From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Cc: Junio C Hamano <gitster@pobox.com>,
git@vger.kernel.org, Jacob Keller <jacob.keller@gmail.com>
Subject: Re: win+VS environment has "cut" but not "paste"?
Date: Thu, 10 Mar 2022 16:23:02 +0100 [thread overview]
Message-ID: <220310.86o82dj02q.gmgdl@evledraar.gmail.com> (raw)
In-Reply-To: <nycvar.QRO.7.76.6.2203091320140.357@tvgsbejvaqbjf.bet>
On Wed, Mar 09 2022, Johannes Schindelin wrote:
> Hi Junio,
>
> On Mon, 7 Mar 2022, Junio C Hamano wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>>
>> > I said that the current output is only useful to veterans. The output that
>> > hides the detailed log, under a separate job that is marked as
>> > non-failing.
>> >
>> > That's still as true as when I said it. :-)
>>
>> I think getting rid of the separate "print failures" CI step and
>> making it more discoverable how to reveal the details of failing
>> test step is a usability improvement.
>
> I'm so glad you're saying that! I was starting to doubt whether my caring
> about getting rid of that `print failures` step was maybe misguided.
I don't think anyone's been maintaining that getting rid of it wouldn't
be ideal. I for one have just been commenting on issues in the proposed
implementation.
I think we might still want to retain some such steps in the future,
i.e. if we have a failure have subsequent steps that on failure() bump
varying levels of verbosity / raw logs etc., or even try re-running the
test in different ways (e.g. narrow it down with --run).
But the failure step you see when something fails should ideally have
the failure plus the relevant error, just as we do with compile errors.
>> I am not Ævar, but I think what was questioned was the improvement
>> justifies multi dozens of seconds extra waiting time, which is a
>> usability dis-improvement. I do not have a good answer to that
>> question.
>
> It is definitely worrisome that we have to pay such a price. And if there
> was a good answer how to improve that (without sacrificing the
> discoverability of the command trace corresponding to the test failure), I
> would be more than just eager to hear it.
Isn't the answer to that what I suggested in [1]; I.e. the performance
problem being that we include N number of lines of the output that
*didn't fail*, and that's what slows down showing the relevant output
that *did* fail.
I.e. if say t3070-wildmatch.sh fails in a couple of tests we'd show a
*lot* of lines between the relevant failing tests, let's just elide the
non-failing ones and show the output for the failing ones specifically.
*Sometimes* (but very rarely) it's relevant to still look at the full
output, since the failure might be due to an earlier silent failure in a
previous test (or the state it left behind), but I think that's rare
enough that the right thing to do is just to stick that in a subsequent
"verbose dump" step or whatever.
>> But new "non-veteran" users may not share that. That is something a
>> bit worrying about the new UI.
>
> Indeed. My goal, after all, is to improve the experience of contributors,
> not to make it harder.
>
> Still, given that you currently have to click quite a few times until you
> get to where you need to be, I have my doubts that what this patch series
> does is actually making things slower, measured in terms of the total time
> from seeing a failed build to being able to diagnose the cause by
> inspecting the command trace.
Yes, but wouldn't the "Test Summary Report" in [1] be the best of both
worlds[1] (with some minor changes to adapt it to the GitHub "grouping"
output, perhaps)?
Then you'd always see the specific of the failing test at the end, if
you had N number of failures you might have a lot of output above that,
but even that we could always tweak with some smart heuristic. I.e. show
verbose "not ok" output if failures <10, if 10..100 elide some for the
raw log, if >100 just print "this is completely screwed" or whatever :)
1. https://lore.kernel.org/git/220302.86mti87cj2.gmgdl@evledraar.gmail.com/
next prev parent reply other threads:[~2022-03-10 15:31 UTC|newest]
Thread overview: 98+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-24 18:56 [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 1/9] ci: fix code style Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 2/9] ci/run-build-and-tests: take a more high-level view Johannes Schindelin via GitGitGadget
2022-01-24 23:22 ` Eric Sunshine
2022-01-25 14:34 ` Johannes Schindelin
2022-01-24 18:56 ` [PATCH 3/9] ci: make it easier to find failed tests' logs in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-01-25 23:48 ` Ævar Arnfjörð Bjarmason
2022-01-24 18:56 ` [PATCH 4/9] ci/run-build-and-tests: add some structure to the GitHub workflow output Johannes Schindelin via GitGitGadget
2022-02-23 12:13 ` Phillip Wood
2022-02-25 13:40 ` Johannes Schindelin
2022-01-24 18:56 ` [PATCH 5/9] tests: refactor --write-junit-xml code Johannes Schindelin via GitGitGadget
2022-01-26 0:10 ` Ævar Arnfjörð Bjarmason
2022-01-24 18:56 ` [PATCH 6/9] test(junit): avoid line feeds in XML attributes Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 7/9] ci: optionally mark up output in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 8/9] ci: use `--github-workflow-markup` " Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 9/9] ci: call `finalize_test_case_output` a little later Johannes Schindelin via GitGitGadget
2022-01-26 0:25 ` [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Ævar Arnfjörð Bjarmason
2022-01-27 16:31 ` CI "grouping" within jobs v.s. lighter split-out jobs (was: [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful) Ævar Arnfjörð Bjarmason
2022-02-19 23:46 ` [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Johannes Schindelin
2022-02-20 2:44 ` Junio C Hamano
2022-02-20 15:25 ` Johannes Schindelin
2022-02-21 8:09 ` Ævar Arnfjörð Bjarmason
2022-02-22 10:26 ` Johannes Schindelin
2022-02-20 12:47 ` Ævar Arnfjörð Bjarmason
2022-02-22 10:30 ` Johannes Schindelin
2022-02-22 13:31 ` Ævar Arnfjörð Bjarmason
2022-02-23 12:07 ` Phillip Wood
2022-02-25 12:39 ` Ævar Arnfjörð Bjarmason
2022-02-25 14:10 ` Johannes Schindelin
2022-02-25 18:16 ` Junio C Hamano
2022-02-26 18:43 ` Junio C Hamano
2022-03-01 2:59 ` Junio C Hamano
2022-03-01 6:35 ` Junio C Hamano
2022-03-01 10:18 ` Johannes Schindelin
2022-03-01 16:52 ` Junio C Hamano
2022-03-01 10:10 ` Johannes Schindelin
2022-03-01 16:57 ` Junio C Hamano
2022-03-01 10:20 ` Johannes Schindelin
2022-03-04 7:38 ` win+VS environment has "cut" but not "paste"? Junio C Hamano
2022-03-04 9:04 ` Ævar Arnfjörð Bjarmason
2022-03-07 15:51 ` Johannes Schindelin
2022-03-07 17:05 ` Junio C Hamano
2022-03-09 13:02 ` Johannes Schindelin
2022-03-10 15:23 ` Ævar Arnfjörð Bjarmason [this message]
2022-03-07 15:48 ` Johannes Schindelin
2022-03-07 16:58 ` Junio C Hamano
2022-03-02 10:58 ` [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Phillip Wood
2022-03-07 16:07 ` Johannes Schindelin
2022-03-07 17:11 ` Junio C Hamano
2022-03-09 11:44 ` Ævar Arnfjörð Bjarmason
2022-03-07 17:12 ` Phillip Wood
2022-03-01 10:24 ` [PATCH v2 " Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 1/9] ci: fix code style Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 2/9] ci/run-build-and-tests: take a more high-level view Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 3/9] ci: make it easier to find failed tests' logs in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 4/9] ci/run-build-and-tests: add some structure to the GitHub workflow output Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 5/9] tests: refactor --write-junit-xml code Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 6/9] test(junit): avoid line feeds in XML attributes Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 7/9] ci: optionally mark up output in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 8/9] ci: use `--github-workflow-markup` " Johannes Schindelin via GitGitGadget
2022-03-01 10:24 ` [PATCH v2 9/9] ci: call `finalize_test_case_output` a little later Johannes Schindelin via GitGitGadget
2022-03-01 19:07 ` [PATCH v2 0/9] ci: make Git's GitHub workflow output much more helpful Junio C Hamano
2022-03-02 12:22 ` Ævar Arnfjörð Bjarmason
2022-03-07 15:57 ` Johannes Schindelin
2022-03-07 16:05 ` Ævar Arnfjörð Bjarmason
2022-03-07 17:36 ` Junio C Hamano
2022-03-09 10:56 ` Ævar Arnfjörð Bjarmason
2022-03-09 13:20 ` Johannes Schindelin
2022-03-09 19:39 ` Junio C Hamano
2022-03-09 19:47 ` Ævar Arnfjörð Bjarmason
2022-03-25 0:48 ` Victoria Dye
2022-03-25 9:02 ` Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` Victoria Dye
2022-05-21 21:42 ` Johannes Schindelin
2022-05-21 23:05 ` Junio C Hamano
2022-05-22 18:48 ` Johannes Schindelin
2022-05-22 19:10 ` Junio C Hamano
2022-05-23 12:58 ` Johannes Schindelin
2022-05-22 23:27 ` Junio C Hamano
2022-05-23 18:55 ` Junio C Hamano
2022-05-23 19:21 ` Johannes Schindelin
2022-05-23 9:05 ` Ævar Arnfjörð Bjarmason
2022-05-23 18:41 ` Johannes Schindelin
2022-05-24 8:40 ` Ævar Arnfjörð Bjarmason
2022-05-21 22:18 ` [PATCH v3 00/12] " Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 01/12] ci: fix code style Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 02/12] tests: refactor --write-junit-xml code Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 03/12] test(junit): avoid line feeds in XML attributes Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 04/12] ci/run-build-and-tests: take a more high-level view Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 05/12] ci: make it easier to find failed tests' logs in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 06/12] ci/run-build-and-tests: add some structure to the GitHub workflow output Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 07/12] ci: optionally mark up output in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 08/12] ci(github): skip the logs of the successful test cases Johannes Schindelin via GitGitGadget
2022-05-24 10:47 ` Ævar Arnfjörð Bjarmason
2022-05-21 22:18 ` [PATCH v3 09/12] ci(github): avoid printing test case preamble twice Victoria Dye via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 10/12] ci: use `--github-workflow-markup` in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 11/12] ci(github): mention where the full logs can be found Johannes Schindelin via GitGitGadget
2022-05-21 22:18 ` [PATCH v3 12/12] ci: call `finalize_test_case_output` a little later Johannes Schindelin via GitGitGadget
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=220310.86o82dj02q.gmgdl@evledraar.gmail.com \
--to=avarab@gmail.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jacob.keller@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).