Re: [PATCH v2 0/9] ci: make Git's GitHub workflow output much more helpful

From: Junio C Hamano <gitster@pobox.com>
To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
	"Johannes Schindelin" <Johannes.Schindelin@gmx.de>
Cc: Victoria Dye <vdye@github.com>,
	Johannes Schindelin via GitGitGadget <gitgitgadget@gmail.com>,
	git@vger.kernel.org, Eric Sunshine <sunshine@sunshineco.com>,
	Derrick Stolee <derrickstolee@github.com>,
	Emily Shaffer <emilyshaffer@google.com>
Subject: Re: [PATCH v2 0/9] ci: make Git's GitHub workflow output much more helpful
Date: Sun, 22 May 2022 16:27:34 -0700	[thread overview]
Message-ID: <xmqqee0l9mhl.fsf@gitster.g> (raw)
In-Reply-To: <xmqq35h2cwrm.fsf@gitster.g> (Junio C. Hamano's message of "Sat, 21 May 2022 16:05:01 -0700")

Junio C Hamano <gitster@pobox.com> writes:

> Which makes a great guinea pig for the CI output improvement topic.
>
> So, I created two variants of 'seen' with this linux-leaks breakage.
> One is with the js/ci-github-workflow-markup topic on this thread.
> The other one is with the ab/ci-github-workflow-markup topic (which
> uses a preliminary clean-up ab/ci-setup-simplify topic as its base).
> They should show the identical test results and failures.

The two runs to look at have been updated.

 - The one with Ævar's change was missing the primary "workflow
   markup" topic (it only had preliminary clean-up topic), so it is
   not a fair feature-to-feature comparison to begin with.

 - The other one with Johannes's change was done with the version
   before the latest round from yesterday, which has improvements.

With all the other in-flight topics (including the one that shows
failures in linux-leaks job) merged to the same base in the same
order, I prepared two variants of 'seen' that resulted in these
logs:

 - https://github.com/git/git/runs/6546816978
 - https://github.com/git/git/runs/6546750379

One is with both of the required topics from Ævar (with a fix-up [*]),
and the other is with the latest from Johannes's series.

I do not want to taint other folks' eyes with my observations, so I'd
send my impression in a separate message as a response to this
message after waiting for some time.

Thanks.

[Footnote]

* 76253615 (ci: optionally mark up output in the GitHub workflow,
  2022-04-21) added references to ci/print-test-failures.sh and
  ci/print-test-failures-github.sh to the workflow file, while the
  latter script does not exist, but it appears that these references
  want to run the same script, so I've made a stupid and obvious
  fix-up today before pushing the result of merging all out.

  This prevented "make test || ci/print-test-failures.sh" from
  running correctly [*], ever since 76253615 (ci: optionally mark up
  output in the GitHub workflow, 2022-04-21) was queued, and it
  seems that nobody noticed nor complained.  Sigh.