From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
"Taylor Blau" <me@ttaylorr.com>,
"SZEDER Gábor" <szeder.dev@gmail.com>,
"Đoàn Trần Công Danh" <congdanhqx@gmail.com>,
"Carlo Marcelo Arenas Belón" <carenas@gmail.com>,
"Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
"Victoria Dye" <vdye@github.com>,
"Matheus Tavares" <matheus.bernardino@usp.br>,
"Lars Schneider" <larsxschneider@gmail.com>,
"Eric Sunshine" <sunshine@sunshineco.com>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH v6 00/14] CI: js/ci-github-workflow-markup rebased on "use $GITHUB_ENV"
Date: Wed, 25 May 2022 13:25:49 +0200 [thread overview]
Message-ID: <cover-v6-00.14-00000000000-20220525T100743Z-avarab@gmail.com> (raw)
In-Reply-To: <RFC-cover-v5-00.10-00000000000-20220421T183001Z-avarab@gmail.com>
This non-RFC re-roll of [1] is based on the just-re-rolled [2]
(hereafter "base") and proposes to replace [3]. It incorporates the
relevant parts of [3] that were new in that re-roll.
Before getting into the details it's helpful to open the following
links:
master: https://github.com/avar/git/actions/runs/2377697338
js: https://github.com/avar/git/actions/runs/2377281297
base: https://github.com/avar/git/actions/runs/2377257487
this: https://github.com/avar/git/actions/runs/2383110429
this with 'raw': https://github.com/avar/git/actions/runs/2383134850
These are all versions of the relevant branches in play with the same
commit on top to trigger CI failures. The "master" and "base" are the
same as those noted in [2], the "js" is [3] (but rebased on master),
and "this" is this topic. The "this with 'raw'" is this topic, but
configured to omit the new GitHub CI Markdown output (see 13/14 and
14/14).
Having these open allows us to compare the output and performance of
these different proposed CI topics. Here clicking on e.g. linux-sha256
(which has a verbose test failure) will on these topics on in the
Chrome debugger[4].
master: https://github.com/avar/git/runs/6573058614?check_suite_focus=true
262 ms Loading
3977 ms Scripting
1632 ms Rendering
641 ms Painting
934 ms System
920 ms Idle
8366 ms Total
js: https://github.com/avar/git/runs/6571972194?check_suite_focus=true
170 ms Loading
12934 ms Scripting
7364 ms Rendering
6940 ms Painting
5489 ms System
5940 ms Idle
38836 ms Total
base: https://github.com/avar/git/runs/6571804381?check_suite_focus=true
215 ms Loading
3270 ms Scripting
1171 ms Rendering
649 ms Painting
941 ms System
731 ms Idle
6976 ms Total
this: https://github.com/avar/git/runs/6588407676?check_suite_focus=true
201 ms Loading
9192 ms Scripting
6104 ms Rendering
5553 ms Painting
4684 ms System
6184 ms Idle
31917 ms Total
this with 'raw': https://github.com/avar/git/runs/6588579493?check_suite_focus=true
173 ms Loading
3036 ms Scripting
1110 ms Rendering
617 ms Painting
1028 ms System
714 ms Idle
6678 ms Total
As noted in previous performance testing [5] what these results
indicate is that the more output you put under GitHub CI's "groups"
the slower it will get. With the "js" variant in particular it can be
seen to struggle to render as you're watching it.
It's faster with "this" because there's less rendering to be done. Due
to the changes in the "base" topic to make "make" and "make test"
top-level commands we can run there's less data in the step where we
need to print failures in detail.
So re: [6] this shows how the two serieses are complimentary and make
the overall CI UX better when combined than stand-alone.
Changes since the RFC v5:
* Incorporated the v3[3] of js/ci-github-workflow-markup.
* Applied fix-up to get rid of broken
ci/print-test-failures-github.sh (which didn't exist anymore)
* The v5 of this series had an issue where we'd "zoom in" on the
"make test" output, instead of the new GitHub CI markdown
output. We now combine the best of both worlds. We'll have a "make
test" and "ci/print-test-failures.sh" steps, both of which fail,
but we'll "zoom in" on the latter, which contains the new output.
* As before this series applied some DRY principles to the code [7]
proposed to add, i.e. the "handle_failed_tests" was a
near-duplication of the code in
ci/print-test-failures-github.sh. The 01-03/14 here allow us to
unify the two & share the code instead.
My RFC v5 had some options we didn't need (--exit-code and
--no-exit-code) that I removed. It furthermore defaults to "auto"
discovering if we should emit the GitHub CI *.markup output based
on if we find those files in t/test-results, which as we'll see in
13-14/14 makes configuring the output simpler.
* Due to the above and other changes here being smaller the "ci: make
it easier to find failed tests' logs in the GitHub workflow" here
is now a squashed version of changes done in several steps in
[3]. We now add the ability to emit GitHub CI markup in one commit,
and enable it in another.
* In the latest "js" topic the "ok" tests were skipped, but we'd
still include "skip" tests, i.e. "not ok .. TODO" tests. These
should be considered "ok" for the purposes of the output, so a new
09/14 here excludes them too.
* An updated commit message for 10/14 notes a subtle and presumably
unintended side-effect/bug in "ci(github): avoid printing test case
preamble twice" new in the v3 of [3]. In tweaking the *.markup
files we also change the output in the *.out files (i.e. the raw
logs).
You thus won't get the same output in *.out (the raw logs) with the
new output.
* New 13/14 and 14/14 commits here make the CI output type
configurable using the same "ci-config" method we use for the
existing "check-ref". As explained in 14/14 this is what I'm using
to produce "this with 'raw'" output the same commit as "this". I
push to a branch whose name ends with "_CI_OUT_RAW".
This allows users (such as me, or per 13/14 probably SZEDER too) to
configure our output to avoid this GitHub markdown in our
ci-config.
* As 13/14 argues for I think a better thing to do would be to avoid
14/14 for now and take this series without making the new "github"
output the default.
But in the interest of not conflating that with this proposed
re-roll 14/14 changes the default from "raw" to "github" again,
thus making the output [3] is proposing the default here too.
We can always revert 14/14 later if we think that was premature,
which as shown there is now a one-line change.
To continue from the last note above: This series also allows for
combining the new & old output in a way where we can have our cake &
eat it too. If we load the "this" output and "zoom in" on the failed
"make test" step instead of the subsequent "ci/print-test-failures.sh"
step we'll load the page in around 1/6 the time, or:
this (zoomed in on 'make test'): https://github.com/avar/git/runs/6588407676?check_suite_focus=true#step:7:1437
149 ms Loading
2559 ms Scripting
934 ms Rendering
591 ms Painting
845 ms System
605 ms Idle
5684 ms Total
Clicking on the "ci/print-test-failures.sh" will then take just as
long. I haven't implemented this as a configurable feature is just a
matter of making ci/print-test-failures.sh exit with 0 where it now
exits with 1 (as we always "zoom in" on the last failed step, which
then will be "make test").
For Junio:
If you eject js/ci-github-workflow-markup you'll get a minor conflict
(as before) with js/use-builtin-add-i, and a new conflict with
jx/l10n-workflow-change. The correct resolution is to "git rm
ci/run-build-and-tests.sh ci/run-static-analysis.sh" and apply this
diff on top:
diff --git a/Makefile b/Makefile
index b602bdaed4a..984b1a1905d 100644
--- a/Makefile
+++ b/Makefile
@@ -3625,3 +3625,4 @@ ci-static-analysis: ci-check-directional-formatting
ci-static-analysis: check-builtins
ci-static-analysis: check-coccicheck
ci-static-analysis: hdr-check
+ci-static-analysis: check-pot
diff --git a/ci/lib.sh b/ci/lib.sh
index b148d3f209f..c6b17b4cb22 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -282,7 +282,7 @@ linux-TEST-vars)
setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1
setenv --test GIT_TEST_MULTI_PACK_INDEX 1
setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1
- setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1
+ setenv --test GIT_TEST_ADD_I_USE_BUILTIN 0
setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master
setenv --test GIT_TEST_WRITE_REV_INDEX 1
setenv --test GIT_TEST_CHECKOUT_WORKERS 2
1. https://lore.kernel.org/git/RFC-cover-v5-00.10-00000000000-20220421T183001Z-avarab@gmail.com/
2. https://lore.kernel.org/git/cover-v6-00.29-00000000000-20220525T094123Z-avarab@gmail.com/
3. https://lore.kernel.org/git/pull.1117.v3.git.1653171536.gitgitgadget@gmail.com/
4. Version 101.0.4951.54 (Official Build) (64-bit). Opening the
debugger in an empty tab (F12), going to "Performance" and
recording a trace (Ctrl+E) (note: *not* Ctrl+Shift+E, as that would
end it after network load, but before rendering is done & the page
is ready to interact with).
The "Total" times here are "real" and don't very with how fast I am
to "stop" the trace (tested by leaving it lingering for a long
time).
5. https://lore.kernel.org/git/220523.86ee0kzjix.gmgdl@evledraar.gmail.com/
6. https://lore.kernel.org/git/nycvar.QRO.7.76.6.2205232040130.352@tvgsbejvaqbjf.bet/
7. https://lore.kernel.org/git/417f702a2454316f0bdc5b66eb16751272ff2989.1653171536.git.gitgitgadget@gmail.com/
Johannes Schindelin (6):
tests: refactor --write-junit-xml code
test(junit): avoid line feeds in XML attributes
ci: optionally mark up output in the GitHub workflow
ci(github): skip the logs of the successful test cases
ci(github): mention where the full logs can be found
ci: call `finalize_test_case_output` a little later
Victoria Dye (1):
ci(github): avoid printing test case preamble twice
Ævar Arnfjörð Bjarmason (7):
CI: don't "cd" in ci/print-test-failures.sh
CI: don't include "test-results/" in ci/print-test-failures.sh output
CI: stop setting FAILED_TEST_ARTIFACTS N times
ci: make it easier to find failed tests' logs in the GitHub workflow
ci(github): skip "skip" tests in --github-workflow-markup
CI: make --github-workflow-markup ci-config, off by default
CI: make the --github-workflow-markup "github" output the default
.github/workflows/main.yml | 24 +++++
ci/lib.sh | 12 +++
ci/print-test-failures.sh | 100 ++++++++++++++++----
t/test-lib-functions.sh | 6 +-
t/test-lib-github-workflow-markup.sh | 56 ++++++++++++
t/test-lib-junit.sh | 132 +++++++++++++++++++++++++++
t/test-lib.sh | 128 ++++----------------------
7 files changed, 332 insertions(+), 126 deletions(-)
create mode 100644 t/test-lib-github-workflow-markup.sh
create mode 100644 t/test-lib-junit.sh
Range-diff against v5:
1: 56910df26e7 = 1: c10a11fdae8 CI: don't "cd" in ci/print-test-failures.sh
3: 577a5be34f4 = 2: 1d07ca58322 CI: don't include "test-results/" in ci/print-test-failures.sh output
4: 1482f840f64 ! 3: fdc5b534644 CI: stop setting FAILED_TEST_ARTIFACTS N times
@@ ci/print-test-failures.sh: do
cp "t/test-results/${TEST_OUT}" t/failed-test-artifacts/
(
cd t &&
-@@ ci/print-test-failures.sh: done
-
- if test -n "$failed"
- then
+@@ ci/print-test-failures.sh: do
+ esac
+ fi
+ done
++
++if test -n "$failed"
++then
+ if test -n "$GITHUB_ENV"
+ then
+ echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
+ fi
-+
- if test -n "$exit_code"
- then
- exit 1
++fi
6: d9b7d5ddf5a = 4: a75846a12f2 tests: refactor --write-junit-xml code
7: 2d233bbdc22 = 5: 5b709dde86a test(junit): avoid line feeds in XML attributes
8: 5dd43a1624b = 6: f5b39f1660d ci: optionally mark up output in the GitHub workflow
2: caec0b1089a ! 7: e713ca6041b CI: add --exit-code to ci/print-test-failures.sh
@@ Metadata
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
## Commit message ##
- CI: add --exit-code to ci/print-test-failures.sh
+ ci: make it easier to find failed tests' logs in the GitHub workflow
- The ci/print-test-failures.sh scripts will exit 0 when there's failed
- tests, let's teach it --exit-code to have it exit non-zero in those
- cases.
+ When investigating a test failure, the time that matters most is the
+ time it takes from getting aware of the failure to displaying the output
+ of the failing test case.
+ You currently have to know a lot of implementation details when
+ investigating test failures in the CI runs. The first step is easy: the
+ failed job is marked quite clearly, but when opening it, the failed step
+ is expanded, which in our case is the one running
+ `make` or 'make test`. When failing in the latter we only show a
+ high-level view of what went wrong: it prints the output of `prove`
+ which merely tells the reader which test script failed, along with the failed
+ test numbers.
+
+ To diagnose those failures in detail the user has then had to know
+ that they should expand the subsequent "Run ci/print-test-failures.sh"
+ step. Because that step always exits successfully it's not expanded by
+ default (the GitHub CI only expands the last failing step), making it
+ easy to miss that there is useful information hidden in there.
+
+ This change does a couple of things to improve the situation:
+
+ * We now make use of the "exit code" feature of
+ ci/print-test-failures.sh added in a preceding commit, thus if "make
+ test" fails we'll pick up that we have failing tests, and fail in
+ ci/print-test-failures.sh too.
+
+ Note that we still exit successfully from ci/print-test-failures.sh
+ if we don't have logs of any failed tests in t/test-results. Thus
+ if we fail in "make" (i.e. when building), or even in "make test"
+ in a way that doesn't yield a failed test (e.g. the t/Makefile
+ itself failing) we'll correctly "zoom in" on the (hopefully
+ failing) "make test" instead.
+
+ As before if there was no previously failed step the
+ ci/print-test-failures.sh step is skipped entirely (see
+ .github/workflows/main.yml)
+
+ To make the script DWYM the ci/print-test-failures.sh defaults to
+ an "auto" mode. Thus since we turn on --github-workflow-markup in
+ ci/lib.sh we'll write *.markup files, and will pick up on that in
+ ci/print-test-failures.sh.
+
+ The "$github_workflow_markup" being "auto" (or "t") on failure is
+ also what makes us exit non-zero. We could also add
+ --[no-]exit-code arguments, but at this point we only need this for
+ the "*.markup" mode, so let's leave that for now.
+
+ By having this be "auto" well pick up on whatever the GIT_TEST_OPTS
+ setting is, which as we'll see in a subsequent commit will be
+ useful if we'd like to configure the CI to emit it only some of the
+ time.
+
+ * The ci/print-test-failures.sh now makes use of the GitHub markdown
+ introduced for t/test-lib.sh in the preceding commit. I.e. in GitHub
+ CI we'll now run with --github-workflow-markup in GIT_TEST_OPTS,
+ which causes our "--verbose" output in
+ "t/test-results/$TEST_OUT.markup" to be in the GitHub CI-specific
+ Markdown format.
+
+ The new output is easier to read since it makes use of collapsible grouping.
+ Initially, readers will see the high-level view of what actually happened.
+ To drill down, the "group" for the specific failing test(s) can be expanded.
+
+ Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
+ ## ci/lib.sh ##
+@@ ci/lib.sh: MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
+ case "$CI_TYPE" in
+ github-actions)
+ setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
+- GIT_TEST_OPTS="--verbose-log -x"
++ GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
+ test Windows != "$RUNNER_OS" ||
+ GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
+ setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
+
## ci/print-test-failures.sh ##
@@ ci/print-test-failures.sh: set -e
. ${0%/*}/lib-ci-type.sh
. ${0%/*}/lib-tput.sh
-+exit_code=
++github_workflow_markup=auto
++case "$CI_TYPE" in
++github-actions)
++ github_workflow_markup=t
++ ;;
++esac
++
+while test $# != 0
+do
+ case "$1" in
-+ --exit-code)
-+ exit_code=t
++ --github-workflow-markup)
++ github_workflow_markup=t
++ ;;
++ --no-github-workflow-markup)
++ github_workflow_markup=
+ ;;
+ *)
+ echo "BUG: invalid $0 argument: $1" >&2
@@ ci/print-test-failures.sh: set -e
TEST_NAME="${TEST_EXIT%.exit}"
TEST_NAME="${TEST_NAME##*/}"
TEST_OUT="${TEST_NAME}.out"
-@@ ci/print-test-failures.sh: do
- esac
+ TEST_MARKUP="${TEST_NAME}.markup"
+
+- echo "------------------------------------------------------------------------"
+- echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
+- echo "------------------------------------------------------------------------"
+- cat "t/test-results/${TEST_OUT}"
++ do_markup=
++ case "$github_workflow_markup" in
++ t)
++ do_markup=t
++ ;;
++ auto)
++ if test -f "t/test-results/$TEST_MARKUP"
++ then
++ do_markup=t
++ fi
++ ;;
++ esac
++
++ if test -n "$do_markup"
++ then
++ printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
++ cat "t/test-results/$TEST_MARKUP"
++ else
++ echo "------------------------------------------------------------------------"
++ echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
++ echo "------------------------------------------------------------------------"
++ cat "t/test-results/${TEST_OUT}"
++ fi
+
+ trash_dir="trash directory.$TEST_NAME"
+ case "$CI_TYPE" in
+@@ ci/print-test-failures.sh: then
+ then
+ echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
fi
- done
+
-+if test -n "$failed"
-+then
-+ if test -n "$exit_code"
-+ then
++ case "$github_workflow_markup" in
++ t|auto)
+ exit 1
-+ fi
-+fi
++ ;;
++ '')
++ exit 0
++ ;;
++ esac
+ fi
-: ----------- > 8: 9c9fd24fffa ci(github): skip the logs of the successful test cases
-: ----------- > 9: 03d7b54d68b ci(github): skip "skip" tests in --github-workflow-markup
-: ----------- > 10: 90a152d79f9 ci(github): avoid printing test case preamble twice
9: 877edff3b4d ! 11: 68eed85162b ci: use `--github-workflow-markup` in the GitHub workflow
@@ Metadata
Author: Johannes Schindelin <Johannes.Schindelin@gmx.de>
## Commit message ##
- ci: use `--github-workflow-markup` in the GitHub workflow
+ ci(github): mention where the full logs can be found
- [Ævar: due to re-structuring on top of my series the {begin,end}_group
- in CI isn't needed at all to get "group" output for the test
- suite. This commit includes the now-squashed "ci/run-build-and-tests:
- add some structure to the GitHub workflow output":]
-
- The current output of Git's GitHub workflow can be quite confusing,
- especially for contributors new to the project.
-
- To make it more helpful, let's introduce some collapsible grouping.
- Initially, readers will see the high-level view of what actually
- happened (did the build fail, or the test suite?). To drill down, the
- respective group can be expanded.
-
- Note: sadly, workflow output currently cannot contain any nested groups
- (see https://github.com/actions/runner/issues/802 for details),
- therefore we take pains to ensure to end any previous group before
- starting a new one.
-
- [Ævar: The above comment isn't true anymore, as that limitation has
- been removed by basing this on my patches to run "make" and "make
- test" directly from the top-level of main.yml.
-
- Those are now effectively their own "group", effectively giving this
- stage another group "level" to use. This means that the equivalent of
- "make test" won't be on the same level as an individual test failure.
-
- We no longer take any pains to ensure balanced group output as a
- result (which was a caveat the previous ci/lib.sh implementation had
- to deal with., We just need to "cat" the generated *.markup]
+ The full logs are contained in the `failed-tests-*.zip` artifacts that
+ are attached to the failed CI run. Since this is not immediately
+ obvious to the well-disposed reader, let's mention it explicitly.
+ Suggested-by: Victoria Dye <vdye@github.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
- ## ci/lib.sh ##
-@@ ci/lib.sh: MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
- case "$CI_TYPE" in
- github-actions)
- setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
-- GIT_TEST_OPTS="--verbose-log -x"
-+ GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
- test Windows != "$RUNNER_OS" ||
- GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
- setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
-
## ci/print-test-failures.sh ##
@@ ci/print-test-failures.sh: do
- if test -n "$github_workflow_markup"
+ if test -n "$do_markup"
then
printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
-+ cat "t/test-results/$TEST_MARKUP"
++ echo "The full logs are in the artifacts attached to this run."
+ cat "t/test-results/$TEST_MARKUP"
else
echo "------------------------------------------------------------------------"
- echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
- echo "------------------------------------------------------------------------"
-+ cat "t/test-results/${TEST_OUT}"
- fi
-- cat "t/test-results/${TEST_OUT}"
-
- trash_dir="trash directory.$TEST_NAME"
- case "$CI_TYPE" in
10: 6b278194f9a = 12: 11dab84b0dc ci: call `finalize_test_case_output` a little later
5: 4291892fbd3 ! 13: fbe0d99c6b3 ci: make it easier to find failed tests' logs in the GitHub workflow
@@
## Metadata ##
-Author: Johannes Schindelin <Johannes.Schindelin@gmx.de>
+Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
## Commit message ##
- ci: make it easier to find failed tests' logs in the GitHub workflow
-
- When investigating a test failure, the time that matters most is the
- time it takes from getting aware of the failure to displaying the output
- of the failing test case.
-
- You currently have to know a lot of implementation details when
- investigating test failures in the CI runs. The first step is easy: the
- failed job is marked quite clearly, but when opening it, the failed step
- is expanded, which in our case is the one running
- `ci/run-build-and-tests.sh`. This step, most notably, only offers a
- high-level view of what went wrong: it prints the output of `prove`
- which merely tells the reader which test script failed.
-
- The actually interesting part is in the detailed log of said failed
- test script. But that log is shown in the CI run's step that runs
- `ci/print-test-failures.sh`. And that step is _not_ expanded in the web
- UI by default. It is even marked as "successful", which makes it very
- easy to miss that there is useful information hidden in there.
-
- Let's help the reader by showing the failed tests' detailed logs in the
- step that is expanded automatically, i.e. directly after the test suite
- failed.
-
- This also helps the situation where the _build_ failed and the
- `print-test-failures` step was executed under the assumption that the
- _test suite_ failed, and consequently failed to find any failed tests.
-
- An alternative way to implement this patch would be to source
- `ci/print-test-failures.sh` in the `handle_test_failures` function to
- show these logs. However, over the course of the next few commits, we
- want to introduce some grouping which would be harder to achieve that
- way (for example, we do want a leaner, and colored, preamble for each
- failed test script, and it would be trickier to accommodate the lack of
- nested groupings in GitHub workflows' output).
-
- Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
+ CI: make --github-workflow-markup ci-config, off by default
+
+ In preceding commits the --github-workflow-markup output was made the
+ default under GitHub CI.
+
+ There's a few outstanding issues with that new output target however:
+
+ A. As noted in [1] it's much slower in some common cases, i.e. the
+ page rendering time (under Firefox & Chrome debugging) goes from ~20s
+ to on the order of 60-80s (less pronounced when not debugging, but
+ still in that ballpark). See also [2] for tests on an earlier iteration
+ of the series.
+
+ B. Preceding commits in this series to omit non-failing test output
+ may have mitigated that somewhat, i.e. skipping "ok" and "skip" tests
+ in the *.markup files (although [1] is a benchmark with that "ok"
+ change).
+
+ Per [3] there are some CI users that prefer to see only the full
+ verbose output, i.e. we can't in the general case assume that we
+ can emit only the trace output for the failing test, and that's
+ going to give us the information we need to debug the test. It
+ might be failing because of earlier setup, or from trace output
+ outside of any test (i.e. in the main body of the file).
+
+ Users such as [3] could still just consult the *.out output, but
+ then we'd make them download the archive instead of viewing the
+ output in the web UX.
+
+ C. Per B above and [3] an earlier change in this series modified
+ the *.out output to change the *.out output inadvertently while trying
+ to modify the *.markup output. See the commit message of "ci(github):
+ avoid printing test case preamble twice" earlier in this series.
+
+ It thus make sense for now to make the "ci/print-test-failures.sh"
+ output be the "raw" one. I.e. the *.out output we'd get before
+ --github-workflow-markup was implemented.
+
+ To make it use ci/print-test-failures.sh we can create a
+ "ci/config/print-test-failures-output-type" script on the "ci-config"
+ branch that does e.g.:
+
+ #!/bin/sh
+ echo github
+
+ That script will (like the existing "check-ref") get the ref as an
+ argument, so we can in the future turn this on experimentally for
+ "seen", and flip the default in the future. The script can also emit
+ nothing to get whatever our idea is of the default (currently "raw").
+
+ Note the corresponding change to ci/print-test-failures.sh, i.e. we'll
+ do the right thing vis-a-vis what we "zoom in" to depending on the
+ setting. If it's "raw" we'd like a failed "make test" to "zoom in" to
+ that step, if it's "github" we'd like the to open the subsequent
+ "ci/print-test-failures.sh" step instead.
+
+ 1. https://lore.kernel.org/git/220523.86ee0kzjix.gmgdl@evledraar.gmail.com/
+ 2. https://lore.kernel.org/git/220222.86tucr6kz5.gmgdl@evledraar.gmail.com/
+ 3. https://lore.kernel.org/git/20210309175249.GE3590451@szeder.dev/
+
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
## .github/workflows/main.yml ##
@@ .github/workflows/main.yml: jobs:
+ runs-on: ubuntu-latest
+ outputs:
+ enabled: ${{ steps.check-ref.outputs.enabled }}${{ steps.skip-if-redundant.outputs.enabled }}
++ print-test-failures-output-type: ${{ steps.print-test-failures-output-type.outputs.type }}
+ steps:
+ - name: try to clone ci-config branch
+ run: |
+@@ .github/workflows/main.yml: jobs:
+ config-repo &&
+ cd config-repo &&
+ git checkout HEAD -- ci/config || : ignore
++ - id: print-test-failures-output-type
++ name: check what output type ci/print-test-failures.sh uses
++ run: |
++ type_default=raw
++ type=$type_default
++
++ if test -x config-repo/ci/config/print-test-failures-output-type
++ then
++ type=$(config-repo/ci/config/print-test-failures-output-type '${{ github.ref }}')
++ if test -z "$type"
++ then
++ type=$type_default
++ fi
++ fi
++ echo "::set-output name=type::$type"
+ - id: check-ref
+ name: check whether CI is enabled for ref
+ run: |
+@@ .github/workflows/main.yml: jobs:
+ - uses: git-for-windows/setup-git-for-windows-sdk@v1
+ - run: ci/lib.sh --test
shell: bash
- - name: test
++ env:
++ GIT_CI_PTF_OUTPUT_TYPE: ${{needs.ci-config.outputs.print-test-failures-output-type}}
+ - name: select tests
+ run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
shell: bash
-- run: . /etc/profile && make -C t -e
-- - name: ci/print-test-failures.sh
-- if: failure()
-- shell: bash
-- run: ci/print-test-failures.sh
-+ run: . /etc/profile && make -C t -e || ci/print-test-failures.sh
- - name: Upload failed tests' directories
- if: failure() && env.FAILED_TEST_ARTIFACTS != ''
- uses: actions/upload-artifact@v2
@@ .github/workflows/main.yml: jobs:
+ run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz
+ - run: ci/lib.sh --test
shell: bash
- - name: test
++ env:
++ GIT_CI_PTF_OUTPUT_TYPE: ${{needs.ci-config.outputs.print-test-failures-output-type}}
+ - name: select tests
+ run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10
shell: bash
-- run: . /etc/profile && make -C t -e
-- - name: ci/print-test-failures.sh
-- if: failure()
-- shell: bash
-- run: ci/print-test-failures.sh
-+ run: . /etc/profile && make -C t -e || ci/print-test-failures-github.sh
- - name: Upload failed tests' directories
- if: failure() && env.FAILED_TEST_ARTIFACTS != ''
- uses: actions/upload-artifact@v2
@@ .github/workflows/main.yml: jobs:
- run: ci/lib.sh --build
- run: make
- run: ci/lib.sh --test
-- - run: make test
-+ - run: make test || ci/print-test-failures-github.sh
++ env:
++ GIT_CI_PTF_OUTPUT_TYPE: ${{needs.ci-config.outputs.print-test-failures-output-type}}
+ - run: make test
if: success()
-- - run: ci/print-test-failures.sh
-- if: failure()
- - name: Upload failed tests' directories
- if: failure() && env.FAILED_TEST_ARTIFACTS != ''
- uses: actions/upload-artifact@v2
+ - run: ci/print-test-failures.sh
@@ .github/workflows/main.yml: jobs:
- run: make
- run: ci/lib.sh --test
if: success() && matrix.vector.skip-tests != 'yes'
-- - run: make test
-+ - run: make test || ci/print-test-failures-github.sh
++ env:
++ GIT_CI_PTF_OUTPUT_TYPE: ${{needs.ci-config.outputs.print-test-failures-output-type}}
+ - run: make test
if: success() && matrix.vector.skip-tests != 'yes'
-- - run: ci/print-test-failures.sh
-- if: failure() && matrix.vector.skip-tests != 'yes'
- - name: Upload failed tests' directories
- if: failure() && env.FAILED_TEST_ARTIFACTS != ''
- uses: actions/upload-artifact@v1
+ - run: ci/print-test-failures.sh
+
+ ## ci/lib.sh ##
+@@ ci/lib.sh: if test -n "$GITHUB_ENV"
+ then
+ echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2
+ fi
++echo "CONFIG: GIT_CI_PTF_OUTPUT_TYPE=$GIT_CI_PTF_OUTPUT_TYPE" >&2
+ echo >&2
+
+ # Helper functions
+@@ ci/lib.sh: MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
+ case "$CI_TYPE" in
+ github-actions)
+ setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
+- GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
++ GIT_TEST_OPTS="--verbose-log -x"
++ if test -n "$GIT_CI_PTF_OUTPUT_TYPE"
++ then
++ # For later use in ci/print-test-failures.sh
++ setenv --test GIT_CI_PTF_OUTPUT_TYPE "$GIT_CI_PTF_OUTPUT_TYPE"
++
++ case "$GIT_CI_PTF_OUTPUT_TYPE" in
++ github)
++ GIT_TEST_OPTS="$GIT_TEST_OPTS --github-workflow-markup"
++ ;;
++ esac
++ fi
+ test Windows != "$RUNNER_OS" ||
+ GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
+ setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
## ci/print-test-failures.sh ##
@@ ci/print-test-failures.sh: set -e
- . ${0%/*}/lib-ci-type.sh
. ${0%/*}/lib-tput.sh
--exit_code=
-+case "$CI_TYPE" in
-+github-actions)
-+ exit_code=t
-+ github_workflow_markup=t
-+ ;;
-+*)
-+ exit_code=
+ github_workflow_markup=auto
+-case "$CI_TYPE" in
+-github-actions)
++case "$GIT_CI_PTF_OUTPUT_TYPE" in
++github)
+ github_workflow_markup=t
+ ;;
++raw)
+ github_workflow_markup=
+ ;;
-+esac
-+
- while test $# != 0
- do
- case "$1" in
- --exit-code)
- exit_code=t
- ;;
-+ --no-exit-code)
-+ exit_code=
-+ ;;
-+ --github-workflow-markup)
-+ github_workflow_markup=t
-+ ;;
-+ --no-github-workflow-markup)
-+ github_workflow_markup=
-+ ;;
- *)
- echo "BUG: invalid $0 argument: $1" >&2
- exit 1
-@@ ci/print-test-failures.sh: do
- TEST_OUT="${TEST_NAME}.out"
- TEST_MARKUP="${TEST_NAME}.markup"
-
-- echo "------------------------------------------------------------------------"
-- echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
-- echo "------------------------------------------------------------------------"
-+ if test -n "$github_workflow_markup"
-+ then
-+ printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
-+ else
-+ echo "------------------------------------------------------------------------"
-+ echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
-+ echo "------------------------------------------------------------------------"
-+ fi
- cat "t/test-results/${TEST_OUT}"
+ esac
- trash_dir="trash directory.$TEST_NAME"
+ while test $# != 0
-: ----------- > 14: 0b02b186c87 CI: make the --github-workflow-markup "github" output the default
--
2.36.1.1045.gf356b5617dd
next prev parent reply other threads:[~2022-05-25 11:26 UTC|newest]
Thread overview: 260+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-21 14:46 [PATCH 00/25] CI: run "make [test]" directly, use $GITHUB_ENV Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 01/25] CI: run "set -ex" early in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 02/25] CI: make "$jobname" explicit, remove fallback Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 03/25] CI: remove more dead Travis CI support Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 04/25] CI: remove dead "tree skipping" code Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 05/25] CI: remove unused Azure ci/* code Ævar Arnfjörð Bjarmason
2022-02-22 10:21 ` Johannes Schindelin
2022-02-22 10:27 ` Ævar Arnfjörð Bjarmason
2022-02-22 12:07 ` Johannes Schindelin
2022-02-22 13:05 ` Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 06/25] CI: don't have "git grep" invoke a pager in tree content check Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 07/25] CI: have "static-analysis" run a "make ci-static-analysis" target Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 08/25] CI: have "static-analysis" run "check-builtins", not "documentation" Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 09/25] CI: move p4 and git-lfs variables to ci/install-dependencies.sh Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 10/25] CI: consistently use "export" in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 11/25] CI: export variables via a wrapper Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 12/25] CI: remove "run-build-and-tests.sh", run "make [test]" directly Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 13/25] CI: check ignored unignored build artifacts in "win[+VS] build" too Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 14/25] CI: invoke "make artifacts-tar" directly in windows-build Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 15/25] CI: split up and reduce "ci/test-documentation.sh" Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 16/25] CI: combine ci/install{,-docker}-dependencies.sh Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 17/25] CI: move "env" definitions into ci/lib.sh Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 18/25] ci/run-test-slice.sh: replace shelling out with "echo" Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 19/25] CI: pre-select test slice in Windows & VS tests Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 20/25] CI: only invoke ci/lib.sh as "steps" in main.yml Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 21/25] CI: narrow down variable definitions in --build and --test Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 22/25] CI: add more variables to MAKEFLAGS, except under vs-build Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 23/25] CI: stop over-setting the $CC variable Ævar Arnfjörð Bjarmason
2022-03-05 8:17 ` SZEDER Gábor
2022-03-05 14:15 ` Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 24/25] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case Ævar Arnfjörð Bjarmason
2022-02-21 14:46 ` [PATCH 25/25] CI: don't use "set -x" in "ci/lib.sh" output Ævar Arnfjörð Bjarmason
2022-02-23 13:55 ` [PATCH 00/25] CI: run "make [test]" directly, use $GITHUB_ENV Phillip Wood
2022-02-23 20:12 ` Junio C Hamano
2022-02-23 21:18 ` Ævar Arnfjörð Bjarmason
2022-03-09 22:10 ` Johannes Schindelin
2022-03-25 18:37 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
2022-03-25 18:37 ` [PATCH v2 01/25] CI: run "set -ex" early in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-03-25 18:37 ` [PATCH v2 02/25] CI: make "$jobname" explicit, remove fallback Ævar Arnfjörð Bjarmason
2022-03-25 18:37 ` [PATCH v2 03/25] CI: remove more dead Travis CI support Ævar Arnfjörð Bjarmason
2022-03-25 18:37 ` [PATCH v2 04/25] CI: remove dead "tree skipping" code Ævar Arnfjörð Bjarmason
2022-03-25 18:37 ` [PATCH v2 05/25] CI: remove unused Azure ci/* code Ævar Arnfjörð Bjarmason
2022-03-25 18:37 ` [PATCH v2 06/25] CI: don't have "git grep" invoke a pager in tree content check Ævar Arnfjörð Bjarmason
2022-03-25 18:37 ` [PATCH v2 07/25] CI: have "static-analysis" run a "make ci-static-analysis" target Ævar Arnfjörð Bjarmason
2022-03-25 18:37 ` [PATCH v2 08/25] CI: have "static-analysis" run "check-builtins", not "documentation" Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 09/25] CI: move p4 and git-lfs variables to ci/install-dependencies.sh Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 10/25] CI: consistently use "export" in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 11/25] CI: export variables via a wrapper Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 12/25] CI: remove "run-build-and-tests.sh", run "make [test]" directly Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 13/25] CI: check ignored unignored build artifacts in "win[+VS] build" too Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 14/25] CI: invoke "make artifacts-tar" directly in windows-build Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 15/25] CI: split up and reduce "ci/test-documentation.sh" Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 16/25] CI: combine ci/install{,-docker}-dependencies.sh Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 17/25] CI: move "env" definitions into ci/lib.sh Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 18/25] ci/run-test-slice.sh: replace shelling out with "echo" Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 19/25] CI: pre-select test slice in Windows & VS tests Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 20/25] CI: only invoke ci/lib.sh as "steps" in main.yml Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 21/25] CI: narrow down variable definitions in --build and --test Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 22/25] CI: add more variables to MAKEFLAGS, except under vs-build Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 23/25] CI: set CC in MAKEFLAGS directly, don't add it to the environment Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 24/25] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case Ævar Arnfjörð Bjarmason
2022-03-25 18:38 ` [PATCH v2 25/25] CI: don't use "set -x" in "ci/lib.sh" output Ævar Arnfjörð Bjarmason
2022-03-25 19:24 ` [RFC PATCH v3 0/6] CI: js/ci-github-workflow-markup rebased on "use $GITHUB_ENV" Ævar Arnfjörð Bjarmason
2022-03-25 19:24 ` [RFC PATCH v3 1/6] ci: make it easier to find failed tests' logs in the GitHub workflow Ævar Arnfjörð Bjarmason
2022-03-25 19:24 ` [RFC PATCH v3 2/6] tests: refactor --write-junit-xml code Ævar Arnfjörð Bjarmason
2022-03-25 19:24 ` [RFC PATCH v3 3/6] test(junit): avoid line feeds in XML attributes Ævar Arnfjörð Bjarmason
2022-03-25 19:24 ` [RFC PATCH v3 4/6] ci: optionally mark up output in the GitHub workflow Ævar Arnfjörð Bjarmason
2022-03-25 19:24 ` [RFC PATCH v3 5/6] ci: use `--github-workflow-markup` " Ævar Arnfjörð Bjarmason
2022-03-25 19:24 ` [RFC PATCH v3 6/6] ci: call `finalize_test_case_output` a little later Ævar Arnfjörð Bjarmason
2022-04-13 19:57 ` [RFC PATCH v4 0/6] CI: js/ci-github-workflow-markup rebased on "use $GITHUB_ENV" Ævar Arnfjörð Bjarmason
2022-04-13 19:57 ` [RFC PATCH v4 1/6] ci: make it easier to find failed tests' logs in the GitHub workflow Ævar Arnfjörð Bjarmason
2022-04-13 19:57 ` [RFC PATCH v4 2/6] tests: refactor --write-junit-xml code Ævar Arnfjörð Bjarmason
2022-04-13 19:57 ` [RFC PATCH v4 3/6] test(junit): avoid line feeds in XML attributes Ævar Arnfjörð Bjarmason
2022-04-13 19:57 ` [RFC PATCH v4 4/6] ci: optionally mark up output in the GitHub workflow Ævar Arnfjörð Bjarmason
2022-04-13 19:57 ` [RFC PATCH v4 5/6] ci: use `--github-workflow-markup` " Ævar Arnfjörð Bjarmason
2022-04-13 19:57 ` [RFC PATCH v4 6/6] ci: call `finalize_test_case_output` a little later Ævar Arnfjörð Bjarmason
2022-04-14 10:26 ` [RFC PATCH v4 0/6] CI: js/ci-github-workflow-markup rebased on "use $GITHUB_ENV" Junio C Hamano
2022-04-14 12:47 ` Ævar Arnfjörð Bjarmason
2022-04-14 18:30 ` Junio C Hamano
2022-04-21 18:36 ` [RFC PATCH v5 00/10] " Ævar Arnfjörð Bjarmason
2022-04-21 18:36 ` [RFC PATCH v5 01/10] CI: don't "cd" in ci/print-test-failures.sh Ævar Arnfjörð Bjarmason
2022-04-21 18:36 ` [RFC PATCH v5 02/10] CI: add --exit-code to ci/print-test-failures.sh Ævar Arnfjörð Bjarmason
2022-04-21 18:36 ` [RFC PATCH v5 03/10] CI: don't include "test-results/" in ci/print-test-failures.sh output Ævar Arnfjörð Bjarmason
2022-04-21 18:36 ` [RFC PATCH v5 04/10] CI: stop setting FAILED_TEST_ARTIFACTS N times Ævar Arnfjörð Bjarmason
2022-04-21 18:36 ` [RFC PATCH v5 05/10] ci: make it easier to find failed tests' logs in the GitHub workflow Ævar Arnfjörð Bjarmason
2022-04-21 18:36 ` [RFC PATCH v5 06/10] tests: refactor --write-junit-xml code Ævar Arnfjörð Bjarmason
2022-04-21 18:36 ` [RFC PATCH v5 07/10] test(junit): avoid line feeds in XML attributes Ævar Arnfjörð Bjarmason
2022-04-21 18:36 ` [RFC PATCH v5 08/10] ci: optionally mark up output in the GitHub workflow Ævar Arnfjörð Bjarmason
2022-04-21 18:36 ` [RFC PATCH v5 09/10] ci: use `--github-workflow-markup` " Ævar Arnfjörð Bjarmason
2022-04-21 18:36 ` [RFC PATCH v5 10/10] ci: call `finalize_test_case_output` a little later Ævar Arnfjörð Bjarmason
2022-05-25 11:25 ` Ævar Arnfjörð Bjarmason [this message]
2022-05-25 11:25 ` [PATCH v6 01/14] CI: don't "cd" in ci/print-test-failures.sh Ævar Arnfjörð Bjarmason
2022-05-25 11:25 ` [PATCH v6 02/14] CI: don't include "test-results/" in ci/print-test-failures.sh output Ævar Arnfjörð Bjarmason
2022-05-25 11:25 ` [PATCH v6 03/14] CI: stop setting FAILED_TEST_ARTIFACTS N times Ævar Arnfjörð Bjarmason
2022-05-25 11:25 ` [PATCH v6 04/14] tests: refactor --write-junit-xml code Ævar Arnfjörð Bjarmason
2022-05-25 11:25 ` [PATCH v6 05/14] test(junit): avoid line feeds in XML attributes Ævar Arnfjörð Bjarmason
2022-05-25 11:25 ` [PATCH v6 06/14] ci: optionally mark up output in the GitHub workflow Ævar Arnfjörð Bjarmason
2022-05-25 11:25 ` [PATCH v6 07/14] ci: make it easier to find failed tests' logs " Ævar Arnfjörð Bjarmason
2022-05-25 11:25 ` [PATCH v6 08/14] ci(github): skip the logs of the successful test cases Ævar Arnfjörð Bjarmason
2022-05-25 11:25 ` [PATCH v6 09/14] ci(github): skip "skip" tests in --github-workflow-markup Ævar Arnfjörð Bjarmason
2022-05-25 11:25 ` [PATCH v6 10/14] ci(github): avoid printing test case preamble twice Ævar Arnfjörð Bjarmason
2022-05-25 11:26 ` [PATCH v6 11/14] ci(github): mention where the full logs can be found Ævar Arnfjörð Bjarmason
2022-05-25 11:26 ` [PATCH v6 12/14] ci: call `finalize_test_case_output` a little later Ævar Arnfjörð Bjarmason
2022-05-25 11:26 ` [PATCH v6 13/14] CI: make --github-workflow-markup ci-config, off by default Ævar Arnfjörð Bjarmason
2022-05-25 11:26 ` [PATCH v6 14/14] CI: make the --github-workflow-markup "github" output the default Ævar Arnfjörð Bjarmason
2022-03-25 20:43 ` [PATCH v2 00/25] CI: run "make [test]" directly, use $GITHUB_ENV Victoria Dye
2022-03-26 0:59 ` Ævar Arnfjörð Bjarmason
2022-04-18 18:33 ` Phillip Wood
2022-03-28 16:34 ` Junio C Hamano
2022-04-05 14:36 ` Johannes Schindelin
2022-04-06 9:29 ` Ævar Arnfjörð Bjarmason
2022-04-06 15:53 ` Junio C Hamano
2022-04-13 19:51 ` [PATCH v3 00/29] CI: run "make" in CI "steps", improve UX Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 01/29] CI: run "set -ex" early in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 02/29] CI: make "$jobname" explicit, remove fallback Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 03/29] CI: remove more dead Travis CI support Ævar Arnfjörð Bjarmason
2022-04-14 6:18 ` Eric Sunshine
2022-04-13 19:51 ` [PATCH v3 04/29] CI: remove dead "tree skipping" code Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 05/29] CI: remove unused Azure ci/* code Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 06/29] CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 07/29] CI: don't have "git grep" invoke a pager in tree content check Ævar Arnfjörð Bjarmason
2022-04-14 6:17 ` Eric Sunshine
2022-04-13 19:51 ` [PATCH v3 08/29] CI: have "static-analysis" run a "make ci-static-analysis" target Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 09/29] CI: have "static-analysis" run "check-builtins", not "documentation" Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 10/29] CI: move p4 and git-lfs variables to ci/install-dependencies.sh Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 11/29] CI: consistently use "export" in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 12/29] CI: export variables via a wrapper Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 13/29] CI: remove "run-build-and-tests.sh", run "make [test]" directly Ævar Arnfjörð Bjarmason
2022-04-14 6:46 ` Eric Sunshine
2022-04-13 19:51 ` [PATCH v3 14/29] ci/lib.sh: use "test" instead of "[" Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 15/29] CI: check ignored unignored build artifacts in "win[+VS] build" too Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 16/29] CI: invoke "make artifacts-tar" directly in windows-build Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 17/29] CI: split up and reduce "ci/test-documentation.sh" Ævar Arnfjörð Bjarmason
2022-04-14 6:51 ` Eric Sunshine
2022-04-13 19:51 ` [PATCH v3 18/29] CI: combine ci/install{,-docker}-dependencies.sh Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 19/29] CI: move "env" definitions into ci/lib.sh Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 20/29] ci/run-test-slice.sh: replace shelling out with "echo" Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 21/29] CI: pre-select test slice in Windows & VS tests Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 22/29] CI: only invoke ci/lib.sh as "steps" in main.yml Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 23/29] CI: narrow down variable definitions in --build and --test Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 24/29] CI: add more variables to MAKEFLAGS, except under vs-build Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment Ævar Arnfjörð Bjarmason
2022-04-14 7:38 ` Eric Sunshine
2022-04-15 2:21 ` Carlo Arenas
2022-04-15 13:47 ` Ævar Arnfjörð Bjarmason
2022-04-15 16:59 ` Junio C Hamano
2022-04-15 19:03 ` Carlo Arenas
2022-04-15 20:28 ` Carlo Arenas
2022-04-13 19:51 ` [PATCH v3 26/29] CI: set SANITIZE=leak in MAKEFLAGS directly Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 27/29] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 28/29] CI: don't use "set -x" in "ci/lib.sh" output Ævar Arnfjörð Bjarmason
2022-04-13 19:51 ` [PATCH v3 29/29] CI: make it easy to use ci/*.sh outside of CI Ævar Arnfjörð Bjarmason
2022-04-14 8:00 ` Eric Sunshine
2022-04-14 13:51 ` Ævar Arnfjörð Bjarmason
2022-04-13 21:08 ` [PATCH v3 00/29] CI: run "make" in CI "steps", improve UX Junio C Hamano
2022-04-18 16:28 ` [PATCH v4 00/31] " Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 01/31] CI: run "set -ex" early in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 02/31] CI: make "$jobname" explicit, remove fallback Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 03/31] CI: remove more dead Travis CI support Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 04/31] CI: remove dead "tree skipping" code Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 05/31] CI: remove unused Azure ci/* code Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 06/31] CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 07/31] CI: don't have "git grep" invoke a pager in tree content check Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 08/31] CI: have "static-analysis" run a "make ci-static-analysis" target Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 09/31] CI: have "static-analysis" run "check-builtins", not "documentation" Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 10/31] CI: move p4 and git-lfs variables to ci/install-dependencies.sh Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 11/31] CI: consistently use "export" in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 12/31] CI: export variables via a wrapper Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 13/31] CI: remove "run-build-and-tests.sh", run "make [test]" directly Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 14/31] ci/lib.sh: use "test" instead of "[" Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 15/31] CI: check ignored unignored build artifacts in "win[+VS] build" too Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 16/31] CI: invoke "make artifacts-tar" directly in windows-build Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 17/31] CI: split up and reduce "ci/test-documentation.sh" Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 18/31] CI: make ci/install-dependencies.sh POSIX-compatible Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 19/31] CI: combine ci/install{,-docker}-dependencies.sh Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 20/31] CI: move "env" definitions into ci/lib.sh Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 21/31] ci/run-test-slice.sh: replace shelling out with "echo" Ævar Arnfjörð Bjarmason
2022-04-18 16:28 ` [PATCH v4 22/31] CI: pre-select test slice in Windows & VS tests Ævar Arnfjörð Bjarmason
2022-04-18 16:29 ` [PATCH v4 23/31] CI: only invoke ci/lib.sh as "steps" in main.yml Ævar Arnfjörð Bjarmason
2022-04-18 16:29 ` [PATCH v4 24/31] CI: narrow down variable definitions in --build and --test Ævar Arnfjörð Bjarmason
2022-04-18 16:29 ` [PATCH v4 25/31] CI: add more variables to MAKEFLAGS, except under vs-build Ævar Arnfjörð Bjarmason
2022-04-18 16:29 ` [PATCH v4 26/31] CI: set CC in MAKEFLAGS directly, don't add it to the environment Ævar Arnfjörð Bjarmason
2022-04-18 16:29 ` [PATCH v4 27/31] CI: set SANITIZE=leak in MAKEFLAGS directly Ævar Arnfjörð Bjarmason
2022-04-18 16:29 ` [PATCH v4 28/31] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case Ævar Arnfjörð Bjarmason
2022-04-18 16:29 ` [PATCH v4 29/31] CI: don't use "set -x" in "ci/lib.sh" output Ævar Arnfjörð Bjarmason
2022-04-18 16:29 ` [PATCH v4 30/31] CI: have osx-gcc use gcc, not clang Ævar Arnfjörð Bjarmason
2022-04-18 18:38 ` Phillip Wood
2022-04-20 8:13 ` Junio C Hamano
2022-04-20 10:11 ` Phillip Wood
2022-04-20 18:45 ` Junio C Hamano
2022-04-21 12:57 ` js/ci-github-workflow-markup, was " Johannes Schindelin
2022-04-21 19:24 ` Ævar Arnfjörð Bjarmason
2022-04-21 20:12 ` Junio C Hamano
2022-04-18 16:29 ` [PATCH v4 31/31] CI: make it easy to use ci/*.sh outside of CI Ævar Arnfjörð Bjarmason
2022-04-18 17:57 ` Eric Sunshine
2022-04-19 9:31 ` Ævar Arnfjörð Bjarmason
2022-04-21 18:22 ` [PATCH v5 00/29] CI: run "make" in CI "steps", improve UX Ævar Arnfjörð Bjarmason
2022-04-21 18:22 ` [PATCH v5 01/29] CI: run "set -ex" early in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-04-21 18:22 ` [PATCH v5 02/29] CI: make "$jobname" explicit, remove fallback Ævar Arnfjörð Bjarmason
2022-04-21 18:22 ` [PATCH v5 03/29] CI: remove more dead Travis CI support Ævar Arnfjörð Bjarmason
2022-04-21 18:22 ` [PATCH v5 04/29] CI: remove dead "tree skipping" code Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 05/29] CI: remove unused Azure ci/* code Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 06/29] CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 07/29] CI: don't have "git grep" invoke a pager in tree content check Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 08/29] CI: have "static-analysis" run a "make ci-static-analysis" target Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 09/29] CI: have "static-analysis" run "check-builtins", not "documentation" Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 10/29] CI: move p4 and git-lfs variables to ci/install-dependencies.sh Ævar Arnfjörð Bjarmason
2022-04-23 20:11 ` [PATCH] fixup! " Carlo Marcelo Arenas Belón
2022-04-23 22:51 ` Junio C Hamano
2022-04-23 23:47 ` Carlo Arenas
2022-04-24 0:25 ` Carlo Arenas
2022-04-21 18:23 ` [PATCH v5 11/29] CI: consistently use "export" in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 12/29] CI: export variables via a wrapper Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 13/29] CI: remove "run-build-and-tests.sh", run "make [test]" directly Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 14/29] CI: make ci/{lib,install-dependencies}.sh POSIX-compatible Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 15/29] CI: check ignored unignored build artifacts in "win[+VS] build" too Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 16/29] CI: invoke "make artifacts-tar" directly in windows-build Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 17/29] CI: split up and reduce "ci/test-documentation.sh" Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 18/29] CI: combine ci/install{,-docker}-dependencies.sh Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 19/29] CI: move "env" definitions into ci/lib.sh Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 20/29] ci/run-test-slice.sh: replace shelling out with "echo" Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 21/29] CI: pre-select test slice in Windows & VS tests Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 22/29] CI: only invoke ci/lib.sh as "steps" in main.yml Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 23/29] CI: narrow down variable definitions in --build and --test Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 24/29] CI: add more variables to MAKEFLAGS, except under vs-build Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 26/29] CI: set SANITIZE=leak in MAKEFLAGS directly Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 27/29] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 28/29] CI: don't use "set -x" in "ci/lib.sh" output Ævar Arnfjörð Bjarmason
2022-04-21 18:23 ` [PATCH v5 29/29] CI: make it easy to use ci/*.sh outside of CI Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 00/29] CI: run "make" in CI "steps", improve UX Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 01/29] CI: run "set -ex" early in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 02/29] CI: make "$jobname" explicit, remove fallback Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 03/29] CI: remove more dead Travis CI support Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 04/29] CI: remove dead "tree skipping" code Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 05/29] CI: remove unused Azure ci/* code Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 06/29] CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 07/29] CI: don't have "git grep" invoke a pager in tree content check Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 08/29] CI: have "static-analysis" run a "make ci-static-analysis" target Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 09/29] CI: have "static-analysis" run "check-builtins", not "documentation" Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 10/29] CI: move p4 and git-lfs variables to ci/install-dependencies.sh Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 11/29] CI: consistently use "export" in ci/lib.sh Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 12/29] CI: export variables via a wrapper Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 13/29] CI: remove "run-build-and-tests.sh", run "make [test]" directly Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 14/29] CI: make ci/{lib,install-dependencies}.sh POSIX-compatible Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 15/29] CI: check ignored unignored build artifacts in "win[+VS] build" too Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 16/29] CI: invoke "make artifacts-tar" directly in windows-build Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 17/29] CI: split up and reduce "ci/test-documentation.sh" Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 18/29] CI: combine ci/install{,-docker}-dependencies.sh Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 19/29] CI: move "env" definitions into ci/lib.sh Ævar Arnfjörð Bjarmason
2022-05-25 10:03 ` [PATCH v6 20/29] ci/run-test-slice.sh: replace shelling out with "echo" Ævar Arnfjörð Bjarmason
2022-05-25 10:04 ` [PATCH v6 21/29] CI: pre-select test slice in Windows & VS tests Ævar Arnfjörð Bjarmason
2022-05-25 10:04 ` [PATCH v6 22/29] CI: only invoke ci/lib.sh as "steps" in main.yml Ævar Arnfjörð Bjarmason
2022-05-25 10:04 ` [PATCH v6 23/29] CI: narrow down variable definitions in --build and --test Ævar Arnfjörð Bjarmason
2022-05-25 10:04 ` [PATCH v6 24/29] CI: add more variables to MAKEFLAGS, except under vs-build Ævar Arnfjörð Bjarmason
2022-05-25 10:04 ` [PATCH v6 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment Ævar Arnfjörð Bjarmason
2022-05-25 10:04 ` [PATCH v6 26/29] CI: set SANITIZE=leak in MAKEFLAGS directly Ævar Arnfjörð Bjarmason
2022-05-25 10:04 ` [PATCH v6 27/29] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case Ævar Arnfjörð Bjarmason
2022-05-25 10:04 ` [PATCH v6 28/29] CI: don't use "set -x" in "ci/lib.sh" output Ævar Arnfjörð Bjarmason
2022-05-25 10:04 ` [PATCH v6 29/29] CI: make it easy to use ci/*.sh outside of CI Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover-v6-00.14-00000000000-20220525T100743Z-avarab@gmail.com \
--to=avarab@gmail.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=carenas@gmail.com \
--cc=congdanhqx@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=larsxschneider@gmail.com \
--cc=matheus.bernardino@usp.br \
--cc=me@ttaylorr.com \
--cc=sunshine@sunshineco.com \
--cc=szeder.dev@gmail.com \
--cc=vdye@github.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).