git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
	"Derrick Stolee" <stolee@gmail.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH 00/10] leak test: add "check" test mode, mark leak-free tests
Date: Tue, 19 Jul 2022 23:05:14 +0200	[thread overview]
Message-ID: <cover-00.10-00000000000-20220719T205710Z-avarab@gmail.com> (raw)

Since release_revisions() we've had much fewer widespread leaks in the
test suite, so I think it's time to mark even more tests as running in
the linux-leaks CI job. The 9/10 here does that.

But before that we introduce a "GIT_TEST_PASSING_SANITIZE_LEAK=check"
leak mode, this uses test-lib.sh --invert-exit-code feature added here
to fail tests whose TEST_PASSES_SANITIZE_LEAK=true marking isn't
current.

I've been using this series for a while to help in submitting all the
other leak fixes I've been sending in. It's very useful to run the
"check" with "git rebase -x", i.e. to atomically mark tests as
leak-free as we fix leaks.

In adding "check" I needed to add a new
GIT_TEST_SANITIZE_LEAK_LOG=true setting, and integrate it with
"check". I.e. we have many cases where we lose exit codes from git, or
otherwise don't notice an abort, 08/10 here discusses why that is.

When "check" and "GIT_TEST_SANITIZE_LEAK_LOG=true" are combined we'll
take into account leaks that we log to decide whether to pass the
tests.

Writing out the leak logs is also very useful to analyze our remaining
leaks. I've got a local script that scrapes the logs, and emits a
sorted list of which leaks provide the best bang-for-the-buck in terms
of getting more tests passing. That's not being submitted here, but I
can follow-up and send that in if there's interest.

Ævar Arnfjörð Bjarmason (10):
  test-lib.sh: use $1, not $@ in test_known_broken_{ok,failure}_
  test-lib.sh: don't set GIT_EXIT_OK before calling test_atexit_handler
  test-lib.sh: fix GIT_EXIT_OK logic errors, use BAIL_OUT
  test-lib.sh: add a --invert-exit-code switch
  t/README: reword the "GIT_TEST_PASSING_SANITIZE_LEAK" description
  test-lib: add a SANITIZE=leak logging mode
  test-lib.sh: add a GIT_TEST_PASSING_SANITIZE_LEAK=check mode
  test-lib: have the "check" mode for SANITIZE=leak consider leak logs
  leak tests: mark passing SANITIZE=leak tests as leak-free
  log tests: don't use "exit 1" outside a sub-shell

 t/README                            |  36 +++++-
 t/t0000-basic.sh                    |  72 +++++++++++
 t/t0027-auto-crlf.sh                |   1 +
 t/t0032-reftable-unittest.sh        |   1 +
 t/t0033-safe-directory.sh           |   1 +
 t/t0050-filesystem.sh               |   1 +
 t/t0095-bloom.sh                    |   2 +
 t/t1405-main-ref-store.sh           |   1 +
 t/t1407-worktree-ref-store.sh       |   1 +
 t/t1418-reflog-exists.sh            |   1 +
 t/t1701-racy-split-index.sh         |   1 +
 t/t2006-checkout-index-basic.sh     |   1 +
 t/t2023-checkout-m.sh               |   1 +
 t/t2205-add-worktree-config.sh      |   1 +
 t/t3012-ls-files-dedup.sh           |   1 +
 t/t4017-diff-retval.sh              |   1 +
 t/t4051-diff-function-context.sh    |   1 +
 t/t4057-diff-combined-paths.sh      |   1 +
 t/t4114-apply-typechange.sh         |   1 +
 t/t4205-log-pretty-formats.sh       |   2 +-
 t/t4301-merge-tree-write-tree.sh    |   1 +
 t/t5315-pack-objects-compression.sh |   1 +
 t/t5351-unpack-large-objects.sh     |   1 +
 t/t5402-post-merge-hook.sh          |   1 +
 t/t5503-tagfollow.sh                |   1 +
 t/t6404-recursive-merge.sh          |   1 +
 t/t6405-merge-symlinks.sh           |   1 +
 t/t6407-merge-binary.sh             |   1 -
 t/t6408-merge-up-to-date.sh         |   1 +
 t/t6411-merge-filemode.sh           |   1 +
 t/t6413-merge-crlf.sh               |   1 +
 t/t6415-merge-dir-to-symlink.sh     |   1 +
 t/t6425-merge-rename-delete.sh      |   1 +
 t/t6431-merge-criscross.sh          |   1 +
 t/t7060-wtstatus.sh                 |   1 +
 t/t7062-wtstatus-ignorecase.sh      |   1 +
 t/t7110-reset-merge.sh              |   1 +
 t/t7111-reset-table.sh              |   1 +
 t/t7609-mergetool--lib.sh           |   1 +
 t/t9100-git-svn-basic.sh            |   1 -
 t/t9700-perl-git.sh                 |   6 +
 t/t9901-git-web--browse.sh          |   1 +
 t/test-lib.sh                       | 187 ++++++++++++++++++++++++----
 43 files changed, 310 insertions(+), 32 deletions(-)

-- 
2.37.1.1062.g385eac7fccf


             reply	other threads:[~2022-07-19 21:05 UTC|newest]

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-07-19 21:05 Ævar Arnfjörð Bjarmason [this message]
2022-07-19 21:05 ` [PATCH 01/10] test-lib.sh: use $1, not $@ in test_known_broken_{ok,failure}_ Ævar Arnfjörð Bjarmason
2022-07-19 21:05 ` [PATCH 02/10] test-lib.sh: don't set GIT_EXIT_OK before calling test_atexit_handler Ævar Arnfjörð Bjarmason
2022-07-19 21:05 ` [PATCH 03/10] test-lib.sh: fix GIT_EXIT_OK logic errors, use BAIL_OUT Ævar Arnfjörð Bjarmason
2022-07-19 21:05 ` [PATCH 04/10] test-lib.sh: add a --invert-exit-code switch Ævar Arnfjörð Bjarmason
2022-07-19 21:05 ` [PATCH 05/10] t/README: reword the "GIT_TEST_PASSING_SANITIZE_LEAK" description Ævar Arnfjörð Bjarmason
2022-07-20  1:38   ` Derrick Stolee
2022-07-19 21:05 ` [PATCH 06/10] test-lib: add a SANITIZE=leak logging mode Ævar Arnfjörð Bjarmason
2022-07-19 21:05 ` [PATCH 07/10] test-lib.sh: add a GIT_TEST_PASSING_SANITIZE_LEAK=check mode Ævar Arnfjörð Bjarmason
2022-07-20  1:43   ` Derrick Stolee
2022-07-19 21:05 ` [PATCH 08/10] test-lib: have the "check" mode for SANITIZE=leak consider leak logs Ævar Arnfjörð Bjarmason
2022-07-20  1:47   ` Derrick Stolee
2022-07-19 21:05 ` [PATCH 09/10] leak tests: mark passing SANITIZE=leak tests as leak-free Ævar Arnfjörð Bjarmason
2022-07-20  1:50   ` Derrick Stolee
2022-07-19 21:05 ` [PATCH 10/10] log tests: don't use "exit 1" outside a sub-shell Ævar Arnfjörð Bjarmason
2022-07-20 17:11   ` Junio C Hamano
2022-07-20 21:21 ` [PATCH v2 00/14] leak test: add "check" test mode, mark leak-free tests Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 01/14] test-lib: use $1, not $@ in test_known_broken_{ok,failure}_ Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 02/14] test-lib: don't set GIT_EXIT_OK before calling test_atexit_handler Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 03/14] test-lib: fix GIT_EXIT_OK logic errors, use BAIL_OUT Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 04/14] test-lib: add a --invert-exit-code switch Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 05/14] t/README: reword the "GIT_TEST_PASSING_SANITIZE_LEAK" description Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 06/14] test-lib: add a SANITIZE=leak logging mode Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 07/14] t/Makefile: don't remove test-results in "clean-except-prove-cache" Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 08/14] tests: move copy/pasted PERL + Test::More checks to a lib-perl.sh Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 09/14] test-lib: simplify by removing test_external Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 10/14] test-lib: add a GIT_TEST_PASSING_SANITIZE_LEAK=check mode Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 11/14] test-lib: have the "check" mode for SANITIZE=leak consider leak logs Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 12/14] leak tests: mark passing SANITIZE=leak tests as leak-free Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 13/14] upload-pack: fix a memory leak in create_pack_file() Ævar Arnfjörð Bjarmason
2022-07-20 21:21   ` [PATCH v2 14/14] CI: use "GIT_TEST_SANITIZE_LEAK_LOG=true" in linux-leaks Ævar Arnfjörð Bjarmason
2022-07-27 23:13   ` [PATCH v3 00/15] leak test: add "check" test mode, mark leak-free tests Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 01/15] test-lib: use $1, not $@ in test_known_broken_{ok,failure}_ Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 02/15] test-lib: don't set GIT_EXIT_OK before calling test_atexit_handler Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 03/15] test-lib: fix GIT_EXIT_OK logic errors, use BAIL_OUT Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 04/15] test-lib: add a --invert-exit-code switch Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 05/15] t/README: reword the "GIT_TEST_PASSING_SANITIZE_LEAK" description Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 06/15] test-lib: add a SANITIZE=leak logging mode Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 07/15] t/Makefile: don't remove test-results in "clean-except-prove-cache" Ævar Arnfjörð Bjarmason
2022-09-20 10:54       ` [PATCH] t/Makefile: remove 'test-results' on 'make clean' SZEDER Gábor
2022-09-20 19:51         ` Jeff King
2022-09-20 20:11           ` SZEDER Gábor
2022-09-20 20:42             ` Jeff King
2022-09-20 20:16         ` [PATCH v2] " SZEDER Gábor
2022-09-21  6:59           ` Ævar Arnfjörð Bjarmason
2022-09-21 17:49             ` Junio C Hamano
2022-09-21 17:52           ` Junio C Hamano
2022-09-26  9:08             ` Ævar Arnfjörð Bjarmason
2022-09-26 19:08               ` Junio C Hamano
2022-07-27 23:13     ` [PATCH v3 08/15] tests: move copy/pasted PERL + Test::More checks to a lib-perl.sh Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 09/15] test-lib: simplify by removing test_external Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 10/15] test-lib: add a GIT_TEST_PASSING_SANITIZE_LEAK=check mode Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 11/15] test-lib: have the "check" mode for SANITIZE=leak consider leak logs Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 12/15] leak tests: don't skip some tests under SANITIZE=leak Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 13/15] leak tests: mark passing SANITIZE=leak tests as leak-free Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 14/15] upload-pack: fix a memory leak in create_pack_file() Ævar Arnfjörð Bjarmason
2022-07-27 23:13     ` [PATCH v3 15/15] CI: use "GIT_TEST_SANITIZE_LEAK_LOG=true" in linux-leaks Ævar Arnfjörð Bjarmason

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover-00.10-00000000000-20220719T205710Z-avarab@gmail.com \
    --to=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=stolee@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).