git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
	"Shubham Mishra" <shivam828787@gmail.com>,
	"Christian Couder" <christian.couder@gmail.com>,
	"Taylor Blau" <me@ttaylorr.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH 00/15] tests: don't ignore "git" exit codes
Date: Wed,  2 Mar 2022 18:27:09 +0100	[thread overview]
Message-ID: <cover-00.15-00000000000-20220302T171755Z-avarab@gmail.com> (raw)

This series fixes issues where we ignored the exit code of "git" due
to it being on the LHS of a pipe, or because we interpolated its
output with $() in a "test" construct, or had missing &&- chains in
helper functions etc.

This series is not made by string-replacing things in our test suite,
if it was it would be much larger. These are all tests I've seen real
hide real failures under SANITIZE=leak, either on current "master", or
in combination with various local leak fixes I've got unsubmitted.

In cases where I was starting to fix a pattern in a file I'd fix the
rest of the file if it was easy, but otherwise these are all cases
where I ran SANITIZE=leak, had a test pass, but having ASAN_OPTIONS
log to a file revealed that we had memory leaks within that test.

As an aside we still have various potential issues with hidden
segfaults etc. in the test suite after this that are tricked to solve,
because:

 * Our tests will (mostly) catch segfaults and abort(), but if we
   invoke a command that invokes another command it needs to ferry the
   exit code up to us.

 * run-command.c notably does not do that, so for e.g. "git push"
   tests where we expect a failure and an underlying "git" command
   fails we won't ferry up the segfault or abort exit code.

 * We have gitweb.perl and some other perl code ignoring return values
   from close(), i.e. ignoring exit codes from "git rev-parse" et al.

 * We have in-tree shellscripts like "git-merge-one-file.sh" invoking
   git commands, and if they fail returning "1", not ferrying up the
   segfault or abort() exit code.

Ævar Arnfjörð Bjarmason (15):
  tests: change some 'test $(git) = "x"' to test_cmp
  tests: use "test_stdout_line_count", not "test $(git [...] | wc -l)"
  read-tree tests: check "diff-files" exit code on failure
  diff tests: don't ignore "git diff" exit code
  diff tests: don't ignore "git diff" exit code in "read" loop
  apply tests: use "test_must_fail" instead of ad-hoc pattern
  merge tests: use "test_must_fail" instead of ad-hoc pattern
  rev-parse tests: don't ignore "git reflog" exit code
  notes tests: don't ignore "git" exit code
  diff tests: don't ignore "git rev-list" exit code
  rev-list tests: don't hide abort() in "test_expect_failure"
  gettext tests: don't ignore "test-tool regex" exit code
  apply tests: don't ignore "git ls-files" exit code, drop sub-shell
  checkout tests: don't ignore "git <cmd>" exit code
  rev-list simplify tests: don't ignore "git" exit code

 t/t0002-gitfile.sh                     |   6 +-
 t/t1001-read-tree-m-2way.sh            |   6 +-
 t/t1002-read-tree-m-u-2way.sh          |   6 +-
 t/t1503-rev-parse-verify.sh            |   5 +-
 t/t2012-checkout-last.sh               |  51 ++++++---
 t/t2200-add-update.sh                  |  33 ++++--
 t/t3302-notes-index-expensive.sh       |   6 +-
 t/t3303-notes-subtrees.sh              |   9 +-
 t/t3305-notes-fanout.sh                |  14 +--
 t/t4020-diff-external.sh               | 153 ++++++++++++-------------
 t/t4027-diff-submodule.sh              |   7 +-
 t/t4123-apply-shrink.sh                |  18 +--
 t/t4128-apply-root.sh                  |  36 +++---
 t/t6005-rev-list-count.sh              |  43 ++++---
 t/t6012-rev-list-simplify.sh           |  12 +-
 t/t6102-rev-list-unexpected-objects.sh |  13 ++-
 t/t6407-merge-binary.sh                |  22 +---
 t/t7103-reset-bare.sh                  |   7 +-
 t/t7812-grep-icase-non-ascii.sh        |  11 +-
 19 files changed, 240 insertions(+), 218 deletions(-)

-- 
2.35.1.1226.g8b497615d32


             reply	other threads:[~2022-03-02 17:27 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-02 17:27 Ævar Arnfjörð Bjarmason [this message]
2022-03-02 17:27 ` [PATCH 01/15] tests: change some 'test $(git) = "x"' to test_cmp Ævar Arnfjörð Bjarmason
2022-03-02 17:27 ` [PATCH 02/15] tests: use "test_stdout_line_count", not "test $(git [...] | wc -l)" Ævar Arnfjörð Bjarmason
2022-03-02 17:27 ` [PATCH 03/15] read-tree tests: check "diff-files" exit code on failure Ævar Arnfjörð Bjarmason
2022-03-02 23:06   ` Junio C Hamano
2022-03-02 17:27 ` [PATCH 04/15] diff tests: don't ignore "git diff" exit code Ævar Arnfjörð Bjarmason
2022-03-02 17:27 ` [PATCH 05/15] diff tests: don't ignore "git diff" exit code in "read" loop Ævar Arnfjörð Bjarmason
2022-03-02 17:27 ` [PATCH 06/15] apply tests: use "test_must_fail" instead of ad-hoc pattern Ævar Arnfjörð Bjarmason
2022-03-02 23:09   ` Junio C Hamano
2022-03-02 17:27 ` [PATCH 07/15] merge " Ævar Arnfjörð Bjarmason
2022-03-02 17:27 ` [PATCH 08/15] rev-parse tests: don't ignore "git reflog" exit code Ævar Arnfjörð Bjarmason
2022-03-02 17:27 ` [PATCH 09/15] notes tests: don't ignore "git" " Ævar Arnfjörð Bjarmason
2022-03-02 17:27 ` [PATCH 10/15] diff tests: don't ignore "git rev-list" " Ævar Arnfjörð Bjarmason
2022-03-02 17:27 ` [PATCH 11/15] rev-list tests: don't hide abort() in "test_expect_failure" Ævar Arnfjörð Bjarmason
2022-03-02 23:16   ` Junio C Hamano
2022-03-03 10:10     ` Ævar Arnfjörð Bjarmason
2022-03-02 17:27 ` [PATCH 12/15] gettext tests: don't ignore "test-tool regex" exit code Ævar Arnfjörð Bjarmason
2022-03-02 23:20   ` Junio C Hamano
2022-03-03 15:46     ` Ævar Arnfjörð Bjarmason
2022-03-03 20:58       ` Junio C Hamano
2022-03-02 17:27 ` [PATCH 13/15] apply tests: don't ignore "git ls-files" exit code, drop sub-shell Ævar Arnfjörð Bjarmason
2022-03-02 17:27 ` [PATCH 14/15] checkout tests: don't ignore "git <cmd>" exit code Ævar Arnfjörð Bjarmason
2022-03-02 17:27 ` [PATCH 15/15] rev-list simplify tests: don't ignore "git" " Ævar Arnfjörð Bjarmason
2022-03-02 23:23 ` [PATCH 00/15] tests: don't ignore "git" exit codes Junio C Hamano
2022-03-03  2:02 ` Derrick Stolee
2022-03-03 10:13   ` Ævar Arnfjörð Bjarmason
2022-03-03 14:06   ` Phillip Wood
2022-03-03 15:35     ` Ævar Arnfjörð Bjarmason
2022-03-04  2:44       ` Junio C Hamano
2022-03-04  6:57         ` Ævar Arnfjörð Bjarmason
2022-03-03  5:09 ` Shubham Mishra
2022-03-03  9:53   ` Ævar Arnfjörð Bjarmason
2022-03-07 12:48 ` [PATCH v2 " Ævar Arnfjörð Bjarmason
2022-03-07 12:48   ` [PATCH v2 01/15] tests: change some 'test $(git) = "x"' to test_cmp Ævar Arnfjörð Bjarmason
2022-03-07 12:48   ` [PATCH v2 02/15] tests: use "test_stdout_line_count", not "test $(git [...] | wc -l)" Ævar Arnfjörð Bjarmason
2022-03-07 12:48   ` [PATCH v2 03/15] read-tree tests: check "diff-files" exit code on failure Ævar Arnfjörð Bjarmason
2022-03-07 12:48   ` [PATCH v2 04/15] diff tests: don't ignore "git diff" exit code Ævar Arnfjörð Bjarmason
2022-03-07 12:48   ` [PATCH v2 05/15] diff tests: don't ignore "git diff" exit code in "read" loop Ævar Arnfjörð Bjarmason
2022-03-07 12:48   ` [PATCH v2 06/15] apply tests: use "test_must_fail" instead of ad-hoc pattern Ævar Arnfjörð Bjarmason
2022-03-07 12:48   ` [PATCH v2 07/15] merge " Ævar Arnfjörð Bjarmason
2022-03-07 12:48   ` [PATCH v2 08/15] rev-parse tests: don't ignore "git reflog" exit code Ævar Arnfjörð Bjarmason
2022-03-07 12:49   ` [PATCH v2 09/15] notes tests: don't ignore "git" " Ævar Arnfjörð Bjarmason
2022-03-07 12:49   ` [PATCH v2 10/15] diff tests: don't ignore "git rev-list" " Ævar Arnfjörð Bjarmason
2022-03-07 12:49   ` [PATCH v2 11/15] rev-list tests: don't hide abort() in "test_expect_failure" Ævar Arnfjörð Bjarmason
2022-03-07 12:49   ` [PATCH v2 12/15] gettext tests: don't ignore "test-tool regex" exit code Ævar Arnfjörð Bjarmason
2022-03-07 12:49   ` [PATCH v2 13/15] apply tests: don't ignore "git ls-files" exit code, drop sub-shell Ævar Arnfjörð Bjarmason
2022-03-07 12:49   ` [PATCH v2 14/15] checkout tests: don't ignore "git <cmd>" exit code Ævar Arnfjörð Bjarmason
2022-03-07 12:49   ` [PATCH v2 15/15] rev-list simplify tests: don't ignore "git" " Ævar Arnfjörð Bjarmason

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover-00.15-00000000000-20220302T171755Z-avarab@gmail.com \
    --to=avarab@gmail.com \
    --cc=christian.couder@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=me@ttaylorr.com \
    --cc=shivam828787@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).