git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jeff King <peff@peff.net>
To: git@vger.kernel.org
Cc: Junio C Hamano <gitster@pobox.com>,
	Eric Sunshine <sunshine@sunshineco.com>,
	Phillip Wood <phillip.wood123@gmail.com>,
	Michael J Gruber <git@grubix.eu>
Subject: [PATCH v2 1/5] tests: run internal chain-linter under "make test"
Date: Thu, 30 Mar 2023 15:27:38 -0400	[thread overview]
Message-ID: <20230330192738.GA27989@coredump.intra.peff.net> (raw)
In-Reply-To: <20230330192712.GA27719@coredump.intra.peff.net>

Since 69b9924b875 (t/Makefile: teach `make test` and `make prove` to run
chainlint.pl, 2022-09-01), we run a single chainlint.pl process for all
scripts, and then instruct each individual script to run with the
equivalent of --no-chain-lint, which tells them not to redundantly run
the chainlint script themselves.

However, this also disables the internal linter run within the shell by
eval-ing "(exit 117) && $1" and confirming we get code 117. In theory
the external linter produces a superset of complaints, and we don't need
the internal one anymore. However, we know there is at least one case
where they differ. A test like:

	test_expect_success 'should fail linter' '
		false &&
		sleep 2 &
		pid=$! &&
		kill $pid
	'

is buggy (it ignores the failure from "false", because it is
backgrounded along with the sleep). The internal linter catches this,
but the external one doesn't (and teaching it to do so is complicated[1]).
So not only does "make test" miss this problem, but it's doubly
confusing because running the script standalone does complain.

Let's teach the suppression in the Makefile to only turn off the
external linter (which we know is redundant, as it was already run) and
leave the internal one intact.

I've used a new environment variable to do this here, and intentionally
did not add a "--no-ext-chain-lint" option. This is an internal
optimization used by the Makefile, and not something that ordinary users
would need to tweak.

[1] For discussion of chainlint.pl and this case, see:
    https://lore.kernel.org/git/CAPig+cQtLFX4PgXyyK_AAkCvg4Aw2RAC5MmLbib-aHHgTBcDuw@mail.gmail.com/

Signed-off-by: Jeff King <peff@peff.net>
---
Same as before, but tweaking the Makefile comment.

 t/Makefile    | 4 ++--
 t/test-lib.sh | 3 ++-
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/t/Makefile b/t/Makefile
index 88fa5049573..3e00cdd801d 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -44,8 +44,8 @@ CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl
 
 # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`)
 # checks all tests in all scripts via a single invocation, so tell individual
-# scripts not to "chainlint" themselves
-CHAINLINTSUPPRESS = GIT_TEST_CHAIN_LINT=0 && export GIT_TEST_CHAIN_LINT &&
+# scripts not to run the external "chainlint.pl" script themselves
+CHAINLINTSUPPRESS = GIT_TEST_EXT_CHAIN_LINT=0 && export GIT_TEST_EXT_CHAIN_LINT &&
 
 all: $(DEFAULT_TEST_TARGET)
 
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 62136caee5a..09789566374 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1593,7 +1593,8 @@ then
 	BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true"
 fi
 
-if test "${GIT_TEST_CHAIN_LINT:-1}" != 0
+if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&
+   test "${GIT_TEST_EXT_CHAIN_LINT:-1}" != 0
 then
 	"$PERL_PATH" "$TEST_DIRECTORY/chainlint.pl" "$0" ||
 		BUG "lint error (see '?!...!? annotations above)"
-- 
2.40.0.692.g7c4c956fc5c


  reply	other threads:[~2023-03-30 19:27 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-28 20:20 [PATCH 0/4] some chainlint fixes and performance improvements Jeff King
2023-03-28 20:22 ` [PATCH 1/4] tests: run internal chain-linter under "make test" Jeff King
2023-03-29 10:20   ` Ævar Arnfjörð Bjarmason
2023-03-29 15:49     ` Junio C Hamano
2023-03-29 23:28       ` Jeff King
2023-03-30 18:45         ` Junio C Hamano
2023-03-28 20:23 ` [PATCH 2/4] tests: replace chainlint subshell with a function Jeff King
2023-03-28 20:40   ` Junio C Hamano
2023-03-28 20:28 ` [PATCH 3/4] tests: drop here-doc check from internal chain-linter Jeff King
2023-03-28 21:46   ` Junio C Hamano
2023-03-29  2:37     ` Jeff King
2023-03-29  3:04       ` Jeff King
2023-03-29  3:13         ` Eric Sunshine
2023-03-29  3:46           ` Eric Sunshine
2023-03-29  4:02             ` Eric Sunshine
2023-03-29  6:07             ` Jeff King
2023-03-29  6:28               ` Eric Sunshine
2023-03-29  3:07       ` Eric Sunshine
2023-03-29  6:28         ` Jeff King
2023-03-28 20:28 ` [PATCH 4/4] tests: skip test_eval_ in internal chain-lint Jeff King
2023-03-28 21:08 ` [PATCH 0/4] some chainlint fixes and performance improvements Jeff King
2023-03-30 22:08   ` Jeff King
2023-03-30 22:16     ` Junio C Hamano
2023-03-30 19:27 ` [PATCH v2 0/5] " Jeff King
2023-03-30 19:27   ` Jeff King [this message]
2023-03-30 19:27   ` [PATCH v2 2/5] tests: replace chainlint subshell with a function Jeff King
2023-03-30 19:30   ` [PATCH v2 3/5] tests: diagnose unclosed here-doc in chainlint.pl Jeff King
2023-03-30 21:26     ` Eric Sunshine
2023-03-30 19:30   ` [PATCH v2 4/5] tests: drop here-doc check from internal chain-linter Jeff King
2023-03-30 19:30   ` [PATCH v2 5/5] tests: skip test_eval_ in internal chain-lint Jeff King
2023-03-30 20:32   ` [PATCH v2 0/5] some chainlint fixes and performance improvements Junio C Hamano
2023-03-30 22:09     ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230330192738.GA27989@coredump.intra.peff.net \
    --to=peff@peff.net \
    --cc=git@grubix.eu \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=phillip.wood123@gmail.com \
    --cc=sunshine@sunshineco.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).