Re: [PATCH 2/2] t/Makefile: always test all lint targets when running tests

From: Jeff King <peff@peff.net>
To: Junio C Hamano <gitster@pobox.com>
Cc: "Jens Lehmann" <Jens.Lehmann@web.de>,
	"Git Mailing List" <git@vger.kernel.org>,
	"Torsten Bögershausen" <tboegi@web.de>
Subject: Re: [PATCH 2/2] t/Makefile: always test all lint targets when running tests
Date: Wed, 9 Jul 2014 01:30:05 -0400	[thread overview]
Message-ID: <20140709053005.GD2318@sigill.intra.peff.net> (raw)
In-Reply-To: <xmqq38eddolk.fsf@gitster.dls.corp.google.com>

On Mon, Jul 07, 2014 at 11:13:11AM -0700, Junio C Hamano wrote:

> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
> > Only the two targets "test-lint-duplicates" and "test-lint-executable" are
> > currently executed when running the test target. This was done on purpose
> > when the TEST_LINT variable was added in 81127d74. But as this does not
> > include the "test-lint-shell-syntax" target added the same day in commit
> > c7ce70ac, it is easy to accidentally add non portable shell constructs
> > without noticing that when running the test suite.
> 
> I not running the lint-shell-syntax that is fundamentally flaky to
> avoid false positives is very much on purpose.  The flakiness is not
> the fault of the implementor of the lint-shell-syntax, but comes
> from the approach taken to pretend that simple pattern matching can
> parse shell scripts.  It may not complain on the current set of
> scripts, but that is not really by design but by accident.
> 
> So I am not very enthusiastic to see this change myself.

Let me play devil's advocate for a moment.

Is lint-shell-syntax in fact flaky? I know we discussed false positives
when it was originally added, but I think the current implementation
tries hard to avoid them. Given that it provides no false positives on
the current code base (without many people running it), it seems likely
to stay that way. And the cost if we are wrong is either fixing the tool
or disabling it (so worst case we are back where we started, modulo a
little effort to enable it and then revert).

What do we gain? We have an extra line of defense that helps newer shell
script writers fix their bugs before they make it to the list. That
catches more bugs, and reduces effort for reviewers. And it is exactly
these newer shell script writers that need the default flipped; they do
not know about portability and the lint target in the first place.

I dunno. I am not that enthusiastic about the change, either, but I tend
to think it will probably not hurt, and may help.

-Peff