Re: [PATCH v3 2/2] test-lib.sh: remove the now-unused "test_untraceable" facility

[Jeff King <peff@peff.net>]

On Mon, Dec 13, 2021 at 10:51:14AM -0800, Junio C Hamano wrote:

> > I don't see any argument pertinent to BASH_XTRACEFD in general in that
> > email, of in favor of its removal in particular, and there are no
> > valid arguments for its removal in earlier emails in this thread
> > either.
> 
> If I am reading Ævar right, the argument is "dash would not be fixed
> with BASH_XTRACEFD, so there needs another way that would work there,
> and if the approach happens to work also for bash, then there is no
> reason to use BASH_XTRACEFD", I think.
> 
> Now, if the way Ævar came up with to help shells with "-x" not to
> contaminate their standard error stream that our test scripts want
> to inspect is worse to write, understand, and maintain, compared to
> the way we have been writing our tests that inspect their standard
> errors, without having to worry about "-x" output thanks to the use
> of BASH_XTRACEFD, it may make a regression to developer
> productivity, but I am not sure if that is the case.

I think the method for handling this in the test scripts _is_ worse to
write, understand, and maintain. The problem to me is less that it's
ugly to workaround (which as you note in this case is not great, but not
_too_ bad), but that it's a subtle friction point that may jump up and
bite any test-writer who does something like:

  (foo && bar) 2>stderr

So my view had always been that BASH_XTRACEFD is the good solution, and
if people want to make "-x" work reliably under other shells, then I
won't stop them. But somewhere along the way Gábor did a bunch of fixes
to get things mostly running, and the use of dash with "-x" got added to
CI, so now it's a de facto requirement (if you care about CI
complaining, which we increasingly do).

Maybe that's OK. We've had fewer incidences of the problem popping up
than I would have expected.

My vision was that we'd leave BASH_XTRACEFD so people using it could
remain oblivious if they chose, but if the ship has sailed via CI, then
that might have less value.

> I think [1/2] of this same series can serve an example of how tests
> must be tweaked to live under the world order without BASH_XTRACEFD?
> Having to set and use a temporary file to capture the standard error
> output and append to it upfront looks uglier than each individual
> test locally capturing the standard error output from a single
> invocation of a helper function, but it does not look _too_ bad to
> me.  Can we find another example to argue for BASH_XTRACEFD, how
> much it makes it easier to write tests that work even under "-x"?

I think the fixes from 571e472dc4 (Merge branch 'sg/test-x', 2018-03-14)
show what had to be done to get where we are today.

-Peff