Re: [PATCH] t1092: use GIT_PROGRESS_DELAY for consistent results

[Derrick Stolee <stolee@gmail.com>]

On 5/24/21 4:28 PM, Jonathan Nieder wrote:
> Hi,
> 
> Derrick Stolee wrote:
> 
>> The t1092-sparse-checkout-compatibility.sh tests compare the stdout and
>> stderr for several Git commands across both full checkouts, sparse
>> checkouts with a full index, and sparse checkouts with a sparse index.
>> Since these are direct comparisons, sometimes a progress indicator can
>> flush at unpredictable points, especially on slower machines. This
>> causes the tests to be flaky.
> 
> Hm, I think this test strategy is going to be fundamentally flaky
> regardless: Git doesn't intend to guarantee any kind of stability in
> the exact stderr output it writes.
> 
> Could the tests be made to check more semantically meaningful
> information such as "git ls-files -s" output instead?

The test is comparing the same exact Git command just with
different configurations. Any change to what Git writes to
stderr should be consistent across these, unless there is
an explicit reason why it would behave differently across
these options (for example, saying "You are in a sparse
checkout" in 'git status').

There are no expectations that stderr is stable across
versions of Git. These tests don't add friction to developers
making new features or changing the error messages that appear
over stderr. It's just that these tests should catch any
unintended inconsistency across these modes.

Thanks,
-Stolee