Re: [RFC PATCH] fetch: stop emitting duplicate transfer.credentialsInUrl=warn warnings

From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: Taylor Blau <me@ttaylorr.com>
Cc: Jeff King <peff@peff.net>,
	git@vger.kernel.org, Junio C Hamano <gitster@pobox.com>,
	Derrick Stolee <derrickstolee@github.com>,
	Johannes Schindelin <johannes.schindelin@gmx.de>
Subject: Re: [RFC PATCH] fetch: stop emitting duplicate transfer.credentialsInUrl=warn warnings
Date: Tue, 01 Nov 2022 23:17:42 +0100	[thread overview]
Message-ID: <221101.86y1su2u2p.gmgdl@evledraar.gmail.com> (raw)
In-Reply-To: <Y2GHjnuyuwGpY3II@nand.local>

On Tue, Nov 01 2022, Taylor Blau wrote:

> On Tue, Nov 01, 2022 at 04:01:18AM +0100, Ævar Arnfjörð Bjarmason wrote:
>> > Yeah, I think it is crappy UX, too. It's just that I think the tests
>> > should not _asserting_ the bad behavior. At most, they should tolerate
>> > the bad behavior as a band-aid. So I think Dscho's patch is doing the
>> > right thing (and I do agree that we should fix the immediate CI pain by
>> > adjusting the tests, and letting the user-visible fix proceed
>> > independently).
>>
>> The tests aren't just asserting the bad behavior, they're also ensuring
>> that it doesn't get worse. 1 warning is ideal, 2-3 is bad, but
>> tolerable, but if we start emitting 500 of these it would be nice to
>> know.
>
> I admit that this kind of argument does not sway me.
>
> Is it likely that we would suddenly start spewing 500 such warnings? If
> we did, are there no other tests that would catch it? And even if *that*
> were the case, would nobody happen to notice it in the meantime either
> during development or when we queue an affected topic onto 'next' for
> wider testing?
>
> I guess the answer is that it's possible that we'd miss such a
> regression in all of those above places, but to me it seems extremely
> unlikely that we'd let such a regression through without noticing.

Literally 500? Probably not, that was hyperbole to make a point, but
several, low tens? Yeah, I know of at least a couple in-tree off the top
of my head.

The point, which I assumed was clear is that we literally wouldn't
notice if it were 500, and that sort of thing is a common pattern in our
tests. I.e. in most cases we'd ideally test_cmp known output (at least
to the extent of assuring ourselves that we're getting it right).

Instead we often just grep it, or don't test it at all. Sometimes for a
good reason (e.g. the output containing absolute paths), but more often
than not for no good reason.