Re: [PATCH 7/9] t9902: fix completion tests for log.d* to match log.diffMerges

From: "SZEDER Gábor" <szeder.dev@gmail.com>
To: Sergey Organov <sorganov@gmail.com>
Cc: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
	"Junio C Hamano" <gitster@pobox.com>, "Jeff King" <peff@peff.net>,
	"Philip Oakley" <philipoakley@iee.email>,
	"Elijah Newren" <newren@gmail.com>,
	"Felipe Contreras" <felipe.contreras@gmail.com>,
	git@vger.kernel.org
Subject: Re: [PATCH 7/9] t9902: fix completion tests for log.d* to match log.diffMerges
Date: Fri, 9 Apr 2021 00:13:43 +0200	[thread overview]
Message-ID: <20210408221343.GC2947267@szeder.dev> (raw)
In-Reply-To: <87k0pc7cap.fsf@osv.gnss.ru>

On Thu, Apr 08, 2021 at 11:26:38PM +0300, Sergey Organov wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> 
> > On Thu, Apr 08 2021, Sergey Organov wrote:
> >
> >> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> >>
> >>> On Thu, Apr 08 2021, Sergey Organov wrote:
> >>>
> >>>> There were 3 completion tests failures due to introduction of
> >>>> log.diffMerges configuration variable that affected the result of
> >>>> completion of log.d. Fixed them accordingly.
> >>>>
> >>>> Signed-off-by: Sergey Organov <sorganov@gmail.com>
> >>>> ---
> >>>>  t/t9902-completion.sh | 3 +++
> >>>>  1 file changed, 3 insertions(+)
> >>>>
> >>>> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> >>>> index 04ce884ef5ac..4d732d6d4f81 100755
> >>>> --- a/t/t9902-completion.sh
> >>>> +++ b/t/t9902-completion.sh
> >>>> @@ -2306,6 +2306,7 @@ test_expect_success 'git config - variable name' '
> >>>>  	test_completion "git config log.d" <<-\EOF
> >>>>  	log.date Z
> >>>>  	log.decorate Z
> >>>> +	log.diffMerges Z
> >>>>  	EOF
> >>>>  '
> >>>>  
> >>>> @@ -2327,6 +2328,7 @@ test_expect_success 'git -c - variable name' '
> >>>>  	test_completion "git -c log.d" <<-\EOF
> >>>>  	log.date=Z
> >>>>  	log.decorate=Z
> >>>> +	log.diffMerges=Z
> >>>>  	EOF
> >>>>  '
> >>>>  
> >>>> @@ -2348,6 +2350,7 @@ test_expect_success 'git clone --config= - variable name' '
> >>>>  	test_completion "git clone --config=log.d" <<-\EOF
> >>>>  	log.date=Z
> >>>>  	log.decorate=Z
> >>>> +	log.diffMerges=Z
> >>>>  	EOF
> >>>>  '
> >>>
> >>> Commits should be made in such a way as to not break the build/tests
> >>> partway through a series, which it seems is happening until this
> >>> fixup.

Well, actually no: it _starts_ to break with this patch, because
'log.diffMerges' is not documented yet, and it will pass again with
the last patch in the series that adds that missing piece of
documentation.

> >> Yep.
> >>
> >> Could these tests be somehow written in a more robust manner, to be
> >> protected against future additions of configuration variables that are
> >> unrelated to the features being tested? If so, I'd prefer to fix them as
> >> a prerequisite to the series rather than adding fixes to unrelated 
> >> existing tests into my patches.
> >
> > Hrm? I mean if you have a commit fixing up failing tests in an earlier
> > commit then that change should in one way or the other be made as part
> > of that earlier change.
> >
> > Yes we can skip the tests or something in the meantime, which we do
> > sometimes as part of some really large changes, but these can just be
> > squashed, no?
> 
> I mean I don't want this change at all.

You'll definitely need this change, though.

> I didn't change completion mechanism, so completion tests should not
> suddenly fail because of my changes.

We auto-generate the list of supported configuration variables from
the documentation and use that list in our Bash completion script to
list possible configuration variables for 'git config <TAB>' and 'git
-c <TAB>'.  And we want to make sure that this feature works as
intended, so we have a couple of tests that try to complete real
config variable sections and names.  You are just unlucky to introduce
a new configuraton variable that happenes to start with the same
prefix that is used in some of those tests.

> I did entirely unrelated change and
> noticed the breakage only by accident, as tests even don't fail unless
> you *install* git, not only make it. So, for example, just "make test"
> doesn't fail, while "make install; make test" will.

It might be related to a bug in the build process that doesn't update
that auto-generated list of supported configuration variables after
e.g. 'Documentation/config/log.txt' was modified; see a proposed fix
at:

  https://public-inbox.org/git/20210408212915.3060286-1-szeder.dev@gmail.com/

> It looks like something is wrong here, a bug or misfeature, or even two,
> and if it's fixed before these series, I won't need this in my series at
> all. Besides, that's yet another reason *not* to squash this change into
> an otherwise unrelated commit.

The introduction of the new configuration variable, its documentation
and this test update should all go into a single patch.  The whole
test suite must pass for every single commit.