yz/p4-py3, was Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Johannes Schindelin <Johannes.Schindelin@gmx.de>]

[-- Attachment #1: Type: text/plain, Size: 2285 bytes --]

Hi Junio,

On Wed, 22 Jan 2020, Junio C Hamano wrote:

> SZEDER Gábor <szeder.dev@gmail.com> writes:
>
> > On Wed, Jan 22, 2020 at 02:18:05PM -0800, Junio C Hamano wrote:
> >> * yz/p4-py3 (2020-01-15) 14 commits
> >>  - ci: also run linux-gcc pipeline with python3.5 environment
> >
> > I still think that this last patch needs to be reworked before this
> > series is merged any further.
> >
> > The only Python script we have is 'git p4', so the Python version is
> > only relevant for 'git p4' tests ('t98*'), while the rest of Git and
> > the test suite couldn't care less [1].  This patch, however, not only
> > builds Git and runs the full test suite for each of the two Python
> > versions, but, worse, runs the full test suite _twice_ for each, first
> > as a "regular" test run and then again with all the GIT_TEST_* knobs
> > enabled.  Consequently, it adds ~50mins to every build's runtime.
> >
> > That's just too wasteful.
>
> Thanks for a reminder.  Yes, I do recall you raised the above point
> and I agree with the assessment.
>
> What's the ideal endgame wrt the tests?  Build with Py$N and run
> full test suite once, and run full test suite again with the unusual
> knobs enabled, which is what is done without this series, plus build
> with Py(5-$N) and run and run only t98?? tests?

Should we declare `t98xx` to be the namespace for the Python-based
scripts, or alternatively declare that we won't ever include another
Python script but `git-p4`?

But yes, I think that we should probably "tack on" the Python 3.x tests to
the `linux-gcc` job.

Or maybe finally split this job into three: one job that does what
`linux-gcc` suggests, a `linux-gcc-knobs` one that sets all those `GIT_*`
variables, and a python3x one that only runs t98*.sh.

The reason to split it off is this: on rare occasion, I have to restart
the `linux-gcc` job because _one_ of those `git-p4` tests failed due to
some reason or other, probably timing-related, I did not have time to
investigate this. Having to re-run the entire test suite, twice, just to
work around those flaky tests is rather wasteful.

That would actually be my prereference. If people agree, I will revive
https://github.com/gitgitgadget/git/pull/266.

Ciao,
Dscho