Re: [PATCH v6 4/4] Jobs based on custom runners: add job definitions for QEMU's machines

[Cleber Rosa <crosa@redhat.com>]

On Thu, Jun 10, 2021 at 2:18 AM Thomas Huth <thuth@redhat.com> wrote:
>
> On 08/06/2021 05.14, Cleber Rosa wrote:
> > The QEMU project has two machines (aarch64 and s390x) that can be used
> > for jobs that do build and run tests.  This introduces those jobs,
> > which are a mapping of custom scripts used for the same purpose.
> >
> > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > ---
> >   .gitlab-ci.d/custom-runners.yml | 208 ++++++++++++++++++++++++++++++++
> >   1 file changed, 208 insertions(+)
> >
> > diff --git a/.gitlab-ci.d/custom-runners.yml b/.gitlab-ci.d/custom-runners.yml
> > index a07b27384c..061d3cdfed 100644
> > --- a/.gitlab-ci.d/custom-runners.yml
> > +++ b/.gitlab-ci.d/custom-runners.yml
> > @@ -12,3 +12,211 @@
> >   # guarantees a fresh repository on each job run.
> >   variables:
> >     GIT_STRATEGY: clone
> > +
> > +# All ubuntu-18.04 jobs should run successfully in an environment
> > +# setup by the scripts/ci/setup/build-environment.yml task
> > +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
> > +ubuntu-18.04-s390x-all-linux-static:
> > + allow_failure: true
> > + needs: []
> > + stage: build
> > + tags:
> > + - ubuntu_18.04
> > + - s390x
> > + rules:
> > + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>
> I don't think this will work very well... sub-maintainers might want to push
> to a "staging" branch in their forked repositories, and without the s390x
> runner, the pipeline gets stuck now:
>
>   https://gitlab.com/thuth/qemu/-/pipelines/317812558
>

Hi Thomas,

As I put it in another response, I saw that actually as a feature, in
the sense that:

* people should indeed be allowed to push to their repos and leverage
their hardware, and
* "staging" is a pretty well scoped word, and has a reasonably well
defined meaning
* one would want to mimic as closely as possible what will be done
before a PR is merged

I agree that having the jobs stuck in any situation is not ideal, but
I honestly find that it would be reasonably hard to accidentally hit
that situation.  I also believe it will end up being inevitable for
entities to do a meta-analysis of the GitLab CI pipeline results,
possibly disregarding jobs that they can not run, or simply do not
care about, in their forks.

> We had the same issue in the kvm-unit-test CI, and we solved it there by
> rather making it depend on an environment variable that has to be set if the
> runner is available:
>
>   only:
>     variables:
>      - $S390X_RUNNER_AVAILABLE
>
> I think that's also nicer in case someone brings their own s390x runner and
> want to use the CI tests on other branches than staging.
>

The problem with this approach, is that it would not be enough to
protect the jobs based on variables for the architecture, as the OS
type and version also play a part in the possibility of running jobs.
For instance, suppose we get s390x machines from LinuxOne running
RHEL.  We'd need variables such as, say,
S390X_RHEL_8_4_RUNNER_AVAILABLE and S390X_RHEL_7_6_RUNNER_AVAILABLE.

> Could you please change your patch accordingly?
>

If you strongly believe now is the time to attempt to handle that
problem, I can go ahead and change it.  I stand behind my original
position that we should start with a simpler, "by convention" approach
and address the more complex scenarios as/if they come up.

>   Thanks,
>    Thomas
>

Thank you!
- Cleber.