qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Daniel P. Berrangé" <berrange@redhat.com>
To: "Philippe Mathieu-Daudé" <f4bug@amsat.org>
Cc: "Peter Maydell" <peter.maydell@linaro.org>,
	"Thomas Huth" <thuth@redhat.com>,
	"Erik Skultety" <eskultet@redhat.com>,
	qemu-devel@nongnu.org,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Willian Rampazzo" <willianr@redhat.com>,
	"Stefan Hajnoczi" <stefanha@redhat.com>,
	"Cleber Rosa" <crosa@redhat.com>,
	"Miroslav Rezanina" <mrezanin@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>
Subject: Re: [RFC PATCH 14/15] gitlab-ci: Allow forks to use different set of jobs
Date: Tue, 11 May 2021 15:21:44 +0100	[thread overview]
Message-ID: <YJqS+FqVttnrw6zb@redhat.com> (raw)
In-Reply-To: <a0e83ef7-13ee-6f45-96b5-b9d848bb8a43@amsat.org>

On Tue, May 11, 2021 at 08:48:44AM +0200, Philippe Mathieu-Daudé wrote:
> +Stefan/Peter
> 
> Not sure if following up this thread or start a new one, but I got
> blocked again from Gitlab, tagged as a crypto miner for running QEMU
> CI...
> [1]
> https://about.gitlab.com/handbook/support/workflows/investigate_blocked_pipeline.html#trends--high-priority-cases
> 
> I pushed 5 different branches to my repository in less than 1h,
> kicking 580 jobs [*].
> 
> I didn't try to stress Gitlab, it was a simple "one time in the month
> rebase unmerged branches, push them before respining on the mailing
> list".
> 
> I'm considering changing my workflow:
> - not push more than 2 branches per hour (I know 3/h works, so choose
>   a lower number, as we want to add more tests).
> - merge multiple branches locally and push the merged result and
>   bisect / re-push on failure
> - run less testing
> - do not run testing
> 
> This sounds counter productive and doesn't scale to a community of
> contributors asked to use Gitlab.
> 
> So far I don't have better idea than this series.
> 
> Who is interested in sending patches to improve our workflow?

So we have a few scenarios for using the CI

 1. Running gating CI before merging to master
 2. Subsystem maintainers running CI before sending a PULL req
 3. Contributors running CI before sending a patch series

Right now we have the same jobs running in all three scenarios.

Given the increasing restrictions on usage, we clearly need to cut
down in general and also make it so that it is harder to accidentally
burn all your available CI allowance.

Currently we always run CI whenever pushing to gitlab. This is
convenient but in retrospect it is overkill. People often push
to gitlab simply as their backup strategy and thus don't need
CI run every time.

Not all changes require all possible jobs to by run, but it is hard
to filter jobs when we're triggering them based on pushes, as the
baseline against which file changes are identified is ill-defined.


For scenario (1) we need all the jobs run to maximise quality.
This is also a case where we're most likely to have custom runners
available, so CI allowance is less of a concern. The job count still
needs to be reasonable to avoid hitting issues at times when the
merges are frequent (just before freeze).


For scenario (2) subsys maintainers, we want them to minimize
the liklihood that a pull request will fail scenario (1) and
require a respin.  Running all jobs achieces this but it is
likely overkill.

eg we have 24 cross compiler builds. If we expect most maintainers
will have either x86-64 or aarch64 hardware for their primary dev
platform, then the key benefit of cross compilers is getting coverage
of

 - 32-bit
 - big endian
 - windows

We don't need 24 jobs todo that. We could simply pick armel as the
most relevant 32-bit arch and s390x as the most relevant big endian
arch, and then the win32/64 platforms. IOW we could potentially only
run 4-6 jobs instead of 24, and still get excellant arch coverage.

Similarly for native builds we test quite alot of different distros.
I think we probably can rationalize that down to just 2 distros,
one covering oldest packages (Debian Stretch) and one covering newest
(Fedora 34), and a "build everything" config.

We have many other jobs that are testing various obscure combinations
of configure args. I'd suggest these rarely fail for most pull requests
so are overkill.

For subsystem maintainers we could potentially get down to just 10-15
jobs if we're ambitious. Leave everything else as manual trigger only.

Perhaps set all the jobs to only run on certain branch name patterns.
eg perhaps "*-next" filter is common for subsystem maintainer's pending
branches ?

For general contributors a similarly short set of jobs to subsystem
maintainers is viable. Perhaps again just let then use a "-next"
branch.

If we can enable manual triggers on any other branches that's good.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



  parent reply	other threads:[~2021-05-11 14:29 UTC|newest]

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-18 23:34 [RFC PATCH 00/15] gitlab-ci: Allow forks to use different pipelines than mainstream Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 01/15] gitlab-ci: Replace YAML anchors by extends (container_job) Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 02/15] gitlab-ci: Replace YAML anchors by extends (native_build_job) Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 03/15] gitlab-ci: Replace YAML anchors by extends (native_test_job) Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 04/15] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job) Philippe Mathieu-Daudé
2021-05-03  9:22   ` Thomas Huth
2021-05-03  9:45     ` Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 05/15] gitlab-ci: Rename acceptance_test_job -> integration_test_job Philippe Mathieu-Daudé
2021-04-19  5:19   ` Thomas Huth
2021-04-23 17:18     ` Willian Rampazzo
2021-04-18 23:34 ` [PATCH 06/15] gitlab-ci: Extract container job template to container-template.yml Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 07/15] gitlab-ci: Extract crossbuild job templates to crossbuild-template.yml Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 08/15] gitlab-ci: Extract DCO/style check jobs to checks.yml Philippe Mathieu-Daudé
2021-04-19  5:26   ` Thomas Huth
2021-04-19 13:44     ` Wainer dos Santos Moschetta
2021-04-18 23:34 ` [PATCH 09/15] gitlab-ci: Extract build stages to stages.yml Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 10/15] gitlab-ci: Extract all default build/test jobs to buildtest.yml Philippe Mathieu-Daudé
2021-04-19  5:39   ` Thomas Huth
2021-04-19 15:11   ` Alex Bennée
2021-05-11  7:19     ` Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 11/15] gitlab-ci: Extract core container jobs to container-core.yml Philippe Mathieu-Daudé
2021-04-19  5:42   ` Thomas Huth
2021-04-18 23:34 ` [PATCH 12/15] gitlab-ci: Move current job set to qemu-project.yml Philippe Mathieu-Daudé
2021-04-18 23:34 ` [RFC PATCH 13/15] gitlab-ci: Switch to dynamically generated pipelines Philippe Mathieu-Daudé
2021-04-18 23:34 ` [RFC PATCH 14/15] gitlab-ci: Allow forks to use different set of jobs Philippe Mathieu-Daudé
2021-04-19  5:48   ` Thomas Huth
2021-04-19  9:40   ` Daniel P. Berrangé
2021-04-19 10:09     ` Philippe Mathieu-Daudé
2021-04-19 10:10     ` Erik Skultety
2021-04-19 10:20       ` Thomas Huth
2021-04-19 10:36         ` Daniel P. Berrangé
2021-04-19 10:48           ` Thomas Huth
2021-04-19 10:51             ` Daniel P. Berrangé
2021-04-19 10:59               ` Thomas Huth
2021-05-11  6:48                 ` Philippe Mathieu-Daudé
2021-05-11 13:55                   ` Stefan Hajnoczi
2021-05-11 14:00                   ` Alex Bennée
2021-05-11 14:21                   ` Daniel P. Berrangé [this message]
2021-05-13 19:01                   ` Philippe Mathieu-Daudé
2021-04-19 10:47         ` Daniel P. Berrangé
2021-04-19 10:44       ` Philippe Mathieu-Daudé
2021-04-19 15:57   ` Alex Bennée
2021-04-19 16:22     ` Daniel P. Berrangé
2021-04-19 16:46       ` Philippe Mathieu-Daudé
2021-04-19 16:58         ` Daniel P. Berrangé
2021-04-19 16:39     ` Philippe Mathieu-Daudé
2021-04-18 23:34 ` [NOTFORMERGE PATCH 15/15] gitlab-ci: Use my own set of jobs for CI pipeline Philippe Mathieu-Daudé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YJqS+FqVttnrw6zb@redhat.com \
    --to=berrange@redhat.com \
    --cc=alex.bennee@linaro.org \
    --cc=crosa@redhat.com \
    --cc=eskultet@redhat.com \
    --cc=f4bug@amsat.org \
    --cc=mrezanin@redhat.com \
    --cc=peter.maydell@linaro.org \
    --cc=qemu-devel@nongnu.org \
    --cc=stefanha@redhat.com \
    --cc=thuth@redhat.com \
    --cc=wainersm@redhat.com \
    --cc=willianr@redhat.com \
    --subject='Re: [RFC PATCH 14/15] gitlab-ci: Allow forks to use different set of jobs' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
on how to clone and mirror all data and code used for this inbox