qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH 00/15] gitlab-ci: Allow forks to use different pipelines than mainstream
@ 2021-04-18 23:34 Philippe Mathieu-Daudé
  2021-04-18 23:34 ` [PATCH 01/15] gitlab-ci: Replace YAML anchors by extends (container_job) Philippe Mathieu-Daudé
                   ` (14 more replies)
  0 siblings, 15 replies; 47+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-04-18 23:34 UTC (permalink / raw)
  To: qemu-devel
  Cc: Thomas Huth, Daniel P . Berrangé,
	Erik Skultety, Philippe Mathieu-Daudé,
	Wainer dos Santos Moschetta, Willian Rampazzo, Cleber Rosa,
	Miroslav Rezanina, Alex Bennée

TL;DR: Patch 13 is an experiment to dynamically generate pipelines.

Hi,

This series is an intent to allow git forks to use different set
of jobs than the ones used by the mainstream repository.

Currently, a fork gets the mainstream CI pipeline YAML configuration
and runs its set of jobs by default. Mainstream tests many cases
which might not be always useful to forks. Beside, one mainstream
pipeline might be a waste of ressources when forks provide their
own runners to their gitlab namespace, or use private instances.

This is not a new / unique problem to our community, and Gitlab
already figured it out. They describe this case as Upstream (our
project mainline) / Downstream (a fork).

The pipeline is divided in 2 steps. First the "upstream" part, which
does some evaluation, and depending on the result, triggers the
"downstream" part. The "upstream" is committed into mainline, and
dynamically generate the "downstream" pipeline. This allows forks
to easily rebase with mainstream, avoiding merge conflicts.

This might not be adapted to the QEMU use case, but I wanted to give
it a try.

Gitlab recommends to use job templates. To be able to re-use /
include templates, they can not use YAML anchors. All current jobs
has first to be converted to use the 'extends' keyworkd.
This is done by patches 1-5 (already reviewed).

Patches 6-12 extract various generic jobs (which can be reused by
forks) as template.

Patches 13-14 follow Gitlab guideline to generate dynamic pipelines

Patch 15 is an example of how a fork could use mainstream templates
to maintain his own set of jobs.

The PRO is forks can contribute to templates.

Regards,

Phil.

Philippe Mathieu-Daudé (15):
  gitlab-ci: Replace YAML anchors by extends (container_job)
  gitlab-ci: Replace YAML anchors by extends (native_build_job)
  gitlab-ci: Replace YAML anchors by extends (native_test_job)
  gitlab-ci: Replace YAML anchors by extends (acceptance_test_job)
  gitlab-ci: Rename acceptance_test_job -> integration_test_job
  gitlab-ci: Extract container job template to container-template.yml
  gitlab-ci: Extract crossbuild job templates to crossbuild-template.yml
  gitlab-ci: Extract DCO/style check jobs to checks.yml
  gitlab-ci: Extract build stages to stages.yml
  gitlab-ci: Extract all default build/test jobs to buildtest.yml
  gitlab-ci: Extract core container jobs to container-core.yml
  gitlab-ci: Move current job set to qemu-project.yml
  gitlab-ci: Switch to dynamically generated pipelines
  gitlab-ci: Allow forks to use different set of jobs
  gitlab-ci: Use my own set of jobs for CI pipeline

 .gitlab-ci.d/buildtest-template.yml  |  80 +++
 .gitlab-ci.d/buildtest.yml           | 744 +++++++++++++++++++++++
 .gitlab-ci.d/checks.yml              |  24 +
 .gitlab-ci.d/container-core.yml      |  17 +
 .gitlab-ci.d/container-template.yml  |  22 +
 .gitlab-ci.d/containers.yml          | 108 ++--
 .gitlab-ci.d/crossbuild-template.yml |  41 ++
 .gitlab-ci.d/crossbuilds.yml         |  43 +-
 .gitlab-ci.d/philmd.yml              |  33 ++
 .gitlab-ci.d/qemu-project.yml        |   8 +
 .gitlab-ci.d/stages.yml              |   8 +
 .gitlab-ci.yml                       | 845 +--------------------------
 12 files changed, 1033 insertions(+), 940 deletions(-)
 create mode 100644 .gitlab-ci.d/buildtest-template.yml
 create mode 100644 .gitlab-ci.d/buildtest.yml
 create mode 100644 .gitlab-ci.d/checks.yml
 create mode 100644 .gitlab-ci.d/container-core.yml
 create mode 100644 .gitlab-ci.d/container-template.yml
 create mode 100644 .gitlab-ci.d/crossbuild-template.yml
 create mode 100644 .gitlab-ci.d/philmd.yml
 create mode 100644 .gitlab-ci.d/qemu-project.yml
 create mode 100644 .gitlab-ci.d/stages.yml

-- 
2.26.3



^ permalink raw reply	[flat|nested] 47+ messages in thread

end of thread, other threads:[~2021-05-13 19:02 UTC | newest]

Thread overview: 47+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-18 23:34 [RFC PATCH 00/15] gitlab-ci: Allow forks to use different pipelines than mainstream Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 01/15] gitlab-ci: Replace YAML anchors by extends (container_job) Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 02/15] gitlab-ci: Replace YAML anchors by extends (native_build_job) Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 03/15] gitlab-ci: Replace YAML anchors by extends (native_test_job) Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 04/15] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job) Philippe Mathieu-Daudé
2021-05-03  9:22   ` Thomas Huth
2021-05-03  9:45     ` Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 05/15] gitlab-ci: Rename acceptance_test_job -> integration_test_job Philippe Mathieu-Daudé
2021-04-19  5:19   ` Thomas Huth
2021-04-23 17:18     ` Willian Rampazzo
2021-04-18 23:34 ` [PATCH 06/15] gitlab-ci: Extract container job template to container-template.yml Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 07/15] gitlab-ci: Extract crossbuild job templates to crossbuild-template.yml Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 08/15] gitlab-ci: Extract DCO/style check jobs to checks.yml Philippe Mathieu-Daudé
2021-04-19  5:26   ` Thomas Huth
2021-04-19 13:44     ` Wainer dos Santos Moschetta
2021-04-18 23:34 ` [PATCH 09/15] gitlab-ci: Extract build stages to stages.yml Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 10/15] gitlab-ci: Extract all default build/test jobs to buildtest.yml Philippe Mathieu-Daudé
2021-04-19  5:39   ` Thomas Huth
2021-04-19 15:11   ` Alex Bennée
2021-05-11  7:19     ` Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 11/15] gitlab-ci: Extract core container jobs to container-core.yml Philippe Mathieu-Daudé
2021-04-19  5:42   ` Thomas Huth
2021-04-18 23:34 ` [PATCH 12/15] gitlab-ci: Move current job set to qemu-project.yml Philippe Mathieu-Daudé
2021-04-18 23:34 ` [RFC PATCH 13/15] gitlab-ci: Switch to dynamically generated pipelines Philippe Mathieu-Daudé
2021-04-18 23:34 ` [RFC PATCH 14/15] gitlab-ci: Allow forks to use different set of jobs Philippe Mathieu-Daudé
2021-04-19  5:48   ` Thomas Huth
2021-04-19  9:40   ` Daniel P. Berrangé
2021-04-19 10:09     ` Philippe Mathieu-Daudé
2021-04-19 10:10     ` Erik Skultety
2021-04-19 10:20       ` Thomas Huth
2021-04-19 10:36         ` Daniel P. Berrangé
2021-04-19 10:48           ` Thomas Huth
2021-04-19 10:51             ` Daniel P. Berrangé
2021-04-19 10:59               ` Thomas Huth
2021-05-11  6:48                 ` Philippe Mathieu-Daudé
2021-05-11 13:55                   ` Stefan Hajnoczi
2021-05-11 14:00                   ` Alex Bennée
2021-05-11 14:21                   ` Daniel P. Berrangé
2021-05-13 19:01                   ` Philippe Mathieu-Daudé
2021-04-19 10:47         ` Daniel P. Berrangé
2021-04-19 10:44       ` Philippe Mathieu-Daudé
2021-04-19 15:57   ` Alex Bennée
2021-04-19 16:22     ` Daniel P. Berrangé
2021-04-19 16:46       ` Philippe Mathieu-Daudé
2021-04-19 16:58         ` Daniel P. Berrangé
2021-04-19 16:39     ` Philippe Mathieu-Daudé
2021-04-18 23:34 ` [NOTFORMERGE PATCH 15/15] gitlab-ci: Use my own set of jobs for CI pipeline Philippe Mathieu-Daudé

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).