All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Philippe Mathieu-Daudé" <f4bug@amsat.org>
To: qemu-devel@nongnu.org
Cc: "Thomas Huth" <thuth@redhat.com>,
	"Daniel P . Berrangé" <berrange@redhat.com>,
	"Erik Skultety" <eskultet@redhat.com>,
	"Philippe Mathieu-Daudé" <f4bug@amsat.org>,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	"Willian Rampazzo" <willianr@redhat.com>,
	"Cleber Rosa" <crosa@redhat.com>,
	"Miroslav Rezanina" <mrezanin@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>
Subject: [RFC PATCH 00/15] gitlab-ci: Allow forks to use different pipelines than mainstream
Date: Mon, 19 Apr 2021 01:34:33 +0200	[thread overview]
Message-ID: <20210418233448.1267991-1-f4bug@amsat.org> (raw)

TL;DR: Patch 13 is an experiment to dynamically generate pipelines.\r
\r
Hi,\r
\r
This series is an intent to allow git forks to use different set\r
of jobs than the ones used by the mainstream repository.\r
\r
Currently, a fork gets the mainstream CI pipeline YAML configuration\r
and runs its set of jobs by default. Mainstream tests many cases\r
which might not be always useful to forks. Beside, one mainstream\r
pipeline might be a waste of ressources when forks provide their\r
own runners to their gitlab namespace, or use private instances.\r
\r
This is not a new / unique problem to our community, and Gitlab\r
already figured it out. They describe this case as Upstream (our\r
project mainline) / Downstream (a fork).\r
\r
The pipeline is divided in 2 steps. First the "upstream" part, which\r
does some evaluation, and depending on the result, triggers the\r
"downstream" part. The "upstream" is committed into mainline, and\r
dynamically generate the "downstream" pipeline. This allows forks\r
to easily rebase with mainstream, avoiding merge conflicts.\r
\r
This might not be adapted to the QEMU use case, but I wanted to give\r
it a try.\r
\r
Gitlab recommends to use job templates. To be able to re-use /\r
include templates, they can not use YAML anchors. All current jobs\r
has first to be converted to use the 'extends' keyworkd.\r
This is done by patches 1-5 (already reviewed).\r
\r
Patches 6-12 extract various generic jobs (which can be reused by\r
forks) as template.\r
\r
Patches 13-14 follow Gitlab guideline to generate dynamic pipelines\r
\r
Patch 15 is an example of how a fork could use mainstream templates\r
to maintain his own set of jobs.\r
\r
The PRO is forks can contribute to templates.\r
\r
Regards,\r
\r
Phil.\r
\r
Philippe Mathieu-Daudé (15):\r
  gitlab-ci: Replace YAML anchors by extends (container_job)\r
  gitlab-ci: Replace YAML anchors by extends (native_build_job)\r
  gitlab-ci: Replace YAML anchors by extends (native_test_job)\r
  gitlab-ci: Replace YAML anchors by extends (acceptance_test_job)\r
  gitlab-ci: Rename acceptance_test_job -> integration_test_job\r
  gitlab-ci: Extract container job template to container-template.yml\r
  gitlab-ci: Extract crossbuild job templates to crossbuild-template.yml\r
  gitlab-ci: Extract DCO/style check jobs to checks.yml\r
  gitlab-ci: Extract build stages to stages.yml\r
  gitlab-ci: Extract all default build/test jobs to buildtest.yml\r
  gitlab-ci: Extract core container jobs to container-core.yml\r
  gitlab-ci: Move current job set to qemu-project.yml\r
  gitlab-ci: Switch to dynamically generated pipelines\r
  gitlab-ci: Allow forks to use different set of jobs\r
  gitlab-ci: Use my own set of jobs for CI pipeline\r
\r
 .gitlab-ci.d/buildtest-template.yml  |  80 +++\r
 .gitlab-ci.d/buildtest.yml           | 744 +++++++++++++++++++++++\r
 .gitlab-ci.d/checks.yml              |  24 +\r
 .gitlab-ci.d/container-core.yml      |  17 +\r
 .gitlab-ci.d/container-template.yml  |  22 +\r
 .gitlab-ci.d/containers.yml          | 108 ++--\r
 .gitlab-ci.d/crossbuild-template.yml |  41 ++\r
 .gitlab-ci.d/crossbuilds.yml         |  43 +-\r
 .gitlab-ci.d/philmd.yml              |  33 ++\r
 .gitlab-ci.d/qemu-project.yml        |   8 +\r
 .gitlab-ci.d/stages.yml              |   8 +\r
 .gitlab-ci.yml                       | 845 +--------------------------\r
 12 files changed, 1033 insertions(+), 940 deletions(-)\r
 create mode 100644 .gitlab-ci.d/buildtest-template.yml\r
 create mode 100644 .gitlab-ci.d/buildtest.yml\r
 create mode 100644 .gitlab-ci.d/checks.yml\r
 create mode 100644 .gitlab-ci.d/container-core.yml\r
 create mode 100644 .gitlab-ci.d/container-template.yml\r
 create mode 100644 .gitlab-ci.d/crossbuild-template.yml\r
 create mode 100644 .gitlab-ci.d/philmd.yml\r
 create mode 100644 .gitlab-ci.d/qemu-project.yml\r
 create mode 100644 .gitlab-ci.d/stages.yml\r
\r
-- \r
2.26.3\r
\r


             reply	other threads:[~2021-04-18 23:36 UTC|newest]

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-18 23:34 Philippe Mathieu-Daudé [this message]
2021-04-18 23:34 ` [PATCH 01/15] gitlab-ci: Replace YAML anchors by extends (container_job) Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 02/15] gitlab-ci: Replace YAML anchors by extends (native_build_job) Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 03/15] gitlab-ci: Replace YAML anchors by extends (native_test_job) Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 04/15] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job) Philippe Mathieu-Daudé
2021-05-03  9:22   ` Thomas Huth
2021-05-03  9:45     ` Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 05/15] gitlab-ci: Rename acceptance_test_job -> integration_test_job Philippe Mathieu-Daudé
2021-04-19  5:19   ` Thomas Huth
2021-04-23 17:18     ` Willian Rampazzo
2021-04-18 23:34 ` [PATCH 06/15] gitlab-ci: Extract container job template to container-template.yml Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 07/15] gitlab-ci: Extract crossbuild job templates to crossbuild-template.yml Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 08/15] gitlab-ci: Extract DCO/style check jobs to checks.yml Philippe Mathieu-Daudé
2021-04-19  5:26   ` Thomas Huth
2021-04-19 13:44     ` Wainer dos Santos Moschetta
2021-04-18 23:34 ` [PATCH 09/15] gitlab-ci: Extract build stages to stages.yml Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 10/15] gitlab-ci: Extract all default build/test jobs to buildtest.yml Philippe Mathieu-Daudé
2021-04-19  5:39   ` Thomas Huth
2021-04-19 15:11   ` Alex Bennée
2021-05-11  7:19     ` Philippe Mathieu-Daudé
2021-04-18 23:34 ` [PATCH 11/15] gitlab-ci: Extract core container jobs to container-core.yml Philippe Mathieu-Daudé
2021-04-19  5:42   ` Thomas Huth
2021-04-18 23:34 ` [PATCH 12/15] gitlab-ci: Move current job set to qemu-project.yml Philippe Mathieu-Daudé
2021-04-18 23:34 ` [RFC PATCH 13/15] gitlab-ci: Switch to dynamically generated pipelines Philippe Mathieu-Daudé
2021-04-18 23:34 ` [RFC PATCH 14/15] gitlab-ci: Allow forks to use different set of jobs Philippe Mathieu-Daudé
2021-04-19  5:48   ` Thomas Huth
2021-04-19  9:40   ` Daniel P. Berrangé
2021-04-19 10:09     ` Philippe Mathieu-Daudé
2021-04-19 10:10     ` Erik Skultety
2021-04-19 10:20       ` Thomas Huth
2021-04-19 10:36         ` Daniel P. Berrangé
2021-04-19 10:48           ` Thomas Huth
2021-04-19 10:51             ` Daniel P. Berrangé
2021-04-19 10:59               ` Thomas Huth
2021-05-11  6:48                 ` Philippe Mathieu-Daudé
2021-05-11 13:55                   ` Stefan Hajnoczi
2021-05-11 14:00                   ` Alex Bennée
2021-05-11 14:21                   ` Daniel P. Berrangé
2021-05-13 19:01                   ` Philippe Mathieu-Daudé
2021-04-19 10:47         ` Daniel P. Berrangé
2021-04-19 10:44       ` Philippe Mathieu-Daudé
2021-04-19 15:57   ` Alex Bennée
2021-04-19 16:22     ` Daniel P. Berrangé
2021-04-19 16:46       ` Philippe Mathieu-Daudé
2021-04-19 16:58         ` Daniel P. Berrangé
2021-04-19 16:39     ` Philippe Mathieu-Daudé
2021-04-18 23:34 ` [NOTFORMERGE PATCH 15/15] gitlab-ci: Use my own set of jobs for CI pipeline Philippe Mathieu-Daudé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210418233448.1267991-1-f4bug@amsat.org \
    --to=f4bug@amsat.org \
    --cc=alex.bennee@linaro.org \
    --cc=berrange@redhat.com \
    --cc=crosa@redhat.com \
    --cc=eskultet@redhat.com \
    --cc=mrezanin@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=thuth@redhat.com \
    --cc=wainersm@redhat.com \
    --cc=willianr@redhat.com \
    --subject='Re: [RFC PATCH 00/15] gitlab-ci: Allow forks to use different pipelines than mainstream' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.