* [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs
@ 2020-11-04 22:45 Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 1/9] gitlab-ci: Replace YAML anchors by extends (cross_system_build_job) Philippe Mathieu-Daudé
` (9 more replies)
0 siblings, 10 replies; 11+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-04 22:45 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Thomas Huth, Daniel P . Berrange, John Snow,
Cornelia Huck, Wainer dos Santos Moschetta,
Philippe Mathieu-Daudé,
Alex Bennée
Hi,
2 months ago GitLab added time limit to their free CI offer [1].
This series provide developers with the possibility to not run
all jobs. By default all jobs are started, but we can disable
a subset of them.
I think this should be the other way around (enable features one
wants to test, with a default to the "all" keyword), but I didn't
want to disrupt the current workflow.
I'm not sure supporting both ("SKIP_BUILD=3Dall INCLUDE_BUILD=3Dsystem")
options is easy to implement, or to maintain (unlikely).
In the next iteration I'll add the possibility to use a project-
wide environment variable [2] to set the default set of enabled /
disabled features a fork is interested in. User will still be
able to overload using git-push on the command line. A friendly
way to use this feature is with git aliases [3]:
$ git config alias.pushci_system \
'push -o ci.variable=3D"SKIP_BUILD=3Duser"'
$ git config alias.pushci_debian \
'push -o ci.variable=3D"SKIP_BUILD=3Dcentos,fedora,ubuntu"'
Then you can run the jobs based on Debian images (only) using:
$ git pushci_debian gitlab_repo my_branch_for_debian
Or run all system-mode emulation jobs only using:
$ git pushci_system my_gitlab_repo branch_with_system_feature
Comments welcomed!
Regards,
Phil.
[1] https://about.gitlab.com/releases/2020/09/01/ci-minutes-update-free-users/
[2] https://docs.gitlab.com/ee/ci/variables/README.html#create-a-custom-varia=
ble-in-the-ui
[3] https://docs.gitlab.com/ee/user/project/push_options.html#useful-git-alia=
ses
Philippe Mathieu-Daud=C3=A9 (9):
gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)
gitlab-ci: Replace YAML anchors by extends (native_build_job)
gitlab-ci: Replace YAML anchors by extends (native_test_job)
gitlab-ci: Replace YAML anchors by extends (acceptance_test_job)
gitlab-ci: Rename acceptance_test_job -> integration_test_job
gitlab-ci: Extract common job definition as 'cross_common_job'
gitlab-ci: Extract common job definition as 'native_common_job'
gitlab-ci: Add rules to skip building cross-jobs
gitlab-ci: Add rules to skip building/testing native jobs
.gitlab-ci.d/crossbuilds.yml | 57 +++++++++++--------
.gitlab-ci.yml | 104 +++++++++++++++++++++--------------
2 files changed, 98 insertions(+), 63 deletions(-)
--=20
2.26.2
^ permalink raw reply [flat|nested] 11+ messages in thread
* [RFC PATCH 1/9] gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)
2020-11-04 22:45 [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
@ 2020-11-04 22:45 ` Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 2/9] gitlab-ci: Replace YAML anchors by extends (native_build_job) Philippe Mathieu-Daudé
` (8 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-04 22:45 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Thomas Huth, Daniel P . Berrange, John Snow,
Cornelia Huck, Wainer dos Santos Moschetta,
Philippe Mathieu-Daudé,
Alex Bennée
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.d/crossbuilds.yml | 40 ++++++++++++++++++------------------
1 file changed, 20 insertions(+), 20 deletions(-)
diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 03ebfabb3fa..099949aaef3 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -1,5 +1,5 @@
-.cross_system_build_job_template: &cross_system_build_job_definition
+.cross_system_build_job:
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
timeout: 80m
@@ -13,7 +13,7 @@
xtensa-softmmu"
- make -j$(expr $(nproc) + 1) all check-build
-.cross_user_build_job_template: &cross_user_build_job_definition
+.cross_user_build_job:
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
script:
@@ -24,91 +24,91 @@
- make -j$(expr $(nproc) + 1) all check-build
cross-armel-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-armel-cross
cross-armel-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-armel-cross
cross-armhf-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-armhf-cross
cross-armhf-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-armhf-cross
cross-arm64-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-arm64-cross
cross-arm64-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-arm64-cross
cross-mips-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-mips-cross
cross-mips-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-mips-cross
cross-mipsel-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-mipsel-cross
cross-mipsel-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-mipsel-cross
cross-mips64el-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-mips64el-cross
cross-mips64el-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-mips64el-cross
cross-ppc64el-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-ppc64el-cross
cross-ppc64el-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-ppc64el-cross
cross-s390x-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-s390x-cross
cross-s390x-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-s390x-cross
cross-win32-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: fedora-win32-cross
cross-win64-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: fedora-win64-cross
--
2.26.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [RFC PATCH 2/9] gitlab-ci: Replace YAML anchors by extends (native_build_job)
2020-11-04 22:45 [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 1/9] gitlab-ci: Replace YAML anchors by extends (cross_system_build_job) Philippe Mathieu-Daudé
@ 2020-11-04 22:45 ` Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 3/9] gitlab-ci: Replace YAML anchors by extends (native_test_job) Philippe Mathieu-Daudé
` (7 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-04 22:45 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Thomas Huth, Daniel P . Berrange, John Snow,
Cornelia Huck, Wainer dos Santos Moschetta,
Philippe Mathieu-Daudé,
Alex Bennée
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 32 ++++++++++++++++----------------
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 3b15ae5c302..ff3a8bd58cf 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -19,7 +19,7 @@ include:
- local: '/.gitlab-ci.d/containers.yml'
- local: '/.gitlab-ci.d/crossbuilds.yml'
-.native_build_job_template: &native_build_job_definition
+.native_build_job:
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
before_script:
@@ -74,7 +74,7 @@ include:
- du -chs ${CI_PROJECT_DIR}/avocado-cache
build-system-ubuntu:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: ubuntu2004
TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu
@@ -105,7 +105,7 @@ acceptance-system-ubuntu:
<<: *acceptance_definition
build-system-debian:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: debian-amd64
TARGETS: arm-softmmu avr-softmmu i386-softmmu mipsel-softmmu
@@ -136,7 +136,7 @@ acceptance-system-debian:
<<: *acceptance_definition
build-system-fedora:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: fedora
CONFIGURE_ARGS: --disable-gcrypt --enable-nettle
@@ -168,7 +168,7 @@ acceptance-system-fedora:
<<: *acceptance_definition
build-system-centos:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: centos8
CONFIGURE_ARGS: --disable-nettle --enable-gcrypt
@@ -200,7 +200,7 @@ acceptance-system-centos:
<<: *acceptance_definition
build-disabled:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: fedora
CONFIGURE_ARGS: --disable-attr --disable-avx2 --disable-bochs
@@ -225,7 +225,7 @@ build-disabled:
MAKE_CHECK_ARGS: check-qtest SPEED=slow
build-tcg-disabled:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: centos8
script:
@@ -245,7 +245,7 @@ build-tcg-disabled:
260 261 262 263 264 270 272 273 277 279
build-user:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: debian-all-test-cross
CONFIGURE_ARGS: --disable-tools --disable-system
@@ -255,7 +255,7 @@ build-user:
# we skip sparc64-linux-user until it has been fixed somewhat
# we skip cris-linux-user as it doesn't use the common run loop
build-user-plugins:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: debian-all-test-cross
CONFIGURE_ARGS: --disable-tools --disable-system --enable-plugins --enable-debug-tcg --target-list-exclude=sparc64-linux-user,cris-linux-user
@@ -263,7 +263,7 @@ build-user-plugins:
timeout: 1h 30m
build-clang:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: fedora
CONFIGURE_ARGS: --cc=clang --cxx=clang++
@@ -273,7 +273,7 @@ build-clang:
# These targets are on the way out
build-deprecated:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: debian-all-test-cross
CONFIGURE_ARGS: --disable-docs --disable-tools
@@ -298,7 +298,7 @@ check-deprecated:
allow_failure: true
build-oss-fuzz:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: fedora
script:
@@ -316,7 +316,7 @@ build-oss-fuzz:
- cd build-oss-fuzz && make check-qtest-i386 check-unit
build-tci:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: fedora
script:
@@ -341,7 +341,7 @@ build-tci:
# These jobs test old gcrypt and nettle from RHEL7
# which had some API differences.
build-crypto-old-nettle:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: centos7
TARGETS: x86_64-softmmu x86_64-linux-user
@@ -362,7 +362,7 @@ check-crypto-old-nettle:
build-crypto-old-gcrypt:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: centos7
TARGETS: x86_64-softmmu x86_64-linux-user
@@ -383,7 +383,7 @@ check-crypto-old-gcrypt:
build-crypto-only-gnutls:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: centos7
TARGETS: x86_64-softmmu x86_64-linux-user
--
2.26.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [RFC PATCH 3/9] gitlab-ci: Replace YAML anchors by extends (native_test_job)
2020-11-04 22:45 [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 1/9] gitlab-ci: Replace YAML anchors by extends (cross_system_build_job) Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 2/9] gitlab-ci: Replace YAML anchors by extends (native_build_job) Philippe Mathieu-Daudé
@ 2020-11-04 22:45 ` Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 4/9] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job) Philippe Mathieu-Daudé
` (6 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-04 22:45 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Thomas Huth, Daniel P . Berrange, John Snow,
Cornelia Huck, Wainer dos Santos Moschetta,
Philippe Mathieu-Daudé,
Alex Bennée
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 26 +++++++++++++-------------
1 file changed, 13 insertions(+), 13 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index ff3a8bd58cf..83beefa2c85 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -40,7 +40,7 @@ include:
make -j"$JOBS" $MAKE_CHECK_ARGS ;
fi
-.native_test_job_template: &native_test_job_definition
+.native_test_job:
stage: test
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
script:
@@ -86,7 +86,7 @@ build-system-ubuntu:
- build
check-system-ubuntu:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-ubuntu
artifacts: true
@@ -95,7 +95,7 @@ check-system-ubuntu:
MAKE_CHECK_ARGS: check
acceptance-system-ubuntu:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-ubuntu
artifacts: true
@@ -117,7 +117,7 @@ build-system-debian:
- build
check-system-debian:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-debian
artifacts: true
@@ -126,7 +126,7 @@ check-system-debian:
MAKE_CHECK_ARGS: check
acceptance-system-debian:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-debian
artifacts: true
@@ -149,7 +149,7 @@ build-system-fedora:
- build
check-system-fedora:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-fedora
artifacts: true
@@ -158,7 +158,7 @@ check-system-fedora:
MAKE_CHECK_ARGS: check
acceptance-system-fedora:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-fedora
artifacts: true
@@ -181,7 +181,7 @@ build-system-centos:
- build
check-system-centos:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-centos
artifacts: true
@@ -190,7 +190,7 @@ check-system-centos:
MAKE_CHECK_ARGS: check
acceptance-system-centos:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-centos
artifacts: true
@@ -288,7 +288,7 @@ build-deprecated:
# We split the check-tcg step as test failures are expected but we still
# want to catch the build breaking.
check-deprecated:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-deprecated
artifacts: true
@@ -352,7 +352,7 @@ build-crypto-old-nettle:
- build
check-crypto-old-nettle:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-crypto-old-nettle
artifacts: true
@@ -373,7 +373,7 @@ build-crypto-old-gcrypt:
- build
check-crypto-old-gcrypt:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-crypto-old-gcrypt
artifacts: true
@@ -394,7 +394,7 @@ build-crypto-only-gnutls:
- build
check-crypto-only-gnutls:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-crypto-only-gnutls
artifacts: true
--
2.26.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [RFC PATCH 4/9] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job)
2020-11-04 22:45 [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
` (2 preceding siblings ...)
2020-11-04 22:45 ` [RFC PATCH 3/9] gitlab-ci: Replace YAML anchors by extends (native_test_job) Philippe Mathieu-Daudé
@ 2020-11-04 22:45 ` Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 5/9] gitlab-ci: Rename acceptance_test_job -> integration_test_job Philippe Mathieu-Daudé
` (5 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-04 22:45 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Thomas Huth, Daniel P . Berrange, John Snow,
Cornelia Huck, Wainer dos Santos Moschetta,
Philippe Mathieu-Daudé,
Alex Bennée
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 15 ++++++---------
1 file changed, 6 insertions(+), 9 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 83beefa2c85..5c64e477c9d 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -48,7 +48,8 @@ include:
- find . -type f -exec touch {} +
- make $MAKE_CHECK_ARGS
-.acceptance_template: &acceptance_definition
+.acceptance_test_job:
+ extends: .native_test_job
cache:
key: "${CI_JOB_NAME}-cache"
paths:
@@ -95,14 +96,13 @@ check-system-ubuntu:
MAKE_CHECK_ARGS: check
acceptance-system-ubuntu:
- extends: .native_test_job
+ extends: .acceptance_test_job
needs:
- job: build-system-ubuntu
artifacts: true
variables:
IMAGE: ubuntu2004
MAKE_CHECK_ARGS: check-acceptance
- <<: *acceptance_definition
build-system-debian:
extends: .native_build_job
@@ -126,14 +126,13 @@ check-system-debian:
MAKE_CHECK_ARGS: check
acceptance-system-debian:
- extends: .native_test_job
+ extends: .acceptance_test_job
needs:
- job: build-system-debian
artifacts: true
variables:
IMAGE: debian-amd64
MAKE_CHECK_ARGS: check-acceptance
- <<: *acceptance_definition
build-system-fedora:
extends: .native_build_job
@@ -158,14 +157,13 @@ check-system-fedora:
MAKE_CHECK_ARGS: check
acceptance-system-fedora:
- extends: .native_test_job
+ extends: .acceptance_test_job
needs:
- job: build-system-fedora
artifacts: true
variables:
IMAGE: fedora
MAKE_CHECK_ARGS: check-acceptance
- <<: *acceptance_definition
build-system-centos:
extends: .native_build_job
@@ -190,14 +188,13 @@ check-system-centos:
MAKE_CHECK_ARGS: check
acceptance-system-centos:
- extends: .native_test_job
+ extends: .acceptance_test_job
needs:
- job: build-system-centos
artifacts: true
variables:
IMAGE: centos8
MAKE_CHECK_ARGS: check-acceptance
- <<: *acceptance_definition
build-disabled:
extends: .native_build_job
--
2.26.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [RFC PATCH 5/9] gitlab-ci: Rename acceptance_test_job -> integration_test_job
2020-11-04 22:45 [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
` (3 preceding siblings ...)
2020-11-04 22:45 ` [RFC PATCH 4/9] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job) Philippe Mathieu-Daudé
@ 2020-11-04 22:45 ` Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 6/9] gitlab-ci: Extract common job definition as 'cross_common_job' Philippe Mathieu-Daudé
` (4 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-04 22:45 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Thomas Huth, Daniel P . Berrange, John Snow,
Cornelia Huck, Wainer dos Santos Moschetta,
Philippe Mathieu-Daudé,
Alex Bennée
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5c64e477c9d..0b77a90dd73 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -48,7 +48,7 @@ include:
- find . -type f -exec touch {} +
- make $MAKE_CHECK_ARGS
-.acceptance_test_job:
+.integration_test_job:
extends: .native_test_job
cache:
key: "${CI_JOB_NAME}-cache"
@@ -95,8 +95,8 @@ check-system-ubuntu:
IMAGE: ubuntu2004
MAKE_CHECK_ARGS: check
-acceptance-system-ubuntu:
- extends: .acceptance_test_job
+integration-system-ubuntu:
+ extends: .integration_test_job
needs:
- job: build-system-ubuntu
artifacts: true
@@ -125,8 +125,8 @@ check-system-debian:
IMAGE: debian-amd64
MAKE_CHECK_ARGS: check
-acceptance-system-debian:
- extends: .acceptance_test_job
+integration-system-debian:
+ extends: .integration_test_job
needs:
- job: build-system-debian
artifacts: true
@@ -156,8 +156,8 @@ check-system-fedora:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-acceptance-system-fedora:
- extends: .acceptance_test_job
+integration-system-fedora:
+ extends: .integration_test_job
needs:
- job: build-system-fedora
artifacts: true
@@ -187,8 +187,8 @@ check-system-centos:
IMAGE: centos8
MAKE_CHECK_ARGS: check
-acceptance-system-centos:
- extends: .acceptance_test_job
+integration-system-centos:
+ extends: .integration_test_job
needs:
- job: build-system-centos
artifacts: true
--
2.26.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [RFC PATCH 6/9] gitlab-ci: Extract common job definition as 'cross_common_job'
2020-11-04 22:45 [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
` (4 preceding siblings ...)
2020-11-04 22:45 ` [RFC PATCH 5/9] gitlab-ci: Rename acceptance_test_job -> integration_test_job Philippe Mathieu-Daudé
@ 2020-11-04 22:45 ` Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 7/9] gitlab-ci: Extract common job definition as 'native_common_job' Philippe Mathieu-Daudé
` (3 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-04 22:45 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Thomas Huth, Daniel P . Berrange, John Snow,
Cornelia Huck, Wainer dos Santos Moschetta,
Philippe Mathieu-Daudé,
Alex Bennée
Extract the common definitions shared by '.cross_system_build_job'
and '.cross_user_build_job' to '.cross_common_job'.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.d/crossbuilds.yml | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 099949aaef3..701550f028c 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -1,7 +1,9 @@
-
-.cross_system_build_job:
+.cross_common_job:
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+
+.cross_system_build_job:
+ extends: .cross_common_job
timeout: 80m
script:
- mkdir build
@@ -14,8 +16,7 @@
- make -j$(expr $(nproc) + 1) all check-build
.cross_user_build_job:
- stage: build
- image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ extends: .cross_common_job
script:
- mkdir build
- cd build
--
2.26.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [RFC PATCH 7/9] gitlab-ci: Extract common job definition as 'native_common_job'
2020-11-04 22:45 [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
` (5 preceding siblings ...)
2020-11-04 22:45 ` [RFC PATCH 6/9] gitlab-ci: Extract common job definition as 'cross_common_job' Philippe Mathieu-Daudé
@ 2020-11-04 22:45 ` Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 8/9] gitlab-ci: Add rules to skip building cross-jobs Philippe Mathieu-Daudé
` (2 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-04 22:45 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Thomas Huth, Daniel P . Berrange, John Snow,
Cornelia Huck, Wainer dos Santos Moschetta,
Philippe Mathieu-Daudé,
Alex Bennée
Extract the common definitions shared by '.native_build_job'
and '.native_test_job' to '.native_common_job'.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 0b77a90dd73..961070d2cbe 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -19,9 +19,12 @@ include:
- local: '/.gitlab-ci.d/containers.yml'
- local: '/.gitlab-ci.d/crossbuilds.yml'
-.native_build_job:
- stage: build
+.native_common_job:
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+
+.native_build_job:
+ extends: .native_common_job
+ stage: build
before_script:
- JOBS=$(expr $(nproc) + 1)
- sed -i s,git.qemu.org/git,gitlab.com/qemu-project, .gitmodules
@@ -41,8 +44,8 @@ include:
fi
.native_test_job:
+ extends: .native_common_job
stage: test
- image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
script:
- cd build
- find . -type f -exec touch {} +
--
2.26.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [RFC PATCH 8/9] gitlab-ci: Add rules to skip building cross-jobs
2020-11-04 22:45 [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
` (6 preceding siblings ...)
2020-11-04 22:45 ` [RFC PATCH 7/9] gitlab-ci: Extract common job definition as 'native_common_job' Philippe Mathieu-Daudé
@ 2020-11-04 22:45 ` Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 9/9] gitlab-ci: Add rules to skip building/testing native jobs Philippe Mathieu-Daudé
2020-11-05 7:22 ` [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
9 siblings, 0 replies; 11+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-04 22:45 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Thomas Huth, Daniel P . Berrange, John Snow,
Cornelia Huck, Wainer dos Santos Moschetta,
Philippe Mathieu-Daudé,
Alex Bennée
Add rules to skip some crossbuild jobs.
The following tags are available to skip CI jobs:
- cross (skip all cross-jobs)
- system (skip all cross-system jobs)
- user (skip all cross-user jobs)
Developers can combine tags in the SKIP_BUILD variable when
pushing a branch (or tag) to repositories. Examples:
$ git push -o ci.variable="SKIP_BUILD=user" myrepo mybranch
$ git push -o ci.variable="SKIP_BUILD=user,system" myrepo mybranch
References:
- https://docs.gitlab.com/ee/ci/yaml/#rulesif
- https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.d/crossbuilds.yml | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 701550f028c..08f27649eb2 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -1,6 +1,16 @@
.cross_common_job:
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ rules:
+ # If the if statement is true, the job is excluded from a pipeline.
+ - if: $SKIP_BUILD =~ /cross/
+ when: never
+ - if: $CI_JOB_NAME =~ /system/ && $SKIP_BUILD =~ /system/
+ when: never
+ - if: $CI_JOB_NAME =~ /user/ && $SKIP_BUILD =~ /user/
+ when: never
+ # In all other cases, the job is added to the pipeline.
+ - when: on_success
.cross_system_build_job:
extends: .cross_common_job
--
2.26.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [RFC PATCH 9/9] gitlab-ci: Add rules to skip building/testing native jobs
2020-11-04 22:45 [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
` (7 preceding siblings ...)
2020-11-04 22:45 ` [RFC PATCH 8/9] gitlab-ci: Add rules to skip building cross-jobs Philippe Mathieu-Daudé
@ 2020-11-04 22:45 ` Philippe Mathieu-Daudé
2020-11-05 7:22 ` [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
9 siblings, 0 replies; 11+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-04 22:45 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Thomas Huth, Daniel P . Berrange, John Snow,
Cornelia Huck, Wainer dos Santos Moschetta,
Philippe Mathieu-Daudé,
Alex Bennée
Add rules to skip various build/test jobs.
The following tags are available to skip CI jobs:
- user (user-mode jobs)
- system (system-mode jobs)
- centos (jobs based on CentOS distribution image)
- debian (... Debian)
- fedora (... Fedora)
- ubuntu (... Ubuntu)
- crypto (jobs testing the crypto feature)
- tci (jobs testing TCI feature)
- fuzz (fuzzer job)
- integration (integration tests)
Developers can combine tags in the SKIP_BUILD variable when
pushing a branch (or tag) to repositories. Examples:
$ git push -o ci.variable="SKIP_BUILD=user" myrepo mybranch
$ git push -o ci.variable="SKIP_BUILD=user,debian,crypto,fuzz" myrepo mybranch
References:
- https://docs.gitlab.com/ee/ci/yaml/#rulesif
- https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 961070d2cbe..432daccf590 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -21,6 +21,30 @@ include:
.native_common_job:
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ rules:
+ # If the if statement is true, the job is excluded from a pipeline.
+ - if: ($TARGETS =~ /softmmu/ || $CONFIGURE_ARGS =~ /disable-user/) && $SKIP_BUILD =~ /system/
+ when: never
+ - if: ($TARGETS =~ /user/ || $CONFIGURE_ARGS =~ /disable-system/) && $SKIP_BUILD =~ /user/
+ when: never
+ - if: $IMAGE =~ /^centos/ && $SKIP_BUILD =~ /centos/
+ when: never
+ - if: $IMAGE =~ /^debian/ && $SKIP_BUILD =~ /debian/
+ when: never
+ - if: $IMAGE =~ /^fedora/ && $SKIP_BUILD =~ /fedora/
+ when: never
+ - if: $IMAGE =~ /^ubuntu/ && $SKIP_BUILD =~ /ubuntu/
+ when: never
+ - if: $CI_JOB_NAME =~ /crypto/ && $SKIP_BUILD =~ /crypto/
+ when: never
+ - if: $CI_JOB_NAME =~ /tci/ && $SKIP_BUILD =~ /tci/
+ when: never
+ - if: $CI_JOB_NAME =~ /fuzz/ && $SKIP_BUILD =~ /fuzz/
+ when: never
+ - if: $CI_JOB_NAME =~ /^acceptance/ && $SKIP_BUILD =~ /integration/
+ when: never
+ # In all other cases, the job is added to the pipeline.
+ - when: on_success
.native_build_job:
extends: .native_common_job
--
2.26.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs
2020-11-04 22:45 [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
` (8 preceding siblings ...)
2020-11-04 22:45 ` [RFC PATCH 9/9] gitlab-ci: Add rules to skip building/testing native jobs Philippe Mathieu-Daudé
@ 2020-11-05 7:22 ` Philippe Mathieu-Daudé
9 siblings, 0 replies; 11+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-05 7:22 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Thomas Huth, Daniel P . Berrange, John Snow,
Cornelia Huck, Wainer dos Santos Moschetta, Alex Bennée
On 11/4/20 11:45 PM, Philippe Mathieu-Daudé wrote:
> Hi,
>
> 2 months ago GitLab added time limit to their free CI offer [1].
> This series provide developers with the possibility to not run
> all jobs. By default all jobs are started, but we can disable
> a subset of them.
>
> I think this should be the other way around (enable features one
> wants to test, with a default to the "all" keyword), but I didn't
> want to disrupt the current workflow.
I was a bit too focused on getting the YAML right to not realize
it is simpler to do:
QEMU_JOBS=all # default, overwritable
QEMU_JOBS=system,fuzz # only build those jobs
# can be set in fork UI config (see [2])
# can be set with git-push option (see below)
> I'm not sure supporting both ("SKIP_BUILD=3Dall INCLUDE_BUILD=3Dsystem")
> options is easy to implement, or to maintain (unlikely).
>
> In the next iteration I'll add the possibility to use a project-
> wide environment variable [2] to set the default set of enabled /
> disabled features a fork is interested in. User will still be
> able to overload using git-push on the command line. A friendly
> way to use this feature is with git aliases [3]:
>
> $ git config alias.pushci_system \
> 'push -o ci.variable=3D"SKIP_BUILD=3Duser"'
> $ git config alias.pushci_debian \
> 'push -o ci.variable=3D"SKIP_BUILD=3Dcentos,fedora,ubuntu"'
>
> Then you can run the jobs based on Debian images (only) using:
>
> $ git pushci_debian gitlab_repo my_branch_for_debian
>
> Or run all system-mode emulation jobs only using:
>
> $ git pushci_system my_gitlab_repo branch_with_system_feature
>
> Comments welcomed!
>
> Regards,
>
> Phil.
>
> [1] https://about.gitlab.com/releases/2020/09/01/ci-minutes-update-free-users/
> [2] https://docs.gitlab.com/ee/ci/variables/README.html#create-a-custom-varia=
> ble-in-the-ui
> [3] https://docs.gitlab.com/ee/user/project/push_options.html#useful-git-alia=
> ses
>
> Philippe Mathieu-Daud=C3=A9 (9):
> gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)
> gitlab-ci: Replace YAML anchors by extends (native_build_job)
> gitlab-ci: Replace YAML anchors by extends (native_test_job)
> gitlab-ci: Replace YAML anchors by extends (acceptance_test_job)
> gitlab-ci: Rename acceptance_test_job -> integration_test_job
> gitlab-ci: Extract common job definition as 'cross_common_job'
> gitlab-ci: Extract common job definition as 'native_common_job'
> gitlab-ci: Add rules to skip building cross-jobs
> gitlab-ci: Add rules to skip building/testing native jobs
>
> .gitlab-ci.d/crossbuilds.yml | 57 +++++++++++--------
> .gitlab-ci.yml | 104 +++++++++++++++++++++--------------
> 2 files changed, 98 insertions(+), 63 deletions(-)
>
> --=20
> 2.26.2
>
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2020-11-05 7:23 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-04 22:45 [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 1/9] gitlab-ci: Replace YAML anchors by extends (cross_system_build_job) Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 2/9] gitlab-ci: Replace YAML anchors by extends (native_build_job) Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 3/9] gitlab-ci: Replace YAML anchors by extends (native_test_job) Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 4/9] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job) Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 5/9] gitlab-ci: Rename acceptance_test_job -> integration_test_job Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 6/9] gitlab-ci: Extract common job definition as 'cross_common_job' Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 7/9] gitlab-ci: Extract common job definition as 'native_common_job' Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 8/9] gitlab-ci: Add rules to skip building cross-jobs Philippe Mathieu-Daudé
2020-11-04 22:45 ` [RFC PATCH 9/9] gitlab-ci: Add rules to skip building/testing native jobs Philippe Mathieu-Daudé
2020-11-05 7:22 ` [RFC PATCH 0/9] gitlab-ci: Allow forks to skip some build jobs Philippe Mathieu-Daudé
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).