* [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs
@ 2020-11-08 22:19 Philippe Mathieu-Daudé
2020-11-08 22:19 ` [PATCH v3 01/11] gitlab-ci: Drop generic cache rule Philippe Mathieu-Daudé
` (10 more replies)
0 siblings, 11 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-08 22:19 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa,
Philippe Mathieu-Daudé
Hi,
2 months ago GitLab added time limit to their free CI offer [1].
This series provide developers with the possibility to not run
all jobs. By default all jobs are started, but we can restrict
by selecting a subset of them.
Since v2:
- no more RFC
- project-wide environment variable works
- do not remove the jobs from the pipeline, mark them 'manual'
so user can still run them
Since v1:
- switch from "all but skip some" to "all or select some"
A friendly way to use this feature is with git aliases [3]:
$ git config alias.pushci_system \
'push -o ci.variable="QEMU_BUILD=system"'
$ git config alias.pushci_debian \
'push -o ci.variable="QEMU_BUILD=debian"'
Then you can run the jobs based on Debian images (only) using:
$ git pushci_debian gitlab_repo my_branch_for_debian
Or run all system-mode emulation jobs only using:
$ git pushci_system my_gitlab_repo branch_with_system_feature
You can also set the default QEMU_BUILD you want for your fork
adding a project-wide environment variable [2]. Your pipelines
will be restricted to this set, but you can overwrite it from
the git-push command line (using QEMU_BUILD=all).
Comments welcomed!
Regards,
Phil.
[1] https://about.gitlab.com/releases/2020/09/01/ci-minutes-update-free-users/
[2] https://docs.gitlab.com/ee/ci/variables/README.html#create-a-custom-variable-in-the-ui
[3] https://docs.gitlab.com/ee/user/project/push_options.html#useful-git-aliases
Philippe Mathieu-Daudé (11):
gitlab-ci: Drop generic cache rule
gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)
gitlab-ci: Replace YAML anchors by extends (native_build_job)
gitlab-ci: Replace YAML anchors by extends (native_test_job)
gitlab-ci: Replace YAML anchors by extends (acceptance_test_job)
gitlab-ci: Rename acceptance_test_job -> integration_test_job
gitlab-ci: Extract common job definition as 'cross_common_job'
gitlab-ci: Extract common job definition as 'native_common_job'
gitlab-ci: Add rules to select cross-jobs to build
gitlab-ci: Add rules to select building/testing native jobs
gitlab-ci: Move artifacts expiry rule to common 'native_build_job'
.gitlab-ci.d/crossbuilds.yml | 82 +++++++++++++------
.gitlab-ci.yml | 153 ++++++++++++++++++++---------------
2 files changed, 146 insertions(+), 89 deletions(-)
--
2.26.2
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH v3 01/11] gitlab-ci: Drop generic cache rule
2020-11-08 22:19 [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs Philippe Mathieu-Daudé
@ 2020-11-08 22:19 ` Philippe Mathieu-Daudé
2020-11-09 9:46 ` Thomas Huth
2020-11-08 22:19 ` [PATCH v3 02/11] gitlab-ci: Replace YAML anchors by extends (cross_system_build_job) Philippe Mathieu-Daudé
` (9 subsequent siblings)
10 siblings, 1 reply; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-08 22:19 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa,
Philippe Mathieu-Daudé
This cache rule is meant for Avocado artifacts, but affects
all jobs. Moreover the 'acceptance_template' template already
include a more detailled rule to cache artifacts.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 6 ------
1 file changed, 6 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 3b15ae5c302..5763318d375 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -7,12 +7,6 @@ stages:
- build
- test
-# We assume GitLab has it's own caching set up for RPM/APT repositories so we
-# just take care of avocado assets here.
-cache:
- paths:
- - $HOME/avocado/data/cache
-
include:
- local: '/.gitlab-ci.d/edk2.yml'
- local: '/.gitlab-ci.d/opensbi.yml'
--
2.26.2
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v3 02/11] gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)
2020-11-08 22:19 [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs Philippe Mathieu-Daudé
2020-11-08 22:19 ` [PATCH v3 01/11] gitlab-ci: Drop generic cache rule Philippe Mathieu-Daudé
@ 2020-11-08 22:19 ` Philippe Mathieu-Daudé
2020-11-10 20:21 ` Wainer dos Santos Moschetta
2020-11-08 22:19 ` [PATCH v3 03/11] gitlab-ci: Replace YAML anchors by extends (native_build_job) Philippe Mathieu-Daudé
` (8 subsequent siblings)
10 siblings, 1 reply; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-08 22:19 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa,
Philippe Mathieu-Daudé
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.d/crossbuilds.yml | 40 ++++++++++++++++++------------------
1 file changed, 20 insertions(+), 20 deletions(-)
diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 03ebfabb3fa..099949aaef3 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -1,5 +1,5 @@
-.cross_system_build_job_template: &cross_system_build_job_definition
+.cross_system_build_job:
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
timeout: 80m
@@ -13,7 +13,7 @@
xtensa-softmmu"
- make -j$(expr $(nproc) + 1) all check-build
-.cross_user_build_job_template: &cross_user_build_job_definition
+.cross_user_build_job:
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
script:
@@ -24,91 +24,91 @@
- make -j$(expr $(nproc) + 1) all check-build
cross-armel-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-armel-cross
cross-armel-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-armel-cross
cross-armhf-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-armhf-cross
cross-armhf-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-armhf-cross
cross-arm64-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-arm64-cross
cross-arm64-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-arm64-cross
cross-mips-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-mips-cross
cross-mips-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-mips-cross
cross-mipsel-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-mipsel-cross
cross-mipsel-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-mipsel-cross
cross-mips64el-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-mips64el-cross
cross-mips64el-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-mips64el-cross
cross-ppc64el-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-ppc64el-cross
cross-ppc64el-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-ppc64el-cross
cross-s390x-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: debian-s390x-cross
cross-s390x-user:
- <<: *cross_user_build_job_definition
+ extends: .cross_user_build_job
variables:
IMAGE: debian-s390x-cross
cross-win32-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: fedora-win32-cross
cross-win64-system:
- <<: *cross_system_build_job_definition
+ extends: .cross_system_build_job
variables:
IMAGE: fedora-win64-cross
--
2.26.2
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v3 03/11] gitlab-ci: Replace YAML anchors by extends (native_build_job)
2020-11-08 22:19 [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs Philippe Mathieu-Daudé
2020-11-08 22:19 ` [PATCH v3 01/11] gitlab-ci: Drop generic cache rule Philippe Mathieu-Daudé
2020-11-08 22:19 ` [PATCH v3 02/11] gitlab-ci: Replace YAML anchors by extends (cross_system_build_job) Philippe Mathieu-Daudé
@ 2020-11-08 22:19 ` Philippe Mathieu-Daudé
2020-11-10 20:23 ` Wainer dos Santos Moschetta
2020-11-08 22:19 ` [PATCH v3 04/11] gitlab-ci: Replace YAML anchors by extends (native_test_job) Philippe Mathieu-Daudé
` (7 subsequent siblings)
10 siblings, 1 reply; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-08 22:19 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa,
Philippe Mathieu-Daudé
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 32 ++++++++++++++++----------------
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5763318d375..a96e7dd23e5 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -13,7 +13,7 @@ include:
- local: '/.gitlab-ci.d/containers.yml'
- local: '/.gitlab-ci.d/crossbuilds.yml'
-.native_build_job_template: &native_build_job_definition
+.native_build_job:
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
before_script:
@@ -68,7 +68,7 @@ include:
- du -chs ${CI_PROJECT_DIR}/avocado-cache
build-system-ubuntu:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: ubuntu2004
TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu
@@ -99,7 +99,7 @@ acceptance-system-ubuntu:
<<: *acceptance_definition
build-system-debian:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: debian-amd64
TARGETS: arm-softmmu avr-softmmu i386-softmmu mipsel-softmmu
@@ -130,7 +130,7 @@ acceptance-system-debian:
<<: *acceptance_definition
build-system-fedora:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: fedora
CONFIGURE_ARGS: --disable-gcrypt --enable-nettle
@@ -162,7 +162,7 @@ acceptance-system-fedora:
<<: *acceptance_definition
build-system-centos:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: centos8
CONFIGURE_ARGS: --disable-nettle --enable-gcrypt
@@ -194,7 +194,7 @@ acceptance-system-centos:
<<: *acceptance_definition
build-disabled:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: fedora
CONFIGURE_ARGS: --disable-attr --disable-avx2 --disable-bochs
@@ -219,7 +219,7 @@ build-disabled:
MAKE_CHECK_ARGS: check-qtest SPEED=slow
build-tcg-disabled:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: centos8
script:
@@ -239,7 +239,7 @@ build-tcg-disabled:
260 261 262 263 264 270 272 273 277 279
build-user:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: debian-all-test-cross
CONFIGURE_ARGS: --disable-tools --disable-system
@@ -249,7 +249,7 @@ build-user:
# we skip sparc64-linux-user until it has been fixed somewhat
# we skip cris-linux-user as it doesn't use the common run loop
build-user-plugins:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: debian-all-test-cross
CONFIGURE_ARGS: --disable-tools --disable-system --enable-plugins --enable-debug-tcg --target-list-exclude=sparc64-linux-user,cris-linux-user
@@ -257,7 +257,7 @@ build-user-plugins:
timeout: 1h 30m
build-clang:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: fedora
CONFIGURE_ARGS: --cc=clang --cxx=clang++
@@ -267,7 +267,7 @@ build-clang:
# These targets are on the way out
build-deprecated:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: debian-all-test-cross
CONFIGURE_ARGS: --disable-docs --disable-tools
@@ -292,7 +292,7 @@ check-deprecated:
allow_failure: true
build-oss-fuzz:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: fedora
script:
@@ -310,7 +310,7 @@ build-oss-fuzz:
- cd build-oss-fuzz && make check-qtest-i386 check-unit
build-tci:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: fedora
script:
@@ -335,7 +335,7 @@ build-tci:
# These jobs test old gcrypt and nettle from RHEL7
# which had some API differences.
build-crypto-old-nettle:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: centos7
TARGETS: x86_64-softmmu x86_64-linux-user
@@ -356,7 +356,7 @@ check-crypto-old-nettle:
build-crypto-old-gcrypt:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: centos7
TARGETS: x86_64-softmmu x86_64-linux-user
@@ -377,7 +377,7 @@ check-crypto-old-gcrypt:
build-crypto-only-gnutls:
- <<: *native_build_job_definition
+ extends: .native_build_job
variables:
IMAGE: centos7
TARGETS: x86_64-softmmu x86_64-linux-user
--
2.26.2
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v3 04/11] gitlab-ci: Replace YAML anchors by extends (native_test_job)
2020-11-08 22:19 [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs Philippe Mathieu-Daudé
` (2 preceding siblings ...)
2020-11-08 22:19 ` [PATCH v3 03/11] gitlab-ci: Replace YAML anchors by extends (native_build_job) Philippe Mathieu-Daudé
@ 2020-11-08 22:19 ` Philippe Mathieu-Daudé
2020-11-10 20:24 ` Wainer dos Santos Moschetta
2020-11-08 22:19 ` [PATCH v3 05/11] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job) Philippe Mathieu-Daudé
` (6 subsequent siblings)
10 siblings, 1 reply; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-08 22:19 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa,
Philippe Mathieu-Daudé
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 26 +++++++++++++-------------
1 file changed, 13 insertions(+), 13 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index a96e7dd23e5..e11f80f6d65 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -34,7 +34,7 @@ include:
make -j"$JOBS" $MAKE_CHECK_ARGS ;
fi
-.native_test_job_template: &native_test_job_definition
+.native_test_job:
stage: test
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
script:
@@ -80,7 +80,7 @@ build-system-ubuntu:
- build
check-system-ubuntu:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-ubuntu
artifacts: true
@@ -89,7 +89,7 @@ check-system-ubuntu:
MAKE_CHECK_ARGS: check
acceptance-system-ubuntu:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-ubuntu
artifacts: true
@@ -111,7 +111,7 @@ build-system-debian:
- build
check-system-debian:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-debian
artifacts: true
@@ -120,7 +120,7 @@ check-system-debian:
MAKE_CHECK_ARGS: check
acceptance-system-debian:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-debian
artifacts: true
@@ -143,7 +143,7 @@ build-system-fedora:
- build
check-system-fedora:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-fedora
artifacts: true
@@ -152,7 +152,7 @@ check-system-fedora:
MAKE_CHECK_ARGS: check
acceptance-system-fedora:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-fedora
artifacts: true
@@ -175,7 +175,7 @@ build-system-centos:
- build
check-system-centos:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-centos
artifacts: true
@@ -184,7 +184,7 @@ check-system-centos:
MAKE_CHECK_ARGS: check
acceptance-system-centos:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-system-centos
artifacts: true
@@ -282,7 +282,7 @@ build-deprecated:
# We split the check-tcg step as test failures are expected but we still
# want to catch the build breaking.
check-deprecated:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-deprecated
artifacts: true
@@ -346,7 +346,7 @@ build-crypto-old-nettle:
- build
check-crypto-old-nettle:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-crypto-old-nettle
artifacts: true
@@ -367,7 +367,7 @@ build-crypto-old-gcrypt:
- build
check-crypto-old-gcrypt:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-crypto-old-gcrypt
artifacts: true
@@ -388,7 +388,7 @@ build-crypto-only-gnutls:
- build
check-crypto-only-gnutls:
- <<: *native_test_job_definition
+ extends: .native_test_job
needs:
- job: build-crypto-only-gnutls
artifacts: true
--
2.26.2
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v3 05/11] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job)
2020-11-08 22:19 [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs Philippe Mathieu-Daudé
` (3 preceding siblings ...)
2020-11-08 22:19 ` [PATCH v3 04/11] gitlab-ci: Replace YAML anchors by extends (native_test_job) Philippe Mathieu-Daudé
@ 2020-11-08 22:19 ` Philippe Mathieu-Daudé
2020-11-10 20:35 ` Wainer dos Santos Moschetta
2020-11-08 22:19 ` [PATCH v3 06/11] gitlab-ci: Rename acceptance_test_job -> integration_test_job Philippe Mathieu-Daudé
` (5 subsequent siblings)
10 siblings, 1 reply; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-08 22:19 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa,
Philippe Mathieu-Daudé
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 15 ++++++---------
1 file changed, 6 insertions(+), 9 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index e11f80f6d65..0ef814764a0 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -42,7 +42,8 @@ include:
- find . -type f -exec touch {} +
- make $MAKE_CHECK_ARGS
-.acceptance_template: &acceptance_definition
+.acceptance_test_job:
+ extends: .native_test_job
cache:
key: "${CI_JOB_NAME}-cache"
paths:
@@ -89,14 +90,13 @@ check-system-ubuntu:
MAKE_CHECK_ARGS: check
acceptance-system-ubuntu:
- extends: .native_test_job
+ extends: .acceptance_test_job
needs:
- job: build-system-ubuntu
artifacts: true
variables:
IMAGE: ubuntu2004
MAKE_CHECK_ARGS: check-acceptance
- <<: *acceptance_definition
build-system-debian:
extends: .native_build_job
@@ -120,14 +120,13 @@ check-system-debian:
MAKE_CHECK_ARGS: check
acceptance-system-debian:
- extends: .native_test_job
+ extends: .acceptance_test_job
needs:
- job: build-system-debian
artifacts: true
variables:
IMAGE: debian-amd64
MAKE_CHECK_ARGS: check-acceptance
- <<: *acceptance_definition
build-system-fedora:
extends: .native_build_job
@@ -152,14 +151,13 @@ check-system-fedora:
MAKE_CHECK_ARGS: check
acceptance-system-fedora:
- extends: .native_test_job
+ extends: .acceptance_test_job
needs:
- job: build-system-fedora
artifacts: true
variables:
IMAGE: fedora
MAKE_CHECK_ARGS: check-acceptance
- <<: *acceptance_definition
build-system-centos:
extends: .native_build_job
@@ -184,14 +182,13 @@ check-system-centos:
MAKE_CHECK_ARGS: check
acceptance-system-centos:
- extends: .native_test_job
+ extends: .acceptance_test_job
needs:
- job: build-system-centos
artifacts: true
variables:
IMAGE: centos8
MAKE_CHECK_ARGS: check-acceptance
- <<: *acceptance_definition
build-disabled:
extends: .native_build_job
--
2.26.2
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v3 06/11] gitlab-ci: Rename acceptance_test_job -> integration_test_job
2020-11-08 22:19 [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs Philippe Mathieu-Daudé
` (4 preceding siblings ...)
2020-11-08 22:19 ` [PATCH v3 05/11] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job) Philippe Mathieu-Daudé
@ 2020-11-08 22:19 ` Philippe Mathieu-Daudé
2020-11-10 20:42 ` Wainer dos Santos Moschetta
2020-11-23 15:36 ` Willian Rampazzo
2020-11-08 22:19 ` [PATCH v3 07/11] gitlab-ci: Extract common job definition as 'cross_common_job' Philippe Mathieu-Daudé
` (4 subsequent siblings)
10 siblings, 2 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-08 22:19 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa,
Philippe Mathieu-Daudé
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 0ef814764a0..d4526323169 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -42,7 +42,7 @@ include:
- find . -type f -exec touch {} +
- make $MAKE_CHECK_ARGS
-.acceptance_test_job:
+.integration_test_job:
extends: .native_test_job
cache:
key: "${CI_JOB_NAME}-cache"
@@ -89,8 +89,8 @@ check-system-ubuntu:
IMAGE: ubuntu2004
MAKE_CHECK_ARGS: check
-acceptance-system-ubuntu:
- extends: .acceptance_test_job
+integration-system-ubuntu:
+ extends: .integration_test_job
needs:
- job: build-system-ubuntu
artifacts: true
@@ -119,8 +119,8 @@ check-system-debian:
IMAGE: debian-amd64
MAKE_CHECK_ARGS: check
-acceptance-system-debian:
- extends: .acceptance_test_job
+integration-system-debian:
+ extends: .integration_test_job
needs:
- job: build-system-debian
artifacts: true
@@ -150,8 +150,8 @@ check-system-fedora:
IMAGE: fedora
MAKE_CHECK_ARGS: check
-acceptance-system-fedora:
- extends: .acceptance_test_job
+integration-system-fedora:
+ extends: .integration_test_job
needs:
- job: build-system-fedora
artifacts: true
@@ -181,8 +181,8 @@ check-system-centos:
IMAGE: centos8
MAKE_CHECK_ARGS: check
-acceptance-system-centos:
- extends: .acceptance_test_job
+integration-system-centos:
+ extends: .integration_test_job
needs:
- job: build-system-centos
artifacts: true
--
2.26.2
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v3 07/11] gitlab-ci: Extract common job definition as 'cross_common_job'
2020-11-08 22:19 [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs Philippe Mathieu-Daudé
` (5 preceding siblings ...)
2020-11-08 22:19 ` [PATCH v3 06/11] gitlab-ci: Rename acceptance_test_job -> integration_test_job Philippe Mathieu-Daudé
@ 2020-11-08 22:19 ` Philippe Mathieu-Daudé
2020-11-09 9:52 ` Thomas Huth
2020-11-10 20:43 ` Wainer dos Santos Moschetta
2020-11-08 22:19 ` [PATCH v3 08/11] gitlab-ci: Extract common job definition as 'native_common_job' Philippe Mathieu-Daudé
` (3 subsequent siblings)
10 siblings, 2 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-08 22:19 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa,
Philippe Mathieu-Daudé
Extract the common definitions shared by '.cross_system_build_job'
and '.cross_user_build_job' to '.cross_common_job'.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.d/crossbuilds.yml | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 099949aaef3..701550f028c 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -1,7 +1,9 @@
-
-.cross_system_build_job:
+.cross_common_job:
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+
+.cross_system_build_job:
+ extends: .cross_common_job
timeout: 80m
script:
- mkdir build
@@ -14,8 +16,7 @@
- make -j$(expr $(nproc) + 1) all check-build
.cross_user_build_job:
- stage: build
- image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ extends: .cross_common_job
script:
- mkdir build
- cd build
--
2.26.2
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v3 08/11] gitlab-ci: Extract common job definition as 'native_common_job'
2020-11-08 22:19 [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs Philippe Mathieu-Daudé
` (6 preceding siblings ...)
2020-11-08 22:19 ` [PATCH v3 07/11] gitlab-ci: Extract common job definition as 'cross_common_job' Philippe Mathieu-Daudé
@ 2020-11-08 22:19 ` Philippe Mathieu-Daudé
2020-11-10 20:54 ` Wainer dos Santos Moschetta
2020-11-08 22:19 ` [PATCH v3 09/11] gitlab-ci: Add rules to select cross-jobs to build Philippe Mathieu-Daudé
` (2 subsequent siblings)
10 siblings, 1 reply; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-08 22:19 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa,
Philippe Mathieu-Daudé
Extract the common definitions shared by '.native_build_job'
and '.native_test_job' to '.native_common_job'.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index d4526323169..f708573884e 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -13,9 +13,12 @@ include:
- local: '/.gitlab-ci.d/containers.yml'
- local: '/.gitlab-ci.d/crossbuilds.yml'
-.native_build_job:
- stage: build
+.native_common_job:
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+
+.native_build_job:
+ extends: .native_common_job
+ stage: build
before_script:
- JOBS=$(expr $(nproc) + 1)
- sed -i s,git.qemu.org/git,gitlab.com/qemu-project, .gitmodules
@@ -35,8 +38,8 @@ include:
fi
.native_test_job:
+ extends: .native_common_job
stage: test
- image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
script:
- cd build
- find . -type f -exec touch {} +
--
2.26.2
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v3 09/11] gitlab-ci: Add rules to select cross-jobs to build
2020-11-08 22:19 [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs Philippe Mathieu-Daudé
` (7 preceding siblings ...)
2020-11-08 22:19 ` [PATCH v3 08/11] gitlab-ci: Extract common job definition as 'native_common_job' Philippe Mathieu-Daudé
@ 2020-11-08 22:19 ` Philippe Mathieu-Daudé
2020-11-09 9:57 ` Thomas Huth
2020-11-08 22:19 ` [PATCH v3 10/11] gitlab-ci: Add rules to select building/testing native jobs Philippe Mathieu-Daudé
2020-11-08 22:19 ` [PATCH v3 11/11] gitlab-ci: Move artifacts expiry rule to common 'native_build_job' Philippe Mathieu-Daudé
10 siblings, 1 reply; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-08 22:19 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa,
Philippe Mathieu-Daudé
Add rules to select some crossbuild jobs.
The following tags are available to restrict the CI jobs:
- all (select all jobs, this is default)
- cross (select all cross-jobs)
- system (select all cross-system jobs)
- user (select all cross-user jobs)
- $ARCH (select an architecture: arm/mips/ppc/sparc/...)
Developers can combine tags in the QEMU_BUILD variable when
pushing a branch (or tag) to repositories. Examples:
$ git push -o ci.variable="QEMU_BUILD=user" myrepo mybranch
$ git push -o ci.variable="QEMU_BUILD=user,system" myrepo mybranch
References:
- https://docs.gitlab.com/ee/ci/yaml/#rulesif
- https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.d/crossbuilds.yml | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 701550f028c..017bc706689 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -1,6 +1,41 @@
.cross_common_job:
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ rules:
+ # If the if statement is true, the job is added to the pipeline.
+ # We only filter for push events
+ - if: '$CI_PIPELINE_SOURCE != "push"'
+ # Build all when no variable defined, or set to "all"
+ - if: $QEMU_BUILD == null || $QEMU_BUILD =~ /^all$/
+ # Build specific job name
+ - if: $QEMU_BUILD =~ /^$CI_JOB_NAME$/
+ # Build set of jobs by feature
+ - if: $QEMU_BUILD =~ /cross/
+ - if: $QEMU_BUILD =~ /system/ && $CI_JOB_NAME =~ /system/
+ - if: $QEMU_BUILD =~ /user/ && $CI_JOB_NAME =~ /user/
+ # Build set of jobs by arch
+ - if: $QEMU_BUILD =~ /aarch64/ && ($CI_JOB_NAME =~ /aarch64/ || $IMAGE =~ /aarch64/)
+ - if: $QEMU_BUILD =~ /alpha/ && ($CI_JOB_NAME =~ /alpha/ || $IMAGE =~ /alpha/)
+ - if: $QEMU_BUILD =~ /arm/ && ($CI_JOB_NAME =~ /arm/ || $IMAGE =~ /arm/)
+ - if: $QEMU_BUILD =~ /avr/ && ($CI_JOB_NAME =~ /avr/ || $IMAGE =~ /avr/)
+ - if: $QEMU_BUILD =~ /hppa/ && ($CI_JOB_NAME =~ /hppa/ || $IMAGE =~ /hppa/)
+ - if: $QEMU_BUILD =~ /i386/ && ($CI_JOB_NAME =~ /i386/ || $IMAGE =~ /i386/)
+ - if: $QEMU_BUILD =~ /lm32/ && ($CI_JOB_NAME =~ /lm32/ || $IMAGE =~ /lm32/)
+ - if: $QEMU_BUILD =~ /m68k/ && ($CI_JOB_NAME =~ /m68k/ || $IMAGE =~ /m68k/)
+ - if: $QEMU_BUILD =~ /mips/ && ($CI_JOB_NAME =~ /mips/ || $IMAGE =~ /mips/)
+ - if: $QEMU_BUILD =~ /ppc/ && ($CI_JOB_NAME =~ /ppc/ || $IMAGE =~ /ppc/)
+ - if: $QEMU_BUILD =~ /riscv/ && ($CI_JOB_NAME =~ /riscv/ || $IMAGE =~ /riscv/)
+ - if: $QEMU_BUILD =~ /s390x/ && ($CI_JOB_NAME =~ /s390x/ || $IMAGE =~ /s390x/)
+ - if: $QEMU_BUILD =~ /sparc/ && ($CI_JOB_NAME =~ /sparc/ || $IMAGE =~ /sparc/)
+ - if: $QEMU_BUILD =~ /tricore/ && ($CI_JOB_NAME =~ /tricore/ || $IMAGE =~ /tricore/)
+ - if: $QEMU_BUILD =~ /x86/ && ($CI_JOB_NAME =~ /x86/ || $IMAGE =~ /x86/)
+ - if: $QEMU_BUILD =~ /xtensa/ && ($CI_JOB_NAME =~ /xtensa/ || $IMAGE =~ /xtensa/)
+ # In all other cases, do not not execute the job automatically. Note the
+ # job is not excluded from the pipeline, user can still start it manually.
+ # As the job is optional, we have to mark it 'allow_failure' to not block
+ # the pipeline.
+ - when: manual
+ allow_failure: true
.cross_system_build_job:
extends: .cross_common_job
--
2.26.2
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v3 10/11] gitlab-ci: Add rules to select building/testing native jobs
2020-11-08 22:19 [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs Philippe Mathieu-Daudé
` (8 preceding siblings ...)
2020-11-08 22:19 ` [PATCH v3 09/11] gitlab-ci: Add rules to select cross-jobs to build Philippe Mathieu-Daudé
@ 2020-11-08 22:19 ` Philippe Mathieu-Daudé
2020-11-08 22:19 ` [PATCH v3 11/11] gitlab-ci: Move artifacts expiry rule to common 'native_build_job' Philippe Mathieu-Daudé
10 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-08 22:19 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa,
Philippe Mathieu-Daudé
Add rules to select various build/test jobs.
The following tags are available to restrict CI jobs:
- user (user-mode jobs)
- system (system-mode jobs)
- centos (jobs based on CentOS distribution image)
- debian (... Debian)
- fedora (... Fedora)
- ubuntu (... Ubuntu)
- crypto (jobs testing the crypto feature)
- tci (jobs testing TCI feature)
- fuzz (fuzzer job)
- integration (integration tests)
- $ARCH (select an architecture: arm/mips/ppc/sparc/...)
Developers can combine tags in the QEMU_BUILD variable when
pushing a branch (or tag) to repositories. Examples:
$ git push -o ci.variable="QEMU_BUILD=user" myrepo mybranch
$ git push -o ci.variable="QEMU_BUILD=user,debian,crypto,fuzz" myrepo mybranch
References:
- https://docs.gitlab.com/ee/ci/yaml/#rulesif
- https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 43 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 43 insertions(+)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index f708573884e..dd5f9a4c505 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -15,6 +15,49 @@ include:
.native_common_job:
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+ rules:
+ # If the if statement is true, the job is added to the pipeline.
+ # We only filter for push events
+ - if: '$CI_PIPELINE_SOURCE != "push"'
+ # Build all when no variable defined, or set to "all"
+ - if: $QEMU_BUILD == null || $QEMU_BUILD =~ /^all$/
+ # Build specific job name
+ - if: $QEMU_BUILD == $CI_JOB_NAME
+ # Build jobs using particular distribution image
+ - if: $QEMU_BUILD =~ /centos/ && $IMAGE =~ /^centos/
+ - if: $QEMU_BUILD =~ /debian/ && $IMAGE =~ /^debian/
+ - if: $QEMU_BUILD =~ /fedora/ && $IMAGE =~ /^fedora/
+ - if: $QEMU_BUILD =~ /ubuntu/ && $IMAGE =~ /^ubuntu/
+ # Build set of jobs by feature
+ - if: $QEMU_BUILD =~ /system/ && ($TARGETS =~ /softmmu/ || $CONFIGURE_ARGS =~ /disable-user/)
+ - if: $QEMU_BUILD =~ /user/ && ($TARGETS =~ /user/ || $CONFIGURE_ARGS =~ /disable-system/)
+ - if: $QEMU_BUILD =~ /integration/ && ($CI_JOB_NAME =~ /^integration/ || $CI_JOB_NAME =~ /^build-system/ || $MAKE_CHECK_ARGS =~ /check-acceptance/) # integration depends on build-system
+ - if: $QEMU_BUILD =~ /crypto/ && $CI_JOB_NAME =~ /crypto/
+ - if: $QEMU_BUILD =~ /tci/ && $CI_JOB_NAME =~ /tci$/
+ - if: $QEMU_BUILD =~ /fuzz/ && $CI_JOB_NAME =~ /fuzz$/
+ # Build set of jobs by arch
+ - if: $QEMU_BUILD =~ /aarch64/ && ($CI_JOB_NAME =~ /aarch64/ || $TARGETS =~ /aarch64/)
+ - if: $QEMU_BUILD =~ /alpha/ && ($CI_JOB_NAME =~ /alpha/ || $TARGETS =~ /alpha/)
+ - if: $QEMU_BUILD =~ /arm/ && ($CI_JOB_NAME =~ /arm/ || $TARGETS =~ /arm/)
+ - if: $QEMU_BUILD =~ /avr/ && ($CI_JOB_NAME =~ /avr/ || $TARGETS =~ /avr/)
+ - if: $QEMU_BUILD =~ /hppa/ && ($CI_JOB_NAME =~ /hppa/ || $TARGETS =~ /hppa/)
+ - if: $QEMU_BUILD =~ /i386/ && ($CI_JOB_NAME =~ /i386/ || $TARGETS =~ /i386/)
+ - if: $QEMU_BUILD =~ /lm32/ && ($CI_JOB_NAME =~ /lm32/ || $TARGETS =~ /lm32/)
+ - if: $QEMU_BUILD =~ /m68k/ && ($CI_JOB_NAME =~ /m68k/ || $TARGETS =~ /m68k/)
+ - if: $QEMU_BUILD =~ /mips/ && ($CI_JOB_NAME =~ /mips/ || $IMAGE =~ /mips/)
+ - if: $QEMU_BUILD =~ /ppc/ && ($CI_JOB_NAME =~ /ppc/ || $TARGETS =~ /ppc/)
+ - if: $QEMU_BUILD =~ /riscv/ && ($CI_JOB_NAME =~ /riscv/ || $TARGETS =~ /riscv/)
+ - if: $QEMU_BUILD =~ /s390x/ && ($CI_JOB_NAME =~ /s390x/ || $TARGETS =~ /s390x/)
+ - if: $QEMU_BUILD =~ /sparc/ && ($CI_JOB_NAME =~ /sparc/ || $TARGETS =~ /sparc/)
+ - if: $QEMU_BUILD =~ /tricore/ && ($CI_JOB_NAME =~ /tricore/ || $TARGETS =~ /tricore/)
+ - if: $QEMU_BUILD =~ /x86/ && ($CI_JOB_NAME =~ /x86/ || $TARGETS =~ /x86/)
+ - if: $QEMU_BUILD =~ /xtensa/ && ($CI_JOB_NAME =~ /xtensa/ || $TARGETS =~ /xtensa/)
+ # In all other cases, do not not execute the job automatically. Note the
+ # job is not excluded from the pipeline, user can still start it manually.
+ # As the job is optional, we have to mark it 'allow_failure' to not block
+ # the pipeline.
+ - when: manual
+ allow_failure: true
.native_build_job:
extends: .native_common_job
--
2.26.2
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH v3 11/11] gitlab-ci: Move artifacts expiry rule to common 'native_build_job'
2020-11-08 22:19 [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs Philippe Mathieu-Daudé
` (9 preceding siblings ...)
2020-11-08 22:19 ` [PATCH v3 10/11] gitlab-ci: Add rules to select building/testing native jobs Philippe Mathieu-Daudé
@ 2020-11-08 22:19 ` Philippe Mathieu-Daudé
2020-11-09 9:59 ` Thomas Huth
10 siblings, 1 reply; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-08 22:19 UTC (permalink / raw)
To: qemu-devel
Cc: Thomas Huth, Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa,
Philippe Mathieu-Daudé
Build jobs include the 'native_build_job' template. Move
the 'artifacts expiry' rule there. Now all build jobs benefit
from it.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
.gitlab-ci.yml | 24 ++++--------------------
1 file changed, 4 insertions(+), 20 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index dd5f9a4c505..27a4cbc5171 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -79,6 +79,10 @@ include:
then
make -j"$JOBS" $MAKE_CHECK_ARGS ;
fi
+ artifacts:
+ expire_in: 2 days
+ paths:
+ - build
.native_test_job:
extends: .native_common_job
@@ -121,10 +125,6 @@ build-system-ubuntu:
TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu
moxie-softmmu microblazeel-softmmu mips64el-softmmu
MAKE_CHECK_ARGS: check-build
- artifacts:
- expire_in: 2 days
- paths:
- - build
check-system-ubuntu:
extends: .native_test_job
@@ -151,10 +151,6 @@ build-system-debian:
TARGETS: arm-softmmu avr-softmmu i386-softmmu mipsel-softmmu
riscv64-softmmu sh4eb-softmmu sparc-softmmu xtensaeb-softmmu
MAKE_CHECK_ARGS: check-build
- artifacts:
- expire_in: 2 days
- paths:
- - build
check-system-debian:
extends: .native_test_job
@@ -182,10 +178,6 @@ build-system-fedora:
TARGETS: tricore-softmmu microblaze-softmmu mips-softmmu
xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu sparc64-softmmu
MAKE_CHECK_ARGS: check-build
- artifacts:
- expire_in: 2 days
- paths:
- - build
check-system-fedora:
extends: .native_test_job
@@ -213,10 +205,6 @@ build-system-centos:
TARGETS: ppc64-softmmu or1k-softmmu s390x-softmmu
x86_64-softmmu rx-softmmu sh4-softmmu nios2-softmmu
MAKE_CHECK_ARGS: check-build
- artifacts:
- expire_in: 2 days
- paths:
- - build
check-system-centos:
extends: .native_test_job
@@ -317,10 +305,6 @@ build-deprecated:
MAKE_CHECK_ARGS: build-tcg
TARGETS: ppc64abi32-linux-user tilegx-linux-user lm32-softmmu
unicore32-softmmu
- artifacts:
- expire_in: 2 days
- paths:
- - build
# We split the check-tcg step as test failures are expected but we still
# want to catch the build breaking.
--
2.26.2
^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCH v3 01/11] gitlab-ci: Drop generic cache rule
2020-11-08 22:19 ` [PATCH v3 01/11] gitlab-ci: Drop generic cache rule Philippe Mathieu-Daudé
@ 2020-11-09 9:46 ` Thomas Huth
0 siblings, 0 replies; 29+ messages in thread
From: Thomas Huth @ 2020-11-09 9:46 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa
On 08/11/2020 23.19, Philippe Mathieu-Daudé wrote:
> This cache rule is meant for Avocado artifacts, but affects
> all jobs. Moreover the 'acceptance_template' template already
> include a more detailled rule to cache artifacts.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> .gitlab-ci.yml | 6 ------
> 1 file changed, 6 deletions(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 3b15ae5c302..5763318d375 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -7,12 +7,6 @@ stages:
> - build
> - test
>
> -# We assume GitLab has it's own caching set up for RPM/APT repositories so we
> -# just take care of avocado assets here.
> -cache:
> - paths:
> - - $HOME/avocado/data/cache
Right, IIRC the path in $HOME wasn't working as expected anyway, that's why
we switched to a more complicated logic in the acceptance_template.
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 07/11] gitlab-ci: Extract common job definition as 'cross_common_job'
2020-11-08 22:19 ` [PATCH v3 07/11] gitlab-ci: Extract common job definition as 'cross_common_job' Philippe Mathieu-Daudé
@ 2020-11-09 9:52 ` Thomas Huth
2020-11-10 10:23 ` Philippe Mathieu-Daudé
2020-11-10 20:43 ` Wainer dos Santos Moschetta
1 sibling, 1 reply; 29+ messages in thread
From: Thomas Huth @ 2020-11-09 9:52 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa
On 08/11/2020 23.19, Philippe Mathieu-Daudé wrote:
> Extract the common definitions shared by '.cross_system_build_job'
> and '.cross_user_build_job' to '.cross_common_job'.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> .gitlab-ci.d/crossbuilds.yml | 9 +++++----
> 1 file changed, 5 insertions(+), 4 deletions(-)
5 insertions, 4 deletions ... is it really that much better?
Thomas
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 09/11] gitlab-ci: Add rules to select cross-jobs to build
2020-11-08 22:19 ` [PATCH v3 09/11] gitlab-ci: Add rules to select cross-jobs to build Philippe Mathieu-Daudé
@ 2020-11-09 9:57 ` Thomas Huth
0 siblings, 0 replies; 29+ messages in thread
From: Thomas Huth @ 2020-11-09 9:57 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa
On 08/11/2020 23.19, Philippe Mathieu-Daudé wrote:
> Add rules to select some crossbuild jobs.
>
> The following tags are available to restrict the CI jobs:
> - all (select all jobs, this is default)
> - cross (select all cross-jobs)
> - system (select all cross-system jobs)
> - user (select all cross-user jobs)
> - $ARCH (select an architecture: arm/mips/ppc/sparc/...)
>
> Developers can combine tags in the QEMU_BUILD variable when
> pushing a branch (or tag) to repositories. Examples:
>
> $ git push -o ci.variable="QEMU_BUILD=user" myrepo mybranch
> $ git push -o ci.variable="QEMU_BUILD=user,system" myrepo mybranch
That looks interesting, but I think the changes are too big for including
them at this point in time, so I'd like to postpone this to the 6.0 cycle
(so please respin the series after the 5.2 release).
Also this certainly needs to be documented in a file in the docs/ folder -
otherwise you'll be the only one who's using this feature, I guess...
Thanks,
Thomas
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 11/11] gitlab-ci: Move artifacts expiry rule to common 'native_build_job'
2020-11-08 22:19 ` [PATCH v3 11/11] gitlab-ci: Move artifacts expiry rule to common 'native_build_job' Philippe Mathieu-Daudé
@ 2020-11-09 9:59 ` Thomas Huth
2020-11-16 13:01 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 29+ messages in thread
From: Thomas Huth @ 2020-11-09 9:59 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa
On 08/11/2020 23.19, Philippe Mathieu-Daudé wrote:
> Build jobs include the 'native_build_job' template. Move
> the 'artifacts expiry' rule there. Now all build jobs benefit
> from it.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> .gitlab-ci.yml | 24 ++++--------------------
> 1 file changed, 4 insertions(+), 20 deletions(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index dd5f9a4c505..27a4cbc5171 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -79,6 +79,10 @@ include:
> then
> make -j"$JOBS" $MAKE_CHECK_ARGS ;
> fi
> + artifacts:
> + expire_in: 2 days
> + paths:
> + - build
Should we also keep artifacts for pipelines that do not need it to pass them
from one job to the next? ... not sure ... considering that gitlab is
thinking about cutting down CI minutes etc., we should maybe be more
conservative and only keep artifacts where they are really needed?
Thomas
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 07/11] gitlab-ci: Extract common job definition as 'cross_common_job'
2020-11-09 9:52 ` Thomas Huth
@ 2020-11-10 10:23 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-10 10:23 UTC (permalink / raw)
To: Thomas Huth, qemu-devel
Cc: Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa
On 11/9/20 10:52 AM, Thomas Huth wrote:
> On 08/11/2020 23.19, Philippe Mathieu-Daudé wrote:
>> Extract the common definitions shared by '.cross_system_build_job'
>> and '.cross_user_build_job' to '.cross_common_job'.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> ---
>> .gitlab-ci.d/crossbuilds.yml | 9 +++++----
>> 1 file changed, 5 insertions(+), 4 deletions(-)
>
> 5 insertions, 4 deletions ... is it really that much better?
It becomes useful later we we add a lot into cross_common_job,
rather than duplicating in cross_[system/user]_build_job.
I understand this commit description lacks that information.
Thanks.
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 02/11] gitlab-ci: Replace YAML anchors by extends (cross_system_build_job)
2020-11-08 22:19 ` [PATCH v3 02/11] gitlab-ci: Replace YAML anchors by extends (cross_system_build_job) Philippe Mathieu-Daudé
@ 2020-11-10 20:21 ` Wainer dos Santos Moschetta
0 siblings, 0 replies; 29+ messages in thread
From: Wainer dos Santos Moschetta @ 2020-11-10 20:21 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Daniel P . Berrange, Thomas Huth, Alex Bennée, Cleber Rosa
On 11/8/20 8:19 PM, Philippe Mathieu-Daudé wrote:
> 'extends' is an alternative to using YAML anchors
> and is a little more flexible and readable. See:
> https://docs.gitlab.com/ee/ci/yaml/#extends
Good idea!
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> .gitlab-ci.d/crossbuilds.yml | 40 ++++++++++++++++++------------------
> 1 file changed, 20 insertions(+), 20 deletions(-)
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
>
> diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
> index 03ebfabb3fa..099949aaef3 100644
> --- a/.gitlab-ci.d/crossbuilds.yml
> +++ b/.gitlab-ci.d/crossbuilds.yml
> @@ -1,5 +1,5 @@
>
> -.cross_system_build_job_template: &cross_system_build_job_definition
> +.cross_system_build_job:
> stage: build
> image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
> timeout: 80m
> @@ -13,7 +13,7 @@
> xtensa-softmmu"
> - make -j$(expr $(nproc) + 1) all check-build
>
> -.cross_user_build_job_template: &cross_user_build_job_definition
> +.cross_user_build_job:
> stage: build
> image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
> script:
> @@ -24,91 +24,91 @@
> - make -j$(expr $(nproc) + 1) all check-build
>
> cross-armel-system:
> - <<: *cross_system_build_job_definition
> + extends: .cross_system_build_job
> variables:
> IMAGE: debian-armel-cross
>
> cross-armel-user:
> - <<: *cross_user_build_job_definition
> + extends: .cross_user_build_job
> variables:
> IMAGE: debian-armel-cross
>
> cross-armhf-system:
> - <<: *cross_system_build_job_definition
> + extends: .cross_system_build_job
> variables:
> IMAGE: debian-armhf-cross
>
> cross-armhf-user:
> - <<: *cross_user_build_job_definition
> + extends: .cross_user_build_job
> variables:
> IMAGE: debian-armhf-cross
>
> cross-arm64-system:
> - <<: *cross_system_build_job_definition
> + extends: .cross_system_build_job
> variables:
> IMAGE: debian-arm64-cross
>
> cross-arm64-user:
> - <<: *cross_user_build_job_definition
> + extends: .cross_user_build_job
> variables:
> IMAGE: debian-arm64-cross
>
> cross-mips-system:
> - <<: *cross_system_build_job_definition
> + extends: .cross_system_build_job
> variables:
> IMAGE: debian-mips-cross
>
> cross-mips-user:
> - <<: *cross_user_build_job_definition
> + extends: .cross_user_build_job
> variables:
> IMAGE: debian-mips-cross
>
> cross-mipsel-system:
> - <<: *cross_system_build_job_definition
> + extends: .cross_system_build_job
> variables:
> IMAGE: debian-mipsel-cross
>
> cross-mipsel-user:
> - <<: *cross_user_build_job_definition
> + extends: .cross_user_build_job
> variables:
> IMAGE: debian-mipsel-cross
>
> cross-mips64el-system:
> - <<: *cross_system_build_job_definition
> + extends: .cross_system_build_job
> variables:
> IMAGE: debian-mips64el-cross
>
> cross-mips64el-user:
> - <<: *cross_user_build_job_definition
> + extends: .cross_user_build_job
> variables:
> IMAGE: debian-mips64el-cross
>
> cross-ppc64el-system:
> - <<: *cross_system_build_job_definition
> + extends: .cross_system_build_job
> variables:
> IMAGE: debian-ppc64el-cross
>
> cross-ppc64el-user:
> - <<: *cross_user_build_job_definition
> + extends: .cross_user_build_job
> variables:
> IMAGE: debian-ppc64el-cross
>
> cross-s390x-system:
> - <<: *cross_system_build_job_definition
> + extends: .cross_system_build_job
> variables:
> IMAGE: debian-s390x-cross
>
> cross-s390x-user:
> - <<: *cross_user_build_job_definition
> + extends: .cross_user_build_job
> variables:
> IMAGE: debian-s390x-cross
>
> cross-win32-system:
> - <<: *cross_system_build_job_definition
> + extends: .cross_system_build_job
> variables:
> IMAGE: fedora-win32-cross
>
> cross-win64-system:
> - <<: *cross_system_build_job_definition
> + extends: .cross_system_build_job
> variables:
> IMAGE: fedora-win64-cross
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 03/11] gitlab-ci: Replace YAML anchors by extends (native_build_job)
2020-11-08 22:19 ` [PATCH v3 03/11] gitlab-ci: Replace YAML anchors by extends (native_build_job) Philippe Mathieu-Daudé
@ 2020-11-10 20:23 ` Wainer dos Santos Moschetta
0 siblings, 0 replies; 29+ messages in thread
From: Wainer dos Santos Moschetta @ 2020-11-10 20:23 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Alex Bennée, Thomas Huth, Daniel P . Berrange, Cleber Rosa
On 11/8/20 8:19 PM, Philippe Mathieu-Daudé wrote:
> 'extends' is an alternative to using YAML anchors
> and is a little more flexible and readable. See:
> https://docs.gitlab.com/ee/ci/yaml/#extends
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> .gitlab-ci.yml | 32 ++++++++++++++++----------------
> 1 file changed, 16 insertions(+), 16 deletions(-)
LGTM
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 5763318d375..a96e7dd23e5 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -13,7 +13,7 @@ include:
> - local: '/.gitlab-ci.d/containers.yml'
> - local: '/.gitlab-ci.d/crossbuilds.yml'
>
> -.native_build_job_template: &native_build_job_definition
> +.native_build_job:
> stage: build
> image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
> before_script:
> @@ -68,7 +68,7 @@ include:
> - du -chs ${CI_PROJECT_DIR}/avocado-cache
>
> build-system-ubuntu:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: ubuntu2004
> TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu
> @@ -99,7 +99,7 @@ acceptance-system-ubuntu:
> <<: *acceptance_definition
>
> build-system-debian:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: debian-amd64
> TARGETS: arm-softmmu avr-softmmu i386-softmmu mipsel-softmmu
> @@ -130,7 +130,7 @@ acceptance-system-debian:
> <<: *acceptance_definition
>
> build-system-fedora:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: fedora
> CONFIGURE_ARGS: --disable-gcrypt --enable-nettle
> @@ -162,7 +162,7 @@ acceptance-system-fedora:
> <<: *acceptance_definition
>
> build-system-centos:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: centos8
> CONFIGURE_ARGS: --disable-nettle --enable-gcrypt
> @@ -194,7 +194,7 @@ acceptance-system-centos:
> <<: *acceptance_definition
>
> build-disabled:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: fedora
> CONFIGURE_ARGS: --disable-attr --disable-avx2 --disable-bochs
> @@ -219,7 +219,7 @@ build-disabled:
> MAKE_CHECK_ARGS: check-qtest SPEED=slow
>
> build-tcg-disabled:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: centos8
> script:
> @@ -239,7 +239,7 @@ build-tcg-disabled:
> 260 261 262 263 264 270 272 273 277 279
>
> build-user:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: debian-all-test-cross
> CONFIGURE_ARGS: --disable-tools --disable-system
> @@ -249,7 +249,7 @@ build-user:
> # we skip sparc64-linux-user until it has been fixed somewhat
> # we skip cris-linux-user as it doesn't use the common run loop
> build-user-plugins:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: debian-all-test-cross
> CONFIGURE_ARGS: --disable-tools --disable-system --enable-plugins --enable-debug-tcg --target-list-exclude=sparc64-linux-user,cris-linux-user
> @@ -257,7 +257,7 @@ build-user-plugins:
> timeout: 1h 30m
>
> build-clang:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: fedora
> CONFIGURE_ARGS: --cc=clang --cxx=clang++
> @@ -267,7 +267,7 @@ build-clang:
>
> # These targets are on the way out
> build-deprecated:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: debian-all-test-cross
> CONFIGURE_ARGS: --disable-docs --disable-tools
> @@ -292,7 +292,7 @@ check-deprecated:
> allow_failure: true
>
> build-oss-fuzz:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: fedora
> script:
> @@ -310,7 +310,7 @@ build-oss-fuzz:
> - cd build-oss-fuzz && make check-qtest-i386 check-unit
>
> build-tci:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: fedora
> script:
> @@ -335,7 +335,7 @@ build-tci:
> # These jobs test old gcrypt and nettle from RHEL7
> # which had some API differences.
> build-crypto-old-nettle:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: centos7
> TARGETS: x86_64-softmmu x86_64-linux-user
> @@ -356,7 +356,7 @@ check-crypto-old-nettle:
>
>
> build-crypto-old-gcrypt:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: centos7
> TARGETS: x86_64-softmmu x86_64-linux-user
> @@ -377,7 +377,7 @@ check-crypto-old-gcrypt:
>
>
> build-crypto-only-gnutls:
> - <<: *native_build_job_definition
> + extends: .native_build_job
> variables:
> IMAGE: centos7
> TARGETS: x86_64-softmmu x86_64-linux-user
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 04/11] gitlab-ci: Replace YAML anchors by extends (native_test_job)
2020-11-08 22:19 ` [PATCH v3 04/11] gitlab-ci: Replace YAML anchors by extends (native_test_job) Philippe Mathieu-Daudé
@ 2020-11-10 20:24 ` Wainer dos Santos Moschetta
0 siblings, 0 replies; 29+ messages in thread
From: Wainer dos Santos Moschetta @ 2020-11-10 20:24 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Daniel P . Berrange, Thomas Huth, Alex Bennée, Cleber Rosa
On 11/8/20 8:19 PM, Philippe Mathieu-Daudé wrote:
> 'extends' is an alternative to using YAML anchors
> and is a little more flexible and readable. See:
> https://docs.gitlab.com/ee/ci/yaml/#extends
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> .gitlab-ci.yml | 26 +++++++++++++-------------
> 1 file changed, 13 insertions(+), 13 deletions(-)
LGTM
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index a96e7dd23e5..e11f80f6d65 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -34,7 +34,7 @@ include:
> make -j"$JOBS" $MAKE_CHECK_ARGS ;
> fi
>
> -.native_test_job_template: &native_test_job_definition
> +.native_test_job:
> stage: test
> image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
> script:
> @@ -80,7 +80,7 @@ build-system-ubuntu:
> - build
>
> check-system-ubuntu:
> - <<: *native_test_job_definition
> + extends: .native_test_job
> needs:
> - job: build-system-ubuntu
> artifacts: true
> @@ -89,7 +89,7 @@ check-system-ubuntu:
> MAKE_CHECK_ARGS: check
>
> acceptance-system-ubuntu:
> - <<: *native_test_job_definition
> + extends: .native_test_job
> needs:
> - job: build-system-ubuntu
> artifacts: true
> @@ -111,7 +111,7 @@ build-system-debian:
> - build
>
> check-system-debian:
> - <<: *native_test_job_definition
> + extends: .native_test_job
> needs:
> - job: build-system-debian
> artifacts: true
> @@ -120,7 +120,7 @@ check-system-debian:
> MAKE_CHECK_ARGS: check
>
> acceptance-system-debian:
> - <<: *native_test_job_definition
> + extends: .native_test_job
> needs:
> - job: build-system-debian
> artifacts: true
> @@ -143,7 +143,7 @@ build-system-fedora:
> - build
>
> check-system-fedora:
> - <<: *native_test_job_definition
> + extends: .native_test_job
> needs:
> - job: build-system-fedora
> artifacts: true
> @@ -152,7 +152,7 @@ check-system-fedora:
> MAKE_CHECK_ARGS: check
>
> acceptance-system-fedora:
> - <<: *native_test_job_definition
> + extends: .native_test_job
> needs:
> - job: build-system-fedora
> artifacts: true
> @@ -175,7 +175,7 @@ build-system-centos:
> - build
>
> check-system-centos:
> - <<: *native_test_job_definition
> + extends: .native_test_job
> needs:
> - job: build-system-centos
> artifacts: true
> @@ -184,7 +184,7 @@ check-system-centos:
> MAKE_CHECK_ARGS: check
>
> acceptance-system-centos:
> - <<: *native_test_job_definition
> + extends: .native_test_job
> needs:
> - job: build-system-centos
> artifacts: true
> @@ -282,7 +282,7 @@ build-deprecated:
> # We split the check-tcg step as test failures are expected but we still
> # want to catch the build breaking.
> check-deprecated:
> - <<: *native_test_job_definition
> + extends: .native_test_job
> needs:
> - job: build-deprecated
> artifacts: true
> @@ -346,7 +346,7 @@ build-crypto-old-nettle:
> - build
>
> check-crypto-old-nettle:
> - <<: *native_test_job_definition
> + extends: .native_test_job
> needs:
> - job: build-crypto-old-nettle
> artifacts: true
> @@ -367,7 +367,7 @@ build-crypto-old-gcrypt:
> - build
>
> check-crypto-old-gcrypt:
> - <<: *native_test_job_definition
> + extends: .native_test_job
> needs:
> - job: build-crypto-old-gcrypt
> artifacts: true
> @@ -388,7 +388,7 @@ build-crypto-only-gnutls:
> - build
>
> check-crypto-only-gnutls:
> - <<: *native_test_job_definition
> + extends: .native_test_job
> needs:
> - job: build-crypto-only-gnutls
> artifacts: true
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 05/11] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job)
2020-11-08 22:19 ` [PATCH v3 05/11] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job) Philippe Mathieu-Daudé
@ 2020-11-10 20:35 ` Wainer dos Santos Moschetta
0 siblings, 0 replies; 29+ messages in thread
From: Wainer dos Santos Moschetta @ 2020-11-10 20:35 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Daniel P . Berrange, Thomas Huth, Alex Bennée, Cleber Rosa
On 11/8/20 8:19 PM, Philippe Mathieu-Daudé wrote:
> 'extends' is an alternative to using YAML anchors
> and is a little more flexible and readable. See:
> https://docs.gitlab.com/ee/ci/yaml/#extends
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> .gitlab-ci.yml | 15 ++++++---------
> 1 file changed, 6 insertions(+), 9 deletions(-)
LGTM
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index e11f80f6d65..0ef814764a0 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -42,7 +42,8 @@ include:
> - find . -type f -exec touch {} +
> - make $MAKE_CHECK_ARGS
>
> -.acceptance_template: &acceptance_definition
> +.acceptance_test_job:
> + extends: .native_test_job
> cache:
> key: "${CI_JOB_NAME}-cache"
> paths:
> @@ -89,14 +90,13 @@ check-system-ubuntu:
> MAKE_CHECK_ARGS: check
>
> acceptance-system-ubuntu:
> - extends: .native_test_job
> + extends: .acceptance_test_job
> needs:
> - job: build-system-ubuntu
> artifacts: true
> variables:
> IMAGE: ubuntu2004
> MAKE_CHECK_ARGS: check-acceptance
> - <<: *acceptance_definition
>
> build-system-debian:
> extends: .native_build_job
> @@ -120,14 +120,13 @@ check-system-debian:
> MAKE_CHECK_ARGS: check
>
> acceptance-system-debian:
> - extends: .native_test_job
> + extends: .acceptance_test_job
> needs:
> - job: build-system-debian
> artifacts: true
> variables:
> IMAGE: debian-amd64
> MAKE_CHECK_ARGS: check-acceptance
> - <<: *acceptance_definition
>
> build-system-fedora:
> extends: .native_build_job
> @@ -152,14 +151,13 @@ check-system-fedora:
> MAKE_CHECK_ARGS: check
>
> acceptance-system-fedora:
> - extends: .native_test_job
> + extends: .acceptance_test_job
> needs:
> - job: build-system-fedora
> artifacts: true
> variables:
> IMAGE: fedora
> MAKE_CHECK_ARGS: check-acceptance
> - <<: *acceptance_definition
>
> build-system-centos:
> extends: .native_build_job
> @@ -184,14 +182,13 @@ check-system-centos:
> MAKE_CHECK_ARGS: check
>
> acceptance-system-centos:
> - extends: .native_test_job
> + extends: .acceptance_test_job
> needs:
> - job: build-system-centos
> artifacts: true
> variables:
> IMAGE: centos8
> MAKE_CHECK_ARGS: check-acceptance
> - <<: *acceptance_definition
>
> build-disabled:
> extends: .native_build_job
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 06/11] gitlab-ci: Rename acceptance_test_job -> integration_test_job
2020-11-08 22:19 ` [PATCH v3 06/11] gitlab-ci: Rename acceptance_test_job -> integration_test_job Philippe Mathieu-Daudé
@ 2020-11-10 20:42 ` Wainer dos Santos Moschetta
2020-11-12 7:02 ` Thomas Huth
2020-11-23 15:36 ` Willian Rampazzo
1 sibling, 1 reply; 29+ messages in thread
From: Wainer dos Santos Moschetta @ 2020-11-10 20:42 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Alex Bennée, Thomas Huth, Daniel P . Berrange, Cleber Rosa
Once Cleber said "acceptance" wasn't a good name for those tests.
Indeed "integration" is widely used, so okay for this renaming.
On 11/8/20 8:19 PM, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> .gitlab-ci.yml | 18 +++++++++---------
> 1 file changed, 9 insertions(+), 9 deletions(-)
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 0ef814764a0..d4526323169 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -42,7 +42,7 @@ include:
> - find . -type f -exec touch {} +
> - make $MAKE_CHECK_ARGS
>
> -.acceptance_test_job:
> +.integration_test_job:
> extends: .native_test_job
> cache:
> key: "${CI_JOB_NAME}-cache"
> @@ -89,8 +89,8 @@ check-system-ubuntu:
> IMAGE: ubuntu2004
> MAKE_CHECK_ARGS: check
>
> -acceptance-system-ubuntu:
> - extends: .acceptance_test_job
> +integration-system-ubuntu:
> + extends: .integration_test_job
> needs:
> - job: build-system-ubuntu
> artifacts: true
> @@ -119,8 +119,8 @@ check-system-debian:
> IMAGE: debian-amd64
> MAKE_CHECK_ARGS: check
>
> -acceptance-system-debian:
> - extends: .acceptance_test_job
> +integration-system-debian:
> + extends: .integration_test_job
> needs:
> - job: build-system-debian
> artifacts: true
> @@ -150,8 +150,8 @@ check-system-fedora:
> IMAGE: fedora
> MAKE_CHECK_ARGS: check
>
> -acceptance-system-fedora:
> - extends: .acceptance_test_job
> +integration-system-fedora:
> + extends: .integration_test_job
> needs:
> - job: build-system-fedora
> artifacts: true
> @@ -181,8 +181,8 @@ check-system-centos:
> IMAGE: centos8
> MAKE_CHECK_ARGS: check
>
> -acceptance-system-centos:
> - extends: .acceptance_test_job
> +integration-system-centos:
> + extends: .integration_test_job
> needs:
> - job: build-system-centos
> artifacts: true
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 07/11] gitlab-ci: Extract common job definition as 'cross_common_job'
2020-11-08 22:19 ` [PATCH v3 07/11] gitlab-ci: Extract common job definition as 'cross_common_job' Philippe Mathieu-Daudé
2020-11-09 9:52 ` Thomas Huth
@ 2020-11-10 20:43 ` Wainer dos Santos Moschetta
1 sibling, 0 replies; 29+ messages in thread
From: Wainer dos Santos Moschetta @ 2020-11-10 20:43 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Daniel P . Berrange, Thomas Huth, Alex Bennée, Cleber Rosa
On 11/8/20 8:19 PM, Philippe Mathieu-Daudé wrote:
> Extract the common definitions shared by '.cross_system_build_job'
> and '.cross_user_build_job' to '.cross_common_job'.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> .gitlab-ci.d/crossbuilds.yml | 9 +++++----
> 1 file changed, 5 insertions(+), 4 deletions(-)
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
>
> diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
> index 099949aaef3..701550f028c 100644
> --- a/.gitlab-ci.d/crossbuilds.yml
> +++ b/.gitlab-ci.d/crossbuilds.yml
> @@ -1,7 +1,9 @@
> -
> -.cross_system_build_job:
> +.cross_common_job:
> stage: build
> image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
> +
> +.cross_system_build_job:
> + extends: .cross_common_job
> timeout: 80m
> script:
> - mkdir build
> @@ -14,8 +16,7 @@
> - make -j$(expr $(nproc) + 1) all check-build
>
> .cross_user_build_job:
> - stage: build
> - image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
> + extends: .cross_common_job
> script:
> - mkdir build
> - cd build
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 08/11] gitlab-ci: Extract common job definition as 'native_common_job'
2020-11-08 22:19 ` [PATCH v3 08/11] gitlab-ci: Extract common job definition as 'native_common_job' Philippe Mathieu-Daudé
@ 2020-11-10 20:54 ` Wainer dos Santos Moschetta
0 siblings, 0 replies; 29+ messages in thread
From: Wainer dos Santos Moschetta @ 2020-11-10 20:54 UTC (permalink / raw)
To: Philippe Mathieu-Daudé, qemu-devel
Cc: Daniel P . Berrange, Thomas Huth, Alex Bennée, Cleber Rosa
On 11/8/20 8:19 PM, Philippe Mathieu-Daudé wrote:
> Extract the common definitions shared by '.native_build_job'
> and '.native_test_job' to '.native_common_job'.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> .gitlab-ci.yml | 9 ++++++---
> 1 file changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index d4526323169..f708573884e 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -13,9 +13,12 @@ include:
> - local: '/.gitlab-ci.d/containers.yml'
> - local: '/.gitlab-ci.d/crossbuilds.yml'
>
> -.native_build_job:
> - stage: build
> +.native_common_job:
> image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
Do you envision that "native_common_job" with more common properties?
Asking because it creates another indirection to just replace two "image"s.
Anyway,
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
> +
> +.native_build_job:
> + extends: .native_common_job
> + stage: build
> before_script:
> - JOBS=$(expr $(nproc) + 1)
> - sed -i s,git.qemu.org/git,gitlab.com/qemu-project, .gitmodules
> @@ -35,8 +38,8 @@ include:
> fi
>
> .native_test_job:
> + extends: .native_common_job
> stage: test
> - image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
> script:
> - cd build
> - find . -type f -exec touch {} +
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 06/11] gitlab-ci: Rename acceptance_test_job -> integration_test_job
2020-11-10 20:42 ` Wainer dos Santos Moschetta
@ 2020-11-12 7:02 ` Thomas Huth
2020-11-12 20:31 ` Wainer dos Santos Moschetta
0 siblings, 1 reply; 29+ messages in thread
From: Thomas Huth @ 2020-11-12 7:02 UTC (permalink / raw)
To: Wainer dos Santos Moschetta, Philippe Mathieu-Daudé, qemu-devel
Cc: Daniel P . Berrange, Alex Bennée, Cleber Rosa
On 10/11/2020 21.42, Wainer dos Santos Moschetta wrote:
> Once Cleber said "acceptance" wasn't a good name for those tests. Indeed
> "integration" is widely used, so okay for this renaming.
Should we maybe also rename the folder and "make check-acceptance" into
"make check-integration" ?
Thomas
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 06/11] gitlab-ci: Rename acceptance_test_job -> integration_test_job
2020-11-12 7:02 ` Thomas Huth
@ 2020-11-12 20:31 ` Wainer dos Santos Moschetta
2020-11-16 12:56 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 29+ messages in thread
From: Wainer dos Santos Moschetta @ 2020-11-12 20:31 UTC (permalink / raw)
To: Thomas Huth, Philippe Mathieu-Daudé, qemu-devel, Cleber Rosa
Cc: Alex Bennée, Daniel P . Berrange
On 11/12/20 5:02 AM, Thomas Huth wrote:
> On 10/11/2020 21.42, Wainer dos Santos Moschetta wrote:
>> Once Cleber said "acceptance" wasn't a good name for those tests. Indeed
>> "integration" is widely used, so okay for this renaming.
> Should we maybe also rename the folder and "make check-acceptance" into
> "make check-integration" ?
I'm okay with that. But I also would like to hear from Cleber.
- Wainer
>
> Thomas
>
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 06/11] gitlab-ci: Rename acceptance_test_job -> integration_test_job
2020-11-12 20:31 ` Wainer dos Santos Moschetta
@ 2020-11-16 12:56 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-16 12:56 UTC (permalink / raw)
To: Wainer dos Santos Moschetta, Thomas Huth, qemu-devel, Cleber Rosa
Cc: Alex Bennée, Daniel P . Berrange
On 11/12/20 9:31 PM, Wainer dos Santos Moschetta wrote:
>
> On 11/12/20 5:02 AM, Thomas Huth wrote:
>> On 10/11/2020 21.42, Wainer dos Santos Moschetta wrote:
>>> Once Cleber said "acceptance" wasn't a good name for those tests.
>>> Indeed
>>> "integration" is widely used, so okay for this renaming.
>> Should we maybe also rename the folder and "make check-acceptance" into
>> "make check-integration" ?
>
> I'm okay with that. But I also would like to hear from Cleber.
This is the plan indeed, but out of the scope of this (gitlab-ci
specific) series.
'check-acceptance' might then depend on 'check-integration'.
>
> - Wainer
>
>>
>> Thomas
>>
>>
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 11/11] gitlab-ci: Move artifacts expiry rule to common 'native_build_job'
2020-11-09 9:59 ` Thomas Huth
@ 2020-11-16 13:01 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-11-16 13:01 UTC (permalink / raw)
To: Thomas Huth, qemu-devel
Cc: Daniel P . Berrange, Alex Bennée,
Wainer dos Santos Moschetta, Cleber Rosa
On 11/9/20 10:59 AM, Thomas Huth wrote:
> On 08/11/2020 23.19, Philippe Mathieu-Daudé wrote:
>> Build jobs include the 'native_build_job' template. Move
>> the 'artifacts expiry' rule there. Now all build jobs benefit
>> from it.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> ---
>> .gitlab-ci.yml | 24 ++++--------------------
>> 1 file changed, 4 insertions(+), 20 deletions(-)
>>
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> index dd5f9a4c505..27a4cbc5171 100644
>> --- a/.gitlab-ci.yml
>> +++ b/.gitlab-ci.yml
>> @@ -79,6 +79,10 @@ include:
>> then
>> make -j"$JOBS" $MAKE_CHECK_ARGS ;
>> fi
>> + artifacts:
>> + expire_in: 2 days
>> + paths:
>> + - build
>
> Should we also keep artifacts for pipelines that do not need it to pass them
> from one job to the next? ... not sure ... considering that gitlab is
> thinking about cutting down CI minutes etc., we should maybe be more
> conservative and only keep artifacts where they are really needed?
As nothing is released from these jobs (except some roms/ and
documentation?) I'd not keep anything at this point. Later we
can be stricter and select the minimum we need.
(I don't think nobody ever downloaded a job artifact to test /
debug it).
>
> Thomas
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v3 06/11] gitlab-ci: Rename acceptance_test_job -> integration_test_job
2020-11-08 22:19 ` [PATCH v3 06/11] gitlab-ci: Rename acceptance_test_job -> integration_test_job Philippe Mathieu-Daudé
2020-11-10 20:42 ` Wainer dos Santos Moschetta
@ 2020-11-23 15:36 ` Willian Rampazzo
1 sibling, 0 replies; 29+ messages in thread
From: Willian Rampazzo @ 2020-11-23 15:36 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Thomas Huth, Daniel P . Berrange, qemu-devel,
Wainer dos Santos Moschetta, Cleber Rosa, Alex Bennée
On Sun, Nov 8, 2020 at 7:27 PM Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
> .gitlab-ci.yml | 18 +++++++++---------
> 1 file changed, 9 insertions(+), 9 deletions(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 0ef814764a0..d4526323169 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -42,7 +42,7 @@ include:
> - find . -type f -exec touch {} +
> - make $MAKE_CHECK_ARGS
>
> -.acceptance_test_job:
> +.integration_test_job:
> extends: .native_test_job
> cache:
> key: "${CI_JOB_NAME}-cache"
> @@ -89,8 +89,8 @@ check-system-ubuntu:
> IMAGE: ubuntu2004
> MAKE_CHECK_ARGS: check
>
> -acceptance-system-ubuntu:
> - extends: .acceptance_test_job
> +integration-system-ubuntu:
> + extends: .integration_test_job
> needs:
> - job: build-system-ubuntu
> artifacts: true
> @@ -119,8 +119,8 @@ check-system-debian:
> IMAGE: debian-amd64
> MAKE_CHECK_ARGS: check
>
> -acceptance-system-debian:
> - extends: .acceptance_test_job
> +integration-system-debian:
> + extends: .integration_test_job
> needs:
> - job: build-system-debian
> artifacts: true
> @@ -150,8 +150,8 @@ check-system-fedora:
> IMAGE: fedora
> MAKE_CHECK_ARGS: check
>
> -acceptance-system-fedora:
> - extends: .acceptance_test_job
> +integration-system-fedora:
> + extends: .integration_test_job
> needs:
> - job: build-system-fedora
> artifacts: true
> @@ -181,8 +181,8 @@ check-system-centos:
> IMAGE: centos8
> MAKE_CHECK_ARGS: check
>
> -acceptance-system-centos:
> - extends: .acceptance_test_job
> +integration-system-centos:
> + extends: .integration_test_job
> needs:
> - job: build-system-centos
> artifacts: true
> --
> 2.26.2
>
>
Makes sense, thanks!
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2020-11-23 15:38 UTC | newest]
Thread overview: 29+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-08 22:19 [PATCH v3 00/11] gitlab-ci: Allow forks to select & restrict build jobs Philippe Mathieu-Daudé
2020-11-08 22:19 ` [PATCH v3 01/11] gitlab-ci: Drop generic cache rule Philippe Mathieu-Daudé
2020-11-09 9:46 ` Thomas Huth
2020-11-08 22:19 ` [PATCH v3 02/11] gitlab-ci: Replace YAML anchors by extends (cross_system_build_job) Philippe Mathieu-Daudé
2020-11-10 20:21 ` Wainer dos Santos Moschetta
2020-11-08 22:19 ` [PATCH v3 03/11] gitlab-ci: Replace YAML anchors by extends (native_build_job) Philippe Mathieu-Daudé
2020-11-10 20:23 ` Wainer dos Santos Moschetta
2020-11-08 22:19 ` [PATCH v3 04/11] gitlab-ci: Replace YAML anchors by extends (native_test_job) Philippe Mathieu-Daudé
2020-11-10 20:24 ` Wainer dos Santos Moschetta
2020-11-08 22:19 ` [PATCH v3 05/11] gitlab-ci: Replace YAML anchors by extends (acceptance_test_job) Philippe Mathieu-Daudé
2020-11-10 20:35 ` Wainer dos Santos Moschetta
2020-11-08 22:19 ` [PATCH v3 06/11] gitlab-ci: Rename acceptance_test_job -> integration_test_job Philippe Mathieu-Daudé
2020-11-10 20:42 ` Wainer dos Santos Moschetta
2020-11-12 7:02 ` Thomas Huth
2020-11-12 20:31 ` Wainer dos Santos Moschetta
2020-11-16 12:56 ` Philippe Mathieu-Daudé
2020-11-23 15:36 ` Willian Rampazzo
2020-11-08 22:19 ` [PATCH v3 07/11] gitlab-ci: Extract common job definition as 'cross_common_job' Philippe Mathieu-Daudé
2020-11-09 9:52 ` Thomas Huth
2020-11-10 10:23 ` Philippe Mathieu-Daudé
2020-11-10 20:43 ` Wainer dos Santos Moschetta
2020-11-08 22:19 ` [PATCH v3 08/11] gitlab-ci: Extract common job definition as 'native_common_job' Philippe Mathieu-Daudé
2020-11-10 20:54 ` Wainer dos Santos Moschetta
2020-11-08 22:19 ` [PATCH v3 09/11] gitlab-ci: Add rules to select cross-jobs to build Philippe Mathieu-Daudé
2020-11-09 9:57 ` Thomas Huth
2020-11-08 22:19 ` [PATCH v3 10/11] gitlab-ci: Add rules to select building/testing native jobs Philippe Mathieu-Daudé
2020-11-08 22:19 ` [PATCH v3 11/11] gitlab-ci: Move artifacts expiry rule to common 'native_build_job' Philippe Mathieu-Daudé
2020-11-09 9:59 ` Thomas Huth
2020-11-16 13:01 ` Philippe Mathieu-Daudé
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.