[PULL v2 00/18] gitlab-ci patches for 2021-05-26

[PULL v2 00/18] gitlab-ci patches for 2021-05-26

[Philippe Mathieu-Daudé]

v2: Rebased on top of 7cf333a3726 ("gitlab-ci: use --meson=git for CFI jobs")

The following changes since commit 8385235ba99c53d1187658f2fc289b953a8090b1:

  Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging (2021-05-26 21:05:36 +0100)

are available in the Git repository at:

  https://github.com/philmd/qemu.git tags/gitlab-ci-20210527

for you to fetch changes up to 2d9e45b9cb0baa7316901238f3795a74206af457:

  gitlab: Convert check-dco/check-patch jobs to the 'rules' syntax (2021-05-27 16:39:23 +0200)

----------------------------------------------------------------
GitLab CI patches queue

- Explode .gitlab-ci.yml in reusable templates
- Add job to cross build/test TCI on i386 host
- Remove CentOS 7 linux-user build job
- Temporarily set Avocado-based jobs in manual mode
- Increase time to hold Avocado reports to 1 week

----------------------------------------------------------------

Daniel P. Berrangé (1):
  gitlab: Drop linux user build job for CentOS 7

Philippe Mathieu-Daudé (15):
  gitlab: Replace YAML anchors by extends (acceptance_test_job)
  gitlab: Extract container job template to container-template.yml
  gitlab: Extract crossbuild job templates to crossbuild-template.yml
  gitlab: Extract DCO/style check jobs to static_checks.yml
  gitlab: Extract build stages to stages.yml
  gitlab: Extract default build/test jobs templates
  gitlab: Extract core container jobs to container-core.yml
  gitlab: Extract all default build/test jobs to buildtest.yml
  gitlab: Move current job set to qemu-project.yml
  gitlab: Document how forks can use different set of jobs
  gitlab: Extract cross-container jobs to container-cross.yml
  gitlab: Keep Avocado reports during 1 week
  gitlab: Run Avocado tests manually (except mainstream CI)
  gitlab: Use $CI_DEFAULT_BRANCH instead of hardcoded 'master'
  gitlab: Convert check-dco/check-patch jobs to the 'rules' syntax

Richard Henderson (2):
  gitlab: Rename ACCEL_CONFIGURE_OPTS to EXTRA_CONFIGURE_OPTS
  gitlab: Enable cross-i386 builds of TCI

 .gitlab-ci.d/buildtest-template.yml  |  81 +++
 .gitlab-ci.d/buildtest.yml           | 730 ++++++++++++++++++++++
 .gitlab-ci.d/container-core.yml      |  17 +
 .gitlab-ci.d/container-cross.yml     | 192 ++++++
 .gitlab-ci.d/container-template.yml  |  21 +
 .gitlab-ci.d/containers.yml          | 232 +------
 .gitlab-ci.d/crossbuild-template.yml |  41 ++
 .gitlab-ci.d/crossbuilds.yml         |  60 +-
 .gitlab-ci.d/qemu-project.yml        |  11 +
 .gitlab-ci.d/stages.yml              |   8 +
 .gitlab-ci.d/static_checks.yml       |  26 +
 .gitlab-ci.yml                       | 889 ++-------------------------
 12 files changed, 1182 insertions(+), 1126 deletions(-)
 create mode 100644 .gitlab-ci.d/buildtest-template.yml
 create mode 100644 .gitlab-ci.d/buildtest.yml
 create mode 100644 .gitlab-ci.d/container-core.yml
 create mode 100644 .gitlab-ci.d/container-cross.yml
 create mode 100644 .gitlab-ci.d/container-template.yml
 create mode 100644 .gitlab-ci.d/crossbuild-template.yml
 create mode 100644 .gitlab-ci.d/qemu-project.yml
 create mode 100644 .gitlab-ci.d/stages.yml
 create mode 100644 .gitlab-ci.d/static_checks.yml

-- 
2.26.3

[PULL v2 08/18] gitlab: Extract default build/test jobs templates

[Philippe Mathieu-Daudé]

To be able to reuse the mainstream build/test jobs templates,
extract them into a new file (buildtest-template.yml).

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-8-f4bug@amsat.org>
[thuth: Keep the "acceptance_test_job_template" name for now]
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/buildtest-template.yml | 69 ++++++++++++++++++++++++++++
 .gitlab-ci.yml                      | 71 +----------------------------
 2 files changed, 70 insertions(+), 70 deletions(-)
 create mode 100644 .gitlab-ci.d/buildtest-template.yml

diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
new file mode 100644
index 00000000000..530d30746c7
--- /dev/null
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -0,0 +1,69 @@
+.native_build_job_template:
+  stage: build
+  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+  before_script:
+    - JOBS=$(expr $(nproc) + 1)
+  script:
+    - if test -n "$LD_JOBS";
+      then
+        scripts/git-submodule.sh update meson ;
+      fi
+    - mkdir build
+    - cd build
+    - if test -n "$TARGETS";
+      then
+        ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list="$TARGETS" ;
+      else
+        ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS ;
+      fi || { cat config.log meson-logs/meson-log.txt && exit 1; }
+    - if test -n "$LD_JOBS";
+      then
+        ../meson/meson.py configure . -Dbackend_max_links="$LD_JOBS" ;
+      fi || exit 1;
+    - make -j"$JOBS"
+    - if test -n "$MAKE_CHECK_ARGS";
+      then
+        make -j"$JOBS" $MAKE_CHECK_ARGS ;
+      fi
+
+.native_test_job_template:
+  stage: test
+  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
+  script:
+    - scripts/git-submodule.sh update
+        $(sed -n '/GIT_SUBMODULES=/ s/.*=// p' build/config-host.mak)
+    - cd build
+    - find . -type f -exec touch {} +
+    # Avoid recompiling by hiding ninja with NINJA=":"
+    - make NINJA=":" $MAKE_CHECK_ARGS
+
+.acceptance_test_job_template:
+  extends: .native_test_job_template
+  cache:
+    key: "${CI_JOB_NAME}-cache"
+    paths:
+      - ${CI_PROJECT_DIR}/avocado-cache
+    policy: pull-push
+  artifacts:
+    name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
+    when: always
+    expire_in: 2 days
+    paths:
+      - build/tests/results/latest/results.xml
+      - build/tests/results/latest/test-results
+    reports:
+      junit: build/tests/results/latest/results.xml
+  before_script:
+    - mkdir -p ~/.config/avocado
+    - echo "[datadir.paths]" > ~/.config/avocado/avocado.conf
+    - echo "cache_dirs = ['${CI_PROJECT_DIR}/avocado-cache']"
+           >> ~/.config/avocado/avocado.conf
+    - echo -e '[job.output.testlogs]\nstatuses = ["FAIL", "INTERRUPT"]'
+           >> ~/.config/avocado/avocado.conf
+    - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then
+        du -chs ${CI_PROJECT_DIR}/avocado-cache ;
+      fi
+    - export AVOCADO_ALLOW_UNTRUSTED_CODE=1
+  after_script:
+    - cd build
+    - du -chs ${CI_PROJECT_DIR}/avocado-cache
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 2be8097f193..ccd11e1625a 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -4,78 +4,9 @@ include:
   - local: '/.gitlab-ci.d/opensbi.yml'
   - local: '/.gitlab-ci.d/containers.yml'
   - local: '/.gitlab-ci.d/crossbuilds.yml'
+  - local: '/.gitlab-ci.d/buildtest-template.yml'
   - local: '/.gitlab-ci.d/static_checks.yml'
 
-.native_build_job_template:
-  stage: build
-  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
-  before_script:
-    - JOBS=$(expr $(nproc) + 1)
-  script:
-    - if test -n "$LD_JOBS";
-      then
-        scripts/git-submodule.sh update meson ;
-      fi
-    - mkdir build
-    - cd build
-    - if test -n "$TARGETS";
-      then
-        ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list="$TARGETS" ;
-      else
-        ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS ;
-      fi || { cat config.log meson-logs/meson-log.txt && exit 1; }
-    - if test -n "$LD_JOBS";
-      then
-        ../meson/meson.py configure . -Dbackend_max_links="$LD_JOBS" ;
-      fi || exit 1;
-    - make -j"$JOBS"
-    - if test -n "$MAKE_CHECK_ARGS";
-      then
-        make -j"$JOBS" $MAKE_CHECK_ARGS ;
-      fi
-
-.native_test_job_template:
-  stage: test
-  image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
-  script:
-    - scripts/git-submodule.sh update
-        $(sed -n '/GIT_SUBMODULES=/ s/.*=// p' build/config-host.mak)
-    - cd build
-    - find . -type f -exec touch {} +
-    # Avoid recompiling by hiding ninja with NINJA=":"
-    - make NINJA=":" $MAKE_CHECK_ARGS
-
-.acceptance_test_job_template:
-  extends: .native_test_job_template
-  cache:
-    key: "${CI_JOB_NAME}-cache"
-    paths:
-      - ${CI_PROJECT_DIR}/avocado-cache
-    policy: pull-push
-  artifacts:
-    name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
-    when: always
-    expire_in: 2 days
-    paths:
-      - build/tests/results/latest/results.xml
-      - build/tests/results/latest/test-results
-    reports:
-      junit: build/tests/results/latest/results.xml
-  before_script:
-    - mkdir -p ~/.config/avocado
-    - echo "[datadir.paths]" > ~/.config/avocado/avocado.conf
-    - echo "cache_dirs = ['${CI_PROJECT_DIR}/avocado-cache']"
-           >> ~/.config/avocado/avocado.conf
-    - echo -e '[job.output.testlogs]\nstatuses = ["FAIL", "INTERRUPT"]'
-           >> ~/.config/avocado/avocado.conf
-    - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then
-        du -chs ${CI_PROJECT_DIR}/avocado-cache ;
-      fi
-    - export AVOCADO_ALLOW_UNTRUSTED_CODE=1
-  after_script:
-    - cd build
-    - du -chs ${CI_PROJECT_DIR}/avocado-cache
-
 build-system-alpine:
   extends: .native_build_job_template
   needs:
-- 
2.26.3

Re: [PULL v2 00/18] gitlab-ci patches for 2021-05-26

[Peter Maydell]

On Thu, 27 May 2021 at 15:48, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>
> v2: Rebased on top of 7cf333a3726 ("gitlab-ci: use --meson=git for CFI jobs")
>
> The following changes since commit 8385235ba99c53d1187658f2fc289b953a8090b1:
>
>   Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging (2021-05-26 21:05:36 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/philmd/qemu.git tags/gitlab-ci-20210527
>
> for you to fetch changes up to 2d9e45b9cb0baa7316901238f3795a74206af457:
>
>   gitlab: Convert check-dco/check-patch jobs to the 'rules' syntax (2021-05-27 16:39:23 +0200)
>
> ----------------------------------------------------------------
> GitLab CI patches queue
>
> - Explode .gitlab-ci.yml in reusable templates
> - Add job to cross build/test TCI on i386 host
> - Remove CentOS 7 linux-user build job
> - Temporarily set Avocado-based jobs in manual mode
> - Increase time to hold Avocado reports to 1 week
>
> ----------------------------------------------------------------


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1
for any user-visible changes.

-- PMM