[PULL 00/10] gitlab and qtest patches

[PULL 00/10] gitlab and qtest patches

[Thomas Huth]

 Hi Peter!

The following changes since commit c79f01c9450bcf90c08a77f13fbf67bdba59a316:

  Merge remote-tracking branch 'remotes/rth-gitlab/tags/pull-hex-20210218' into staging (2021-02-18 16:33:36 +0000)

are available in the Git repository at:

  https://gitlab.com/huth/qemu.git tags/pull-request-2021-02-19

for you to fetch changes up to fc4241827fffc761f0ca41536c1c7d88b1b1d0c4:

  travis.yml: Limit simultaneous jobs to 3 (2021-02-19 06:31:40 +0100)

----------------------------------------------------------------
* Always build the container images in the gitlab-CI
* Some other small gitlab-CI improvements
* Some qtest fixes
* One patch to (hopefully) silence the failing Travis jobs
----------------------------------------------------------------

Daniel P. Berrangé (3):
      gitlab: always build container images
      gitlab: add fine grained job deps for all build jobs
      gitlab: fix inconsistent indentation

Philippe Mathieu-Daudé (3):
      tests/qtest/boot-serial-test: Test Virt machine with 'max'
      gitlab-ci: Display Avocado log content when tests timeout
      gitlab-ci: Disable vhost-kernel in build-disable job

Thomas Huth (4):
      scripts/checkpatch: Improve the check for authors mangled by the mailing list
      tests/qtest/boot-sector: Check that the guest did not panic
      gitlab-ci.yml: Run check-tcg with TCI
      travis.yml: Limit simultaneous jobs to 3

 .gitlab-ci.d/containers.yml    |  7 ----
 .gitlab-ci.d/crossbuilds.yml   | 46 ++++++++++++++++++++++++
 .gitlab-ci.yml                 | 81 ++++++++++++++++++++++++++++++++++++------
 .travis.yml                    |  4 ++-
 scripts/checkpatch.pl          |  2 +-
 tests/qtest/boot-sector.c      |  9 +++++
 tests/qtest/boot-serial-test.c |  2 +-
 7 files changed, 130 insertions(+), 21 deletions(-)

[PULL 01/10] tests/qtest/boot-serial-test: Test Virt machine with 'max'

[Thomas Huth]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

When using KVM, using a specific cpu type will only work if the
host CPU really is that exact CPU type.

During testing we can simply use the 'max' CPU which will select
all the features available from the host.

This allow running this test on a Cavium CN8890 (ThunderX cores).

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210205144345.2068758-4-f4bug@amsat.org>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qtest/boot-serial-test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/qtest/boot-serial-test.c b/tests/qtest/boot-serial-test.c
index b6b1c23cd0..d74509b1c5 100644
--- a/tests/qtest/boot-serial-test.c
+++ b/tests/qtest/boot-serial-test.c
@@ -149,7 +149,7 @@ static testdef_t tests[] = {
     { "arm", "raspi2", "", "TT", sizeof(bios_raspi2), 0, bios_raspi2 },
     /* For hppa, force bios to output to serial by disabling graphics. */
     { "hppa", "hppa", "-vga none", "SeaBIOS wants SYSTEM HALT" },
-    { "aarch64", "virt", "-cpu cortex-a57", "TT", sizeof(kernel_aarch64),
+    { "aarch64", "virt", "-cpu max", "TT", sizeof(kernel_aarch64),
       kernel_aarch64 },
     { "arm", "microbit", "", "T", sizeof(kernel_nrf51), kernel_nrf51 },
 
-- 
2.27.0

[PULL 02/10] gitlab: always build container images

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

Currently we attempt to skip building container images if the commits do
not involve changes to the dockerfiles or gitlab CI definitions.

Conceptually this makes sense, but there is a challenge in the real
world implementation of this in gitlab.

In the case of a CI pipeline triggered from a merge request, GitLab
knows the common ancestor of the merge request and the main git repo,
so it can trivially determine if any of the commits associated with
the MR change the dockerfiles.

In the case of a CI pipeline triggered from a push to a branch, it is
much more difficult. There is no concept of a common ancestor in this
case. Instead GitLab looks at the set of commits in the git push event.

On the surface this may sound reasonable, but it doesn't take into
account that a push event does not always contain the full set of
patches from a branch.

For example, consider pushing 5 commits, one of which contains a
dockerfile change. This will trigger a CI pipeline for the
containers. Now consider you do some more work on the branch and push 3
further commits, so you now have a branch of 8 commits. For the second
push GitLab will only look at the 3 most recent commits, the other 5
were already present. Thus GitLab will not realize that the branch has
dockerfile changes that need to trigger the container build.

This can cause real world problems:

 - Push 5 commits to branch "foo", including a dockerfile change

    => rebuilds the container images with content from "foo"
    => build jobs runs against containers from "foo"

 - Refresh your master branch with latest upstream master

    => rebuilds the container images with content from "master"
    => build jobs runs against containers from "master"

 - Push 3 more commits to branch "foo", with no dockerfile change

    => no container rebuild triggers
    => build jobs runs against containers from "master"

The "changes" conditional in gitlab is OK, *provided* your build
jobs are not relying on any external state from previous builds.

This is NOT the case in QEMU, because we are building container
images and these are cached. This is a scenario in which the
"changes" conditional is not usuable.

The only other way to avoid this problem would be to use the git
branch name as the container image tag, instead of always using
"latest". The downside of this approach is that the user's gitlab
registry will grow significantly until it starts to trigger
GitLab's automatic deletion policy.  Every time the user starts
a new branch they will have to trigger a rebuild of the container
images. Given this, we might as well just drop the conditional
and always build the container images. Most of the time docker
will be able to use the layer cache to avoid the most expensive
part of the rebuild process (installing all the RPMs/debs/etc)

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210216132954.295906-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/containers.yml | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
index 90fac85ce4..33e4046e23 100644
--- a/.gitlab-ci.d/containers.yml
+++ b/.gitlab-ci.d/containers.yml
@@ -20,13 +20,6 @@
     - docker push "$TAG"
   after_script:
     - docker logout
-  rules:
-    - changes:
-      - .gitlab-ci.d/containers.yml
-      - tests/docker/*
-      - tests/docker/dockerfiles/*
-    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
-    - if: '$CI_COMMIT_REF_NAME == "testing/next"'
 
 amd64-alpine-container:
   <<: *container_job_definition
-- 
2.27.0

[PULL 03/10] gitlab: add fine grained job deps for all build jobs

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

This allows the build jobs to start running as soon as their respective
container image is ready, instead of waiting for all container builds
to finish.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210216132954.295906-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.d/crossbuilds.yml | 46 ++++++++++++++++++++++++++++
 .gitlab-ci.yml               | 58 ++++++++++++++++++++++++++++++++++++
 2 files changed, 104 insertions(+)

diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 66547b6683..d5098c986b 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -41,114 +41,158 @@
 
 cross-armel-system:
   extends: .cross_system_build_job
+  needs:
+    job: armel-debian-cross-container
   variables:
     IMAGE: debian-armel-cross
 
 cross-armel-user:
   extends: .cross_user_build_job
+  needs:
+    job: armel-debian-cross-container
   variables:
     IMAGE: debian-armel-cross
 
 cross-armhf-system:
   extends: .cross_system_build_job
+  needs:
+    job: armhf-debian-cross-container
   variables:
     IMAGE: debian-armhf-cross
 
 cross-armhf-user:
   extends: .cross_user_build_job
+  needs:
+    job: armhf-debian-cross-container
   variables:
     IMAGE: debian-armhf-cross
 
 cross-arm64-system:
   extends: .cross_system_build_job
+  needs:
+    job: arm64-debian-cross-container
   variables:
     IMAGE: debian-arm64-cross
 
 cross-arm64-user:
   extends: .cross_user_build_job
+  needs:
+    job: arm64-debian-cross-container
   variables:
     IMAGE: debian-arm64-cross
 
 cross-i386-system:
   extends: .cross_system_build_job
+  needs:
+    job: i386-fedora-cross-container
   variables:
     IMAGE: fedora-i386-cross
     MAKE_CHECK_ARGS: check-qtest
 
 cross-i386-user:
   extends: .cross_user_build_job
+  needs:
+    job: i386-fedora-cross-container
   variables:
     IMAGE: fedora-i386-cross
     MAKE_CHECK_ARGS: check
 
 cross-mips-system:
   extends: .cross_system_build_job
+  needs:
+    job: mips-debian-cross-container
   variables:
     IMAGE: debian-mips-cross
 
 cross-mips-user:
   extends: .cross_user_build_job
+  needs:
+    job: mips-debian-cross-container
   variables:
     IMAGE: debian-mips-cross
 
 cross-mipsel-system:
   extends: .cross_system_build_job
+  needs:
+    job: mipsel-debian-cross-container
   variables:
     IMAGE: debian-mipsel-cross
 
 cross-mipsel-user:
   extends: .cross_user_build_job
+  needs:
+    job: mipsel-debian-cross-container
   variables:
     IMAGE: debian-mipsel-cross
 
 cross-mips64el-system:
   extends: .cross_system_build_job
+  needs:
+    job: mips64el-debian-cross-container
   variables:
     IMAGE: debian-mips64el-cross
 
 cross-mips64el-user:
   extends: .cross_user_build_job
+  needs:
+    job: mips64el-debian-cross-container
   variables:
     IMAGE: debian-mips64el-cross
 
 cross-ppc64el-system:
   extends: .cross_system_build_job
+  needs:
+    job: ppc64el-debian-cross-container
   variables:
     IMAGE: debian-ppc64el-cross
 
 cross-ppc64el-user:
   extends: .cross_user_build_job
+  needs:
+    job: ppc64el-debian-cross-container
   variables:
     IMAGE: debian-ppc64el-cross
 
 cross-s390x-system:
   extends: .cross_system_build_job
+  needs:
+    job: s390x-debian-cross-container
   variables:
     IMAGE: debian-s390x-cross
 
 cross-s390x-user:
   extends: .cross_user_build_job
+  needs:
+    job: s390x-debian-cross-container
   variables:
     IMAGE: debian-s390x-cross
 
 cross-s390x-kvm-only:
   extends: .cross_accel_build_job
+  needs:
+    job: s390x-debian-cross-container
   variables:
     IMAGE: debian-s390x-cross
     ACCEL_CONFIGURE_OPTS: --disable-tcg
 
 cross-win32-system:
   extends: .cross_system_build_job
+  needs:
+    job: win32-fedora-cross-container
   variables:
     IMAGE: fedora-win32-cross
 
 cross-win64-system:
   extends: .cross_system_build_job
+  needs:
+    job: win64-fedora-cross-container
   variables:
     IMAGE: fedora-win64-cross
 
 cross-amd64-xen-only:
   extends: .cross_accel_build_job
+  needs:
+    job: amd64-debian-cross-container
   variables:
     IMAGE: debian-amd64-cross
     ACCEL: xen
@@ -156,6 +200,8 @@ cross-amd64-xen-only:
 
 cross-arm64-xen-only:
   extends: .cross_accel_build_job
+  needs:
+    job: arm64-debian-cross-container
   variables:
     IMAGE: debian-arm64-cross
     ACCEL: xen
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 7adb9a4cef..32cc6bd4a2 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -109,6 +109,8 @@ acceptance-system-alpine:
 
 build-system-ubuntu:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-ubuntu2004-container
   variables:
     IMAGE: ubuntu2004
     CONFIGURE_ARGS: --enable-fdt=system --enable-slirp=system
@@ -141,6 +143,8 @@ acceptance-system-ubuntu:
 
 build-system-debian:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-debian-container
   variables:
     IMAGE: debian-amd64
     CONFIGURE_ARGS: --enable-fdt=system
@@ -186,6 +190,8 @@ acceptance-system-debian:
 
 build-system-fedora:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-fedora-container
   variables:
     IMAGE: fedora
     CONFIGURE_ARGS: --disable-gcrypt --enable-nettle --enable-docs
@@ -219,6 +225,8 @@ acceptance-system-fedora:
 
 build-system-centos:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-centos8-container
   variables:
     IMAGE: centos8
     CONFIGURE_ARGS: --disable-nettle --enable-gcrypt --enable-fdt=system
@@ -252,6 +260,8 @@ acceptance-system-centos:
 
 build-system-opensuse:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-opensuse-leap-container
   variables:
     IMAGE: opensuse-leap
     CONFIGURE_ARGS: --enable-fdt=system
@@ -284,6 +294,8 @@ acceptance-system-opensuse:
 
 build-disabled:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-fedora-container
   variables:
     IMAGE: fedora
     CONFIGURE_ARGS:
@@ -366,6 +378,8 @@ build-disabled:
 # available.
 build-tcg-disabled:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-centos8-container
   variables:
     IMAGE: centos8
   script:
@@ -386,6 +400,8 @@ build-tcg-disabled:
 
 build-user:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-debian-user-cross-container
   variables:
     IMAGE: debian-all-test-cross
     CONFIGURE_ARGS: --disable-tools --disable-system
@@ -393,6 +409,8 @@ build-user:
 
 build-user-static:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-debian-user-cross-container
   variables:
     IMAGE: debian-all-test-cross
     CONFIGURE_ARGS: --disable-tools --disable-system --static
@@ -401,6 +419,8 @@ build-user-static:
 # Only build the softmmu targets we have check-tcg tests for
 build-some-softmmu:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-debian-user-cross-container
   variables:
     IMAGE: debian-all-test-cross
     CONFIGURE_ARGS: --disable-tools --enable-debug
@@ -412,6 +432,8 @@ build-some-softmmu:
 # we skip cris-linux-user as it doesn't use the common run loop
 build-user-plugins:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-debian-user-cross-container
   variables:
     IMAGE: debian-all-test-cross
     CONFIGURE_ARGS: --disable-tools --disable-system --enable-plugins --enable-debug-tcg --target-list-exclude=sparc64-linux-user,cris-linux-user
@@ -427,6 +449,8 @@ build-user-centos7:
 
 build-some-softmmu-plugins:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-debian-user-cross-container
   variables:
     IMAGE: debian-all-test-cross
     CONFIGURE_ARGS: --disable-tools --disable-user --enable-plugins --enable-debug-tcg
@@ -435,6 +459,8 @@ build-some-softmmu-plugins:
 
 clang-system:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-fedora-container
   variables:
     IMAGE: fedora
     CONFIGURE_ARGS: --cc=clang --cxx=clang++
@@ -464,6 +490,8 @@ tsan-build:
 # These targets are on the way out
 build-deprecated:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-debian-user-cross-container
   variables:
     IMAGE: debian-all-test-cross
     CONFIGURE_ARGS: --disable-docs --disable-tools
@@ -501,6 +529,8 @@ gprof-gcov:
 
 build-oss-fuzz:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-fedora-container
   variables:
     IMAGE: fedora
   script:
@@ -519,6 +549,8 @@ build-oss-fuzz:
 
 build-tci:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-fedora-container
   variables:
     IMAGE: fedora
   script:
@@ -542,6 +574,8 @@ build-tci:
 # However we can't test against KVM on Gitlab-CI so we can only run unit tests
 build-coroutine-ucontext:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-ubuntu2004-container
   variables:
     IMAGE: ubuntu2004
     CONFIGURE_ARGS: --with-coroutine=ucontext --disable-tcg
@@ -549,6 +583,8 @@ build-coroutine-ucontext:
 
 build-coroutine-sigaltstack:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-ubuntu2004-container
   variables:
     IMAGE: ubuntu2004
     CONFIGURE_ARGS: --with-coroutine=sigaltstack --disable-tcg
@@ -560,6 +596,8 @@ build-coroutine-sigaltstack:
 # which had some API differences.
 build-crypto-old-nettle:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-centos7-container
   variables:
     IMAGE: centos7
     TARGETS: x86_64-softmmu x86_64-linux-user
@@ -581,6 +619,8 @@ check-crypto-old-nettle:
 
 build-crypto-old-gcrypt:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-centos7-container
   variables:
     IMAGE: centos7
     TARGETS: x86_64-softmmu x86_64-linux-user
@@ -602,6 +642,8 @@ check-crypto-old-gcrypt:
 
 build-crypto-only-gnutls:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-centos7-container
   variables:
     IMAGE: centos7
     TARGETS: x86_64-softmmu x86_64-linux-user
@@ -623,18 +665,24 @@ check-crypto-only-gnutls:
 # We don't need to exercise every backend with every front-end
 build-trace-multi-user:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-ubuntu2004-container
   variables:
     IMAGE: ubuntu2004
     CONFIGURE_ARGS: --enable-trace-backends=log,simple,syslog --disable-system
 
 build-trace-ftrace-system:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-ubuntu2004-container
   variables:
     IMAGE: ubuntu2004
     CONFIGURE_ARGS: --enable-trace-backends=ftrace --target-list=x86_64-softmmu
 
 build-trace-ust-system:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-ubuntu2004-container
   variables:
     IMAGE: ubuntu2004
     CONFIGURE_ARGS: --enable-trace-backends=ust --target-list=x86_64-softmmu
@@ -642,12 +690,16 @@ build-trace-ust-system:
 # Check our reduced build configurations
 build-without-default-devices:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-centos8-container
   variables:
     IMAGE: centos8
     CONFIGURE_ARGS: --without-default-devices --disable-user
 
 build-without-default-features:
   <<: *native_build_job_definition
+  needs:
+    job: amd64-debian-container
   variables:
     IMAGE: debian-amd64
     CONFIGURE_ARGS: --without-default-features --disable-user
@@ -657,6 +709,8 @@ build-without-default-features:
 check-patch:
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
+  needs:
+    job: amd64-centos8-container
   script: .gitlab-ci.d/check-patch.py
   except:
     variables:
@@ -668,6 +722,8 @@ check-patch:
 check-dco:
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
+  needs:
+    job: amd64-centos8-container
   script: .gitlab-ci.d/check-dco.py
   except:
     variables:
@@ -678,6 +734,8 @@ check-dco:
 build-libvhost-user:
   stage: build
   image: $CI_REGISTRY_IMAGE/qemu/fedora:latest
+  needs:
+    job: amd64-fedora-container
   before_script:
     - dnf install -y meson ninja-build
   script:
-- 
2.27.0

[PULL 04/10] gitlab: fix inconsistent indentation

[Thomas Huth]

From: Daniel P. Berrangé <berrange@redhat.com>

The standard is to use 2 space indent, not 3.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210216132954.295906-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.yml | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 32cc6bd4a2..870d5f83f5 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -282,14 +282,14 @@ check-system-opensuse:
     MAKE_CHECK_ARGS: check
 
 acceptance-system-opensuse:
-   <<: *native_test_job_definition
-   needs:
-     - job: build-system-opensuse
-       artifacts: true
-   variables:
-     IMAGE: opensuse-leap
-     MAKE_CHECK_ARGS: check-acceptance
-   <<: *acceptance_definition
+  <<: *native_test_job_definition
+  needs:
+    - job: build-system-opensuse
+      artifacts: true
+  variables:
+    IMAGE: opensuse-leap
+    MAKE_CHECK_ARGS: check-acceptance
+  <<: *acceptance_definition
 
 
 build-disabled:
-- 
2.27.0

[PULL 05/10] gitlab-ci: Display Avocado log content when tests timeout

[Thomas Huth]

From: Philippe Mathieu-Daudé <philmd@redhat.com>

Since commit ba2232bae6d ("gitlab-ci: Refactor code that show logs
of failed acceptances") we display the log content of failing tests
(Avocado "FAIL" event).

Since we are also interested in tests timeouting, update our global
Avocado config to display log content for the "INTERRUPT" event,
"possible when the timeout is reached" (See [*]).

[*] https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#test-statuses

Suggested-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210215171438.935665-1-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 870d5f83f5..fcbec77a91 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -64,7 +64,7 @@ include:
     - echo "[datadir.paths]" > ~/.config/avocado/avocado.conf
     - echo "cache_dirs = ['${CI_PROJECT_DIR}/avocado-cache']"
            >> ~/.config/avocado/avocado.conf
-    - echo -e '[job.output.testlogs]\nstatuses = ["FAIL"]'
+    - echo -e '[job.output.testlogs]\nstatuses = ["FAIL", "INTERRUPT"]'
            >> ~/.config/avocado/avocado.conf
     - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then
         du -chs ${CI_PROJECT_DIR}/avocado-cache ;
-- 
2.27.0

[PULL 06/10] scripts/checkpatch: Improve the check for authors mangled by the mailing list

[Thomas Huth]

There were recently some patches on the list which had their "From:"
line mangled like this:

 From: qemu_oss--- via <qemu-devel@nongnu.org>

Since our test in the checkpatch.pl script did not trigger here, the
patches finally also ended up in a pull request, with the wrong author
set. So let's improve the regular expression to also complain on
these new patterns, too.

Message-Id: <20210216071512.1199827-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 scripts/checkpatch.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index e47ad878d8..7f194c842b 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -1499,7 +1499,7 @@ sub process {
 			$is_patch = 1;
 		}
 
-		if ($line =~ /^Author: .*via Qemu-devel.*<qemu-devel\@nongnu.org>/) {
+		if ($line =~ /^(Author|From): .* via .*<qemu-devel\@nongnu.org>/) {
 		    ERROR("Author email address is mangled by the mailing list\n" . $herecurr);
 		}
 
-- 
2.27.0

[PULL 07/10] gitlab-ci: Disable vhost-kernel in build-disable job

[Thomas Huth]

From: Philippe Mathieu-Daudé <f4bug@amsat.org>

Commit 299e6f19b3e ("vhost-net: revamp configure logic") added
the --enable-vhost-kernel option.
Disable it in the build-disable job.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210131104621.221602-1-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index fcbec77a91..c06c20be6c 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -358,6 +358,7 @@ build-disabled:
       --disable-vhost-crypto
       --disable-vhost-net
       --disable-vhost-scsi
+      --disable-vhost-kernel
       --disable-vhost-user
       --disable-vhost-vdpa
       --disable-vhost-vsock
-- 
2.27.0

[PULL 08/10] tests/qtest/boot-sector: Check that the guest did not panic

[Thomas Huth]

The s390-ccw bios code panics if it can not boot successfully. In
this case, it does not make sense that we wait the full 600 seconds
for the boot sector test to finish and can signal the failure
immediately, thus let's check the status of the guest with the
"query-status" QMP command here, too.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20210212113141.854871-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/qtest/boot-sector.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/tests/qtest/boot-sector.c b/tests/qtest/boot-sector.c
index 24df5c4734..ea8f264661 100644
--- a/tests/qtest/boot-sector.c
+++ b/tests/qtest/boot-sector.c
@@ -138,6 +138,7 @@ void boot_sector_test(QTestState *qts)
     uint8_t signature_low;
     uint8_t signature_high;
     uint16_t signature;
+    QDict *qrsp, *qret;
     int i;
 
     /* Wait at most 600 seconds (test is slow with TCI and --enable-debug) */
@@ -155,6 +156,14 @@ void boot_sector_test(QTestState *qts)
         if (signature == SIGNATURE) {
             break;
         }
+
+        /* check that guest is still in "running" state and did not panic */
+        qrsp = qtest_qmp(qts, "{ 'execute': 'query-status' }");
+        qret = qdict_get_qdict(qrsp, "return");
+        g_assert_nonnull(qret);
+        g_assert_cmpstr(qdict_get_try_str(qret, "status"), ==, "running");
+        qobject_unref(qrsp);
+
         g_usleep(TEST_DELAY);
     }
 
-- 
2.27.0

[PULL 09/10] gitlab-ci.yml: Run check-tcg with TCI

[Thomas Huth]

It's now possible to also run the non-x86 TCG tests with TCI.

Message-Id: <20210127055903.40148-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.yml | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index c06c20be6c..8b6d495288 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -551,9 +551,9 @@ build-oss-fuzz:
 build-tci:
   <<: *native_build_job_definition
   needs:
-    job: amd64-fedora-container
+    job: amd64-debian-user-cross-container
   variables:
-    IMAGE: fedora
+    IMAGE: debian-all-test-cross
   script:
     - TARGETS="aarch64 alpha arm hppa m68k microblaze moxie ppc64 s390x x86_64"
     - mkdir build
@@ -561,7 +561,6 @@ build-tci:
     - ../configure --enable-tcg-interpreter
         --target-list="$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; done)" || { cat config.log meson-logs/meson-log.txt && exit 1; }
     - make -j"$JOBS"
-    - make run-tcg-tests-x86_64-softmmu
     - make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest/pxe-test
     - for tg in $TARGETS ; do
         export QTEST_QEMU_BINARY="./qemu-system-${tg}" ;
@@ -570,6 +569,7 @@ build-tci:
       done
     - QTEST_QEMU_BINARY="./qemu-system-x86_64" ./tests/qtest/pxe-test
     - QTEST_QEMU_BINARY="./qemu-system-s390x" ./tests/qtest/pxe-test -m slow
+    - make check-tcg
 
 # Alternate coroutines implementations are only really of interest to KVM users
 # However we can't test against KVM on Gitlab-CI so we can only run unit tests
-- 
2.27.0

[PULL 10/10] travis.yml: Limit simultaneous jobs to 3

[Thomas Huth]

Even though the host machines that run the Travis CI jobs have
quite a lot of CPUs (e.g. nproc in an aarch64 job reports 32), the
containers on Travis are still limited to 2 vCPUs according to:

 https://docs.travis-ci.com/user/reference/overview/#approx-boot-time

So we do not gain much when compiling with a job number based on
the output of "getconf _NPROCESSORS_ONLN" - quite the contrary, the
aarch64 containers are currently aborting quite often since they
are running out of memory. Thus let's rather use a fixed number
like 3 in the jobs here, so that e.g. two threads can actively run
while a third one might be waiting for I/O operations to complete.
This should hopefully fix the out-of-memory failures in the aarch64
CI jobs.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210217102531.1441557-1-thuth@redhat.com>
[AJB: add comment]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210217121932.19986-6-alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .travis.yml | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index fc27fd6330..4609240b5a 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -86,9 +86,11 @@ git:
   submodules: false
 
 # Common first phase for all steps
+# We no longer use nproc to calculate jobs:
+# https://travis-ci.community/t/nproc-reports-32-cores-on-arm64/5851
 before_install:
   - if command -v ccache ; then ccache --zero-stats ; fi
-  - export JOBS=$(($(getconf _NPROCESSORS_ONLN) + 1))
+  - export JOBS=3
   - echo "=== Using ${JOBS} simultaneous jobs ==="
 
 # Configure step - may be overridden
-- 
2.27.0

Re: [PULL 00/10] gitlab and qtest patches

[Peter Maydell]

On Fri, 19 Feb 2021 at 07:57, Thomas Huth <thuth@redhat.com> wrote:
>
>  Hi Peter!
>
> The following changes since commit c79f01c9450bcf90c08a77f13fbf67bdba59a316:
>
>   Merge remote-tracking branch 'remotes/rth-gitlab/tags/pull-hex-20210218' into staging (2021-02-18 16:33:36 +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/huth/qemu.git tags/pull-request-2021-02-19
>
> for you to fetch changes up to fc4241827fffc761f0ca41536c1c7d88b1b1d0c4:
>
>   travis.yml: Limit simultaneous jobs to 3 (2021-02-19 06:31:40 +0100)
>
> ----------------------------------------------------------------
> * Always build the container images in the gitlab-CI
> * Some other small gitlab-CI improvements
> * Some qtest fixes
> * One patch to (hopefully) silence the failing Travis jobs
> ----------------------------------------------------------------


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/6.0
for any user-visible changes.

-- PMM