[PATCH 0/7] Gitlab CI improvements and fixes

[PATCH 0/7] Gitlab CI improvements and fixes

[Thomas Huth]

Currently, almost all piplines of our Gitlab CI are broken, since there
has been a change in the default Debian-based container that causes the
QEMU build process to fail (aka. the "BITS_PER_LONG" macro problem).

We could likely either wait for the containers to get fixed, or we could
try to work-around that issue in the QEMU sources, but considering that
we test with Ubuntu (and thus Debian-based) containers in our Travis-CI
already, we should maybe do something different here and rather switch
our Gitlab-CI to some RPM-based distros instead, to get even more test
coverage. This is now done in the 5th patch. Along the way, there are
clean-ups and some minor improvements, and a fix to be able to build QEMU
with Clang again (last patch).

After applying this patch series, the Gitlab-CI shows "green" again:

 https://gitlab.com/huth/qemu/pipelines/149326595

Alex Bennée (1):
  linux-user: limit check to HOST_LONG_BITS < TARGET_ABI_BITS

Cleber Rosa (1):
  GitLab CI: avoid calling before_scripts on unintended jobs

Philippe Mathieu-Daudé (1):
  gitlab-ci: Remove flex/bison packages

Thomas Huth (4):
  MAINTAINERS: Add Philippe, Alex and Wainer to the Gitlab-CI section
  gitlab-ci: Move edk2 and opensbi YAML files to .gitlab-ci.d folder
  gitlab-ci: Do not use the standard container images from gitlab
  gitlab-ci: Determine the number of jobs dynamically

 .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml  |  0
 .../opensbi.yml                               |  0
 .gitlab-ci.yml                                | 67 +++++++++++++------
 MAINTAINERS                                   |  5 +-
 linux-user/elfload.c                          |  2 +
 5 files changed, 51 insertions(+), 23 deletions(-)
 rename .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml (100%)
 rename .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml (100%)

-- 
2.18.1

[PATCH 1/7] MAINTAINERS: Add Philippe, Alex and Wainer to the Gitlab-CI section

[Thomas Huth]

Initially, I was the only one who was using Gitlab while most developers
had their git trees still on other systems, but that has changed nowadays.
There is now much more interest in the Gitlab-CI today, so it would be
good to have more than only one maintainer / reviewer for the gitlab-ci.yml
file. Alex, Wainer and Philippe kindly offered their help here, so let's
add them to the corresponding section in the MAINTAINERS file now.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 MAINTAINERS | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 87a412c229..bde5fd480f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2893,6 +2893,9 @@ W: https://cirrus-ci.com/github/qemu/qemu
 
 GitLab Continuous Integration
 M: Thomas Huth <thuth@redhat.com>
+M: Philippe Mathieu-Daudé <philmd@redhat.com>
+M: Alex Bennée <alex.bennee@linaro.org>
+R: Wainer dos Santos Moschetta <wainersm@redhat.com>
 S: Maintained
 F: .gitlab-ci.yml
 
-- 
2.18.1

[PATCH 2/7] gitlab-ci: Remove flex/bison packages

[Thomas Huth]

From: Philippe Mathieu-Daudé <philmd@redhat.com>

QEMU does not use flex/bison packages.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200515163029.12917-4-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index b889fb96b6..994774250f 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -4,7 +4,7 @@ include:
 
 before_script:
  - apt-get update -qq
- - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
+ - apt-get install -y -qq libglib2.0-dev libpixman-1-dev genisoimage
 
 build-system1:
  script:
-- 
2.18.1

[PATCH 3/7] GitLab CI: avoid calling before_scripts on unintended jobs

[Thomas Huth]

From: Cleber Rosa <crosa@redhat.com>

At this point it seems that all jobs depend on those steps, with
maybe the EDK2 jobs as exceptions.

The jobs that will be added later will not want those scripts to be
run, so let's move these steps to the appropriate jobs, while
still trying to avoid repetition.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
[thuth: Rebased to current master branch, use separate template]
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.yml | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 994774250f..bc6aee6aba 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -2,11 +2,13 @@ include:
   - local: '/.gitlab-ci-edk2.yml'
   - local: '/.gitlab-ci-opensbi.yml'
 
-before_script:
- - apt-get update -qq
- - apt-get install -y -qq libglib2.0-dev libpixman-1-dev genisoimage
+.update_apt_template: &before_script_apt
+ before_script:
+  - apt-get update -qq
+  - apt-get install -y -qq libglib2.0-dev libpixman-1-dev genisoimage
 
 build-system1:
+ <<: *before_script_apt
  script:
  - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
       libusb-dev libvde-dev libspice-protocol-dev libgl1-mesa-dev libvdeplug-dev
@@ -19,6 +21,7 @@ build-system1:
  - make -j2 check
 
 build-system2:
+ <<: *before_script_apt
  script:
  - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
       libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
@@ -32,6 +35,7 @@ build-system2:
  - make -j2 check
 
 build-disabled:
+ <<: *before_script_apt
  script:
  - mkdir build
  - cd build
@@ -46,6 +50,7 @@ build-disabled:
  - make -j2 check-qtest SPEED=slow
 
 build-tcg-disabled:
+ <<: *before_script_apt
  script:
  - apt-get install -y -qq clang libgtk-3-dev libusb-dev
  - mkdir build
@@ -64,6 +69,7 @@ build-tcg-disabled:
             260 261 262 263 264 270 272 273 277 279
 
 build-user:
+ <<: *before_script_apt
  script:
  - mkdir build
  - cd build
@@ -73,6 +79,7 @@ build-user:
  - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user
 
 build-clang:
+ <<: *before_script_apt
  script:
  - apt-get install -y -qq clang libsdl2-dev libattr1-dev libcap-ng-dev
       xfslibs-dev libiscsi-dev libnfs-dev libseccomp-dev gnutls-dev librbd-dev
@@ -85,6 +92,7 @@ build-clang:
  - make -j2 check
 
 build-tci:
+ <<: *before_script_apt
  script:
  - TARGETS="aarch64 alpha arm hppa m68k microblaze moxie ppc64 s390x x86_64"
  - mkdir build
-- 
2.18.1

[PATCH 4/7] gitlab-ci: Move edk2 and opensbi YAML files to .gitlab-ci.d folder

[Thomas Huth]

We have a dedicated folder for the gitlab-ci - so there is no need
to clutter the top directory with these .yml files.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml       | 0
 .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml | 0
 .gitlab-ci.yml                                     | 4 ++--
 MAINTAINERS                                        | 2 +-
 4 files changed, 3 insertions(+), 3 deletions(-)
 rename .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml (100%)
 rename .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml (100%)

diff --git a/.gitlab-ci-edk2.yml b/.gitlab-ci.d/edk2.yml
similarity index 100%
rename from .gitlab-ci-edk2.yml
rename to .gitlab-ci.d/edk2.yml
diff --git a/.gitlab-ci-opensbi.yml b/.gitlab-ci.d/opensbi.yml
similarity index 100%
rename from .gitlab-ci-opensbi.yml
rename to .gitlab-ci.d/opensbi.yml
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index bc6aee6aba..5208d93ff8 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,6 +1,6 @@
 include:
-  - local: '/.gitlab-ci-edk2.yml'
-  - local: '/.gitlab-ci-opensbi.yml'
+  - local: '/.gitlab-ci.d/edk2.yml'
+  - local: '/.gitlab-ci.d/opensbi.yml'
 
 .update_apt_template: &before_script_apt
  before_script:
diff --git a/MAINTAINERS b/MAINTAINERS
index bde5fd480f..d43c98115c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2534,7 +2534,7 @@ F: roms/edk2
 F: roms/edk2-*
 F: tests/data/uefi-boot-images/
 F: tests/uefi-test-tools/
-F: .gitlab-ci-edk2.yml
+F: .gitlab-ci.d/edk2.yml
 F: .gitlab-ci.d/edk2/
 
 Usermode Emulation
-- 
2.18.1

[PATCH 5/7] gitlab-ci: Do not use the standard container images from gitlab

[Thomas Huth]

Currently all pipelines of the gitlab CI are failing, except for the
"build-user" pipeline. There is an issue with the default container
image (likely Debian stable) where they imported something bad in one
of the system headers:

 /usr/include/linux/swab.h: In function '__swab':
 /builds/huth/qemu/include/qemu/bitops.h:20:34: error: "sizeof" is not
  defined, evaluates to 0 [-Werror=undef]
 #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)

We could maybe work-around this issue or wait for the default containers
to get fixed, but considering that we use Ubuntu (and thus Debian-style)
CI in Travis already to a very large extent, we should consider to use
some RPM-based distros in our gitlab CI instead. Thus let's change the
failing pipelines to use Fedora and CentOS (and also one Ubuntu 19.10,
since 20.04 is broken, too) now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.yml | 37 +++++++++++++++++++++++++------------
 1 file changed, 25 insertions(+), 12 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5208d93ff8..559ec2ab4d 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -5,9 +5,17 @@ include:
 .update_apt_template: &before_script_apt
  before_script:
   - apt-get update -qq
-  - apt-get install -y -qq libglib2.0-dev libpixman-1-dev genisoimage
+  - apt-get install -y -qq git gcc libglib2.0-dev libpixman-1-dev make
+        genisoimage
+
+.update_dnf_template: &before_script_dnf
+ before_script:
+  - dnf update -y
+  - dnf install -y bzip2 diffutils gcc git genisoimage findutils glib2-devel
+        make python3 perl-podlators perl-Test-Harness pixman-devel zlib-devel
 
 build-system1:
+ image: ubuntu:19.10
  <<: *before_script_apt
  script:
  - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
@@ -21,11 +29,12 @@ build-system1:
  - make -j2 check
 
 build-system2:
- <<: *before_script_apt
+ image: fedora:latest
+ <<: *before_script_dnf
  script:
- - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
-      libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
-      libzstd-dev
+ - yum install -y SDL2-devel libgcrypt-devel brlapi-devel libaio-devel
+       libfdt-devel lzo-devel librdmacm-devel libibverbs-devel libibumad-devel
+       libzstd-devel
  - mkdir build
  - cd build
  - ../configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
@@ -35,7 +44,8 @@ build-system2:
  - make -j2 check
 
 build-disabled:
- <<: *before_script_apt
+ image: fedora:latest
+ <<: *before_script_dnf
  script:
  - mkdir build
  - cd build
@@ -50,9 +60,10 @@ build-disabled:
  - make -j2 check-qtest SPEED=slow
 
 build-tcg-disabled:
- <<: *before_script_apt
+ image: centos:8
+ <<: *before_script_dnf
  script:
- - apt-get install -y -qq clang libgtk-3-dev libusb-dev
+ - dnf install -y clang gtk3-devel libusbx-devel libgcrypt-devel
  - mkdir build
  - cd build
  - ../configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
@@ -79,10 +90,11 @@ build-user:
  - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user
 
 build-clang:
- <<: *before_script_apt
+ image: fedora:latest
+ <<: *before_script_dnf
  script:
- - apt-get install -y -qq clang libsdl2-dev libattr1-dev libcap-ng-dev
-      xfslibs-dev libiscsi-dev libnfs-dev libseccomp-dev gnutls-dev librbd-dev
+ - yum install -y clang SDL2-devel libattr-devel libcap-ng-devel xfsprogs-devel
+       libiscsi-devel libnfs-devel libseccomp-devel gnutls-devel librbd-devel
  - mkdir build
  - cd build
  - ../configure --cc=clang --cxx=clang++ --enable-werror
@@ -92,7 +104,8 @@ build-clang:
  - make -j2 check
 
 build-tci:
- <<: *before_script_apt
+ image: centos:8
+ <<: *before_script_dnf
  script:
  - TARGETS="aarch64 alpha arm hppa m68k microblaze moxie ppc64 s390x x86_64"
  - mkdir build
-- 
2.18.1

[PATCH 6/7] gitlab-ci: Determine the number of jobs dynamically

[Thomas Huth]

Some people might want to run the gitlab CI pipelines in an environment
where multiple CPUs are available to the runners, so let's rather get
the number for "-j" from the "nproc" program (increased by 1 to compensate
for jobs that wait for I/O) instead of hard-coding it.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 .gitlab-ci.yml | 24 +++++++++++++-----------
 1 file changed, 13 insertions(+), 11 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 559ec2ab4d..349c77aa58 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -7,12 +7,14 @@ include:
   - apt-get update -qq
   - apt-get install -y -qq git gcc libglib2.0-dev libpixman-1-dev make
         genisoimage
+  - JOBS=$(expr $(nproc) + 1)
 
 .update_dnf_template: &before_script_dnf
  before_script:
   - dnf update -y
   - dnf install -y bzip2 diffutils gcc git genisoimage findutils glib2-devel
         make python3 perl-podlators perl-Test-Harness pixman-devel zlib-devel
+  - JOBS=$(expr $(nproc) + 1)
 
 build-system1:
  image: ubuntu:19.10
@@ -25,8 +27,8 @@ build-system1:
  - ../configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
       cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
       mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
- - make -j2
- - make -j2 check
+ - make -j"$JOBS"
+ - make -j"$JOBS" check
 
 build-system2:
  image: fedora:latest
@@ -40,8 +42,8 @@ build-system2:
  - ../configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
       microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
       sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
- - make -j2
- - make -j2 check
+ - make -j"$JOBS"
+ - make -j"$JOBS" check
 
 build-disabled:
  image: fedora:latest
@@ -56,8 +58,8 @@ build-disabled:
       --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
       --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
       --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
- - make -j2
- - make -j2 check-qtest SPEED=slow
+ - make -j"$JOBS"
+ - make -j"$JOBS" check-qtest SPEED=slow
 
 build-tcg-disabled:
  image: centos:8
@@ -67,7 +69,7 @@ build-tcg-disabled:
  - mkdir build
  - cd build
  - ../configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
- - make -j2
+ - make -j"$JOBS"
  - make check-unit
  - make check-qapi-schema
  - cd tests/qemu-iotests/
@@ -86,7 +88,7 @@ build-user:
  - cd build
  - ../configure --enable-werror --disable-system --disable-guest-agent
                --disable-capstone --disable-slirp --disable-fdt
- - make -j2
+ - make -j"$JOBS"
  - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user
 
 build-clang:
@@ -100,8 +102,8 @@ build-clang:
  - ../configure --cc=clang --cxx=clang++ --enable-werror
       --target-list="alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu
                      ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user"
- - make -j2
- - make -j2 check
+ - make -j"$JOBS"
+ - make -j"$JOBS" check
 
 build-tci:
  image: centos:8
@@ -112,7 +114,7 @@ build-tci:
  - cd build
  - ../configure --enable-tcg-interpreter
       --target-list="$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; done)"
- - make -j2
+ - make -j"$JOBS"
  - make run-tcg-tests-x86_64-softmmu
  - make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest/pxe-test
  - for tg in $TARGETS ; do
-- 
2.18.1

[PATCH 7/7] linux-user: limit check to HOST_LONG_BITS < TARGET_ABI_BITS

[Thomas Huth]

From: Alex Bennée <alex.bennee@linaro.org>

Newer clangs rightly spot that you can never exceed the full address
space of 64 bit hosts with:

  linux-user/elfload.c:2076:41: error: result of comparison 'unsigned
  long' > 18446744073709551615 is always false
  [-Werror,-Wtautological-type-limit-compare]
  4685         if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
  4686             ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~
  4687 1 error generated.

So lets limit the check to 32 bit hosts only.

Fixes: ee94743034bf
Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
[thuth: Use HOST_LONG_BITS < TARGET_ABI_BITS instead of HOST_LONG_BITS == 32]
Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 linux-user/elfload.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 01a9323a63..ebc663ea0b 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -2073,12 +2073,14 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
             exit(EXIT_FAILURE);
         }
     } else {
+#if HOST_LONG_BITS < TARGET_ABI_BITS
         if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
             error_report("%s: requires more virtual address space "
                          "than the host can provide (0x%" PRIx64 ")",
                          image_name, (uint64_t)guest_hiaddr - guest_base);
             exit(EXIT_FAILURE);
         }
+#endif
     }
 
     /*
-- 
2.18.1

Re: [PATCH 4/7] gitlab-ci: Move edk2 and opensbi YAML files to .gitlab-ci.d folder

[Philippe Mathieu-Daudé]

On 5/25/20 3:18 PM, Thomas Huth wrote:
> We have a dedicated folder for the gitlab-ci - so there is no need
> to clutter the top directory with these .yml files.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml       | 0
>  .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml | 0
>  .gitlab-ci.yml                                     | 4 ++--
>  MAINTAINERS                                        | 2 +-
>  4 files changed, 3 insertions(+), 3 deletions(-)
>  rename .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml (100%)
>  rename .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml (100%)
> 
> diff --git a/.gitlab-ci-edk2.yml b/.gitlab-ci.d/edk2.yml
> similarity index 100%
> rename from .gitlab-ci-edk2.yml
> rename to .gitlab-ci.d/edk2.yml
> diff --git a/.gitlab-ci-opensbi.yml b/.gitlab-ci.d/opensbi.yml
> similarity index 100%
> rename from .gitlab-ci-opensbi.yml
> rename to .gitlab-ci.d/opensbi.yml
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index bc6aee6aba..5208d93ff8 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -1,6 +1,6 @@
>  include:
> -  - local: '/.gitlab-ci-edk2.yml'
> -  - local: '/.gitlab-ci-opensbi.yml'
> +  - local: '/.gitlab-ci.d/edk2.yml'
> +  - local: '/.gitlab-ci.d/opensbi.yml'
>  
>  .update_apt_template: &before_script_apt
>   before_script:
> diff --git a/MAINTAINERS b/MAINTAINERS
> index bde5fd480f..d43c98115c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2534,7 +2534,7 @@ F: roms/edk2
>  F: roms/edk2-*
>  F: tests/data/uefi-boot-images/
>  F: tests/uefi-test-tools/
> -F: .gitlab-ci-edk2.yml
> +F: .gitlab-ci.d/edk2.yml
>  F: .gitlab-ci.d/edk2/
>  
>  Usermode Emulation
> 

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Re: [PATCH 7/7] linux-user: limit check to HOST_LONG_BITS < TARGET_ABI_BITS

[Laurent Vivier]

Le 25/05/2020 à 15:18, Thomas Huth a écrit :
> From: Alex Bennée <alex.bennee@linaro.org>
> 
> Newer clangs rightly spot that you can never exceed the full address
> space of 64 bit hosts with:
> 
>   linux-user/elfload.c:2076:41: error: result of comparison 'unsigned
>   long' > 18446744073709551615 is always false
>   [-Werror,-Wtautological-type-limit-compare]
>   4685         if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
>   4686             ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~
>   4687 1 error generated.
> 
> So lets limit the check to 32 bit hosts only.
> 
> Fixes: ee94743034bf
> Reported-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> [thuth: Use HOST_LONG_BITS < TARGET_ABI_BITS instead of HOST_LONG_BITS == 32]
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  linux-user/elfload.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/linux-user/elfload.c b/linux-user/elfload.c
> index 01a9323a63..ebc663ea0b 100644
> --- a/linux-user/elfload.c
> +++ b/linux-user/elfload.c
> @@ -2073,12 +2073,14 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
>              exit(EXIT_FAILURE);
>          }
>      } else {
> +#if HOST_LONG_BITS < TARGET_ABI_BITS
>          if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
>              error_report("%s: requires more virtual address space "
>                           "than the host can provide (0x%" PRIx64 ")",
>                           image_name, (uint64_t)guest_hiaddr - guest_base);
>              exit(EXIT_FAILURE);
>          }
> +#endif
>      }
>  
>      /*
> 

Philippe sent the same patch:

https://www.mail-archive.com/qemu-devel@nongnu.org/msg699796.html

Thanks,
Laurent

Re: [PATCH 7/7] linux-user: limit check to HOST_LONG_BITS < TARGET_ABI_BITS

[Thomas Huth]

On 27/05/2020 16.44, Laurent Vivier wrote:
> Le 25/05/2020 à 15:18, Thomas Huth a écrit :
>> From: Alex Bennée <alex.bennee@linaro.org>
>>
>> Newer clangs rightly spot that you can never exceed the full address
>> space of 64 bit hosts with:
>>
>>   linux-user/elfload.c:2076:41: error: result of comparison 'unsigned
>>   long' > 18446744073709551615 is always false
>>   [-Werror,-Wtautological-type-limit-compare]
>>   4685         if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
>>   4686             ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~
>>   4687 1 error generated.
>>
>> So lets limit the check to 32 bit hosts only.
>>
>> Fixes: ee94743034bf
>> Reported-by: Thomas Huth <thuth@redhat.com>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> [thuth: Use HOST_LONG_BITS < TARGET_ABI_BITS instead of HOST_LONG_BITS == 32]
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>  linux-user/elfload.c | 2 ++
>>  1 file changed, 2 insertions(+)
>>
>> diff --git a/linux-user/elfload.c b/linux-user/elfload.c
>> index 01a9323a63..ebc663ea0b 100644
>> --- a/linux-user/elfload.c
>> +++ b/linux-user/elfload.c
>> @@ -2073,12 +2073,14 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
>>              exit(EXIT_FAILURE);
>>          }
>>      } else {
>> +#if HOST_LONG_BITS < TARGET_ABI_BITS
>>          if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
>>              error_report("%s: requires more virtual address space "
>>                           "than the host can provide (0x%" PRIx64 ")",
>>                           image_name, (uint64_t)guest_hiaddr - guest_base);
>>              exit(EXIT_FAILURE);
>>          }
>> +#endif
>>      }
>>  
>>      /*
>>
> 
> Philippe sent the same patch:
> 
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg699796.html

Indeed, but looking more closely, he's using slightly different
locations for the #if and #endif ... not sure what's better though...?

 Thomas

Re: [PATCH 7/7] linux-user: limit check to HOST_LONG_BITS < TARGET_ABI_BITS

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> On 27/05/2020 16.44, Laurent Vivier wrote:
>> Le 25/05/2020 à 15:18, Thomas Huth a écrit :
>>> From: Alex Bennée <alex.bennee@linaro.org>
>>>
>>> Newer clangs rightly spot that you can never exceed the full address
>>> space of 64 bit hosts with:
>>>
>>>   linux-user/elfload.c:2076:41: error: result of comparison 'unsigned
>>>   long' > 18446744073709551615 is always false
>>>   [-Werror,-Wtautological-type-limit-compare]
>>>   4685         if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
>>>   4686             ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~
>>>   4687 1 error generated.
>>>
>>> So lets limit the check to 32 bit hosts only.
>>>
>>> Fixes: ee94743034bf
>>> Reported-by: Thomas Huth <thuth@redhat.com>
>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>> [thuth: Use HOST_LONG_BITS < TARGET_ABI_BITS instead of HOST_LONG_BITS == 32]
>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>> ---
>>>  linux-user/elfload.c | 2 ++
>>>  1 file changed, 2 insertions(+)
>>>
>>> diff --git a/linux-user/elfload.c b/linux-user/elfload.c
>>> index 01a9323a63..ebc663ea0b 100644
>>> --- a/linux-user/elfload.c
>>> +++ b/linux-user/elfload.c
>>> @@ -2073,12 +2073,14 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
>>>              exit(EXIT_FAILURE);
>>>          }
>>>      } else {
>>> +#if HOST_LONG_BITS < TARGET_ABI_BITS
>>>          if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
>>>              error_report("%s: requires more virtual address space "
>>>                           "than the host can provide (0x%" PRIx64 ")",
>>>                           image_name, (uint64_t)guest_hiaddr - guest_base);
>>>              exit(EXIT_FAILURE);
>>>          }
>>> +#endif
>>>      }
>>>  
>>>      /*
>>>
>> 
>> Philippe sent the same patch:
>> 
>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg699796.html
>
> Indeed, but looking more closely, he's using slightly different
> locations for the #if and #endif ... not sure what's better though...?

Richard was more inclined to suppress the warning:

  Subject: Re: [PATCH v2] linux-user: limit check to HOST_LONG_BITS == 32
  From: Richard Henderson <richard.henderson@linaro.org>
  Message-ID: <3069bc1b-115d-f361-8271-c775bf6957ea@linaro.org>
  Date: Thu, 21 May 2020 20:15:51 -0700

One reason I dropped the f32 patch from my last PR was because this
wasn't the only warning the latest clang picks up.

-- 
Alex Bennée

Re: [PATCH 2/7] gitlab-ci: Remove flex/bison packages

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> From: Philippe Mathieu-Daudé <philmd@redhat.com>
>
> QEMU does not use flex/bison packages.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Message-Id: <20200515163029.12917-4-philmd@redhat.com>
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  .gitlab-ci.yml | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index b889fb96b6..994774250f 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -4,7 +4,7 @@ include:
>  
>  before_script:
>   - apt-get update -qq
> - - apt-get install -y -qq flex bison libglib2.0-dev libpixman-1-dev genisoimage
> + - apt-get install -y -qq libglib2.0-dev libpixman-1-dev genisoimage
>  
>  build-system1:
>   script:


-- 
Alex Bennée

Re: [PATCH 3/7] GitLab CI: avoid calling before_scripts on unintended jobs

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> From: Cleber Rosa <crosa@redhat.com>
>
> At this point it seems that all jobs depend on those steps, with
> maybe the EDK2 jobs as exceptions.
>
> The jobs that will be added later will not want those scripts to be
> run, so let's move these steps to the appropriate jobs, while
> still trying to avoid repetition.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> [thuth: Rebased to current master branch, use separate template]
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée

Re: [PATCH 4/7] gitlab-ci: Move edk2 and opensbi YAML files to .gitlab-ci.d folder

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> We have a dedicated folder for the gitlab-ci - so there is no need
> to clutter the top directory with these .yml files.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

> ---
>  .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml       | 0
>  .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml | 0
>  .gitlab-ci.yml                                     | 4 ++--
>  MAINTAINERS                                        | 2 +-
>  4 files changed, 3 insertions(+), 3 deletions(-)
>  rename .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml (100%)
>  rename .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml (100%)
>
> diff --git a/.gitlab-ci-edk2.yml b/.gitlab-ci.d/edk2.yml
> similarity index 100%
> rename from .gitlab-ci-edk2.yml
> rename to .gitlab-ci.d/edk2.yml
> diff --git a/.gitlab-ci-opensbi.yml b/.gitlab-ci.d/opensbi.yml
> similarity index 100%
> rename from .gitlab-ci-opensbi.yml
> rename to .gitlab-ci.d/opensbi.yml
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index bc6aee6aba..5208d93ff8 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -1,6 +1,6 @@
>  include:
> -  - local: '/.gitlab-ci-edk2.yml'
> -  - local: '/.gitlab-ci-opensbi.yml'
> +  - local: '/.gitlab-ci.d/edk2.yml'
> +  - local: '/.gitlab-ci.d/opensbi.yml'
>  
>  .update_apt_template: &before_script_apt
>   before_script:
> diff --git a/MAINTAINERS b/MAINTAINERS
> index bde5fd480f..d43c98115c 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2534,7 +2534,7 @@ F: roms/edk2
>  F: roms/edk2-*
>  F: tests/data/uefi-boot-images/
>  F: tests/uefi-test-tools/
> -F: .gitlab-ci-edk2.yml
> +F: .gitlab-ci.d/edk2.yml
>  F: .gitlab-ci.d/edk2/
>  
>  Usermode Emulation


-- 
Alex Bennée

Re: [PATCH 7/7] linux-user: limit check to HOST_LONG_BITS < TARGET_ABI_BITS

[Thomas Huth]

On 27/05/2020 18.36, Alex Bennée wrote:
> 
> Thomas Huth <thuth@redhat.com> writes:
> 
>> On 27/05/2020 16.44, Laurent Vivier wrote:
>>> Le 25/05/2020 à 15:18, Thomas Huth a écrit :
>>>> From: Alex Bennée <alex.bennee@linaro.org>
>>>>
>>>> Newer clangs rightly spot that you can never exceed the full address
>>>> space of 64 bit hosts with:
>>>>
>>>>   linux-user/elfload.c:2076:41: error: result of comparison 'unsigned
>>>>   long' > 18446744073709551615 is always false
>>>>   [-Werror,-Wtautological-type-limit-compare]
>>>>   4685         if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
>>>>   4686             ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~
>>>>   4687 1 error generated.
>>>>
>>>> So lets limit the check to 32 bit hosts only.
>>>>
>>>> Fixes: ee94743034bf
>>>> Reported-by: Thomas Huth <thuth@redhat.com>
>>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>>> [thuth: Use HOST_LONG_BITS < TARGET_ABI_BITS instead of HOST_LONG_BITS == 32]
>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>> ---
>>>>  linux-user/elfload.c | 2 ++
>>>>  1 file changed, 2 insertions(+)
>>>>
>>>> diff --git a/linux-user/elfload.c b/linux-user/elfload.c
>>>> index 01a9323a63..ebc663ea0b 100644
>>>> --- a/linux-user/elfload.c
>>>> +++ b/linux-user/elfload.c
>>>> @@ -2073,12 +2073,14 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
>>>>              exit(EXIT_FAILURE);
>>>>          }
>>>>      } else {
>>>> +#if HOST_LONG_BITS < TARGET_ABI_BITS
>>>>          if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
>>>>              error_report("%s: requires more virtual address space "
>>>>                           "than the host can provide (0x%" PRIx64 ")",
>>>>                           image_name, (uint64_t)guest_hiaddr - guest_base);
>>>>              exit(EXIT_FAILURE);
>>>>          }
>>>> +#endif
>>>>      }
>>>>  
>>>>      /*
>>>>
>>>
>>> Philippe sent the same patch:
>>>
>>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg699796.html
>>
>> Indeed, but looking more closely, he's using slightly different
>> locations for the #if and #endif ... not sure what's better though...?
> 
> Richard was more inclined to suppress the warning:
> 
>   Subject: Re: [PATCH v2] linux-user: limit check to HOST_LONG_BITS == 32
>   From: Richard Henderson <richard.henderson@linaro.org>
>   Message-ID: <3069bc1b-115d-f361-8271-c775bf6957ea@linaro.org>
>   Date: Thu, 21 May 2020 20:15:51 -0700
> 
> One reason I dropped the f32 patch from my last PR was because this
> wasn't the only warning the latest clang picks up.

... but this is currently the only spot that is required to get the
gitlab CI going again, so I think we should include this patch until we
have a final decision whether to disable the warning or not (and we can
still revert this patch after we disabled the warning). Ok?

 Thomas

Re: [PATCH 7/7] linux-user: limit check to HOST_LONG_BITS < TARGET_ABI_BITS

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> On 27/05/2020 18.36, Alex Bennée wrote:
>> 
>> Thomas Huth <thuth@redhat.com> writes:
>> 
>>> On 27/05/2020 16.44, Laurent Vivier wrote:
>>>> Le 25/05/2020 à 15:18, Thomas Huth a écrit :
>>>>> From: Alex Bennée <alex.bennee@linaro.org>
>>>>>
>>>>> Newer clangs rightly spot that you can never exceed the full address
>>>>> space of 64 bit hosts with:
>>>>>
>>>>>   linux-user/elfload.c:2076:41: error: result of comparison 'unsigned
>>>>>   long' > 18446744073709551615 is always false
>>>>>   [-Werror,-Wtautological-type-limit-compare]
>>>>>   4685         if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
>>>>>   4686             ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~
>>>>>   4687 1 error generated.
>>>>>
>>>>> So lets limit the check to 32 bit hosts only.
>>>>>
>>>>> Fixes: ee94743034bf
>>>>> Reported-by: Thomas Huth <thuth@redhat.com>
>>>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>>>>> [thuth: Use HOST_LONG_BITS < TARGET_ABI_BITS instead of HOST_LONG_BITS == 32]
>>>>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>>>>> ---
>>>>>  linux-user/elfload.c | 2 ++
>>>>>  1 file changed, 2 insertions(+)
>>>>>
>>>>> diff --git a/linux-user/elfload.c b/linux-user/elfload.c
>>>>> index 01a9323a63..ebc663ea0b 100644
>>>>> --- a/linux-user/elfload.c
>>>>> +++ b/linux-user/elfload.c
>>>>> @@ -2073,12 +2073,14 @@ static void pgb_have_guest_base(const char *image_name, abi_ulong guest_loaddr,
>>>>>              exit(EXIT_FAILURE);
>>>>>          }
>>>>>      } else {
>>>>> +#if HOST_LONG_BITS < TARGET_ABI_BITS
>>>>>          if ((guest_hiaddr - guest_base) > ~(uintptr_t)0) {
>>>>>              error_report("%s: requires more virtual address space "
>>>>>                           "than the host can provide (0x%" PRIx64 ")",
>>>>>                           image_name, (uint64_t)guest_hiaddr - guest_base);
>>>>>              exit(EXIT_FAILURE);
>>>>>          }
>>>>> +#endif
>>>>>      }
>>>>>  
>>>>>      /*
>>>>>
>>>>
>>>> Philippe sent the same patch:
>>>>
>>>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg699796.html
>>>
>>> Indeed, but looking more closely, he's using slightly different
>>> locations for the #if and #endif ... not sure what's better though...?
>> 
>> Richard was more inclined to suppress the warning:
>> 
>>   Subject: Re: [PATCH v2] linux-user: limit check to HOST_LONG_BITS == 32
>>   From: Richard Henderson <richard.henderson@linaro.org>
>>   Message-ID: <3069bc1b-115d-f361-8271-c775bf6957ea@linaro.org>
>>   Date: Thu, 21 May 2020 20:15:51 -0700
>> 
>> One reason I dropped the f32 patch from my last PR was because this
>> wasn't the only warning the latest clang picks up.
>
> ... but this is currently the only spot that is required to get the
> gitlab CI going again, so I think we should include this patch until we
> have a final decision whether to disable the warning or not (and we can
> still revert this patch after we disabled the warning). Ok?

I'm certainly happy with that if it gets gitlab working.

My experience with make docker-test-vlang@fedora (with 32) was there
where more things to fix. I guess gitlab didn't trigger them.

-- 
Alex Bennée

Re: [PATCH 5/7] gitlab-ci: Do not use the standard container images from gitlab

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> Currently all pipelines of the gitlab CI are failing, except for the
> "build-user" pipeline. There is an issue with the default container
> image (likely Debian stable) where they imported something bad in one
> of the system headers:
>
>  /usr/include/linux/swab.h: In function '__swab':
>  /builds/huth/qemu/include/qemu/bitops.h:20:34: error: "sizeof" is not
>   defined, evaluates to 0 [-Werror=undef]
>  #define BITS_PER_LONG           (sizeof (unsigned long) * BITS_PER_BYTE)
>
> We could maybe work-around this issue or wait for the default containers
> to get fixed, but considering that we use Ubuntu (and thus Debian-style)
> CI in Travis already to a very large extent, we should consider to use
> some RPM-based distros in our gitlab CI instead. Thus let's change the
> failing pipelines to use Fedora and CentOS (and also one Ubuntu 19.10,
> since 20.04 is broken, too) now.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>

Acked-by: Alex Bennée <alex.bennee@linaro.org>

I will say Fedora/CentOS won't be immune to this failure if they update
the kernel headers to somewhere between the breakage and the fix.

> ---
>  .gitlab-ci.yml | 37 +++++++++++++++++++++++++------------
>  1 file changed, 25 insertions(+), 12 deletions(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 5208d93ff8..559ec2ab4d 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -5,9 +5,17 @@ include:
>  .update_apt_template: &before_script_apt
>   before_script:
>    - apt-get update -qq
> -  - apt-get install -y -qq libglib2.0-dev libpixman-1-dev genisoimage
> +  - apt-get install -y -qq git gcc libglib2.0-dev libpixman-1-dev make
> +        genisoimage
> +
> +.update_dnf_template: &before_script_dnf
> + before_script:
> +  - dnf update -y
> +  - dnf install -y bzip2 diffutils gcc git genisoimage findutils glib2-devel
> +        make python3 perl-podlators perl-Test-Harness pixman-devel zlib-devel
>  
>  build-system1:
> + image: ubuntu:19.10
>   <<: *before_script_apt
>   script:
>   - apt-get install -y -qq libgtk-3-dev libvte-dev nettle-dev libcacard-dev
> @@ -21,11 +29,12 @@ build-system1:
>   - make -j2 check
>  
>  build-system2:
> - <<: *before_script_apt
> + image: fedora:latest
> + <<: *before_script_dnf
>   script:
> - - apt-get install -y -qq libsdl2-dev libgcrypt-dev libbrlapi-dev libaio-dev
> -      libfdt-dev liblzo2-dev librdmacm-dev libibverbs-dev libibumad-dev
> -      libzstd-dev
> + - yum install -y SDL2-devel libgcrypt-devel brlapi-devel libaio-devel
> +       libfdt-devel lzo-devel librdmacm-devel libibverbs-devel libibumad-devel
> +       libzstd-devel
>   - mkdir build
>   - cd build
>   - ../configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
> @@ -35,7 +44,8 @@ build-system2:
>   - make -j2 check
>  
>  build-disabled:
> - <<: *before_script_apt
> + image: fedora:latest
> + <<: *before_script_dnf
>   script:
>   - mkdir build
>   - cd build
> @@ -50,9 +60,10 @@ build-disabled:
>   - make -j2 check-qtest SPEED=slow
>  
>  build-tcg-disabled:
> - <<: *before_script_apt
> + image: centos:8
> + <<: *before_script_dnf
>   script:
> - - apt-get install -y -qq clang libgtk-3-dev libusb-dev
> + - dnf install -y clang gtk3-devel libusbx-devel libgcrypt-devel
>   - mkdir build
>   - cd build
>   - ../configure --cc=clang --enable-werror --disable-tcg --audio-drv-list=""
> @@ -79,10 +90,11 @@ build-user:
>   - make run-tcg-tests-i386-linux-user run-tcg-tests-x86_64-linux-user
>  
>  build-clang:
> - <<: *before_script_apt
> + image: fedora:latest
> + <<: *before_script_dnf
>   script:
> - - apt-get install -y -qq clang libsdl2-dev libattr1-dev libcap-ng-dev
> -      xfslibs-dev libiscsi-dev libnfs-dev libseccomp-dev gnutls-dev librbd-dev
> + - yum install -y clang SDL2-devel libattr-devel libcap-ng-devel xfsprogs-devel
> +       libiscsi-devel libnfs-devel libseccomp-devel gnutls-devel librbd-devel
>   - mkdir build
>   - cd build
>   - ../configure --cc=clang --cxx=clang++ --enable-werror
> @@ -92,7 +104,8 @@ build-clang:
>   - make -j2 check
>  
>  build-tci:
> - <<: *before_script_apt
> + image: centos:8
> + <<: *before_script_dnf
>   script:
>   - TARGETS="aarch64 alpha arm hppa m68k microblaze moxie ppc64 s390x x86_64"
>   - mkdir build


-- 
Alex Bennée

Re: [PATCH 6/7] gitlab-ci: Determine the number of jobs dynamically

[Alex Bennée]

Thomas Huth <thuth@redhat.com> writes:

> Some people might want to run the gitlab CI pipelines in an environment
> where multiple CPUs are available to the runners, so let's rather get
> the number for "-j" from the "nproc" program (increased by 1 to compensate
> for jobs that wait for I/O) instead of hard-coding it.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
<snip>
> @@ -25,8 +27,8 @@ build-system1:
>   - ../configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
>        cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
>        mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
> - - make -j2
> - - make -j2 check
> + - make -j"$JOBS"
> + - make -j"$JOBS" check
>  
>  build-system2:
>   image: fedora:latest
> @@ -40,8 +42,8 @@ build-system2:
>   - ../configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
>        microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
>        sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
> - - make -j2
> - - make -j2 check
> + - make -j"$JOBS"
> + - make -j"$JOBS" check
>  
>  build-disabled:
>   image: fedora:latest
> @@ -56,8 +58,8 @@ build-disabled:
>        --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
>        --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
>        --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
> - - make -j2
> - - make -j2 check-qtest SPEED=slow
> + - make -j"$JOBS"
> + - make -j"$JOBS" check-qtest SPEED=slow

I would make all the check jobs use a single core as it otherwise gets
hard to figure out exactly where something broke/hung.

<snip>
> @@ -100,8 +102,8 @@ build-clang:
>   - ../configure --cc=clang --cxx=clang++ --enable-werror
>        --target-list="alpha-softmmu arm-softmmu m68k-softmmu mips64-softmmu
>                       ppc-softmmu s390x-softmmu x86_64-softmmu arm-linux-user"
> - - make -j2
> - - make -j2 check
> + - make -j"$JOBS"
> + - make -j"$JOBS" check

Ditto for this check

>  
>  build-tci:
>   image: centos:8
> @@ -112,7 +114,7 @@ build-tci:
>   - cd build
>   - ../configure --enable-tcg-interpreter
>        --target-list="$(for tg in $TARGETS; do echo -n ${tg}'-softmmu '; done)"
> - - make -j2
> + - make -j"$JOBS"
>   - make run-tcg-tests-x86_64-softmmu
>   - make tests/qtest/boot-serial-test tests/qtest/cdrom-test tests/qtest/pxe-test
>   - for tg in $TARGETS ; do

Otherwise:

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée

Re: [PATCH 6/7] gitlab-ci: Determine the number of jobs dynamically

[Thomas Huth]

On 28/05/2020 10.41, Alex Bennée wrote:
> 
> Thomas Huth <thuth@redhat.com> writes:
> 
>> Some people might want to run the gitlab CI pipelines in an environment
>> where multiple CPUs are available to the runners, so let's rather get
>> the number for "-j" from the "nproc" program (increased by 1 to compensate
>> for jobs that wait for I/O) instead of hard-coding it.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
> <snip>
>> @@ -25,8 +27,8 @@ build-system1:
>>   - ../configure --enable-werror --target-list="aarch64-softmmu alpha-softmmu
>>        cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu
>>        mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu"
>> - - make -j2
>> - - make -j2 check
>> + - make -j"$JOBS"
>> + - make -j"$JOBS" check
>>  
>>  build-system2:
>>   image: fedora:latest
>> @@ -40,8 +42,8 @@ build-system2:
>>   - ../configure --enable-werror --target-list="tricore-softmmu unicore32-softmmu
>>        microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu
>>        sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu"
>> - - make -j2
>> - - make -j2 check
>> + - make -j"$JOBS"
>> + - make -j"$JOBS" check
>>  
>>  build-disabled:
>>   image: fedora:latest
>> @@ -56,8 +58,8 @@ build-disabled:
>>        --disable-qom-cast-debug --disable-spice --disable-vhost-vsock
>>        --disable-vhost-net --disable-vhost-crypto --disable-vhost-user
>>        --target-list="i386-softmmu ppc64-softmmu mips64-softmmu i386-linux-user"
>> - - make -j2
>> - - make -j2 check-qtest SPEED=slow
>> + - make -j"$JOBS"
>> + - make -j"$JOBS" check-qtest SPEED=slow
> 
> I would make all the check jobs use a single core as it otherwise gets
> hard to figure out exactly where something broke/hung.

It's a somewhat double-edged sword ... either faster CI test times, or
more deterministic output ... so far I didn't suffer the problem with
the deterministic output in the gitlab-CI yet (unlike with Travis), so
I'd rather keep the -j here for now. We can still remove it later if we
hit a bug that is hard to debug otherwise.

 Thomas