* [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile)
@ 2022-05-27 15:35 Alex Bennée
2022-05-27 15:35 ` [PATCH v1 01/33] .gitlab-ci.d/container-cross: Fix RISC-V container dependencies / stages Alex Bennée
` (32 more replies)
0 siblings, 33 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée
Hi,
After a delay caused with other priorities I've finally managed to
catch up with some of my maintainer duties. The result is the current
testing/next branch which contains:
- some GitLab fixes from Thomas
- exposing JUnit to gitlab from Marc-André
- more lcitool docker conversions from me
- sharing the testing cross compilers with rom builds from Paolo
- disable testing on forks by default from Daniel
the last one is important with the upcoming rationing of CI minutes
as well as hopefully avoiding too much wasteful testing while
developing. See the doc tips about setting up aliases to make it easy
to trigger a CI build with a push.
So far it all seems to be hanging together fairly well. I'll probably
look to cut a PR from this next week if the soak testing doesn't throw
up anything else.
My patches could do with someone casting an eye over them as they are
un-reviewed and written on a Friday afternoon ;-)
Alex Bennée (9):
meson.build: fix summary display of test compilers
tests/lcitool: fix up indentation to correct style
tests/docker: update debian-armhf-cross with lcitool
tests/docker: update debian-armel-cross with lcitool
tests/docker: update debian-mipsel-cross with lcitool
tests/docker: update debian-mips64el-cross with lcitool
tests/docker: update debian-ppc64el-cross with lcitool
tests/docker: update debian-amd64 with lcitool
docs/devel: clean-up the CI links in the docs
Daniel P. Berrangé (5):
gitlab: introduce a common base job template
gitlab: convert Cirrus jobs to .base_job_template
gitlab: convert static checks to .base_job_template
gitlab: convert build/container jobs to .base_job_template
gitlab: don't run CI jobs in forks by default
Marc-André Lureau (1):
gitlab-ci: add meson JUnit test result into report
Paolo Bonzini (16):
configure: do not define or use the CPP variable
build: clean up ninja invocation
build: add a more generic way to specify make->ninja dependencies
build: do a full build before running TCG tests
configure, meson: move symlinking of ROMs to meson
tests/tcg: correct target CPU for sparc32
tests/tcg: merge configure.sh back into main configure script
configure: add missing cross compiler fallbacks
configure: handle host compiler in probe_target_compiler
configure: introduce --cross-prefix-*=
configure: include more binutils in tests/tcg makefile
configure: move symlink configuration earlier
configure: enable cross-compilation of s390-ccw
configure: enable cross-compilation of optionrom
configure: enable cross compilation of vof
configure: remove unused variables from config-host.mak
Thomas Huth (2):
.gitlab-ci.d/container-cross: Fix RISC-V container dependencies /
stages
.gitlab-ci.d/crossbuilds: Fix the dependency of the cross-i386-tci job
docs/devel/ci-jobs.rst.inc | 116 +++-
docs/devel/ci.rst | 11 +-
docs/devel/submitting-a-patch.rst | 36 +-
docs/devel/testing.rst | 2 +
configure | 605 +++++++++++++++---
Makefile | 9 +-
pc-bios/s390-ccw/netboot.mak | 2 +-
meson.build | 8 +-
.gitlab-ci.d/base.yml | 72 +++
.gitlab-ci.d/buildtest-template.yml | 18 +-
.gitlab-ci.d/buildtest.yml | 28 +-
.gitlab-ci.d/cirrus.yml | 16 +-
.gitlab-ci.d/container-cross.yml | 24 +-
.gitlab-ci.d/container-template.yml | 1 +
.gitlab-ci.d/containers.yml | 3 +-
.gitlab-ci.d/crossbuild-template.yml | 3 +
.gitlab-ci.d/crossbuilds.yml | 2 +
.gitlab-ci.d/qemu-project.yml | 1 +
.gitlab-ci.d/static_checks.yml | 19 +-
.gitlab-ci.d/windows.yml | 1 +
pc-bios/meson.build | 17 +-
pc-bios/optionrom/Makefile | 4 +-
pc-bios/s390-ccw/Makefile | 9 +-
pc-bios/vof/Makefile | 17 +-
scripts/mtest2make.py | 8 +-
tests/Makefile.include | 4 +-
tests/docker/Makefile.include | 5 -
tests/docker/dockerfiles/debian-amd64.docker | 194 ++++--
.../dockerfiles/debian-armel-cross.docker | 178 +++++-
.../dockerfiles/debian-armhf-cross.docker | 184 +++++-
.../dockerfiles/debian-mips64el-cross.docker | 177 ++++-
.../dockerfiles/debian-mipsel-cross.docker | 179 +++++-
.../dockerfiles/debian-ppc64el-cross.docker | 178 +++++-
tests/lcitool/refresh | 178 ++++--
tests/tcg/configure.sh | 376 -----------
35 files changed, 1884 insertions(+), 801 deletions(-)
create mode 100644 .gitlab-ci.d/base.yml
delete mode 100755 tests/tcg/configure.sh
--
2.30.2
^ permalink raw reply [flat|nested] 55+ messages in thread
* [PATCH v1 01/33] .gitlab-ci.d/container-cross: Fix RISC-V container dependencies / stages
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-31 0:18 ` Alistair Francis
2022-05-27 15:35 ` [PATCH v1 02/33] .gitlab-ci.d/crossbuilds: Fix the dependency of the cross-i386-tci job Alex Bennée
` (31 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Thomas Huth, Alex Bennée, Wainer dos Santos Moschetta,
Beraldo Leal
From: Thomas Huth <thuth@redhat.com>
The "riscv64-debian-cross-container" job does not depend on any other
container job from the first stage, so we can move it to the first
stage, too.
The "riscv64-debian-test-cross-container" job needs the debian11
container, so we should add a proper "needs:" statement here.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220524093141.91012-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
.gitlab-ci.d/container-cross.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index e622ac2d21..ac15fce9b6 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -125,7 +125,7 @@ ppc64el-debian-cross-container:
riscv64-debian-cross-container:
extends: .container_job_template
- stage: containers-layer2
+ stage: containers
# as we are currently based on 'sid/unstable' we may break so...
allow_failure: true
variables:
@@ -135,6 +135,7 @@ riscv64-debian-cross-container:
riscv64-debian-test-cross-container:
extends: .container_job_template
stage: containers-layer2
+ needs: ['amd64-debian11-container']
variables:
NAME: debian-riscv64-test-cross
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 02/33] .gitlab-ci.d/crossbuilds: Fix the dependency of the cross-i386-tci job
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
2022-05-27 15:35 ` [PATCH v1 01/33] .gitlab-ci.d/container-cross: Fix RISC-V container dependencies / stages Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 03/33] gitlab-ci: add meson JUnit test result into report Alex Bennée
` (30 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Thomas Huth, Richard Henderson, Alex Bennée,
Wainer dos Santos Moschetta, Beraldo Leal
From: Thomas Huth <thuth@redhat.com>
The cross-i386-tci job uses the fedora-i386-cross image, so we should make sure
that the corresponding job that builds it (the i386-fedora-cross-container job)
has finished before we start the TCI job.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220524092600.89997-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
.gitlab-ci.d/crossbuilds.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index 17d6cb3e45..4a5fb6ea2a 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -62,6 +62,8 @@ cross-i386-user:
cross-i386-tci:
extends: .cross_accel_build_job
timeout: 60m
+ needs:
+ job: i386-fedora-cross-container
variables:
IMAGE: fedora-i386-cross
ACCEL: tcg-interpreter
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 03/33] gitlab-ci: add meson JUnit test result into report
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
2022-05-27 15:35 ` [PATCH v1 01/33] .gitlab-ci.d/container-cross: Fix RISC-V container dependencies / stages Alex Bennée
2022-05-27 15:35 ` [PATCH v1 02/33] .gitlab-ci.d/crossbuilds: Fix the dependency of the cross-i386-tci job Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-30 9:03 ` Thomas Huth
2022-05-27 15:35 ` [PATCH v1 04/33] meson.build: fix summary display of test compilers Alex Bennée
` (29 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Marc-André Lureau, Alex Bennée, Thomas Huth,
Wainer dos Santos Moschetta, Beraldo Leal
From: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220525173411.612224-1-marcandre.lureau@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
.gitlab-ci.d/buildtest-template.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index dc6d67aacf..b381345dbc 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -44,6 +44,8 @@
expire_in: 7 days
paths:
- build/meson-logs/testlog.txt
+ reports:
+ junit: build/meson-logs/testlog.junit.xml
.avocado_test_job_template:
extends: .common_test_job_template
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 04/33] meson.build: fix summary display of test compilers
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (2 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 03/33] gitlab-ci: add meson JUnit test result into report Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 17:19 ` Richard Henderson
2022-05-27 15:35 ` [PATCH v1 05/33] tests/lcitool: fix up indentation to correct style Alex Bennée
` (28 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée
The recent refactoring of configure.sh dropped a number of variables
we relied on for printing out information. Make it simpler.
Fixes: eebf199c09 (tests/tcg: invoke Makefile.target directly from QEMU's makefile)
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
meson.build | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/meson.build b/meson.build
index df7c34b076..b622f37a34 100644
--- a/meson.build
+++ b/meson.build
@@ -3732,12 +3732,8 @@ foreach target: target_dirs
config_cross_tcg = keyval.load(tcg_mak)
target = config_cross_tcg['TARGET_NAME']
compiler = ''
- if 'DOCKER_CROSS_CC_GUEST' in config_cross_tcg
- summary_info += {target + ' tests': config_cross_tcg['DOCKER_CROSS_CC_GUEST'] +
- ' via ' + config_cross_tcg['DOCKER_IMAGE']}
- elif 'CROSS_CC_GUEST' in config_cross_tcg
- summary_info += {target + ' tests'
- : config_cross_tcg['CROSS_CC_GUEST'] }
+ if 'CC' in config_cross_tcg
+ summary_info += {target + ' tests': config_cross_tcg['CC']}
endif
endif
endforeach
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 05/33] tests/lcitool: fix up indentation to correct style
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (3 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 04/33] meson.build: fix summary display of test compilers Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:43 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 06/33] tests/docker: update debian-armhf-cross with lcitool Alex Bennée
` (27 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, Thomas Huth, Wainer dos Santos Moschetta,
Beraldo Leal
3 space indentation snuck into the initial commit. Clean it up before
we let it get established. I've also:
- removed unused os import
- added double lines between functions
- added some comments and grouped and sorted the generation stanzas
My lint tool is also recommending using f-strings but that requires
python 3.6.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Daniel P. Berrangé <berrange@redhat.com>
---
tests/lcitool/refresh | 134 ++++++++++++++++++++++++------------------
1 file changed, 76 insertions(+), 58 deletions(-)
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index fb49bbc441..dc1fc21ef9 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -13,14 +13,13 @@
# the top-level directory.
import sys
-import os
import subprocess
from pathlib import Path
if len(sys.argv) != 1:
- print("syntax: %s" % sys.argv[0], file=sys.stderr)
- sys.exit(1)
+ print("syntax: %s" % sys.argv[0], file=sys.stderr)
+ sys.exit(1)
self_dir = Path(__file__).parent
src_dir = self_dir.parent.parent
@@ -30,76 +29,95 @@ lcitool_path = Path(self_dir, "libvirt-ci", "lcitool")
lcitool_cmd = [lcitool_path, "--data-dir", self_dir]
+
def atomic_write(filename, content):
- tmp = filename.with_suffix(filename.suffix + ".tmp")
- try:
- with tmp.open("w") as fp:
- print(content, file=fp, end="")
- tmp.rename(filename)
- except Exception as ex:
- tmp.unlink()
- raise
+ tmp = filename.with_suffix(filename.suffix + ".tmp")
+ try:
+ with tmp.open("w") as fp:
+ print(content, file=fp, end="")
+ tmp.rename(filename)
+ except Exception as ex:
+ tmp.unlink()
+ raise
+
def generate(filename, cmd, trailer):
- print("Generate %s" % filename)
- lcitool=subprocess.run(cmd, capture_output=True)
+ print("Generate %s" % filename)
+ lcitool = subprocess.run(cmd, capture_output=True)
- if lcitool.returncode != 0:
- raise Exception("Failed to generate %s: %s" % (filename, lcitool.stderr))
+ if lcitool.returncode != 0:
+ raise Exception("Failed to generate %s: %s" % (filename, lcitool.stderr))
+
+ content = lcitool.stdout.decode("utf8")
+ if trailer is not None:
+ content += trailer
+ atomic_write(filename, content)
- content = lcitool.stdout.decode("utf8")
- if trailer is not None:
- content += trailer
- atomic_write(filename, content)
def generate_dockerfile(host, target, cross=None, trailer=None):
- filename = Path(src_dir, "tests", "docker", "dockerfiles", host + ".docker")
- cmd = lcitool_cmd + ["dockerfile"]
- if cross is not None:
- cmd.extend(["--cross", cross])
- cmd.extend([target, "qemu"])
- generate(filename, cmd, trailer)
+ filename = Path(src_dir, "tests", "docker", "dockerfiles", host + ".docker")
+ cmd = lcitool_cmd + ["dockerfile"]
+ if cross is not None:
+ cmd.extend(["--cross", cross])
+ cmd.extend([target, "qemu"])
+ generate(filename, cmd, trailer)
+
def generate_cirrus(target, trailer=None):
- filename = Path(src_dir, ".gitlab-ci.d", "cirrus", target + ".vars")
- cmd = lcitool_cmd + ["variables", target, "qemu"]
- generate(filename, cmd, trailer)
+ filename = Path(src_dir, ".gitlab-ci.d", "cirrus", target + ".vars")
+ cmd = lcitool_cmd + ["variables", target, "qemu"]
+ generate(filename, cmd, trailer)
+
ubuntu2004_tsanhack = [
- "# Apply patch https://reviews.llvm.org/D75820\n",
- "# This is required for TSan in clang-10 to compile with QEMU.\n",
- "RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h\n"
+ "# Apply patch https://reviews.llvm.org/D75820\n",
+ "# This is required for TSan in clang-10 to compile with QEMU.\n",
+ "RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h\n"
]
+
def debian_cross_build(prefix, targets):
- conf = "ENV QEMU_CONFIGURE_OPTS --cross-prefix=%s\n" % (prefix)
- targets = "ENV DEF_TARGET_LIST %s\n" % (targets)
- return "".join([conf, targets])
+ conf = "ENV QEMU_CONFIGURE_OPTS --cross-prefix=%s\n" % (prefix)
+ targets = "ENV DEF_TARGET_LIST %s\n" % (targets)
+ return "".join([conf, targets])
+#
+# Update all the various build configurations.
+# Please keep each group sorted alphabetically for easy reading.
+#
try:
- generate_dockerfile("centos8", "centos-stream-8")
- generate_dockerfile("fedora", "fedora-35")
- generate_dockerfile("ubuntu2004", "ubuntu-2004",
- trailer="".join(ubuntu2004_tsanhack))
- generate_dockerfile("opensuse-leap", "opensuse-leap-152")
- generate_dockerfile("alpine", "alpine-edge")
-
- generate_dockerfile("debian-arm64-cross", "debian-11",
- cross="aarch64",
- trailer=debian_cross_build("aarch64-linux-gnu-",
- "aarch64-softmmu,aarch64-linux-user"))
-
- generate_dockerfile("debian-s390x-cross", "debian-11",
- cross="s390x",
- trailer=debian_cross_build("s390x-linux-gnu-",
- "s390x-softmmu,s390x-linux-user"))
-
- generate_cirrus("freebsd-12")
- generate_cirrus("freebsd-13")
- generate_cirrus("macos-11")
-
- sys.exit(0)
+ #
+ # Standard native builds
+ #
+ generate_dockerfile("alpine", "alpine-edge")
+ generate_dockerfile("centos8", "centos-stream-8")
+ generate_dockerfile("fedora", "fedora-35")
+ generate_dockerfile("opensuse-leap", "opensuse-leap-152")
+ generate_dockerfile("ubuntu2004", "ubuntu-2004",
+ trailer="".join(ubuntu2004_tsanhack))
+
+ #
+ # Cross compiling builds
+ #
+ generate_dockerfile("debian-arm64-cross", "debian-11",
+ cross="aarch64",
+ trailer=debian_cross_build("aarch64-linux-gnu-",
+ "aarch64-softmmu,aarch64-linux-user"))
+
+ generate_dockerfile("debian-s390x-cross", "debian-11",
+ cross="s390x",
+ trailer=debian_cross_build("s390x-linux-gnu-",
+ "s390x-softmmu,s390x-linux-user"))
+
+ #
+ # Cirrus packages lists for GitLab
+ #
+ generate_cirrus("freebsd-12")
+ generate_cirrus("freebsd-13")
+ generate_cirrus("macos-11")
+
+ sys.exit(0)
except Exception as ex:
- print(str(ex), file=sys.stderr)
- sys.exit(1)
+ print(str(ex), file=sys.stderr)
+ sys.exit(1)
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 06/33] tests/docker: update debian-armhf-cross with lcitool
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (4 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 05/33] tests/lcitool: fix up indentation to correct style Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:44 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 07/33] tests/docker: update debian-armel-cross " Alex Bennée
` (26 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, Thomas Huth, Wainer dos Santos Moschetta,
Beraldo Leal
Use lcitool to update debian-armhf-cross to a Debian 11 based system.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
.gitlab-ci.d/container-cross.yml | 3 +-
tests/docker/Makefile.include | 1 -
.../dockerfiles/debian-armhf-cross.docker | 184 +++++++++++++++---
tests/lcitool/refresh | 5 +
4 files changed, 166 insertions(+), 27 deletions(-)
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index ac15fce9b6..4d1830f3fc 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -34,8 +34,7 @@ armel-debian-cross-container:
armhf-debian-cross-container:
extends: .container_job_template
- stage: containers-layer2
- needs: ['amd64-debian10-container']
+ stage: containers
variables:
NAME: debian-armhf-cross
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index ca2157db46..d6e0710554 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -90,7 +90,6 @@ endif
docker-image-debian-alpha-cross: docker-image-debian10
docker-image-debian-armel-cross: docker-image-debian10
-docker-image-debian-armhf-cross: docker-image-debian10
docker-image-debian-hppa-cross: docker-image-debian10
docker-image-debian-m68k-cross: docker-image-debian10
docker-image-debian-mips-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-armhf-cross.docker b/tests/docker/dockerfiles/debian-armhf-cross.docker
index 25d7618833..a2ebce96f8 100644
--- a/tests/docker/dockerfiles/debian-armhf-cross.docker
+++ b/tests/docker/dockerfiles/debian-armhf-cross.docker
@@ -1,29 +1,165 @@
+# THIS FILE WAS AUTO-GENERATED
#
-# Docker armhf cross-compiler target
+# $ lcitool dockerfile --layers all --cross armv7l debian-11 qemu
#
-# This docker target builds on the debian Stretch base image.
-#
-FROM qemu/debian10
+# https://gitlab.com/libvirt/libvirt-ci
-# Add the foreign architecture we want and install dependencies
-RUN dpkg --add-architecture armhf
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt install -y --no-install-recommends \
- crossbuild-essential-armhf
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt build-dep -yy -a armhf --arch-only qemu
+FROM docker.io/library/debian:11-slim
-# Specify the cross prefix for this image (see tests/docker/common.rc)
-ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabihf-
-ENV DEF_TARGET_LIST arm-softmmu,arm-linux-user,armeb-linux-user
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ apt-get update && \
+ apt-get install -y eatmydata && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y \
+ bash \
+ bc \
+ bsdextrautils \
+ bzip2 \
+ ca-certificates \
+ ccache \
+ dbus \
+ debianutils \
+ diffutils \
+ exuberant-ctags \
+ findutils \
+ gcovr \
+ genisoimage \
+ gettext \
+ git \
+ hostname \
+ libpcre2-dev \
+ libspice-protocol-dev \
+ llvm \
+ locales \
+ make \
+ meson \
+ ncat \
+ ninja-build \
+ openssh-client \
+ perl-base \
+ pkgconf \
+ python3 \
+ python3-numpy \
+ python3-opencv \
+ python3-pillow \
+ python3-pip \
+ python3-sphinx \
+ python3-sphinx-rtd-theme \
+ python3-venv \
+ python3-yaml \
+ rpm2cpio \
+ sed \
+ sparse \
+ tar \
+ tesseract-ocr \
+ tesseract-ocr-eng \
+ texinfo && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+ dpkg-reconfigure locales
+
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt install -y --no-install-recommends \
- libbz2-dev:armhf \
- liblzo2-dev:armhf \
- librdmacm-dev:armhf \
- libsnappy-dev:armhf \
- libxen-dev:armhf
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ dpkg --add-architecture armhf && \
+ eatmydata apt-get update && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y dpkg-dev && \
+ eatmydata apt-get install --no-install-recommends -y \
+ g++-arm-linux-gnueabihf \
+ gcc-arm-linux-gnueabihf \
+ libaio-dev:armhf \
+ libasan5:armhf \
+ libasound2-dev:armhf \
+ libattr1-dev:armhf \
+ libbpf-dev:armhf \
+ libbrlapi-dev:armhf \
+ libbz2-dev:armhf \
+ libc6-dev:armhf \
+ libcacard-dev:armhf \
+ libcap-ng-dev:armhf \
+ libcapstone-dev:armhf \
+ libcurl4-gnutls-dev:armhf \
+ libdaxctl-dev:armhf \
+ libdrm-dev:armhf \
+ libepoxy-dev:armhf \
+ libfdt-dev:armhf \
+ libffi-dev:armhf \
+ libfuse3-dev:armhf \
+ libgbm-dev:armhf \
+ libgcrypt20-dev:armhf \
+ libglib2.0-dev:armhf \
+ libglusterfs-dev:armhf \
+ libgnutls28-dev:armhf \
+ libgtk-3-dev:armhf \
+ libibumad-dev:armhf \
+ libibverbs-dev:armhf \
+ libiscsi-dev:armhf \
+ libjemalloc-dev:armhf \
+ libjpeg62-turbo-dev:armhf \
+ liblttng-ust-dev:armhf \
+ liblzo2-dev:armhf \
+ libncursesw5-dev:armhf \
+ libnfs-dev:armhf \
+ libnuma-dev:armhf \
+ libpam0g-dev:armhf \
+ libpixman-1-dev:armhf \
+ libpng-dev:armhf \
+ libpulse-dev:armhf \
+ librbd-dev:armhf \
+ librdmacm-dev:armhf \
+ libsasl2-dev:armhf \
+ libsdl2-dev:armhf \
+ libsdl2-image-dev:armhf \
+ libseccomp-dev:armhf \
+ libselinux1-dev:armhf \
+ libslirp-dev:armhf \
+ libsnappy-dev:armhf \
+ libspice-server-dev:armhf \
+ libssh-gcrypt-dev:armhf \
+ libsystemd-dev:armhf \
+ libtasn1-6-dev:armhf \
+ libubsan1:armhf \
+ libudev-dev:armhf \
+ liburing-dev:armhf \
+ libusb-1.0-0-dev:armhf \
+ libusbredirhost-dev:armhf \
+ libvdeplug-dev:armhf \
+ libvirglrenderer-dev:armhf \
+ libvte-2.91-dev:armhf \
+ libxen-dev:armhf \
+ libzstd-dev:armhf \
+ nettle-dev:armhf \
+ systemtap-sdt-dev:armhf \
+ xfslibs-dev:armhf \
+ zlib1g-dev:armhf && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ mkdir -p /usr/local/share/meson/cross && \
+ echo "[binaries]\n\
+c = '/usr/bin/arm-linux-gnueabihf-gcc'\n\
+ar = '/usr/bin/arm-linux-gnueabihf-gcc-ar'\n\
+strip = '/usr/bin/arm-linux-gnueabihf-strip'\n\
+pkgconfig = '/usr/bin/arm-linux-gnueabihf-pkg-config'\n\
+\n\
+[host_machine]\n\
+system = 'linux'\n\
+cpu_family = 'arm'\n\
+cpu = 'armhf'\n\
+endian = 'little'" > /usr/local/share/meson/cross/arm-linux-gnueabihf && \
+ dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-c++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-g++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabihf-gcc
+
+ENV ABI "arm-linux-gnueabihf"
+ENV MESON_OPTS "--cross-file=arm-linux-gnueabihf"
+ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabihf-
+ENV DEF_TARGET_LIST arm-softmmu,arm-linux-user
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index dc1fc21ef9..7e4a92f630 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -105,6 +105,11 @@ try:
trailer=debian_cross_build("aarch64-linux-gnu-",
"aarch64-softmmu,aarch64-linux-user"))
+ generate_dockerfile("debian-armhf-cross", "debian-11",
+ cross="armv7l",
+ trailer=debian_cross_build("arm-linux-gnueabihf-",
+ "arm-softmmu,arm-linux-user"))
+
generate_dockerfile("debian-s390x-cross", "debian-11",
cross="s390x",
trailer=debian_cross_build("s390x-linux-gnu-",
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 07/33] tests/docker: update debian-armel-cross with lcitool
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (5 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 06/33] tests/docker: update debian-armhf-cross with lcitool Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:45 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 08/33] tests/docker: update debian-mipsel-cross " Alex Bennée
` (25 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, Thomas Huth, Wainer dos Santos Moschetta,
Beraldo Leal
Use lcitool to update debian-armel-cross to a Debian 11 based system.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
.gitlab-ci.d/container-cross.yml | 3 +-
tests/docker/Makefile.include | 1 -
.../dockerfiles/debian-armel-cross.docker | 178 ++++++++++++++++--
tests/lcitool/refresh | 5 +
4 files changed, 164 insertions(+), 23 deletions(-)
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 4d1830f3fc..caef7decf4 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -27,8 +27,7 @@ arm64-debian-cross-container:
armel-debian-cross-container:
extends: .container_job_template
- stage: containers-layer2
- needs: ['amd64-debian10-container']
+ stage: containers
variables:
NAME: debian-armel-cross
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index d6e0710554..d9109bcc77 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -89,7 +89,6 @@ DOCKER_PARTIAL_IMAGES += fedora
endif
docker-image-debian-alpha-cross: docker-image-debian10
-docker-image-debian-armel-cross: docker-image-debian10
docker-image-debian-hppa-cross: docker-image-debian10
docker-image-debian-m68k-cross: docker-image-debian10
docker-image-debian-mips-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-armel-cross.docker b/tests/docker/dockerfiles/debian-armel-cross.docker
index b7b1a3585f..a6153e5a83 100644
--- a/tests/docker/dockerfiles/debian-armel-cross.docker
+++ b/tests/docker/dockerfiles/debian-armel-cross.docker
@@ -1,26 +1,164 @@
+# THIS FILE WAS AUTO-GENERATED
#
-# Docker armel cross-compiler target
+# $ lcitool dockerfile --layers all --cross armv6l debian-11 qemu
#
-# This docker target builds on the debian Stretch base image.
-#
-FROM qemu/debian10
-MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
+# https://gitlab.com/libvirt/libvirt-ci
+
+FROM docker.io/library/debian:11-slim
+
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ apt-get update && \
+ apt-get install -y eatmydata && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y \
+ bash \
+ bc \
+ bsdextrautils \
+ bzip2 \
+ ca-certificates \
+ ccache \
+ dbus \
+ debianutils \
+ diffutils \
+ exuberant-ctags \
+ findutils \
+ gcovr \
+ genisoimage \
+ gettext \
+ git \
+ hostname \
+ libpcre2-dev \
+ libspice-protocol-dev \
+ llvm \
+ locales \
+ make \
+ meson \
+ ncat \
+ ninja-build \
+ openssh-client \
+ perl-base \
+ pkgconf \
+ python3 \
+ python3-numpy \
+ python3-opencv \
+ python3-pillow \
+ python3-pip \
+ python3-sphinx \
+ python3-sphinx-rtd-theme \
+ python3-venv \
+ python3-yaml \
+ rpm2cpio \
+ sed \
+ sparse \
+ tar \
+ tesseract-ocr \
+ tesseract-ocr-eng \
+ texinfo && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+ dpkg-reconfigure locales
-# Add the foreign architecture we want and install dependencies
-RUN dpkg --add-architecture armel && \
- apt update && \
- apt install -yy crossbuild-essential-armel && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt build-dep -yy -a armel --arch-only qemu
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-# Specify the cross prefix for this image (see tests/docker/common.rc)
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ dpkg --add-architecture armel && \
+ eatmydata apt-get update && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y dpkg-dev && \
+ eatmydata apt-get install --no-install-recommends -y \
+ g++-arm-linux-gnueabi \
+ gcc-arm-linux-gnueabi \
+ libaio-dev:armel \
+ libasan5:armel \
+ libasound2-dev:armel \
+ libattr1-dev:armel \
+ libbpf-dev:armel \
+ libbrlapi-dev:armel \
+ libbz2-dev:armel \
+ libc6-dev:armel \
+ libcacard-dev:armel \
+ libcap-ng-dev:armel \
+ libcapstone-dev:armel \
+ libcurl4-gnutls-dev:armel \
+ libdaxctl-dev:armel \
+ libdrm-dev:armel \
+ libepoxy-dev:armel \
+ libfdt-dev:armel \
+ libffi-dev:armel \
+ libfuse3-dev:armel \
+ libgbm-dev:armel \
+ libgcrypt20-dev:armel \
+ libglib2.0-dev:armel \
+ libglusterfs-dev:armel \
+ libgnutls28-dev:armel \
+ libgtk-3-dev:armel \
+ libibumad-dev:armel \
+ libibverbs-dev:armel \
+ libiscsi-dev:armel \
+ libjemalloc-dev:armel \
+ libjpeg62-turbo-dev:armel \
+ liblttng-ust-dev:armel \
+ liblzo2-dev:armel \
+ libncursesw5-dev:armel \
+ libnfs-dev:armel \
+ libnuma-dev:armel \
+ libpam0g-dev:armel \
+ libpixman-1-dev:armel \
+ libpng-dev:armel \
+ libpulse-dev:armel \
+ librbd-dev:armel \
+ librdmacm-dev:armel \
+ libsasl2-dev:armel \
+ libsdl2-dev:armel \
+ libsdl2-image-dev:armel \
+ libseccomp-dev:armel \
+ libselinux1-dev:armel \
+ libslirp-dev:armel \
+ libsnappy-dev:armel \
+ libspice-server-dev:armel \
+ libssh-gcrypt-dev:armel \
+ libsystemd-dev:armel \
+ libtasn1-6-dev:armel \
+ libubsan1:armel \
+ libudev-dev:armel \
+ liburing-dev:armel \
+ libusb-1.0-0-dev:armel \
+ libusbredirhost-dev:armel \
+ libvdeplug-dev:armel \
+ libvirglrenderer-dev:armel \
+ libvte-2.91-dev:armel \
+ libzstd-dev:armel \
+ nettle-dev:armel \
+ systemtap-sdt-dev:armel \
+ xfslibs-dev:armel \
+ zlib1g-dev:armel && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ mkdir -p /usr/local/share/meson/cross && \
+ echo "[binaries]\n\
+c = '/usr/bin/arm-linux-gnueabi-gcc'\n\
+ar = '/usr/bin/arm-linux-gnueabi-gcc-ar'\n\
+strip = '/usr/bin/arm-linux-gnueabi-strip'\n\
+pkgconfig = '/usr/bin/arm-linux-gnueabi-pkg-config'\n\
+\n\
+[host_machine]\n\
+system = 'linux'\n\
+cpu_family = 'arm'\n\
+cpu = 'arm'\n\
+endian = 'little'" > /usr/local/share/meson/cross/arm-linux-gnueabi && \
+ dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-c++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-g++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/arm-linux-gnueabi-gcc
+
+ENV ABI "arm-linux-gnueabi"
+ENV MESON_OPTS "--cross-file=arm-linux-gnueabi"
ENV QEMU_CONFIGURE_OPTS --cross-prefix=arm-linux-gnueabi-
ENV DEF_TARGET_LIST arm-softmmu,arm-linux-user,armeb-linux-user
-
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt install -y --no-install-recommends \
- libbz2-dev:armel \
- liblzo2-dev:armel \
- librdmacm-dev:armel \
- libsnappy-dev:armel
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 7e4a92f630..4dc5527234 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -105,6 +105,11 @@ try:
trailer=debian_cross_build("aarch64-linux-gnu-",
"aarch64-softmmu,aarch64-linux-user"))
+ generate_dockerfile("debian-armel-cross", "debian-11",
+ cross="armv6l",
+ trailer=debian_cross_build("arm-linux-gnueabi-",
+ "arm-softmmu,arm-linux-user,armeb-linux-user"))
+
generate_dockerfile("debian-armhf-cross", "debian-11",
cross="armv7l",
trailer=debian_cross_build("arm-linux-gnueabihf-",
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 08/33] tests/docker: update debian-mipsel-cross with lcitool
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (6 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 07/33] tests/docker: update debian-armel-cross " Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:45 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 09/33] tests/docker: update debian-mips64el-cross " Alex Bennée
` (24 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, Thomas Huth, Wainer dos Santos Moschetta,
Beraldo Leal, Jiaxun Yang
Use lcitool to update debian-mipsel-cross to a Debian 11 based system.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
.gitlab-ci.d/container-cross.yml | 3 +-
tests/docker/Makefile.include | 1 -
.../dockerfiles/debian-mipsel-cross.docker | 179 +++++++++++++++---
tests/lcitool/refresh | 5 +
4 files changed, 161 insertions(+), 27 deletions(-)
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index caef7decf4..1a533e6fc0 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -102,8 +102,7 @@ mips-debian-cross-container:
mipsel-debian-cross-container:
extends: .container_job_template
- stage: containers-layer2
- needs: ['amd64-debian10-container']
+ stage: containers
variables:
NAME: debian-mipsel-cross
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index d9109bcc77..0ac5975419 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -94,7 +94,6 @@ docker-image-debian-m68k-cross: docker-image-debian10
docker-image-debian-mips-cross: docker-image-debian10
docker-image-debian-mips64-cross: docker-image-debian10
docker-image-debian-mips64el-cross: docker-image-debian10
-docker-image-debian-mipsel-cross: docker-image-debian10
docker-image-debian-ppc64el-cross: docker-image-debian10
docker-image-debian-sh4-cross: docker-image-debian10
docker-image-debian-sparc64-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-mipsel-cross.docker b/tests/docker/dockerfiles/debian-mipsel-cross.docker
index 0e5dd42d3c..b6d99ae324 100644
--- a/tests/docker/dockerfiles/debian-mipsel-cross.docker
+++ b/tests/docker/dockerfiles/debian-mipsel-cross.docker
@@ -1,31 +1,162 @@
+# THIS FILE WAS AUTO-GENERATED
#
-# Docker mipsel cross-compiler target
+# $ lcitool dockerfile --layers all --cross mipsel debian-11 qemu
#
-# This docker target builds on the debian Stretch base image.
-#
-FROM qemu/debian10
+# https://gitlab.com/libvirt/libvirt-ci
-MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
+FROM docker.io/library/debian:11-slim
-# Add the foreign architecture we want and install dependencies
-RUN dpkg --add-architecture mipsel
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt install -y --no-install-recommends \
- gcc-mipsel-linux-gnu
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ apt-get update && \
+ apt-get install -y eatmydata && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y \
+ bash \
+ bc \
+ bsdextrautils \
+ bzip2 \
+ ca-certificates \
+ ccache \
+ dbus \
+ debianutils \
+ diffutils \
+ exuberant-ctags \
+ findutils \
+ gcovr \
+ genisoimage \
+ gettext \
+ git \
+ hostname \
+ libpcre2-dev \
+ libspice-protocol-dev \
+ llvm \
+ locales \
+ make \
+ meson \
+ ncat \
+ ninja-build \
+ openssh-client \
+ perl-base \
+ pkgconf \
+ python3 \
+ python3-numpy \
+ python3-opencv \
+ python3-pillow \
+ python3-pip \
+ python3-sphinx \
+ python3-sphinx-rtd-theme \
+ python3-venv \
+ python3-yaml \
+ rpm2cpio \
+ sed \
+ sparse \
+ tar \
+ tesseract-ocr \
+ tesseract-ocr-eng \
+ texinfo && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+ dpkg-reconfigure locales
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt build-dep -yy -a mipsel --arch-only qemu
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-# Specify the cross prefix for this image (see tests/docker/common.rc)
-ENV QEMU_CONFIGURE_OPTS --cross-prefix=mipsel-linux-gnu-
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ dpkg --add-architecture mipsel && \
+ eatmydata apt-get update && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y dpkg-dev && \
+ eatmydata apt-get install --no-install-recommends -y \
+ g++-mipsel-linux-gnu \
+ gcc-mipsel-linux-gnu \
+ libaio-dev:mipsel \
+ libasound2-dev:mipsel \
+ libattr1-dev:mipsel \
+ libbpf-dev:mipsel \
+ libbrlapi-dev:mipsel \
+ libbz2-dev:mipsel \
+ libc6-dev:mipsel \
+ libcacard-dev:mipsel \
+ libcap-ng-dev:mipsel \
+ libcapstone-dev:mipsel \
+ libcurl4-gnutls-dev:mipsel \
+ libdaxctl-dev:mipsel \
+ libdrm-dev:mipsel \
+ libepoxy-dev:mipsel \
+ libfdt-dev:mipsel \
+ libffi-dev:mipsel \
+ libfuse3-dev:mipsel \
+ libgbm-dev:mipsel \
+ libgcrypt20-dev:mipsel \
+ libglib2.0-dev:mipsel \
+ libglusterfs-dev:mipsel \
+ libgnutls28-dev:mipsel \
+ libgtk-3-dev:mipsel \
+ libibumad-dev:mipsel \
+ libibverbs-dev:mipsel \
+ libiscsi-dev:mipsel \
+ libjemalloc-dev:mipsel \
+ libjpeg62-turbo-dev:mipsel \
+ liblttng-ust-dev:mipsel \
+ liblzo2-dev:mipsel \
+ libncursesw5-dev:mipsel \
+ libnfs-dev:mipsel \
+ libnuma-dev:mipsel \
+ libpam0g-dev:mipsel \
+ libpixman-1-dev:mipsel \
+ libpng-dev:mipsel \
+ libpulse-dev:mipsel \
+ librbd-dev:mipsel \
+ librdmacm-dev:mipsel \
+ libsasl2-dev:mipsel \
+ libsdl2-dev:mipsel \
+ libsdl2-image-dev:mipsel \
+ libseccomp-dev:mipsel \
+ libselinux1-dev:mipsel \
+ libslirp-dev:mipsel \
+ libsnappy-dev:mipsel \
+ libspice-server-dev:mipsel \
+ libssh-gcrypt-dev:mipsel \
+ libsystemd-dev:mipsel \
+ libtasn1-6-dev:mipsel \
+ libudev-dev:mipsel \
+ liburing-dev:mipsel \
+ libusb-1.0-0-dev:mipsel \
+ libusbredirhost-dev:mipsel \
+ libvdeplug-dev:mipsel \
+ libvirglrenderer-dev:mipsel \
+ libvte-2.91-dev:mipsel \
+ libzstd-dev:mipsel \
+ nettle-dev:mipsel \
+ systemtap-sdt-dev:mipsel \
+ xfslibs-dev:mipsel \
+ zlib1g-dev:mipsel && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ mkdir -p /usr/local/share/meson/cross && \
+ echo "[binaries]\n\
+c = '/usr/bin/mipsel-linux-gnu-gcc'\n\
+ar = '/usr/bin/mipsel-linux-gnu-gcc-ar'\n\
+strip = '/usr/bin/mipsel-linux-gnu-strip'\n\
+pkgconfig = '/usr/bin/mipsel-linux-gnu-pkg-config'\n\
+\n\
+[host_machine]\n\
+system = 'linux'\n\
+cpu_family = 'mips'\n\
+cpu = 'mipsel'\n\
+endian = 'little'" > /usr/local/share/meson/cross/mipsel-linux-gnu && \
+ dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-c++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-g++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mipsel-linux-gnu-gcc
-# Install extra libraries to increase code coverage
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt install -y --no-install-recommends \
- libbz2-dev:mipsel \
- liblzo2-dev:mipsel \
- librdmacm-dev:mipsel \
- libsnappy-dev:mipsel
+ENV ABI "mipsel-linux-gnu"
+ENV MESON_OPTS "--cross-file=mipsel-linux-gnu"
+ENV QEMU_CONFIGURE_OPTS --cross-prefix=mipsel-linux-gnu-
+ENV DEF_TARGET_LIST mipsel-softmmu,mipsel-linux-user
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 4dc5527234..a4b7452189 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -115,6 +115,11 @@ try:
trailer=debian_cross_build("arm-linux-gnueabihf-",
"arm-softmmu,arm-linux-user"))
+ generate_dockerfile("debian-mipsel-cross", "debian-11",
+ cross="mipsel",
+ trailer=debian_cross_build("mipsel-linux-gnu-",
+ "mipsel-softmmu,mipsel-linux-user"))
+
generate_dockerfile("debian-s390x-cross", "debian-11",
cross="s390x",
trailer=debian_cross_build("s390x-linux-gnu-",
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 09/33] tests/docker: update debian-mips64el-cross with lcitool
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (7 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 08/33] tests/docker: update debian-mipsel-cross " Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:46 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 10/33] tests/docker: update debian-ppc64el-cross " Alex Bennée
` (23 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, Thomas Huth, Wainer dos Santos Moschetta,
Beraldo Leal, Jiaxun Yang
Use lcitool to update debian-mips64el-cross to a Debian 11 based system.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
.gitlab-ci.d/container-cross.yml | 3 +-
tests/docker/Makefile.include | 1 -
.../dockerfiles/debian-mips64el-cross.docker | 177 +++++++++++++++---
tests/lcitool/refresh | 5 +
4 files changed, 159 insertions(+), 27 deletions(-)
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 1a533e6fc0..411dc06bf8 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -88,8 +88,7 @@ mips64-debian-cross-container:
mips64el-debian-cross-container:
extends: .container_job_template
- stage: containers-layer2
- needs: ['amd64-debian10-container']
+ stage: containers
variables:
NAME: debian-mips64el-cross
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 0ac5975419..d9f37ae8fa 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -93,7 +93,6 @@ docker-image-debian-hppa-cross: docker-image-debian10
docker-image-debian-m68k-cross: docker-image-debian10
docker-image-debian-mips-cross: docker-image-debian10
docker-image-debian-mips64-cross: docker-image-debian10
-docker-image-debian-mips64el-cross: docker-image-debian10
docker-image-debian-ppc64el-cross: docker-image-debian10
docker-image-debian-sh4-cross: docker-image-debian10
docker-image-debian-sparc64-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-mips64el-cross.docker b/tests/docker/dockerfiles/debian-mips64el-cross.docker
index c990b683b7..b02dcb7fd9 100644
--- a/tests/docker/dockerfiles/debian-mips64el-cross.docker
+++ b/tests/docker/dockerfiles/debian-mips64el-cross.docker
@@ -1,33 +1,162 @@
+# THIS FILE WAS AUTO-GENERATED
#
-# Docker mips64el cross-compiler target
-#
-# This docker target builds on the debian Stretch base image.
+# $ lcitool dockerfile --layers all --cross mips64el debian-11 qemu
#
+# https://gitlab.com/libvirt/libvirt-ci
-FROM qemu/debian10
+FROM docker.io/library/debian:11-slim
-MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ apt-get update && \
+ apt-get install -y eatmydata && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y \
+ bash \
+ bc \
+ bsdextrautils \
+ bzip2 \
+ ca-certificates \
+ ccache \
+ dbus \
+ debianutils \
+ diffutils \
+ exuberant-ctags \
+ findutils \
+ gcovr \
+ genisoimage \
+ gettext \
+ git \
+ hostname \
+ libpcre2-dev \
+ libspice-protocol-dev \
+ llvm \
+ locales \
+ make \
+ meson \
+ ncat \
+ ninja-build \
+ openssh-client \
+ perl-base \
+ pkgconf \
+ python3 \
+ python3-numpy \
+ python3-opencv \
+ python3-pillow \
+ python3-pip \
+ python3-sphinx \
+ python3-sphinx-rtd-theme \
+ python3-venv \
+ python3-yaml \
+ rpm2cpio \
+ sed \
+ sparse \
+ tar \
+ tesseract-ocr \
+ tesseract-ocr-eng \
+ texinfo && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+ dpkg-reconfigure locales
-# Add the foreign architecture we want and install dependencies
-RUN dpkg --add-architecture mips64el && \
- apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt install -y --no-install-recommends \
- gcc-mips64el-linux-gnuabi64
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt build-dep -yy -a mips64el --arch-only qemu
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ dpkg --add-architecture mips64el && \
+ eatmydata apt-get update && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y dpkg-dev && \
+ eatmydata apt-get install --no-install-recommends -y \
+ g++-mips64el-linux-gnuabi64 \
+ gcc-mips64el-linux-gnuabi64 \
+ libaio-dev:mips64el \
+ libasound2-dev:mips64el \
+ libattr1-dev:mips64el \
+ libbpf-dev:mips64el \
+ libbrlapi-dev:mips64el \
+ libbz2-dev:mips64el \
+ libc6-dev:mips64el \
+ libcacard-dev:mips64el \
+ libcap-ng-dev:mips64el \
+ libcapstone-dev:mips64el \
+ libcurl4-gnutls-dev:mips64el \
+ libdaxctl-dev:mips64el \
+ libdrm-dev:mips64el \
+ libepoxy-dev:mips64el \
+ libfdt-dev:mips64el \
+ libffi-dev:mips64el \
+ libfuse3-dev:mips64el \
+ libgbm-dev:mips64el \
+ libgcrypt20-dev:mips64el \
+ libglib2.0-dev:mips64el \
+ libglusterfs-dev:mips64el \
+ libgnutls28-dev:mips64el \
+ libgtk-3-dev:mips64el \
+ libibumad-dev:mips64el \
+ libibverbs-dev:mips64el \
+ libiscsi-dev:mips64el \
+ libjemalloc-dev:mips64el \
+ libjpeg62-turbo-dev:mips64el \
+ liblttng-ust-dev:mips64el \
+ liblzo2-dev:mips64el \
+ libncursesw5-dev:mips64el \
+ libnfs-dev:mips64el \
+ libnuma-dev:mips64el \
+ libpam0g-dev:mips64el \
+ libpixman-1-dev:mips64el \
+ libpng-dev:mips64el \
+ libpulse-dev:mips64el \
+ librbd-dev:mips64el \
+ librdmacm-dev:mips64el \
+ libsasl2-dev:mips64el \
+ libsdl2-dev:mips64el \
+ libsdl2-image-dev:mips64el \
+ libseccomp-dev:mips64el \
+ libselinux1-dev:mips64el \
+ libslirp-dev:mips64el \
+ libsnappy-dev:mips64el \
+ libspice-server-dev:mips64el \
+ libssh-gcrypt-dev:mips64el \
+ libsystemd-dev:mips64el \
+ libtasn1-6-dev:mips64el \
+ libudev-dev:mips64el \
+ liburing-dev:mips64el \
+ libusb-1.0-0-dev:mips64el \
+ libusbredirhost-dev:mips64el \
+ libvdeplug-dev:mips64el \
+ libvirglrenderer-dev:mips64el \
+ libvte-2.91-dev:mips64el \
+ libzstd-dev:mips64el \
+ nettle-dev:mips64el \
+ systemtap-sdt-dev:mips64el \
+ xfslibs-dev:mips64el \
+ zlib1g-dev:mips64el && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ mkdir -p /usr/local/share/meson/cross && \
+ echo "[binaries]\n\
+c = '/usr/bin/mips64el-linux-gnuabi64-gcc'\n\
+ar = '/usr/bin/mips64el-linux-gnuabi64-gcc-ar'\n\
+strip = '/usr/bin/mips64el-linux-gnuabi64-strip'\n\
+pkgconfig = '/usr/bin/mips64el-linux-gnuabi64-pkg-config'\n\
+\n\
+[host_machine]\n\
+system = 'linux'\n\
+cpu_family = 'mips64'\n\
+cpu = 'mips64el'\n\
+endian = 'little'" > /usr/local/share/meson/cross/mips64el-linux-gnuabi64 && \
+ dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-c++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-g++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/mips64el-linux-gnuabi64-gcc
-# Specify the cross prefix for this image (see tests/docker/common.rc)
+ENV ABI "mips64el-linux-gnuabi64"
+ENV MESON_OPTS "--cross-file=mips64el-linux-gnuabi64"
ENV QEMU_CONFIGURE_OPTS --cross-prefix=mips64el-linux-gnuabi64-
ENV DEF_TARGET_LIST mips64el-softmmu,mips64el-linux-user
-
-# Install extra libraries to increase code coverage
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt install -y --no-install-recommends \
- libbz2-dev:mips64el \
- liblzo2-dev:mips64el \
- librdmacm-dev:mips64el \
- libsnappy-dev:mips64el
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index a4b7452189..341a07e677 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -115,6 +115,11 @@ try:
trailer=debian_cross_build("arm-linux-gnueabihf-",
"arm-softmmu,arm-linux-user"))
+ generate_dockerfile("debian-mips64el-cross", "debian-11",
+ cross="mips64el",
+ trailer=debian_cross_build("mips64el-linux-gnuabi64-",
+ "mips64el-softmmu,mips64el-linux-user"))
+
generate_dockerfile("debian-mipsel-cross", "debian-11",
cross="mipsel",
trailer=debian_cross_build("mipsel-linux-gnu-",
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 10/33] tests/docker: update debian-ppc64el-cross with lcitool
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (8 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 09/33] tests/docker: update debian-mips64el-cross " Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:46 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 11/33] tests/docker: update debian-amd64 " Alex Bennée
` (22 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, Thomas Huth, Wainer dos Santos Moschetta,
Beraldo Leal
Use lcitool to update debian-ppc64el-cross to a Debian 11 based system.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
.gitlab-ci.d/container-cross.yml | 3 +-
tests/docker/Makefile.include | 1 -
.../dockerfiles/debian-ppc64el-cross.docker | 178 +++++++++++++++---
tests/lcitool/refresh | 5 +
4 files changed, 163 insertions(+), 24 deletions(-)
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 411dc06bf8..147e667744 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -114,8 +114,7 @@ powerpc-test-cross-container:
ppc64el-debian-cross-container:
extends: .container_job_template
- stage: containers-layer2
- needs: ['amd64-debian10-container']
+ stage: containers
variables:
NAME: debian-ppc64el-cross
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index d9f37ae8fa..e68f91b853 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -93,7 +93,6 @@ docker-image-debian-hppa-cross: docker-image-debian10
docker-image-debian-m68k-cross: docker-image-debian10
docker-image-debian-mips-cross: docker-image-debian10
docker-image-debian-mips64-cross: docker-image-debian10
-docker-image-debian-ppc64el-cross: docker-image-debian10
docker-image-debian-sh4-cross: docker-image-debian10
docker-image-debian-sparc64-cross: docker-image-debian10
diff --git a/tests/docker/dockerfiles/debian-ppc64el-cross.docker b/tests/docker/dockerfiles/debian-ppc64el-cross.docker
index 5de12b01cd..bcf04bc90b 100644
--- a/tests/docker/dockerfiles/debian-ppc64el-cross.docker
+++ b/tests/docker/dockerfiles/debian-ppc64el-cross.docker
@@ -1,28 +1,164 @@
+# THIS FILE WAS AUTO-GENERATED
#
-# Docker ppc64el cross-compiler target
+# $ lcitool dockerfile --layers all --cross ppc64le debian-11 qemu
#
-# This docker target builds on the debian Stretch base image.
-#
-FROM qemu/debian10
+# https://gitlab.com/libvirt/libvirt-ci
+
+FROM docker.io/library/debian:11-slim
-# Add the foreign architecture we want and install dependencies
-RUN dpkg --add-architecture ppc64el && \
- apt update && \
- apt install -yy crossbuild-essential-ppc64el
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ apt-get update && \
+ apt-get install -y eatmydata && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y \
+ bash \
+ bc \
+ bsdextrautils \
+ bzip2 \
+ ca-certificates \
+ ccache \
+ dbus \
+ debianutils \
+ diffutils \
+ exuberant-ctags \
+ findutils \
+ gcovr \
+ genisoimage \
+ gettext \
+ git \
+ hostname \
+ libpcre2-dev \
+ libspice-protocol-dev \
+ llvm \
+ locales \
+ make \
+ meson \
+ ncat \
+ ninja-build \
+ openssh-client \
+ perl-base \
+ pkgconf \
+ python3 \
+ python3-numpy \
+ python3-opencv \
+ python3-pillow \
+ python3-pip \
+ python3-sphinx \
+ python3-sphinx-rtd-theme \
+ python3-venv \
+ python3-yaml \
+ rpm2cpio \
+ sed \
+ sparse \
+ tar \
+ tesseract-ocr \
+ tesseract-ocr-eng \
+ texinfo && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+ dpkg-reconfigure locales
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt build-dep -yy -a ppc64el --arch-only qemu
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
-# Specify the cross prefix for this image (see tests/docker/common.rc)
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ dpkg --add-architecture ppc64el && \
+ eatmydata apt-get update && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y dpkg-dev && \
+ eatmydata apt-get install --no-install-recommends -y \
+ g++-powerpc64le-linux-gnu \
+ gcc-powerpc64le-linux-gnu \
+ libaio-dev:ppc64el \
+ libasan5:ppc64el \
+ libasound2-dev:ppc64el \
+ libattr1-dev:ppc64el \
+ libbpf-dev:ppc64el \
+ libbrlapi-dev:ppc64el \
+ libbz2-dev:ppc64el \
+ libc6-dev:ppc64el \
+ libcacard-dev:ppc64el \
+ libcap-ng-dev:ppc64el \
+ libcapstone-dev:ppc64el \
+ libcurl4-gnutls-dev:ppc64el \
+ libdaxctl-dev:ppc64el \
+ libdrm-dev:ppc64el \
+ libepoxy-dev:ppc64el \
+ libfdt-dev:ppc64el \
+ libffi-dev:ppc64el \
+ libfuse3-dev:ppc64el \
+ libgbm-dev:ppc64el \
+ libgcrypt20-dev:ppc64el \
+ libglib2.0-dev:ppc64el \
+ libglusterfs-dev:ppc64el \
+ libgnutls28-dev:ppc64el \
+ libgtk-3-dev:ppc64el \
+ libibumad-dev:ppc64el \
+ libibverbs-dev:ppc64el \
+ libiscsi-dev:ppc64el \
+ libjemalloc-dev:ppc64el \
+ libjpeg62-turbo-dev:ppc64el \
+ liblttng-ust-dev:ppc64el \
+ liblzo2-dev:ppc64el \
+ libncursesw5-dev:ppc64el \
+ libnfs-dev:ppc64el \
+ libnuma-dev:ppc64el \
+ libpam0g-dev:ppc64el \
+ libpixman-1-dev:ppc64el \
+ libpng-dev:ppc64el \
+ libpulse-dev:ppc64el \
+ librbd-dev:ppc64el \
+ librdmacm-dev:ppc64el \
+ libsasl2-dev:ppc64el \
+ libsdl2-dev:ppc64el \
+ libsdl2-image-dev:ppc64el \
+ libseccomp-dev:ppc64el \
+ libselinux1-dev:ppc64el \
+ libslirp-dev:ppc64el \
+ libsnappy-dev:ppc64el \
+ libspice-server-dev:ppc64el \
+ libssh-gcrypt-dev:ppc64el \
+ libsystemd-dev:ppc64el \
+ libtasn1-6-dev:ppc64el \
+ libubsan1:ppc64el \
+ libudev-dev:ppc64el \
+ liburing-dev:ppc64el \
+ libusb-1.0-0-dev:ppc64el \
+ libusbredirhost-dev:ppc64el \
+ libvdeplug-dev:ppc64el \
+ libvirglrenderer-dev:ppc64el \
+ libvte-2.91-dev:ppc64el \
+ libzstd-dev:ppc64el \
+ nettle-dev:ppc64el \
+ systemtap-sdt-dev:ppc64el \
+ xfslibs-dev:ppc64el \
+ zlib1g-dev:ppc64el && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ mkdir -p /usr/local/share/meson/cross && \
+ echo "[binaries]\n\
+c = '/usr/bin/powerpc64le-linux-gnu-gcc'\n\
+ar = '/usr/bin/powerpc64le-linux-gnu-gcc-ar'\n\
+strip = '/usr/bin/powerpc64le-linux-gnu-strip'\n\
+pkgconfig = '/usr/bin/powerpc64le-linux-gnu-pkg-config'\n\
+\n\
+[host_machine]\n\
+system = 'linux'\n\
+cpu_family = 'ppc64'\n\
+cpu = 'powerpc64le'\n\
+endian = 'little'" > /usr/local/share/meson/cross/powerpc64le-linux-gnu && \
+ dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-c++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-g++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/powerpc64le-linux-gnu-gcc
+
+ENV ABI "powerpc64le-linux-gnu"
+ENV MESON_OPTS "--cross-file=powerpc64le-linux-gnu"
ENV QEMU_CONFIGURE_OPTS --cross-prefix=powerpc64le-linux-gnu-
ENV DEF_TARGET_LIST ppc64-softmmu,ppc64-linux-user
-
-# Install extra libraries to increase code coverage
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt install -y --no-install-recommends \
- libbz2-dev:ppc64el \
- liblzo2-dev:ppc64el \
- librdmacm-dev:ppc64el \
- libsnappy-dev:ppc64el
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 341a07e677..b947bb170d 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -125,6 +125,11 @@ try:
trailer=debian_cross_build("mipsel-linux-gnu-",
"mipsel-softmmu,mipsel-linux-user"))
+ generate_dockerfile("debian-ppc64el-cross", "debian-11",
+ cross="ppc64le",
+ trailer=debian_cross_build("powerpc64le-linux-gnu-",
+ "ppc64-softmmu,ppc64-linux-user"))
+
generate_dockerfile("debian-s390x-cross", "debian-11",
cross="s390x",
trailer=debian_cross_build("s390x-linux-gnu-",
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 11/33] tests/docker: update debian-amd64 with lcitool
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (9 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 10/33] tests/docker: update debian-ppc64el-cross " Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:51 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 12/33] configure: do not define or use the CPP variable Alex Bennée
` (21 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, Luigi Rizzo, Giuseppe Lettieri,
Vincenzo Maffione, Thomas Huth, Wainer dos Santos Moschetta,
Beraldo Leal
The one minor wrinkle we need to account for is the netmap support
still requires building from source. We also include cscope and GNU
global as they are used in one of the builds.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Luigi Rizzo <rizzo@iet.unipi.it>
Cc: Giuseppe Lettieri <g.lettieri@iet.unipi.it>
Cc: Vincenzo Maffione <v.maffione@gmail.com>
---
.gitlab-ci.d/containers.yml | 3 +-
tests/docker/dockerfiles/debian-amd64.docker | 194 ++++++++++++++-----
tests/lcitool/refresh | 19 ++
3 files changed, 164 insertions(+), 52 deletions(-)
diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
index e9df90bbdd..be34cbc7ba 100644
--- a/.gitlab-ci.d/containers.yml
+++ b/.gitlab-ci.d/containers.yml
@@ -14,8 +14,7 @@ amd64-debian11-container:
amd64-debian-container:
extends: .container_job_template
- stage: containers-layer2
- needs: ['amd64-debian10-container']
+ stage: containers
variables:
NAME: debian-amd64
diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-amd64.docker
index ed546edcd6..503e282802 100644
--- a/tests/docker/dockerfiles/debian-amd64.docker
+++ b/tests/docker/dockerfiles/debian-amd64.docker
@@ -1,59 +1,153 @@
+# THIS FILE WAS AUTO-GENERATED
#
-# Docker x86_64 target
+# $ lcitool dockerfile --layers all debian-11 qemu
#
-# This docker target builds on the Debian Buster base image. Further
-# libraries which are not widely available are installed by hand.
-#
-FROM qemu/debian10
-MAINTAINER Philippe Mathieu-Daudé <f4bug@amsat.org>
-
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt build-dep -yy qemu
+# https://gitlab.com/libvirt/libvirt-ci
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt install -y --no-install-recommends \
- cscope \
- genisoimage \
- exuberant-ctags \
- global \
- libbz2-dev \
- liblzo2-dev \
- libgcrypt20-dev \
- libfdt-dev \
- librdmacm-dev \
- libsasl2-dev \
- libsnappy-dev \
- libvte-dev \
- netcat-openbsd \
- openssh-client \
- python3-numpy \
- python3-opencv \
- python3-venv
+FROM docker.io/library/debian:11-slim
-# virgl
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt install -y --no-install-recommends \
- libegl1-mesa-dev \
- libepoxy-dev \
- libgbm-dev
-RUN git clone https://gitlab.freedesktop.org/virgl/virglrenderer.git /usr/src/virglrenderer && \
- cd /usr/src/virglrenderer && git checkout virglrenderer-0.8.0
-RUN cd /usr/src/virglrenderer && ./autogen.sh && ./configure --disable-tests && make install
+RUN export DEBIAN_FRONTEND=noninteractive && \
+ apt-get update && \
+ apt-get install -y eatmydata && \
+ eatmydata apt-get dist-upgrade -y && \
+ eatmydata apt-get install --no-install-recommends -y \
+ bash \
+ bc \
+ bsdextrautils \
+ bzip2 \
+ ca-certificates \
+ ccache \
+ clang \
+ dbus \
+ debianutils \
+ diffutils \
+ exuberant-ctags \
+ findutils \
+ g++ \
+ gcc \
+ gcovr \
+ genisoimage \
+ gettext \
+ git \
+ hostname \
+ libaio-dev \
+ libasan5 \
+ libasound2-dev \
+ libattr1-dev \
+ libbpf-dev \
+ libbrlapi-dev \
+ libbz2-dev \
+ libc6-dev \
+ libcacard-dev \
+ libcap-ng-dev \
+ libcapstone-dev \
+ libcurl4-gnutls-dev \
+ libdaxctl-dev \
+ libdrm-dev \
+ libepoxy-dev \
+ libfdt-dev \
+ libffi-dev \
+ libfuse3-dev \
+ libgbm-dev \
+ libgcrypt20-dev \
+ libglib2.0-dev \
+ libglusterfs-dev \
+ libgnutls28-dev \
+ libgtk-3-dev \
+ libibumad-dev \
+ libibverbs-dev \
+ libiscsi-dev \
+ libjemalloc-dev \
+ libjpeg62-turbo-dev \
+ liblttng-ust-dev \
+ liblzo2-dev \
+ libncursesw5-dev \
+ libnfs-dev \
+ libnuma-dev \
+ libpam0g-dev \
+ libpcre2-dev \
+ libpixman-1-dev \
+ libpmem-dev \
+ libpng-dev \
+ libpulse-dev \
+ librbd-dev \
+ librdmacm-dev \
+ libsasl2-dev \
+ libsdl2-dev \
+ libsdl2-image-dev \
+ libseccomp-dev \
+ libselinux1-dev \
+ libslirp-dev \
+ libsnappy-dev \
+ libspice-protocol-dev \
+ libspice-server-dev \
+ libssh-gcrypt-dev \
+ libsystemd-dev \
+ libtasn1-6-dev \
+ libubsan1 \
+ libudev-dev \
+ liburing-dev \
+ libusb-1.0-0-dev \
+ libusbredirhost-dev \
+ libvdeplug-dev \
+ libvirglrenderer-dev \
+ libvte-2.91-dev \
+ libxen-dev \
+ libzstd-dev \
+ llvm \
+ locales \
+ make \
+ meson \
+ multipath-tools \
+ ncat \
+ nettle-dev \
+ ninja-build \
+ openssh-client \
+ perl-base \
+ pkgconf \
+ python3 \
+ python3-numpy \
+ python3-opencv \
+ python3-pillow \
+ python3-pip \
+ python3-sphinx \
+ python3-sphinx-rtd-theme \
+ python3-venv \
+ python3-yaml \
+ rpm2cpio \
+ sed \
+ sparse \
+ systemtap-sdt-dev \
+ tar \
+ tesseract-ocr \
+ tesseract-ocr-eng \
+ texinfo \
+ xfslibs-dev \
+ zlib1g-dev && \
+ eatmydata apt-get autoremove -y && \
+ eatmydata apt-get autoclean -y && \
+ sed -Ei 's,^# (en_US\.UTF-8 .*)$,\1,' /etc/locale.gen && \
+ dpkg-reconfigure locales && \
+ dpkg-query --showformat '${Package}_${Version}_${Architecture}\n' --show > /packages.txt && \
+ mkdir -p /usr/libexec/ccache-wrappers && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/c++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/cc && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \
+ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc
-# netmap
-RUN apt update && \
- DEBIAN_FRONTEND=noninteractive eatmydata \
- apt install -y --no-install-recommends \
- linux-headers-amd64
+ENV LANG "en_US.UTF-8"
+ENV MAKE "/usr/bin/make"
+ENV NINJA "/usr/bin/ninja"
+ENV PYTHON "/usr/bin/python3"
+ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers"
+# netmap/cscope/global
+RUN DEBIAN_FRONTEND=noninteractive eatmydata \
+ apt install -y --no-install-recommends \
+ cscope\
+ global\
+ linux-headers-amd64
RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap
RUN cd /usr/src/netmap && git checkout v11.3
RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && make install
ENV QEMU_CONFIGURE_OPTS --enable-netmap
-
-RUN ldconfig
-
-# gcrypt
-ENV QEMU_CONFIGURE_OPTS $QEMU_CONFIGURE_OPTS --enable-gcrypt
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index b947bb170d..5e260f8cd6 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -76,6 +76,23 @@ ubuntu2004_tsanhack = [
]
+# Netmap still needs to be manually built as it is yet to be packaged
+# into a distro. We also add cscope and gtags which are used in the CI
+# test
+debian11_extras = [
+ "# netmap/cscope/global\n",
+ "RUN DEBIAN_FRONTEND=noninteractive eatmydata \\\n",
+ " apt install -y --no-install-recommends \\\n",
+ " cscope\\\n",
+ " global\\\n",
+ " linux-headers-amd64\n",
+ "RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap\n",
+ "RUN cd /usr/src/netmap && git checkout v11.3\n",
+ "RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && make install\n",
+ "ENV QEMU_CONFIGURE_OPTS --enable-netmap\n"
+]
+
+
def debian_cross_build(prefix, targets):
conf = "ENV QEMU_CONFIGURE_OPTS --cross-prefix=%s\n" % (prefix)
targets = "ENV DEF_TARGET_LIST %s\n" % (targets)
@@ -92,6 +109,8 @@ try:
#
generate_dockerfile("alpine", "alpine-edge")
generate_dockerfile("centos8", "centos-stream-8")
+ generate_dockerfile("debian-amd64", "debian-11",
+ trailer="".join(debian11_extras))
generate_dockerfile("fedora", "fedora-35")
generate_dockerfile("opensuse-leap", "opensuse-leap-152")
generate_dockerfile("ubuntu2004", "ubuntu-2004",
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 12/33] configure: do not define or use the CPP variable
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (10 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 11/33] tests/docker: update debian-amd64 " Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 13/33] build: clean up ninja invocation Alex Bennée
` (20 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
Just hardcode $(CC) -E, it should be enough.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-2-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 3 ---
pc-bios/optionrom/Makefile | 2 +-
2 files changed, 1 insertion(+), 4 deletions(-)
diff --git a/configure b/configure
index 180ee688dc..7a071c161a 100755
--- a/configure
+++ b/configure
@@ -376,7 +376,6 @@ fi
ar="${AR-${cross_prefix}ar}"
as="${AS-${cross_prefix}as}"
ccas="${CCAS-$cc}"
-cpp="${CPP-$cc -E}"
objcopy="${OBJCOPY-${cross_prefix}objcopy}"
ld="${LD-${cross_prefix}ld}"
ranlib="${RANLIB-${cross_prefix}ranlib}"
@@ -2012,7 +2011,6 @@ echo "CC=$cc" >> $config_host_mak
echo "AR=$ar" >> $config_host_mak
echo "AS=$as" >> $config_host_mak
echo "CCAS=$ccas" >> $config_host_mak
-echo "CPP=$cpp" >> $config_host_mak
echo "OBJCOPY=$objcopy" >> $config_host_mak
echo "LD=$ld" >> $config_host_mak
echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak
@@ -2254,7 +2252,6 @@ preserve_env() {
preserve_env AR
preserve_env AS
preserve_env CC
-preserve_env CPP
preserve_env CFLAGS
preserve_env CXX
preserve_env CXXFLAGS
diff --git a/pc-bios/optionrom/Makefile b/pc-bios/optionrom/Makefile
index 2494ad9c25..17ccc76241 100644
--- a/pc-bios/optionrom/Makefile
+++ b/pc-bios/optionrom/Makefile
@@ -50,7 +50,7 @@ override LDFLAGS = -m $(LD_I386_EMULATION) -T $(SRC_DIR)/flat.lds
pvh.img: pvh.o pvh_main.o
%.o: %.S
- $(call quiet-command,$(CPP) $(CPPFLAGS) -c -o - $< | $(AS) $(ASFLAGS) -o $@,"AS","$@")
+ $(call quiet-command,$(CC) $(CPPFLAGS) -E -o - $< | $(AS) $(ASFLAGS) -o $@,"AS","$@")
%.o: %.c
$(call quiet-command,$(CC) $(CPPFLAGS) $(CFLAGS) -c $< -o $@,"CC","$@")
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 13/33] build: clean up ninja invocation
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (11 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 12/33] configure: do not define or use the CPP variable Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 14/33] build: add a more generic way to specify make->ninja dependencies Alex Bennée
` (19 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
Fix an incorrect "@@:" and move "-d keepdepfile" to the NINJAFLAGS variable.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-3-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
Makefile | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/Makefile b/Makefile
index b842dbccdb..fad312040f 100644
--- a/Makefile
+++ b/Makefile
@@ -143,7 +143,7 @@ MAKE.q = $(findstring q,$(firstword $(filter-out --%,$(MAKEFLAGS))))
MAKE.nq = $(if $(word 2, $(MAKE.n) $(MAKE.q)),nq)
NINJAFLAGS = $(if $V,-v) $(if $(MAKE.n), -n) $(if $(MAKE.k), -k0) \
$(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS)))) \
-
+ -d keepdepfile
ninja-cmd-goals = $(or $(MAKECMDGOALS), all)
ninja-cmd-goals += $(foreach t, $(.check.build-suites), $(.check-$t.deps))
ninja-cmd-goals += $(foreach t, $(.bench.build-suites), $(.bench-$t.deps))
@@ -160,8 +160,8 @@ $(ninja-targets): run-ninja
# --output-sync line.
run-ninja: config-host.mak
ifneq ($(filter $(ninja-targets), $(ninja-cmd-goals)),)
- +$(quiet-@)$(if $(MAKE.nq),@:, $(NINJA) -d keepdepfile \
- $(NINJAFLAGS) $(sort $(filter $(ninja-targets), $(ninja-cmd-goals))) | cat)
+ +$(if $(MAKE.nq),@:,$(quiet-@)$(NINJA) $(NINJAFLAGS) \
+ $(sort $(filter $(ninja-targets), $(ninja-cmd-goals))) | cat)
endif
endif
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 14/33] build: add a more generic way to specify make->ninja dependencies
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (12 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 13/33] build: clean up ninja invocation Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 15/33] build: do a full build before running TCG tests Alex Bennée
` (18 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, John Snow
From: Paolo Bonzini <pbonzini@redhat.com>
Let any make target specify ninja goals that needs to be built for it
(though selecting the goals is _not_ recursive on depending targets)
instead of having a custom mechanism only for "make check" and "make
bench".
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-4-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
Makefile | 3 +--
scripts/mtest2make.py | 8 ++++----
2 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/Makefile b/Makefile
index fad312040f..3c0d89057e 100644
--- a/Makefile
+++ b/Makefile
@@ -145,8 +145,7 @@ NINJAFLAGS = $(if $V,-v) $(if $(MAKE.n), -n) $(if $(MAKE.k), -k0) \
$(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS)))) \
-d keepdepfile
ninja-cmd-goals = $(or $(MAKECMDGOALS), all)
-ninja-cmd-goals += $(foreach t, $(.check.build-suites), $(.check-$t.deps))
-ninja-cmd-goals += $(foreach t, $(.bench.build-suites), $(.bench-$t.deps))
+ninja-cmd-goals += $(foreach g, $(MAKECMDGOALS), $(.ninja-goals.$g))))
makefile-targets := build.ninja ctags TAGS cscope dist clean uninstall
# "ninja -t targets" also lists all prerequisites. If build system
diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py
index 304634b71e..0fe81efbbc 100644
--- a/scripts/mtest2make.py
+++ b/scripts/mtest2make.py
@@ -81,12 +81,12 @@ def emit_prolog(suites, prefix):
def emit_suite_deps(name, suite, prefix):
deps = ' '.join(suite.deps)
- targets = f'{prefix}-{name} {prefix}-report-{name}.junit.xml {prefix} {prefix}-report.junit.xml'
+ targets = [f'{prefix}-{name}', f'{prefix}-report-{name}.junit.xml', f'{prefix}', f'{prefix}-report.junit.xml',
+ f'{prefix}-build']
print()
print(f'.{prefix}-{name}.deps = {deps}')
- print(f'ifneq ($(filter {prefix}-build {targets}, $(MAKECMDGOALS)),)')
- print(f'.{prefix}.build-suites += {name}')
- print(f'endif')
+ for t in targets:
+ print(f'.ninja-goals.{t} += $(.{prefix}-{name}.deps)')
def emit_suite(name, suite, prefix):
emit_suite_deps(name, suite, prefix)
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 15/33] build: do a full build before running TCG tests
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (13 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 14/33] build: add a more generic way to specify make->ninja dependencies Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 16/33] configure, meson: move symlinking of ROMs to meson Alex Bennée
` (17 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
TCG tests need both QEMU and firmware to be built, so do "ninja all" before
trying to run them.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-5-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/Makefile.include | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tests/Makefile.include b/tests/Makefile.include
index ec84b2ebc0..72ce0561f4 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -57,7 +57,7 @@ $(TCG_TESTS_TARGETS:%=build-tcg-tests-%): build-tcg-tests-%: $(BUILD_DIR)/tests/
"BUILD","$* guest-tests")
.PHONY: $(TCG_TESTS_TARGETS:%=run-tcg-tests-%)
-$(TCG_TESTS_TARGETS:%=run-tcg-tests-%): run-tcg-tests-%: build-tcg-tests-% $(if $(CONFIG_PLUGIN),test-plugins)
+$(TCG_TESTS_TARGETS:%=run-tcg-tests-%): run-tcg-tests-%: build-tcg-tests-%
$(call quiet-command, \
$(MAKE) -C tests/tcg/$* -f ../Makefile.target $(SUBDIR_MAKEFLAGS) \
TARGET="$*" SRC_PATH="$(SRC_PATH)" SPEED=$(SPEED) run, \
@@ -74,6 +74,7 @@ $(TCG_TESTS_TARGETS:%=clean-tcg-tests-%): clean-tcg-tests-%:
build-tcg: $(BUILD_TCG_TARGET_RULES)
.PHONY: check-tcg
+.ninja-goals.check-tcg = all $(if $(CONFIG_PLUGIN),test-plugins)
check-tcg: $(RUN_TCG_TARGET_RULES)
.PHONY: clean-tcg
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 16/33] configure, meson: move symlinking of ROMs to meson
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (14 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 15/33] build: do a full build before running TCG tests Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 17/33] tests/tcg: correct target CPU for sparc32 Alex Bennée
` (16 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
This is useful because pc-bios/meson.build already has a list of all ROM
files, and thus does not need to use wildcards. The problems with
wildcards are mentioned above the definition of the LINKS variable,
but then the recommendation is disattended.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-6-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 15 ---------------
pc-bios/meson.build | 17 ++++++++++++-----
2 files changed, 12 insertions(+), 20 deletions(-)
diff --git a/configure b/configure
index 7a071c161a..2138f61e54 100755
--- a/configure
+++ b/configure
@@ -2097,21 +2097,6 @@ LINKS="$LINKS tests/avocado tests/data"
LINKS="$LINKS tests/qemu-iotests/check"
LINKS="$LINKS python"
LINKS="$LINKS contrib/plugins/Makefile "
-for bios_file in \
- $source_path/pc-bios/*.bin \
- $source_path/pc-bios/*.elf \
- $source_path/pc-bios/*.lid \
- $source_path/pc-bios/*.rom \
- $source_path/pc-bios/*.dtb \
- $source_path/pc-bios/*.img \
- $source_path/pc-bios/openbios-* \
- $source_path/pc-bios/u-boot.* \
- $source_path/pc-bios/palcode-* \
- $source_path/pc-bios/qemu_vga.ndrv
-
-do
- LINKS="$LINKS pc-bios/$(basename $bios_file)"
-done
for f in $LINKS ; do
if [ -e "$source_path/$f" ]; then
mkdir -p `dirname ./$f`
diff --git a/pc-bios/meson.build b/pc-bios/meson.build
index c86dedf7df..41ba1c0ec7 100644
--- a/pc-bios/meson.build
+++ b/pc-bios/meson.build
@@ -23,7 +23,7 @@ if unpack_edk2_blobs
endforeach
endif
-blobs = files(
+blobs = [
'bios.bin',
'bios-256k.bin',
'bios-microvm.bin',
@@ -83,11 +83,18 @@ blobs = files(
'npcm7xx_bootrom.bin',
'vof.bin',
'vof-nvram.bin',
-)
+]
-if get_option('install_blobs')
- install_data(blobs, install_dir: qemu_datadir)
-endif
+ln_s = [find_program('ln', required: true), '-sf']
+foreach f : blobs
+ roms += custom_target(f,
+ build_by_default: have_system,
+ output: f,
+ input: files('meson.build'), # dummy input
+ install: get_option('install_blobs'),
+ install_dir: qemu_datadir,
+ command: [ ln_s, meson.project_source_root() / 'pc-bios' / f, '@OUTPUT@' ])
+endforeach
subdir('descriptors')
subdir('keymaps')
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 17/33] tests/tcg: correct target CPU for sparc32
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (15 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 16/33] configure, meson: move symlinking of ROMs to meson Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 18/33] tests/tcg: merge configure.sh back into main configure script Alex Bennée
` (15 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
We do not want v8plus for pure sparc32, as the difference with the V8 ABI
are only meaningful on 64-bit CPUs suh as ultrasparc; supersparc is the
best CPU to use for 32-bit.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-7-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
tests/tcg/configure.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
index 691d90abac..59f2403d1a 100755
--- a/tests/tcg/configure.sh
+++ b/tests/tcg/configure.sh
@@ -70,7 +70,7 @@ fi
: ${cross_cc_riscv64="riscv64-linux-gnu-gcc"}
: ${cross_cc_s390x="s390x-linux-gnu-gcc"}
: ${cross_cc_sh4="sh4-linux-gnu-gcc"}
-: ${cross_cc_cflags_sparc="-m32 -mv8plus -mcpu=ultrasparc"}
+: ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
: ${cross_cc_sparc64="sparc64-linux-gnu-gcc"}
: ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
: ${cross_cc_x86_64="x86_64-linux-gnu-gcc"}
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 18/33] tests/tcg: merge configure.sh back into main configure script
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (16 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 17/33] tests/tcg: correct target CPU for sparc32 Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 19/33] configure: add missing cross compiler fallbacks Alex Bennée
` (14 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
tests/tcg/configure.sh has a complicated story.
In the beginning its code ran as part of the creation of config-target.mak
files, and that is where it placed the information on the target compiler.
However, probing for the buildability of TCG tests required multiple
inclusions of config-target.mak in the _main_ Makefile (not in
Makefile.target, which took care of building the QEMU executables in
the pre-Meson era), which polluted the namespace.
Thus, it was moved to a separate directory. It created small config-*.mak
files in $(BUILD_DIR)/tests/tcg. Those were also included multiple
times, but at least they were small and manageable; this was also an
important step in disentangling the TCG tests from Makefile.target.
Since then, Meson has allowed the configure script to go on a diet.
A few compilation tests survive (mostly for sanitizers) but these days
it mostly takes care of command line parsing, looking for tools, and
setting up the environment for Meson to do its stuff.
It's time to extend configure with the capability to build for more
than just one target: not just tests, but also firmware. As a first
step, integrate all the logic to find cross compilers in the configure
script, and move tests/tcg/configure.sh back there (though as a
separate loop, not integrated in the one that generates target
configurations for Meson).
tests/tcg is actually very close to being buildable as a standalone
project, so I actually expect the compiler tests to move back to
tests/tcg, as a "configure" script of sorts which would run at Make
time after the docker images are built. The GCC tree has a similar idea
of doing only bare-bones tree-wide configuration and leaving the rest
for Make time.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-8-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 398 +++++++++++++++++++++++++++++++++++++++--
tests/Makefile.include | 1 -
tests/tcg/configure.sh | 376 --------------------------------------
3 files changed, 388 insertions(+), 387 deletions(-)
delete mode 100755 tests/tcg/configure.sh
diff --git a/configure b/configure
index 2138f61e54..bb05e70bcc 100755
--- a/configure
+++ b/configure
@@ -109,6 +109,20 @@ error_exit() {
}
do_compiler() {
+ # Run the compiler, capturing its output to the log. First argument
+ # is compiler binary to execute.
+ local compiler="$1"
+ shift
+ if test -n "$BASH_VERSION"; then eval '
+ echo >>config.log "
+funcs: ${FUNCNAME[*]}
+lines: ${BASH_LINENO[*]}"
+ '; fi
+ echo $compiler "$@" >> config.log
+ $compiler "$@" >> config.log 2>&1 || return $?
+}
+
+do_compiler_werror() {
# Run the compiler, capturing its output to the log. First argument
# is compiler binary to execute.
compiler="$1"
@@ -142,15 +156,15 @@ lines: ${BASH_LINENO[*]}"
}
do_cc() {
- do_compiler "$cc" $CPU_CFLAGS "$@"
+ do_compiler_werror "$cc" $CPU_CFLAGS "$@"
}
do_cxx() {
- do_compiler "$cxx" $CPU_CFLAGS "$@"
+ do_compiler_werror "$cxx" $CPU_CFLAGS "$@"
}
do_objc() {
- do_compiler "$objcc" $CPU_CFLAGS "$@"
+ do_compiler_werror "$objcc" $CPU_CFLAGS "$@"
}
# Append $2 to the variable named $1, with space separation
@@ -345,11 +359,9 @@ for opt do
;;
--cross-cc-cflags-*) cc_arch=${opt#--cross-cc-cflags-}; cc_arch=${cc_arch%%=*}
eval "cross_cc_cflags_${cc_arch}=\$optarg"
- cross_cc_vars="$cross_cc_vars cross_cc_cflags_${cc_arch}"
;;
--cross-cc-*) cc_arch=${opt#--cross-cc-}; cc_arch=${cc_arch%%=*}
eval "cross_cc_${cc_arch}=\$optarg"
- cross_cc_vars="$cross_cc_vars cross_cc_${cc_arch}"
;;
esac
done
@@ -944,7 +956,6 @@ esac
if eval test -z "\${cross_cc_$cpu}"; then
eval "cross_cc_${cpu}=\$cc"
- cross_cc_vars="$cross_cc_vars cross_cc_${cpu}"
fi
default_target_list=""
@@ -1795,6 +1806,248 @@ case "$slirp" in
;;
esac
+##########################################
+# functions to probe cross compilers
+
+container="no"
+if test $use_containers = "yes"; then
+ if has "docker" || has "podman"; then
+ container=$($python $source_path/tests/docker/docker.py probe)
+ fi
+fi
+
+# cross compilers defaults, can be overridden with --cross-cc-ARCH
+: ${cross_cc_aarch64="aarch64-linux-gnu-gcc"}
+: ${cross_cc_aarch64_be="$cross_cc_aarch64"}
+: ${cross_cc_cflags_aarch64_be="-mbig-endian"}
+: ${cross_cc_alpha="alpha-linux-gnu-gcc"}
+: ${cross_cc_arm="arm-linux-gnueabihf-gcc"}
+: ${cross_cc_cflags_armeb="-mbig-endian"}
+: ${cross_cc_hexagon="hexagon-unknown-linux-musl-clang"}
+: ${cross_cc_cflags_hexagon="-mv67 -O2 -static"}
+: ${cross_cc_hppa="hppa-linux-gnu-gcc"}
+: ${cross_cc_i386="i686-linux-gnu-gcc"}
+: ${cross_cc_cflags_i386="-m32"}
+: ${cross_cc_m68k="m68k-linux-gnu-gcc"}
+: ${cross_cc_microblaze="microblaze-linux-musl-gcc"}
+: ${cross_cc_mips64el="mips64el-linux-gnuabi64-gcc"}
+: ${cross_cc_mips64="mips64-linux-gnuabi64-gcc"}
+: ${cross_cc_mipsel="mipsel-linux-gnu-gcc"}
+: ${cross_cc_mips="mips-linux-gnu-gcc"}
+: ${cross_cc_nios2="nios2-linux-gnu-gcc"}
+: ${cross_cc_ppc="powerpc-linux-gnu-gcc"}
+: ${cross_cc_cflags_ppc="-m32"}
+: ${cross_cc_ppc64="powerpc64-linux-gnu-gcc"}
+: ${cross_cc_cflags_ppc64="-m64 -mbig-endian"}
+: ${cross_cc_ppc64le="$cross_cc_ppc64"}
+: ${cross_cc_cflags_ppc64le="-m64 -mlittle-endian"}
+: ${cross_cc_riscv64="riscv64-linux-gnu-gcc"}
+: ${cross_cc_s390x="s390x-linux-gnu-gcc"}
+: ${cross_cc_sh4="sh4-linux-gnu-gcc"}
+: ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
+: ${cross_cc_sparc64="sparc64-linux-gnu-gcc"}
+: ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
+: ${cross_cc_x86_64="x86_64-linux-gnu-gcc"}
+: ${cross_cc_cflags_x86_64="-m64"}
+
+# tricore is special as it doesn't have a compiler
+: ${cross_as_tricore="tricore-as"}
+: ${cross_ld_tricore="tricore-ld"}
+
+probe_target_compiler() {
+ # reset all output variables
+ container_image=
+ container_hosts=
+ container_cross_cc=
+ container_cross_as=
+ container_cross_ld=
+ target_cc=
+ target_as=
+ target_ld=
+
+ case $1 in
+ aarch64) container_hosts="x86_64 aarch64" ;;
+ alpha) container_hosts=x86_64 ;;
+ arm) container_hosts="x86_64 aarch64" ;;
+ cris) container_hosts=x86_64 ;;
+ hexagon) container_hosts=x86_64 ;;
+ hppa) container_hosts=x86_64 ;;
+ i386) container_hosts=x86_64 ;;
+ m68k) container_hosts=x86_64 ;;
+ microblaze) container_hosts=x86_64 ;;
+ mips64el) container_hosts=x86_64 ;;
+ mips64) container_hosts=x86_64 ;;
+ mipsel) container_hosts=x86_64 ;;
+ mips) container_hosts=x86_64 ;;
+ nios2) container_hosts=x86_64 ;;
+ ppc) container_hosts=x86_64 ;;
+ ppc64|ppc64le) container_hosts=x86_64 ;;
+ riscv64) container_hosts=x86_64 ;;
+ s390x) container_hosts=x86_64 ;;
+ sh4) container_hosts=x86_64 ;;
+ sparc64) container_hosts=x86_64 ;;
+ tricore) container_hosts=x86_64 ;;
+ x86_64) container_hosts="aarch64 ppc64el x86_64" ;;
+ xtensa*) container_hosts=x86_64 ;;
+ esac
+
+ for host in $container_hosts; do
+ test "$container" != no || continue
+ test "$host" = "$cpu" || continue
+ case $1 in
+ aarch64)
+ # We don't have any bigendian build tools so we only use this for AArch64
+ container_image=debian-arm64-cross
+ container_cross_cc=aarch64-linux-gnu-gcc-10
+ ;;
+ alpha)
+ container_image=debian-alpha-cross
+ container_cross_cc=alpha-linux-gnu-gcc
+ ;;
+ arm)
+ # We don't have any bigendian build tools so we only use this for ARM
+ container_image=debian-armhf-cross
+ container_cross_cc=arm-linux-gnueabihf-gcc
+ ;;
+ cris)
+ container_image=fedora-cris-cross
+ container_cross_cc=cris-linux-gnu-gcc
+ ;;
+ hexagon)
+ container_image=debian-hexagon-cross
+ container_cross_cc=hexagon-unknown-linux-musl-clang
+ ;;
+ hppa)
+ container_image=debian-hppa-cross
+ container_cross_cc=hppa-linux-gnu-gcc
+ ;;
+ i386)
+ container_image=fedora-i386-cross
+ container_cross_cc=gcc
+ ;;
+ m68k)
+ container_image=debian-m68k-cross
+ container_cross_cc=m68k-linux-gnu-gcc
+ ;;
+ microblaze)
+ container_image=debian-microblaze-cross
+ container_cross_cc=microblaze-linux-musl-gcc
+ ;;
+ mips64el)
+ container_image=debian-mips64el-cross
+ container_cross_cc=mips64el-linux-gnuabi64-gcc
+ ;;
+ mips64)
+ container_image=debian-mips64-cross
+ container_cross_cc=mips64-linux-gnuabi64-gcc
+ ;;
+ mipsel)
+ container_image=debian-mipsel-cross
+ container_cross_cc=mipsel-linux-gnu-gcc
+ ;;
+ mips)
+ container_image=debian-mips-cross
+ container_cross_cc=mips-linux-gnu-gcc
+ ;;
+ nios2)
+ container_image=debian-nios2-cross
+ container_cross_cc=nios2-linux-gnu-gcc
+ ;;
+ ppc)
+ container_image=debian-powerpc-test-cross
+ container_cross_cc=powerpc-linux-gnu-gcc-10
+ ;;
+ ppc64|ppc64le)
+ container_image=debian-powerpc-test-cross
+ container_cross_cc=powerpc${1#ppc}-linux-gnu-gcc-10
+ ;;
+ riscv64)
+ container_image=debian-riscv64-test-cross
+ container_cross_cc=riscv64-linux-gnu-gcc
+ ;;
+ s390x)
+ container_image=debian-s390x-cross
+ container_cross_cc=s390x-linux-gnu-gcc
+ ;;
+ sh4)
+ container_image=debian-sh4-cross
+ container_cross_cc=sh4-linux-gnu-gcc
+ ;;
+ sparc64)
+ container_image=debian-sparc64-cross
+ container_cross_cc=sparc64-linux-gnu-gcc
+ ;;
+ tricore)
+ container_image=debian-tricore-cross
+ container_cross_as=tricore-as
+ container_cross_ld=tricore-ld
+ ;;
+ x86_64)
+ container_image=debian-amd64-cross
+ container_cross_cc=x86_64-linux-gnu-gcc
+ ;;
+ xtensa*)
+ # FIXME: xtensa-linux-user?
+ container_hosts=x86_64
+ container_image=debian-xtensa-cross
+
+ # default to the dc232b cpu
+ container_cross_cc=/opt/2020.07/xtensa-dc232b-elf/bin/xtensa-dc232b-elf-gcc
+ ;;
+ esac
+ done
+
+ eval "target_cflags=\${cross_cc_cflags_$1}"
+ if eval test -n "\"\${cross_cc_$1}\""; then
+ if eval has "\"\${cross_cc_$1}\""; then
+ eval "target_cc=\"\${cross_cc_$1}\""
+ case $1 in
+ i386|x86_64)
+ if $target_cc --version | grep -qi "clang"; then
+ unset target_cc
+ fi
+ ;;
+ esac
+ fi
+ fi
+ if eval test -n "\"\${cross_as_$1}\""; then
+ if eval has "\"\${cross_as_$1}\""; then
+ eval "target_as=\"\${cross_as_$1}\""
+ fi
+ fi
+ if eval test -n "\"\${cross_ld_$1}\""; then
+ if eval has "\"\${cross_ld_$1}\""; then
+ eval "target_ld=\"\${cross_ld_$1}\""
+ fi
+ fi
+}
+
+write_target_makefile() {
+ if test -n "$target_cc"; then
+ echo "CC=$target_cc"
+ fi
+ if test -n "$target_as"; then
+ echo "AS=$target_as"
+ fi
+ if test -n "$target_ld"; then
+ echo "LD=$target_ld"
+ fi
+}
+
+write_container_target_makefile() {
+ if test -n "$container_cross_cc"; then
+ echo "CC=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --"
+ fi
+ if test -n "$container_cross_as"; then
+ echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_as -i qemu/$container_image -s $source_path --"
+ fi
+ if test -n "$container_cross_ld"; then
+ echo "LD=\$(DOCKER_SCRIPT) cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --"
+ fi
+}
+
+
+
##########################################
# End of CC checks
# After here, no more $cc or $ld runs
@@ -2104,11 +2357,136 @@ for f in $LINKS ; do
fi
done
-(for i in $cross_cc_vars; do
- export $i
+# tests/tcg configuration
+(makefile=tests/tcg/Makefile.prereqs
+echo "# Automatically generated by configure - do not modify" > $makefile
+
+config_host_mak=tests/tcg/config-host.mak
+echo "# Automatically generated by configure - do not modify" > $config_host_mak
+echo "SRC_PATH=$source_path" >> $config_host_mak
+echo "HOST_CC=$host_cc" >> $config_host_mak
+
+tcg_tests_targets=
+for target in $target_list; do
+ arch=${target%%-*}
+
+ probe_target_compiler ${arch}
+ config_target_mak=tests/tcg/config-$target.mak
+
+ echo "# Automatically generated by configure - do not modify" > $config_target_mak
+ echo "TARGET_NAME=$arch" >> $config_target_mak
+ case $target in
+ *-softmmu)
+ test -f $source_path/tests/tcg/$arch/Makefile.softmmu-target || continue
+ qemu="qemu-system-$arch"
+ ;;
+ *-linux-user|*-bsd-user)
+ qemu="qemu-$arch"
+ ;;
+ esac
+
+ got_cross_cc=no
+ unset build_static
+
+ if test -n "$target_cc"; then
+ write_c_skeleton
+ if ! do_compiler "$target_cc" $target_cflags \
+ -o $TMPE $TMPC -static ; then
+ # For host systems we might get away with building without -static
+ if do_compiler "$target_cc" $target_cflags \
+ -o $TMPE $TMPC ; then
+ got_cross_cc=yes
+ fi
+ else
+ got_cross_cc=yes
+ build_static=y
+ fi
+ elif test -n "$target_as" && test -n "$target_ld"; then
+ # Special handling for assembler only tests
+ case $target in
+ tricore-softmmu) got_cross_cc=yes ;;
+ esac
+ fi
+
+ if test $got_cross_cc = yes; then
+ # Test for compiler features for optional tests. We only do this
+ # for cross compilers because ensuring the docker containers based
+ # compilers is a requirememt for adding a new test that needs a
+ # compiler feature.
+
+ echo "BUILD_STATIC=$build_static" >> $config_target_mak
+ write_target_makefile >> $config_target_mak
+ case $target in
+ aarch64-*)
+ if do_compiler "$target_cc" $target_cflags \
+ -march=armv8.1-a+sve -o $TMPE $TMPC; then
+ echo "CROSS_CC_HAS_SVE=y" >> $config_target_mak
+ fi
+ if do_compiler "$target_cc" $target_cflags \
+ -march=armv8.1-a+sve2 -o $TMPE $TMPC; then
+ echo "CROSS_CC_HAS_SVE2=y" >> $config_target_mak
+ fi
+ if do_compiler "$target_cc" $target_cflags \
+ -march=armv8.3-a -o $TMPE $TMPC; then
+ echo "CROSS_CC_HAS_ARMV8_3=y" >> $config_target_mak
+ fi
+ if do_compiler "$target_cc" $target_cflags \
+ -mbranch-protection=standard -o $TMPE $TMPC; then
+ echo "CROSS_CC_HAS_ARMV8_BTI=y" >> $config_target_mak
+ fi
+ if do_compiler "$target_cc" $target_cflags \
+ -march=armv8.5-a+memtag -o $TMPE $TMPC; then
+ echo "CROSS_CC_HAS_ARMV8_MTE=y" >> $config_target_mak
+ fi
+ ;;
+ ppc*)
+ if do_compiler "$target_cc" $target_cflags \
+ -mpower8-vector -o $TMPE $TMPC; then
+ echo "CROSS_CC_HAS_POWER8_VECTOR=y" >> $config_target_mak
+ fi
+ if do_compiler "$target_cc" $target_cflags \
+ -mpower10 -o $TMPE $TMPC; then
+ echo "CROSS_CC_HAS_POWER10=y" >> $config_target_mak
+ fi
+ ;;
+ i386-linux-user)
+ if do_compiler "$target_cc" $target_cflags \
+ -Werror -fno-pie -o $TMPE $TMPC; then
+ echo "CROSS_CC_HAS_I386_NOPIE=y" >> $config_target_mak
+ fi
+ ;;
+ esac
+ elif test -n "$container_image"; then
+ echo "build-tcg-tests-$target: docker-image-$container_image" >> $makefile
+ echo "BUILD_STATIC=y" >> $config_target_mak
+ write_container_target_makefile >> $config_target_mak
+ case $target in
+ aarch64-*)
+ echo "CROSS_CC_HAS_SVE=y" >> $config_target_mak
+ echo "CROSS_CC_HAS_SVE2=y" >> $config_target_mak
+ echo "CROSS_CC_HAS_ARMV8_3=y" >> $config_target_mak
+ echo "CROSS_CC_HAS_ARMV8_BTI=y" >> $config_target_mak
+ echo "CROSS_CC_HAS_ARMV8_MTE=y" >> $config_target_mak
+ ;;
+ ppc*)
+ echo "CROSS_CC_HAS_POWER8_VECTOR=y" >> $config_target_mak
+ echo "CROSS_CC_HAS_POWER10=y" >> $config_target_mak
+ ;;
+ i386-linux-user)
+ echo "CROSS_CC_HAS_I386_NOPIE=y" >> $config_target_mak
+ ;;
+ esac
+ got_cross_cc=yes
+ fi
+ if test $got_cross_cc = yes; then
+ mkdir -p tests/tcg/$target
+ echo "QEMU=$PWD/$qemu" >> $config_target_mak
+ echo "EXTRA_CFLAGS=$target_cflags" >> $config_target_mak
+ echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> $makefile
+ tcg_tests_targets="$tcg_tests_targets $target"
+ fi
done
-export target_list source_path use_containers cpu host_cc
-$source_path/tests/tcg/configure.sh)
+echo "TCG_TESTS_TARGETS=$tcg_tests_targets" >> $makefile)
config_mak=pc-bios/optionrom/config.mak
echo "# Automatically generated by configure - do not modify" > $config_mak
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 72ce0561f4..6a1688e33e 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -37,7 +37,6 @@ export SRC_PATH
SPEED = quick
-include tests/tcg/Makefile.prereqs
-config-host.mak: $(SRC_PATH)/tests/tcg/configure.sh
tests/tcg/Makefile.prereqs: config-host.mak
# Per guest TCG tests
diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh
deleted file mode 100755
index 59f2403d1a..0000000000
--- a/tests/tcg/configure.sh
+++ /dev/null
@@ -1,376 +0,0 @@
-#! /bin/sh
-
-if test -z "$source_path"; then
- echo Do not invoke this script directly. It is called
- echo automatically by configure.
- exit 1
-fi
-
-write_c_skeleton() {
- cat > $TMPC <<EOF
-int main(void) { return 0; }
-EOF
-}
-
-has() {
- command -v "$1" >/dev/null 2>&1
-}
-
-do_compiler() {
- # Run the compiler, capturing its output to the log. First argument
- # is compiler binary to execute.
- local compiler="$1"
- shift
- if test -n "$BASH_VERSION"; then eval '
- echo >>config.log "
-funcs: ${FUNCNAME[*]}
-lines: ${BASH_LINENO[*]}"
- '; fi
- echo $compiler "$@" >> config.log
- $compiler "$@" >> config.log 2>&1 || return $?
-}
-
-
-TMPDIR1="config-temp"
-TMPC="${TMPDIR1}/qemu-conf.c"
-TMPE="${TMPDIR1}/qemu-conf.exe"
-
-container="no"
-if test $use_containers = "yes"; then
- if has "docker" || has "podman"; then
- container=$($python $source_path/tests/docker/docker.py probe)
- fi
-fi
-
-# cross compilers defaults, can be overridden with --cross-cc-ARCH
-: ${cross_cc_aarch64="aarch64-linux-gnu-gcc"}
-: ${cross_cc_aarch64_be="$cross_cc_aarch64"}
-: ${cross_cc_cflags_aarch64_be="-mbig-endian"}
-: ${cross_cc_alpha="alpha-linux-gnu-gcc"}
-: ${cross_cc_arm="arm-linux-gnueabihf-gcc"}
-: ${cross_cc_cflags_armeb="-mbig-endian"}
-: ${cross_cc_hexagon="hexagon-unknown-linux-musl-clang"}
-: ${cross_cc_cflags_hexagon="-mv67 -O2 -static"}
-: ${cross_cc_hppa="hppa-linux-gnu-gcc"}
-: ${cross_cc_i386="i686-linux-gnu-gcc"}
-: ${cross_cc_cflags_i386="-m32"}
-: ${cross_cc_m68k="m68k-linux-gnu-gcc"}
-: ${cross_cc_microblaze="microblaze-linux-musl-gcc"}
-: ${cross_cc_mips64el="mips64el-linux-gnuabi64-gcc"}
-: ${cross_cc_mips64="mips64-linux-gnuabi64-gcc"}
-: ${cross_cc_mipsel="mipsel-linux-gnu-gcc"}
-: ${cross_cc_mips="mips-linux-gnu-gcc"}
-: ${cross_cc_nios2="nios2-linux-gnu-gcc"}
-: ${cross_cc_ppc="powerpc-linux-gnu-gcc"}
-: ${cross_cc_cflags_ppc="-m32"}
-: ${cross_cc_ppc64="powerpc64-linux-gnu-gcc"}
-: ${cross_cc_cflags_ppc64="-m64 -mbig-endian"}
-: ${cross_cc_ppc64le="$cross_cc_ppc64"}
-: ${cross_cc_cflags_ppc64le="-m64 -mlittle-endian"}
-: ${cross_cc_riscv64="riscv64-linux-gnu-gcc"}
-: ${cross_cc_s390x="s390x-linux-gnu-gcc"}
-: ${cross_cc_sh4="sh4-linux-gnu-gcc"}
-: ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
-: ${cross_cc_sparc64="sparc64-linux-gnu-gcc"}
-: ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
-: ${cross_cc_x86_64="x86_64-linux-gnu-gcc"}
-: ${cross_cc_cflags_x86_64="-m64"}
-
-# tricore is special as it doesn't have a compiler
-: ${cross_as_tricore="tricore-as"}
-: ${cross_ld_tricore="tricore-ld"}
-
-makefile=tests/tcg/Makefile.prereqs
-echo "# Automatically generated by configure - do not modify" > $makefile
-
-config_host_mak=tests/tcg/config-host.mak
-echo "# Automatically generated by configure - do not modify" > $config_host_mak
-echo "SRC_PATH=$source_path" >> $config_host_mak
-echo "HOST_CC=$host_cc" >> $config_host_mak
-
-tcg_tests_targets=
-for target in $target_list; do
- arch=${target%%-*}
-
- # reset all container fields
- container_image=
- container_hosts=
- container_cross_cc=
- container_cross_as=
- container_cross_ld=
-
- # suppress clang
- supress_clang=
-
- case $target in
- aarch64-*)
- # We don't have any bigendian build tools so we only use this for AArch64
- container_hosts="x86_64 aarch64"
- container_image=debian-arm64-cross
- container_cross_cc=aarch64-linux-gnu-gcc-10
- ;;
- alpha-*)
- container_hosts=x86_64
- container_image=debian-alpha-cross
- container_cross_cc=alpha-linux-gnu-gcc
- ;;
- arm-*)
- # We don't have any bigendian build tools so we only use this for ARM
- container_hosts="x86_64 aarch64"
- container_image=debian-armhf-cross
- container_cross_cc=arm-linux-gnueabihf-gcc
- ;;
- cris-*)
- container_hosts=x86_64
- container_image=fedora-cris-cross
- container_cross_cc=cris-linux-gnu-gcc
- ;;
- hexagon-*)
- container_hosts=x86_64
- container_image=debian-hexagon-cross
- container_cross_cc=hexagon-unknown-linux-musl-clang
- ;;
- hppa-*)
- container_hosts=x86_64
- container_image=debian-hppa-cross
- container_cross_cc=hppa-linux-gnu-gcc
- ;;
- i386-*)
- container_hosts=x86_64
- container_image=fedora-i386-cross
- container_cross_cc=gcc
- supress_clang=yes
- ;;
- m68k-*)
- container_hosts=x86_64
- container_image=debian-m68k-cross
- container_cross_cc=m68k-linux-gnu-gcc
- ;;
- microblaze-*)
- container_hosts=x86_64
- container_image=debian-microblaze-cross
- container_cross_cc=microblaze-linux-musl-gcc
- ;;
- mips64el-*)
- container_hosts=x86_64
- container_image=debian-mips64el-cross
- container_cross_cc=mips64el-linux-gnuabi64-gcc
- ;;
- mips64-*)
- container_hosts=x86_64
- container_image=debian-mips64-cross
- container_cross_cc=mips64-linux-gnuabi64-gcc
- ;;
- mipsel-*)
- container_hosts=x86_64
- container_image=debian-mipsel-cross
- container_cross_cc=mipsel-linux-gnu-gcc
- ;;
- mips-*)
- container_hosts=x86_64
- container_image=debian-mips-cross
- container_cross_cc=mips-linux-gnu-gcc
- ;;
- nios2-*)
- container_hosts=x86_64
- container_image=debian-nios2-cross
- container_cross_cc=nios2-linux-gnu-gcc
- ;;
- ppc-*)
- container_hosts=x86_64
- container_image=debian-powerpc-test-cross
- container_cross_cc=powerpc-linux-gnu-gcc-10
- ;;
- ppc64-*|ppc64le-*)
- container_hosts=x86_64
- container_image=debian-powerpc-test-cross
- container_cross_cc=${target%%-*}-linux-gnu-gcc-10
- container_cross_cc=powerpc${container_cross_cc#ppc}
- ;;
- riscv64-*)
- container_hosts=x86_64
- container_image=debian-riscv64-test-cross
- container_cross_cc=riscv64-linux-gnu-gcc
- ;;
- s390x-*)
- container_hosts=x86_64
- container_image=debian-s390x-cross
- container_cross_cc=s390x-linux-gnu-gcc
- ;;
- sh4-*)
- container_hosts=x86_64
- container_image=debian-sh4-cross
- container_cross_cc=sh4-linux-gnu-gcc
- ;;
- sparc64-*)
- container_hosts=x86_64
- container_image=debian-sparc64-cross
- container_cross_cc=sparc64-linux-gnu-gcc
- ;;
- tricore-softmmu)
- container_hosts=x86_64
- container_image=debian-tricore-cross
- container_cross_as=tricore-as
- container_cross_ld=tricore-ld
- ;;
- x86_64-*)
- container_hosts="aarch64 ppc64el x86_64"
- container_image=debian-amd64-cross
- container_cross_cc=x86_64-linux-gnu-gcc
- supress_clang=yes
- ;;
- xtensa*-softmmu)
- container_hosts=x86_64
- container_image=debian-xtensa-cross
-
- # default to the dc232b cpu
- container_cross_cc=/opt/2020.07/xtensa-dc232b-elf/bin/xtensa-dc232b-elf-gcc
- ;;
- esac
-
- config_target_mak=tests/tcg/config-$target.mak
-
- echo "# Automatically generated by configure - do not modify" > $config_target_mak
- echo "TARGET_NAME=$arch" >> $config_target_mak
- case $target in
- *-softmmu)
- test -f $source_path/tests/tcg/$arch/Makefile.softmmu-target || continue
- qemu="qemu-system-$arch"
- ;;
- *-linux-user|*-bsd-user)
- qemu="qemu-$arch"
- ;;
- esac
-
- eval "target_compiler_cflags=\${cross_cc_cflags_$arch}"
-
- got_cross_cc=no
-
- if eval test "x\"\${cross_cc_$arch}\"" != xyes; then
- eval "target_compiler=\"\${cross_cc_$arch}\""
-
- if has $target_compiler; then
- if test "$supress_clang" = yes &&
- $target_compiler --version | grep -qi "clang"; then
- got_cross_cc=no
- else
- write_c_skeleton
- if ! do_compiler "$target_compiler" $target_compiler_cflags \
- -o $TMPE $TMPC -static ; then
- # For host systems we might get away with building without -static
- if do_compiler "$target_compiler" $target_compiler_cflags \
- -o $TMPE $TMPC ; then
- got_cross_cc=yes
- echo "CC=$target_compiler" >> $config_target_mak
- fi
- else
- got_cross_cc=yes
- echo "BUILD_STATIC=y" >> $config_target_mak
- echo "CC=$target_compiler" >> $config_target_mak
- fi
- fi
- fi
-
- # Special handling for assembler only tests
- eval "target_as=\"\${cross_as_$arch}\""
- eval "target_ld=\"\${cross_ld_$arch}\""
- if has $target_as && has $target_ld; then
- case $target in
- tricore-softmmu)
- echo "AS=$target_as" >> $config_target_mak
- echo "LD=$target_ld" >> $config_target_mak
- got_cross_cc=yes
- ;;
- esac
- fi
- fi
-
- if test $got_cross_cc = yes; then
- # Test for compiler features for optional tests. We only do this
- # for cross compilers because ensuring the docker containers based
- # compilers is a requirememt for adding a new test that needs a
- # compiler feature.
-
- case $target in
- aarch64-*)
- if do_compiler "$target_compiler" $target_compiler_cflags \
- -march=armv8.1-a+sve -o $TMPE $TMPC; then
- echo "CROSS_CC_HAS_SVE=y" >> $config_target_mak
- fi
- if do_compiler "$target_compiler" $target_compiler_cflags \
- -march=armv8.1-a+sve2 -o $TMPE $TMPC; then
- echo "CROSS_CC_HAS_SVE2=y" >> $config_target_mak
- fi
- if do_compiler "$target_compiler" $target_compiler_cflags \
- -march=armv8.3-a -o $TMPE $TMPC; then
- echo "CROSS_CC_HAS_ARMV8_3=y" >> $config_target_mak
- fi
- if do_compiler "$target_compiler" $target_compiler_cflags \
- -mbranch-protection=standard -o $TMPE $TMPC; then
- echo "CROSS_CC_HAS_ARMV8_BTI=y" >> $config_target_mak
- fi
- if do_compiler "$target_compiler" $target_compiler_cflags \
- -march=armv8.5-a+memtag -o $TMPE $TMPC; then
- echo "CROSS_CC_HAS_ARMV8_MTE=y" >> $config_target_mak
- fi
- ;;
- ppc*)
- if do_compiler "$target_compiler" $target_compiler_cflags \
- -mpower8-vector -o $TMPE $TMPC; then
- echo "CROSS_CC_HAS_POWER8_VECTOR=y" >> $config_target_mak
- fi
- if do_compiler "$target_compiler" $target_compiler_cflags \
- -mpower10 -o $TMPE $TMPC; then
- echo "CROSS_CC_HAS_POWER10=y" >> $config_target_mak
- fi
- ;;
- i386-linux-user)
- if do_compiler "$target_compiler" $target_compiler_cflags \
- -Werror -fno-pie -o $TMPE $TMPC; then
- echo "CROSS_CC_HAS_I386_NOPIE=y" >> $config_target_mak
- fi
- ;;
- esac
- elif test $got_cross_cc = no && test "$container" != no && \
- test -n "$container_image"; then
- for host in $container_hosts; do
- if test "$host" = "$cpu"; then
- echo "build-tcg-tests-$target: docker-image-$container_image" >> $makefile
- echo "BUILD_STATIC=y" >> $config_target_mak
- echo "CC=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --" >> $config_target_mak
- if test -n "$container_cross_as"; then
- echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_as -i qemu/$container_image -s $source_path --" >> $config_target_mak
- fi
- if test -n "$container_cross_ld"; then
- echo "LD=\$(DOCKER_SCRIPT) cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --" >> $config_target_mak
- fi
- case $target in
- aarch64-*)
- echo "CROSS_CC_HAS_SVE=y" >> $config_target_mak
- echo "CROSS_CC_HAS_SVE2=y" >> $config_target_mak
- echo "CROSS_CC_HAS_ARMV8_3=y" >> $config_target_mak
- echo "CROSS_CC_HAS_ARMV8_BTI=y" >> $config_target_mak
- echo "CROSS_CC_HAS_ARMV8_MTE=y" >> $config_target_mak
- ;;
- ppc*)
- echo "CROSS_CC_HAS_POWER8_VECTOR=y" >> $config_target_mak
- echo "CROSS_CC_HAS_POWER10=y" >> $config_target_mak
- ;;
- i386-linux-user)
- echo "CROSS_CC_HAS_I386_NOPIE=y" >> $config_target_mak
- ;;
- esac
- got_cross_cc=yes
- break
- fi
- done
- fi
- if test $got_cross_cc = yes; then
- mkdir -p tests/tcg/$target
- echo "QEMU=$PWD/$qemu" >> $config_target_mak
- echo "EXTRA_CFLAGS=$target_compiler_cflags" >> $config_target_mak
- echo "run-tcg-tests-$target: $qemu\$(EXESUF)" >> $makefile
- tcg_tests_targets="$tcg_tests_targets $target"
- fi
-done
-echo "TCG_TESTS_TARGETS=$tcg_tests_targets" >> $makefile
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 19/33] configure: add missing cross compiler fallbacks
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (17 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 18/33] tests/tcg: merge configure.sh back into main configure script Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 20/33] configure: handle host compiler in probe_target_compiler Alex Bennée
` (13 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
The arm compiler can be used for armeb, and the sparc64 compiler
can be used for sparc.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-9-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/configure b/configure
index bb05e70bcc..31c1ab2579 100755
--- a/configure
+++ b/configure
@@ -1822,6 +1822,7 @@ fi
: ${cross_cc_cflags_aarch64_be="-mbig-endian"}
: ${cross_cc_alpha="alpha-linux-gnu-gcc"}
: ${cross_cc_arm="arm-linux-gnueabihf-gcc"}
+: ${cross_cc_armeb="$cross_cc_arm"}
: ${cross_cc_cflags_armeb="-mbig-endian"}
: ${cross_cc_hexagon="hexagon-unknown-linux-musl-clang"}
: ${cross_cc_cflags_hexagon="-mv67 -O2 -static"}
@@ -1844,9 +1845,10 @@ fi
: ${cross_cc_riscv64="riscv64-linux-gnu-gcc"}
: ${cross_cc_s390x="s390x-linux-gnu-gcc"}
: ${cross_cc_sh4="sh4-linux-gnu-gcc"}
-: ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
: ${cross_cc_sparc64="sparc64-linux-gnu-gcc"}
: ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
+: ${cross_cc_sparc="$cross_cc_sparc64"}
+: ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
: ${cross_cc_x86_64="x86_64-linux-gnu-gcc"}
: ${cross_cc_cflags_x86_64="-m64"}
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 20/33] configure: handle host compiler in probe_target_compiler
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (18 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 19/33] configure: add missing cross compiler fallbacks Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 21/33] configure: introduce --cross-prefix-*= Alex Bennée
` (12 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
In preparation for handling more binaries than just cc, handle
the case of "probe_target_compiler $cpu" directly in the function,
setting the target_* variables based on the ones that are used to
build QEMU. The clang check also needs to be moved after this
fallback.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220517092616.1272238-10-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 25 ++++++++++++++-----------
1 file changed, 14 insertions(+), 11 deletions(-)
diff --git a/configure b/configure
index 31c1ab2579..addbb0fe44 100755
--- a/configure
+++ b/configure
@@ -954,10 +954,6 @@ case $git_submodules_action in
;;
esac
-if eval test -z "\${cross_cc_$cpu}"; then
- eval "cross_cc_${cpu}=\$cc"
-fi
-
default_target_list=""
mak_wilds=""
@@ -2003,13 +1999,6 @@ probe_target_compiler() {
if eval test -n "\"\${cross_cc_$1}\""; then
if eval has "\"\${cross_cc_$1}\""; then
eval "target_cc=\"\${cross_cc_$1}\""
- case $1 in
- i386|x86_64)
- if $target_cc --version | grep -qi "clang"; then
- unset target_cc
- fi
- ;;
- esac
fi
fi
if eval test -n "\"\${cross_as_$1}\""; then
@@ -2022,6 +2011,20 @@ probe_target_compiler() {
eval "target_ld=\"\${cross_ld_$1}\""
fi
fi
+ if test "$1" = $cpu; then
+ : ${target_cc:=$cc}
+ : ${target_as:=$as}
+ : ${target_ld:=$ld}
+ fi
+ if test -n "$target_cc"; then
+ case $1 in
+ i386|x86_64)
+ if $target_cc --version | grep -qi "clang"; then
+ unset target_cc
+ fi
+ ;;
+ esac
+ fi
}
write_target_makefile() {
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 21/33] configure: introduce --cross-prefix-*=
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (19 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 20/33] configure: handle host compiler in probe_target_compiler Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 22/33] configure: include more binutils in tests/tcg makefile Alex Bennée
` (11 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
Also in preparation for handling more binaries from the cross binutils,
support an option --cross-prefix-ARCH. All cross_cc_* defaults are
replaced with cross_prefix_*; the cross_cc_* fallbacks are extended
to the cross-compilation prefix, but the compiler fallbacks remain
as well. This way, for example, --cross-cc-arm=arm-linux-gnueabihf-clang
also applies to armeb binaries.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-11-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 137 ++++++++++++++++++++++++++++++------------------------
1 file changed, 77 insertions(+), 60 deletions(-)
diff --git a/configure b/configure
index addbb0fe44..c2b16c17b9 100755
--- a/configure
+++ b/configure
@@ -363,6 +363,11 @@ for opt do
--cross-cc-*) cc_arch=${opt#--cross-cc-}; cc_arch=${cc_arch%%=*}
eval "cross_cc_${cc_arch}=\$optarg"
;;
+ --cross-prefix-*[!a-zA-Z0-9_-]*=*) error_exit "Passed bad --cross-prefix-FOO option"
+ ;;
+ --cross-prefix-*) cc_arch=${opt#--cross-prefix-}; cc_arch=${cc_arch%%=*}
+ eval "cross_prefix_${cc_arch}=\$optarg"
+ ;;
esac
done
# OS specific
@@ -728,6 +733,8 @@ for opt do
;;
--cross-cc-*)
;;
+ --cross-prefix-*)
+ ;;
--enable-debug-info) meson_option_add -Ddebug=true
;;
--disable-debug-info) meson_option_add -Ddebug=false
@@ -1016,6 +1023,7 @@ Advanced options (experts only):
--extra-ldflags=LDFLAGS append extra linker flags LDFLAGS
--cross-cc-ARCH=CC use compiler when building ARCH guest test cases
--cross-cc-cflags-ARCH= use compiler flags when building ARCH guest tests
+ --cross-prefix-ARCH=PREFIX cross compiler prefix when building ARCH guest test cases
--make=MAKE use specified make [$make]
--python=PYTHON use specified python [$python]
--meson=MESON use specified meson [$meson]
@@ -1813,44 +1821,54 @@ if test $use_containers = "yes"; then
fi
# cross compilers defaults, can be overridden with --cross-cc-ARCH
-: ${cross_cc_aarch64="aarch64-linux-gnu-gcc"}
+: ${cross_prefix_aarch64="aarch64-linux-gnu-"}
+: ${cross_prefix_aarch64_be="$cross_prefix_aarch64"}
+: ${cross_prefix_alpha="alpha-linux-gnu-"}
+: ${cross_prefix_arm="arm-linux-gnueabihf-"}
+: ${cross_prefix_armeb="$cross_prefix_arm"}
+: ${cross_prefix_hexagon="hexagon-unknown-linux-musl-"}
+: ${cross_prefix_hppa="hppa-linux-gnu-"}
+: ${cross_prefix_i386="i686-linux-gnu-"}
+: ${cross_prefix_m68k="m68k-linux-gnu-"}
+: ${cross_prefix_microblaze="microblaze-linux-musl-"}
+: ${cross_prefix_mips64el="mips64el-linux-gnuabi64-"}
+: ${cross_prefix_mips64="mips64-linux-gnuabi64-"}
+: ${cross_prefix_mipsel="mipsel-linux-gnu-"}
+: ${cross_prefix_mips="mips-linux-gnu-"}
+: ${cross_prefix_nios2="nios2-linux-gnu-"}
+: ${cross_prefix_ppc="powerpc-linux-gnu-"}
+: ${cross_prefix_ppc64="powerpc64-linux-gnu-"}
+: ${cross_prefix_ppc64le="$cross_prefix_ppc64"}
+: ${cross_prefix_riscv64="riscv64-linux-gnu-"}
+: ${cross_prefix_s390x="s390x-linux-gnu-"}
+: ${cross_prefix_sh4="sh4-linux-gnu-"}
+: ${cross_prefix_sparc64="sparc64-linux-gnu-"}
+: ${cross_prefix_sparc="$cross_prefix_sparc64"}
+: ${cross_prefix_x86_64="x86_64-linux-gnu-"}
+
: ${cross_cc_aarch64_be="$cross_cc_aarch64"}
: ${cross_cc_cflags_aarch64_be="-mbig-endian"}
-: ${cross_cc_alpha="alpha-linux-gnu-gcc"}
-: ${cross_cc_arm="arm-linux-gnueabihf-gcc"}
: ${cross_cc_armeb="$cross_cc_arm"}
: ${cross_cc_cflags_armeb="-mbig-endian"}
: ${cross_cc_hexagon="hexagon-unknown-linux-musl-clang"}
: ${cross_cc_cflags_hexagon="-mv67 -O2 -static"}
-: ${cross_cc_hppa="hppa-linux-gnu-gcc"}
-: ${cross_cc_i386="i686-linux-gnu-gcc"}
: ${cross_cc_cflags_i386="-m32"}
-: ${cross_cc_m68k="m68k-linux-gnu-gcc"}
-: ${cross_cc_microblaze="microblaze-linux-musl-gcc"}
-: ${cross_cc_mips64el="mips64el-linux-gnuabi64-gcc"}
-: ${cross_cc_mips64="mips64-linux-gnuabi64-gcc"}
-: ${cross_cc_mipsel="mipsel-linux-gnu-gcc"}
-: ${cross_cc_mips="mips-linux-gnu-gcc"}
-: ${cross_cc_nios2="nios2-linux-gnu-gcc"}
-: ${cross_cc_ppc="powerpc-linux-gnu-gcc"}
: ${cross_cc_cflags_ppc="-m32"}
-: ${cross_cc_ppc64="powerpc64-linux-gnu-gcc"}
: ${cross_cc_cflags_ppc64="-m64 -mbig-endian"}
: ${cross_cc_ppc64le="$cross_cc_ppc64"}
: ${cross_cc_cflags_ppc64le="-m64 -mlittle-endian"}
-: ${cross_cc_riscv64="riscv64-linux-gnu-gcc"}
-: ${cross_cc_s390x="s390x-linux-gnu-gcc"}
-: ${cross_cc_sh4="sh4-linux-gnu-gcc"}
-: ${cross_cc_sparc64="sparc64-linux-gnu-gcc"}
: ${cross_cc_cflags_sparc64="-m64 -mcpu=ultrasparc"}
: ${cross_cc_sparc="$cross_cc_sparc64"}
: ${cross_cc_cflags_sparc="-m32 -mcpu=supersparc"}
-: ${cross_cc_x86_64="x86_64-linux-gnu-gcc"}
: ${cross_cc_cflags_x86_64="-m64"}
-# tricore is special as it doesn't have a compiler
-: ${cross_as_tricore="tricore-as"}
-: ${cross_ld_tricore="tricore-ld"}
+compute_target_variable() {
+ if eval test -n "\"\${cross_prefix_$1}\""; then
+ if eval has "\"\${cross_prefix_$1}\$3\""; then
+ eval "$2=\"\${cross_prefix_$1}\$3\""
+ fi
+ fi
+}
probe_target_compiler() {
# reset all output variables
@@ -1896,93 +1914,99 @@ probe_target_compiler() {
aarch64)
# We don't have any bigendian build tools so we only use this for AArch64
container_image=debian-arm64-cross
- container_cross_cc=aarch64-linux-gnu-gcc-10
+ container_cross_prefix=aarch64-linux-gnu-
+ container_cross_cc=${container_cross_prefix}gcc-10
;;
alpha)
container_image=debian-alpha-cross
- container_cross_cc=alpha-linux-gnu-gcc
+ container_cross_prefix=alpha-linux-gnu-
;;
arm)
# We don't have any bigendian build tools so we only use this for ARM
container_image=debian-armhf-cross
- container_cross_cc=arm-linux-gnueabihf-gcc
+ container_cross_prefix=arm-linux-gnueabihf-
;;
cris)
container_image=fedora-cris-cross
- container_cross_cc=cris-linux-gnu-gcc
+ container_cross_prefix=cris-linux-gnu-
;;
hexagon)
container_image=debian-hexagon-cross
- container_cross_cc=hexagon-unknown-linux-musl-clang
+ container_cross_prefix=hexagon-unknown-linux-musl-
+ container_cross_cc=${container_cross_prefix}clang
;;
hppa)
container_image=debian-hppa-cross
- container_cross_cc=hppa-linux-gnu-gcc
+ container_cross_prefix=hppa-linux-gnu-
;;
i386)
container_image=fedora-i386-cross
- container_cross_cc=gcc
+ container_cross_prefix=
;;
m68k)
container_image=debian-m68k-cross
- container_cross_cc=m68k-linux-gnu-gcc
+ container_cross_prefix=m68k-linux-gnu-
;;
microblaze)
container_image=debian-microblaze-cross
- container_cross_cc=microblaze-linux-musl-gcc
+ container_cross_prefix=microblaze-linux-musl-
;;
mips64el)
container_image=debian-mips64el-cross
- container_cross_cc=mips64el-linux-gnuabi64-gcc
+ container_cross_prefix=mips64el-linux-gnuabi64-
;;
mips64)
container_image=debian-mips64-cross
- container_cross_cc=mips64-linux-gnuabi64-gcc
+ container_cross_prefix=mips64-linux-gnuabi64-
;;
mipsel)
container_image=debian-mipsel-cross
- container_cross_cc=mipsel-linux-gnu-gcc
+ container_cross_prefix=mipsel-linux-gnu-
;;
mips)
container_image=debian-mips-cross
- container_cross_cc=mips-linux-gnu-gcc
+ container_cross_prefix=mips-linux-gnu-
;;
nios2)
container_image=debian-nios2-cross
- container_cross_cc=nios2-linux-gnu-gcc
+ container_cross_prefix=nios2-linux-gnu-
;;
ppc)
container_image=debian-powerpc-test-cross
- container_cross_cc=powerpc-linux-gnu-gcc-10
+ container_cross_prefix=powerpc-linux-gnu-
+ container_cross_cc=${container_cross_prefix}gcc-10
;;
ppc64|ppc64le)
container_image=debian-powerpc-test-cross
- container_cross_cc=powerpc${1#ppc}-linux-gnu-gcc-10
+ container_cross_prefix=powerpc${1#ppc}-linux-gnu-
+ container_cross_cc=${container_cross_prefix}gcc-10
;;
riscv64)
container_image=debian-riscv64-test-cross
- container_cross_cc=riscv64-linux-gnu-gcc
+ container_cross_prefix=riscv64-linux-gnu-
;;
s390x)
container_image=debian-s390x-cross
- container_cross_cc=s390x-linux-gnu-gcc
+ container_cross_prefix=s390x-linux-gnu-
;;
sh4)
container_image=debian-sh4-cross
- container_cross_cc=sh4-linux-gnu-gcc
+ container_cross_prefix=sh4-linux-gnu-
;;
sparc64)
container_image=debian-sparc64-cross
- container_cross_cc=sparc64-linux-gnu-gcc
+ container_cross_prefix=sparc64-linux-gnu-
;;
tricore)
container_image=debian-tricore-cross
+ container_cross_prefix=tricore-
container_cross_as=tricore-as
container_cross_ld=tricore-ld
+ break
;;
x86_64)
container_image=debian-amd64-cross
- container_cross_cc=x86_64-linux-gnu-gcc
+ container_cross_prefix=x86_64-linux-gnu-
;;
xtensa*)
# FIXME: xtensa-linux-user?
@@ -1990,9 +2014,12 @@ probe_target_compiler() {
container_image=debian-xtensa-cross
# default to the dc232b cpu
- container_cross_cc=/opt/2020.07/xtensa-dc232b-elf/bin/xtensa-dc232b-elf-gcc
+ container_cross_prefix=/opt/2020.07/xtensa-dc232b-elf/bin/xtensa-dc232b-elf-
;;
esac
+ : ${container_cross_cc:=${container_cross_prefix}gcc}
+ : ${container_cross_as:=${container_cross_prefix}as}
+ : ${container_cross_ld:=${container_cross_prefix}ld}
done
eval "target_cflags=\${cross_cc_cflags_$1}"
@@ -2000,17 +2027,11 @@ probe_target_compiler() {
if eval has "\"\${cross_cc_$1}\""; then
eval "target_cc=\"\${cross_cc_$1}\""
fi
+ else
+ compute_target_variable $1 target_cc gcc
fi
- if eval test -n "\"\${cross_as_$1}\""; then
- if eval has "\"\${cross_as_$1}\""; then
- eval "target_as=\"\${cross_as_$1}\""
- fi
- fi
- if eval test -n "\"\${cross_ld_$1}\""; then
- if eval has "\"\${cross_ld_$1}\""; then
- eval "target_ld=\"\${cross_ld_$1}\""
- fi
- fi
+ compute_target_variable $1 target_as as
+ compute_target_variable $1 target_ld ld
if test "$1" = $cpu; then
: ${target_cc:=$cc}
: ${target_as:=$as}
@@ -2043,12 +2064,8 @@ write_container_target_makefile() {
if test -n "$container_cross_cc"; then
echo "CC=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --"
fi
- if test -n "$container_cross_as"; then
- echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_as -i qemu/$container_image -s $source_path --"
- fi
- if test -n "$container_cross_ld"; then
- echo "LD=\$(DOCKER_SCRIPT) cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --"
- fi
+ echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_as -i qemu/$container_image -s $source_path --"
+ echo "LD=\$(DOCKER_SCRIPT) cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --"
}
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 22/33] configure: include more binutils in tests/tcg makefile
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (20 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 21/33] configure: introduce --cross-prefix-*= Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 23/33] configure: move symlink configuration earlier Alex Bennée
` (10 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
Firmware builds require paths to all the binutils; it is not enough to
use only cc, or even as/ld as in the case of tests/tcg/tricore.
Adjust the cross-compiler configurator to detect also ar, nm, objcopy,
ranlib and strip.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-12-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 51 insertions(+)
diff --git a/configure b/configure
index c2b16c17b9..b8c21e096c 100755
--- a/configure
+++ b/configure
@@ -1875,11 +1875,21 @@ probe_target_compiler() {
container_image=
container_hosts=
container_cross_cc=
+ container_cross_ar=
container_cross_as=
container_cross_ld=
+ container_cross_nm=
+ container_cross_objcopy=
+ container_cross_ranlib=
+ container_cross_strip=
target_cc=
+ target_ar=
target_as=
target_ld=
+ target_nm=
+ target_objcopy=
+ target_ranlib=
+ target_strip=
case $1 in
aarch64) container_hosts="x86_64 aarch64" ;;
@@ -2018,8 +2028,13 @@ probe_target_compiler() {
;;
esac
: ${container_cross_cc:=${container_cross_prefix}gcc}
+ : ${container_cross_ar:=${container_cross_prefix}ar}
: ${container_cross_as:=${container_cross_prefix}as}
: ${container_cross_ld:=${container_cross_prefix}ld}
+ : ${container_cross_nm:=${container_cross_prefix}nm}
+ : ${container_cross_objcopy:=${container_cross_prefix}objcopy}
+ : ${container_cross_ranlib:=${container_cross_prefix}ranlib}
+ : ${container_cross_strip:=${container_cross_prefix}strip}
done
eval "target_cflags=\${cross_cc_cflags_$1}"
@@ -2030,12 +2045,26 @@ probe_target_compiler() {
else
compute_target_variable $1 target_cc gcc
fi
+ target_ccas=$target_cc
+ compute_target_variable $1 target_ar ar
compute_target_variable $1 target_as as
compute_target_variable $1 target_ld ld
+ compute_target_variable $1 target_nm nm
+ compute_target_variable $1 target_objcopy objcopy
+ compute_target_variable $1 target_ranlib ranlib
+ compute_target_variable $1 target_strip strip
if test "$1" = $cpu; then
: ${target_cc:=$cc}
+ : ${target_ccas:=$ccas}
: ${target_as:=$as}
: ${target_ld:=$ld}
+ : ${target_ar:=$ar}
+ : ${target_as:=$as}
+ : ${target_ld:=$ld}
+ : ${target_nm:=$nm}
+ : ${target_objcopy:=$objcopy}
+ : ${target_ranlib:=$ranlib}
+ : ${target_strip:=$strip}
fi
if test -n "$target_cc"; then
case $1 in
@@ -2051,6 +2080,10 @@ probe_target_compiler() {
write_target_makefile() {
if test -n "$target_cc"; then
echo "CC=$target_cc"
+ echo "CCAS=$target_ccas"
+ fi
+ if test -n "$target_ar"; then
+ echo "AR=$target_ar"
fi
if test -n "$target_as"; then
echo "AS=$target_as"
@@ -2058,14 +2091,32 @@ write_target_makefile() {
if test -n "$target_ld"; then
echo "LD=$target_ld"
fi
+ if test -n "$target_nm"; then
+ echo "NM=$target_nm"
+ fi
+ if test -n "$target_objcopy"; then
+ echo "OBJCOPY=$target_objcopy"
+ fi
+ if test -n "$target_ranlib"; then
+ echo "RANLIB=$target_ranlib"
+ fi
+ if test -n "$target_strip"; then
+ echo "STRIP=$target_strip"
+ fi
}
write_container_target_makefile() {
if test -n "$container_cross_cc"; then
echo "CC=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --"
+ echo "CCAS=\$(DOCKER_SCRIPT) cc --cc $container_cross_cc -i qemu/$container_image -s $source_path --"
fi
+ echo "AR=\$(DOCKER_SCRIPT) cc --cc $container_cross_ar -i qemu/$container_image -s $source_path --"
echo "AS=\$(DOCKER_SCRIPT) cc --cc $container_cross_as -i qemu/$container_image -s $source_path --"
echo "LD=\$(DOCKER_SCRIPT) cc --cc $container_cross_ld -i qemu/$container_image -s $source_path --"
+ echo "NM=\$(DOCKER_SCRIPT) cc --cc $container_cross_nm -i qemu/$container_image -s $source_path --"
+ echo "OBJCOPY=\$(DOCKER_SCRIPT) cc --cc $container_cross_objcopy -i qemu/$container_image -s $source_path --"
+ echo "RANLIB=\$(DOCKER_SCRIPT) cc --cc $container_cross_ranlib -i qemu/$container_image -s $source_path --"
+ echo "STRIP=\$(DOCKER_SCRIPT) cc --cc $container_cross_strip -i qemu/$container_image -s $source_path --"
}
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 23/33] configure: move symlink configuration earlier
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (21 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 22/33] configure: include more binutils in tests/tcg makefile Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-30 8:31 ` Paolo Bonzini
2022-05-27 15:35 ` [PATCH v1 24/33] configure: enable cross-compilation of s390-ccw Alex Bennée
` (9 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
Ensure that the pc-bios/optionrom and pc-bios/s390-ccw directory
exist at the time when we'll write out the compiler configuration
for them.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-13-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 49 ++++++++++++++++++++++++-------------------------
1 file changed, 24 insertions(+), 25 deletions(-)
diff --git a/configure b/configure
index b8c21e096c..82c2ddc79a 100755
--- a/configure
+++ b/configure
@@ -2187,6 +2187,30 @@ fi
QEMU_GA_MSI_MINGW_BIN_PATH="$($pkg_config --variable=prefix glib-2.0)/bin"
+# Set up build tree symlinks that point back into the source tree
+# (these can be both files and directories).
+# Caution: avoid adding files or directories here using wildcards. This
+# will result in problems later if a new file matching the wildcard is
+# added to the source tree -- nothing will cause configure to be rerun
+# so the build tree will be missing the link back to the new file, and
+# tests might fail. Prefer to keep the relevant files in their own
+# directory and symlink the directory instead.
+LINKS="Makefile"
+LINKS="$LINKS tests/tcg/Makefile.target"
+LINKS="$LINKS pc-bios/optionrom/Makefile"
+LINKS="$LINKS pc-bios/s390-ccw/Makefile"
+LINKS="$LINKS .gdbinit scripts" # scripts needed by relative path in .gdbinit
+LINKS="$LINKS tests/avocado tests/data"
+LINKS="$LINKS tests/qemu-iotests/check"
+LINKS="$LINKS python"
+LINKS="$LINKS contrib/plugins/Makefile "
+for f in $LINKS ; do
+ if [ -e "$source_path/$f" ]; then
+ mkdir -p `dirname ./$f`
+ symlink "$source_path/$f" "$f"
+ fi
+done
+
# Mac OS X ships with a broken assembler
roms=
if { test "$cpu" = "i386" || test "$cpu" = "x86_64"; } && \
@@ -2405,31 +2429,6 @@ if test "$safe_stack" = "yes"; then
echo "CONFIG_SAFESTACK=y" >> $config_host_mak
fi
-# If we're using a separate build tree, set it up now.
-# LINKS are things to symlink back into the source tree
-# (these can be both files and directories).
-# Caution: do not add files or directories here using wildcards. This
-# will result in problems later if a new file matching the wildcard is
-# added to the source tree -- nothing will cause configure to be rerun
-# so the build tree will be missing the link back to the new file, and
-# tests might fail. Prefer to keep the relevant files in their own
-# directory and symlink the directory instead.
-LINKS="Makefile"
-LINKS="$LINKS tests/tcg/Makefile.target"
-LINKS="$LINKS pc-bios/optionrom/Makefile"
-LINKS="$LINKS pc-bios/s390-ccw/Makefile"
-LINKS="$LINKS .gdbinit scripts" # scripts needed by relative path in .gdbinit
-LINKS="$LINKS tests/avocado tests/data"
-LINKS="$LINKS tests/qemu-iotests/check"
-LINKS="$LINKS python"
-LINKS="$LINKS contrib/plugins/Makefile "
-for f in $LINKS ; do
- if [ -e "$source_path/$f" ]; then
- mkdir -p `dirname ./$f`
- symlink "$source_path/$f" "$f"
- fi
-done
-
# tests/tcg configuration
(makefile=tests/tcg/Makefile.prereqs
echo "# Automatically generated by configure - do not modify" > $makefile
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 24/33] configure: enable cross-compilation of s390-ccw
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (22 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 23/33] configure: move symlink configuration earlier Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-30 7:40 ` Thomas Huth
2022-05-27 15:35 ` [PATCH v1 25/33] configure: enable cross-compilation of optionrom Alex Bennée
` (8 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée, Christian Borntraeger,
Thomas Huth, Cornelia Huck, open list:S390-ccw boot
From: Paolo Bonzini <pbonzini@redhat.com>
While container-based cross compilers are not supported, this already makes
it possible to build s390-ccw on any machine that has s390x GCC and binutils
installed.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-14-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 18 +++++++++++++-----
pc-bios/s390-ccw/netboot.mak | 2 +-
pc-bios/s390-ccw/Makefile | 9 +++++----
3 files changed, 19 insertions(+), 10 deletions(-)
diff --git a/configure b/configure
index 82c2ddc79a..99626df869 100755
--- a/configure
+++ b/configure
@@ -2228,24 +2228,32 @@ if { test "$cpu" = "i386" || test "$cpu" = "x86_64"; } && \
done
fi
-# Only build s390-ccw bios if we're on s390x and the compiler has -march=z900
-# or -march=z10 (which is the lowest architecture level that Clang supports)
-if test "$cpu" = "s390x" ; then
+# Only build s390-ccw bios if the compiler has -march=z900 or -march=z10
+# (which is the lowest architecture level that Clang supports)
+probe_target_compiler s390x
+if test -n "$target_cc" && test "$softmmu" = yes; then
write_c_skeleton
- compile_prog "-march=z900" ""
+ do_compiler "$target_cc" $target_cc_cflags -march=z900 -o $TMPO -c $TMPC
has_z900=$?
- if [ $has_z900 = 0 ] || compile_object "-march=z10 -msoft-float -Werror"; then
+ if [ $has_z900 = 0 ] || do_compiler "$target_cc" $target_cc_cflags -march=z10 -msoft-float -Werror -o $TMPO -c $TMPC; then
if [ $has_z900 != 0 ]; then
echo "WARNING: Your compiler does not support the z900!"
echo " The s390-ccw bios will only work with guest CPUs >= z10."
fi
roms="$roms s390-ccw"
+ config_mak=pc-bios/s390-ccw/config-host.mak
+ echo "# Automatically generated by configure - do not modify" > $config_mak
+ echo "SRC_PATH=$source_path/pc-bios/s390-ccw" >> $config_mak
+ write_target_makefile >> $config_mak
# SLOF is required for building the s390-ccw firmware on s390x,
# since it is using the libnet code from SLOF for network booting.
git_submodules="${git_submodules} roms/SLOF"
fi
fi
+#######################################
+# generate config-host.mak
+
# Check that the C++ compiler exists and works with the C compiler.
# All the QEMU_CXXFLAGS are based on QEMU_CFLAGS. Keep this at the end to don't miss any other that could be added.
if has $cxx; then
diff --git a/pc-bios/s390-ccw/netboot.mak b/pc-bios/s390-ccw/netboot.mak
index 68b4d7edcb..1a06befa4b 100644
--- a/pc-bios/s390-ccw/netboot.mak
+++ b/pc-bios/s390-ccw/netboot.mak
@@ -1,5 +1,5 @@
-SLOF_DIR := $(SRC_PATH)/roms/SLOF
+SLOF_DIR := $(SRC_PATH)/../../roms/SLOF
NETOBJS := start.o sclp.o cio.o virtio.o virtio-net.o jump2ipl.o netmain.o
diff --git a/pc-bios/s390-ccw/Makefile b/pc-bios/s390-ccw/Makefile
index 0eb68efc7b..6eb713bf37 100644
--- a/pc-bios/s390-ccw/Makefile
+++ b/pc-bios/s390-ccw/Makefile
@@ -2,8 +2,9 @@ all: build-all
# Dummy command so that make thinks it has done something
@true
-include ../../config-host.mak
+include config-host.mak
CFLAGS = -O2 -g
+MAKEFLAGS += -rR
quiet-command = $(if $(V),$1,$(if $(2),@printf " %-7s %s\n" $2 $3 && $1, @$1))
cc-option = $(if $(shell $(CC) $1 $2 -S -o /dev/null -xc /dev/null \
@@ -11,7 +12,7 @@ cc-option = $(if $(shell $(CC) $1 $2 -S -o /dev/null -xc /dev/null \
VPATH_SUFFIXES = %.c %.h %.S %.m %.mak %.sh %.rc Kconfig% %.json.in
set-vpath = $(if $1,$(foreach PATTERN,$(VPATH_SUFFIXES),$(eval vpath $(PATTERN) $1)))
-$(call set-vpath, $(SRC_PATH)/pc-bios/s390-ccw)
+$(call set-vpath, $(SRC_PATH))
# Flags for dependency generation
QEMU_DGFLAGS = -MMD -MP -MT $@ -MF $(@D)/$(*F).d
@@ -49,8 +50,8 @@ s390-ccw.img: s390-ccw.elf
$(OBJECTS): Makefile
-ifneq ($(wildcard $(SRC_PATH)/roms/SLOF/lib/libnet),)
-include $(SRC_PATH)/pc-bios/s390-ccw/netboot.mak
+ifneq ($(wildcard $(SRC_PATH)/../../roms/SLOF/lib/libnet),)
+include $(SRC_PATH)/netboot.mak
else
s390-netboot.img:
@echo "s390-netboot.img not built since roms/SLOF/ is not available."
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 25/33] configure: enable cross-compilation of optionrom
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (23 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 24/33] configure: enable cross-compilation of s390-ccw Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-30 8:32 ` Paolo Bonzini
2022-05-27 15:35 ` [PATCH v1 26/33] configure: enable cross compilation of vof Alex Bennée
` (7 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
While container-based cross compilers are not supported, this already makes
it possible to build x86 optionroms on any machine that has an installation
of GCC and binutils for 32- or 64-bit x86.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-15-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 29 +++++++++++++++++++++--------
pc-bios/optionrom/Makefile | 2 --
2 files changed, 21 insertions(+), 10 deletions(-)
diff --git a/configure b/configure
index 99626df869..b974db3ebd 100755
--- a/configure
+++ b/configure
@@ -2077,6 +2077,13 @@ probe_target_compiler() {
fi
}
+probe_target_compilers() {
+ for i; do
+ probe_target_compiler $i
+ test -n "$target_cc" && return 0
+ done
+}
+
write_target_makefile() {
if test -n "$target_cc"; then
echo "CC=$target_cc"
@@ -2187,6 +2194,9 @@ fi
QEMU_GA_MSI_MINGW_BIN_PATH="$($pkg_config --variable=prefix glib-2.0)/bin"
+#######################################
+# cross-compiled firmware targets
+
# Set up build tree symlinks that point back into the source tree
# (these can be both files and directories).
# Caution: avoid adding files or directories here using wildcards. This
@@ -2213,19 +2223,27 @@ done
# Mac OS X ships with a broken assembler
roms=
-if { test "$cpu" = "i386" || test "$cpu" = "x86_64"; } && \
+probe_target_compilers i386 x86_64
+if test -n "$target_cc" &&
test "$targetos" != "darwin" && test "$targetos" != "sunos" && \
test "$targetos" != "haiku" && test "$softmmu" = yes ; then
# Different host OS linkers have different ideas about the name of the ELF
# emulation. Linux and OpenBSD/amd64 use 'elf_i386'; FreeBSD uses the _fbsd
# variant; OpenBSD/i386 uses the _obsd variant; and Windows uses i386pe.
for emu in elf_i386 elf_i386_fbsd elf_i386_obsd i386pe; do
- if "$ld" -verbose 2>&1 | grep -q "^[[:space:]]*$emu[[:space:]]*$"; then
+ if "$target_ld" -verbose 2>&1 | grep -q "^[[:space:]]*$emu[[:space:]]*$"; then
ld_i386_emulation="$emu"
- roms="optionrom"
break
fi
done
+ if test -n "$ld_i386_emulation"; then
+ roms="optionrom"
+ config_mak=pc-bios/optionrom/config.mak
+ echo "# Automatically generated by configure - do not modify" > $config_mak
+ echo "TOPSRC_DIR=$source_path" >> $config_mak
+ echo "LD_I386_EMULATION=$ld_i386_emulation" >> $config_mak
+ write_target_makefile >> $config_mak
+ fi
fi
# Only build s390-ccw bios if the compiler has -march=z900 or -march=z10
@@ -2378,7 +2396,6 @@ echo "GLIB_CFLAGS=$glib_cflags" >> $config_host_mak
echo "GLIB_LIBS=$glib_libs" >> $config_host_mak
echo "GLIB_VERSION=$(pkg-config --modversion glib-2.0)" >> $config_host_mak
echo "QEMU_LDFLAGS=$QEMU_LDFLAGS" >> $config_host_mak
-echo "LD_I386_EMULATION=$ld_i386_emulation" >> $config_host_mak
echo "STRIP=$strip" >> $config_host_mak
echo "EXESUF=$EXESUF" >> $config_host_mak
@@ -2568,10 +2585,6 @@ for target in $target_list; do
done
echo "TCG_TESTS_TARGETS=$tcg_tests_targets" >> $makefile)
-config_mak=pc-bios/optionrom/config.mak
-echo "# Automatically generated by configure - do not modify" > $config_mak
-echo "TOPSRC_DIR=$source_path" >> $config_mak
-
if test "$skip_meson" = no; then
cross="config-meson.cross.new"
meson_quote() {
diff --git a/pc-bios/optionrom/Makefile b/pc-bios/optionrom/Makefile
index 17ccc76241..f639915b4f 100644
--- a/pc-bios/optionrom/Makefile
+++ b/pc-bios/optionrom/Makefile
@@ -6,7 +6,6 @@ all: multiboot.bin multiboot_dma.bin linuxboot.bin linuxboot_dma.bin kvmvapic.bi
# Dummy command so that make thinks it has done something
@true
-include ../../config-host.mak
CFLAGS = -O2 -g
quiet-command = $(if $(V),$1,$(if $(2),@printf " %-7s %s\n" $2 $3 && $1, @$1))
@@ -44,7 +43,6 @@ Wa = -Wa,
override ASFLAGS += -32
override CFLAGS += $(call cc-option, $(Wa)-32)
-LD_I386_EMULATION ?= elf_i386
override LDFLAGS = -m $(LD_I386_EMULATION) -T $(SRC_DIR)/flat.lds
pvh.img: pvh.o pvh_main.o
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 26/33] configure: enable cross compilation of vof
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (24 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 25/33] configure: enable cross-compilation of optionrom Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 27/33] configure: remove unused variables from config-host.mak Alex Bennée
` (6 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Alex Bennée, Alexey Kardashevskiy,
Cédric Le Goater, Daniel Henrique Barboza, David Gibson,
Greg Kurz, open list:Virtual Open Firm...
From: Paolo Bonzini <pbonzini@redhat.com>
While container-based cross compilers are not supported, this already
makes it possible to build vof on any machine that has an installation
of GCC and binutils for 32- or 64-bit PowerPC.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-16-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 10 ++++++++++
pc-bios/vof/Makefile | 17 +++++++++--------
2 files changed, 19 insertions(+), 8 deletions(-)
diff --git a/configure b/configure
index b974db3ebd..89a0470cc2 100755
--- a/configure
+++ b/configure
@@ -2209,6 +2209,7 @@ LINKS="Makefile"
LINKS="$LINKS tests/tcg/Makefile.target"
LINKS="$LINKS pc-bios/optionrom/Makefile"
LINKS="$LINKS pc-bios/s390-ccw/Makefile"
+LINKS="$LINKS pc-bios/vof/Makefile"
LINKS="$LINKS .gdbinit scripts" # scripts needed by relative path in .gdbinit
LINKS="$LINKS tests/avocado tests/data"
LINKS="$LINKS tests/qemu-iotests/check"
@@ -2246,6 +2247,15 @@ if test -n "$target_cc" &&
fi
fi
+probe_target_compilers ppc ppc64
+if test -n "$target_cc" && test "$softmmu" = yes; then
+ roms="$roms vof"
+ config_mak=pc-bios/vof/config.mak
+ echo "# Automatically generated by configure - do not modify" > $config_mak
+ echo "SRC_DIR=$source_path/pc-bios/vof" >> $config_mak
+ write_target_makefile >> $config_mak
+fi
+
# Only build s390-ccw bios if the compiler has -march=z900 or -march=z10
# (which is the lowest architecture level that Clang supports)
probe_target_compiler s390x
diff --git a/pc-bios/vof/Makefile b/pc-bios/vof/Makefile
index aa1678c4d8..391ac0d600 100644
--- a/pc-bios/vof/Makefile
+++ b/pc-bios/vof/Makefile
@@ -1,11 +1,10 @@
-all: build-all
+include config.mak
+VPATH=$(SRC_DIR)
+all: vof.bin
-build-all: vof.bin
-
-CROSS ?=
-CC = $(CROSS)gcc
-LD = $(CROSS)ld
-OBJCOPY = $(CROSS)objcopy
+CC ?= $(CROSS)gcc
+LD ?= $(CROSS)ld
+OBJCOPY ?= $(CROSS)objcopy
%.o: %.S
$(CC) -m32 -mbig-endian -mcpu=power4 -c -o $@ $<
@@ -14,10 +13,12 @@ OBJCOPY = $(CROSS)objcopy
$(CC) -m32 -mbig-endian -mcpu=power4 -c -fno-stack-protector -o $@ $<
vof.elf: entry.o main.o ci.o bootmem.o libc.o
- $(LD) -nostdlib -e_start -Tvof.lds -EB -o $@ $^
+ $(LD) -nostdlib -e_start -T$(SRC_DIR)/vof.lds -EB -o $@ $^
%.bin: %.elf
$(OBJCOPY) -O binary -j .text -j .data -j .toc -j .got2 $^ $@
clean:
rm -f *.o vof.bin vof.elf *~
+
+.PHONY: all clean
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 27/33] configure: remove unused variables from config-host.mak
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (25 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 26/33] configure: enable cross compilation of vof Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-27 15:35 ` [PATCH v1 28/33] gitlab: introduce a common base job template Alex Bennée
` (5 subsequent siblings)
32 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée
From: Paolo Bonzini <pbonzini@redhat.com>
The only compiler variable that is still needed is $(CC), for
contrib/plugins/Makefile. All firmware builds have their own
config-host.mak file.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220517092616.1272238-17-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
configure | 6 ------
1 file changed, 6 deletions(-)
diff --git a/configure b/configure
index 89a0470cc2..4c01625459 100755
--- a/configure
+++ b/configure
@@ -2394,11 +2394,6 @@ echo "GENISOIMAGE=$genisoimage" >> $config_host_mak
echo "MESON=$meson" >> $config_host_mak
echo "NINJA=$ninja" >> $config_host_mak
echo "CC=$cc" >> $config_host_mak
-echo "AR=$ar" >> $config_host_mak
-echo "AS=$as" >> $config_host_mak
-echo "CCAS=$ccas" >> $config_host_mak
-echo "OBJCOPY=$objcopy" >> $config_host_mak
-echo "LD=$ld" >> $config_host_mak
echo "QEMU_CFLAGS=$QEMU_CFLAGS" >> $config_host_mak
echo "QEMU_CXXFLAGS=$QEMU_CXXFLAGS" >> $config_host_mak
echo "QEMU_OBJCFLAGS=$QEMU_OBJCFLAGS" >> $config_host_mak
@@ -2406,7 +2401,6 @@ echo "GLIB_CFLAGS=$glib_cflags" >> $config_host_mak
echo "GLIB_LIBS=$glib_libs" >> $config_host_mak
echo "GLIB_VERSION=$(pkg-config --modversion glib-2.0)" >> $config_host_mak
echo "QEMU_LDFLAGS=$QEMU_LDFLAGS" >> $config_host_mak
-echo "STRIP=$strip" >> $config_host_mak
echo "EXESUF=$EXESUF" >> $config_host_mak
# use included Linux headers
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 28/33] gitlab: introduce a common base job template
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (26 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 27/33] configure: remove unused variables from config-host.mak Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-30 7:50 ` Thomas Huth
2022-05-27 15:35 ` [PATCH v1 29/33] gitlab: convert Cirrus jobs to .base_job_template Alex Bennée
` (4 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, Thomas Huth, Wainer dos Santos Moschetta,
Beraldo Leal
From: Daniel P. Berrangé <berrange@redhat.com>
Currently job rules are spread across the various templates
and jobs, making it hard to understand exactly what runs in
what scenario. This leads to inconsistency in the rules and
increased maint burden.
The intent is that we introduce a common '.base_job_template'
which will have a general purpose 'rules:' block. No other
template or job should define 'rules:', but instead they must
rely on the inherited rules. To allow behaviour to be tweaked,
rules will be influenced by a number of variables with the
naming scheme 'QEMU_JOB_nnnn'.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
docs/devel/ci-jobs.rst.inc | 36 ++++++++++++++++++++++++++++++++++-
.gitlab-ci.d/base.yml | 28 +++++++++++++++++++++++++++
.gitlab-ci.d/qemu-project.yml | 1 +
3 files changed, 64 insertions(+), 1 deletion(-)
create mode 100644 .gitlab-ci.d/base.yml
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index 92e25872aa..eb6a9e6122 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -28,7 +28,35 @@ For further information about how to set these variables, please refer to::
https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd
-Here is a list of the most used variables:
+Variable naming and grouping
+----------------------------
+
+The variables used by QEMU's CI configuration are grouped together
+in a handful of namespaces
+
+ * QEMU_JOB_nnnn - variables to be defined in individual jobs
+ or templates, to influence the shared rules defined in the
+ .base_job_template.
+
+ * QEMU_CI_nnn - variables to be set by contributors in their
+ repository CI settings, or as git push variables, to influence
+ which jobs get run in a pipeline
+
+ * nnn - other misc variables not falling into the above
+ categories, or using different names for historical reasons
+ and not yet converted.
+
+Maintainer controlled job variables
+-----------------------------------
+
+The following variables may be set when defining a job in the
+CI configuration file.
+
+Contributor controlled runtime variables
+----------------------------------------
+
+The following variables may be set by contributors to control
+job execution
QEMU_CI_AVOCADO_TESTING
~~~~~~~~~~~~~~~~~~~~~~~
@@ -38,6 +66,12 @@ these artifacts are not already cached, downloading them make the jobs
reach the timeout limit). Set this variable to have the tests using the
Avocado framework run automatically.
+Other misc variables
+--------------------
+
+These variables are primarily to control execution of jobs on
+private runners
+
AARCH64_RUNNER_AVAILABLE
~~~~~~~~~~~~~~~~~~~~~~~~
If you've got access to an aarch64 host that can be used as a gitlab-CI
diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
new file mode 100644
index 0000000000..10eb6ab8bc
--- /dev/null
+++ b/.gitlab-ci.d/base.yml
@@ -0,0 +1,28 @@
+
+# The order of rules defined here is critically important.
+# They are evaluated in order and first match wins.
+#
+# Thus we group them into a number of stages, ordered from
+# most restrictive to least restrictive
+#
+.base_job_template:
+ rules:
+ #############################################################
+ # Stage 1: exclude scenarios where we definitely don't
+ # want jobs to run
+ #############################################################
+
+
+ #############################################################
+ # Stage 2: fine tune execution of jobs in specific scenarios
+ # where the catch all logic is inapprorpaite
+ #############################################################
+
+
+ #############################################################
+ # Stage 3: catch all logic applying to any job not matching
+ # an earlier criteria
+ #############################################################
+
+ # Jobs can run if any jobs they depend on were successfull
+ - when: on_success
diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
index 871262fe0e..691d9bf5dc 100644
--- a/.gitlab-ci.d/qemu-project.yml
+++ b/.gitlab-ci.d/qemu-project.yml
@@ -2,6 +2,7 @@
# https://gitlab.com/qemu-project/qemu/-/pipelines
include:
+ - local: '/.gitlab-ci.d/base.yml'
- local: '/.gitlab-ci.d/stages.yml'
- local: '/.gitlab-ci.d/edk2.yml'
- local: '/.gitlab-ci.d/opensbi.yml'
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 29/33] gitlab: convert Cirrus jobs to .base_job_template
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (27 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 28/33] gitlab: introduce a common base job template Alex Bennée
@ 2022-05-27 15:35 ` Alex Bennée
2022-05-30 7:54 ` Thomas Huth
2022-05-27 15:36 ` [PATCH v1 30/33] gitlab: convert static checks " Alex Bennée
` (3 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:35 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, Thomas Huth, Wainer dos Santos Moschetta,
Beraldo Leal
From: Daniel P. Berrangé <berrange@redhat.com>
This folds the Cirrus job rules into the base job
template, introducing two new variables
- QEMU_JOB_CIRRUS - identifies the job as making
use of Cirrus CI via cirrus-run
- QEMU_JOB_OPTIONAL - identifies the job as one
that is not run by default, primarily due to
resource constraints. It can be manually invoked
by users if they wish to validate that scenario.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
docs/devel/ci-jobs.rst.inc | 14 ++++++++++++++
.gitlab-ci.d/base.yml | 9 +++++++++
.gitlab-ci.d/cirrus.yml | 16 ++++++++--------
3 files changed, 31 insertions(+), 8 deletions(-)
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index eb6a9e6122..a539f502da 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -52,6 +52,20 @@ Maintainer controlled job variables
The following variables may be set when defining a job in the
CI configuration file.
+QEMU_JOB_CIRRUS
+~~~~~~~~~~~~~~~
+
+The job makes use of Cirrus CI infrastructure, requiring the
+configuration setup for cirrus-run to be present in the repository
+
+QEMU_JOB_OPTIONAL
+~~~~~~~~~~~~~~~~~
+
+The job is expected to be successful in general, but is not run
+by default due to need to conserve limited CI resources. It is
+available to be started manually by the contributor in the CI
+pipelines UI.
+
Contributor controlled runtime variables
----------------------------------------
diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
index 10eb6ab8bc..5734caf9fe 100644
--- a/.gitlab-ci.d/base.yml
+++ b/.gitlab-ci.d/base.yml
@@ -12,12 +12,21 @@
# want jobs to run
#############################################################
+ # Cirrus jobs can't run unless the creds / target repo are set
+ - if: '$QEMU_JOB_CIRRUS && ($CIRRUS_GITHUB_REPO == "" || $CIRRUS_API_TOKEN == "")'
+ when: never
+
#############################################################
# Stage 2: fine tune execution of jobs in specific scenarios
# where the catch all logic is inapprorpaite
#############################################################
+ # Optional jobs should not be run unless manually triggered
+ - if: '$QEMU_JOB_OPTIONAL'
+ when: manual
+ allow_failure: true
+
#############################################################
# Stage 3: catch all logic applying to any job not matching
diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index b96b22e269..609c364308 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -11,6 +11,7 @@
# special care, because we can't just override it at the GitLab CI job
# definition level or we risk breaking it completely.
.cirrus_build_job:
+ extends: .base_job_template
stage: build
image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master
needs: []
@@ -40,11 +41,8 @@
<.gitlab-ci.d/cirrus/build.yml >.gitlab-ci.d/cirrus/$NAME.yml
- cat .gitlab-ci.d/cirrus/$NAME.yml
- cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
- rules:
- # Allow on 'staging' branch and 'stable-X.Y-staging' branches only
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH !~ /staging/'
- when: never
- - if: "$CIRRUS_GITHUB_REPO && $CIRRUS_API_TOKEN"
+ variables:
+ QEMU_JOB_CIRRUS: 1
x64-freebsd-12-build:
extends: .cirrus_build_job
@@ -90,11 +88,11 @@ x64-macos-11-base-build:
# The following jobs run VM-based tests via KVM on a Linux-based Cirrus-CI job
.cirrus_kvm_job:
+ extends: .base_job_template
stage: build
image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master
needs: []
timeout: 80m
- allow_failure: true
script:
- sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g"
-e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g"
@@ -105,8 +103,10 @@ x64-macos-11-base-build:
<.gitlab-ci.d/cirrus/kvm-build.yml >.gitlab-ci.d/cirrus/$NAME.yml
- cat .gitlab-ci.d/cirrus/$NAME.yml
- cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
- rules:
- - when: manual
+ variables:
+ QEMU_JOB_CIRRUS: 1
+ QEMU_JOB_OPTIONAL: 1
+
x86-netbsd:
extends: .cirrus_kvm_job
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 30/33] gitlab: convert static checks to .base_job_template
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (28 preceding siblings ...)
2022-05-27 15:35 ` [PATCH v1 29/33] gitlab: convert Cirrus jobs to .base_job_template Alex Bennée
@ 2022-05-27 15:36 ` Alex Bennée
2022-05-30 8:04 ` Thomas Huth
2022-05-27 15:36 ` [PATCH v1 31/33] gitlab: convert build/container jobs " Alex Bennée
` (2 subsequent siblings)
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:36 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, Thomas Huth, Wainer dos Santos Moschetta,
Beraldo Leal
From: Daniel P. Berrangé <berrange@redhat.com>
This folds the static checks into using the base job
template rules, introducing one new variable
- QEMU_JOB_ONLY_FORKS - a job that should never run
on an upstream pipeline. The information it reports
is only applicable to contributors in a pre-submission
scenario, not time of merge.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-4-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
docs/devel/ci-jobs.rst.inc | 7 +++++++
.gitlab-ci.d/base.yml | 4 ++++
.gitlab-ci.d/static_checks.yml | 19 +++++++------------
3 files changed, 18 insertions(+), 12 deletions(-)
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index a539f502da..4c7e30ab08 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -66,6 +66,13 @@ by default due to need to conserve limited CI resources. It is
available to be started manually by the contributor in the CI
pipelines UI.
+QEMU_JOB_ONLY_FORKS
+~~~~~~~~~~~~~~~~~~~
+
+The job results are only of interest to contributors prior to
+submitting code. They are not required as part of the gating
+CI pipeline.
+
Contributor controlled runtime variables
----------------------------------------
diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
index 5734caf9fe..9a0b8d7f97 100644
--- a/.gitlab-ci.d/base.yml
+++ b/.gitlab-ci.d/base.yml
@@ -16,6 +16,10 @@
- if: '$QEMU_JOB_CIRRUS && ($CIRRUS_GITHUB_REPO == "" || $CIRRUS_API_TOKEN == "")'
when: never
+ # Jobs only intended for forks should always be skipped on upstram
+ - if: '$QEMU_JOB_ONLY_FORKS == "1" && $CI_PROJECT_NAMESPACE == "qemu-project"'
+ when: never
+
#############################################################
# Stage 2: fine tune execution of jobs in specific scenarios
diff --git a/.gitlab-ci.d/static_checks.yml b/.gitlab-ci.d/static_checks.yml
index 94858e3272..289ad1359e 100644
--- a/.gitlab-ci.d/static_checks.yml
+++ b/.gitlab-ci.d/static_checks.yml
@@ -1,4 +1,5 @@
check-patch:
+ extends: .base_job_template
stage: build
image: python:3.10-alpine
needs: []
@@ -6,15 +7,13 @@ check-patch:
- .gitlab-ci.d/check-patch.py
variables:
GIT_DEPTH: 1000
+ QEMU_JOB_ONLY_FORKS: 1
before_script:
- apk -U add git perl
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project"'
- when: never
- - when: on_success
- allow_failure: true
+ allow_failure: true
check-dco:
+ extends: .base_job_template
stage: build
image: python:3.10-alpine
needs: []
@@ -23,12 +22,9 @@ check-dco:
GIT_DEPTH: 1000
before_script:
- apk -U add git
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- when: never
- - when: on_success
check-python-pipenv:
+ extends: .base_job_template
stage: test
image: $CI_REGISTRY_IMAGE/qemu/python:latest
script:
@@ -39,6 +35,7 @@ check-python-pipenv:
job: python-container
check-python-tox:
+ extends: .base_job_template
stage: test
image: $CI_REGISTRY_IMAGE/qemu/python:latest
script:
@@ -46,8 +43,6 @@ check-python-tox:
variables:
GIT_DEPTH: 1
QEMU_TOX_EXTRA_ARGS: --skip-missing-interpreters=false
+ QEMU_JOB_OPTIONAL: 1
needs:
job: python-container
- rules:
- - when: manual
- allow_failure: true
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 31/33] gitlab: convert build/container jobs to .base_job_template
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (29 preceding siblings ...)
2022-05-27 15:36 ` [PATCH v1 30/33] gitlab: convert static checks " Alex Bennée
@ 2022-05-27 15:36 ` Alex Bennée
2022-05-30 8:53 ` Thomas Huth
2022-05-27 15:36 ` [PATCH v1 32/33] gitlab: don't run CI jobs in forks by default Alex Bennée
2022-05-27 15:36 ` [PATCH v1 33/33] docs/devel: clean-up the CI links in the docs Alex Bennée
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:36 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, Thomas Huth, Wainer dos Santos Moschetta,
Beraldo Leal
From: Daniel P. Berrangé <berrange@redhat.com>
This converts the main build and container jobs to use the
base job rules, defining the following new variables
- QEMU_JOB_SKIPPED - jobs that are known to be currently
broken and should not be run. Can still be manually
launched if desired.
- QEMU_JOB_AVOCADO - jobs that run the Avocado integration
test harness.
- QEMU_JOB_PUBLISH - jobs that publish content after the
branch is merged upstream
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-5-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
docs/devel/ci-jobs.rst.inc | 19 +++++++++++++++++++
.gitlab-ci.d/base.yml | 22 ++++++++++++++++++++++
.gitlab-ci.d/buildtest-template.yml | 16 ++++------------
.gitlab-ci.d/buildtest.yml | 28 +++++++++++++---------------
.gitlab-ci.d/container-cross.yml | 6 ++----
.gitlab-ci.d/container-template.yml | 1 +
.gitlab-ci.d/crossbuild-template.yml | 3 +++
.gitlab-ci.d/windows.yml | 1 +
8 files changed, 65 insertions(+), 31 deletions(-)
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index 4c7e30ab08..0b4926e537 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -73,6 +73,25 @@ The job results are only of interest to contributors prior to
submitting code. They are not required as part of the gating
CI pipeline.
+QEMU_JOB_SKIPPED
+~~~~~~~~~~~~~~~~
+
+The job is not reliably successsful in general, so is not
+currently suitable to be run by default. Ideally this should
+be a temporary marker until the problems can be addressed, or
+the job permanently removed.
+
+QEMU_JOB_PUBLISH
+~~~~~~~~~~~~~~~~
+
+The job is for publishing content after a branch has been
+merged into the upstream default branch.
+
+QEMU_JOB_AVOCADO
+~~~~~~~~~~~~~~~~
+
+The job runs the Avocado integration test suite
+
Contributor controlled runtime variables
----------------------------------------
diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
index 9a0b8d7f97..6a918abbda 100644
--- a/.gitlab-ci.d/base.yml
+++ b/.gitlab-ci.d/base.yml
@@ -16,10 +16,22 @@
- if: '$QEMU_JOB_CIRRUS && ($CIRRUS_GITHUB_REPO == "" || $CIRRUS_API_TOKEN == "")'
when: never
+ # Publishing jobs should only run on the default branch in upstream
+ - if: '$QEMU_JOB_PUBLISH == "1" && $CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH'
+ when: never
+
+ # Non-publishing jobs should only run on staging branches in upstream
+ - if: '$QEMU_JOB_PUBLISH != "1" && $CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH !~ /staging/'
+ when: never
+
# Jobs only intended for forks should always be skipped on upstram
- if: '$QEMU_JOB_ONLY_FORKS == "1" && $CI_PROJECT_NAMESPACE == "qemu-project"'
when: never
+ # Avocado jobs don't run in forks unless $QEMU_CI_AVOCADO_TESTING is set
+ - if: '$QEMU_JOB_AVOCADO && $QEMU_CI_AVOCADO_TESTING != "1" && $CI_PROJECT_NAMESPACE != "qemu-project"'
+ when: never
+
#############################################################
# Stage 2: fine tune execution of jobs in specific scenarios
@@ -31,6 +43,16 @@
when: manual
allow_failure: true
+ # Skipped jobs should not be run unless manually triggered
+ - if: '$QEMU_JOB_SKIPPED'
+ when: manual
+ allow_failure: true
+
+ # Avocado jobs can be manually start in forks if $QEMU_CI_AVOCADO_TESTING is unset
+ - if: '$QEMU_JOB_AVOCADO && $CI_PROJECT_NAMESPACE != "qemu-project"'
+ when: manual
+ allow_failure: true
+
#############################################################
# Stage 3: catch all logic applying to any job not matching
diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
index b381345dbc..73ecfabb8d 100644
--- a/.gitlab-ci.d/buildtest-template.yml
+++ b/.gitlab-ci.d/buildtest-template.yml
@@ -1,4 +1,5 @@
.native_build_job_template:
+ extends: .base_job_template
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
before_script:
@@ -27,6 +28,7 @@
fi
.common_test_job_template:
+ extends: .base_job_template
stage: test
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
script:
@@ -77,15 +79,5 @@
after_script:
- cd build
- du -chs ${CI_PROJECT_DIR}/avocado-cache
- rules:
- # Only run these jobs if running on the mainstream namespace,
- # or if the user set the QEMU_CI_AVOCADO_TESTING variable (either
- # in its namespace setting or via git-push option, see documentation
- # in /.gitlab-ci.yml of this repository).
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project"'
- when: on_success
- - if: '$QEMU_CI_AVOCADO_TESTING'
- when: on_success
- # Otherwise, set to manual (the jobs are created but not run).
- - when: manual
- allow_failure: true
+ variables:
+ QEMU_JOB_AVOCADO: 1
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index e9620c3074..ecac3ec50c 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -360,12 +360,11 @@ build-cfi-aarch64:
expire_in: 2 days
paths:
- build
- rules:
+ variables:
# FIXME: This job is often failing, likely due to out-of-memory problems in
# the constrained containers of the shared runners. Thus this is marked as
- # manual until the situation has been solved.
- - when: manual
- allow_failure: true
+ # skipped until the situation has been solved.
+ QEMU_JOB_SKIPPED: 1
check-cfi-aarch64:
extends: .native_test_job_template
@@ -402,12 +401,11 @@ build-cfi-ppc64-s390x:
expire_in: 2 days
paths:
- build
- rules:
+ variables:
# FIXME: This job is often failing, likely due to out-of-memory problems in
# the constrained containers of the shared runners. Thus this is marked as
- # manual until the situation has been solved.
- - when: manual
- allow_failure: true
+ # skipped until the situation has been solved.
+ QEMU_JOB_SKIPPED: 1
check-cfi-ppc64-s390x:
extends: .native_test_job_template
@@ -579,6 +577,7 @@ build-without-default-features:
MAKE_CHECK_ARGS: check-unit check-qtest SPEED=slow
build-libvhost-user:
+ extends: .base_job_template
stage: build
image: $CI_REGISTRY_IMAGE/qemu/fedora:latest
needs:
@@ -595,10 +594,13 @@ build-tools-and-docs-debian:
extends: .native_build_job_template
needs:
job: amd64-debian-container
+ # when running on 'master' we use pre-existing container
+ optional: true
variables:
IMAGE: debian-amd64
MAKE_CHECK_ARGS: check-unit check-softfloat ctags TAGS cscope
CONFIGURE_ARGS: --disable-system --disable-user --enable-docs --enable-tools
+ QEMU_JOB_PUBLISH: 1
artifacts:
expire_in: 2 days
paths:
@@ -618,6 +620,7 @@ build-tools-and-docs-debian:
# that users can see the results of their commits, regardless
# of what topic branch they're currently using
pages:
+ extends: .base_job_template
image: $CI_REGISTRY_IMAGE/qemu/debian-amd64:latest
stage: test
needs:
@@ -635,10 +638,5 @@ pages:
artifacts:
paths:
- public
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- when: on_success
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project"'
- when: never
- - if: '$CI_PROJECT_NAMESPACE != "qemu-project"'
- when: on_success
+ variables:
+ QEMU_JOB_PUBLISH: 1
diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 147e667744..b7963498a3 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -40,15 +40,13 @@ armhf-debian-cross-container:
# We never want to build hexagon in the CI system and by default we
# always want to refer to the master registry where it lives.
hexagon-cross-container:
+ extends: .base_job_template
image: docker:stable
stage: containers
- rules:
- - if: '$CI_PROJECT_NAMESPACE == "qemu-project"'
- when: never
- - when: always
variables:
NAME: debian-hexagon-cross
GIT_DEPTH: 1
+ QEMU_JOB_ONLY_FORKS: 1
services:
- docker:dind
before_script:
diff --git a/.gitlab-ci.d/container-template.yml b/.gitlab-ci.d/container-template.yml
index 1baecd9460..c434b9c8f3 100644
--- a/.gitlab-ci.d/container-template.yml
+++ b/.gitlab-ci.d/container-template.yml
@@ -1,4 +1,5 @@
.container_job_template:
+ extends: .base_job_template
image: docker:stable
stage: containers
services:
diff --git a/.gitlab-ci.d/crossbuild-template.yml b/.gitlab-ci.d/crossbuild-template.yml
index 29c3c2b826..28b2142ec2 100644
--- a/.gitlab-ci.d/crossbuild-template.yml
+++ b/.gitlab-ci.d/crossbuild-template.yml
@@ -1,4 +1,5 @@
.cross_system_build_job:
+ extends: .base_job_template
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
timeout: 80m
@@ -24,6 +25,7 @@
# KVM), and set extra options (such disabling other accelerators) via the
# $EXTRA_CONFIGURE_OPTS variable.
.cross_accel_build_job:
+ extends: .base_job_template
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
timeout: 30m
@@ -36,6 +38,7 @@
- make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
.cross_user_build_job:
+ extends: .base_job_template
stage: build
image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest
script:
diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index cf7724b8e5..1b2ede49e1 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -1,4 +1,5 @@
.shared_msys2_builder:
+ extends: .base_job_template
tags:
- shared-windows
- windows
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 32/33] gitlab: don't run CI jobs in forks by default
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (30 preceding siblings ...)
2022-05-27 15:36 ` [PATCH v1 31/33] gitlab: convert build/container jobs " Alex Bennée
@ 2022-05-27 15:36 ` Alex Bennée
2022-05-27 15:57 ` Daniel P. Berrangé
2022-05-30 9:00 ` Thomas Huth
2022-05-27 15:36 ` [PATCH v1 33/33] docs/devel: clean-up the CI links in the docs Alex Bennée
32 siblings, 2 replies; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:36 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée, Thomas Huth, Wainer dos Santos Moschetta,
Beraldo Leal
From: Daniel P. Berrangé <berrange@redhat.com>
To preserve CI shared runner credits we don't want to run
pipelines on every push.
This sets up the config so that pipelines are never created
for contributors by default. To override this the QEMU_CI
variable can be set to a non-zero value. If set to 1, the
pipeline will be created but all jobs will remain manually
started. The contributor can selectively run jobs that they
care about. If set to 2, the pipeline will be created and
all jobs will immediately start.
This behavior can be controlled using push variables
git push -o ci.variable=QEMU_CI=1
To make this more convenient define an alias
git config --local alias.push-ci "push -o ci.variable=QEMU_CI=1"
git config --local alias.push-ci-now "push -o ci.variable=QEMU_CI=2"
Which lets you run
git push-ci
to create the pipeline, or
git push-ci-now
to create and run the pipeline
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-6-berrange@redhat.com>
[AJB: fix typo, replicate alias tips in ci.rst]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
docs/devel/ci-jobs.rst.inc | 38 ++++++++++++++++++++++++++++++++++++++
.gitlab-ci.d/base.yml | 9 +++++++++
2 files changed, 47 insertions(+)
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index 0b4926e537..13d448b54d 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -28,6 +28,32 @@ For further information about how to set these variables, please refer to::
https://docs.gitlab.com/ee/user/project/push_options.html#push-options-for-gitlab-cicd
+Setting aliases in your git config
+----------------------------------
+
+You can use aliases to make it easier to push branches with different
+CI configurations. For example define an alias for triggering CI:
+
+.. code::
+
+ git config --local alias.push-ci "push -o ci.variable=QEMU_CI=1"
+ git config --local alias.push-ci-now "push -o ci.variable=QEMU_CI=2"
+
+Which lets you run:
+
+.. code::
+
+ git push-ci
+
+to create the pipeline, or:
+
+.. code::
+
+ git push-ci-now
+
+to create and run the pipeline
+
+
Variable naming and grouping
----------------------------
@@ -98,6 +124,18 @@ Contributor controlled runtime variables
The following variables may be set by contributors to control
job execution
+QEMU_CI
+~~~~~~~
+
+By default, no pipelines will be created on contributor forks
+in order to preserve CI credits
+
+Set this variable to 1 to create the pipelines, but leave all
+the jobs to be manually started from the UI
+
+Set this variable to 2 to create the pipelines and run all
+the jobs immediately, as was historicaly behaviour
+
QEMU_CI_AVOCADO_TESTING
~~~~~~~~~~~~~~~~~~~~~~~
By default, tests using the Avocado framework are not run automatically in
diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
index 6a918abbda..62f2a850c3 100644
--- a/.gitlab-ci.d/base.yml
+++ b/.gitlab-ci.d/base.yml
@@ -28,6 +28,10 @@
- if: '$QEMU_JOB_ONLY_FORKS == "1" && $CI_PROJECT_NAMESPACE == "qemu-project"'
when: never
+ # Forks don't get pipelines unless QEMU_CI=1 or QEMU_CI=2 is set
+ - if: '$QEMU_CI != "1" && $QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != "qemu-project"'
+ when: never
+
# Avocado jobs don't run in forks unless $QEMU_CI_AVOCADO_TESTING is set
- if: '$QEMU_JOB_AVOCADO && $QEMU_CI_AVOCADO_TESTING != "1" && $CI_PROJECT_NAMESPACE != "qemu-project"'
when: never
@@ -59,5 +63,10 @@
# an earlier criteria
#############################################################
+ # Forks pipeline jobs don't start automatically unless
+ # QEMU_CI=2 is set
+ - if: '$QEMU_CI != "2" && $CI_PROJECT_NAMESPACE != "qemu-project"'
+ when: manual
+
# Jobs can run if any jobs they depend on were successfull
- when: on_success
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH v1 33/33] docs/devel: clean-up the CI links in the docs
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
` (31 preceding siblings ...)
2022-05-27 15:36 ` [PATCH v1 32/33] gitlab: don't run CI jobs in forks by default Alex Bennée
@ 2022-05-27 15:36 ` Alex Bennée
2022-05-27 15:52 ` Daniel P. Berrangé
32 siblings, 1 reply; 55+ messages in thread
From: Alex Bennée @ 2022-05-27 15:36 UTC (permalink / raw)
To: qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Alex Bennée
There where some broken links so fix those up with proper references
to the devel docs. I also did a little light copy-editing to reflect
the current state and broke up a paragraph to reduce the "wall of
text" effect.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
docs/devel/ci-jobs.rst.inc | 2 ++
docs/devel/ci.rst | 11 +++++-----
docs/devel/submitting-a-patch.rst | 36 ++++++++++++++++---------------
docs/devel/testing.rst | 2 ++
4 files changed, 29 insertions(+), 22 deletions(-)
diff --git a/docs/devel/ci-jobs.rst.inc b/docs/devel/ci-jobs.rst.inc
index 13d448b54d..6d88449250 100644
--- a/docs/devel/ci-jobs.rst.inc
+++ b/docs/devel/ci-jobs.rst.inc
@@ -1,3 +1,5 @@
+.. _ci_var:
+
Custom CI/CD variables
======================
diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
index d106610096..8f4a28fbdf 100644
--- a/docs/devel/ci.rst
+++ b/docs/devel/ci.rst
@@ -1,12 +1,13 @@
+.. _ci:
+
==
CI
==
-QEMU has configurations enabled for a number of different CI services.
-The most up to date information about them and their status can be
-found at::
-
- https://wiki.qemu.org/Testing/CI
+Most of QEMU's CI is run on GitLab's infrastructure although a number
+of other CI services are used for specialised purposes. The most up to
+date information about them and their status can be found on the
+`project wiki testing page<https://wiki.qemu.org/Testing/CI>`_.
.. include:: ci-definitions.rst.inc
.. include:: ci-jobs.rst.inc
diff --git a/docs/devel/submitting-a-patch.rst b/docs/devel/submitting-a-patch.rst
index e51259eb9c..d3876ec1b7 100644
--- a/docs/devel/submitting-a-patch.rst
+++ b/docs/devel/submitting-a-patch.rst
@@ -204,23 +204,25 @@ log`` for these keywords for example usage.
Test your patches
~~~~~~~~~~~~~~~~~
-Although QEMU has `continuous integration
-services <Testing#Continuous_Integration>`__ that attempt to test
-patches submitted to the list, it still saves everyone time if you have
-already tested that your patch compiles and works. Because QEMU is such
-a large project, it's okay to use configure arguments to limit what is
-built for faster turnaround during your development time; but it is
-still wise to also check that your patches work with a full build before
-submitting a series, especially if your changes might have an unintended
-effect on other areas of the code you don't normally experiment with.
-See `Testing <Testing>`__ for more details on what tests are available.
-Also, it is a wise idea to include a testsuite addition as part of your
-patches - either to ensure that future changes won't regress your new
-feature, or to add a test which exposes the bug that the rest of your
-series fixes. Keeping separate commits for the test and the fix allows
-reviewers to rebase the test to occur first to prove it catches the
-problem, then again to place it last in the series so that bisection
-doesn't land on a known-broken state.
+Although QEMU uses various :ref:`ci` services that attempt to test
+patches submitted to the list, it still saves everyone time if you
+have already tested that your patch compiles and works. Because QEMU
+is such a large project the default configuration won't create a
+testing pipeline on GitLab when a branch is pushed. See the :ref:`CI
+variable documentation<ci_var>` for details on how to control the
+running of tests; but it is still wise to also check that your patches
+work with a full build before submitting a series, especially if your
+changes might have an unintended effect on other areas of the code you
+don't normally experiment with. See :ref:`testing` for more details on
+what tests are available.
+
+Also, it is a wise idea to include a testsuite addition as part of
+your patches - either to ensure that future changes won't regress your
+new feature, or to add a test which exposes the bug that the rest of
+your series fixes. Keeping separate commits for the test and the fix
+allows reviewers to rebase the test to occur first to prove it catches
+the problem, then again to place it last in the series so that
+bisection doesn't land on a known-broken state.
.. _submitting_your_patches:
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 5b60a31807..3f6ebd5073 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -1,3 +1,5 @@
+.. _testing:
+
Testing in QEMU
===============
--
2.30.2
^ permalink raw reply related [flat|nested] 55+ messages in thread
* Re: [PATCH v1 05/33] tests/lcitool: fix up indentation to correct style
2022-05-27 15:35 ` [PATCH v1 05/33] tests/lcitool: fix up indentation to correct style Alex Bennée
@ 2022-05-27 15:43 ` Daniel P. Berrangé
0 siblings, 0 replies; 55+ messages in thread
From: Daniel P. Berrangé @ 2022-05-27 15:43 UTC (permalink / raw)
To: Alex Bennée
Cc: qemu-devel, fam, f4bug, aurelien, pbonzini, stefanha, crosa,
Thomas Huth, Wainer dos Santos Moschetta, Beraldo Leal
On Fri, May 27, 2022 at 04:35:35PM +0100, Alex Bennée wrote:
> 3 space indentation snuck into the initial commit. Clean it up before
> we let it get established. I've also:
>
> - removed unused os import
> - added double lines between functions
> - added some comments and grouped and sorted the generation stanzas
>
> My lint tool is also recommending using f-strings but that requires
> python 3.6.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: Daniel P. Berrangé <berrange@redhat.com>
> ---
> tests/lcitool/refresh | 134 ++++++++++++++++++++++++------------------
> 1 file changed, 76 insertions(+), 58 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 06/33] tests/docker: update debian-armhf-cross with lcitool
2022-05-27 15:35 ` [PATCH v1 06/33] tests/docker: update debian-armhf-cross with lcitool Alex Bennée
@ 2022-05-27 15:44 ` Daniel P. Berrangé
0 siblings, 0 replies; 55+ messages in thread
From: Daniel P. Berrangé @ 2022-05-27 15:44 UTC (permalink / raw)
To: Alex Bennée
Cc: qemu-devel, fam, f4bug, aurelien, pbonzini, stefanha, crosa,
Thomas Huth, Wainer dos Santos Moschetta, Beraldo Leal
On Fri, May 27, 2022 at 04:35:36PM +0100, Alex Bennée wrote:
> Use lcitool to update debian-armhf-cross to a Debian 11 based system.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .gitlab-ci.d/container-cross.yml | 3 +-
> tests/docker/Makefile.include | 1 -
> .../dockerfiles/debian-armhf-cross.docker | 184 +++++++++++++++---
> tests/lcitool/refresh | 5 +
> 4 files changed, 166 insertions(+), 27 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 07/33] tests/docker: update debian-armel-cross with lcitool
2022-05-27 15:35 ` [PATCH v1 07/33] tests/docker: update debian-armel-cross " Alex Bennée
@ 2022-05-27 15:45 ` Daniel P. Berrangé
0 siblings, 0 replies; 55+ messages in thread
From: Daniel P. Berrangé @ 2022-05-27 15:45 UTC (permalink / raw)
To: Alex Bennée
Cc: qemu-devel, fam, f4bug, aurelien, pbonzini, stefanha, crosa,
Thomas Huth, Wainer dos Santos Moschetta, Beraldo Leal
On Fri, May 27, 2022 at 04:35:37PM +0100, Alex Bennée wrote:
> Use lcitool to update debian-armel-cross to a Debian 11 based system.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .gitlab-ci.d/container-cross.yml | 3 +-
> tests/docker/Makefile.include | 1 -
> .../dockerfiles/debian-armel-cross.docker | 178 ++++++++++++++++--
> tests/lcitool/refresh | 5 +
> 4 files changed, 164 insertions(+), 23 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 08/33] tests/docker: update debian-mipsel-cross with lcitool
2022-05-27 15:35 ` [PATCH v1 08/33] tests/docker: update debian-mipsel-cross " Alex Bennée
@ 2022-05-27 15:45 ` Daniel P. Berrangé
0 siblings, 0 replies; 55+ messages in thread
From: Daniel P. Berrangé @ 2022-05-27 15:45 UTC (permalink / raw)
To: Alex Bennée
Cc: qemu-devel, fam, f4bug, aurelien, pbonzini, stefanha, crosa,
Thomas Huth, Wainer dos Santos Moschetta, Beraldo Leal,
Jiaxun Yang
On Fri, May 27, 2022 at 04:35:38PM +0100, Alex Bennée wrote:
> Use lcitool to update debian-mipsel-cross to a Debian 11 based system.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .gitlab-ci.d/container-cross.yml | 3 +-
> tests/docker/Makefile.include | 1 -
> .../dockerfiles/debian-mipsel-cross.docker | 179 +++++++++++++++---
> tests/lcitool/refresh | 5 +
> 4 files changed, 161 insertions(+), 27 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 09/33] tests/docker: update debian-mips64el-cross with lcitool
2022-05-27 15:35 ` [PATCH v1 09/33] tests/docker: update debian-mips64el-cross " Alex Bennée
@ 2022-05-27 15:46 ` Daniel P. Berrangé
0 siblings, 0 replies; 55+ messages in thread
From: Daniel P. Berrangé @ 2022-05-27 15:46 UTC (permalink / raw)
To: Alex Bennée
Cc: qemu-devel, fam, f4bug, aurelien, pbonzini, stefanha, crosa,
Thomas Huth, Wainer dos Santos Moschetta, Beraldo Leal,
Jiaxun Yang
On Fri, May 27, 2022 at 04:35:39PM +0100, Alex Bennée wrote:
> Use lcitool to update debian-mips64el-cross to a Debian 11 based system.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .gitlab-ci.d/container-cross.yml | 3 +-
> tests/docker/Makefile.include | 1 -
> .../dockerfiles/debian-mips64el-cross.docker | 177 +++++++++++++++---
> tests/lcitool/refresh | 5 +
> 4 files changed, 159 insertions(+), 27 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 10/33] tests/docker: update debian-ppc64el-cross with lcitool
2022-05-27 15:35 ` [PATCH v1 10/33] tests/docker: update debian-ppc64el-cross " Alex Bennée
@ 2022-05-27 15:46 ` Daniel P. Berrangé
0 siblings, 0 replies; 55+ messages in thread
From: Daniel P. Berrangé @ 2022-05-27 15:46 UTC (permalink / raw)
To: Alex Bennée
Cc: qemu-devel, fam, f4bug, aurelien, pbonzini, stefanha, crosa,
Thomas Huth, Wainer dos Santos Moschetta, Beraldo Leal
On Fri, May 27, 2022 at 04:35:40PM +0100, Alex Bennée wrote:
> Use lcitool to update debian-ppc64el-cross to a Debian 11 based system.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .gitlab-ci.d/container-cross.yml | 3 +-
> tests/docker/Makefile.include | 1 -
> .../dockerfiles/debian-ppc64el-cross.docker | 178 +++++++++++++++---
> tests/lcitool/refresh | 5 +
> 4 files changed, 163 insertions(+), 24 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 11/33] tests/docker: update debian-amd64 with lcitool
2022-05-27 15:35 ` [PATCH v1 11/33] tests/docker: update debian-amd64 " Alex Bennée
@ 2022-05-27 15:51 ` Daniel P. Berrangé
0 siblings, 0 replies; 55+ messages in thread
From: Daniel P. Berrangé @ 2022-05-27 15:51 UTC (permalink / raw)
To: Alex Bennée
Cc: qemu-devel, fam, f4bug, aurelien, pbonzini, stefanha, crosa,
Luigi Rizzo, Giuseppe Lettieri, Vincenzo Maffione, Thomas Huth,
Wainer dos Santos Moschetta, Beraldo Leal
On Fri, May 27, 2022 at 04:35:41PM +0100, Alex Bennée wrote:
> The one minor wrinkle we need to account for is the netmap support
> still requires building from source. We also include cscope and GNU
> global as they are used in one of the builds.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Cc: Philippe Mathieu-Daudé <f4bug@amsat.org>
> Cc: Luigi Rizzo <rizzo@iet.unipi.it>
> Cc: Giuseppe Lettieri <g.lettieri@iet.unipi.it>
> Cc: Vincenzo Maffione <v.maffione@gmail.com>
> ---
> .gitlab-ci.d/containers.yml | 3 +-
> tests/docker/dockerfiles/debian-amd64.docker | 194 ++++++++++++++-----
> tests/lcitool/refresh | 19 ++
> 3 files changed, 164 insertions(+), 52 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> +# Netmap still needs to be manually built as it is yet to be packaged
> +# into a distro. We also add cscope and gtags which are used in the CI
> +# test
> +debian11_extras = [
> + "# netmap/cscope/global\n",
> + "RUN DEBIAN_FRONTEND=noninteractive eatmydata \\\n",
> + " apt install -y --no-install-recommends \\\n",
> + " cscope\\\n",
> + " global\\\n",
> + " linux-headers-amd64\n",
> + "RUN git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap\n",
> + "RUN cd /usr/src/netmap && git checkout v11.3\n",
> + "RUN cd /usr/src/netmap/LINUX && ./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && make install\n",
This will result in many layers bloating the image a little.
We could optimize it with approximately
[
RUN export DEBIAN_FRONTEND=noninteractive && \
eatmydata apt install -y --no-install-recommends \
cscope \
global \
linux-headers-amd64 &&
git clone https://github.com/luigirizzo/netmap.git /usr/src/netmap && \
cd /usr/src/netmap && \
git checkout v11.3 && \0
cd LINUX && \
./configure --no-drivers --no-apps --kernel-dir=$(ls -d /usr/src/linux-headers-*-amd64) && \
make install && \
cd / && \
rm -rf /usr/src/netmap &&
eatmydata apt remove cscope global linux-headers-amd64
(untested)
Essentially this means we only add 1 layer to the docker image
and it only contains the final binary bits
Still, what you have is what already exists in tree, so on
that basis
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 33/33] docs/devel: clean-up the CI links in the docs
2022-05-27 15:36 ` [PATCH v1 33/33] docs/devel: clean-up the CI links in the docs Alex Bennée
@ 2022-05-27 15:52 ` Daniel P. Berrangé
0 siblings, 0 replies; 55+ messages in thread
From: Daniel P. Berrangé @ 2022-05-27 15:52 UTC (permalink / raw)
To: Alex Bennée
Cc: qemu-devel, fam, f4bug, aurelien, pbonzini, stefanha, crosa
On Fri, May 27, 2022 at 04:36:03PM +0100, Alex Bennée wrote:
> There where some broken links so fix those up with proper references
> to the devel docs. I also did a little light copy-editing to reflect
> the current state and broke up a paragraph to reduce the "wall of
> text" effect.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> docs/devel/ci-jobs.rst.inc | 2 ++
> docs/devel/ci.rst | 11 +++++-----
> docs/devel/submitting-a-patch.rst | 36 ++++++++++++++++---------------
> docs/devel/testing.rst | 2 ++
> 4 files changed, 29 insertions(+), 22 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 32/33] gitlab: don't run CI jobs in forks by default
2022-05-27 15:36 ` [PATCH v1 32/33] gitlab: don't run CI jobs in forks by default Alex Bennée
@ 2022-05-27 15:57 ` Daniel P. Berrangé
2022-05-30 9:00 ` Thomas Huth
1 sibling, 0 replies; 55+ messages in thread
From: Daniel P. Berrangé @ 2022-05-27 15:57 UTC (permalink / raw)
To: Alex Bennée
Cc: qemu-devel, fam, f4bug, aurelien, pbonzini, stefanha, crosa,
Thomas Huth, Wainer dos Santos Moschetta, Beraldo Leal
On Fri, May 27, 2022 at 04:36:02PM +0100, Alex Bennée wrote:
> From: Daniel P. Berrangé <berrange@redhat.com>
>
> To preserve CI shared runner credits we don't want to run
> pipelines on every push.
>
> This sets up the config so that pipelines are never created
> for contributors by default. To override this the QEMU_CI
> variable can be set to a non-zero value. If set to 1, the
> pipeline will be created but all jobs will remain manually
> started. The contributor can selectively run jobs that they
> care about. If set to 2, the pipeline will be created and
> all jobs will immediately start.
>
> This behavior can be controlled using push variables
>
> git push -o ci.variable=QEMU_CI=1
>
> To make this more convenient define an alias
>
> git config --local alias.push-ci "push -o ci.variable=QEMU_CI=1"
> git config --local alias.push-ci-now "push -o ci.variable=QEMU_CI=2"
>
> Which lets you run
>
> git push-ci
>
> to create the pipeline, or
>
> git push-ci-now
>
> to create and run the pipeline
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> Message-Id: <20220526110705.59952-6-berrange@redhat.com>
> [AJB: fix typo, replicate alias tips in ci.rst]
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Looks good to me.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 04/33] meson.build: fix summary display of test compilers
2022-05-27 15:35 ` [PATCH v1 04/33] meson.build: fix summary display of test compilers Alex Bennée
@ 2022-05-27 17:19 ` Richard Henderson
0 siblings, 0 replies; 55+ messages in thread
From: Richard Henderson @ 2022-05-27 17:19 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa
On 5/27/22 08:35, Alex Bennée wrote:
> The recent refactoring of configure.sh dropped a number of variables
> we relied on for printing out information. Make it simpler.
>
> Fixes: eebf199c09 (tests/tcg: invoke Makefile.target directly from QEMU's makefile)
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> meson.build | 8 ++------
> 1 file changed, 2 insertions(+), 6 deletions(-)
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
r~
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 24/33] configure: enable cross-compilation of s390-ccw
2022-05-27 15:35 ` [PATCH v1 24/33] configure: enable cross-compilation of s390-ccw Alex Bennée
@ 2022-05-30 7:40 ` Thomas Huth
0 siblings, 0 replies; 55+ messages in thread
From: Thomas Huth @ 2022-05-30 7:40 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Richard Henderson, Christian Borntraeger, Cornelia Huck,
open list:S390-ccw boot
On 27/05/2022 17.35, Alex Bennée wrote:
> From: Paolo Bonzini <pbonzini@redhat.com>
>
> While container-based cross compilers are not supported, this already makes
> it possible to build s390-ccw on any machine that has s390x GCC and binutils
> installed.
>
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Message-Id: <20220517092616.1272238-14-pbonzini@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> configure | 18 +++++++++++++-----
> pc-bios/s390-ccw/netboot.mak | 2 +-
> pc-bios/s390-ccw/Makefile | 9 +++++----
> 3 files changed, 19 insertions(+), 10 deletions(-)
Acked-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 28/33] gitlab: introduce a common base job template
2022-05-27 15:35 ` [PATCH v1 28/33] gitlab: introduce a common base job template Alex Bennée
@ 2022-05-30 7:50 ` Thomas Huth
0 siblings, 0 replies; 55+ messages in thread
From: Thomas Huth @ 2022-05-30 7:50 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Wainer dos Santos Moschetta, Beraldo Leal
On 27/05/2022 17.35, Alex Bennée wrote:
> From: Daniel P. Berrangé <berrange@redhat.com>
>
> Currently job rules are spread across the various templates
> and jobs, making it hard to understand exactly what runs in
> what scenario. This leads to inconsistency in the rules and
> increased maint burden.
>
> The intent is that we introduce a common '.base_job_template'
> which will have a general purpose 'rules:' block. No other
> template or job should define 'rules:', but instead they must
> rely on the inherited rules. To allow behaviour to be tweaked,
> rules will be influenced by a number of variables with the
> naming scheme 'QEMU_JOB_nnnn'.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> Message-Id: <20220526110705.59952-2-berrange@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> docs/devel/ci-jobs.rst.inc | 36 ++++++++++++++++++++++++++++++++++-
> .gitlab-ci.d/base.yml | 28 +++++++++++++++++++++++++++
> .gitlab-ci.d/qemu-project.yml | 1 +
> 3 files changed, 64 insertions(+), 1 deletion(-)
> create mode 100644 .gitlab-ci.d/base.yml
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 29/33] gitlab: convert Cirrus jobs to .base_job_template
2022-05-27 15:35 ` [PATCH v1 29/33] gitlab: convert Cirrus jobs to .base_job_template Alex Bennée
@ 2022-05-30 7:54 ` Thomas Huth
0 siblings, 0 replies; 55+ messages in thread
From: Thomas Huth @ 2022-05-30 7:54 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Wainer dos Santos Moschetta, Beraldo Leal
On 27/05/2022 17.35, Alex Bennée wrote:
> From: Daniel P. Berrangé <berrange@redhat.com>
>
> This folds the Cirrus job rules into the base job
> template, introducing two new variables
>
> - QEMU_JOB_CIRRUS - identifies the job as making
> use of Cirrus CI via cirrus-run
>
> - QEMU_JOB_OPTIONAL - identifies the job as one
> that is not run by default, primarily due to
> resource constraints. It can be manually invoked
> by users if they wish to validate that scenario.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> Message-Id: <20220526110705.59952-3-berrange@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> docs/devel/ci-jobs.rst.inc | 14 ++++++++++++++
> .gitlab-ci.d/base.yml | 9 +++++++++
> .gitlab-ci.d/cirrus.yml | 16 ++++++++--------
> 3 files changed, 31 insertions(+), 8 deletions(-)
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 30/33] gitlab: convert static checks to .base_job_template
2022-05-27 15:36 ` [PATCH v1 30/33] gitlab: convert static checks " Alex Bennée
@ 2022-05-30 8:04 ` Thomas Huth
0 siblings, 0 replies; 55+ messages in thread
From: Thomas Huth @ 2022-05-30 8:04 UTC (permalink / raw)
To: Alex Bennée, qemu-devel, Peter Maydell, Richard Henderson
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Wainer dos Santos Moschetta, Beraldo Leal
On 27/05/2022 17.36, Alex Bennée wrote:
> From: Daniel P. Berrangé <berrange@redhat.com>
>
> This folds the static checks into using the base job
> template rules, introducing one new variable
>
> - QEMU_JOB_ONLY_FORKS - a job that should never run
> on an upstream pipeline. The information it reports
> is only applicable to contributors in a pre-submission
> scenario, not time of merge.
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> Message-Id: <20220526110705.59952-4-berrange@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
...
> diff --git a/.gitlab-ci.d/base.yml b/.gitlab-ci.d/base.yml
> index 5734caf9fe..9a0b8d7f97 100644
> --- a/.gitlab-ci.d/base.yml
> +++ b/.gitlab-ci.d/base.yml
> @@ -16,6 +16,10 @@
> - if: '$QEMU_JOB_CIRRUS && ($CIRRUS_GITHUB_REPO == "" || $CIRRUS_API_TOKEN == "")'
> when: never
>
> + # Jobs only intended for forks should always be skipped on upstram
s/upstram/upstream/
With the typo fixed:
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 23/33] configure: move symlink configuration earlier
2022-05-27 15:35 ` [PATCH v1 23/33] configure: move symlink configuration earlier Alex Bennée
@ 2022-05-30 8:31 ` Paolo Bonzini
0 siblings, 0 replies; 55+ messages in thread
From: Paolo Bonzini @ 2022-05-30 8:31 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, f4bug, aurelien, stefanha, crosa, Richard Henderson
On 5/27/22 17:35, Alex Bennée wrote:
> configure | 49 ++++++++++++++++++++++++-------------------------
> 1 file changed, 24 insertions(+), 25 deletions(-)
>
> diff --git a/configure b/configure
> index b8c21e096c..82c2ddc79a 100755
> --- a/configure
> +++ b/configure
> @@ -2187,6 +2187,30 @@ fi
>
> QEMU_GA_MSI_MINGW_BIN_PATH="$($pkg_config --variable=prefix glib-2.0)/bin"
>
> +# Set up build tree symlinks that point back into the source tree
> +# (these can be both files and directories).
> +# Caution: avoid adding files or directories here using wildcards. This
This now conflicts with master due to the QEMU_GA_MSI_MINGW_BIN_PATH
being gone. Fixing the conflict is just a matter of removing the line
from the "======"....">>>>>>" section.
Paolo
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 25/33] configure: enable cross-compilation of optionrom
2022-05-27 15:35 ` [PATCH v1 25/33] configure: enable cross-compilation of optionrom Alex Bennée
@ 2022-05-30 8:32 ` Paolo Bonzini
0 siblings, 0 replies; 55+ messages in thread
From: Paolo Bonzini @ 2022-05-30 8:32 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, f4bug, aurelien, stefanha, crosa, Richard Henderson
On 5/27/22 17:35, Alex Bennée wrote:
>
> QEMU_GA_MSI_MINGW_BIN_PATH="$($pkg_config --variable=prefix glib-2.0)/bin"
>
> +#######################################
> +# cross-compiled firmware targets
> +
> # Set up build tree symlinks that point back into the source tree
> # (these can be both files and directories).
> # Caution: avoid adding files or directories here usin
This conflicts for the same reason.
Paolo
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 31/33] gitlab: convert build/container jobs to .base_job_template
2022-05-27 15:36 ` [PATCH v1 31/33] gitlab: convert build/container jobs " Alex Bennée
@ 2022-05-30 8:53 ` Thomas Huth
2022-06-01 14:43 ` Alex Bennée
0 siblings, 1 reply; 55+ messages in thread
From: Thomas Huth @ 2022-05-30 8:53 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Wainer dos Santos Moschetta, Beraldo Leal
On 27/05/2022 17.36, Alex Bennée wrote:
> From: Daniel P. Berrangé <berrange@redhat.com>
>
> This converts the main build and container jobs to use the
> base job rules, defining the following new variables
>
> - QEMU_JOB_SKIPPED - jobs that are known to be currently
> broken and should not be run. Can still be manually
> launched if desired.
>
> - QEMU_JOB_AVOCADO - jobs that run the Avocado integration
> test harness.
>
> - QEMU_JOB_PUBLISH - jobs that publish content after the
> branch is merged upstream
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> Message-Id: <20220526110705.59952-5-berrange@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
...
> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
> index e9620c3074..ecac3ec50c 100644
> --- a/.gitlab-ci.d/buildtest.yml
> +++ b/.gitlab-ci.d/buildtest.yml
> @@ -360,12 +360,11 @@ build-cfi-aarch64:
> expire_in: 2 days
> paths:
> - build
> - rules:
> + variables:
> # FIXME: This job is often failing, likely due to out-of-memory problems in
> # the constrained containers of the shared runners. Thus this is marked as
> - # manual until the situation has been solved.
> - - when: manual
> - allow_failure: true
> + # skipped until the situation has been solved.
> + QEMU_JOB_SKIPPED: 1
>
> check-cfi-aarch64:
> extends: .native_test_job_template
> @@ -402,12 +401,11 @@ build-cfi-ppc64-s390x:
> expire_in: 2 days
> paths:
> - build
> - rules:
> + variables:
> # FIXME: This job is often failing, likely due to out-of-memory problems in
> # the constrained containers of the shared runners. Thus this is marked as
> - # manual until the situation has been solved.
> - - when: manual
> - allow_failure: true
> + # skipped until the situation has been solved.
> + QEMU_JOB_SKIPPED: 1
>
> check-cfi-ppc64-s390x:
> extends: .native_test_job_template
> @@ -579,6 +577,7 @@ build-without-default-features:
> MAKE_CHECK_ARGS: check-unit check-qtest SPEED=slow
>
> build-libvhost-user:
> + extends: .base_job_template
> stage: build
> image: $CI_REGISTRY_IMAGE/qemu/fedora:latest
> needs:
> @@ -595,10 +594,13 @@ build-tools-and-docs-debian:
> extends: .native_build_job_template
> needs:
> job: amd64-debian-container
> + # when running on 'master' we use pre-existing container
> + optional: true
This change doesn't look like it's related to the other changes in here?
Maybe mention it in the patch description at least?
Apart from that:
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 32/33] gitlab: don't run CI jobs in forks by default
2022-05-27 15:36 ` [PATCH v1 32/33] gitlab: don't run CI jobs in forks by default Alex Bennée
2022-05-27 15:57 ` Daniel P. Berrangé
@ 2022-05-30 9:00 ` Thomas Huth
1 sibling, 0 replies; 55+ messages in thread
From: Thomas Huth @ 2022-05-30 9:00 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Wainer dos Santos Moschetta, Beraldo Leal
On 27/05/2022 17.36, Alex Bennée wrote:
> From: Daniel P. Berrangé <berrange@redhat.com>
>
> To preserve CI shared runner credits we don't want to run
> pipelines on every push.
>
> This sets up the config so that pipelines are never created
> for contributors by default. To override this the QEMU_CI
> variable can be set to a non-zero value. If set to 1, the
> pipeline will be created but all jobs will remain manually
> started. The contributor can selectively run jobs that they
> care about. If set to 2, the pipeline will be created and
> all jobs will immediately start.
>
> This behavior can be controlled using push variables
>
> git push -o ci.variable=QEMU_CI=1
>
> To make this more convenient define an alias
>
> git config --local alias.push-ci "push -o ci.variable=QEMU_CI=1"
> git config --local alias.push-ci-now "push -o ci.variable=QEMU_CI=2"
>
> Which lets you run
>
> git push-ci
>
> to create the pipeline, or
>
> git push-ci-now
>
> to create and run the pipeline
>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> Message-Id: <20220526110705.59952-6-berrange@redhat.com>
> [AJB: fix typo, replicate alias tips in ci.rst]
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> docs/devel/ci-jobs.rst.inc | 38 ++++++++++++++++++++++++++++++++++++++
> .gitlab-ci.d/base.yml | 9 +++++++++
> 2 files changed, 47 insertions(+)
Please add:
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/751
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 03/33] gitlab-ci: add meson JUnit test result into report
2022-05-27 15:35 ` [PATCH v1 03/33] gitlab-ci: add meson JUnit test result into report Alex Bennée
@ 2022-05-30 9:03 ` Thomas Huth
0 siblings, 0 replies; 55+ messages in thread
From: Thomas Huth @ 2022-05-30 9:03 UTC (permalink / raw)
To: Alex Bennée, qemu-devel
Cc: fam, berrange, f4bug, aurelien, pbonzini, stefanha, crosa,
Marc-André Lureau, Wainer dos Santos Moschetta,
Beraldo Leal
On 27/05/2022 17.35, Alex Bennée wrote:
> From: Marc-André Lureau <marcandre.lureau@redhat.com>
Please add the patch description that Marc-André provided here:
https://lists.gnu.org/archive/html/qemu-devel/2022-05/msg05797.html
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> Message-Id: <20220525173411.612224-1-marcandre.lureau@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> .gitlab-ci.d/buildtest-template.yml | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/.gitlab-ci.d/buildtest-template.yml b/.gitlab-ci.d/buildtest-template.yml
> index dc6d67aacf..b381345dbc 100644
> --- a/.gitlab-ci.d/buildtest-template.yml
> +++ b/.gitlab-ci.d/buildtest-template.yml
> @@ -44,6 +44,8 @@
> expire_in: 7 days
> paths:
> - build/meson-logs/testlog.txt
> + reports:
> + junit: build/meson-logs/testlog.junit.xml
>
> .avocado_test_job_template:
> extends: .common_test_job_template
Reviewed-by: Thomas Huth <thuth@redhat.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 01/33] .gitlab-ci.d/container-cross: Fix RISC-V container dependencies / stages
2022-05-27 15:35 ` [PATCH v1 01/33] .gitlab-ci.d/container-cross: Fix RISC-V container dependencies / stages Alex Bennée
@ 2022-05-31 0:18 ` Alistair Francis
0 siblings, 0 replies; 55+ messages in thread
From: Alistair Francis @ 2022-05-31 0:18 UTC (permalink / raw)
To: Alex Bennée
Cc: qemu-devel@nongnu.org Developers, Fam Zheng, Daniel P. Berrange,
Philippe Mathieu-Daudé,
Aurelien Jarno, Paolo Bonzini, Stefan Hajnoczi, Cleber Rosa,
Thomas Huth, Wainer dos Santos Moschetta, Beraldo Leal
On Sat, May 28, 2022 at 1:36 AM Alex Bennée <alex.bennee@linaro.org> wrote:
>
> From: Thomas Huth <thuth@redhat.com>
>
> The "riscv64-debian-cross-container" job does not depend on any other
> container job from the first stage, so we can move it to the first
> stage, too.
>
> The "riscv64-debian-test-cross-container" job needs the debian11
> container, so we should add a proper "needs:" statement here.
>
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> Message-Id: <20220524093141.91012-1-thuth@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Alistair
> ---
> .gitlab-ci.d/container-cross.yml | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
> index e622ac2d21..ac15fce9b6 100644
> --- a/.gitlab-ci.d/container-cross.yml
> +++ b/.gitlab-ci.d/container-cross.yml
> @@ -125,7 +125,7 @@ ppc64el-debian-cross-container:
>
> riscv64-debian-cross-container:
> extends: .container_job_template
> - stage: containers-layer2
> + stage: containers
> # as we are currently based on 'sid/unstable' we may break so...
> allow_failure: true
> variables:
> @@ -135,6 +135,7 @@ riscv64-debian-cross-container:
> riscv64-debian-test-cross-container:
> extends: .container_job_template
> stage: containers-layer2
> + needs: ['amd64-debian11-container']
> variables:
> NAME: debian-riscv64-test-cross
>
> --
> 2.30.2
>
>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH v1 31/33] gitlab: convert build/container jobs to .base_job_template
2022-05-30 8:53 ` Thomas Huth
@ 2022-06-01 14:43 ` Alex Bennée
0 siblings, 0 replies; 55+ messages in thread
From: Alex Bennée @ 2022-06-01 14:43 UTC (permalink / raw)
To: Thomas Huth
Cc: qemu-devel, fam, berrange, f4bug, aurelien, pbonzini, stefanha,
crosa, Wainer dos Santos Moschetta, Beraldo Leal
Thomas Huth <thuth@redhat.com> writes:
> On 27/05/2022 17.36, Alex Bennée wrote:
>> From: Daniel P. Berrangé <berrange@redhat.com>
>> This converts the main build and container jobs to use the
>> base job rules, defining the following new variables
>> - QEMU_JOB_SKIPPED - jobs that are known to be currently
>> broken and should not be run. Can still be manually
>> launched if desired.
>> - QEMU_JOB_AVOCADO - jobs that run the Avocado integration
>> test harness.
>> - QEMU_JOB_PUBLISH - jobs that publish content after the
>> branch is merged upstream
>> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>> Message-Id: <20220526110705.59952-5-berrange@redhat.com>
>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
>> ---
> ...
>> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
>> index e9620c3074..ecac3ec50c 100644
>> --- a/.gitlab-ci.d/buildtest.yml
>> +++ b/.gitlab-ci.d/buildtest.yml
>> @@ -360,12 +360,11 @@ build-cfi-aarch64:
>> expire_in: 2 days
>> paths:
>> - build
>> - rules:
>> + variables:
>> # FIXME: This job is often failing, likely due to out-of-memory problems in
>> # the constrained containers of the shared runners. Thus this is marked as
>> - # manual until the situation has been solved.
>> - - when: manual
>> - allow_failure: true
>> + # skipped until the situation has been solved.
>> + QEMU_JOB_SKIPPED: 1
>> check-cfi-aarch64:
>> extends: .native_test_job_template
>> @@ -402,12 +401,11 @@ build-cfi-ppc64-s390x:
>> expire_in: 2 days
>> paths:
>> - build
>> - rules:
>> + variables:
>> # FIXME: This job is often failing, likely due to out-of-memory problems in
>> # the constrained containers of the shared runners. Thus this is marked as
>> - # manual until the situation has been solved.
>> - - when: manual
>> - allow_failure: true
>> + # skipped until the situation has been solved.
>> + QEMU_JOB_SKIPPED: 1
>> check-cfi-ppc64-s390x:
>> extends: .native_test_job_template
>> @@ -579,6 +577,7 @@ build-without-default-features:
>> MAKE_CHECK_ARGS: check-unit check-qtest SPEED=slow
>> build-libvhost-user:
>> + extends: .base_job_template
>> stage: build
>> image: $CI_REGISTRY_IMAGE/qemu/fedora:latest
>> needs:
>> @@ -595,10 +594,13 @@ build-tools-and-docs-debian:
>> extends: .native_build_job_template
>> needs:
>> job: amd64-debian-container
>> + # when running on 'master' we use pre-existing container
>> + optional: true
>
> This change doesn't look like it's related to the other changes in
> here? Maybe mention it in the patch description at least?
It is because the QEMU_JOB_PUBLISH stage is run on master so we don't
really need to rebuild the container from staging to run it. I've
mentioned it in the commit.
>
> Apart from that:
> Reviewed-by: Thomas Huth <thuth@redhat.com>
--
Alex Bennée
^ permalink raw reply [flat|nested] 55+ messages in thread
end of thread, other threads:[~2022-06-01 14:46 UTC | newest]
Thread overview: 55+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-27 15:35 [PATCH v1 00/33] testing/next (gitlab, junit, lcitool, x-compile) Alex Bennée
2022-05-27 15:35 ` [PATCH v1 01/33] .gitlab-ci.d/container-cross: Fix RISC-V container dependencies / stages Alex Bennée
2022-05-31 0:18 ` Alistair Francis
2022-05-27 15:35 ` [PATCH v1 02/33] .gitlab-ci.d/crossbuilds: Fix the dependency of the cross-i386-tci job Alex Bennée
2022-05-27 15:35 ` [PATCH v1 03/33] gitlab-ci: add meson JUnit test result into report Alex Bennée
2022-05-30 9:03 ` Thomas Huth
2022-05-27 15:35 ` [PATCH v1 04/33] meson.build: fix summary display of test compilers Alex Bennée
2022-05-27 17:19 ` Richard Henderson
2022-05-27 15:35 ` [PATCH v1 05/33] tests/lcitool: fix up indentation to correct style Alex Bennée
2022-05-27 15:43 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 06/33] tests/docker: update debian-armhf-cross with lcitool Alex Bennée
2022-05-27 15:44 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 07/33] tests/docker: update debian-armel-cross " Alex Bennée
2022-05-27 15:45 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 08/33] tests/docker: update debian-mipsel-cross " Alex Bennée
2022-05-27 15:45 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 09/33] tests/docker: update debian-mips64el-cross " Alex Bennée
2022-05-27 15:46 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 10/33] tests/docker: update debian-ppc64el-cross " Alex Bennée
2022-05-27 15:46 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 11/33] tests/docker: update debian-amd64 " Alex Bennée
2022-05-27 15:51 ` Daniel P. Berrangé
2022-05-27 15:35 ` [PATCH v1 12/33] configure: do not define or use the CPP variable Alex Bennée
2022-05-27 15:35 ` [PATCH v1 13/33] build: clean up ninja invocation Alex Bennée
2022-05-27 15:35 ` [PATCH v1 14/33] build: add a more generic way to specify make->ninja dependencies Alex Bennée
2022-05-27 15:35 ` [PATCH v1 15/33] build: do a full build before running TCG tests Alex Bennée
2022-05-27 15:35 ` [PATCH v1 16/33] configure, meson: move symlinking of ROMs to meson Alex Bennée
2022-05-27 15:35 ` [PATCH v1 17/33] tests/tcg: correct target CPU for sparc32 Alex Bennée
2022-05-27 15:35 ` [PATCH v1 18/33] tests/tcg: merge configure.sh back into main configure script Alex Bennée
2022-05-27 15:35 ` [PATCH v1 19/33] configure: add missing cross compiler fallbacks Alex Bennée
2022-05-27 15:35 ` [PATCH v1 20/33] configure: handle host compiler in probe_target_compiler Alex Bennée
2022-05-27 15:35 ` [PATCH v1 21/33] configure: introduce --cross-prefix-*= Alex Bennée
2022-05-27 15:35 ` [PATCH v1 22/33] configure: include more binutils in tests/tcg makefile Alex Bennée
2022-05-27 15:35 ` [PATCH v1 23/33] configure: move symlink configuration earlier Alex Bennée
2022-05-30 8:31 ` Paolo Bonzini
2022-05-27 15:35 ` [PATCH v1 24/33] configure: enable cross-compilation of s390-ccw Alex Bennée
2022-05-30 7:40 ` Thomas Huth
2022-05-27 15:35 ` [PATCH v1 25/33] configure: enable cross-compilation of optionrom Alex Bennée
2022-05-30 8:32 ` Paolo Bonzini
2022-05-27 15:35 ` [PATCH v1 26/33] configure: enable cross compilation of vof Alex Bennée
2022-05-27 15:35 ` [PATCH v1 27/33] configure: remove unused variables from config-host.mak Alex Bennée
2022-05-27 15:35 ` [PATCH v1 28/33] gitlab: introduce a common base job template Alex Bennée
2022-05-30 7:50 ` Thomas Huth
2022-05-27 15:35 ` [PATCH v1 29/33] gitlab: convert Cirrus jobs to .base_job_template Alex Bennée
2022-05-30 7:54 ` Thomas Huth
2022-05-27 15:36 ` [PATCH v1 30/33] gitlab: convert static checks " Alex Bennée
2022-05-30 8:04 ` Thomas Huth
2022-05-27 15:36 ` [PATCH v1 31/33] gitlab: convert build/container jobs " Alex Bennée
2022-05-30 8:53 ` Thomas Huth
2022-06-01 14:43 ` Alex Bennée
2022-05-27 15:36 ` [PATCH v1 32/33] gitlab: don't run CI jobs in forks by default Alex Bennée
2022-05-27 15:57 ` Daniel P. Berrangé
2022-05-30 9:00 ` Thomas Huth
2022-05-27 15:36 ` [PATCH v1 33/33] docs/devel: clean-up the CI links in the docs Alex Bennée
2022-05-27 15:52 ` Daniel P. Berrangé
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.