[PATCH v3 00/12] automation: improvements (mostly) for arm64

[PATCH v3 00/12] automation: improvements (mostly) for arm64

[Stefano Stabellini]

Hi all,

This series does a few things:

1) it introduces a simple Xen arm64 dom0less smoke test based on QEMU
2) it introduces alpine linux builds x86 and arm64
3) it introduces two tests artifacts containers
4) it uses said artifacts to create a dom0/domU arm64 test based on QEMU

The series is v3, but in reality only 1) above was sent out before (the
first two patches). Everything else is new. All tests succeed currently.



Stefano Stabellini (12):
      automation: add a QEMU aarch64 smoke test
      automation: add dom0less to the QEMU aarch64 smoke test
      automation: pass --disable-werror for QEMUU builds if libc is musl
      automation: add alpine linux 3.12 arm64 build container
      automation: add alpine linux arm64 build test
      automation: add alpine linux 3.12 x86 build container
      automation: add alpine linux x86 build jobs
      automation: add tests artifacts
      automation: make available the tests artifacts to the pipeline
      automation: create an alpine linux arm64 test job
      automation: use the tests-artifacts kernel for qemu-smoke-arm64-gcc
      automation: add domU creation to dom0 alpine linux test

 automation/build/alpine/3.12-arm64v8.dockerfile    |  52 +++++++++
 automation/build/alpine/3.12.dockerfile            |  52 +++++++++
 automation/gitlab-ci/build.yaml                    |  60 ++++++++++
 automation/gitlab-ci/test.yaml                     |  47 ++++++++
 automation/scripts/build                           |  12 +-
 automation/scripts/qemu-alpine-arm64.sh            | 121 +++++++++++++++++++++
 automation/scripts/qemu-smoke-arm64.sh             |  93 ++++++++++++++++
 automation/tests-artifacts/Makefile                |  19 ++++
 .../tests-artifacts/alpine/3.12-arm64v8.dockerfile |  67 ++++++++++++
 .../kernel/5.9.9-arm64v8.dockerfile                |  34 ++++++
 10 files changed, 554 insertions(+), 3 deletions(-)
 create mode 100644 automation/build/alpine/3.12-arm64v8.dockerfile
 create mode 100644 automation/build/alpine/3.12.dockerfile
 create mode 100755 automation/scripts/qemu-alpine-arm64.sh
 create mode 100755 automation/scripts/qemu-smoke-arm64.sh
 create mode 100644 automation/tests-artifacts/Makefile
 create mode 100644 automation/tests-artifacts/alpine/3.12-arm64v8.dockerfile
 create mode 100644 automation/tests-artifacts/kernel/5.9.9-arm64v8.dockerfile

[PATCH v3 01/12] automation: add a QEMU aarch64 smoke test

[Stefano Stabellini]

Use QEMU to start Xen (just the hypervisor) up until it stops because
there is no dom0 kernel to boot.

It is based on the existing build job unstable-arm64v8.

Also use make -j$(nproc) to build Xen.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
Changes in v2:
- fix x86_32 build
---
 automation/gitlab-ci/test.yaml         | 22 ++++++++++++++++++
 automation/scripts/build               |  6 ++---
 automation/scripts/qemu-smoke-arm64.sh | 32 ++++++++++++++++++++++++++
 3 files changed, 57 insertions(+), 3 deletions(-)
 create mode 100755 automation/scripts/qemu-smoke-arm64.sh

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 793feafe8b..35346e3f6e 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -22,6 +22,28 @@ build-each-commit-gcc:
     - /^coverity-tested\/.*/
     - /^stable-.*/
 
+qemu-smoke-arm64-gcc:
+  stage: test
+  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
+  variables:
+    CONTAINER: debian:unstable-arm64v8
+  script:
+    - ./automation/scripts/qemu-smoke-arm64.sh 2>&1 | tee qemu-smoke-arm64.log
+  dependencies:
+    - debian-unstable-gcc-arm64
+  artifacts:
+    paths:
+      - smoke.serial
+      - '*.log'
+    when: always
+  tags:
+    - arm64
+  except:
+    - master
+    - smoke
+    - /^coverity-tested\/.*/
+    - /^stable-.*/
+
 qemu-smoke-x86-64-gcc:
   stage: test
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
diff --git a/automation/scripts/build b/automation/scripts/build
index 0cd0f3971d..7038e5eb50 100755
--- a/automation/scripts/build
+++ b/automation/scripts/build
@@ -10,9 +10,9 @@ cc-ver()
 
 # random config or default config
 if [[ "${RANDCONFIG}" == "y" ]]; then
-    make -C xen KCONFIG_ALLCONFIG=tools/kconfig/allrandom.config randconfig
+    make -j$(nproc) -C xen KCONFIG_ALLCONFIG=tools/kconfig/allrandom.config randconfig
 else
-    make -C xen defconfig
+    make -j$(nproc) -C xen defconfig
 fi
 
 # build up our configure options
@@ -45,7 +45,7 @@ make -j$(nproc) dist
 # Extract artifacts to avoid getting rewritten by customised builds
 cp xen/.config xen-config
 mkdir binaries
-if [[ "${XEN_TARGET_ARCH}" == "x86_64" ]]; then
+if [[ "${XEN_TARGET_ARCH}" != "x86_32" ]]; then
     cp xen/xen binaries/xen
 fi
 
diff --git a/automation/scripts/qemu-smoke-arm64.sh b/automation/scripts/qemu-smoke-arm64.sh
new file mode 100755
index 0000000000..a7efbf8b6f
--- /dev/null
+++ b/automation/scripts/qemu-smoke-arm64.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+set -ex
+
+# Install QEMU
+export DEBIAN_FRONTENT=noninteractive
+apt-get -qy update
+apt-get -qy install --no-install-recommends qemu-system-aarch64 \
+                                            u-boot-qemu
+
+# XXX Silly workaround to get the following QEMU command to work
+cp /usr/share/qemu/pvh.bin /usr/share/qemu/efi-virtio.rom
+qemu-system-aarch64 \
+   -machine virtualization=true \
+   -cpu cortex-a57 -machine type=virt \
+   -m 512 -display none \
+   -machine dumpdtb=binaries/virt-gicv3.dtb
+
+rm -f smoke.serial
+set +e
+echo "  booti 0x49000000 - 0x44000000" | timeout -k 1 30 qemu-system-aarch64 \
+    -machine virtualization=true \
+    -cpu cortex-a57 -machine type=virt \
+    -m 512 -monitor none -serial stdio \
+    -no-reboot \
+    -device loader,file=binaries/virt-gicv3.dtb,force-raw=on,addr=0x44000000 \
+    -device loader,file=binaries/xen,force-raw=on,addr=0x49000000 \
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin |& tee smoke.serial
+
+set -e
+grep -q 'LOADING DOMAIN 0' smoke.serial || exit 1
+exit 0
-- 
2.17.1

[PATCH v3 02/12] automation: add dom0less to the QEMU aarch64 smoke test

[Stefano Stabellini]

Add a trivial dom0less test:
- fetch the Debian arm64 kernel and use it ad dom0/U kernel
- use busybox-static to create a trivial dom0/U ramdisk
- use ImageBuilder to generate the uboot boot script automatically
- install and use u-boot from the Debian package to start the test
- binaries are loaded from uboot via tftp

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
Changes in v3:
- don't hardcode linux kernel version in testing script

Changes in v2:
- use the Debian kernel for testing
---
 automation/scripts/qemu-smoke-arm64.sh | 81 +++++++++++++++++++++++---
 1 file changed, 74 insertions(+), 7 deletions(-)

diff --git a/automation/scripts/qemu-smoke-arm64.sh b/automation/scripts/qemu-smoke-arm64.sh
index a7efbf8b6f..9bf4488115 100755
--- a/automation/scripts/qemu-smoke-arm64.sh
+++ b/automation/scripts/qemu-smoke-arm64.sh
@@ -6,27 +6,94 @@ set -ex
 export DEBIAN_FRONTENT=noninteractive
 apt-get -qy update
 apt-get -qy install --no-install-recommends qemu-system-aarch64 \
-                                            u-boot-qemu
+                                            u-boot-qemu \
+                                            u-boot-tools \
+                                            device-tree-compiler \
+                                            busybox-static \
+                                            cpio
+
+cd binaries
+apt-get download linux-image-*[0-9]-arm64
+dpkg -i --ignore-depends=initramfs-tools ./linux-image-*arm64.deb || true
+cp /boot/vmlinuz-*arm64 ./Image
+cd ..
 
 # XXX Silly workaround to get the following QEMU command to work
 cp /usr/share/qemu/pvh.bin /usr/share/qemu/efi-virtio.rom
 qemu-system-aarch64 \
    -machine virtualization=true \
    -cpu cortex-a57 -machine type=virt \
-   -m 512 -display none \
+   -m 1024 -display none \
    -machine dumpdtb=binaries/virt-gicv3.dtb
+# XXX disable pl061 to avoid Linux crash
+dtc -I dtb -O dts binaries/virt-gicv3.dtb > binaries/virt-gicv3.dts
+sed 's/compatible = "arm,pl061.*/status = "disabled";/g' binaries/virt-gicv3.dts > binaries/virt-gicv3-edited.dts
+dtc -I dts -O dtb binaries/virt-gicv3-edited.dts > binaries/virt-gicv3.dtb
+
+
+# Busybox Dom0
+mkdir -p initrd
+mkdir -p initrd/bin
+mkdir -p initrd/sbin
+mkdir -p initrd/etc
+mkdir -p initrd/dev
+mkdir -p initrd/proc
+mkdir -p initrd/sys
+mkdir -p initrd/lib
+mkdir -p initrd/var
+mkdir -p initrd/mnt
+cp /bin/busybox initrd/bin/busybox
+initrd/bin/busybox --install initrd/bin
+echo "#!/bin/sh
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+mount -t devtmpfs devtmpfs /dev
+/bin/sh" > initrd/init
+chmod +x initrd/init
+cd initrd
+find . | cpio --create --format='newc' | gzip > ../binaries/initrd
+cd ..
+
+
+# ImageBuilder
+echo 'MEMORY_START="0x40000000"
+MEMORY_END="0x80000000"
 
+DEVICE_TREE="virt-gicv3.dtb"
+XEN="xen"
+DOM0_KERNEL="Image"
+DOM0_RAMDISK="initrd"
+XEN_CMD="console=dtuart dom0_mem=512M"
+
+NUM_DOMUS=1
+DOMU_KERNEL[0]="Image"
+DOMU_RAMDISK[0]="initrd"
+DOMU_MEM[0]="256"
+
+LOAD_CMD="tftpb"
+UBOOT_SOURCE="boot.source"
+UBOOT_SCRIPT="boot.scr"' > binaries/config
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/config
+
+
+# Run the test
 rm -f smoke.serial
 set +e
-echo "  booti 0x49000000 - 0x44000000" | timeout -k 1 30 qemu-system-aarch64 \
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"| \
+timeout -k 1 240 \
+qemu-system-aarch64 \
     -machine virtualization=true \
     -cpu cortex-a57 -machine type=virt \
-    -m 512 -monitor none -serial stdio \
+    -m 1024 -monitor none -serial stdio \
+    -smp 2 \
     -no-reboot \
-    -device loader,file=binaries/virt-gicv3.dtb,force-raw=on,addr=0x44000000 \
-    -device loader,file=binaries/xen,force-raw=on,addr=0x49000000 \
+    -device virtio-net-pci,netdev=n0 \
+    -netdev user,id=n0,tftp=binaries \
     -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin |& tee smoke.serial
 
 set -e
-grep -q 'LOADING DOMAIN 0' smoke.serial || exit 1
+(grep -q "^BusyBox" smoke.serial && grep -q "DOM1: BusyBox" smoke.serial) || exit 1
 exit 0
-- 
2.17.1

[PATCH v3 03/12] automation: pass --disable-werror for QEMUU builds if libc is musl

[Stefano Stabellini]

QEMU upstream builds with warnings when libc is musl:

  #warning redirecting incorrect #include <sys/signal.h> to <signal.h>

Disable -Werror by passing --disable-werror to the QEMUU config script
if libc is musl.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
 automation/scripts/build | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/automation/scripts/build b/automation/scripts/build
index 7038e5eb50..3fb2fe134c 100755
--- a/automation/scripts/build
+++ b/automation/scripts/build
@@ -28,6 +28,11 @@ if [[ "${CC}" == "clang"* ]]; then
     cfgargs+=("--disable-stubdom")
 fi
 
+# disable --disable-werror for QEMUU when building with MUSL
+if ! test -z "$(ldd /bin/ls|grep musl|head -1)"; then
+	cfgargs+=("--with-extra-qemuu-configure-args=\"--disable-werror\"")
+fi
+
 # Qemu requires Python 3.5 or later
 if ! type python3 || python3 -c "import sys; res = sys.version_info < (3, 5); exit(not(res))"; then
     cfgargs+=("--with-system-qemu=/bin/false")
-- 
2.17.1

[PATCH v3 04/12] automation: add alpine linux 3.12 arm64 build container

[Stefano Stabellini]

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

The build container will be used for a new Alpine Linux 3.12 arm64 build
test.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
 .../build/alpine/3.12-arm64v8.dockerfile      | 52 +++++++++++++++++++
 1 file changed, 52 insertions(+)
 create mode 100644 automation/build/alpine/3.12-arm64v8.dockerfile

diff --git a/automation/build/alpine/3.12-arm64v8.dockerfile b/automation/build/alpine/3.12-arm64v8.dockerfile
new file mode 100644
index 0000000000..d6cdf5b200
--- /dev/null
+++ b/automation/build/alpine/3.12-arm64v8.dockerfile
@@ -0,0 +1,52 @@
+FROM arm64v8/alpine:3.12
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN \
+  # apk
+  apk update && \
+  \
+  # xen build deps
+  apk add argp-standalone && \
+  apk add autoconf && \
+  apk add automake && \
+  apk add bash && \
+  apk add curl && \
+  apk add curl-dev && \
+  apk add dev86 && \
+  apk add dtc-dev && \
+  apk add gcc  && \
+  apk add gettext && \
+  apk add git && \
+  apk add iasl && \
+  apk add libaio-dev && \
+  apk add libfdt && \
+  apk add linux-headers && \
+  apk add make && \
+  apk add musl-dev  && \
+  apk add ncurses-dev && \
+  apk add patch  && \
+  apk add python3-dev && \
+  apk add texinfo && \
+  apk add util-linux-dev && \
+  apk add xz-dev && \
+  apk add yajl-dev && \
+  apk add zlib-dev && \
+  \
+  # qemu build deps
+  apk add bison && \
+  apk add flex && \
+  apk add glib-dev && \
+  apk add libattr && \
+  apk add libcap-ng-dev && \
+  apk add pixman-dev && \
+  \
+  # cleanup
+  rm -rf /tmp/* && \
+  rm -f /var/cache/apk/*
-- 
2.17.1

[PATCH v3 05/12] automation: add alpine linux arm64 build test

[Stefano Stabellini]

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

Based on the arm64 3.12 build container

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
 automation/gitlab-ci/build.yaml | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 4bd1cfc1c0..fa38c39d6a 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -434,3 +434,12 @@ debian-unstable-gcc-debug-arm64-randconfig:
     CONTAINER: debian:unstable-arm64v8
     RANDCONFIG: y
 
+alpine-3.12-gcc-arm64:
+  extends: .gcc-arm64-build
+  variables:
+    CONTAINER: alpine:3.12-arm64v8
+
+alpine-3.12-gcc-debug-arm64:
+  extends: .gcc-arm64-build-debug
+  variables:
+    CONTAINER: alpine:3.12-arm64v8
-- 
2.17.1

[PATCH v3 06/12] automation: add alpine linux 3.12 x86 build container

[Stefano Stabellini]

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
 automation/build/alpine/3.12.dockerfile | 52 +++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
 create mode 100644 automation/build/alpine/3.12.dockerfile

diff --git a/automation/build/alpine/3.12.dockerfile b/automation/build/alpine/3.12.dockerfile
new file mode 100644
index 0000000000..2c02417ee6
--- /dev/null
+++ b/automation/build/alpine/3.12.dockerfile
@@ -0,0 +1,52 @@
+FROM alpine:3.12
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN \
+  # apk
+  apk update && \
+  \
+  # xen build deps
+  apk add argp-standalone && \
+  apk add autoconf && \
+  apk add automake && \
+  apk add bash && \
+  apk add curl && \
+  apk add curl-dev && \
+  apk add dev86 && \
+  apk add gcc  && \
+  apk add clang  && \
+  apk add gettext && \
+  apk add git && \
+  apk add iasl && \
+  apk add libaio-dev && \
+  apk add linux-headers && \
+  apk add make && \
+  apk add musl-dev  && \
+  apk add libc6-compat && \
+  apk add ncurses-dev && \
+  apk add patch  && \
+  apk add python3-dev && \
+  apk add texinfo && \
+  apk add util-linux-dev && \
+  apk add xz-dev && \
+  apk add yajl-dev && \
+  apk add zlib-dev && \
+  \
+  # qemu build deps
+  apk add bison && \
+  apk add flex && \
+  apk add glib-dev && \
+  apk add libattr && \
+  apk add libcap-ng-dev && \
+  apk add pixman-dev && \
+  \
+  # cleanup
+  rm -rf /tmp/* && \
+  rm -f /var/cache/apk/*
-- 
2.17.1

[PATCH v3 07/12] automation: add alpine linux x86 build jobs

[Stefano Stabellini]

Allow failure for these jobs. Currently they fail because hvmloader
doesn't build with musl. The failures don't block the pipeline.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
This patch is probably required: https://github.com/alpinelinux/aports/blob/master/main/xen/musl-hvmloader-fix-stdint.patch
---
 automation/gitlab-ci/build.yaml | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index fa38c39d6a..c48c0f3d66 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -410,6 +410,31 @@ opensuse-leap-gcc-debug:
   variables:
     CONTAINER: suse:opensuse-leap
 
+alpine-3.12-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: alpine:3.12
+  allow_failure: true
+
+alpine-3.12-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: alpine:3.12
+  allow_failure: true
+
+alpine-3.12-clang:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: alpine:3.12
+  allow_failure: true
+
+alpine-3.12-clang-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: alpine:3.12
+  allow_failure: true
+
+
 # Arm builds
 
 debian-unstable-gcc-arm64:
-- 
2.17.1

[PATCH v3 08/12] automation: add tests artifacts

[Stefano Stabellini]

Some tests (soon to come) will require pre-built binaries to run, such
as the Linux kernel binary. We don't want to rebuild the Linux kernel
for each gitlab-ci run: these builds should not be added to the current
list of build jobs.

Instead, create additional containers that today are built and uploaded
manually, but could be re-built automatically. The containers build the
required binarires during the "docker build" step and store them inside
the container itself.

gitlab-ci will be able to fetch these pre-built binaries during the
regular test runs, saving cycles.

Add two tests artifacts containers:
- one to build the Linux kernel ARM64
- one to create an Alpine Linux ARM64 rootfs for Dom0

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
 automation/tests-artifacts/Makefile           | 19 ++++++
 .../alpine/3.12-arm64v8.dockerfile            | 67 +++++++++++++++++++
 .../kernel/5.9.9-arm64v8.dockerfile           | 34 ++++++++++
 3 files changed, 120 insertions(+)
 create mode 100644 automation/tests-artifacts/Makefile
 create mode 100644 automation/tests-artifacts/alpine/3.12-arm64v8.dockerfile
 create mode 100644 automation/tests-artifacts/kernel/5.9.9-arm64v8.dockerfile

diff --git a/automation/tests-artifacts/Makefile b/automation/tests-artifacts/Makefile
new file mode 100644
index 0000000000..8ca71b78ad
--- /dev/null
+++ b/automation/tests-artifacts/Makefile
@@ -0,0 +1,19 @@
+
+# the base of where these containers will appear
+REGISTRY := registry.gitlab.com/xen-project/xen/tests-artifacts
+CONTAINERS = $(subst .dockerfile,,$(wildcard */*.dockerfile))
+
+help:
+	@echo "Containers to build and export tests artifacts."
+	@echo "To build one run 'make ARTIFACT/VERSION'. Available containers:"
+	@$(foreach file,$(sort $(CONTAINERS)),echo ${file};)
+	@echo "To push container builds, set the env var PUSH"
+
+%: %.dockerfile ## Builds containers
+	docker build -t $(REGISTRY)/$(@D):$(@F) -f $< $(<D)
+	@if [ ! -z $${PUSH+x} ]; then \
+		docker push $(REGISTRY)/$(@D):$(@F); \
+	fi
+
+.PHONY: all
+all: $(CONTAINERS)
diff --git a/automation/tests-artifacts/alpine/3.12-arm64v8.dockerfile b/automation/tests-artifacts/alpine/3.12-arm64v8.dockerfile
new file mode 100644
index 0000000000..9457009452
--- /dev/null
+++ b/automation/tests-artifacts/alpine/3.12-arm64v8.dockerfile
@@ -0,0 +1,67 @@
+FROM arm64v8/alpine:3.12
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+RUN \
+  # apk
+  apk update && \
+  \
+  # xen runtime deps
+  apk add musl && \
+  apk add openrc && \
+  apk add busybox && \
+  apk add sudo && \
+  apk add dbus && \
+  apk add bash && \
+  apk add python2 && \
+  apk add gettext && \
+  apk add zlib && \
+  apk add ncurses && \
+  apk add texinfo && \
+  apk add yajl && \
+  apk add libaio && \
+  apk add xz-dev && \
+  apk add util-linux && \
+  apk add argp-standalone && \
+  apk add libfdt && \
+  apk add glib && \
+  apk add pixman && \
+  apk add curl && \
+  apk add udev && \
+  \
+  # Xen
+  cd / && \
+  # Minimal ramdisk environment in case of cpio output
+  rc-update add udev && \
+  rc-update add udev-trigger && \
+  rc-update add udev-settle && \
+  rc-update add networking sysinit && \
+  rc-update add loopback sysinit && \
+  rc-update add bootmisc boot && \
+  rc-update add devfs sysinit && \
+  rc-update add dmesg sysinit && \
+  rc-update add hostname boot && \
+  rc-update add hwclock boot && \
+  rc-update add hwdrivers sysinit && \
+  rc-update add killprocs shutdown && \
+  rc-update add modloop sysinit && \
+  rc-update add modules boot && \
+  rc-update add mount-ro shutdown && \
+  rc-update add savecache shutdown && \
+  rc-update add sysctl boot && \
+  rc-update add local default && \
+  cp -a /sbin/init /init && \
+  echo "ttyS0" >> /etc/securetty && \
+  echo "hvc0" >> /etc/securetty && \
+  echo "ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100" >> /etc/inittab && \
+  echo "hvc0::respawn:/sbin/getty -L hvc0 115200 vt100" >> /etc/inittab && \
+  passwd -d "root" root && \
+  \
+  # Create rootfs
+  cd / && \
+  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var
diff --git a/automation/tests-artifacts/kernel/5.9.9-arm64v8.dockerfile b/automation/tests-artifacts/kernel/5.9.9-arm64v8.dockerfile
new file mode 100644
index 0000000000..053d65a345
--- /dev/null
+++ b/automation/tests-artifacts/kernel/5.9.9-arm64v8.dockerfile
@@ -0,0 +1,34 @@
+FROM arm64v8/debian:unstable
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV LINUX_VERSION=5.9.9
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apt-get update && \
+    apt-get --quiet --yes install \
+        build-essential \
+        libssl-dev \
+        bc \
+        curl \
+        flex \
+        bison \
+        && \
+    \
+    # Build the kernel
+    curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz && \
+    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
+    cd linux-"$LINUX_VERSION" && \
+    make defconfig && \
+    make -j$(nproc) Image.gz && \
+    cp arch/arm64/boot/Image / && \
+    cd /build && \
+    rm -rf linux-"$LINUX_VERSION"* && \
+    apt-get autoremove -y && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
-- 
2.17.1

[PATCH v3 09/12] automation: make available the tests artifacts to the pipeline

[Stefano Stabellini]

In order to make available the pre-built binaries of the
automation/tests-artifacts containers to the gitlab-ci pipeline we need
to export them as gitlab artifacts.

To do that, we create two "fake" jobs that simply export the require
binaries as artifacts and do nothing else.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
 automation/gitlab-ci/build.yaml | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index c48c0f3d66..e5246828f8 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -468,3 +468,29 @@ alpine-3.12-gcc-debug-arm64:
   extends: .gcc-arm64-build-debug
   variables:
     CONTAINER: alpine:3.12-arm64v8
+
+
+# Arm test artifacts
+
+alpine-3.12-arm64-rootfs-export:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.12-arm64v8
+  script:
+    - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
+  artifacts:
+    paths:
+      - binaries/initrd.tar.gz
+  tags:
+    - arm64
+
+kernel-5.9.9-arm64-export:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:5.9.9-arm64v8
+  script:
+    - mkdir binaries && cp /Image binaries/Image
+  artifacts:
+    paths:
+      - binaries/Image
+  tags:
+    - arm64
+
-- 
2.17.1

[PATCH v3 10/12] automation: create an alpine linux arm64 test job

[Stefano Stabellini]

Create a test job that starts Xen and Dom0 on QEMU based on the alpine
linux rootfs. Use the Linux kernel and rootfs from the tests-artifacts
containers. Add the Xen tools binaries from the Alpine Linux build job.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
 automation/gitlab-ci/test.yaml          | 24 +++++++
 automation/scripts/build                |  1 +
 automation/scripts/qemu-alpine-arm64.sh | 84 +++++++++++++++++++++++++
 3 files changed, 109 insertions(+)
 create mode 100755 automation/scripts/qemu-alpine-arm64.sh

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 35346e3f6e..a291538d68 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -22,6 +22,30 @@ build-each-commit-gcc:
     - /^coverity-tested\/.*/
     - /^stable-.*/
 
+qemu-alpine-arm64-gcc:
+  stage: test
+  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
+  variables:
+    CONTAINER: debian:unstable-arm64v8
+  script:
+    - ./automation/scripts/qemu-alpine-arm64.sh 2>&1 | tee qemu-smoke-arm64.log
+  dependencies:
+    - alpine-3.12-gcc-arm64
+    - alpine-3.12-arm64-rootfs-export
+    - kernel-5.9.9-arm64-export
+  artifacts:
+    paths:
+      - smoke.serial
+      - '*.log'
+    when: always
+  tags:
+    - arm64
+  except:
+    - master
+    - smoke
+    - /^coverity-tested\/.*/
+    - /^stable-.*/
+
 qemu-smoke-arm64-gcc:
   stage: test
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
diff --git a/automation/scripts/build b/automation/scripts/build
index 3fb2fe134c..0e1ebd60f2 100755
--- a/automation/scripts/build
+++ b/automation/scripts/build
@@ -52,6 +52,7 @@ cp xen/.config xen-config
 mkdir binaries
 if [[ "${XEN_TARGET_ARCH}" != "x86_32" ]]; then
     cp xen/xen binaries/xen
+    cp -r dist binaries/
 fi
 
 # Build all the configs we care about
diff --git a/automation/scripts/qemu-alpine-arm64.sh b/automation/scripts/qemu-alpine-arm64.sh
new file mode 100755
index 0000000000..7d1eb7d1b6
--- /dev/null
+++ b/automation/scripts/qemu-alpine-arm64.sh
@@ -0,0 +1,84 @@
+#!/bin/bash
+
+set -ex
+
+apt-get -qy update
+apt-get -qy install --no-install-recommends qemu-system-aarch64 \
+                                            u-boot-qemu \
+                                            u-boot-tools \
+                                            device-tree-compiler \
+                                            cpio \
+                                            curl
+
+mkdir -p binaries/rootfs
+cd binaries/rootfs
+tar xvzf ../initrd.tar.gz
+mkdir proc
+mkdir run
+mkdir srv
+mkdir sys
+rm var/run
+cp -ar ../dist/install/* .
+echo "#!/bin/bash
+
+export LD_LIBRARY_PATH=/usr/local/lib
+bash /etc/init.d/xencommons start
+
+xl list
+
+" > etc/local.d/xen.start
+chmod +x etc/local.d/xen.start
+echo "rc_verbose=yes" >> etc/rc.conf
+find . |cpio -H newc -o|gzip > ../xen-rootfs.cpio.gz
+cd ../..
+
+# XXX Silly workaround to get the following QEMU command to work
+cp /usr/share/qemu/pvh.bin /usr/share/qemu/efi-virtio.rom
+qemu-system-aarch64 \
+   -machine virtualization=true \
+   -cpu cortex-a57 -machine type=virt \
+   -m 1024 -display none \
+   -machine dumpdtb=binaries/virt-gicv3.dtb
+# XXX disable pl061 to avoid Linux crash
+dtc -I dtb -O dts binaries/virt-gicv3.dtb > binaries/virt-gicv3.dts
+sed 's/compatible = "arm,pl061.*/status = "disabled";/g' binaries/virt-gicv3.dts > binaries/virt-gicv3-edited.dts
+dtc -I dts -O dtb binaries/virt-gicv3-edited.dts > binaries/virt-gicv3.dtb
+
+# ImageBuilder
+echo 'MEMORY_START="0x40000000"
+MEMORY_END="0x80000000"
+
+DEVICE_TREE="virt-gicv3.dtb"
+XEN="xen"
+DOM0_KERNEL="Image"
+DOM0_RAMDISK="xen-rootfs.cpio.gz"
+XEN_CMD="console=dtuart dom0_mem=1024M"
+
+NUM_DOMUS=0
+
+LOAD_CMD="tftpb"
+UBOOT_SOURCE="boot.source"
+UBOOT_SCRIPT="boot.scr"' > binaries/config
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/config
+
+
+# Run the test
+rm -f smoke.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"| \
+timeout -k 1 480 \
+qemu-system-aarch64 \
+    -machine virtualization=true \
+    -cpu cortex-a57 -machine type=virt \
+    -m 2048 -monitor none -serial stdio \
+    -smp 2 \
+    -no-reboot \
+    -device virtio-net-pci,netdev=n0 \
+    -netdev user,id=n0,tftp=binaries \
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin |& tee smoke.serial
+
+set -e
+grep -q "Domain-0" smoke.serial || exit 1
+exit 0
-- 
2.17.1

[PATCH v3 11/12] automation: use the tests-artifacts kernel for qemu-smoke-arm64-gcc

[Stefano Stabellini]

Use the tests-artifacts kernel, instead of the Debian kernel, for the
qemu-smoke-arm64-gcc job.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
 automation/gitlab-ci/test.yaml         | 1 +
 automation/scripts/qemu-smoke-arm64.sh | 6 ------
 2 files changed, 1 insertion(+), 6 deletions(-)

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index a291538d68..9448651187 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -55,6 +55,7 @@ qemu-smoke-arm64-gcc:
     - ./automation/scripts/qemu-smoke-arm64.sh 2>&1 | tee qemu-smoke-arm64.log
   dependencies:
     - debian-unstable-gcc-arm64
+    - kernel-5.9.9-arm64-export
   artifacts:
     paths:
       - smoke.serial
diff --git a/automation/scripts/qemu-smoke-arm64.sh b/automation/scripts/qemu-smoke-arm64.sh
index 9bf4488115..d614227f0a 100755
--- a/automation/scripts/qemu-smoke-arm64.sh
+++ b/automation/scripts/qemu-smoke-arm64.sh
@@ -12,12 +12,6 @@ apt-get -qy install --no-install-recommends qemu-system-aarch64 \
                                             busybox-static \
                                             cpio
 
-cd binaries
-apt-get download linux-image-*[0-9]-arm64
-dpkg -i --ignore-depends=initramfs-tools ./linux-image-*arm64.deb || true
-cp /boot/vmlinuz-*arm64 ./Image
-cd ..
-
 # XXX Silly workaround to get the following QEMU command to work
 cp /usr/share/qemu/pvh.bin /usr/share/qemu/efi-virtio.rom
 qemu-system-aarch64 \
-- 
2.17.1

[PATCH v3 12/12] automation: add domU creation to dom0 alpine linux test

[Stefano Stabellini]

Add a trivial Busybox based domU.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
 automation/scripts/qemu-alpine-arm64.sh | 47 ++++++++++++++++++++++---
 1 file changed, 42 insertions(+), 5 deletions(-)

diff --git a/automation/scripts/qemu-alpine-arm64.sh b/automation/scripts/qemu-alpine-arm64.sh
index 7d1eb7d1b6..bd8965e6f1 100755
--- a/automation/scripts/qemu-alpine-arm64.sh
+++ b/automation/scripts/qemu-alpine-arm64.sh
@@ -8,10 +8,36 @@ apt-get -qy install --no-install-recommends qemu-system-aarch64 \
                                             u-boot-tools \
                                             device-tree-compiler \
                                             cpio \
-                                            curl
+                                            curl \
+                                            busybox-static
 
-mkdir -p binaries/rootfs
-cd binaries/rootfs
+# DomU Busybox
+cd binaries
+mkdir -p initrd
+mkdir -p initrd/bin
+mkdir -p initrd/sbin
+mkdir -p initrd/etc
+mkdir -p initrd/dev
+mkdir -p initrd/proc
+mkdir -p initrd/sys
+mkdir -p initrd/lib
+mkdir -p initrd/var
+mkdir -p initrd/mnt
+cp /bin/busybox initrd/bin/busybox
+initrd/bin/busybox --install initrd/bin
+echo "#!/bin/sh
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+mount -t devtmpfs devtmpfs /dev
+/bin/sh" > initrd/init
+chmod +x initrd/init
+cd initrd
+find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+cd ..
+
+mkdir -p rootfs
+cd rootfs
 tar xvzf ../initrd.tar.gz
 mkdir proc
 mkdir run
@@ -19,6 +45,15 @@ mkdir srv
 mkdir sys
 rm var/run
 cp -ar ../dist/install/* .
+mv ../initrd.cpio.gz ./root
+cp ../Image ./root
+echo "name=\"test\"
+memory=512
+vcpus=1
+kernel=\"/root/Image\"
+ramdisk=\"/root/initrd.cpio.gz\"
+extra=\"console=hvc0 root=/dev/ram0 rdinit=/bin/sh\"
+" > root/test.cfg
 echo "#!/bin/bash
 
 export LD_LIBRARY_PATH=/usr/local/lib
@@ -26,6 +61,8 @@ bash /etc/init.d/xencommons start
 
 xl list
 
+xl create -c /root/test.cfg
+
 " > etc/local.d/xen.start
 chmod +x etc/local.d/xen.start
 echo "rc_verbose=yes" >> etc/rc.conf
@@ -68,7 +105,7 @@ bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/conf
 rm -f smoke.serial
 set +e
 echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"| \
-timeout -k 1 480 \
+timeout -k 1 720 \
 qemu-system-aarch64 \
     -machine virtualization=true \
     -cpu cortex-a57 -machine type=virt \
@@ -80,5 +117,5 @@ qemu-system-aarch64 \
     -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin |& tee smoke.serial
 
 set -e
-grep -q "Domain-0" smoke.serial || exit 1
+(grep -q "Domain-0" smoke.serial && grep -q "BusyBox" smoke.serial) || exit 1
 exit 0
-- 
2.17.1

Re: [PATCH v3 07/12] automation: add alpine linux x86 build jobs

[Stefano Stabellini]

On Tue, 24 Nov 2020, Stefano Stabellini wrote:
> Allow failure for these jobs. Currently they fail because hvmloader
> doesn't build with musl. The failures don't block the pipeline.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> ---
> This patch is probably required: https://github.com/alpinelinux/aports/blob/master/main/xen/musl-hvmloader-fix-stdint.patch


We could simply disable the hvmloader build when it is not necessary.
When OVMF, SeaBios, and Rombios are all disabled, it doesn't make sense
to build hvmloader. If this assumption is correct, then the patch below
fixes the Alpine Linux build (as long as we pass --disable-seabios and
--disable-rombios appropriately)


diff --git a/config/Tools.mk.in b/config/Tools.mk.in
index 48bd9ab731..a6aada576f 100644
--- a/config/Tools.mk.in
+++ b/config/Tools.mk.in
@@ -55,6 +55,15 @@ CONFIG_QEMU_XEN     := @qemu_xen@
 CONFIG_QEMUU_EXTRA_ARGS:= @EXTRA_QEMUU_CONFIGURE_ARGS@
 CONFIG_LIBNL        := @libnl@
 CONFIG_GOLANG       := @golang@
+ifeq ($(CONFIG_ROMBIOS),y)
+CONFIG_FIRMWARE=y
+endif
+ifeq ($(CONFIG_SEABIOS),y)
+CONFIG_FIRMWARE=y
+endif
+ifeq ($(CONFIG_OVMF),y)
+CONFIG_FIRMWARE=y
+endif
 
 CONFIG_SYSTEMD      := @systemd@
 SYSTEMD_CFLAGS      := @SYSTEMD_CFLAGS@
diff --git a/tools/Makefile b/tools/Makefile
index ed71474421..9821a7f5d5 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -14,7 +14,7 @@ SUBDIRS-y += examples
 SUBDIRS-y += hotplug
 SUBDIRS-y += xentrace
 SUBDIRS-$(CONFIG_XCUTILS) += xcutils
-SUBDIRS-$(CONFIG_X86) += firmware
+SUBDIRS-$(CONFIG_FIRMWARE) += firmware
 SUBDIRS-y += console
 SUBDIRS-y += xenmon
 SUBDIRS-y += xentop

Re: [PATCH v3 01/12] automation: add a QEMU aarch64 smoke test

[Wei Liu]

On Tue, Nov 24, 2020 at 08:27:34PM -0800, Stefano Stabellini wrote:
> Use QEMU to start Xen (just the hypervisor) up until it stops because
> there is no dom0 kernel to boot.
> 
> It is based on the existing build job unstable-arm64v8.
> 
> Also use make -j$(nproc) to build Xen.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> ---
> Changes in v2:
> - fix x86_32 build
> ---
>  automation/gitlab-ci/test.yaml         | 22 ++++++++++++++++++
>  automation/scripts/build               |  6 ++---
>  automation/scripts/qemu-smoke-arm64.sh | 32 ++++++++++++++++++++++++++
>  3 files changed, 57 insertions(+), 3 deletions(-)
>  create mode 100755 automation/scripts/qemu-smoke-arm64.sh
> 
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index 793feafe8b..35346e3f6e 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -22,6 +22,28 @@ build-each-commit-gcc:
>      - /^coverity-tested\/.*/
>      - /^stable-.*/
>  
> +qemu-smoke-arm64-gcc:
> +  stage: test
> +  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> +  variables:
> +    CONTAINER: debian:unstable-arm64v8
> +  script:
> +    - ./automation/scripts/qemu-smoke-arm64.sh 2>&1 | tee qemu-smoke-arm64.log
> +  dependencies:
> +    - debian-unstable-gcc-arm64
> +  artifacts:
> +    paths:
> +      - smoke.serial
> +      - '*.log'
> +    when: always
> +  tags:
> +    - arm64
> +  except:
> +    - master
> +    - smoke
> +    - /^coverity-tested\/.*/
> +    - /^stable-.*/
> +
>  qemu-smoke-x86-64-gcc:
>    stage: test
>    image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> diff --git a/automation/scripts/build b/automation/scripts/build
> index 0cd0f3971d..7038e5eb50 100755
> --- a/automation/scripts/build
> +++ b/automation/scripts/build
> @@ -10,9 +10,9 @@ cc-ver()
>  
>  # random config or default config
>  if [[ "${RANDCONFIG}" == "y" ]]; then
> -    make -C xen KCONFIG_ALLCONFIG=tools/kconfig/allrandom.config randconfig
> +    make -j$(nproc) -C xen KCONFIG_ALLCONFIG=tools/kconfig/allrandom.config randconfig
>  else
> -    make -C xen defconfig
> +    make -j$(nproc) -C xen defconfig
>  fi
>  
>  # build up our configure options
> @@ -45,7 +45,7 @@ make -j$(nproc) dist
>  # Extract artifacts to avoid getting rewritten by customised builds
>  cp xen/.config xen-config
>  mkdir binaries
> -if [[ "${XEN_TARGET_ARCH}" == "x86_64" ]]; then
> +if [[ "${XEN_TARGET_ARCH}" != "x86_32" ]]; then
>      cp xen/xen binaries/xen
>  fi
>  
> diff --git a/automation/scripts/qemu-smoke-arm64.sh b/automation/scripts/qemu-smoke-arm64.sh
> new file mode 100755
> index 0000000000..a7efbf8b6f
> --- /dev/null
> +++ b/automation/scripts/qemu-smoke-arm64.sh
> @@ -0,0 +1,32 @@
> +#!/bin/bash
> +
> +set -ex
> +
> +# Install QEMU
> +export DEBIAN_FRONTENT=noninteractive
> +apt-get -qy update
> +apt-get -qy install --no-install-recommends qemu-system-aarch64 \
> +                                            u-boot-qemu
> +
> +# XXX Silly workaround to get the following QEMU command to work
> +cp /usr/share/qemu/pvh.bin /usr/share/qemu/efi-virtio.rom

Can you explain a bit more why this workaround works at all?

Not a blocking comment, but this will help other people who try to
modify this script.

Wei.

Re: [PATCH v3 00/12] automation: improvements (mostly) for arm64

[Wei Liu]

On Tue, Nov 24, 2020 at 08:27:21PM -0800, Stefano Stabellini wrote:
> Hi all,
> 
> This series does a few things:
> 
> 1) it introduces a simple Xen arm64 dom0less smoke test based on QEMU
> 2) it introduces alpine linux builds x86 and arm64
> 3) it introduces two tests artifacts containers
> 4) it uses said artifacts to create a dom0/domU arm64 test based on QEMU
> 
> The series is v3, but in reality only 1) above was sent out before (the
> first two patches). Everything else is new. All tests succeed currently.
> 

Acked-by: Wei Liu <wl@xen.org>

Re: [PATCH v3 01/12] automation: add a QEMU aarch64 smoke test

[Stefano Stabellini]

On Fri, 4 Dec 2020, Wei Liu wrote:
> On Tue, Nov 24, 2020 at 08:27:34PM -0800, Stefano Stabellini wrote:
> > Use QEMU to start Xen (just the hypervisor) up until it stops because
> > there is no dom0 kernel to boot.
> > 
> > It is based on the existing build job unstable-arm64v8.
> > 
> > Also use make -j$(nproc) to build Xen.
> > 
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> > ---
> > Changes in v2:
> > - fix x86_32 build
> > ---
> >  automation/gitlab-ci/test.yaml         | 22 ++++++++++++++++++
> >  automation/scripts/build               |  6 ++---
> >  automation/scripts/qemu-smoke-arm64.sh | 32 ++++++++++++++++++++++++++
> >  3 files changed, 57 insertions(+), 3 deletions(-)
> >  create mode 100755 automation/scripts/qemu-smoke-arm64.sh
> > 
> > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> > index 793feafe8b..35346e3f6e 100644
> > --- a/automation/gitlab-ci/test.yaml
> > +++ b/automation/gitlab-ci/test.yaml
> > @@ -22,6 +22,28 @@ build-each-commit-gcc:
> >      - /^coverity-tested\/.*/
> >      - /^stable-.*/
> >  
> > +qemu-smoke-arm64-gcc:
> > +  stage: test
> > +  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> > +  variables:
> > +    CONTAINER: debian:unstable-arm64v8
> > +  script:
> > +    - ./automation/scripts/qemu-smoke-arm64.sh 2>&1 | tee qemu-smoke-arm64.log
> > +  dependencies:
> > +    - debian-unstable-gcc-arm64
> > +  artifacts:
> > +    paths:
> > +      - smoke.serial
> > +      - '*.log'
> > +    when: always
> > +  tags:
> > +    - arm64
> > +  except:
> > +    - master
> > +    - smoke
> > +    - /^coverity-tested\/.*/
> > +    - /^stable-.*/
> > +
> >  qemu-smoke-x86-64-gcc:
> >    stage: test
> >    image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> > diff --git a/automation/scripts/build b/automation/scripts/build
> > index 0cd0f3971d..7038e5eb50 100755
> > --- a/automation/scripts/build
> > +++ b/automation/scripts/build
> > @@ -10,9 +10,9 @@ cc-ver()
> >  
> >  # random config or default config
> >  if [[ "${RANDCONFIG}" == "y" ]]; then
> > -    make -C xen KCONFIG_ALLCONFIG=tools/kconfig/allrandom.config randconfig
> > +    make -j$(nproc) -C xen KCONFIG_ALLCONFIG=tools/kconfig/allrandom.config randconfig
> >  else
> > -    make -C xen defconfig
> > +    make -j$(nproc) -C xen defconfig
> >  fi
> >  
> >  # build up our configure options
> > @@ -45,7 +45,7 @@ make -j$(nproc) dist
> >  # Extract artifacts to avoid getting rewritten by customised builds
> >  cp xen/.config xen-config
> >  mkdir binaries
> > -if [[ "${XEN_TARGET_ARCH}" == "x86_64" ]]; then
> > +if [[ "${XEN_TARGET_ARCH}" != "x86_32" ]]; then
> >      cp xen/xen binaries/xen
> >  fi
> >  
> > diff --git a/automation/scripts/qemu-smoke-arm64.sh b/automation/scripts/qemu-smoke-arm64.sh
> > new file mode 100755
> > index 0000000000..a7efbf8b6f
> > --- /dev/null
> > +++ b/automation/scripts/qemu-smoke-arm64.sh
> > @@ -0,0 +1,32 @@
> > +#!/bin/bash
> > +
> > +set -ex
> > +
> > +# Install QEMU
> > +export DEBIAN_FRONTENT=noninteractive
> > +apt-get -qy update
> > +apt-get -qy install --no-install-recommends qemu-system-aarch64 \
> > +                                            u-boot-qemu
> > +
> > +# XXX Silly workaround to get the following QEMU command to work
> > +cp /usr/share/qemu/pvh.bin /usr/share/qemu/efi-virtio.rom
> 
> Can you explain a bit more why this workaround works at all?
> 
> Not a blocking comment, but this will help other people who try to
> modify this script.

Yeah: the following QEMU command just after the copy is:

  qemu-system-aarch64 \
     -machine virtualization=true \
     -cpu cortex-a57 -machine type=virt \
     -m 512 -display none \
     -machine dumpdtb=binaries/virt-gicv3.dtb

The purpose for this command is just to generate the dtb for the
platform, see the "dumpdtb" option.

This version of QEMU refuses to do that unless it can load
"efi-virtio.rom"; although it doesn't make any sense because:
- we are not running anything here, only generating a DTB, no ROMs
  should be needed
- below when we actualy start QEMU to do emulation with the same
  options, "efi-virtio.rom" is not actually needed


I can expand a bit more on the comment, maybe:

# XXX Silly workaround to get the following QEMU command to work
# QEMU looks for "efi-virtio.rom" even if it is unneeded


Thank you for the ack on the series by the way. If you are OK with it, I
am going to wait for a couple of days in case of further comments, and
if there aren't any I'll commit the series making this change on commit.