[PATCH 0/3] automation: introduce an x86_64 Dom0/DomU test

[PATCH 0/3] automation: introduce an x86_64 Dom0/DomU test

[Stefano Stabellini]

Hi all,

This small patch series introduces a new QEMU-based test to Gitlab-CI.
It uses QEMU to emulate an x86_64 machine and run Xen, Dom0 and start a
DomU. It is very similar to the existing qemu-alpine-arm64-gcc test but
based on x86_64 rather than ARM64. I think it is important because all
the other x86_64 gitlab-ci tests we currently have are more narrow and
based on XTF. This would be the first end-to-end x86_64 test in
gitlab-ci.

To make it happen, we need an Alpine Linux rootfs and a Linux kernel.
The first two patches introduce them to gitlab-ci. Note that actually
nothing will get build during gitlab-ci runs, it has already been done
beforehand and uploaded as containers. They only import *existing*
containers and binaries into a gitlab-ci run, thus, they cannot fail.
The risk to the release of the first two patches is as close to zero as
possible.

The last patch is the one introducing a new test. This one can fail.
However, it is a new test at the end of the pipeline: it doesn't impact
the existing tests. In the worst case, the test will fail and the whole
pipeline will be marked as "failed" but looking at the jobs all the
other will continue to be marked as successful. In short, if it fails,
we can simply ignore it. Also, at the moment it is actually succeeding.

Cheers,

Stefano


Stefano Stabellini (3):
      automation: add x86_64 alpine 3.12 test-artifact
      automation: Linux 5.10.74 test-artifact
      automation: add a QEMU based x86_64 Dom0/DomU test

 automation/gitlab-ci/build.yaml                    | 24 ++++++
 automation/gitlab-ci/test.yaml                     | 24 ++++++
 automation/scripts/qemu-alpine-x86_64.sh           | 92 ++++++++++++++++++++++
 automation/tests-artifacts/alpine/3.12.dockerfile  | 68 ++++++++++++++++
 .../tests-artifacts/kernel/5.10.74.dockerfile      | 38 +++++++++
 5 files changed, 246 insertions(+)
 create mode 100644 automation/scripts/qemu-alpine-x86_64.sh
 create mode 100644 automation/tests-artifacts/alpine/3.12.dockerfile
 create mode 100644 automation/tests-artifacts/kernel/5.10.74.dockerfile

[PATCH 1/3] automation: add x86_64 alpine 3.12 test-artifact

[Stefano Stabellini]

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

It is the same as the existing ARM64 alpine 3.12 test-artifact. It is
used to export an Alpine rootfs for Dom0 used for testing.

Also add the exporting job to build.yaml so that the binaries can be
used during gitlab-ci runs.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
 automation/gitlab-ci/build.yaml               | 13 ++++
 .../tests-artifacts/alpine/3.12.dockerfile    | 68 +++++++++++++++++++
 2 files changed, 81 insertions(+)
 create mode 100644 automation/tests-artifacts/alpine/3.12.dockerfile

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index d177da1710..76b73beead 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -596,3 +596,16 @@ qemu-system-aarch64-5.2.0-arm64-export:
   tags:
     - arm64
 
+
+# x86_64 test artifacts
+
+alpine-3.12-rootfs-export:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/alpine:3.12
+  script:
+    - mkdir binaries && cp /initrd.tar.gz binaries/initrd.tar.gz
+  artifacts:
+    paths:
+      - binaries/initrd.tar.gz
+  tags:
+    - x86_64
diff --git a/automation/tests-artifacts/alpine/3.12.dockerfile b/automation/tests-artifacts/alpine/3.12.dockerfile
new file mode 100644
index 0000000000..fbc26b9e36
--- /dev/null
+++ b/automation/tests-artifacts/alpine/3.12.dockerfile
@@ -0,0 +1,68 @@
+FROM alpine:3.12
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+RUN \
+  # apk
+  apk update && \
+  \
+  # xen runtime deps
+  apk add musl && \
+  apk add openrc && \
+  apk add busybox && \
+  apk add sudo && \
+  apk add dbus && \
+  apk add bash && \
+  apk add python2 && \
+  # gettext for Xen < 4.13
+  apk add gettext && \
+  apk add zlib && \
+  apk add ncurses && \
+  apk add texinfo && \
+  apk add yajl && \
+  apk add libaio && \
+  apk add xz-dev && \
+  apk add util-linux && \
+  apk add argp-standalone && \
+  apk add libfdt && \
+  apk add glib && \
+  apk add pixman && \
+  apk add curl && \
+  apk add udev && \
+  \
+  # Xen
+  cd / && \
+  # Minimal ramdisk environment in case of cpio output
+  rc-update add udev && \
+  rc-update add udev-trigger && \
+  rc-update add udev-settle && \
+  rc-update add networking sysinit && \
+  rc-update add loopback sysinit && \
+  rc-update add bootmisc boot && \
+  rc-update add devfs sysinit && \
+  rc-update add dmesg sysinit && \
+  rc-update add hostname boot && \
+  rc-update add hwclock boot && \
+  rc-update add hwdrivers sysinit && \
+  rc-update add killprocs shutdown && \
+  rc-update add modloop sysinit && \
+  rc-update add modules boot && \
+  rc-update add mount-ro shutdown && \
+  rc-update add savecache shutdown && \
+  rc-update add sysctl boot && \
+  rc-update add local default && \
+  cp -a /sbin/init /init && \
+  echo "ttyS0" >> /etc/securetty && \
+  echo "hvc0" >> /etc/securetty && \
+  echo "ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100" >> /etc/inittab && \
+  echo "hvc0::respawn:/sbin/getty -L hvc0 115200 vt100" >> /etc/inittab && \
+  passwd -d "root" root && \
+  \
+  # Create rootfs
+  cd / && \
+  tar cvzf /initrd.tar.gz bin dev etc home init lib mnt opt root sbin usr var
-- 
2.17.1

Re: [PATCH 1/3] automation: add x86_64 alpine 3.12 test-artifact

[Anthony PERARD]

On Thu, Oct 21, 2021 at 04:08:37PM -0700, Stefano Stabellini wrote:
> From: Stefano Stabellini <stefano.stabellini@xilinx.com>
> 
> It is the same as the existing ARM64 alpine 3.12 test-artifact. It is
> used to export an Alpine rootfs for Dom0 used for testing.
> 
> Also add the exporting job to build.yaml so that the binaries can be
> used during gitlab-ci runs.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
> ---

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

This feels weird to me that we have a container just to share a file,
but I guess that works :-).

Thanks,

-- 
Anthony PERARD

Re: [PATCH 1/3] automation: add x86_64 alpine 3.12 test-artifact

[Andrew Cooper]

On 22/10/2021 00:08, Stefano Stabellini wrote:
> +RUN \
> +  # apk
> +  apk update && \
> +  \
> +  # xen runtime deps
> +  apk add musl && \
> +  apk add openrc && \
> +  apk add busybox && \
> +  apk add sudo && \
> +  apk add dbus && \
> +  apk add bash && \
> +  apk add python2 && \
> +  # gettext for Xen < 4.13
> +  apk add gettext && \

Unless we're planning on backporting this to staging-4.13, we can drop
gettext.  (That's the whole point of the comments highlighting version)

~Andrew

Re: [PATCH 1/3] automation: add x86_64 alpine 3.12 test-artifact

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 600 bytes --]

On Fri, 22 Oct 2021, Andrew Cooper wrote:
> On 22/10/2021 00:08, Stefano Stabellini wrote:
> > +RUN \
> > +  # apk
> > +  apk update && \
> > +  \
> > +  # xen runtime deps
> > +  apk add musl && \
> > +  apk add openrc && \
> > +  apk add busybox && \
> > +  apk add sudo && \
> > +  apk add dbus && \
> > +  apk add bash && \
> > +  apk add python2 && \
> > +  # gettext for Xen < 4.13
> > +  apk add gettext && \
> 
> Unless we're planning on backporting this to staging-4.13, we can drop
> gettext.  (That's the whole point of the comments highlighting version)

Good point, I'll make the change

[PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Stefano Stabellini]

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

Build a 5.10 kernel to be used as Dom0 and DomU kernel for testing. This
is almost the same as the existing ARM64 recipe for Linux 5.9, the
only differences are:
- upgrade to latest 5.10.x stable
- force Xen modules to built-in (on ARM it was already done by defconfig)

Also add the exporting job to build.yaml so that the binary can be used
during gitlab-ci runs.

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
 automation/gitlab-ci/build.yaml               | 11 ++++++
 .../tests-artifacts/kernel/5.10.74.dockerfile | 38 +++++++++++++++++++
 2 files changed, 49 insertions(+)
 create mode 100644 automation/tests-artifacts/kernel/5.10.74.dockerfile

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 76b73beead..0034c50950 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -609,3 +609,14 @@ alpine-3.12-rootfs-export:
       - binaries/initrd.tar.gz
   tags:
     - x86_64
+
+kernel-5.10.74-export:
+  stage: build
+  image: registry.gitlab.com/xen-project/xen/tests-artifacts/kernel:5.10.74
+  script:
+    - mkdir binaries && cp /bzImage binaries/bzImage
+  artifacts:
+    paths:
+      - binaries/bzImage
+  tags:
+    - x86_64
diff --git a/automation/tests-artifacts/kernel/5.10.74.dockerfile b/automation/tests-artifacts/kernel/5.10.74.dockerfile
new file mode 100644
index 0000000000..f2dbbecf74
--- /dev/null
+++ b/automation/tests-artifacts/kernel/5.10.74.dockerfile
@@ -0,0 +1,38 @@
+FROM debian:unstable
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV LINUX_VERSION=5.10.74
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+# build depends
+RUN apt-get update && \
+    apt-get --quiet --yes install \
+        build-essential \
+        libssl-dev \
+        bc \
+        curl \
+        flex \
+        bison \
+        libelf-dev \
+        && \
+    \
+    # Build the kernel
+    curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz && \
+    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
+    cd linux-"$LINUX_VERSION" && \
+    make defconfig && \
+    make xen.config && \
+    cp .config .config.orig && \
+    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
+    make -j$(nproc) bzImage && \
+    cp arch/x86/boot/bzImage / && \
+    cd /build && \
+    rm -rf linux-"$LINUX_VERSION"* && \
+    apt-get autoremove -y && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
-- 
2.17.1

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Anthony PERARD]

On Thu, Oct 21, 2021 at 04:08:38PM -0700, Stefano Stabellini wrote:
> diff --git a/automation/tests-artifacts/kernel/5.10.74.dockerfile b/automation/tests-artifacts/kernel/5.10.74.dockerfile
> new file mode 100644
> index 0000000000..f2dbbecf74
> --- /dev/null
> +++ b/automation/tests-artifacts/kernel/5.10.74.dockerfile
> @@ -0,0 +1,38 @@
> +FROM debian:unstable
> +LABEL maintainer.name="The Xen Project" \
> +      maintainer.email="xen-devel@lists.xenproject.org"
> +
> +ENV DEBIAN_FRONTEND=noninteractive
> +ENV LINUX_VERSION=5.10.74
> +ENV USER root
> +
> +RUN mkdir /build
> +WORKDIR /build
> +
> +# build depends
> +RUN apt-get update && \
> +    apt-get --quiet --yes install \
> +        build-essential \
> +        libssl-dev \
> +        bc \
> +        curl \
> +        flex \
> +        bison \
> +        libelf-dev \
> +        && \
> +    \
> +    # Build the kernel

You could start a new docker "RUN" step to build the kernel.

> +    curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz && \
> +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
> +    cd linux-"$LINUX_VERSION" && \
> +    make defconfig && \
> +    make xen.config && \
> +    cp .config .config.orig && \
> +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \

This command line could be replaced by:
    sed -i '/XEN/s/=m$/=y/' .config
But I guess Kconfig don't mind duplicated symbols.

> +    make -j$(nproc) bzImage && \
> +    cp arch/x86/boot/bzImage / && \
> +    cd /build && \
> +    rm -rf linux-"$LINUX_VERSION"* && \


> +    apt-get autoremove -y && \
> +    apt-get clean && \
> +    rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*

This apt cleaning stuff should be done right after `apt install`, and
thus before starting to build the kernel, I think.

Any any case:
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Stefano Stabellini]

On Fri, 22 Oct 2021, Anthony PERARD wrote:
> On Thu, Oct 21, 2021 at 04:08:38PM -0700, Stefano Stabellini wrote:
> > diff --git a/automation/tests-artifacts/kernel/5.10.74.dockerfile b/automation/tests-artifacts/kernel/5.10.74.dockerfile
> > new file mode 100644
> > index 0000000000..f2dbbecf74
> > --- /dev/null
> > +++ b/automation/tests-artifacts/kernel/5.10.74.dockerfile
> > @@ -0,0 +1,38 @@
> > +FROM debian:unstable
> > +LABEL maintainer.name="The Xen Project" \
> > +      maintainer.email="xen-devel@lists.xenproject.org"
> > +
> > +ENV DEBIAN_FRONTEND=noninteractive
> > +ENV LINUX_VERSION=5.10.74
> > +ENV USER root
> > +
> > +RUN mkdir /build
> > +WORKDIR /build
> > +
> > +# build depends
> > +RUN apt-get update && \
> > +    apt-get --quiet --yes install \
> > +        build-essential \
> > +        libssl-dev \
> > +        bc \
> > +        curl \
> > +        flex \
> > +        bison \
> > +        libelf-dev \
> > +        && \
> > +    \
> > +    # Build the kernel
> 
> You could start a new docker "RUN" step to build the kernel.

OK


> > +    curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz && \
> > +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
> > +    cd linux-"$LINUX_VERSION" && \
> > +    make defconfig && \
> > +    make xen.config && \
> > +    cp .config .config.orig && \
> > +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
> 
> This command line could be replaced by:
>     sed -i '/XEN/s/=m$/=y/' .config
> But I guess Kconfig don't mind duplicated symbols.

Yeah the latter overwrites the former


> > +    make -j$(nproc) bzImage && \
> > +    cp arch/x86/boot/bzImage / && \
> > +    cd /build && \
> > +    rm -rf linux-"$LINUX_VERSION"* && \
> 
> 
> > +    apt-get autoremove -y && \
> > +    apt-get clean && \
> > +    rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
> 
> This apt cleaning stuff should be done right after `apt install`, and
> thus before starting to build the kernel, I think.

OK

> Any any case:
> Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks Anthony!

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Andrew Cooper]

On 22/10/2021 00:08, Stefano Stabellini wrote:
> +# build depends
> +RUN apt-get update && \
> +    apt-get --quiet --yes install \
> +        build-essential \
> +        libssl-dev \
> +        bc \
> +        curl \
> +        flex \
> +        bison \
> +        libelf-dev \
> +        && \
> +    \
> +    # Build the kernel
> +    curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz && \
> +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
> +    cd linux-"$LINUX_VERSION" && \
> +    make defconfig && \
> +    make xen.config && \
> +    cp .config .config.orig && \
> +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
> +    make -j$(nproc) bzImage && \

defconfig is huuuuge.  Can we use tinyconfig instead?

Also, you want to turn off CONFIG_MODULES seeing as you only copy
bzImage to the test.

This also fixes the fact that `grep XEN` also matches CONFIG_NETXEN_NIC
which has nothing to do with Xen.

~Andrew

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 10274 bytes --]

+Juergen

On Fri, 22 Oct 2021, Andrew Cooper wrote:
> On 22/10/2021 00:08, Stefano Stabellini wrote:
> > +# build depends
> > +RUN apt-get update && \
> > +    apt-get --quiet --yes install \
> > +        build-essential \
> > +        libssl-dev \
> > +        bc \
> > +        curl \
> > +        flex \
> > +        bison \
> > +        libelf-dev \
> > +        && \
> > +    \
> > +    # Build the kernel
> > +    curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz && \
> > +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
> > +    cd linux-"$LINUX_VERSION" && \
> > +    make defconfig && \
> > +    make xen.config && \
> > +    cp .config .config.orig && \
> > +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
> > +    make -j$(nproc) bzImage && \
> 
> defconfig is huuuuge.  Can we use tinyconfig instead?
> 
> Also, you want to turn off CONFIG_MODULES seeing as you only copy
> bzImage to the test.
> 
> This also fixes the fact that `grep XEN` also matches CONFIG_NETXEN_NIC
> which has nothing to do with Xen.

Tinyconfig would be a good idea but it crashes on QEMU with the appended
boot log. I did:

# make tinyconfig
# make xen.config
# make bzImage

Of course we could have our own tailored special kernel config for this
kernel but I like the simplicify of "make tinyconfig" or "make
defconfig" followed by "make xen.config".

Unless you know one kconfig option or two that we need to add to
tinyconfig to make it work my preference is to go with defconfig for
now.



(XEN) Xen version 4.16-unstable (sstabellini@) (gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0) debug=y Tue Oct 19 19:29:09 PDT 2021
(XEN) Latest ChangeSet: Tue Oct 19 18:19:21 2021 -0700 git:9d0c6610bf-dirty
(XEN) build-id: 204da5ded1385bd7f645957893dba95d08ffc6e2
(XEN) Bootloader: iPXE 1.20.1+ (g4bd0)
(XEN) Command line: console=com1
(XEN) Xen image load base address: 0
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V1 V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 0 EDD information structures
(XEN) CPU Vendor: AMD, Family 15 (0xf), Model 107 (0x6b), Stepping 1 (raw 00060fb1)
(XEN) Xen-e820 RAM map:
(XEN)  [0000000000000000, 000000000009fbff] (usable)
(XEN)  [000000000009fc00, 000000000009ffff] (reserved)
(XEN)  [00000000000f0000, 00000000000fffff] (reserved)
(XEN)  [0000000000100000, 000000007ffdffff] (usable)
(XEN)  [000000007ffe0000, 000000007fffffff] (reserved)
(XEN)  [00000000fffc0000, 00000000ffffffff] (reserved)
(XEN) New Xen image base address: 0x7f800000
(XEN) System RAM: 2047MB (2096636kB)
(XEN) ACPI: RSDP 000F58D0, 0014 (r0 BOCHS )
(XEN) ACPI: RSDT 7FFE1960, 0034 (r1 BOCHS  BXPC            1 BXPC        1)
(XEN) ACPI: FACP 7FFE180C, 0074 (r1 BOCHS  BXPC            1 BXPC        1)
(XEN) ACPI: DSDT 7FFE0040, 17CC (r1 BOCHS  BXPC            1 BXPC        1)
(XEN) ACPI: FACS 7FFE0000, 0040
(XEN) ACPI: APIC 7FFE1880, 0080 (r1 BOCHS  BXPC            1 BXPC        1)
(XEN) ACPI: HPET 7FFE1900, 0038 (r1 BOCHS  BXPC            1 BXPC        1)
(XEN) ACPI: WAET 7FFE1938, 0028 (r1 BOCHS  BXPC            1 BXPC        1)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000007ffe0000
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f5ab0
(XEN) DMI 2.8 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x608 (24 bits)
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:604,1:0], pm1x_evt[1:600,1:0]
(XEN) ACPI:             wakeup_vec[7ffe000c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ5 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) ACPI: IRQ10 used by override.
(XEN) ACPI: IRQ11 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 2 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 392 MSI/MSI-X
(XEN) Microcode loading not available
(XEN) No SSBD controls available
(XEN) CPU0: AMD K8 machine check reporting enabled
(XEN) Speculative mitigation facilities:
(XEN)   Hardware hints:
(XEN)   Hardware features:
(XEN)   Compiled-in support: INDIRECT_THUNK SHADOW_PAGING
(XEN)   Xen settings: BTI-Thunk LFENCE, SPEC_CTRL: No, Other: BRANCH_HARDEN
(XEN)   Support for HVM VMs: RSB
(XEN)   Support for PV VMs: RSB
(XEN)   XPTI (64-bit PV only): Dom0 disabled, DomU disabled (without PCID)
(XEN)   PV L1TF shadowing: Dom0 disabled, DomU disabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Platform timer is 100.000MHz HPET
(XEN) Detected 1993.167 MHz processor.
(XEN) Freed 1024kB unused BSS memory
(XEN) alt table ffff82d04048f570 -> ffff82d04049cf34
(XEN) I/O virtualisation disabled
(XEN) nr_sockets: 1
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC only partially writable
(XEN) Allocated console ring of 16 KiB.
(XEN) mwait-idle: does not run on family 15 model 107
(XEN) HVM: ASIDs enabled.
(XEN) SVM: Supported advanced features:
(XEN)  - none
(XEN) HVM: SVM enabled
(XEN) HVM: Hardware Assisted Paging (HAP) not detected
(XEN) alt table ffff82d04048f570 -> ffff82d04049cf34
(XEN) Brought up 2 CPUs
(XEN) Scheduling granularity: cpu, 1 CPU per sched-resource
(XEN) Adding cpu 0 to runqueue 0
(XEN)  First cpu on runqueue, activating
(XEN) Adding cpu 1 to runqueue 0
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) mtrr: your CPUs had inconsistent fixed MTRR settings
(XEN) mtrr: your CPUs had inconsistent variable MTRR settings
(XEN) mtrr: your CPUs had inconsistent MTRRdefType settings
(XEN) mtrr: probably your BIOS does not setup all CPUs.
(XEN) mtrr: corrected configuration.
(XEN) MTRR default type: write-back
(XEN) MTRR fixed ranges enabled:
(XEN)   00000-9ffff write-back
(XEN)   a0000-bffff uncachable
(XEN)   c0000-fffff write-protect
(XEN) MTRR variable ranges enabled:
(XEN)   0 base 0080000000 mask ff80000000 uncachable
(XEN)   1 disabled
(XEN)   2 disabled
(XEN)   3 disabled
(XEN)   4 disabled
(XEN)   5 disabled
(XEN)   6 disabled
(XEN)   7 disabled
(XEN) Running stub recovery selftests...
(XEN) Fixup #UD[0000]: ffff82d07fffe040 [ffff82d07fffe040] -> ffff82d04038e83a
(XEN) Selftest 1 failed: Opc 90 02 00 c3 expected 13[0000], got 14[0001]
(XEN) Selftest 2 failed: Opc 02 04 04 c3 expected 12[0000], got 14[0001]
(XEN) Fixup #BP[0000]: ffff82d07fffe041 [ffff82d07fffe041] -> ffff82d04038e83a
(XEN) NX (Execute Disable) protection active
(XEN) Dom0 has maximum 416 PIRQs
(XEN) *** Building a PV Dom0 ***
(XEN) ELF: phdr: paddr=0x1000000 memsz=0x8532f0
(XEN) ELF: phdr: paddr=0x1a00000 memsz=0x75000
(XEN) ELF: phdr: paddr=0x1a75000 memsz=0x1a1000
(XEN) ELF: memory: 0x1000000 -> 0x1c16000
(XEN) ELF: note: GUEST_OS = "linux"
(XEN) ELF: note: GUEST_VERSION = "2.6"
(XEN) ELF: note: XEN_VERSION = "xen-3.0"
(XEN) ELF: note: VIRT_BASE = 0xffffffff80000000
(XEN) ELF: note: INIT_P2M = 0x8000000000
(XEN) ELF: note: ENTRY = 0xffffffff81a75160
(XEN) ELF: note: HYPERCALL_PAGE = 0xffffffff81001000
(XEN) ELF: note: FEATURES = "!writable_page_tables|pae_pgdir_above_4gb"
(XEN) ELF: note: SUPPORTED_FEATURES = 0x8801
(XEN) ELF: note: PAE_MODE = "yes"
(XEN) ELF: note: LOADER = "generic"
(XEN) ELF: note: unknown (0xd)
(XEN) ELF: note: SUSPEND_CANCEL = 0x1
(XEN) ELF: note: MOD_START_PFN = 0x1
(XEN) ELF: note: HV_START_LOW = 0xffff800000000000
(XEN) ELF: note: PADDR_OFFSET = 0
(XEN) ELF: addresses:
(XEN)     virt_base        = 0xffffffff80000000
(XEN)     elf_paddr_offset = 0x0
(XEN)     virt_offset      = 0xffffffff80000000
(XEN)     virt_kstart      = 0xffffffff81000000
(XEN)     virt_kend        = 0xffffffff81c16000
(XEN)     virt_entry       = 0xffffffff81a75160
(XEN)     p2m_base         = 0x8000000000
(XEN)  Xen  kernel: 64-bit, lsb
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1c16000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   000000007a000000->000000007c000000 (476919 pages to be allocated)
(XEN)  Init. ramdisk: 000000007f806000->000000007f9ffe77
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff81c16000
(XEN)  Phys-Mach map: 0000008000000000->00000080003b4788
(XEN)  Start info:    ffffffff81c16000->ffffffff81c164b8
(XEN)  Page tables:   ffffffff81c17000->ffffffff81c2a000
(XEN)  Boot stack:    ffffffff81c2a000->ffffffff81c2b000
(XEN)  TOTAL:         ffffffff80000000->ffffffff82000000
(XEN)  ENTRY ADDRESS: ffffffff81a75160
(XEN) Dom0 has maximum 2 VCPUs
(XEN) ELF: phdr 0 at 0xffffffff81000000 -> 0xffffffff818532f0
(XEN) ELF: phdr 1 at 0xffffffff81a00000 -> 0xffffffff81a75000
(XEN) ELF: phdr 2 at 0xffffffff81a75000 -> 0xffffffff81afcc4d
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM in background
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) ***************************************************
(XEN) SELFTEST FAILURE: CORRECT BEHAVIOR CANNOT BE GUARANTEED
(XEN) ***************************************************
(XEN) 3... 2... 1... 
(XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
(XEN) Freed 592kB init memory
mapping kernel into physical memory
about to get started...
(XEN) emul-priv-op.c:1018:d0v0 RDMSR 0xc0010055 unimplemented
(XEN) Hardware Dom0 crashed: rebooting machine in 5 seconds.
qemu-system-x86_64: terminating on signal 2

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Juergen Gross]

[-- Attachment #1.1.1: Type: text/plain, Size: 1878 bytes --]

On 22.10.21 21:41, Stefano Stabellini wrote:
> +Juergen
> 
> On Fri, 22 Oct 2021, Andrew Cooper wrote:
>> On 22/10/2021 00:08, Stefano Stabellini wrote:
>>> +# build depends
>>> +RUN apt-get update && \
>>> +    apt-get --quiet --yes install \
>>> +        build-essential \
>>> +        libssl-dev \
>>> +        bc \
>>> +        curl \
>>> +        flex \
>>> +        bison \
>>> +        libelf-dev \
>>> +        && \
>>> +    \
>>> +    # Build the kernel
>>> +    curl -fsSLO https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz && \
>>> +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
>>> +    cd linux-"$LINUX_VERSION" && \
>>> +    make defconfig && \
>>> +    make xen.config && \
>>> +    cp .config .config.orig && \
>>> +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config && \
>>> +    make -j$(nproc) bzImage && \
>>
>> defconfig is huuuuge.  Can we use tinyconfig instead?
>>
>> Also, you want to turn off CONFIG_MODULES seeing as you only copy
>> bzImage to the test.
>>
>> This also fixes the fact that `grep XEN` also matches CONFIG_NETXEN_NIC
>> which has nothing to do with Xen.
> 
> Tinyconfig would be a good idea but it crashes on QEMU with the appended
> boot log. I did:
> 
> # make tinyconfig
> # make xen.config
> # make bzImage
> 
> Of course we could have our own tailored special kernel config for this
> kernel but I like the simplicify of "make tinyconfig" or "make
> defconfig" followed by "make xen.config".
> 
> Unless you know one kconfig option or two that we need to add to
> tinyconfig to make it work my preference is to go with defconfig for
> now.

You will need to add the dom0 and SMP related config options at least.

I'll have a look at config.xen, which is a little bit weird: it
configures the backends, but not dom0.


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 2207 bytes --]

On Mon, 25 Oct 2021, Juergen Gross wrote:
> On 22.10.21 21:41, Stefano Stabellini wrote:
> > +Juergen
> > 
> > On Fri, 22 Oct 2021, Andrew Cooper wrote:
> > > On 22/10/2021 00:08, Stefano Stabellini wrote:
> > > > +# build depends
> > > > +RUN apt-get update && \
> > > > +    apt-get --quiet --yes install \
> > > > +        build-essential \
> > > > +        libssl-dev \
> > > > +        bc \
> > > > +        curl \
> > > > +        flex \
> > > > +        bison \
> > > > +        libelf-dev \
> > > > +        && \
> > > > +    \
> > > > +    # Build the kernel
> > > > +    curl -fsSLO
> > > > https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz
> > > > && \
> > > > +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
> > > > +    cd linux-"$LINUX_VERSION" && \
> > > > +    make defconfig && \
> > > > +    make xen.config && \
> > > > +    cp .config .config.orig && \
> > > > +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config
> > > > && \
> > > > +    make -j$(nproc) bzImage && \
> > > 
> > > defconfig is huuuuge.  Can we use tinyconfig instead?
> > > 
> > > Also, you want to turn off CONFIG_MODULES seeing as you only copy
> > > bzImage to the test.
> > > 
> > > This also fixes the fact that `grep XEN` also matches CONFIG_NETXEN_NIC
> > > which has nothing to do with Xen.
> > 
> > Tinyconfig would be a good idea but it crashes on QEMU with the appended
> > boot log. I did:
> > 
> > # make tinyconfig
> > # make xen.config
> > # make bzImage
> > 
> > Of course we could have our own tailored special kernel config for this
> > kernel but I like the simplicify of "make tinyconfig" or "make
> > defconfig" followed by "make xen.config".
> > 
> > Unless you know one kconfig option or two that we need to add to
> > tinyconfig to make it work my preference is to go with defconfig for
> > now.
> 
> You will need to add the dom0 and SMP related config options at least.
> 
> I'll have a look at config.xen, which is a little bit weird: it
> configures the backends, but not dom0.

Thanks, it would be good if make tinyconfig && make xen.config would
just work on QEMU. In the meantime I'll keep defconfig for this test, we
can always improve later.

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Juergen Gross]

[-- Attachment #1.1.1: Type: text/plain, Size: 2321 bytes --]

On 26.10.21 02:54, Stefano Stabellini wrote:
> On Mon, 25 Oct 2021, Juergen Gross wrote:
>> On 22.10.21 21:41, Stefano Stabellini wrote:
>>> +Juergen
>>>
>>> On Fri, 22 Oct 2021, Andrew Cooper wrote:
>>>> On 22/10/2021 00:08, Stefano Stabellini wrote:
>>>>> +# build depends
>>>>> +RUN apt-get update && \
>>>>> +    apt-get --quiet --yes install \
>>>>> +        build-essential \
>>>>> +        libssl-dev \
>>>>> +        bc \
>>>>> +        curl \
>>>>> +        flex \
>>>>> +        bison \
>>>>> +        libelf-dev \
>>>>> +        && \
>>>>> +    \
>>>>> +    # Build the kernel
>>>>> +    curl -fsSLO
>>>>> https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz
>>>>> && \
>>>>> +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
>>>>> +    cd linux-"$LINUX_VERSION" && \
>>>>> +    make defconfig && \
>>>>> +    make xen.config && \
>>>>> +    cp .config .config.orig && \
>>>>> +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >> .config
>>>>> && \
>>>>> +    make -j$(nproc) bzImage && \
>>>>
>>>> defconfig is huuuuge.  Can we use tinyconfig instead?
>>>>
>>>> Also, you want to turn off CONFIG_MODULES seeing as you only copy
>>>> bzImage to the test.
>>>>
>>>> This also fixes the fact that `grep XEN` also matches CONFIG_NETXEN_NIC
>>>> which has nothing to do with Xen.
>>>
>>> Tinyconfig would be a good idea but it crashes on QEMU with the appended
>>> boot log. I did:
>>>
>>> # make tinyconfig
>>> # make xen.config
>>> # make bzImage
>>>
>>> Of course we could have our own tailored special kernel config for this
>>> kernel but I like the simplicify of "make tinyconfig" or "make
>>> defconfig" followed by "make xen.config".
>>>
>>> Unless you know one kconfig option or two that we need to add to
>>> tinyconfig to make it work my preference is to go with defconfig for
>>> now.
>>
>> You will need to add the dom0 and SMP related config options at least.
>>
>> I'll have a look at config.xen, which is a little bit weird: it
>> configures the backends, but not dom0.
> 
> Thanks, it would be good if make tinyconfig && make xen.config would
> just work on QEMU. In the meantime I'll keep defconfig for this test, we
> can always improve later.

Can you please test the attached patch?


Juergen

[-- Attachment #1.1.2: 0001-x86-xen-update-xen.config.patch --]
[-- Type: text/x-patch, Size: 1306 bytes --]

From 8ffe2d58a3507dfd42996a38f0de66c1e7ca2c6a Mon Sep 17 00:00:00 2001
From: Juergen Gross <jgross@suse.com>
Date: Wed, 27 Oct 2021 09:05:28 +0200
Subject: [PATCH] x86/xen: update xen.config

Update some entries in xen.config:

- add ACPI as being required for running as dom0
- add SMP for convenience
- remove HIGHMEM64G as this option no longer exists
- remove XEN_MAX_DOMAIN_MEMORY as this option no longer exists

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/configs/xen.config | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/arch/x86/configs/xen.config b/arch/x86/configs/xen.config
index d9fc7139fd46..e59fdb0f6b67 100644
--- a/arch/x86/configs/xen.config
+++ b/arch/x86/configs/xen.config
@@ -1,6 +1,4 @@
 # global x86 required specific stuff
-# On 32-bit HIGHMEM4G is not allowed
-CONFIG_HIGHMEM64G=y
 CONFIG_64BIT=y
 
 # These enable us to allow some of the
@@ -9,12 +7,13 @@ CONFIG_HYPERVISOR_GUEST=y
 CONFIG_PCI=y
 CONFIG_PCI_MSI=y
 CONFIG_X86_MCE=y
+CONFIG_ACPI=y
 CONFIG_ACPI_PROCESSOR=y
 CONFIG_CPU_FREQ=y
+CONFIG_SMP=y
 
 # x86 xen specific config options
 CONFIG_XEN_PVH=y
-CONFIG_XEN_MAX_DOMAIN_MEMORY=500
 CONFIG_XEN_SAVE_RESTORE=y
 # CONFIG_XEN_DEBUG_FS is not set
 CONFIG_XEN_MCE_LOG=y
-- 
2.26.2


[-- Attachment #1.1.3: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 2786 bytes --]

On Wed, 27 Oct 2021, Juergen Gross wrote:
> On 26.10.21 02:54, Stefano Stabellini wrote:
> > On Mon, 25 Oct 2021, Juergen Gross wrote:
> > > On 22.10.21 21:41, Stefano Stabellini wrote:
> > > > +Juergen
> > > > 
> > > > On Fri, 22 Oct 2021, Andrew Cooper wrote:
> > > > > On 22/10/2021 00:08, Stefano Stabellini wrote:
> > > > > > +# build depends
> > > > > > +RUN apt-get update && \
> > > > > > +    apt-get --quiet --yes install \
> > > > > > +        build-essential \
> > > > > > +        libssl-dev \
> > > > > > +        bc \
> > > > > > +        curl \
> > > > > > +        flex \
> > > > > > +        bison \
> > > > > > +        libelf-dev \
> > > > > > +        && \
> > > > > > +    \
> > > > > > +    # Build the kernel
> > > > > > +    curl -fsSLO
> > > > > > https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz
> > > > > > && \
> > > > > > +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
> > > > > > +    cd linux-"$LINUX_VERSION" && \
> > > > > > +    make defconfig && \
> > > > > > +    make xen.config && \
> > > > > > +    cp .config .config.orig && \
> > > > > > +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >>
> > > > > > .config
> > > > > > && \
> > > > > > +    make -j$(nproc) bzImage && \
> > > > > 
> > > > > defconfig is huuuuge.  Can we use tinyconfig instead?
> > > > > 
> > > > > Also, you want to turn off CONFIG_MODULES seeing as you only copy
> > > > > bzImage to the test.
> > > > > 
> > > > > This also fixes the fact that `grep XEN` also matches
> > > > > CONFIG_NETXEN_NIC
> > > > > which has nothing to do with Xen.
> > > > 
> > > > Tinyconfig would be a good idea but it crashes on QEMU with the appended
> > > > boot log. I did:
> > > > 
> > > > # make tinyconfig
> > > > # make xen.config
> > > > # make bzImage
> > > > 
> > > > Of course we could have our own tailored special kernel config for this
> > > > kernel but I like the simplicify of "make tinyconfig" or "make
> > > > defconfig" followed by "make xen.config".
> > > > 
> > > > Unless you know one kconfig option or two that we need to add to
> > > > tinyconfig to make it work my preference is to go with defconfig for
> > > > now.
> > > 
> > > You will need to add the dom0 and SMP related config options at least.
> > > 
> > > I'll have a look at config.xen, which is a little bit weird: it
> > > configures the backends, but not dom0.
> > 
> > Thanks, it would be good if make tinyconfig && make xen.config would
> > just work on QEMU. In the meantime I'll keep defconfig for this test, we
> > can always improve later.
> 
> Can you please test the attached patch?

Yes, that works, thanks!

We don't currently have any patches under automation/ but I guess I
could add this patch under automation/tests-artifacts/kernel/

Cheers,

Stefano

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 3839 bytes --]

On Wed, 27 Oct 2021, Stefano Stabellini wrote:
> On Wed, 27 Oct 2021, Juergen Gross wrote:
> > On 26.10.21 02:54, Stefano Stabellini wrote:
> > > On Mon, 25 Oct 2021, Juergen Gross wrote:
> > > > On 22.10.21 21:41, Stefano Stabellini wrote:
> > > > > +Juergen
> > > > > 
> > > > > On Fri, 22 Oct 2021, Andrew Cooper wrote:
> > > > > > On 22/10/2021 00:08, Stefano Stabellini wrote:
> > > > > > > +# build depends
> > > > > > > +RUN apt-get update && \
> > > > > > > +    apt-get --quiet --yes install \
> > > > > > > +        build-essential \
> > > > > > > +        libssl-dev \
> > > > > > > +        bc \
> > > > > > > +        curl \
> > > > > > > +        flex \
> > > > > > > +        bison \
> > > > > > > +        libelf-dev \
> > > > > > > +        && \
> > > > > > > +    \
> > > > > > > +    # Build the kernel
> > > > > > > +    curl -fsSLO
> > > > > > > https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz
> > > > > > > && \
> > > > > > > +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
> > > > > > > +    cd linux-"$LINUX_VERSION" && \
> > > > > > > +    make defconfig && \
> > > > > > > +    make xen.config && \
> > > > > > > +    cp .config .config.orig && \
> > > > > > > +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >>
> > > > > > > .config
> > > > > > > && \
> > > > > > > +    make -j$(nproc) bzImage && \
> > > > > > 
> > > > > > defconfig is huuuuge.  Can we use tinyconfig instead?
> > > > > > 
> > > > > > Also, you want to turn off CONFIG_MODULES seeing as you only copy
> > > > > > bzImage to the test.
> > > > > > 
> > > > > > This also fixes the fact that `grep XEN` also matches
> > > > > > CONFIG_NETXEN_NIC
> > > > > > which has nothing to do with Xen.
> > > > > 
> > > > > Tinyconfig would be a good idea but it crashes on QEMU with the appended
> > > > > boot log. I did:
> > > > > 
> > > > > # make tinyconfig
> > > > > # make xen.config
> > > > > # make bzImage
> > > > > 
> > > > > Of course we could have our own tailored special kernel config for this
> > > > > kernel but I like the simplicify of "make tinyconfig" or "make
> > > > > defconfig" followed by "make xen.config".
> > > > > 
> > > > > Unless you know one kconfig option or two that we need to add to
> > > > > tinyconfig to make it work my preference is to go with defconfig for
> > > > > now.
> > > > 
> > > > You will need to add the dom0 and SMP related config options at least.
> > > > 
> > > > I'll have a look at config.xen, which is a little bit weird: it
> > > > configures the backends, but not dom0.
> > > 
> > > Thanks, it would be good if make tinyconfig && make xen.config would
> > > just work on QEMU. In the meantime I'll keep defconfig for this test, we
> > > can always improve later.
> > 
> > Can you please test the attached patch?
> 
> Yes, that works, thanks!

Sorry, I spoke too soon: same error as before:

https://gitlab.com/xen-project/people/sstabellini/xen/-/jobs/1722909607/raw

(XEN) Hardware Dom0 crashed: rebooting machine in 5 seconds.

No output from the kernel.

By the way you can repro this at your end by doing the following:

# docker run -v /tmp:/tmp registry.gitlab.com/xen-project/xen/debian:stretch

Then inside the container follow pretty much what this script does:

xen.git/automation/scripts/qemu-alpine-x86_64.sh

For instance, you can do:

---
cat >> binaries/pxelinux.0 << EOF
#!ipxe

kernel /tmp/xen console=com1
module /tmp/bzImage console=hvc0
module /tmp/xen-rootfs.cpio.gz
boot
EOF

qemu-system-x86_64 \
    -cpu qemu64,+svm \
    -m 2G -smp 2 \
    -monitor none -serial stdio \
    -nographic \
    -device virtio-net-pci,netdev=n0 \
    -netdev user,id=n0,tftp=binaries,bootfile=/pxelinux.0
---

Just place your xen, bzImage and xen-rootfs.cpio.gz (the rootfs is not
required) binaries under /tmp and you are good to go.

Cheers,

Stefano

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Juergen Gross]

[-- Attachment #1.1.1: Type: text/plain, Size: 3352 bytes --]

On 28.10.21 01:24, Stefano Stabellini wrote:
> On Wed, 27 Oct 2021, Stefano Stabellini wrote:
>> On Wed, 27 Oct 2021, Juergen Gross wrote:
>>> On 26.10.21 02:54, Stefano Stabellini wrote:
>>>> On Mon, 25 Oct 2021, Juergen Gross wrote:
>>>>> On 22.10.21 21:41, Stefano Stabellini wrote:
>>>>>> +Juergen
>>>>>>
>>>>>> On Fri, 22 Oct 2021, Andrew Cooper wrote:
>>>>>>> On 22/10/2021 00:08, Stefano Stabellini wrote:
>>>>>>>> +# build depends
>>>>>>>> +RUN apt-get update && \
>>>>>>>> +    apt-get --quiet --yes install \
>>>>>>>> +        build-essential \
>>>>>>>> +        libssl-dev \
>>>>>>>> +        bc \
>>>>>>>> +        curl \
>>>>>>>> +        flex \
>>>>>>>> +        bison \
>>>>>>>> +        libelf-dev \
>>>>>>>> +        && \
>>>>>>>> +    \
>>>>>>>> +    # Build the kernel
>>>>>>>> +    curl -fsSLO
>>>>>>>> https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz
>>>>>>>> && \
>>>>>>>> +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
>>>>>>>> +    cd linux-"$LINUX_VERSION" && \
>>>>>>>> +    make defconfig && \
>>>>>>>> +    make xen.config && \
>>>>>>>> +    cp .config .config.orig && \
>>>>>>>> +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >>
>>>>>>>> .config
>>>>>>>> && \
>>>>>>>> +    make -j$(nproc) bzImage && \
>>>>>>>
>>>>>>> defconfig is huuuuge.  Can we use tinyconfig instead?
>>>>>>>
>>>>>>> Also, you want to turn off CONFIG_MODULES seeing as you only copy
>>>>>>> bzImage to the test.
>>>>>>>
>>>>>>> This also fixes the fact that `grep XEN` also matches
>>>>>>> CONFIG_NETXEN_NIC
>>>>>>> which has nothing to do with Xen.
>>>>>>
>>>>>> Tinyconfig would be a good idea but it crashes on QEMU with the appended
>>>>>> boot log. I did:
>>>>>>
>>>>>> # make tinyconfig
>>>>>> # make xen.config
>>>>>> # make bzImage
>>>>>>
>>>>>> Of course we could have our own tailored special kernel config for this
>>>>>> kernel but I like the simplicify of "make tinyconfig" or "make
>>>>>> defconfig" followed by "make xen.config".
>>>>>>
>>>>>> Unless you know one kconfig option or two that we need to add to
>>>>>> tinyconfig to make it work my preference is to go with defconfig for
>>>>>> now.
>>>>>
>>>>> You will need to add the dom0 and SMP related config options at least.
>>>>>
>>>>> I'll have a look at config.xen, which is a little bit weird: it
>>>>> configures the backends, but not dom0.
>>>>
>>>> Thanks, it would be good if make tinyconfig && make xen.config would
>>>> just work on QEMU. In the meantime I'll keep defconfig for this test, we
>>>> can always improve later.
>>>
>>> Can you please test the attached patch?
>>
>> Yes, that works, thanks!
> 
> Sorry, I spoke too soon: same error as before:
> 
> https://gitlab.com/xen-project/people/sstabellini/xen/-/jobs/1722909607/raw
> 
> (XEN) Hardware Dom0 crashed: rebooting machine in 5 seconds.
> 
> No output from the kernel.

This is not very surprising, as tinyconfig doesn't have CONFIG_PRINTK.
BTW, it doesn't have initrd support either (BLK_DEV_RAM and
BLK_DEV_INITRD), so I added those to xen.config as well.

With the (updated) attached patch the kernel boots fine as dom0 until
rootfs is tried to be mounted on a physical machine.

I've added CONFIG_BUG, too, in order to have more debugging output if
needed.


Juergen

[-- Attachment #1.1.2: 0001-x86-xen-update-xen.config.patch --]
[-- Type: text/x-patch, Size: 1493 bytes --]

From cb08391f332c850ba569b9232c8225ebe0e0758c Mon Sep 17 00:00:00 2001
From: Juergen Gross <jgross@suse.com>
Date: Wed, 27 Oct 2021 09:05:28 +0200
Subject: [PATCH] x86/xen: update xen.config

Update some entries in xen.config:

- add ACPI as being required for running as dom0
- add SMP for convenience
- add PRINTK and BUG for diagnosis support
- add BLK_DEV_INITRD and BLK_DEV_RAM for initrd support
- remove HIGHMEM64G as this option no longer exists
- remove XEN_MAX_DOMAIN_MEMORY as this option no longer exists

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/configs/xen.config | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/arch/x86/configs/xen.config b/arch/x86/configs/xen.config
index d9fc7139fd46..8db14cadf5b0 100644
--- a/arch/x86/configs/xen.config
+++ b/arch/x86/configs/xen.config
@@ -1,6 +1,4 @@
 # global x86 required specific stuff
-# On 32-bit HIGHMEM4G is not allowed
-CONFIG_HIGHMEM64G=y
 CONFIG_64BIT=y
 
 # These enable us to allow some of the
@@ -9,12 +7,17 @@ CONFIG_HYPERVISOR_GUEST=y
 CONFIG_PCI=y
 CONFIG_PCI_MSI=y
 CONFIG_X86_MCE=y
+CONFIG_ACPI=y
 CONFIG_ACPI_PROCESSOR=y
 CONFIG_CPU_FREQ=y
+CONFIG_SMP=y
+CONFIG_PRINTK=y
+CONFIG_BUG=y
+CONFIG_BLK_DEV_RAM=y
+CONFIG_BLK_DEV_INITRD=y
 
 # x86 xen specific config options
 CONFIG_XEN_PVH=y
-CONFIG_XEN_MAX_DOMAIN_MEMORY=500
 CONFIG_XEN_SAVE_RESTORE=y
 # CONFIG_XEN_DEBUG_FS is not set
 CONFIG_XEN_MCE_LOG=y
-- 
2.26.2


[-- Attachment #1.1.3: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 4193 bytes --]

On Thu, 28 Oct 2021, Juergen Gross wrote:
> On 28.10.21 01:24, Stefano Stabellini wrote:
> > On Wed, 27 Oct 2021, Stefano Stabellini wrote:
> > > On Wed, 27 Oct 2021, Juergen Gross wrote:
> > > > On 26.10.21 02:54, Stefano Stabellini wrote:
> > > > > On Mon, 25 Oct 2021, Juergen Gross wrote:
> > > > > > On 22.10.21 21:41, Stefano Stabellini wrote:
> > > > > > > +Juergen
> > > > > > > 
> > > > > > > On Fri, 22 Oct 2021, Andrew Cooper wrote:
> > > > > > > > On 22/10/2021 00:08, Stefano Stabellini wrote:
> > > > > > > > > +# build depends
> > > > > > > > > +RUN apt-get update && \
> > > > > > > > > +    apt-get --quiet --yes install \
> > > > > > > > > +        build-essential \
> > > > > > > > > +        libssl-dev \
> > > > > > > > > +        bc \
> > > > > > > > > +        curl \
> > > > > > > > > +        flex \
> > > > > > > > > +        bison \
> > > > > > > > > +        libelf-dev \
> > > > > > > > > +        && \
> > > > > > > > > +    \
> > > > > > > > > +    # Build the kernel
> > > > > > > > > +    curl -fsSLO
> > > > > > > > > https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz
> > > > > > > > > && \
> > > > > > > > > +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
> > > > > > > > > +    cd linux-"$LINUX_VERSION" && \
> > > > > > > > > +    make defconfig && \
> > > > > > > > > +    make xen.config && \
> > > > > > > > > +    cp .config .config.orig && \
> > > > > > > > > +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >>
> > > > > > > > > .config
> > > > > > > > > && \
> > > > > > > > > +    make -j$(nproc) bzImage && \
> > > > > > > > 
> > > > > > > > defconfig is huuuuge.  Can we use tinyconfig instead?
> > > > > > > > 
> > > > > > > > Also, you want to turn off CONFIG_MODULES seeing as you only
> > > > > > > > copy
> > > > > > > > bzImage to the test.
> > > > > > > > 
> > > > > > > > This also fixes the fact that `grep XEN` also matches
> > > > > > > > CONFIG_NETXEN_NIC
> > > > > > > > which has nothing to do with Xen.
> > > > > > > 
> > > > > > > Tinyconfig would be a good idea but it crashes on QEMU with the
> > > > > > > appended
> > > > > > > boot log. I did:
> > > > > > > 
> > > > > > > # make tinyconfig
> > > > > > > # make xen.config
> > > > > > > # make bzImage
> > > > > > > 
> > > > > > > Of course we could have our own tailored special kernel config for
> > > > > > > this
> > > > > > > kernel but I like the simplicify of "make tinyconfig" or "make
> > > > > > > defconfig" followed by "make xen.config".
> > > > > > > 
> > > > > > > Unless you know one kconfig option or two that we need to add to
> > > > > > > tinyconfig to make it work my preference is to go with defconfig
> > > > > > > for
> > > > > > > now.
> > > > > > 
> > > > > > You will need to add the dom0 and SMP related config options at
> > > > > > least.
> > > > > > 
> > > > > > I'll have a look at config.xen, which is a little bit weird: it
> > > > > > configures the backends, but not dom0.
> > > > > 
> > > > > Thanks, it would be good if make tinyconfig && make xen.config would
> > > > > just work on QEMU. In the meantime I'll keep defconfig for this test,
> > > > > we
> > > > > can always improve later.
> > > > 
> > > > Can you please test the attached patch?
> > > 
> > > Yes, that works, thanks!
> > 
> > Sorry, I spoke too soon: same error as before:
> > 
> > https://gitlab.com/xen-project/people/sstabellini/xen/-/jobs/1722909607/raw
> > 
> > (XEN) Hardware Dom0 crashed: rebooting machine in 5 seconds.
> > 
> > No output from the kernel.
> 
> This is not very surprising, as tinyconfig doesn't have CONFIG_PRINTK.
> BTW, it doesn't have initrd support either (BLK_DEV_RAM and
> BLK_DEV_INITRD), so I added those to xen.config as well.
> 
> With the (updated) attached patch the kernel boots fine as dom0 until
> rootfs is tried to be mounted on a physical machine.
> 
> I've added CONFIG_BUG, too, in order to have more debugging output if
> needed.

Much better, now there is output:
https://gitlab.com/xen-project/people/sstabellini/xen/-/jobs/1726396600/raw

This kernel is used as dom0 and from the logs it looks like for dom0 we
need a couple more things:
- flocks
- the kconfig to recognize #!/bin/sh

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Juergen Gross]

[-- Attachment #1.1.1: Type: text/plain, Size: 4127 bytes --]

On 28.10.21 18:41, Stefano Stabellini wrote:
> On Thu, 28 Oct 2021, Juergen Gross wrote:
>> On 28.10.21 01:24, Stefano Stabellini wrote:
>>> On Wed, 27 Oct 2021, Stefano Stabellini wrote:
>>>> On Wed, 27 Oct 2021, Juergen Gross wrote:
>>>>> On 26.10.21 02:54, Stefano Stabellini wrote:
>>>>>> On Mon, 25 Oct 2021, Juergen Gross wrote:
>>>>>>> On 22.10.21 21:41, Stefano Stabellini wrote:
>>>>>>>> +Juergen
>>>>>>>>
>>>>>>>> On Fri, 22 Oct 2021, Andrew Cooper wrote:
>>>>>>>>> On 22/10/2021 00:08, Stefano Stabellini wrote:
>>>>>>>>>> +# build depends
>>>>>>>>>> +RUN apt-get update && \
>>>>>>>>>> +    apt-get --quiet --yes install \
>>>>>>>>>> +        build-essential \
>>>>>>>>>> +        libssl-dev \
>>>>>>>>>> +        bc \
>>>>>>>>>> +        curl \
>>>>>>>>>> +        flex \
>>>>>>>>>> +        bison \
>>>>>>>>>> +        libelf-dev \
>>>>>>>>>> +        && \
>>>>>>>>>> +    \
>>>>>>>>>> +    # Build the kernel
>>>>>>>>>> +    curl -fsSLO
>>>>>>>>>> https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz
>>>>>>>>>> && \
>>>>>>>>>> +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
>>>>>>>>>> +    cd linux-"$LINUX_VERSION" && \
>>>>>>>>>> +    make defconfig && \
>>>>>>>>>> +    make xen.config && \
>>>>>>>>>> +    cp .config .config.orig && \
>>>>>>>>>> +    cat .config.orig | grep XEN | grep =m |sed 's/=m/=y/g' >>
>>>>>>>>>> .config
>>>>>>>>>> && \
>>>>>>>>>> +    make -j$(nproc) bzImage && \
>>>>>>>>>
>>>>>>>>> defconfig is huuuuge.  Can we use tinyconfig instead?
>>>>>>>>>
>>>>>>>>> Also, you want to turn off CONFIG_MODULES seeing as you only
>>>>>>>>> copy
>>>>>>>>> bzImage to the test.
>>>>>>>>>
>>>>>>>>> This also fixes the fact that `grep XEN` also matches
>>>>>>>>> CONFIG_NETXEN_NIC
>>>>>>>>> which has nothing to do with Xen.
>>>>>>>>
>>>>>>>> Tinyconfig would be a good idea but it crashes on QEMU with the
>>>>>>>> appended
>>>>>>>> boot log. I did:
>>>>>>>>
>>>>>>>> # make tinyconfig
>>>>>>>> # make xen.config
>>>>>>>> # make bzImage
>>>>>>>>
>>>>>>>> Of course we could have our own tailored special kernel config for
>>>>>>>> this
>>>>>>>> kernel but I like the simplicify of "make tinyconfig" or "make
>>>>>>>> defconfig" followed by "make xen.config".
>>>>>>>>
>>>>>>>> Unless you know one kconfig option or two that we need to add to
>>>>>>>> tinyconfig to make it work my preference is to go with defconfig
>>>>>>>> for
>>>>>>>> now.
>>>>>>>
>>>>>>> You will need to add the dom0 and SMP related config options at
>>>>>>> least.
>>>>>>>
>>>>>>> I'll have a look at config.xen, which is a little bit weird: it
>>>>>>> configures the backends, but not dom0.
>>>>>>
>>>>>> Thanks, it would be good if make tinyconfig && make xen.config would
>>>>>> just work on QEMU. In the meantime I'll keep defconfig for this test,
>>>>>> we
>>>>>> can always improve later.
>>>>>
>>>>> Can you please test the attached patch?
>>>>
>>>> Yes, that works, thanks!
>>>
>>> Sorry, I spoke too soon: same error as before:
>>>
>>> https://gitlab.com/xen-project/people/sstabellini/xen/-/jobs/1722909607/raw
>>>
>>> (XEN) Hardware Dom0 crashed: rebooting machine in 5 seconds.
>>>
>>> No output from the kernel.
>>
>> This is not very surprising, as tinyconfig doesn't have CONFIG_PRINTK.
>> BTW, it doesn't have initrd support either (BLK_DEV_RAM and
>> BLK_DEV_INITRD), so I added those to xen.config as well.
>>
>> With the (updated) attached patch the kernel boots fine as dom0 until
>> rootfs is tried to be mounted on a physical machine.
>>
>> I've added CONFIG_BUG, too, in order to have more debugging output if
>> needed.
> 
> Much better, now there is output:
> https://gitlab.com/xen-project/people/sstabellini/xen/-/jobs/1726396600/raw
> 
> This kernel is used as dom0 and from the logs it looks like for dom0 we
> need a couple more things:
> - flocks

CONFIG_FILE_LOCKING probably wants to be added.

> - the kconfig to recognize #!/bin/sh

This is probably CONFIG_BINFMT_SCRIPT.

I'm not sure those should be added to the upstream xen.config.


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3135 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH 2/3] automation: Linux 5.10.74 test-artifact

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 5247 bytes --]

On Fri, 29 Oct 2021, Juergen Gross wrote:
> On 28.10.21 18:41, Stefano Stabellini wrote:
> > On Thu, 28 Oct 2021, Juergen Gross wrote:
> > > On 28.10.21 01:24, Stefano Stabellini wrote:
> > > > On Wed, 27 Oct 2021, Stefano Stabellini wrote:
> > > > > On Wed, 27 Oct 2021, Juergen Gross wrote:
> > > > > > On 26.10.21 02:54, Stefano Stabellini wrote:
> > > > > > > On Mon, 25 Oct 2021, Juergen Gross wrote:
> > > > > > > > On 22.10.21 21:41, Stefano Stabellini wrote:
> > > > > > > > > +Juergen
> > > > > > > > > 
> > > > > > > > > On Fri, 22 Oct 2021, Andrew Cooper wrote:
> > > > > > > > > > On 22/10/2021 00:08, Stefano Stabellini wrote:
> > > > > > > > > > > +# build depends
> > > > > > > > > > > +RUN apt-get update && \
> > > > > > > > > > > +    apt-get --quiet --yes install \
> > > > > > > > > > > +        build-essential \
> > > > > > > > > > > +        libssl-dev \
> > > > > > > > > > > +        bc \
> > > > > > > > > > > +        curl \
> > > > > > > > > > > +        flex \
> > > > > > > > > > > +        bison \
> > > > > > > > > > > +        libelf-dev \
> > > > > > > > > > > +        && \
> > > > > > > > > > > +    \
> > > > > > > > > > > +    # Build the kernel
> > > > > > > > > > > +    curl -fsSLO
> > > > > > > > > > > https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-"$LINUX_VERSION".tar.xz
> > > > > > > > > > > && \
> > > > > > > > > > > +    tar xvJf linux-"$LINUX_VERSION".tar.xz && \
> > > > > > > > > > > +    cd linux-"$LINUX_VERSION" && \
> > > > > > > > > > > +    make defconfig && \
> > > > > > > > > > > +    make xen.config && \
> > > > > > > > > > > +    cp .config .config.orig && \
> > > > > > > > > > > +    cat .config.orig | grep XEN | grep =m |sed
> > > > > > > > > > > 's/=m/=y/g' >>
> > > > > > > > > > > .config
> > > > > > > > > > > && \
> > > > > > > > > > > +    make -j$(nproc) bzImage && \
> > > > > > > > > > 
> > > > > > > > > > defconfig is huuuuge.  Can we use tinyconfig instead?
> > > > > > > > > > 
> > > > > > > > > > Also, you want to turn off CONFIG_MODULES seeing as you only
> > > > > > > > > > copy
> > > > > > > > > > bzImage to the test.
> > > > > > > > > > 
> > > > > > > > > > This also fixes the fact that `grep XEN` also matches
> > > > > > > > > > CONFIG_NETXEN_NIC
> > > > > > > > > > which has nothing to do with Xen.
> > > > > > > > > 
> > > > > > > > > Tinyconfig would be a good idea but it crashes on QEMU with
> > > > > > > > > the
> > > > > > > > > appended
> > > > > > > > > boot log. I did:
> > > > > > > > > 
> > > > > > > > > # make tinyconfig
> > > > > > > > > # make xen.config
> > > > > > > > > # make bzImage
> > > > > > > > > 
> > > > > > > > > Of course we could have our own tailored special kernel config
> > > > > > > > > for
> > > > > > > > > this
> > > > > > > > > kernel but I like the simplicify of "make tinyconfig" or "make
> > > > > > > > > defconfig" followed by "make xen.config".
> > > > > > > > > 
> > > > > > > > > Unless you know one kconfig option or two that we need to add
> > > > > > > > > to
> > > > > > > > > tinyconfig to make it work my preference is to go with
> > > > > > > > > defconfig
> > > > > > > > > for
> > > > > > > > > now.
> > > > > > > > 
> > > > > > > > You will need to add the dom0 and SMP related config options at
> > > > > > > > least.
> > > > > > > > 
> > > > > > > > I'll have a look at config.xen, which is a little bit weird: it
> > > > > > > > configures the backends, but not dom0.
> > > > > > > 
> > > > > > > Thanks, it would be good if make tinyconfig && make xen.config
> > > > > > > would
> > > > > > > just work on QEMU. In the meantime I'll keep defconfig for this
> > > > > > > test,
> > > > > > > we
> > > > > > > can always improve later.
> > > > > > 
> > > > > > Can you please test the attached patch?
> > > > > 
> > > > > Yes, that works, thanks!
> > > > 
> > > > Sorry, I spoke too soon: same error as before:
> > > > 
> > > > https://gitlab.com/xen-project/people/sstabellini/xen/-/jobs/1722909607/raw
> > > > 
> > > > (XEN) Hardware Dom0 crashed: rebooting machine in 5 seconds.
> > > > 
> > > > No output from the kernel.
> > > 
> > > This is not very surprising, as tinyconfig doesn't have CONFIG_PRINTK.
> > > BTW, it doesn't have initrd support either (BLK_DEV_RAM and
> > > BLK_DEV_INITRD), so I added those to xen.config as well.
> > > 
> > > With the (updated) attached patch the kernel boots fine as dom0 until
> > > rootfs is tried to be mounted on a physical machine.
> > > 
> > > I've added CONFIG_BUG, too, in order to have more debugging output if
> > > needed.
> > 
> > Much better, now there is output:
> > https://gitlab.com/xen-project/people/sstabellini/xen/-/jobs/1726396600/raw
> > 
> > This kernel is used as dom0 and from the logs it looks like for dom0 we
> > need a couple more things:
> > - flocks
> 
> CONFIG_FILE_LOCKING probably wants to be added.
> 
> > - the kconfig to recognize #!/bin/sh
> 
> This is probably CONFIG_BINFMT_SCRIPT.
> 
> I'm not sure those should be added to the upstream xen.config.

I don't know either... They are certainly required for dom0, but not for
domU. Do we want xen.config to be for domU only or for domU and dom0? If
we want it to be for both, then CONFIG_FILE_LOCKING and
CONFIG_BINFMT_SCRIPT should be added otherwise not.

[PATCH 3/3] automation: add a QEMU based x86_64 Dom0/DomU test

[Stefano Stabellini]

From: Stefano Stabellini <stefano.stabellini@xilinx.com>

Introduce a test based on QEMU to run Xen, Dom0 and start a DomU.
This is similar to the existing qemu-alpine-arm64.sh script and test.
The only differences are:
- use Debian's qemu-system-x86_64 (on ARM we build our own)
- use ipxe instead of u-boot and ImageBuilder

Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
---
 automation/gitlab-ci/test.yaml           | 24 +++++++
 automation/scripts/qemu-alpine-x86_64.sh | 92 ++++++++++++++++++++++++
 2 files changed, 116 insertions(+)
 create mode 100644 automation/scripts/qemu-alpine-x86_64.sh

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 91a10febbf..c1d67ec4b5 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -47,6 +47,30 @@ qemu-alpine-arm64-gcc:
     - /^coverity-tested\/.*/
     - /^stable-.*/
 
+qemu-alpine-x86_64-gcc:
+  stage: test
+  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
+  variables:
+    CONTAINER: debian:unstable
+  script:
+    - ./automation/scripts/qemu-alpine-x86_64.sh 2>&1 | tee qemu-smoke-arm64.log
+  dependencies:
+    - alpine-3.12-gcc
+    - alpine-3.12-rootfs-export
+    - kernel-5.10.74-export
+  artifacts:
+    paths:
+      - smoke.serial
+      - '*.log'
+    when: always
+  tags:
+    - x86_64
+  except:
+    - master
+    - smoke
+    - /^coverity-tested\/.*/
+    - /^stable-.*/
+
 qemu-smoke-arm64-gcc:
   stage: test
   image: registry.gitlab.com/xen-project/xen/${CONTAINER}
diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
new file mode 100644
index 0000000000..41b05210d6
--- /dev/null
+++ b/automation/scripts/qemu-alpine-x86_64.sh
@@ -0,0 +1,92 @@
+#!/bin/bash
+
+set -ex
+
+apt-get -qy update
+apt-get -qy install --no-install-recommends qemu-system-x86 \
+                                            cpio \
+                                            curl \
+                                            busybox-static
+
+# DomU Busybox
+cd binaries
+mkdir -p initrd
+mkdir -p initrd/bin
+mkdir -p initrd/sbin
+mkdir -p initrd/etc
+mkdir -p initrd/dev
+mkdir -p initrd/proc
+mkdir -p initrd/sys
+mkdir -p initrd/lib
+mkdir -p initrd/var
+mkdir -p initrd/mnt
+cp /bin/busybox initrd/bin/busybox
+initrd/bin/busybox --install initrd/bin
+echo "#!/bin/sh
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+mount -t devtmpfs devtmpfs /dev
+/bin/sh" > initrd/init
+chmod +x initrd/init
+cd initrd
+find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
+cd ..
+
+mkdir -p rootfs
+cd rootfs
+tar xvzf ../initrd.tar.gz
+mkdir proc
+mkdir run
+mkdir srv
+mkdir sys
+rm var/run
+cp -ar ../dist/install/* .
+mv ../initrd.cpio.gz ./root
+cp ../bzImage ./root
+echo "name=\"test\"
+memory=512
+vcpus=1
+kernel=\"/root/bzImage\"
+ramdisk=\"/root/initrd.cpio.gz\"
+extra=\"console=hvc0 root=/dev/ram0 rdinit=/bin/sh\"
+" > root/test.cfg
+echo "#!/bin/bash
+
+export LD_LIBRARY_PATH=/usr/local/lib
+bash /etc/init.d/xencommons start
+
+xl list
+
+xl create -c /root/test.cfg
+
+" > etc/local.d/xen.start
+chmod +x etc/local.d/xen.start
+echo "rc_verbose=yes" >> etc/rc.conf
+find . |cpio -H newc -o|gzip > ../xen-rootfs.cpio.gz
+cd ../..
+
+cat >> binaries/pxelinux.0 <<- EOF
+#!ipxe
+
+kernel xen console=com1
+module bzImage console=hvc0
+module xen-rootfs.cpio.gz
+boot
+EOF
+
+# Run the test
+rm -f smoke.serial
+set +e
+timeout -k 1 720 \
+qemu-system-x86_64 \
+    -cpu qemu64,+svm \
+    -m 2G -smp 2 \
+    -monitor none -serial stdio \
+    -nographic \
+    -device virtio-net-pci,netdev=n0 \
+    -netdev user,id=n0,tftp=binaries,bootfile=/pxelinux.0 |& tee smoke.serial
+
+set -e
+(grep -q "Domain-0" smoke.serial && grep -q "BusyBox" smoke.serial) || exit 1
+exit 0
-- 
2.17.1

Re: [PATCH 3/3] automation: add a QEMU based x86_64 Dom0/DomU test

[Anthony PERARD]

On Thu, Oct 21, 2021 at 04:08:39PM -0700, Stefano Stabellini wrote:
> diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
> new file mode 100644
> index 0000000000..41b05210d6
> --- /dev/null
> +++ b/automation/scripts/qemu-alpine-x86_64.sh
> @@ -0,0 +1,92 @@
> +#!/bin/bash
> +
> +set -ex
> +
> +apt-get -qy update
> +apt-get -qy install --no-install-recommends qemu-system-x86 \
> +                                            cpio \
> +                                            curl \
> +                                            busybox-static

Please, don't install packages during the CI job. If you need new
packages, update the container.

That said, "curl" doesn't seems to be needed.

> +# DomU Busybox
> +cd binaries
> +mkdir -p initrd
> +mkdir -p initrd/bin
> +mkdir -p initrd/sbin
> +mkdir -p initrd/etc
> +mkdir -p initrd/dev
> +mkdir -p initrd/proc
> +mkdir -p initrd/sys
> +mkdir -p initrd/lib
> +mkdir -p initrd/var
> +mkdir -p initrd/mnt
> +cp /bin/busybox initrd/bin/busybox
> +initrd/bin/busybox --install initrd/bin
> +echo "#!/bin/sh
> +
> +mount -t proc proc /proc
> +mount -t sysfs sysfs /sys
> +mount -t devtmpfs devtmpfs /dev
> +/bin/sh" > initrd/init
> +chmod +x initrd/init
> +cd initrd
> +find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz

This isn't confusing at all, depending on the tool used to make an
archive, the resulting initrd has a different purpose :-).
    initrd.tar.gz -> dom0
    initrd.cpio.gz -> domU

> +cd ..

Maybe add a comment here saying that we are now preparing dom0 root
filesystem? (as there is one for domu)

> +mkdir -p rootfs
> +cd rootfs
> +tar xvzf ../initrd.tar.gz
> +mkdir proc
> +mkdir run
> +mkdir srv
> +mkdir sys
> +rm var/run
> +cp -ar ../dist/install/* .
> +mv ../initrd.cpio.gz ./root
> +cp ../bzImage ./root
> +echo "name=\"test\"
> +memory=512
> +vcpus=1
> +kernel=\"/root/bzImage\"
> +ramdisk=\"/root/initrd.cpio.gz\"
> +extra=\"console=hvc0 root=/dev/ram0 rdinit=/bin/sh\"
> +" > root/test.cfg
> +echo "#!/bin/bash
> +

Maybe add `set -x` ?

> +export LD_LIBRARY_PATH=/usr/local/lib
> +bash /etc/init.d/xencommons start
> +
> +xl list
> +
> +xl create -c /root/test.cfg
> +
> +" > etc/local.d/xen.start
> +chmod +x etc/local.d/xen.start
> +echo "rc_verbose=yes" >> etc/rc.conf
> +find . |cpio -H newc -o|gzip > ../xen-rootfs.cpio.gz
> +cd ../..
> +
> +cat >> binaries/pxelinux.0 <<- EOF

So, I've look at <<- meaning as I never used that before and it seems to
remove all leading tab chr from the input, yet they are no tab in the
input. So maybe use just <<EOF instead, without the dash -.

> +#!ipxe
> +
> +kernel xen console=com1
> +module bzImage console=hvc0
> +module xen-rootfs.cpio.gz
> +boot
> +EOF

Thanks,

-- 
Anthony PERARD

Re: [PATCH 3/3] automation: add a QEMU based x86_64 Dom0/DomU test

[Stefano Stabellini]

On Fri, 22 Oct 2021, Anthony PERARD wrote:
> On Thu, Oct 21, 2021 at 04:08:39PM -0700, Stefano Stabellini wrote:
> > diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
> > new file mode 100644
> > index 0000000000..41b05210d6
> > --- /dev/null
> > +++ b/automation/scripts/qemu-alpine-x86_64.sh
> > @@ -0,0 +1,92 @@
> > +#!/bin/bash
> > +
> > +set -ex
> > +
> > +apt-get -qy update
> > +apt-get -qy install --no-install-recommends qemu-system-x86 \
> > +                                            cpio \
> > +                                            curl \
> > +                                            busybox-static
> 
> Please, don't install packages during the CI job. If you need new
> packages, update the container.

The container used to run this script is the one used for the Xen build:
automation/build/debian/unstable.dockerfile, AKA
registry.gitlab.com/xen-project/xen/debian:unstable. We don't have a
specific container for the sole purpose of running tests.

Thus, I could add qemu-system-x86 to
automation/build/debian/unstable.dockerfile, but then we increase the
size of the debian unstable build container needlessly for all related
build jobs.

Or we could add one more special container just for running tests, but
then it is one more container to maintain and keep up-to-date.

This is why I chose as a compromise to keep the underlying container as
is, and just apt-get the extra 3-4 packages here. It is the same thing
we do on ARM: automation/scripts/qemu-alpine-arm64.sh. Also keep in mind
that this job is run in the "test" step where we have far fewer jobs at
the moment and the runners are not busy. (It would be different in the
"build" step where we have many jobs.)

I am not entirely sure what is the best solution overall, but for this
series at this stage I would prefer to keep the same strategy used for
the ARM tests (i.e. reuse the debian unstable build container and
apt-get the few missing packages.) If we do change the way we do it, I
would rather change both x86 and ARM at the same time.


> That said, "curl" doesn't seems to be needed.

Yeah I'll take away curl.


> > +# DomU Busybox
> > +cd binaries
> > +mkdir -p initrd
> > +mkdir -p initrd/bin
> > +mkdir -p initrd/sbin
> > +mkdir -p initrd/etc
> > +mkdir -p initrd/dev
> > +mkdir -p initrd/proc
> > +mkdir -p initrd/sys
> > +mkdir -p initrd/lib
> > +mkdir -p initrd/var
> > +mkdir -p initrd/mnt
> > +cp /bin/busybox initrd/bin/busybox
> > +initrd/bin/busybox --install initrd/bin
> > +echo "#!/bin/sh
> > +
> > +mount -t proc proc /proc
> > +mount -t sysfs sysfs /sys
> > +mount -t devtmpfs devtmpfs /dev
> > +/bin/sh" > initrd/init
> > +chmod +x initrd/init
> > +cd initrd
> > +find . | cpio --create --format='newc' | gzip > ../initrd.cpio.gz
> 
> This isn't confusing at all, depending on the tool used to make an
> archive, the resulting initrd has a different purpose :-).
>     initrd.tar.gz -> dom0
>     initrd.cpio.gz -> domU
> 
> > +cd ..
> 
> Maybe add a comment here saying that we are now preparing dom0 root
> filesystem? (as there is one for domu)

Sure I can add a comment


> > +mkdir -p rootfs
> > +cd rootfs
> > +tar xvzf ../initrd.tar.gz
> > +mkdir proc
> > +mkdir run
> > +mkdir srv
> > +mkdir sys
> > +rm var/run
> > +cp -ar ../dist/install/* .
> > +mv ../initrd.cpio.gz ./root
> > +cp ../bzImage ./root
> > +echo "name=\"test\"
> > +memory=512
> > +vcpus=1
> > +kernel=\"/root/bzImage\"
> > +ramdisk=\"/root/initrd.cpio.gz\"
> > +extra=\"console=hvc0 root=/dev/ram0 rdinit=/bin/sh\"
> > +" > root/test.cfg
> > +echo "#!/bin/bash
> > +
> 
> Maybe add `set -x` ?

OK


> > +export LD_LIBRARY_PATH=/usr/local/lib
> > +bash /etc/init.d/xencommons start
> > +
> > +xl list
> > +
> > +xl create -c /root/test.cfg
> > +
> > +" > etc/local.d/xen.start
> > +chmod +x etc/local.d/xen.start
> > +echo "rc_verbose=yes" >> etc/rc.conf
> > +find . |cpio -H newc -o|gzip > ../xen-rootfs.cpio.gz
> > +cd ../..
> > +
> > +cat >> binaries/pxelinux.0 <<- EOF
> 
> So, I've look at <<- meaning as I never used that before and it seems to
> remove all leading tab chr from the input, yet they are no tab in the
> input. So maybe use just <<EOF instead, without the dash -.

OK


> > +#!ipxe
> > +
> > +kernel xen console=com1
> > +module bzImage console=hvc0
> > +module xen-rootfs.cpio.gz
> > +boot
> > +EOF

Re: [PATCH 3/3] automation: add a QEMU based x86_64 Dom0/DomU test

[Anthony PERARD]

On Fri, Oct 22, 2021 at 01:05:35PM -0700, Stefano Stabellini wrote:
> On Fri, 22 Oct 2021, Anthony PERARD wrote:
> > On Thu, Oct 21, 2021 at 04:08:39PM -0700, Stefano Stabellini wrote:
> > > diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
> > > new file mode 100644
> > > index 0000000000..41b05210d6
> > > --- /dev/null
> > > +++ b/automation/scripts/qemu-alpine-x86_64.sh
> > > @@ -0,0 +1,92 @@
> > > +#!/bin/bash
> > > +
> > > +set -ex
> > > +
> > > +apt-get -qy update
> > > +apt-get -qy install --no-install-recommends qemu-system-x86 \
> > > +                                            cpio \
> > > +                                            curl \
> > > +                                            busybox-static
> > 
> > Please, don't install packages during the CI job. If you need new
> > packages, update the container.
> 
> The container used to run this script is the one used for the Xen build:
> automation/build/debian/unstable.dockerfile, AKA
> registry.gitlab.com/xen-project/xen/debian:unstable. We don't have a
> specific container for the sole purpose of running tests.

I've added qemu to our debian:stretch container recently, in order to
run the "qemu-smoke-*" tests without running apt commands. Unless more
recent software are needed, you could base the "qemu-alpine-x86*" test
on debian:stretch which might only then missing cpio and busybox. Update
of the stretch container should go smoothly as it has been done
recently.

> Thus, I could add qemu-system-x86 to
> automation/build/debian/unstable.dockerfile, but then we increase the
> size of the debian unstable build container needlessly for all related
> build jobs.

There is something I'm missing, how is it a problem to have a container
that is a bit bigger? What sort of problem could we have to deal with?
On the other hand, doing more task, installing software, downloading
anything from the internet, makes the job much less reliable. It
increase the risk of a failure do to external factors and it takes more
time to run the test.

> Or we could add one more special container just for running tests, but
> then it is one more container to maintain and keep up-to-date.
> 
> This is why I chose as a compromise to keep the underlying container as
> is, and just apt-get the extra 3-4 packages here. It is the same thing
> we do on ARM: automation/scripts/qemu-alpine-arm64.sh. Also keep in mind
> that this job is run in the "test" step where we have far fewer jobs at
> the moment and the runners are not busy. (It would be different in the
> "build" step where we have many jobs.)

I don't really see any difference between a "test" job and a "build"
jobs, both kind use the same resource/runner.

On that note, they're is something I've learned recently: "test" job
don't even have to wait for all "build" job to complete, they can run in
parallel. We would just need to replace "dependencies" by "needs":
    https://docs.gitlab.com/ee/ci/yaml/index.html#needs
But that could be an improvement for an other time and only a side note
for the patch.

> I am not entirely sure what is the best solution overall, but for this
> series at this stage I would prefer to keep the same strategy used for
> the ARM tests (i.e. reuse the debian unstable build container and
> apt-get the few missing packages.) If we do change the way we do it, I
> would rather change both x86 and ARM at the same time.

I'm pretty sure the best strategy would be to do as little as possible
during a job, download as little as possible and possibly cache as much
as possible and do as much as possible ahead of time. Feel free to
change the Arm test, but I don't think it is necessary to change the Arm
test at the same time as introducing an x86 test.

Cheers,

-- 
Anthony PERARD

Re: [PATCH 3/3] automation: add a QEMU based x86_64 Dom0/DomU test

[Stefano Stabellini]

On Mon, 25 Oct 2021, Anthony PERARD wrote:
> On Fri, Oct 22, 2021 at 01:05:35PM -0700, Stefano Stabellini wrote:
> > On Fri, 22 Oct 2021, Anthony PERARD wrote:
> > > On Thu, Oct 21, 2021 at 04:08:39PM -0700, Stefano Stabellini wrote:
> > > > diff --git a/automation/scripts/qemu-alpine-x86_64.sh b/automation/scripts/qemu-alpine-x86_64.sh
> > > > new file mode 100644
> > > > index 0000000000..41b05210d6
> > > > --- /dev/null
> > > > +++ b/automation/scripts/qemu-alpine-x86_64.sh
> > > > @@ -0,0 +1,92 @@
> > > > +#!/bin/bash
> > > > +
> > > > +set -ex
> > > > +
> > > > +apt-get -qy update
> > > > +apt-get -qy install --no-install-recommends qemu-system-x86 \
> > > > +                                            cpio \
> > > > +                                            curl \
> > > > +                                            busybox-static
> > > 
> > > Please, don't install packages during the CI job. If you need new
> > > packages, update the container.
> > 
> > The container used to run this script is the one used for the Xen build:
> > automation/build/debian/unstable.dockerfile, AKA
> > registry.gitlab.com/xen-project/xen/debian:unstable. We don't have a
> > specific container for the sole purpose of running tests.
> 
> I've added qemu to our debian:stretch container recently, in order to
> run the "qemu-smoke-*" tests without running apt commands. Unless more
> recent software are needed, you could base the "qemu-alpine-x86*" test
> on debian:stretch which might only then missing cpio and busybox. Update
> of the stretch container should go smoothly as it has been done
> recently.

I am happy to use debian:stretch in this patch series. I'll make the
change. That only leaves cpio and busybox-static to apt-get which are
small. In general, I am not too happy about increasing the size of build
containers with testing binaries so I would rather not increase it any
further, see below.


> > Thus, I could add qemu-system-x86 to
> > automation/build/debian/unstable.dockerfile, but then we increase the
> > size of the debian unstable build container needlessly for all related
> > build jobs.
> 
> There is something I'm missing, how is it a problem to have a container
> that is a bit bigger? What sort of problem could we have to deal with?

It takes time to clone the container in the gitlab-ci, the bigger the
container the more time it takes. It is fetched over the network. Now we
are fetching qemu (as part of the container) 10 times during the build
although it is not needed.

There is a compromise to be made. Let's say the QEMU package is 100MB.
If we add QEMU to debian:stretch we are going to increase the overall
data usage by about 1GB, which means we are waiting to fetch 1GB of
additional data during the pipeline.

If we needed QEMU only once, then we could just apt-get it and we would
only fetch 100MB once. Although using apt-get will be a bit slower
because the deb needs to be unpacked, etc. I think that if we used QEMU
only in one test then it would be faster to apt-get it.

However in our case we are using QEMU 4 times during the tests, so
adding QEMU to debian:stretch probably made things a bit faster or at
least not slower.

Does it make sense?

This is why we should have specialized containers for the tests instead
of adding test packages to the build containers. However, today all
containers are updated manually so it would be best to make containers
automatically updatable first.  Until then I think it is OK to apt-get
stuff.


> On the other hand, doing more task, installing software, downloading
> anything from the internet, makes the job much less reliable. It
> increase the risk of a failure do to external factors and it takes more
> time to run the test.

That is fair but it is more a theoretical problem than a real issue at
the moment. In reality I haven't seen Debian's apt-get failing even a
single time in our gitlab-ci (it failed because our debian:unstable
container went out of date but that's our fault).

But we do have a severe problem at the moment with external sources: our
"git clones" keep failing during the build on x86. That is definitely
something worth improving (see my other email thread on the subject) and
it is the main problem affecting gitlab-ci at the moment, I keep having
to restart jobs almost daily to get the overall pipeline to "pass".

If you have any ideas on how to stop fetching things using "git" from
external repositories in gitlab-ci that would be fantastic :-)
The only thing I could think of to "fix it" is moving all external repos
to gitlab repositories mirrors.


> > Or we could add one more special container just for running tests, but
> > then it is one more container to maintain and keep up-to-date.
> > 
> > This is why I chose as a compromise to keep the underlying container as
> > is, and just apt-get the extra 3-4 packages here. It is the same thing
> > we do on ARM: automation/scripts/qemu-alpine-arm64.sh. Also keep in mind
> > that this job is run in the "test" step where we have far fewer jobs at
> > the moment and the runners are not busy. (It would be different in the
> > "build" step where we have many jobs.)
> 
> I don't really see any difference between a "test" job and a "build"
> jobs, both kind use the same resource/runner.
> 
> On that note, they're is something I've learned recently: "test" job
> don't even have to wait for all "build" job to complete, they can run in
> parallel. We would just need to replace "dependencies" by "needs":
>     https://docs.gitlab.com/ee/ci/yaml/index.html#needs
> But that could be an improvement for an other time and only a side note
> for the patch.

That is really cool! I didn't know about it. Yes, currently there is a
big distinction between build and test jobs because build jobs are the
bottleneck and test jobs don't start before all the build jobs finish.


> > I am not entirely sure what is the best solution overall, but for this
> > series at this stage I would prefer to keep the same strategy used for
> > the ARM tests (i.e. reuse the debian unstable build container and
> > apt-get the few missing packages.) If we do change the way we do it, I
> > would rather change both x86 and ARM at the same time.
> 
> I'm pretty sure the best strategy would be to do as little as possible
> during a job, download as little as possible and possibly cache as much
> as possible and do as much as possible ahead of time. Feel free to
> change the Arm test, but I don't think it is necessary to change the Arm
> test at the same time as introducing an x86 test.

I agree.

At the same time it would be nice to follow the same strategy between
x86 and ARM going forward: if one optimization is made for one, it is
also made for the other.

Re: [PATCH 3/3] automation: add a QEMU based x86_64 Dom0/DomU test

[Anthony PERARD]

On Mon, Oct 25, 2021 at 06:33:53PM -0700, Stefano Stabellini wrote:
> On Mon, 25 Oct 2021, Anthony PERARD wrote:
> > There is something I'm missing, how is it a problem to have a container
> > that is a bit bigger? What sort of problem could we have to deal with?
> 
> It takes time to clone the container in the gitlab-ci, the bigger the
> container the more time it takes. It is fetched over the network. Now we
> are fetching qemu (as part of the container) 10 times during the build
> although it is not needed.

I guess the issue would be more like we don't do enough caching with our
gitlab runners. So package installation is just a workaround.

> But we do have a severe problem at the moment with external sources: our
> "git clones" keep failing during the build on x86. That is definitely
> something worth improving (see my other email thread on the subject) and
> it is the main problem affecting gitlab-ci at the moment, I keep having
> to restart jobs almost daily to get the overall pipeline to "pass".
> 
> If you have any ideas on how to stop fetching things using "git" from
> external repositories in gitlab-ci that would be fantastic :-)
> The only thing I could think of to "fix it" is moving all external repos
> to gitlab repositories mirrors.

I don't think that would work, I've seen the initial clone/fetch of a
job fail as well, so from gitlab. If we could have a cache of those
external resources closer to the runners, that would be better.

> > > I am not entirely sure what is the best solution overall, but for this
> > > series at this stage I would prefer to keep the same strategy used for
> > > the ARM tests (i.e. reuse the debian unstable build container and
> > > apt-get the few missing packages.) If we do change the way we do it, I
> > > would rather change both x86 and ARM at the same time.
> > 
> > I'm pretty sure the best strategy would be to do as little as possible
> > during a job, download as little as possible and possibly cache as much
> > as possible and do as much as possible ahead of time. Feel free to
> > change the Arm test, but I don't think it is necessary to change the Arm
> > test at the same time as introducing an x86 test.
> 
> I agree.
> 
> At the same time it would be nice to follow the same strategy between
> x86 and ARM going forward: if one optimization is made for one, it is
> also made for the other.

Probably better, yes.

Thanks,

-- 
Anthony PERARD

Solving the gitlab-ci git fetch issue, was: [PATCH 3/3] automation: add a QEMU based x86_64 Dom0/DomU test

[Stefano Stabellini]

On Wed, 27 Oct 2021, Anthony PERARD wrote:
> > But we do have a severe problem at the moment with external sources: our
> > "git clones" keep failing during the build on x86. That is definitely
> > something worth improving (see my other email thread on the subject) and
> > it is the main problem affecting gitlab-ci at the moment, I keep having
> > to restart jobs almost daily to get the overall pipeline to "pass".
> > 
> > If you have any ideas on how to stop fetching things using "git" from
> > external repositories in gitlab-ci that would be fantastic :-)
> > The only thing I could think of to "fix it" is moving all external repos
> > to gitlab repositories mirrors.
> 
> I don't think that would work, I've seen the initial clone/fetch of a
> job fail as well, so from gitlab. If we could have a cache of those
> external resources closer to the runners, that would be better.

You mean like a git repository mirror inside the Rackspace network (the
provider of the x86 runner), right? Then we would force the git client
to go to the Rackspace mirror instead of directly to the target using
"insteadOf".

Is that what you meant? Doug, do you think it would work?

Re: Solving the gitlab-ci git fetch issue, was: [PATCH 3/3] automation: add a QEMU based x86_64 Dom0/DomU test

[Anthony PERARD]

On Wed, Oct 27, 2021 at 02:43:46PM -0700, Stefano Stabellini wrote:
> On Wed, 27 Oct 2021, Anthony PERARD wrote:
> > > But we do have a severe problem at the moment with external sources: our
> > > "git clones" keep failing during the build on x86. That is definitely
> > > something worth improving (see my other email thread on the subject) and
> > > it is the main problem affecting gitlab-ci at the moment, I keep having
> > > to restart jobs almost daily to get the overall pipeline to "pass".
> > > 
> > > If you have any ideas on how to stop fetching things using "git" from
> > > external repositories in gitlab-ci that would be fantastic :-)
> > > The only thing I could think of to "fix it" is moving all external repos
> > > to gitlab repositories mirrors.
> > 
> > I don't think that would work, I've seen the initial clone/fetch of a
> > job fail as well, so from gitlab. If we could have a cache of those
> > external resources closer to the runners, that would be better.
> 
> You mean like a git repository mirror inside the Rackspace network (the
> provider of the x86 runner), right? Then we would force the git client
> to go to the Rackspace mirror instead of directly to the target using
> "insteadOf".

That would seems the best to me. If we could install Ian's
git-cache-proxy that is used in osstest, that would be good I think.
Having a mirror instead might work too but that would mean figure out
which repo we would need a mirror of.

I did try a different alternative a while back, I tried to use gitlab's
caching capability:
    automation: Cache sub-project git tree in build jobs
    https://lore.kernel.org/xen-devel/20191219144217.305851-3-anthony.perard@citrix.com/
It mostly works but I'm not sure how useful it is as it seems there is
10 computers that would maintain 10 different caches, and most of them
for a short while.

> Is that what you meant? Doug, do you think it would work?

-- 
Anthony PERARD

Re: Solving the gitlab-ci git fetch issue, was: [PATCH 3/3] automation: add a QEMU based x86_64 Dom0/DomU test

[Stefano Stabellini]

On Thu, 28 Oct 2021, Anthony PERARD wrote:
> On Wed, Oct 27, 2021 at 02:43:46PM -0700, Stefano Stabellini wrote:
> > On Wed, 27 Oct 2021, Anthony PERARD wrote:
> > > > But we do have a severe problem at the moment with external sources: our
> > > > "git clones" keep failing during the build on x86. That is definitely
> > > > something worth improving (see my other email thread on the subject) and
> > > > it is the main problem affecting gitlab-ci at the moment, I keep having
> > > > to restart jobs almost daily to get the overall pipeline to "pass".
> > > > 
> > > > If you have any ideas on how to stop fetching things using "git" from
> > > > external repositories in gitlab-ci that would be fantastic :-)
> > > > The only thing I could think of to "fix it" is moving all external repos
> > > > to gitlab repositories mirrors.
> > > 
> > > I don't think that would work, I've seen the initial clone/fetch of a
> > > job fail as well, so from gitlab. If we could have a cache of those
> > > external resources closer to the runners, that would be better.
> > 
> > You mean like a git repository mirror inside the Rackspace network (the
> > provider of the x86 runner), right? Then we would force the git client
> > to go to the Rackspace mirror instead of directly to the target using
> > "insteadOf".
> 
> That would seems the best to me. If we could install Ian's
> git-cache-proxy that is used in osstest, that would be good I think.
> Having a mirror instead might work too but that would mean figure out
> which repo we would need a mirror of.
> 
> I did try a different alternative a while back, I tried to use gitlab's
> caching capability:
>     automation: Cache sub-project git tree in build jobs
>     https://lore.kernel.org/xen-devel/20191219144217.305851-3-anthony.perard@citrix.com/
> It mostly works but I'm not sure how useful it is as it seems there is
> 10 computers that would maintain 10 different caches, and most of them
> for a short while.

Interesting!

With the current runners being short-lived, it makes sense only to have
git-cache-proxy on a different always-on host. But even after the
runners are changed to be always running, instead of spawned on demand,
it might be better to have git-cache-proxy on a separate host so that
all runners can use it.  It should be more efficient?

Re: [PATCH 0/3] automation: introduce an x86_64 Dom0/DomU test

[Ian Jackson]

Stefano Stabellini writes ("[PATCH 0/3] automation: introduce an x86_64 Dom0/DomU test"):
> The last patch is the one introducing a new test. This one can fail.
> However, it is a new test at the end of the pipeline: it doesn't impact
> the existing tests. In the worst case, the test will fail and the whole
> pipeline will be marked as "failed" but looking at the jobs all the
> other will continue to be marked as successful. In short, if it fails,
> we can simply ignore it. Also, at the moment it is actually succeeding.

Thanks.  For 4.16:

Release-Acked-by: Ian Jackson <iwj@xenproject.org>