[PATCH v2 0/2] XTF on arm64 support in CI

[PATCH v2 0/2] XTF on arm64 support in CI

[Michal Orzel]

This series adds support into our CI for using XTF to perform low-level
testing on arm64 (dom0less use case for now). For the purpose of the CI
testing, we are going to use XTF fork with implemented arm64 support.

This series populates the CI jobs to run hyp-xen-version XTF test to validate
some of the xen_version hypercall operations.

Changes in v2:
Let's take the opportunity to run XTF as a domU in pure dom0less configuration,
as this is something our testing do not cover at the moment.

Michal Orzel (2):
  automation: Add support for using XTF for arm64 testing
  automation: Add test jobs to run XTF hypercall xen_version test

 automation/gitlab-ci/test.yaml                | 16 +++++
 automation/scripts/qemu-xtf-dom0less-arm64.sh | 68 +++++++++++++++++++
 2 files changed, 84 insertions(+)
 create mode 100755 automation/scripts/qemu-xtf-dom0less-arm64.sh

-- 
2.25.1

[PATCH v2 1/2] automation: Add support for using XTF for arm64 testing

[Michal Orzel]

Introduce support for using XTF on Arm to perform low-level testing.
For the purpose of the CI testing, let's use the fork [1] from upstream
XTF with implemented support for arm64 (the upstream XTF only supports
x86).

Add a new script under automation/scripts to be used by the CI XTF test
jobs to perform the following tasks:
 - Compiling XTF,
 - Generating u-boot script using ImageBuilder,
 - Running Xen with XTF as domU in pure dom0less configuration using Qemu,
 - Checking test result.

The script takes the name of the XTF test to run as a first parameter.

[1] https://gitlab.com/xen-project/fusa/xtf.git (branch xtf-arm)

Signed-off-by: Michal Orzel <michal.orzel@amd.com>
---
Changes in v2:
 - run XTF in pure dom0less configuration
---
 automation/scripts/qemu-xtf-dom0less-arm64.sh | 68 +++++++++++++++++++
 1 file changed, 68 insertions(+)
 create mode 100755 automation/scripts/qemu-xtf-dom0less-arm64.sh

diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh
new file mode 100755
index 000000000000..3ec9cf74e129
--- /dev/null
+++ b/automation/scripts/qemu-xtf-dom0less-arm64.sh
@@ -0,0 +1,68 @@
+#!/bin/bash
+
+set -ex
+
+# Name of the XTF test
+xtf_test=$1
+
+# Message returned by XTF in case of success
+passed="Test result: SUCCESS"
+
+# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
+curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
+./binaries/qemu-system-aarch64 \
+   -machine virtualization=true \
+   -cpu cortex-a57 -machine type=virt \
+   -m 2048 -smp 2 -display none \
+   -machine dumpdtb=binaries/virt-gicv2.dtb
+
+# XTF
+# Build a single XTF test passed as a first parameter to the script.
+# Build XTF with GICv2 support to match Qemu configuration and with SBSA UART
+# support, so that the test will use an emulated UART for printing messages.
+# This will allow us to run the test on both debug and non-debug Xen builds.
+rm -rf xtf
+git clone https://gitlab.com/xen-project/fusa/xtf.git -b xtf-arm
+make -C xtf TESTS=tests/${xtf_test} CONFIG_SBSA_UART=y CONFIG_GICV2=y -j$(nproc)
+cp xtf/tests/${xtf_test}/test-mmu64le-${xtf_test} binaries/xtf-test
+
+# ImageBuilder
+echo 'MEMORY_START="0x40000000"
+MEMORY_END="0xC0000000"
+
+XEN="xen"
+DEVICE_TREE="virt-gicv2.dtb"
+
+XEN_CMD="console=dtuart"
+
+DOMU_KERNEL[0]="xtf-test"
+DOMU_MEM[0]="128"
+
+NUM_DOMUS=1
+
+LOAD_CMD="tftpb"
+UBOOT_SOURCE="boot.source"
+UBOOT_SCRIPT="boot.scr"' > binaries/config
+
+rm -rf imagebuilder
+git clone https://gitlab.com/ViryaOS/imagebuilder
+bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/config
+
+# Run the test
+rm -f smoke.serial
+set +e
+echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"| \
+timeout -k 1 120 \
+./binaries/qemu-system-aarch64 \
+    -machine virtualization=true \
+    -cpu cortex-a57 -machine type=virt \
+    -m 2048 -monitor none -serial stdio \
+    -smp 2 \
+    -no-reboot \
+    -device virtio-net-pci,netdev=n0 \
+    -netdev user,id=n0,tftp=binaries \
+    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin |& tee smoke.serial
+
+set -e
+(grep -q "${passed}" smoke.serial) || exit 1
+exit 0
-- 
2.25.1

[PATCH v2 2/2] automation: Add test jobs to run XTF hypercall xen_version test

[Michal Orzel]

Add test jobs in both debug and non-debug versions to run hyp-xen-version
XTF test as a dom0less domU on arm64. The purpose of this test is to
validate the functional behavior of xen_version hypercall.

Signed-off-by: Michal Orzel <michal.orzel@amd.com>
---
Changes in v2:
 - as we are going to run XTF in pure dom0less configuration, specify only
   the need for Xen and Qemu
---
 automation/gitlab-ci/test.yaml | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 2d57f4feb74a..afd80adfe17c 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -165,6 +165,22 @@ qemu-smoke-dom0less-arm64-gcc-debug-boot-cpupools:
     - *arm64-test-needs
     - alpine-3.12-gcc-debug-arm64-boot-cpupools
 
+qemu-xtf-dom0less-arm64-gcc-hyp-xen-version:
+  extends: .qemu-arm64
+  script:
+    - ./automation/scripts/qemu-xtf-dom0less-arm64.sh hyp-xen-version 2>&1 | tee ${LOGFILE}
+  needs:
+    - alpine-3.12-gcc-arm64
+    - qemu-system-aarch64-6.0.0-arm64-export
+
+qemu-xtf-dom0less-arm64-gcc-debug-hyp-xen-version:
+  extends: .qemu-arm64
+  script:
+    - ./automation/scripts/qemu-xtf-dom0less-arm64.sh hyp-xen-version 2>&1 | tee ${LOGFILE}
+  needs:
+    - alpine-3.12-gcc-debug-arm64
+    - qemu-system-aarch64-6.0.0-arm64-export
+
 qemu-smoke-dom0-arm32-gcc:
   extends: .qemu-arm32
   script:
-- 
2.25.1

Re: [PATCH v2 1/2] automation: Add support for using XTF for arm64 testing

[Stefano Stabellini]

On Fri, 16 Dec 2022, Michal Orzel wrote:
> Introduce support for using XTF on Arm to perform low-level testing.
> For the purpose of the CI testing, let's use the fork [1] from upstream
> XTF with implemented support for arm64 (the upstream XTF only supports
> x86).
> 
> Add a new script under automation/scripts to be used by the CI XTF test
> jobs to perform the following tasks:
>  - Compiling XTF,
>  - Generating u-boot script using ImageBuilder,
>  - Running Xen with XTF as domU in pure dom0less configuration using Qemu,
>  - Checking test result.
> 
> The script takes the name of the XTF test to run as a first parameter.
> 
> [1] https://gitlab.com/xen-project/fusa/xtf.git (branch xtf-arm)
> 
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in v2:
>  - run XTF in pure dom0less configuration
> ---
>  automation/scripts/qemu-xtf-dom0less-arm64.sh | 68 +++++++++++++++++++
>  1 file changed, 68 insertions(+)
>  create mode 100755 automation/scripts/qemu-xtf-dom0less-arm64.sh
> 
> diff --git a/automation/scripts/qemu-xtf-dom0less-arm64.sh b/automation/scripts/qemu-xtf-dom0less-arm64.sh
> new file mode 100755
> index 000000000000..3ec9cf74e129
> --- /dev/null
> +++ b/automation/scripts/qemu-xtf-dom0less-arm64.sh
> @@ -0,0 +1,68 @@
> +#!/bin/bash
> +
> +set -ex
> +
> +# Name of the XTF test
> +xtf_test=$1
> +
> +# Message returned by XTF in case of success
> +passed="Test result: SUCCESS"
> +
> +# XXX QEMU looks for "efi-virtio.rom" even if it is unneeded
> +curl -fsSLO https://github.com/qemu/qemu/raw/v5.2.0/pc-bios/efi-virtio.rom
> +./binaries/qemu-system-aarch64 \
> +   -machine virtualization=true \
> +   -cpu cortex-a57 -machine type=virt \
> +   -m 2048 -smp 2 -display none \
> +   -machine dumpdtb=binaries/virt-gicv2.dtb
> +
> +# XTF
> +# Build a single XTF test passed as a first parameter to the script.
> +# Build XTF with GICv2 support to match Qemu configuration and with SBSA UART
> +# support, so that the test will use an emulated UART for printing messages.
> +# This will allow us to run the test on both debug and non-debug Xen builds.
> +rm -rf xtf
> +git clone https://gitlab.com/xen-project/fusa/xtf.git -b xtf-arm
> +make -C xtf TESTS=tests/${xtf_test} CONFIG_SBSA_UART=y CONFIG_GICV2=y -j$(nproc)
> +cp xtf/tests/${xtf_test}/test-mmu64le-${xtf_test} binaries/xtf-test
> +
> +# ImageBuilder
> +echo 'MEMORY_START="0x40000000"
> +MEMORY_END="0xC0000000"
> +
> +XEN="xen"
> +DEVICE_TREE="virt-gicv2.dtb"
> +
> +XEN_CMD="console=dtuart"
> +
> +DOMU_KERNEL[0]="xtf-test"
> +DOMU_MEM[0]="128"
> +
> +NUM_DOMUS=1
> +
> +LOAD_CMD="tftpb"
> +UBOOT_SOURCE="boot.source"
> +UBOOT_SCRIPT="boot.scr"' > binaries/config
> +
> +rm -rf imagebuilder
> +git clone https://gitlab.com/ViryaOS/imagebuilder
> +bash imagebuilder/scripts/uboot-script-gen -t tftp -d binaries/ -c binaries/config
> +
> +# Run the test
> +rm -f smoke.serial
> +set +e
> +echo "  virtio scan; dhcp; tftpb 0x40000000 boot.scr; source 0x40000000"| \
> +timeout -k 1 120 \
> +./binaries/qemu-system-aarch64 \
> +    -machine virtualization=true \
> +    -cpu cortex-a57 -machine type=virt \
> +    -m 2048 -monitor none -serial stdio \
> +    -smp 2 \
> +    -no-reboot \
> +    -device virtio-net-pci,netdev=n0 \
> +    -netdev user,id=n0,tftp=binaries \
> +    -bios /usr/lib/u-boot/qemu_arm64/u-boot.bin |& tee smoke.serial
> +
> +set -e
> +(grep -q "${passed}" smoke.serial) || exit 1
> +exit 0
> -- 
> 2.25.1
> 

Re: [PATCH v2 2/2] automation: Add test jobs to run XTF hypercall xen_version test

[Stefano Stabellini]

On Fri, 16 Dec 2022, Michal Orzel wrote:
> Add test jobs in both debug and non-debug versions to run hyp-xen-version
> XTF test as a dom0less domU on arm64. The purpose of this test is to
> validate the functional behavior of xen_version hypercall.
> 
> Signed-off-by: Michal Orzel <michal.orzel@amd.com>

Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>


> ---
> Changes in v2:
>  - as we are going to run XTF in pure dom0less configuration, specify only
>    the need for Xen and Qemu
> ---
>  automation/gitlab-ci/test.yaml | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
> 
> diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
> index 2d57f4feb74a..afd80adfe17c 100644
> --- a/automation/gitlab-ci/test.yaml
> +++ b/automation/gitlab-ci/test.yaml
> @@ -165,6 +165,22 @@ qemu-smoke-dom0less-arm64-gcc-debug-boot-cpupools:
>      - *arm64-test-needs
>      - alpine-3.12-gcc-debug-arm64-boot-cpupools
>  
> +qemu-xtf-dom0less-arm64-gcc-hyp-xen-version:
> +  extends: .qemu-arm64
> +  script:
> +    - ./automation/scripts/qemu-xtf-dom0less-arm64.sh hyp-xen-version 2>&1 | tee ${LOGFILE}
> +  needs:
> +    - alpine-3.12-gcc-arm64
> +    - qemu-system-aarch64-6.0.0-arm64-export
> +
> +qemu-xtf-dom0less-arm64-gcc-debug-hyp-xen-version:
> +  extends: .qemu-arm64
> +  script:
> +    - ./automation/scripts/qemu-xtf-dom0less-arm64.sh hyp-xen-version 2>&1 | tee ${LOGFILE}
> +  needs:
> +    - alpine-3.12-gcc-debug-arm64
> +    - qemu-system-aarch64-6.0.0-arm64-export
> +
>  qemu-smoke-dom0-arm32-gcc:
>    extends: .qemu-arm32
>    script:
> -- 
> 2.25.1
>