[PATCH 0/3] selftests/nolibc: various build improvements

[PATCH 0/3] selftests/nolibc: various build improvements

[Thomas Weißschuh]

With the out-of-tree builds it's possible do incremental tests fairly fast:

time ./run-tests.sh 
i386:          162 test(s): 162 passed,   0 skipped,   0 failed => status: success
x86_64:        162 test(s): 162 passed,   0 skipped,   0 failed => status: success
arm64:         162 test(s): 162 passed,   0 skipped,   0 failed => status: success
arm:           162 test(s): 162 passed,   0 skipped,   0 failed => status: success
mips:          162 test(s): 161 passed,   1 skipped,   0 failed => status: warning
ppc:           162 test(s): 162 passed,   0 skipped,   0 failed => status: success
ppc64:         162 test(s): 162 passed,   0 skipped,   0 failed => status: success
ppc64le:       162 test(s): 162 passed,   0 skipped,   0 failed => status: success
riscv:         162 test(s): 162 passed,   0 skipped,   0 failed => status: success
s390:          162 test(s): 161 passed,   1 skipped,   0 failed => status: warning
loongarch:     162 test(s): 161 passed,   1 skipped,   0 failed => status: warning

real	1m56.226s
user	2m42.457s
sys	0m57.979s

This is with an incremental kernel rebuild and testrun inside qemu.

---
Changes in v2:
- Drop already applied qemu-system-ppc64le patch
- Drop config generation patch
- Add Co-developed-by for out-of-tree patch
- Link to v1: https://lore.kernel.org/lkml/20231010-nolibc-out-of-tree-v1-0-b6a263859596@weissschuh.net/

---
Thomas Weißschuh (3):
      selftests/nolibc: use EFI -bios for LoongArch qemu
      selftests/nolibc: anchor paths in $(srcdir) if possible
      selftests/nolibc: support out-of-tree builds

 tools/testing/selftests/nolibc/Makefile | 31 ++++++++++++++++++++++++-------
 1 file changed, 24 insertions(+), 7 deletions(-)
---
base-commit: 5a6a09e97199d6600d31383055f9d43fbbcbe86f
change-id: 20231010-nolibc-out-of-tree-b6684c6cf0e3

Best regards,
-- 
Thomas Weißschuh <linux@weissschuh.net>

[PATCH 1/3] selftests/nolibc: use EFI -bios for LoongArch qemu

[Thomas Weißschuh]

qemu for LoongArch does not work properly with direct kernel boot.
The kernel will panic during initialization and hang without any output.

When booting in EFI mode everything work correctly.

While users most likely don't have the LoongArch EFI binary installed at
least an explicit error about 'file not found' is better than a hanging
test without output that can never succeed.

Link: https://lore.kernel.org/loongarch/1738d60a-df3a-4102-b1da-d16a29b6e06a@t-8ch.de/
Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 tools/testing/selftests/nolibc/Makefile | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index a0fc07253baf..eb258ae1d948 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -88,6 +88,13 @@ QEMU_ARCH_s390       = s390x
 QEMU_ARCH_loongarch  = loongarch64
 QEMU_ARCH            = $(QEMU_ARCH_$(XARCH))
 
+QEMU_BIOS_DIR = /usr/share/edk2/
+QEMU_BIOS_loongarch = $(QEMU_BIOS_DIR)/loongarch64/OVMF_CODE.fd
+
+ifneq ($(QEMU_BIOS_$(XARCH)),)
+QEMU_ARGS_BIOS = -bios $(QEMU_BIOS_$(XARCH))
+endif
+
 # QEMU_ARGS : some arch-specific args to pass to qemu
 QEMU_ARGS_i386       = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_x86_64     = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)"
@@ -101,7 +108,7 @@ QEMU_ARGS_ppc64le    = -M powernv -append "console=hvc0 panic=-1 $(TEST:%=NOLIBC
 QEMU_ARGS_riscv      = -M virt -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_s390       = -M s390-ccw-virtio -m 1G -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
 QEMU_ARGS_loongarch  = -M virt -append "console=ttyS0,115200 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
-QEMU_ARGS            = $(QEMU_ARGS_$(XARCH)) $(QEMU_ARGS_EXTRA)
+QEMU_ARGS            = $(QEMU_ARGS_$(XARCH)) $(QEMU_ARGS_BIOS) $(QEMU_ARGS_EXTRA)
 
 # OUTPUT is only set when run from the main makefile, otherwise
 # it defaults to this nolibc directory.

-- 
2.42.0

[PATCH 2/3] selftests/nolibc: anchor paths in $(srcdir) if possible

[Thomas Weißschuh]

It is easier to recognize paths from their well-known location in the
source tree than having to resolve the relative path in ones head.

Acked-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 tools/testing/selftests/nolibc/Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index eb258ae1d948..b1129be98376 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -174,7 +174,7 @@ sysroot: sysroot/$(ARCH)/include
 sysroot/$(ARCH)/include:
 	$(Q)rm -rf sysroot/$(ARCH) sysroot/sysroot
 	$(QUIET_MKDIR)mkdir -p sysroot
-	$(Q)$(MAKE) -C ../../../include/nolibc ARCH=$(ARCH) OUTPUT=$(CURDIR)/sysroot/ headers_standalone
+	$(Q)$(MAKE) -C $(srctree)/tools/include/nolibc ARCH=$(ARCH) OUTPUT=$(CURDIR)/sysroot/ headers_standalone
 	$(Q)mv sysroot/sysroot sysroot/$(ARCH)
 
 ifneq ($(NOLIBC_SYSROOT),0)
@@ -184,7 +184,7 @@ nolibc-test: nolibc-test.c nolibc-test-linkage.c sysroot/$(ARCH)/include
 else
 nolibc-test: nolibc-test.c nolibc-test-linkage.c
 	$(QUIET_CC)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ \
-	  -nostdlib -static -include ../../../include/nolibc/nolibc.h nolibc-test.c nolibc-test-linkage.c -lgcc
+	  -nostdlib -static -include $(srctree)/tools/include/nolibc/nolibc.h nolibc-test.c nolibc-test-linkage.c -lgcc
 endif
 
 libc-test: nolibc-test.c nolibc-test-linkage.c

-- 
2.42.0

[PATCH 3/3] selftests/nolibc: support out-of-tree builds

[Thomas Weißschuh]

Out of tree builds are much more convenient when building for multiple
architectures or configurations in parallel.

Only absolute O= parameters are supported as Makefile.include will
always resolve relative paths in relation to $(srctree) instead of the
current directory.

Add a call to "make outputmakefile" to verify that the sourcetree is
clean.

This is based on Zhangjins out-of-tree patch.
It extends that work for get_init_cpio support and also drops relative
O= specifications explicitly.

Link: https://lore.kernel.org/lkml/06d96bd81fe812a9718098a383678ad3beba98b1.1691215074.git.falcon@tinylab.org/
Co-developed-by: Zhangjin Wu <falcon@tinylab.org>
Signed-off-by: Zhangjin Wu <falcon@tinylab.org>
Acked-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 tools/testing/selftests/nolibc/Makefile | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index b1129be98376..c5df1617cbc3 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -1,9 +1,16 @@
 # SPDX-License-Identifier: GPL-2.0
 # Makefile for nolibc tests
 include ../../../scripts/Makefile.include
+include ../../../scripts/utilities.mak
 # We need this for the "cc-option" macro.
 include ../../../build/Build.include
 
+ifneq ($(O),)
+ifneq ($(call is-absolute,$(O)),y)
+$(error Only absolute O= parameters are supported)
+endif
+endif
+
 # we're in ".../tools/testing/selftests/nolibc"
 ifeq ($(srctree),)
 srctree := $(patsubst %/tools/testing/selftests/,%,$(dir $(CURDIR)))
@@ -14,6 +21,8 @@ include $(srctree)/scripts/subarch.include
 ARCH = $(SUBARCH)
 endif
 
+objtree ?= $(srctree)
+
 # XARCH extends the kernel's ARCH with a few variants of the same
 # architecture that only differ by the configuration, the toolchain
 # and the Qemu program used. It is copied as-is into ARCH except for
@@ -52,7 +61,7 @@ IMAGE_ppc64le    = arch/powerpc/boot/zImage
 IMAGE_riscv      = arch/riscv/boot/Image
 IMAGE_s390       = arch/s390/boot/bzImage
 IMAGE_loongarch  = arch/loongarch/boot/vmlinuz.efi
-IMAGE            = $(IMAGE_$(XARCH))
+IMAGE            = $(objtree)/$(IMAGE_$(XARCH))
 IMAGE_NAME       = $(notdir $(IMAGE))
 
 # default kernel configurations that appear to be usable
@@ -174,6 +183,7 @@ sysroot: sysroot/$(ARCH)/include
 sysroot/$(ARCH)/include:
 	$(Q)rm -rf sysroot/$(ARCH) sysroot/sysroot
 	$(QUIET_MKDIR)mkdir -p sysroot
+	$(Q)$(MAKE) -C $(srctree) outputmakefile
 	$(Q)$(MAKE) -C $(srctree)/tools/include/nolibc ARCH=$(ARCH) OUTPUT=$(CURDIR)/sysroot/ headers_standalone
 	$(Q)mv sysroot/sysroot sysroot/$(ARCH)
 
@@ -206,7 +216,7 @@ run-user: nolibc-test
 	$(Q)$(REPORT) $(CURDIR)/run.out
 
 initramfs.cpio: kernel nolibc-test
-	$(QUIET_GEN)echo 'file /init nolibc-test 755 0 0' | $(srctree)/usr/gen_init_cpio - > initramfs.cpio
+	$(QUIET_GEN)echo 'file /init nolibc-test 755 0 0' | $(objtree)/usr/gen_init_cpio - > initramfs.cpio
 
 initramfs: nolibc-test
 	$(QUIET_MKDIR)mkdir -p initramfs
@@ -224,12 +234,12 @@ kernel-standalone: initramfs
 
 # run the tests after building the kernel
 run: kernel initramfs.cpio
-	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -initrd initramfs.cpio -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
+	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(IMAGE)" -initrd initramfs.cpio -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
 	$(Q)$(REPORT) $(CURDIR)/run.out
 
 # re-run the tests from an existing kernel
 rerun:
-	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -initrd initramfs.cpio -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
+	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(IMAGE)" -initrd initramfs.cpio -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
 	$(Q)$(REPORT) $(CURDIR)/run.out
 
 # report with existing test log

-- 
2.42.0

Re: [PATCH 1/3] selftests/nolibc: use EFI -bios for LoongArch qemu

[Willy Tarreau]

On Tue, Oct 31, 2023 at 09:36:58PM +0100, Thomas Weißschuh wrote:
> qemu for LoongArch does not work properly with direct kernel boot.
> The kernel will panic during initialization and hang without any output.
> 
> When booting in EFI mode everything work correctly.
> 
> While users most likely don't have the LoongArch EFI binary installed at
> least an explicit error about 'file not found' is better than a hanging
> test without output that can never succeed.
> 
> Link: https://lore.kernel.org/loongarch/1738d60a-df3a-4102-b1da-d16a29b6e06a@t-8ch.de/
> Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
> ---
>  tools/testing/selftests/nolibc/Makefile | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
> index a0fc07253baf..eb258ae1d948 100644
> --- a/tools/testing/selftests/nolibc/Makefile
> +++ b/tools/testing/selftests/nolibc/Makefile
> @@ -88,6 +88,13 @@ QEMU_ARCH_s390       = s390x
>  QEMU_ARCH_loongarch  = loongarch64
>  QEMU_ARCH            = $(QEMU_ARCH_$(XARCH))
>  
> +QEMU_BIOS_DIR = /usr/share/edk2/
> +QEMU_BIOS_loongarch = $(QEMU_BIOS_DIR)/loongarch64/OVMF_CODE.fd
> +
> +ifneq ($(QEMU_BIOS_$(XARCH)),)
> +QEMU_ARGS_BIOS = -bios $(QEMU_BIOS_$(XARCH))
> +endif
> +
>  # QEMU_ARGS : some arch-specific args to pass to qemu
>  QEMU_ARGS_i386       = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)"
>  QEMU_ARGS_x86_64     = -M pc -append "console=ttyS0,9600 i8042.noaux panic=-1 $(TEST:%=NOLIBC_TEST=%)"
> @@ -101,7 +108,7 @@ QEMU_ARGS_ppc64le    = -M powernv -append "console=hvc0 panic=-1 $(TEST:%=NOLIBC
>  QEMU_ARGS_riscv      = -M virt -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
>  QEMU_ARGS_s390       = -M s390-ccw-virtio -m 1G -append "console=ttyS0 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
>  QEMU_ARGS_loongarch  = -M virt -append "console=ttyS0,115200 panic=-1 $(TEST:%=NOLIBC_TEST=%)"
> -QEMU_ARGS            = $(QEMU_ARGS_$(XARCH)) $(QEMU_ARGS_EXTRA)
> +QEMU_ARGS            = $(QEMU_ARGS_$(XARCH)) $(QEMU_ARGS_BIOS) $(QEMU_ARGS_EXTRA)
>  
>  # OUTPUT is only set when run from the main makefile, otherwise
>  # it defaults to this nolibc directory.

Looks much nicer this way, thanks ;-)

Acked-by: Willy Tarreau <w@1wt.eu>

Willy

Re: [PATCH 0/3] selftests/nolibc: various build improvements

[Willy Tarreau]

On Tue, Oct 31, 2023 at 09:36:57PM +0100, Thomas Weißschuh wrote:
> With the out-of-tree builds it's possible do incremental tests fairly fast:
> 
> time ./run-tests.sh 
> i386:          162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> x86_64:        162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> arm64:         162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> arm:           162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> mips:          162 test(s): 161 passed,   1 skipped,   0 failed => status: warning
> ppc:           162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> ppc64:         162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> ppc64le:       162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> riscv:         162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> s390:          162 test(s): 161 passed,   1 skipped,   0 failed => status: warning
> loongarch:     162 test(s): 161 passed,   1 skipped,   0 failed => status: warning
> 
> real	1m56.226s
> user	2m42.457s
> sys	0m57.979s
> 
> This is with an incremental kernel rebuild and testrun inside qemu.
> 
> ---
> Changes in v2:
> - Drop already applied qemu-system-ppc64le patch
> - Drop config generation patch
> - Add Co-developed-by for out-of-tree patch
> - Link to v1: https://lore.kernel.org/lkml/20231010-nolibc-out-of-tree-v1-0-b6a263859596@weissschuh.net/

Thanks Thomas for these, feel free to merge them!
Willy

Re: [PATCH 0/3] selftests/nolibc: various build improvements

[Thomas Weißschuh]

On 2023-11-02 09:50:38+0100, Willy Tarreau wrote:
> On Tue, Oct 31, 2023 at 09:36:57PM +0100, Thomas Weißschuh wrote:
> > With the out-of-tree builds it's possible do incremental tests fairly fast:
> > 
> > time ./run-tests.sh 
> > i386:          162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> > x86_64:        162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> > arm64:         162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> > arm:           162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> > mips:          162 test(s): 161 passed,   1 skipped,   0 failed => status: warning
> > ppc:           162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> > ppc64:         162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> > ppc64le:       162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> > riscv:         162 test(s): 162 passed,   0 skipped,   0 failed => status: success
> > s390:          162 test(s): 161 passed,   1 skipped,   0 failed => status: warning
> > loongarch:     162 test(s): 161 passed,   1 skipped,   0 failed => status: warning
> > 
> > real	1m56.226s
> > user	2m42.457s
> > sys	0m57.979s
> > 
> > This is with an incremental kernel rebuild and testrun inside qemu.
> > 
> > ---
> > Changes in v2:
> > - Drop already applied qemu-system-ppc64le patch
> > - Drop config generation patch
> > - Add Co-developed-by for out-of-tree patch
> > - Link to v1: https://lore.kernel.org/lkml/20231010-nolibc-out-of-tree-v1-0-b6a263859596@weissschuh.net/
> 
> Thanks Thomas for these, feel free to merge them!

Thanks for the review!

Applied and pushed to nolibc/next.

Thomas