[PATCH RFC 0/3] selftests/nolibc: avoid spurious kernel relinks

[PATCH RFC 0/3] selftests/nolibc: avoid spurious kernel relinks

[Thomas Weißschuh]

Currently the nolibc testsuite embeds the test executable into a kernel
image with CONFIG_INITRAMFS_SOURCE.
This forces a full kernel relink everytime the test executable is
updated.

This relinking step dominates the test cycle.
It is slower than building and running the test in qemu together.

With a bit of Makefile-shuffling the relinking can be avoided.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
Thomas Weißschuh (3):
      kbuild: add toplevel target for usr/gen_init_cpio
      selftests/nolibc: don't embed initramfs into kernel image
      selftests/nolibc: drop target "rerun"

 Makefile                                |  4 +++
 tools/testing/selftests/nolibc/Makefile | 50 +++++++++++++++------------------
 2 files changed, 27 insertions(+), 27 deletions(-)
---
base-commit: 3f79a57865b33f49fdae6655510bd27c8e6610e0
change-id: 20230916-nolibc-initramfs-4fd00eac3256

Best regards,
-- 
Thomas Weißschuh <linux@weissschuh.net>

[PATCH RFC 1/3] kbuild: add toplevel target for usr/gen_init_cpio

[Thomas Weißschuh]

The nolibc testsuite wants to generate an initrams without linking it
into the kernel for which it needs access to gen_init_cpio.
Add a new toplevel target for it so it can be built standalone.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>

---

Kbuild maintainers:

If there is a way that does not require modification of the toplevel
Makefile that would be nice, too.
I followed the example of the "scripts_unifdef" target.
---
 Makefile | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/Makefile b/Makefile
index ceb23eed4dce..1caa4429eceb 100644
--- a/Makefile
+++ b/Makefile
@@ -1812,6 +1812,10 @@ endif
 
 endif # KBUILD_EXTMOD
 
+PHONY += usr_gen_init_cpio
+usr_gen_init_cpio:
+	$(Q)$(MAKE) $(build)=usr usr/gen_init_cpio
+
 # ---------------------------------------------------------------------------
 # Modules
 

-- 
2.42.0

[PATCH RFC 2/3] selftests/nolibc: don't embed initramfs into kernel image

[Thomas Weißschuh]

When the initramfs is embedded into the kernel each rebuild of it will
trigger a full kernel relink and all the expensive postprocessing steps.

Currently nolibc-test and therefore the initramfs are always rebuild,
even without source changes, leading to lots of slow kernel relinks.

Instead of linking the initramfs into the kernel assemble it manually
and pass it explicitly to qemu.
This avoids all of the kernel relinks.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 tools/testing/selftests/nolibc/Makefile | 47 +++++++++++++++++----------------
 1 file changed, 24 insertions(+), 23 deletions(-)

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index 689658f81a19..c62efc62a060 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -131,18 +131,18 @@ REPORT  ?= awk '/\[OK\][\r]*$$/{p++} /\[FAIL\][\r]*$$/{if (!f) printf("\n"); f++
 
 help:
 	@echo "Supported targets under selftests/nolibc:"
-	@echo "  all          call the \"run\" target below"
-	@echo "  help         this help"
-	@echo "  sysroot      create the nolibc sysroot here (uses \$$ARCH)"
-	@echo "  nolibc-test  build the executable (uses \$$CC and \$$CROSS_COMPILE)"
-	@echo "  libc-test    build an executable using the compiler's default libc instead"
-	@echo "  run-user     runs the executable under QEMU (uses \$$XARCH, \$$TEST)"
-	@echo "  initramfs    prepare the initramfs with nolibc-test"
-	@echo "  defconfig    create a fresh new default config (uses \$$XARCH)"
-	@echo "  kernel       (re)build the kernel with the initramfs (uses \$$XARCH)"
-	@echo "  run          runs the kernel in QEMU after building it (uses \$$XARCH, \$$TEST)"
-	@echo "  rerun        runs a previously prebuilt kernel in QEMU (uses \$$XARCH, \$$TEST)"
-	@echo "  clean        clean the sysroot, initramfs, build and output files"
+	@echo "  all             call the \"run\" target below"
+	@echo "  help            this help"
+	@echo "  sysroot         create the nolibc sysroot here (uses \$$ARCH)"
+	@echo "  nolibc-test     build the executable (uses \$$CC and \$$CROSS_COMPILE)"
+	@echo "  libc-test       build an executable using the compiler's default libc instead"
+	@echo "  run-user        runs the executable under QEMU (uses \$$XARCH, \$$TEST)"
+	@echo "  initramfs.cpio  prepare the initramfs with nolibc-test"
+	@echo "  defconfig       create a fresh new default config (uses \$$XARCH)"
+	@echo "  kernel          (re)build the kernel with the initramfs (uses \$$XARCH)"
+	@echo "  run             runs the kernel in QEMU after building it (uses \$$XARCH, \$$TEST)"
+	@echo "  rerun           runs a previously prebuilt kernel in QEMU (uses \$$XARCH, \$$TEST)"
+	@echo "  clean           clean the sysroot, initramfs, build and output files"
 	@echo ""
 	@echo "The output file is \"run.out\". Test ranges may be passed using \$$TEST."
 	@echo ""
@@ -195,25 +195,26 @@ run-user: nolibc-test
 	$(Q)qemu-$(QEMU_ARCH) ./nolibc-test > "$(CURDIR)/run.out" || :
 	$(Q)$(REPORT) $(CURDIR)/run.out
 
-initramfs: nolibc-test
-	$(QUIET_MKDIR)mkdir -p initramfs
-	$(call QUIET_INSTALL, initramfs/init)
-	$(Q)cp nolibc-test initramfs/init
+$(srctree)/usr/gen_init_cpio:
+	$(Q)$(MAKE) -C $(srctree) usr_gen_init_cpio
+
+initramfs.cpio: $(srctree)/usr/gen_init_cpio nolibc-test
+	$(QUIET_GEN)echo 'file /init nolibc-test 755 0 0' | $(srctree)/usr/gen_init_cpio - > initramfs.cpio
 
 defconfig:
 	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) mrproper $(DEFCONFIG) prepare
 
-kernel: initramfs
-	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME) CONFIG_INITRAMFS_SOURCE=$(CURDIR)/initramfs
+kernel:
+	$(Q)$(MAKE) -C $(srctree) ARCH=$(ARCH) CC=$(CC) CROSS_COMPILE=$(CROSS_COMPILE) $(IMAGE_NAME)
 
 # run the tests after building the kernel
-run: kernel
-	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
+run: kernel initramfs.cpio
+	qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -initrd initramfs.cpio -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
 	$(Q)$(REPORT) $(CURDIR)/run.out
 
 # re-run the tests from an existing kernel
 rerun:
-	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
+	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -initrd initramfs.cpio -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
 	$(Q)$(REPORT) $(CURDIR)/run.out
 
 # report with existing test log
@@ -227,8 +228,8 @@ clean:
 	$(Q)rm -f nolibc-test
 	$(call QUIET_CLEAN, libc-test)
 	$(Q)rm -f libc-test
-	$(call QUIET_CLEAN, initramfs)
-	$(Q)rm -rf initramfs
+	$(call QUIET_CLEAN, initramfs.cpio)
+	$(Q)rm -rf initramfs.cpio
 	$(call QUIET_CLEAN, run.out)
 	$(Q)rm -rf run.out
 

-- 
2.42.0

[PATCH RFC 3/3] selftests/nolibc: drop target "rerun"

[Thomas Weißschuh]

Now that the target "run" does not relink the kernel all the time
anymore it is much faster and we don't need "rerun" anymore.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>

---

Willy:

This is an optional commit, "rerun" is still a bit faster than "run".
Also maybe it should be squashed into the previous commit.
---
 tools/testing/selftests/nolibc/Makefile | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile
index c62efc62a060..ca9ec9c6f2f4 100644
--- a/tools/testing/selftests/nolibc/Makefile
+++ b/tools/testing/selftests/nolibc/Makefile
@@ -212,11 +212,6 @@ run: kernel initramfs.cpio
 	qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -initrd initramfs.cpio -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
 	$(Q)$(REPORT) $(CURDIR)/run.out
 
-# re-run the tests from an existing kernel
-rerun:
-	$(Q)qemu-system-$(QEMU_ARCH) -display none -no-reboot -kernel "$(srctree)/$(IMAGE)" -initrd initramfs.cpio -serial stdio $(QEMU_ARGS) > "$(CURDIR)/run.out"
-	$(Q)$(REPORT) $(CURDIR)/run.out
-
 # report with existing test log
 report:
 	$(Q)$(REPORT) $(CURDIR)/run.out

-- 
2.42.0

Re: [PATCH RFC 1/3] kbuild: add toplevel target for usr/gen_init_cpio

[Masahiro Yamada]

On Sat, Sep 16, 2023 at 4:13 PM Thomas Weißschuh <linux@weissschuh.net> wrote:
>
> The nolibc testsuite wants to generate an initrams without linking it
> into the kernel for which it needs access to gen_init_cpio.
> Add a new toplevel target for it so it can be built standalone.
>
> Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
>
> ---
>
> Kbuild maintainers:
>
> If there is a way that does not require modification of the toplevel
> Makefile that would be nice, too.
> I followed the example of the "scripts_unifdef" target.


This worked for me.


masahiro@zoe:~/ref/linux$ make -s mrproper
masahiro@zoe:~/ref/linux$ make -s defconfig
masahiro@zoe:~/ref/linux$ make run-command KBUILD_RUN_COMMAND='$(MAKE)
$(build)=usr usr/gen_init_cpio'
  HOSTCC  usr/gen_init_cpio








> ---
>  Makefile | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/Makefile b/Makefile
> index ceb23eed4dce..1caa4429eceb 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1812,6 +1812,10 @@ endif
>
>  endif # KBUILD_EXTMOD
>
> +PHONY += usr_gen_init_cpio
> +usr_gen_init_cpio:
> +       $(Q)$(MAKE) $(build)=usr usr/gen_init_cpio
> +
>  # ---------------------------------------------------------------------------
>  # Modules
>
>
> --
> 2.42.0
>


-- 
Best Regards
Masahiro Yamada

Re: [PATCH RFC 0/3] selftests/nolibc: avoid spurious kernel relinks

[Willy Tarreau]

On Sat, Sep 16, 2023 at 09:13:26AM +0200, Thomas Weißschuh wrote:
> Currently the nolibc testsuite embeds the test executable into a kernel
> image with CONFIG_INITRAMFS_SOURCE.
> This forces a full kernel relink everytime the test executable is
> updated.
> 
> This relinking step dominates the test cycle.
> It is slower than building and running the test in qemu together.
> 
> With a bit of Makefile-shuffling the relinking can be avoided.

That's pretty nice as indeed it still takes a while to relink it into
the kernel. I agree that for running nolibc-test in qemu we don't need
a unified image. However I've seldom used it on real hardware and I
find it significantly more convenient to use as a single image in this
case. Maybe we should just rename targets so that everything qemu-related
just uses two distinct images while a "unified-image" (or anything else)
still assembles the image into the kernel (BTW the help on the "kernel"
target still mentions initramfs).

Note that we don't need to modify anything in the build system to create
an initrd, I usually make them by hand using "cpio -o -H newc", we don't
need anything else here.

Regarding rerun, I'd rather keep it for the sole reason that I've used
it to check for randomly failing errors (typically the timing-based
ones). It's convenient to run the same image 100 times if needed. I'm
not strongly attached to it, but if "make run" is slower, then we can
keep it. However if you really want to drop it, it also needs to be
dropped from the help message ;-)

Cheers,
Willy

Re: [PATCH RFC 0/3] selftests/nolibc: avoid spurious kernel relinks

[Thomas Weißschuh]

On 2023-09-17 05:22:19+0200, Willy Tarreau wrote:
> On Sat, Sep 16, 2023 at 09:13:26AM +0200, Thomas Weißschuh wrote:
> > Currently the nolibc testsuite embeds the test executable into a kernel
> > image with CONFIG_INITRAMFS_SOURCE.
> > This forces a full kernel relink everytime the test executable is
> > updated.
> > 
> > This relinking step dominates the test cycle.
> > It is slower than building and running the test in qemu together.
> > 
> > With a bit of Makefile-shuffling the relinking can be avoided.
> 
> That's pretty nice as indeed it still takes a while to relink it into
> the kernel. I agree that for running nolibc-test in qemu we don't need
> a unified image. However I've seldom used it on real hardware and I
> find it significantly more convenient to use as a single image in this
> case. Maybe we should just rename targets so that everything qemu-related
> just uses two distinct images while a "unified-image" (or anything else)
> still assembles the image into the kernel (BTW the help on the "kernel"
> target still mentions initramfs).

Sounds good, "unified-image" is a bit close to "unified kernel image"
(UKI) which is similar but different.

What about kernel-standalone?

> Note that we don't need to modify anything in the build system to create
> an initrd, I usually make them by hand using "cpio -o -H newc", we don't
> need anything else here.

I'd like to keep the build self-contained. But actually the kernel build
will always build a minimal initramfs if CONFIG_BLK_DEV_INITRD is set.
This config is needed for the nolibc testsuite in any case.

The fact that the initrd is always build means that usr/gen_init_cpio
is also always build.
So we can add "kernel" as a prerequisite to "initramfs.cpio" and
everything should work out without any buildsystem modifications.

> Regarding rerun, I'd rather keep it for the sole reason that I've used
> it to check for randomly failing errors (typically the timing-based
> ones). It's convenient to run the same image 100 times if needed. I'm
> not strongly attached to it, but if "make run" is slower, then we can
> keep it. However if you really want to drop it, it also needs to be
> dropped from the help message ;-)

Fine for me, let's keep it :-)