All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 1/2] ARM: fix nm error message when GNU Make >= 4.4 is used
@ 2023-01-07 21:20 ` Masahiro Yamada
  0 siblings, 0 replies; 5+ messages in thread
From: Masahiro Yamada @ 2023-01-07 21:20 UTC (permalink / raw)
  To: linux-kbuild
  Cc: linux-kernel, Nathan Chancellor, Masahiro Yamada, Arnd Bergmann,
	Nick Desaulniers, Nicolas Schier, Russell King (Oracle),
	Russell King, Seung-Woo Kim, Xin Li, linux-arm-kernel

Nathan Chancellor reports an error message from $(NM) if GNU Make 4.4
is used to build the ARM decompressor.

  $ make-4.4 ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- O=build defconfig all
    [snip]
    LD      vmlinux
    NM      System.map
    SORTTAB vmlinux
    OBJCOPY arch/arm/boot/Image
    Kernel: arch/arm/boot/Image is ready
  arm-linux-gnueabi-nm: 'arch/arm/boot/compressed/../../../../vmlinux': No such file
  /bin/sh: 1: arithmetic expression: expecting primary: " "
    LDS     arch/arm/boot/compressed/vmlinux.lds
    AS      arch/arm/boot/compressed/head.o
    GZIP    arch/arm/boot/compressed/piggy_data
    AS      arch/arm/boot/compressed/piggy.o
    CC      arch/arm/boot/compressed/misc.o

This occurs with GNU Make commit 98da874c4303 ("[SV 10593] Export
variables to $(shell ...) commands"), and the O= option is needed to
reproduce it. The generated zImage is correct despite the error message.

As the commit description of 98da874c4303 [1] says, exported variables
are passed down to $(shell ) functions, which means exported recursive
variables might be expanded earlier than before, in the parse stage.

The following test code demonstrates the change for GNU Make 4.4.

[Test Makefile]

  $(shell echo hello > foo)
  export foo = $(shell cat bar/../foo)
  $(shell mkdir bar)

  all:
          @echo $(foo)

[GNU Make 4.3]

  $ rm -rf bar; make-4.3
  hello

[GNU Make 4.4]

  $ rm -rf bar; make-4.4
  cat: bar/../foo: No such file or directory
  hello

The 'foo' is a resursively expanded (or lazily expanded) variable.

GNU Make 4.3 expands it just before running the recipe '@echo $(foo)',
at this point, the directory 'bar' exists.

GNU Make 4.4 expands it to evaluate $(shell mkdir bar) because 'foo' is
exported. At this point, the directory 'bar' does not exit yet. The cat
command cannot resolve the bar/../foo path, hence the error message.

Let's get back to the kernel Makefile.

'KBSS_SZ' in arch/arm/boot/compressed/Makefile is a recursive variable,
which is referenced by 'LDFLAGS_vmlinux', which is also a recursive
variable.

GNU Make 4.3 expands 'KBSS_SZ' just before running the recipes. Before
that, $(shell mkdir -p $(obj-dirs)) in scripts/Makefile.build creates
the output directory, arch/arm/boot/compressed.

GNU Make 4.4 expands 'KBSS_SZ' in the parse stage because LDFLAGS_vmlinux
was (accidentally) exported by commit 5d4aeffbf709 ("kbuild: rebuild
.vmlinux.export.o when its prerequisite is updated"). $(NM) cannot
resolve the path arch/arm/boot/compressed/../../../../vmlinux.

I admit this is a bug caused by 5d4aeffbf709 (I will fix it in the next
commit), but do not see any good reason in writing the vmlinux path in
such an indirect way. Just say 'vmlinux'.

[1]: https://git.savannah.gnu.org/cgit/make.git/commit/?id=98da874c43035a490cdca81331724f233a3d0c9a

Link: https://lore.kernel.org/all/Y7i8+EjwdnhHtlrr@dev-arch.thelio-3990X/
Fixes: 5d4aeffbf709 ("kbuild: rebuild .vmlinux.export.o when its prerequisite is updated")
Reported-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 arch/arm/boot/compressed/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
index 2ef651a78fa2..726ecabcef09 100644
--- a/arch/arm/boot/compressed/Makefile
+++ b/arch/arm/boot/compressed/Makefile
@@ -107,7 +107,7 @@ ccflags-remove-$(CONFIG_FUNCTION_TRACER) += -pg
 asflags-y := -DZIMAGE
 
 # Supply kernel BSS size to the decompressor via a linker symbol.
-KBSS_SZ = $(shell echo $$(($$($(NM) $(obj)/../../../../vmlinux | \
+KBSS_SZ = $(shell echo $$(($$($(NM) vmlinux | \
 		sed -n -e 's/^\([^ ]*\) [ABD] __bss_start$$/-0x\1/p' \
 		       -e 's/^\([^ ]*\) [ABD] __bss_stop$$/+0x\1/p') )) )
 LDFLAGS_vmlinux = --defsym _kernel_bss_size=$(KBSS_SZ)
-- 
2.34.1


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 1/2] ARM: fix nm error message when GNU Make >= 4.4 is used
@ 2023-01-07 21:20 ` Masahiro Yamada
  0 siblings, 0 replies; 5+ messages in thread
From: Masahiro Yamada @ 2023-01-07 21:20 UTC (permalink / raw)
  To: linux-kbuild
  Cc: linux-kernel, Nathan Chancellor, Masahiro Yamada, Arnd Bergmann,
	Nick Desaulniers, Nicolas Schier, Russell King (Oracle),
	Russell King, Seung-Woo Kim, Xin Li, linux-arm-kernel

Nathan Chancellor reports an error message from $(NM) if GNU Make 4.4
is used to build the ARM decompressor.

  $ make-4.4 ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- O=build defconfig all
    [snip]
    LD      vmlinux
    NM      System.map
    SORTTAB vmlinux
    OBJCOPY arch/arm/boot/Image
    Kernel: arch/arm/boot/Image is ready
  arm-linux-gnueabi-nm: 'arch/arm/boot/compressed/../../../../vmlinux': No such file
  /bin/sh: 1: arithmetic expression: expecting primary: " "
    LDS     arch/arm/boot/compressed/vmlinux.lds
    AS      arch/arm/boot/compressed/head.o
    GZIP    arch/arm/boot/compressed/piggy_data
    AS      arch/arm/boot/compressed/piggy.o
    CC      arch/arm/boot/compressed/misc.o

This occurs with GNU Make commit 98da874c4303 ("[SV 10593] Export
variables to $(shell ...) commands"), and the O= option is needed to
reproduce it. The generated zImage is correct despite the error message.

As the commit description of 98da874c4303 [1] says, exported variables
are passed down to $(shell ) functions, which means exported recursive
variables might be expanded earlier than before, in the parse stage.

The following test code demonstrates the change for GNU Make 4.4.

[Test Makefile]

  $(shell echo hello > foo)
  export foo = $(shell cat bar/../foo)
  $(shell mkdir bar)

  all:
          @echo $(foo)

[GNU Make 4.3]

  $ rm -rf bar; make-4.3
  hello

[GNU Make 4.4]

  $ rm -rf bar; make-4.4
  cat: bar/../foo: No such file or directory
  hello

The 'foo' is a resursively expanded (or lazily expanded) variable.

GNU Make 4.3 expands it just before running the recipe '@echo $(foo)',
at this point, the directory 'bar' exists.

GNU Make 4.4 expands it to evaluate $(shell mkdir bar) because 'foo' is
exported. At this point, the directory 'bar' does not exit yet. The cat
command cannot resolve the bar/../foo path, hence the error message.

Let's get back to the kernel Makefile.

'KBSS_SZ' in arch/arm/boot/compressed/Makefile is a recursive variable,
which is referenced by 'LDFLAGS_vmlinux', which is also a recursive
variable.

GNU Make 4.3 expands 'KBSS_SZ' just before running the recipes. Before
that, $(shell mkdir -p $(obj-dirs)) in scripts/Makefile.build creates
the output directory, arch/arm/boot/compressed.

GNU Make 4.4 expands 'KBSS_SZ' in the parse stage because LDFLAGS_vmlinux
was (accidentally) exported by commit 5d4aeffbf709 ("kbuild: rebuild
.vmlinux.export.o when its prerequisite is updated"). $(NM) cannot
resolve the path arch/arm/boot/compressed/../../../../vmlinux.

I admit this is a bug caused by 5d4aeffbf709 (I will fix it in the next
commit), but do not see any good reason in writing the vmlinux path in
such an indirect way. Just say 'vmlinux'.

[1]: https://git.savannah.gnu.org/cgit/make.git/commit/?id=98da874c43035a490cdca81331724f233a3d0c9a

Link: https://lore.kernel.org/all/Y7i8+EjwdnhHtlrr@dev-arch.thelio-3990X/
Fixes: 5d4aeffbf709 ("kbuild: rebuild .vmlinux.export.o when its prerequisite is updated")
Reported-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 arch/arm/boot/compressed/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm/boot/compressed/Makefile b/arch/arm/boot/compressed/Makefile
index 2ef651a78fa2..726ecabcef09 100644
--- a/arch/arm/boot/compressed/Makefile
+++ b/arch/arm/boot/compressed/Makefile
@@ -107,7 +107,7 @@ ccflags-remove-$(CONFIG_FUNCTION_TRACER) += -pg
 asflags-y := -DZIMAGE
 
 # Supply kernel BSS size to the decompressor via a linker symbol.
-KBSS_SZ = $(shell echo $$(($$($(NM) $(obj)/../../../../vmlinux | \
+KBSS_SZ = $(shell echo $$(($$($(NM) vmlinux | \
 		sed -n -e 's/^\([^ ]*\) [ABD] __bss_start$$/-0x\1/p' \
 		       -e 's/^\([^ ]*\) [ABD] __bss_stop$$/+0x\1/p') )) )
 LDFLAGS_vmlinux = --defsym _kernel_bss_size=$(KBSS_SZ)
-- 
2.34.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH 2/2] kbuild: do not export LDFLAGS_vmlinux to decompressor Makefiles
  2023-01-07 21:20 ` Masahiro Yamada
  (?)
@ 2023-01-07 21:20 ` Masahiro Yamada
  -1 siblings, 0 replies; 5+ messages in thread
From: Masahiro Yamada @ 2023-01-07 21:20 UTC (permalink / raw)
  To: linux-kbuild
  Cc: linux-kernel, Nathan Chancellor, Masahiro Yamada,
	Nick Desaulniers, Nicolas Schier

Commit 3ec8a5b33dea ("kbuild: do not export LDFLAGS_vmlinux") unexported
LDFLAGS_vmlinux because LDFLAGS_vmlinux in the top Makefile and the one
in arch/*/boot/compressed/Makefile are unrelated to each other. It is
just the decompressors, arch/*/boot/compressed/vmlinux just happen to
have the same base name.

Commit 5d4aeffbf709 ("kbuild: rebuild .vmlinux.export.o when its
prerequisite is updated") accidentally exported it again.

The issue reported so far is $(NM) error message when building the
ARM decompressor with GNU Make >= 4.4. This was fixed by refactoring
arch/arm/boot/compressed/Makefile.

The error message fixed by 3ec8a5b33dea is very likely to happen in the
current cleaning code, but it would still be possible on systems with a
huge number of CPUs.

Anyway, LDFLAGS_vmlinux should not be exported to the decompressor
Makefiles.

Fixes: 5d4aeffbf709 ("kbuild: rebuild .vmlinux.export.o when its prerequisite is updated")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 Makefile | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index dfba294ae790..146cdf195e56 100644
--- a/Makefile
+++ b/Makefile
@@ -549,7 +549,7 @@ LDFLAGS_MODULE  =
 CFLAGS_KERNEL	=
 RUSTFLAGS_KERNEL =
 AFLAGS_KERNEL	=
-export LDFLAGS_vmlinux =
+LDFLAGS_vmlinux =
 
 # Use USERINCLUDE when you must reference the UAPI directories only.
 USERINCLUDE    := \
@@ -1248,6 +1248,11 @@ vmlinux.o modules.builtin.modinfo modules.builtin: vmlinux_o
 	@:
 
 PHONY += vmlinux
+# LDFLAGS_vmlinux flags from the top Makefile and arch/*/Makefile should be
+# exported for building the top vmlinux, but not for building decompressors.
+# They have the same base name (arch/*/boot/compressed/vmlinux), but the
+# decompressor Makefiles should not inherit LDFLAGS_vmlinux.
+vmlinux: export LDFLAGS_vmlinux := $(LDFLAGS_vmlinux)
 vmlinux: vmlinux.o $(KBUILD_LDS) modpost
 	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.vmlinux
 
-- 
2.34.1


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: [PATCH 1/2] ARM: fix nm error message when GNU Make >= 4.4 is used
  2023-01-07 21:20 ` Masahiro Yamada
@ 2023-01-08  9:50   ` Masahiro Yamada
  -1 siblings, 0 replies; 5+ messages in thread
From: Masahiro Yamada @ 2023-01-08  9:50 UTC (permalink / raw)
  To: linux-kbuild
  Cc: linux-kernel, Nathan Chancellor, Arnd Bergmann, Nick Desaulniers,
	Nicolas Schier, Russell King (Oracle),
	Russell King, Seung-Woo Kim, Xin Li, linux-arm-kernel

On Sun, Jan 8, 2023 at 6:20 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> Nathan Chancellor reports an error message from $(NM) if GNU Make 4.4
> is used to build the ARM decompressor.
>
>   $ make-4.4 ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- O=build defconfig all
>     [snip]
>     LD      vmlinux
>     NM      System.map
>     SORTTAB vmlinux
>     OBJCOPY arch/arm/boot/Image
>     Kernel: arch/arm/boot/Image is ready
>   arm-linux-gnueabi-nm: 'arch/arm/boot/compressed/../../../../vmlinux': No such file
>   /bin/sh: 1: arithmetic expression: expecting primary: " "
>     LDS     arch/arm/boot/compressed/vmlinux.lds
>     AS      arch/arm/boot/compressed/head.o
>     GZIP    arch/arm/boot/compressed/piggy_data
>     AS      arch/arm/boot/compressed/piggy.o
>     CC      arch/arm/boot/compressed/misc.o
>
> This occurs with GNU Make commit 98da874c4303 ("[SV 10593] Export
> variables to $(shell ...) commands"), and the O= option is needed to
> reproduce it. The generated zImage is correct despite the error message.
>
> As the commit description of 98da874c4303 [1] says, exported variables
> are passed down to $(shell ) functions, which means exported recursive
> variables might be expanded earlier than before, in the parse stage.
>
> The following test code demonstrates the change for GNU Make 4.4.
>
> [Test Makefile]
>
>   $(shell echo hello > foo)
>   export foo = $(shell cat bar/../foo)
>   $(shell mkdir bar)
>
>   all:
>           @echo $(foo)
>
> [GNU Make 4.3]
>
>   $ rm -rf bar; make-4.3
>   hello
>
> [GNU Make 4.4]
>
>   $ rm -rf bar; make-4.4
>   cat: bar/../foo: No such file or directory
>   hello
>
> The 'foo' is a resursively expanded (or lazily expanded) variable.
>
> GNU Make 4.3 expands it just before running the recipe '@echo $(foo)',
> at this point, the directory 'bar' exists.
>
> GNU Make 4.4 expands it to evaluate $(shell mkdir bar) because 'foo' is
> exported. At this point, the directory 'bar' does not exit yet. The cat
> command cannot resolve the bar/../foo path, hence the error message.
>
> Let's get back to the kernel Makefile.
>
> 'KBSS_SZ' in arch/arm/boot/compressed/Makefile is a recursive variable,
> which is referenced by 'LDFLAGS_vmlinux', which is also a recursive
> variable.
>
> GNU Make 4.3 expands 'KBSS_SZ' just before running the recipes. Before
> that, $(shell mkdir -p $(obj-dirs)) in scripts/Makefile.build creates
> the output directory, arch/arm/boot/compressed.
>
> GNU Make 4.4 expands 'KBSS_SZ' in the parse stage because LDFLAGS_vmlinux
> was (accidentally) exported by commit 5d4aeffbf709 ("kbuild: rebuild
> .vmlinux.export.o when its prerequisite is updated"). $(NM) cannot
> resolve the path arch/arm/boot/compressed/../../../../vmlinux.
>
> I admit this is a bug caused by 5d4aeffbf709 (I will fix it in the next
> commit), but do not see any good reason in writing the vmlinux path in
> such an indirect way. Just say 'vmlinux'.
>
> [1]: https://git.savannah.gnu.org/cgit/make.git/commit/?id=98da874c43035a490cdca81331724f233a3d0c9a
>
> Link: https://lore.kernel.org/all/Y7i8+EjwdnhHtlrr@dev-arch.thelio-3990X/
> Fixes: 5d4aeffbf709 ("kbuild: rebuild .vmlinux.export.o when its prerequisite is updated")
> Reported-by: Nathan Chancellor <nathan@kernel.org>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---



I changed my mind. I will fix 2/2 only for now,
so that backing to stable kernels gets easier.


I will send v2 with updated commit description.




-- 
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH 1/2] ARM: fix nm error message when GNU Make >= 4.4 is used
@ 2023-01-08  9:50   ` Masahiro Yamada
  0 siblings, 0 replies; 5+ messages in thread
From: Masahiro Yamada @ 2023-01-08  9:50 UTC (permalink / raw)
  To: linux-kbuild
  Cc: linux-kernel, Nathan Chancellor, Arnd Bergmann, Nick Desaulniers,
	Nicolas Schier, Russell King (Oracle),
	Russell King, Seung-Woo Kim, Xin Li, linux-arm-kernel

On Sun, Jan 8, 2023 at 6:20 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> Nathan Chancellor reports an error message from $(NM) if GNU Make 4.4
> is used to build the ARM decompressor.
>
>   $ make-4.4 ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- O=build defconfig all
>     [snip]
>     LD      vmlinux
>     NM      System.map
>     SORTTAB vmlinux
>     OBJCOPY arch/arm/boot/Image
>     Kernel: arch/arm/boot/Image is ready
>   arm-linux-gnueabi-nm: 'arch/arm/boot/compressed/../../../../vmlinux': No such file
>   /bin/sh: 1: arithmetic expression: expecting primary: " "
>     LDS     arch/arm/boot/compressed/vmlinux.lds
>     AS      arch/arm/boot/compressed/head.o
>     GZIP    arch/arm/boot/compressed/piggy_data
>     AS      arch/arm/boot/compressed/piggy.o
>     CC      arch/arm/boot/compressed/misc.o
>
> This occurs with GNU Make commit 98da874c4303 ("[SV 10593] Export
> variables to $(shell ...) commands"), and the O= option is needed to
> reproduce it. The generated zImage is correct despite the error message.
>
> As the commit description of 98da874c4303 [1] says, exported variables
> are passed down to $(shell ) functions, which means exported recursive
> variables might be expanded earlier than before, in the parse stage.
>
> The following test code demonstrates the change for GNU Make 4.4.
>
> [Test Makefile]
>
>   $(shell echo hello > foo)
>   export foo = $(shell cat bar/../foo)
>   $(shell mkdir bar)
>
>   all:
>           @echo $(foo)
>
> [GNU Make 4.3]
>
>   $ rm -rf bar; make-4.3
>   hello
>
> [GNU Make 4.4]
>
>   $ rm -rf bar; make-4.4
>   cat: bar/../foo: No such file or directory
>   hello
>
> The 'foo' is a resursively expanded (or lazily expanded) variable.
>
> GNU Make 4.3 expands it just before running the recipe '@echo $(foo)',
> at this point, the directory 'bar' exists.
>
> GNU Make 4.4 expands it to evaluate $(shell mkdir bar) because 'foo' is
> exported. At this point, the directory 'bar' does not exit yet. The cat
> command cannot resolve the bar/../foo path, hence the error message.
>
> Let's get back to the kernel Makefile.
>
> 'KBSS_SZ' in arch/arm/boot/compressed/Makefile is a recursive variable,
> which is referenced by 'LDFLAGS_vmlinux', which is also a recursive
> variable.
>
> GNU Make 4.3 expands 'KBSS_SZ' just before running the recipes. Before
> that, $(shell mkdir -p $(obj-dirs)) in scripts/Makefile.build creates
> the output directory, arch/arm/boot/compressed.
>
> GNU Make 4.4 expands 'KBSS_SZ' in the parse stage because LDFLAGS_vmlinux
> was (accidentally) exported by commit 5d4aeffbf709 ("kbuild: rebuild
> .vmlinux.export.o when its prerequisite is updated"). $(NM) cannot
> resolve the path arch/arm/boot/compressed/../../../../vmlinux.
>
> I admit this is a bug caused by 5d4aeffbf709 (I will fix it in the next
> commit), but do not see any good reason in writing the vmlinux path in
> such an indirect way. Just say 'vmlinux'.
>
> [1]: https://git.savannah.gnu.org/cgit/make.git/commit/?id=98da874c43035a490cdca81331724f233a3d0c9a
>
> Link: https://lore.kernel.org/all/Y7i8+EjwdnhHtlrr@dev-arch.thelio-3990X/
> Fixes: 5d4aeffbf709 ("kbuild: rebuild .vmlinux.export.o when its prerequisite is updated")
> Reported-by: Nathan Chancellor <nathan@kernel.org>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---



I changed my mind. I will fix 2/2 only for now,
so that backing to stable kernels gets easier.


I will send v2 with updated commit description.




-- 
Best Regards
Masahiro Yamada

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2023-01-08  9:52 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-01-07 21:20 [PATCH 1/2] ARM: fix nm error message when GNU Make >= 4.4 is used Masahiro Yamada
2023-01-07 21:20 ` Masahiro Yamada
2023-01-07 21:20 ` [PATCH 2/2] kbuild: do not export LDFLAGS_vmlinux to decompressor Makefiles Masahiro Yamada
2023-01-08  9:50 ` [PATCH 1/2] ARM: fix nm error message when GNU Make >= 4.4 is used Masahiro Yamada
2023-01-08  9:50   ` Masahiro Yamada

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.