* [PATCH v7 1/8] kbuild: replace $(linked-object) with CONFIG options
2022-05-27 10:01 [PATCH v7 0/8] kbuild: yet another series of cleanups Masahiro Yamada
@ 2022-05-27 10:01 ` Masahiro Yamada
2022-05-27 15:00 ` Josh Poimboeuf
` (2 more replies)
2022-05-27 10:01 ` [PATCH v7 2/8] kbuild: do not create *.prelink.o for Clang LTO or IBT Masahiro Yamada
` (6 subsequent siblings)
7 siblings, 3 replies; 18+ messages in thread
From: Masahiro Yamada @ 2022-05-27 10:01 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm, Helge Deller,
linux-parisc, Masahiro Yamada, Michal Marek, Nick Desaulniers
*.prelink.o is created when CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT
is enabled.
Replace $(linked-object) with $(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT)
so you will get better idea when the --link option is passed.
No functional change is intended.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---
Changes in v7:
- New patch
scripts/Makefile.build | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 06400504150b..f80196eef03a 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -219,7 +219,7 @@ objtool_args = \
$(if $(CONFIG_STACK_VALIDATION), --stackval) \
$(if $(CONFIG_HAVE_STATIC_CALL_INLINE), --static-call) \
--uaccess \
- $(if $(linked-object), --link) \
+ $(if $($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT)), --link) \
$(if $(part-of-module), --module) \
$(if $(CONFIG_GCOV_KERNEL), --no-unreachable)
@@ -284,7 +284,6 @@ quiet_cmd_cc_prelink_modules = LD [M] $@
# modules into native code
$(obj)/%.prelink.o: objtool-enabled = y
$(obj)/%.prelink.o: part-of-module := y
-$(obj)/%.prelink.o: linked-object := y
$(obj)/%.prelink.o: $(obj)/%.o FORCE
$(call if_changed,cc_prelink_modules)
--
2.32.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH v7 1/8] kbuild: replace $(linked-object) with CONFIG options
2022-05-27 10:01 ` [PATCH v7 1/8] kbuild: replace $(linked-object) with CONFIG options Masahiro Yamada
@ 2022-05-27 15:00 ` Josh Poimboeuf
2022-05-27 17:32 ` Nick Desaulniers
2022-05-28 2:32 ` Sedat Dilek
2 siblings, 0 replies; 18+ messages in thread
From: Josh Poimboeuf @ 2022-05-27 15:00 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm,
Helge Deller, linux-parisc, Michal Marek, Nick Desaulniers
On Fri, May 27, 2022 at 07:01:48PM +0900, Masahiro Yamada wrote:
> *.prelink.o is created when CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT
> is enabled.
>
> Replace $(linked-object) with $(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT)
> so you will get better idea when the --link option is passed.
>
> No functional change is intended.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Josh Poimboeuf <jpoimboe@kernel.org>
--
Josh
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v7 1/8] kbuild: replace $(linked-object) with CONFIG options
2022-05-27 10:01 ` [PATCH v7 1/8] kbuild: replace $(linked-object) with CONFIG options Masahiro Yamada
2022-05-27 15:00 ` Josh Poimboeuf
@ 2022-05-27 17:32 ` Nick Desaulniers
2022-05-28 2:32 ` Sedat Dilek
2 siblings, 0 replies; 18+ messages in thread
From: Nick Desaulniers @ 2022-05-27 17:32 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm,
Helge Deller, linux-parisc, Michal Marek
On Fri, May 27, 2022 at 3:04 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> *.prelink.o is created when CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT
> is enabled.
>
> Replace $(linked-object) with $(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT)
> so you will get better idea when the --link option is passed.
>
> No functional change is intended.
It triggered my PTSD to see this ^ expression; many commits to LLVM
are labeled "NFCI."
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
>
> Changes in v7:
> - New patch
>
> scripts/Makefile.build | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index 06400504150b..f80196eef03a 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -219,7 +219,7 @@ objtool_args = \
> $(if $(CONFIG_STACK_VALIDATION), --stackval) \
> $(if $(CONFIG_HAVE_STATIC_CALL_INLINE), --static-call) \
> --uaccess \
> - $(if $(linked-object), --link) \
> + $(if $($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT)), --link) \
> $(if $(part-of-module), --module) \
> $(if $(CONFIG_GCOV_KERNEL), --no-unreachable)
>
> @@ -284,7 +284,6 @@ quiet_cmd_cc_prelink_modules = LD [M] $@
> # modules into native code
> $(obj)/%.prelink.o: objtool-enabled = y
> $(obj)/%.prelink.o: part-of-module := y
> -$(obj)/%.prelink.o: linked-object := y
>
> $(obj)/%.prelink.o: $(obj)/%.o FORCE
> $(call if_changed,cc_prelink_modules)
> --
> 2.32.0
>
--
Thanks,
~Nick Desaulniers
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v7 1/8] kbuild: replace $(linked-object) with CONFIG options
2022-05-27 10:01 ` [PATCH v7 1/8] kbuild: replace $(linked-object) with CONFIG options Masahiro Yamada
2022-05-27 15:00 ` Josh Poimboeuf
2022-05-27 17:32 ` Nick Desaulniers
@ 2022-05-28 2:32 ` Sedat Dilek
2022-05-28 7:10 ` Masahiro Yamada
2 siblings, 1 reply; 18+ messages in thread
From: Sedat Dilek @ 2022-05-28 2:32 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm,
Helge Deller, linux-parisc, Michal Marek, Nick Desaulniers
On Fri, May 27, 2022 at 1:56 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> *.prelink.o is created when CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT
> is enabled.
>
> Replace $(linked-object) with $(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT)
> so you will get better idea when the --link option is passed.
>
> No functional change is intended.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Hi Masahiroy,
I was not CCed on the cover-letter and only on some patches in this series.
So, I have re-tested this series by pulling from kbuild.git#kbuild
(saw later you renewed the parisc/nm patch).
While testing my selfmade LLVM version 14.0.4 (PGO + ThinLTO
optimized) toolchain I included this series.
Just built fine and I was able to boot on bare metal.
-Sedat -
> ---
>
> Changes in v7:
> - New patch
>
> scripts/Makefile.build | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index 06400504150b..f80196eef03a 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -219,7 +219,7 @@ objtool_args = \
> $(if $(CONFIG_STACK_VALIDATION), --stackval) \
> $(if $(CONFIG_HAVE_STATIC_CALL_INLINE), --static-call) \
> --uaccess \
> - $(if $(linked-object), --link) \
> + $(if $($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT)), --link) \
> $(if $(part-of-module), --module) \
> $(if $(CONFIG_GCOV_KERNEL), --no-unreachable)
>
> @@ -284,7 +284,6 @@ quiet_cmd_cc_prelink_modules = LD [M] $@
> # modules into native code
> $(obj)/%.prelink.o: objtool-enabled = y
> $(obj)/%.prelink.o: part-of-module := y
> -$(obj)/%.prelink.o: linked-object := y
>
> $(obj)/%.prelink.o: $(obj)/%.o FORCE
> $(call if_changed,cc_prelink_modules)
> --
> 2.32.0
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v7 1/8] kbuild: replace $(linked-object) with CONFIG options
2022-05-28 2:32 ` Sedat Dilek
@ 2022-05-28 7:10 ` Masahiro Yamada
0 siblings, 0 replies; 18+ messages in thread
From: Masahiro Yamada @ 2022-05-28 7:10 UTC (permalink / raw)
To: Sedat Dilek
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Peter Zijlstra, Josh Poimboeuf, clang-built-linux, Helge Deller,
Parisc List, Michal Marek, Nick Desaulniers
On Sat, May 28, 2022 at 11:32 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Fri, May 27, 2022 at 1:56 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > *.prelink.o is created when CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT
> > is enabled.
> >
> > Replace $(linked-object) with $(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT)
> > so you will get better idea when the --link option is passed.
> >
> > No functional change is intended.
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
>
> Hi Masahiroy,
>
> I was not CCed on the cover-letter and only on some patches in this series.
>
> So, I have re-tested this series by pulling from kbuild.git#kbuild
> (saw later you renewed the parisc/nm patch).
>
> While testing my selfmade LLVM version 14.0.4 (PGO + ThinLTO
> optimized) toolchain I included this series.
>
> Just built fine and I was able to boot on bare metal.
>
> -Sedat -
>
Thanks for your testing!
I added your Tested-by tag.
> > ---
> >
> > Changes in v7:
> > - New patch
> >
> > scripts/Makefile.build | 3 +--
> > 1 file changed, 1 insertion(+), 2 deletions(-)
> >
> > diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> > index 06400504150b..f80196eef03a 100644
> > --- a/scripts/Makefile.build
> > +++ b/scripts/Makefile.build
> > @@ -219,7 +219,7 @@ objtool_args = \
> > $(if $(CONFIG_STACK_VALIDATION), --stackval) \
> > $(if $(CONFIG_HAVE_STATIC_CALL_INLINE), --static-call) \
> > --uaccess \
> > - $(if $(linked-object), --link) \
> > + $(if $($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT)), --link) \
> > $(if $(part-of-module), --module) \
> > $(if $(CONFIG_GCOV_KERNEL), --no-unreachable)
> >
> > @@ -284,7 +284,6 @@ quiet_cmd_cc_prelink_modules = LD [M] $@
> > # modules into native code
> > $(obj)/%.prelink.o: objtool-enabled = y
> > $(obj)/%.prelink.o: part-of-module := y
> > -$(obj)/%.prelink.o: linked-object := y
> >
> > $(obj)/%.prelink.o: $(obj)/%.o FORCE
> > $(call if_changed,cc_prelink_modules)
> > --
> > 2.32.0
> >
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v7 2/8] kbuild: do not create *.prelink.o for Clang LTO or IBT
2022-05-27 10:01 [PATCH v7 0/8] kbuild: yet another series of cleanups Masahiro Yamada
2022-05-27 10:01 ` [PATCH v7 1/8] kbuild: replace $(linked-object) with CONFIG options Masahiro Yamada
@ 2022-05-27 10:01 ` Masahiro Yamada
2022-05-27 15:31 ` Josh Poimboeuf
2022-05-27 10:01 ` [PATCH v7 3/8] parisc: fix the exit status of arch/parisc/nm Masahiro Yamada
` (5 subsequent siblings)
7 siblings, 1 reply; 18+ messages in thread
From: Masahiro Yamada @ 2022-05-27 10:01 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm, Helge Deller,
linux-parisc, Masahiro Yamada, Nicolas Schier, Nathan Chancellor,
Sami Tolvanen, Sedat Dilek, Michal Marek, Nick Desaulniers,
Tom Rix
When CONFIG_LTO_CLANG=y, additional intermediate *.prelink.o is created
for each module. Also, objtool is postponed until LLVM IR is converted
to ELF.
CONFIG_X86_KERNEL_IBT works in a similar way to postpone objtool until
objects are merged together.
This commit stops generating *.prelink.o, so the build flow will look
similar with/without LTO.
The following figures show how the LTO build currently works, and
how this commit is changing it.
Current build flow
==================
[1] single-object module
$(LD)
$(CC) +objtool $(LD)
foo.c --------------------> foo.o -----> foo.prelink.o -----> foo.ko
(LLVM IR) (ELF) |
|
foo.mod.o --/
[2] multi-object module
$(LD)
$(CC) $(AR) +objtool $(LD)
foo1.c -----> foo1.o -----> foo.o -----> foo.prelink.o -----> foo.ko
| (archive) (ELF) |
foo2.c -----> foo2.o --/ |
(LLVM IR) foo.mod.o --/
One confusion is foo.o in multi-object module is an archive despite of
its suffix.
New build flow
==============
[1] single-object module
Since there is only one object, there is no need to keep the LLVM IR.
Use $(CC)+$(LD) to generate an ELF object in one build rule. When LTO
is disabled, $(LD) is unneeded because $(CC) produces an ELF object.
$(CC)+$(LD)+objtool $(LD)
foo.c ----------------------------> foo.o ---------> foo.ko
(ELF) |
|
foo.mod.o --/
[2] multi-object module
Previously, $(AR) was used to combine LLVM bitcode into an archive,
but there was no technical reason to do so. Use $(LD) to merge them
into a single ELF object.
$(LD)
$(CC) +objtool $(LD)
foo1.c ---------> foo1.o ---------> foo.o ---------> foo.ko
| (ELF) |
foo2.c ---------> foo2.o ----/ |
(LLVM IR) foo.mod.o --/
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
---
(no changes since v2)
Changes in v2:
- replace the chain of $(if ...) with $(and )
scripts/Kbuild.include | 4 +++
scripts/Makefile.build | 60 ++++++++++++---------------------------
scripts/Makefile.lib | 7 -----
scripts/Makefile.modfinal | 5 ++--
scripts/Makefile.modpost | 9 ++----
scripts/mod/modpost.c | 7 -----
6 files changed, 26 insertions(+), 66 deletions(-)
diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index 3514c2149e9d..455a0a6ce12d 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -15,6 +15,10 @@ pound := \#
# Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o
dot-target = $(dir $@).$(notdir $@)
+###
+# Name of target with a '.tmp_' as filename prefix. foo/bar.o => foo/.tmp_bar.o
+tmp-target = $(dir $@).tmp_$(notdir $@)
+
###
# The temporary file to save gcc -MMD generated dependencies must not
# contain a comma
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index f80196eef03a..23e1fb19f9ad 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -88,10 +88,6 @@ endif
targets-for-modules := $(foreach x, o mod $(if $(CONFIG_TRIM_UNUSED_KSYMS), usyms), \
$(patsubst %.o, %.$x, $(filter %.o, $(obj-m))))
-ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
-targets-for-modules += $(patsubst %.o, %.prelink.o, $(filter %.o, $(obj-m)))
-endif
-
ifdef need-modorder
targets-for-modules += $(obj)/modules.order
endif
@@ -152,8 +148,16 @@ $(obj)/%.ll: $(src)/%.c FORCE
# The C file is compiled and updated dependency information is generated.
# (See cmd_cc_o_c + relevant part of rule_cc_o_c)
+is-single-obj-m = $(and $(part-of-module),$(filter $@, $(obj-m)),y)
+
+ifdef CONFIG_LTO_CLANG
+cmd_ld_single_m = $(if $(is-single-obj-m), ; $(LD) $(ld_flags) -r -o $(tmp-target) $@; mv $(tmp-target) $@)
+endif
+
quiet_cmd_cc_o_c = CC $(quiet_modtag) $@
- cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $< $(cmd_objtool)
+ cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $< \
+ $(cmd_ld_single_m) \
+ $(cmd_objtool)
ifdef CONFIG_MODVERSIONS
# When module versioning is enabled the following steps are executed:
@@ -219,7 +223,7 @@ objtool_args = \
$(if $(CONFIG_STACK_VALIDATION), --stackval) \
$(if $(CONFIG_HAVE_STATIC_CALL_INLINE), --static-call) \
--uaccess \
- $(if $($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT)), --link) \
+ $(if $(delay-objtool), --link) \
$(if $(part-of-module), --module) \
$(if $(CONFIG_GCOV_KERNEL), --no-unreachable)
@@ -228,21 +232,16 @@ cmd_gen_objtooldep = $(if $(objtool-enabled), { echo ; echo '$@: $$(wildcard $(o
endif # CONFIG_OBJTOOL
-ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
-
-# Skip objtool for LLVM bitcode
-$(obj)/%.o: objtool-enabled :=
-
-else
# 'OBJECT_FILES_NON_STANDARD := y': skip objtool checking for a directory
# 'OBJECT_FILES_NON_STANDARD_foo.o := 'y': skip objtool checking for a file
# 'OBJECT_FILES_NON_STANDARD_foo.o := 'n': override directory skip for a file
-$(obj)/%.o: objtool-enabled = $(if $(filter-out y%, \
- $(OBJECT_FILES_NON_STANDARD_$(basetarget).o)$(OBJECT_FILES_NON_STANDARD)n),y)
+is-standard-object = $(if $(filter-out y%, $(OBJECT_FILES_NON_STANDARD_$(basetarget).o)$(OBJECT_FILES_NON_STANDARD)n),y)
-endif
+delay-objtool := $(or $(CONFIG_LTO_CLANG),$(CONFIG_X86_KERNEL_IBT))
+
+$(obj)/%.o: objtool-enabled = $(if $(is-standard-object),$(if $(delay-objtool),$(is-single-obj-m),y))
ifdef CONFIG_TRIM_UNUSED_KSYMS
cmd_gen_ksymdeps = \
@@ -271,24 +270,6 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE
$(call if_changed_rule,cc_o_c)
$(call cmd,force_checksrc)
-ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
-# Module .o files may contain LLVM bitcode, compile them into native code
-# before ELF processing
-quiet_cmd_cc_prelink_modules = LD [M] $@
- cmd_cc_prelink_modules = \
- $(LD) $(ld_flags) -r -o $@ \
- --whole-archive $(filter-out FORCE,$^) \
- $(cmd_objtool)
-
-# objtool was skipped for LLVM bitcode, run it now that we have compiled
-# modules into native code
-$(obj)/%.prelink.o: objtool-enabled = y
-$(obj)/%.prelink.o: part-of-module := y
-
-$(obj)/%.prelink.o: $(obj)/%.o FORCE
- $(call if_changed,cc_prelink_modules)
-endif
-
cmd_mod = echo $(addprefix $(obj)/, $(call real-search, $*.o, .o, -objs -y -m)) | \
$(AWK) -v RS='( |\n)' '!x[$$0]++' > $@
@@ -298,7 +279,7 @@ $(obj)/%.mod: FORCE
# List module undefined symbols
cmd_undefined_syms = $(NM) $< | sed -n 's/^ *U //p' > $@
-$(obj)/%.usyms: $(obj)/%$(mod-prelink-ext).o FORCE
+$(obj)/%.usyms: $(obj)/%.o FORCE
$(call if_changed,undefined_syms)
quiet_cmd_cc_lst_c = MKLST $@
@@ -420,16 +401,11 @@ $(obj)/modules.order: $(obj-m) FORCE
$(obj)/lib.a: $(lib-y) FORCE
$(call if_changed,ar)
-ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
-quiet_cmd_link_multi-m = AR [M] $@
-cmd_link_multi-m = \
- rm -f $@; \
- $(AR) cDPrsT $@ @$(patsubst %.o,%.mod,$@)
-else
quiet_cmd_link_multi-m = LD [M] $@
- cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@)
-endif
+ cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool)
+$(multi-obj-m): objtool-enabled := $(delay-objtool)
+$(multi-obj-m): part-of-module := y
$(multi-obj-m): %.o: %.mod FORCE
$(call if_changed,link_multi-m)
$(call multi_depend, $(multi-obj-m), .o, -objs -y -m)
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 0453a1904646..f75138385449 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -225,13 +225,6 @@ dtc_cpp_flags = -Wp,-MMD,$(depfile).pre.tmp -nostdinc \
$(addprefix -I,$(DTC_INCLUDE)) \
-undef -D__DTS__
-ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
-# With CONFIG_LTO_CLANG, .o files in modules might be LLVM bitcode, so we
-# need to run LTO to compile them into native code (.lto.o) before further
-# processing.
-mod-prelink-ext := .prelink
-endif
-
# Useful for describing the dependency of composite objects
# Usage:
# $(call multi_depend, multi_used_targets, suffix_to_remove, suffix_to_add)
diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
index 7f39599e9fae..35100e981f4a 100644
--- a/scripts/Makefile.modfinal
+++ b/scripts/Makefile.modfinal
@@ -9,7 +9,7 @@ __modfinal:
include include/config/auto.conf
include $(srctree)/scripts/Kbuild.include
-# for c_flags and mod-prelink-ext
+# for c_flags
include $(srctree)/scripts/Makefile.lib
# find all modules listed in modules.order
@@ -54,9 +54,8 @@ if_changed_except = $(if $(call newer_prereqs_except,$(2))$(cmd-check), \
$(cmd); \
printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:)
-
# Re-generate module BTFs if either module's .ko or vmlinux changed
-$(modules): %.ko: %$(mod-prelink-ext).o %.mod.o scripts/module.lds $(if $(KBUILD_BUILTIN),vmlinux) FORCE
+$(modules): %.ko: %.o %.mod.o scripts/module.lds $(if $(KBUILD_BUILTIN),vmlinux) FORCE
+$(call if_changed_except,ld_ko_o,vmlinux)
ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+$(if $(newer-prereqs),$(call cmd,btf_ko))
diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
index 8b343480813d..911606496341 100644
--- a/scripts/Makefile.modpost
+++ b/scripts/Makefile.modpost
@@ -41,9 +41,6 @@ __modpost:
include include/config/auto.conf
include $(srctree)/scripts/Kbuild.include
-# for mod-prelink-ext
-include $(srctree)/scripts/Makefile.lib
-
MODPOST = scripts/mod/modpost \
$(if $(CONFIG_MODVERSIONS),-m) \
$(if $(CONFIG_MODULE_SRCVERSION_ALL),-a) \
@@ -117,8 +114,6 @@ $(input-symdump):
@echo >&2 ' Modules may not have dependencies or modversions.'
@echo >&2 ' You may get many unresolved symbol warnings.'
-modules := $(sort $(shell cat $(MODORDER)))
-
# KBUILD_MODPOST_WARN can be set to avoid error out in case of undefined symbols
ifneq ($(KBUILD_MODPOST_WARN)$(filter-out $(existing-input-symdump), $(input-symdump)),)
MODPOST += -w
@@ -127,9 +122,9 @@ endif
# Read out modules.order to pass in modpost.
# Otherwise, allmodconfig would fail with "Argument list too long".
quiet_cmd_modpost = MODPOST $@
- cmd_modpost = sed 's/\.ko$$/$(mod-prelink-ext)\.o/' $< | $(MODPOST) -T -
+ cmd_modpost = sed 's/ko$$/o/' $< | $(MODPOST) -T -
-$(output-symdump): $(MODORDER) $(input-symdump) $(modules:.ko=$(mod-prelink-ext).o) FORCE
+$(output-symdump): $(MODORDER) $(input-symdump) FORCE
$(call if_changed,modpost)
targets += $(output-symdump)
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index b70a31b4245a..e74224a6a8e8 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -1903,10 +1903,6 @@ static char *remove_dot(char *s)
size_t m = strspn(s + n + 1, "0123456789");
if (m && (s[n + m + 1] == '.' || s[n + m + 1] == 0))
s[n] = 0;
-
- /* strip trailing .prelink */
- if (strends(s, ".prelink"))
- s[strlen(s) - 8] = '\0';
}
return s;
}
@@ -2028,9 +2024,6 @@ static void read_symbols(const char *modname)
/* strip trailing .o */
tmp = NOFAIL(strdup(modname));
tmp[strlen(tmp) - 2] = '\0';
- /* strip trailing .prelink */
- if (strends(tmp, ".prelink"))
- tmp[strlen(tmp) - 8] = '\0';
mod = new_module(tmp);
free(tmp);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH v7 2/8] kbuild: do not create *.prelink.o for Clang LTO or IBT
2022-05-27 10:01 ` [PATCH v7 2/8] kbuild: do not create *.prelink.o for Clang LTO or IBT Masahiro Yamada
@ 2022-05-27 15:31 ` Josh Poimboeuf
0 siblings, 0 replies; 18+ messages in thread
From: Josh Poimboeuf @ 2022-05-27 15:31 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm,
Helge Deller, linux-parisc, Nicolas Schier, Nathan Chancellor,
Sami Tolvanen, Sedat Dilek, Michal Marek, Nick Desaulniers,
Tom Rix
On Fri, May 27, 2022 at 07:01:49PM +0900, Masahiro Yamada wrote:
> New build flow
> ==============
>
> [1] single-object module
>
> Since there is only one object, there is no need to keep the LLVM IR.
> Use $(CC)+$(LD) to generate an ELF object in one build rule. When LTO
> is disabled, $(LD) is unneeded because $(CC) produces an ELF object.
>
> $(CC)+$(LD)+objtool $(LD)
> foo.c ----------------------------> foo.o ---------> foo.ko
> (ELF) |
> |
> foo.mod.o --/
>
> [2] multi-object module
>
> Previously, $(AR) was used to combine LLVM bitcode into an archive,
> but there was no technical reason to do so. Use $(LD) to merge them
> into a single ELF object.
>
> $(LD)
> $(CC) +objtool $(LD)
> foo1.c ---------> foo1.o ---------> foo.o ---------> foo.ko
> | (ELF) |
> foo2.c ---------> foo2.o ----/ |
> (LLVM IR) foo.mod.o --/
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
> Tested-by: Nathan Chancellor <nathan@kernel.org>
> Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
Acked-by: Josh Poimboeuf <jpoimboe@kernel.org>
--
Josh
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v7 3/8] parisc: fix the exit status of arch/parisc/nm
2022-05-27 10:01 [PATCH v7 0/8] kbuild: yet another series of cleanups Masahiro Yamada
2022-05-27 10:01 ` [PATCH v7 1/8] kbuild: replace $(linked-object) with CONFIG options Masahiro Yamada
2022-05-27 10:01 ` [PATCH v7 2/8] kbuild: do not create *.prelink.o for Clang LTO or IBT Masahiro Yamada
@ 2022-05-27 10:01 ` Masahiro Yamada
2022-05-27 13:57 ` Helge Deller
2022-05-27 10:01 ` [PATCH v7 4/8] kbuild: check static EXPORT_SYMBOL* by script instead of modpost Masahiro Yamada
` (4 subsequent siblings)
7 siblings, 1 reply; 18+ messages in thread
From: Masahiro Yamada @ 2022-05-27 10:01 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm, Helge Deller,
linux-parisc, Masahiro Yamada, James E.J. Bottomley
parisc overrides 'nm' with a shell script. I do not know the reason,
but anyway it is how it has worked since 2003. [1]
A problem is that this script returns the exit code of grep instead of
${CROSS_COMPILE}nm.
grep(1) says:
Normally the exit status is 0 if a line is selected, 1 if no lines
were selected, and 2 if an error occurred. However, if the -q or
--quiet or --silent is used and a line is selected, the exit status
is 0 even if an error occurred.
When the given object has no symbol, grep returns 1, while the true nm
returns 0. Hence, build rules using ${NM} fail on ARCH=parisc even if
the given object is valid.
This commit corrects the exit status of the script.
- A pipeline returns the exit status of the last command (here, grep).
The exit status of ${CROSS_COMPILE}nm is just ignored. Use bash's
pipefail flag to catch errors of ${CROSS_COMPILE}nm.
- If grep returns 1, this script should return 0 in order to mimic
true nm. If grep returns 2, it is a real and fatal error. Return
it as is.
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/history/history.git/commit/?id=36eaa6e4c0e0b6950136b956b72fd08155b92ca3
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---
Changes in v7:
- New patch
arch/parisc/Makefile | 2 +-
arch/parisc/nm | 12 ++++++++++--
2 files changed, 11 insertions(+), 3 deletions(-)
mode change 100644 => 100755 arch/parisc/nm
diff --git a/arch/parisc/Makefile b/arch/parisc/Makefile
index aca1710fd658..e7139955367d 100644
--- a/arch/parisc/Makefile
+++ b/arch/parisc/Makefile
@@ -18,7 +18,7 @@
boot := arch/parisc/boot
KBUILD_IMAGE := $(boot)/bzImage
-NM = sh $(srctree)/arch/parisc/nm
+NM = $(srctree)/arch/parisc/nm
CHECKFLAGS += -D__hppa__=1
ifdef CONFIG_64BIT
diff --git a/arch/parisc/nm b/arch/parisc/nm
old mode 100644
new mode 100755
index c788308de33f..3e72238a91f3
--- a/arch/parisc/nm
+++ b/arch/parisc/nm
@@ -1,6 +1,14 @@
-#!/bin/sh
+#!/bin/bash
##
# Hack to have an nm which removes the local symbols. We also rely
# on this nm being hidden out of the ordinarily executable path
##
-${CROSS_COMPILE}nm $* | grep -v '.LC*[0-9]*$'
+
+# use pipefail to catch error of ${CROSS_COMPILE}nm
+set -o pipefail
+
+# grep exits with 1 if no lines were selected.
+# If the given object has no symbol, grep returns 1, but it is not an error.
+
+${CROSS_COMPILE}nm "$@" |
+{ grep -v '.LC*[0-9]*$' || { exit_code=$?; test $exit_code -eq 1 || exit $exit_code; } }
--
2.32.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH v7 3/8] parisc: fix the exit status of arch/parisc/nm
2022-05-27 10:01 ` [PATCH v7 3/8] parisc: fix the exit status of arch/parisc/nm Masahiro Yamada
@ 2022-05-27 13:57 ` Helge Deller
2022-05-27 14:21 ` Masahiro Yamada
0 siblings, 1 reply; 18+ messages in thread
From: Helge Deller @ 2022-05-27 13:57 UTC (permalink / raw)
To: Masahiro Yamada, linux-kbuild
Cc: linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm, linux-parisc,
James E.J. Bottomley
Hello Masahiro,
On 5/27/22 12:01, Masahiro Yamada wrote:
> parisc overrides 'nm' with a shell script. I do not know the reason,
> but anyway it is how it has worked since 2003. [1]
I don't know the reason either...
I assume it was that the older toolchains had bugs and kept lots of local
symbols like .LC? in the object files.
I did a small build without the nm script (and removed it's reference
in the Makefile), and it did not seem to break anything.
> A problem is that this script returns the exit code of grep instead of
> ${CROSS_COMPILE}nm.
Instead of fixing this, I'd suggest that you simply remove the nm script
alltogether. If you like I can apply such a patch in the parisc git tree,
and you just drop this specific patch. Or you change your patch to remove it.
Just let me know what you prefer.
Helge
> grep(1) says:
> Normally the exit status is 0 if a line is selected, 1 if no lines
> were selected, and 2 if an error occurred. However, if the -q or
> --quiet or --silent is used and a line is selected, the exit status
> is 0 even if an error occurred.
>
> When the given object has no symbol, grep returns 1, while the true nm
> returns 0. Hence, build rules using ${NM} fail on ARCH=parisc even if
> the given object is valid.
>
> This commit corrects the exit status of the script.
>
> - A pipeline returns the exit status of the last command (here, grep).
> The exit status of ${CROSS_COMPILE}nm is just ignored. Use bash's
> pipefail flag to catch errors of ${CROSS_COMPILE}nm.
>
> - If grep returns 1, this script should return 0 in order to mimic
> true nm. If grep returns 2, it is a real and fatal error. Return
> it as is.
>
> [1]: https://git.kernel.org/pub/scm/linux/kernel/git/history/history.git/commit/?id=36eaa6e4c0e0b6950136b956b72fd08155b92ca3
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
>
> Changes in v7:
> - New patch
>
> arch/parisc/Makefile | 2 +-
> arch/parisc/nm | 12 ++++++++++--
> 2 files changed, 11 insertions(+), 3 deletions(-)
> mode change 100644 => 100755 arch/parisc/nm
>
> diff --git a/arch/parisc/Makefile b/arch/parisc/Makefile
> index aca1710fd658..e7139955367d 100644
> --- a/arch/parisc/Makefile
> +++ b/arch/parisc/Makefile
> @@ -18,7 +18,7 @@
> boot := arch/parisc/boot
> KBUILD_IMAGE := $(boot)/bzImage
>
> -NM = sh $(srctree)/arch/parisc/nm
> +NM = $(srctree)/arch/parisc/nm
> CHECKFLAGS += -D__hppa__=1
>
> ifdef CONFIG_64BIT
> diff --git a/arch/parisc/nm b/arch/parisc/nm
> old mode 100644
> new mode 100755
> index c788308de33f..3e72238a91f3
> --- a/arch/parisc/nm
> +++ b/arch/parisc/nm
> @@ -1,6 +1,14 @@
> -#!/bin/sh
> +#!/bin/bash
> ##
> # Hack to have an nm which removes the local symbols. We also rely
> # on this nm being hidden out of the ordinarily executable path
> ##
> -${CROSS_COMPILE}nm $* | grep -v '.LC*[0-9]*$'
> +
> +# use pipefail to catch error of ${CROSS_COMPILE}nm
> +set -o pipefail
> +
> +# grep exits with 1 if no lines were selected.
> +# If the given object has no symbol, grep returns 1, but it is not an error.
> +
> +${CROSS_COMPILE}nm "$@" |
> +{ grep -v '.LC*[0-9]*$' || { exit_code=$?; test $exit_code -eq 1 || exit $exit_code; } }
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [PATCH v7 3/8] parisc: fix the exit status of arch/parisc/nm
2022-05-27 13:57 ` Helge Deller
@ 2022-05-27 14:21 ` Masahiro Yamada
0 siblings, 0 replies; 18+ messages in thread
From: Masahiro Yamada @ 2022-05-27 14:21 UTC (permalink / raw)
To: Helge Deller
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Peter Zijlstra, Josh Poimboeuf, clang-built-linux, Parisc List,
James E.J. Bottomley
Hi Helge,
On Fri, May 27, 2022 at 10:58 PM Helge Deller <deller@gmx.de> wrote:
>
> Hello Masahiro,
>
> On 5/27/22 12:01, Masahiro Yamada wrote:
> > parisc overrides 'nm' with a shell script. I do not know the reason,
> > but anyway it is how it has worked since 2003. [1]
>
> I don't know the reason either...
> I assume it was that the older toolchains had bugs and kept lots of local
> symbols like .LC? in the object files.
>
> I did a small build without the nm script (and removed it's reference
> in the Makefile), and it did not seem to break anything.
>
> > A problem is that this script returns the exit code of grep instead of
> > ${CROSS_COMPILE}nm.
>
> Instead of fixing this, I'd suggest that you simply remove the nm script
> alltogether. If you like I can apply such a patch in the parisc git tree,
> and you just drop this specific patch. Or you change your patch to remove it.
> Just let me know what you prefer.
This is a prerequisite of my kbuild work:
https://lore.kernel.org/linux-kbuild/20220527100155.1996314-5-masahiroy@kernel.org/T/#u
Without this, ARCH=parisc builds will be broken due to ${NM} returning 1.
I will send a patch to remove arch/parisc/nm.
Please give me your ack.
Thank you.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH v7 4/8] kbuild: check static EXPORT_SYMBOL* by script instead of modpost
2022-05-27 10:01 [PATCH v7 0/8] kbuild: yet another series of cleanups Masahiro Yamada
` (2 preceding siblings ...)
2022-05-27 10:01 ` [PATCH v7 3/8] parisc: fix the exit status of arch/parisc/nm Masahiro Yamada
@ 2022-05-27 10:01 ` Masahiro Yamada
2022-05-27 10:01 ` [PATCH v7 5/8] kbuild: make built-in.a rule robust against too long argument error Masahiro Yamada
` (3 subsequent siblings)
7 siblings, 0 replies; 18+ messages in thread
From: Masahiro Yamada @ 2022-05-27 10:01 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm, Helge Deller,
linux-parisc, Masahiro Yamada, Nick Desaulniers,
Nathan Chancellor, Sedat Dilek, Michal Marek, Tom Rix
The 'static' specifier and EXPORT_SYMBOL() are an odd combination.
Commit 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL*
functions") tried to detect it, but this check has false negatives.
Here is the sample code.
Makefile:
obj-y += foo1.o foo2.o
foo1.c:
#include <linux/export.h>
static void foo(void) {}
EXPORT_SYMBOL(foo);
foo2.c:
void foo(void) {}
foo1.c exports the static symbol 'foo', but modpost cannot catch it
because it is fooled by foo2.c, which has a global symbol with the
same name.
s->is_static is cleared if a global symbol with the same name is found
somewhere, but EXPORT_SYMBOL() and the global symbol do not necessarily
belong to the same compilation unit.
This check should be done per compilation unit, but I do not know how
to do it in modpost. modpost runs against vmlinux.o or modules, which
merges multiple objects, then forgets their origin.
modpost cannot parse individual objects because they may not be ELF but
LLVM IR when CONFIG_LTO_CLANG=y.
Add a simple bash script to parse the output from ${NM}. This works for
CONFIG_LTO_CLANG=y because llvm-nm can dump symbols of LLVM IR files.
Revert 15bfc2348d54.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
---
Changes in v7:
- Fix sparc64 build (skip the line if the number of fields is less than 3)
Changes in v5:
- Add more comments in the script
Changes in v4:
- New patch
scripts/Makefile.build | 4 +++
scripts/check-local-export | 65 ++++++++++++++++++++++++++++++++++++++
scripts/mod/modpost.c | 28 +---------------
3 files changed, 70 insertions(+), 27 deletions(-)
create mode 100755 scripts/check-local-export
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 23e1fb19f9ad..0135b4e7f78b 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -248,9 +248,12 @@ cmd_gen_ksymdeps = \
$(CONFIG_SHELL) $(srctree)/scripts/gen_ksymdeps.sh $@ >> $(dot-target).cmd
endif
+cmd_check_local_export = $(srctree)/scripts/check-local-export $@
+
define rule_cc_o_c
$(call cmd_and_fixdep,cc_o_c)
$(call cmd,gen_ksymdeps)
+ $(call cmd,check_local_export)
$(call cmd,checksrc)
$(call cmd,checkdoc)
$(call cmd,gen_objtooldep)
@@ -261,6 +264,7 @@ endef
define rule_as_o_S
$(call cmd_and_fixdep,as_o_S)
$(call cmd,gen_ksymdeps)
+ $(call cmd,check_local_export)
$(call cmd,gen_objtooldep)
$(call cmd,gen_symversions_S)
endef
diff --git a/scripts/check-local-export b/scripts/check-local-export
new file mode 100755
index 000000000000..017119d89cd2
--- /dev/null
+++ b/scripts/check-local-export
@@ -0,0 +1,65 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Masahiro Yamada <masahiroy@kernel.org>
+#
+# Exit with error if a local exported symbol is found.
+# EXPORT_SYMBOL should be used for global symbols.
+
+set -e
+
+declare -A symbol_types
+declare -a export_symbols
+
+exit_code=0
+
+while read value type name
+do
+ # Skip the number of fields is less than 3.
+ #
+ # case 1)
+ # For undefined symbols, the first field (value) is empty.
+ # The outout looks like this:
+ # " U _printk"
+ # It is unneeded to record undefined symbols.
+ #
+ # case 2)
+ # For Clang LTO, llvm-nm outputs a line with type 't' but empty name:
+ # "---------------- t"
+ if [[ -z ${name} ]]; then
+ continue
+ fi
+
+ # save (name, type) in the associative array
+ symbol_types[${name}]=${type}
+
+ # append the exported symbol to the array
+ if [[ ${name} == __ksymtab_* ]]; then
+ export_symbols+=(${name#__ksymtab_})
+ fi
+
+ # If there is no symbol in the object, ${NM} (both GNU nm and llvm-nm)
+ # shows 'no symbols' diagnostic (but exits with 0). It is harmless and
+ # hidden by '2>/dev/null'. However, it suppresses real error messages
+ # as well. Add a hand-crafted error message here.
+ #
+ # Use --quiet instead of 2>/dev/null when we upgrade the minimum version
+ # of binutils to 2.37, llvm to 13.0.0.
+ #
+ # Then, the following line will be really simple:
+ # done < <(${NM} --quiet ${1})
+done < <(${NM} ${1} 2>/dev/null || { echo "${0}: ${NM} failed" >&2; false; } )
+
+# Catch error in the process substitution
+wait $!
+
+for name in "${export_symbols[@]}"
+do
+ # nm(3) says "If lowercase, the symbol is usually local"
+ if [[ ${symbol_types[$name]} =~ [a-z] ]]; then
+ echo "$@: error: local symbol '${name}' was exported" >&2
+ exit_code=1
+ fi
+done
+
+exit ${exit_code}
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index e74224a6a8e8..9269735f85c5 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -212,7 +212,6 @@ struct symbol {
unsigned int crc;
bool crc_valid;
bool weak;
- bool is_static; /* true if symbol is not global */
bool is_gpl_only; /* exported by EXPORT_SYMBOL_GPL */
char name[];
};
@@ -242,7 +241,7 @@ static struct symbol *alloc_symbol(const char *name)
memset(s, 0, sizeof(*s));
strcpy(s->name, name);
- s->is_static = true;
+
return s;
}
@@ -2064,20 +2063,6 @@ static void read_symbols(const char *modname)
sym_get_data(&info, sym));
}
- // check for static EXPORT_SYMBOL_* functions && global vars
- for (sym = info.symtab_start; sym < info.symtab_stop; sym++) {
- unsigned char bind = ELF_ST_BIND(sym->st_info);
-
- if (bind == STB_GLOBAL || bind == STB_WEAK) {
- struct symbol *s =
- find_symbol(remove_dot(info.strtab +
- sym->st_name));
-
- if (s)
- s->is_static = false;
- }
- }
-
check_sec_ref(modname, &info);
if (!mod->is_vmlinux) {
@@ -2507,7 +2492,6 @@ static void read_dump(const char *fname)
mod->from_dump = true;
}
s = sym_add_exported(symname, mod, gpl_only);
- s->is_static = false;
sym_set_crc(s, crc);
sym_update_namespace(symname, namespace);
}
@@ -2572,7 +2556,6 @@ int main(int argc, char **argv)
char *missing_namespace_deps = NULL;
char *dump_write = NULL, *files_source = NULL;
int opt;
- int n;
LIST_HEAD(dump_lists);
struct dump_list *dl, *dl2;
@@ -2648,15 +2631,6 @@ int main(int argc, char **argv)
if (sec_mismatch_count && !sec_mismatch_warn_only)
error("Section mismatches detected.\n"
"Set CONFIG_SECTION_MISMATCH_WARN_ONLY=y to allow them.\n");
- for (n = 0; n < SYMBOL_HASH_SIZE; n++) {
- struct symbol *s;
-
- for (s = symbolhash[n]; s; s = s->next) {
- if (s->is_static)
- error("\"%s\" [%s] is a static EXPORT_SYMBOL\n",
- s->name, s->module->name);
- }
- }
if (nr_unresolved > MAX_UNRESOLVED_REPORTS)
warn("suppressed %u unresolved symbol warnings because there were too many)\n",
--
2.32.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH v7 5/8] kbuild: make built-in.a rule robust against too long argument error
2022-05-27 10:01 [PATCH v7 0/8] kbuild: yet another series of cleanups Masahiro Yamada
` (3 preceding siblings ...)
2022-05-27 10:01 ` [PATCH v7 4/8] kbuild: check static EXPORT_SYMBOL* by script instead of modpost Masahiro Yamada
@ 2022-05-27 10:01 ` Masahiro Yamada
2022-05-27 10:01 ` [PATCH v7 6/8] kbuild: make *.mod " Masahiro Yamada
` (2 subsequent siblings)
7 siblings, 0 replies; 18+ messages in thread
From: Masahiro Yamada @ 2022-05-27 10:01 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm, Helge Deller,
linux-parisc, Masahiro Yamada, Nicolas Schier, Nathan Chancellor,
Sedat Dilek, Michal Marek, Nick Desaulniers, Tom Rix
Kbuild runs at the top of objtree instead of changing the working
directory to subdirectories. I think this design is nice overall but
some commands have a scalability issue.
The build command of built-in.a is one of them whose length scales with:
O(D * N)
Here, D is the length of the directory path (i.e. $(obj)/ prefix),
N is the number of objects in the Makefile, O() is the big O notation.
The deeper directory the Makefile directory is located, the more easily
it will hit the too long argument error.
We can make it better. Trim the $(obj)/ by Make's builtin function, and
restore it by a shell command (sed).
With this, the command length scales with:
O(D + N)
In-tree modules still have some room to the limit (ARG_MAX=2097152),
but this is more future-proof for big modules in a deep directory.
For example, you can build i915 as builtin (CONFIG_DRM_I915=y) and
compare drivers/gpu/drm/i915/.built-in.a.cmd with/without this commit.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
---
Changes in v7:
- Add comment to the code
Changes in v2:
- New patch
scripts/Makefile.build | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 0135b4e7f78b..014ebee70fd6 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -376,9 +376,14 @@ $(subdir-modorder): $(obj)/%/modules.order: $(obj)/% ;
#
# Rule to compile a set of .o files into one .a file (without symbol table)
#
+# To make this rule robust against "Argument list too long" error,
+# remove $(obj)/ prefix, and restore it by a shell command.
quiet_cmd_ar_builtin = AR $@
- cmd_ar_builtin = rm -f $@; $(AR) cDPrST $@ $(real-prereqs)
+ cmd_ar_builtin = rm -f $@; \
+ echo $(patsubst $(obj)/%,%,$(real-prereqs)) | \
+ sed -E 's:([^ ]+):$(obj)/\1:g' | \
+ xargs $(AR) cDPrST $@
$(obj)/built-in.a: $(real-obj-y) FORCE
$(call if_changed,ar_builtin)
--
2.32.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH v7 6/8] kbuild: make *.mod rule robust against too long argument error
2022-05-27 10:01 [PATCH v7 0/8] kbuild: yet another series of cleanups Masahiro Yamada
` (4 preceding siblings ...)
2022-05-27 10:01 ` [PATCH v7 5/8] kbuild: make built-in.a rule robust against too long argument error Masahiro Yamada
@ 2022-05-27 10:01 ` Masahiro Yamada
2022-05-27 10:01 ` [PATCH v7 7/8] kbuild: add cmd_and_savecmd macro Masahiro Yamada
2022-05-27 10:01 ` [PATCH v7 8/8] kbuild: rebuild multi-object modules when objtool is updated Masahiro Yamada
7 siblings, 0 replies; 18+ messages in thread
From: Masahiro Yamada @ 2022-05-27 10:01 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm, Helge Deller,
linux-parisc, Masahiro Yamada, Nicolas Schier, Nathan Chancellor,
Sedat Dilek, Michal Marek, Nick Desaulniers, Tom Rix
Like built-in.a, the command length of the *.mod rule scales with
the depth of the directory times the number of objects in the Makefile.
Add $(obj)/ by the shell command (awk) instead of by Make's builtin
function.
In-tree modules still have some room to the limit (ARG_MAX=2097152),
but this is more future-proof for big modules in a deep directory.
For example, you can build i915 as a module (CONFIG_DRM_I915=m) and
compare drivers/gpu/drm/i915/.i915.mod.cmd with/without this commit.
The issue is more critical for external modules because the M= path
can be very long as Jeff Johnson reported before [1].
[1] https://lore.kernel.org/linux-kbuild/4c02050c4e95e4cb8cc04282695f8404@codeaurora.org/
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
---
Changes in v7:
- Add comment to the code
scripts/Makefile.build | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 014ebee70fd6..31feb798e16e 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -274,8 +274,10 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE
$(call if_changed_rule,cc_o_c)
$(call cmd,force_checksrc)
-cmd_mod = echo $(addprefix $(obj)/, $(call real-search, $*.o, .o, -objs -y -m)) | \
- $(AWK) -v RS='( |\n)' '!x[$$0]++' > $@
+# To make this rule robust against "Argument list too long" error,
+# ensure to add $(obj)/ prefix by a shell command.
+cmd_mod = echo $(call real-search, $*.o, .o, -objs -y -m) | \
+ $(AWK) -v RS='( |\n)' '!x[$$0]++ { print("$(obj)/"$$0) }' > $@
$(obj)/%.mod: FORCE
$(call if_changed,mod)
--
2.32.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH v7 7/8] kbuild: add cmd_and_savecmd macro
2022-05-27 10:01 [PATCH v7 0/8] kbuild: yet another series of cleanups Masahiro Yamada
` (5 preceding siblings ...)
2022-05-27 10:01 ` [PATCH v7 6/8] kbuild: make *.mod " Masahiro Yamada
@ 2022-05-27 10:01 ` Masahiro Yamada
2022-05-27 10:01 ` [PATCH v7 8/8] kbuild: rebuild multi-object modules when objtool is updated Masahiro Yamada
7 siblings, 0 replies; 18+ messages in thread
From: Masahiro Yamada @ 2022-05-27 10:01 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm, Helge Deller,
linux-parisc, Masahiro Yamada, Kees Cook, Nathan Chancellor,
Nicolas Schier, Sedat Dilek, Michal Marek, Nick Desaulniers,
Tom Rix
Separate out the command execution part of if_changed, as we did
for if_changed_dep.
This allows us to reuse it in if_changed_rule.
define rule_foo
$(call cmd_and_savecmd,foo)
$(call cmd,bar)
endef
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
---
(no changes since v4)
Changes in v4:
- New.
Resent of my previous submission.
https://lore.kernel.org/all/20210831074004.3195284-10-masahiroy@kernel.org/
scripts/Kbuild.include | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index 455a0a6ce12d..ece44b735061 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -142,9 +142,11 @@ check-FORCE = $(if $(filter FORCE, $^),,$(warning FORCE prerequisite is missing)
if-changed-cond = $(newer-prereqs)$(cmd-check)$(check-FORCE)
# Execute command if command has changed or prerequisite(s) are updated.
-if_changed = $(if $(if-changed-cond), \
+if_changed = $(if $(if-changed-cond),$(cmd_and_savecmd),@:)
+
+cmd_and_savecmd = \
$(cmd); \
- printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:)
+ printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd
# Execute the command and also postprocess generated .d dependencies file.
if_changed_dep = $(if $(if-changed-cond),$(cmd_and_fixdep),@:)
--
2.32.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* [PATCH v7 8/8] kbuild: rebuild multi-object modules when objtool is updated
2022-05-27 10:01 [PATCH v7 0/8] kbuild: yet another series of cleanups Masahiro Yamada
` (6 preceding siblings ...)
2022-05-27 10:01 ` [PATCH v7 7/8] kbuild: add cmd_and_savecmd macro Masahiro Yamada
@ 2022-05-27 10:01 ` Masahiro Yamada
2022-05-27 15:50 ` Josh Poimboeuf
7 siblings, 1 reply; 18+ messages in thread
From: Masahiro Yamada @ 2022-05-27 10:01 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm, Helge Deller,
linux-parisc, Masahiro Yamada, Kees Cook, Nathan Chancellor,
Nicolas Schier, Sedat Dilek, Michal Marek, Nick Desaulniers,
Tom Rix
When CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT is enabled, objtool for
multi-object modules is postponed until the objects are linked together.
Make sure to re-run objtool and re-link multi-object modules when
objtool is updated.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
---
(no changes since v4)
Changes in v4:
- New
Resent of my previous submission
https://lore.kernel.org/linux-kbuild/20210831074004.3195284-11-masahiroy@kernel.org/
scripts/Makefile.build | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 31feb798e16e..bd5fc2b37387 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -412,13 +412,18 @@ $(obj)/modules.order: $(obj-m) FORCE
$(obj)/lib.a: $(lib-y) FORCE
$(call if_changed,ar)
-quiet_cmd_link_multi-m = LD [M] $@
- cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool)
+quiet_cmd_ld_multi_m = LD [M] $@
+ cmd_ld_multi_m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool)
+
+define rule_ld_multi_m
+ $(call cmd_and_savecmd,ld_multi_m)
+ $(call cmd,gen_objtooldep)
+endef
$(multi-obj-m): objtool-enabled := $(delay-objtool)
$(multi-obj-m): part-of-module := y
$(multi-obj-m): %.o: %.mod FORCE
- $(call if_changed,link_multi-m)
+ $(call if_changed_rule,ld_multi_m)
$(call multi_depend, $(multi-obj-m), .o, -objs -y -m)
targets := $(filter-out $(PHONY), $(targets))
--
2.32.0
^ permalink raw reply related [flat|nested] 18+ messages in thread* Re: [PATCH v7 8/8] kbuild: rebuild multi-object modules when objtool is updated
2022-05-27 10:01 ` [PATCH v7 8/8] kbuild: rebuild multi-object modules when objtool is updated Masahiro Yamada
@ 2022-05-27 15:50 ` Josh Poimboeuf
2022-05-27 17:41 ` Masahiro Yamada
0 siblings, 1 reply; 18+ messages in thread
From: Josh Poimboeuf @ 2022-05-27 15:50 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, linux-kernel, Peter Zijlstra, Josh Poimboeuf, llvm,
Helge Deller, linux-parisc, Kees Cook, Nathan Chancellor,
Nicolas Schier, Sedat Dilek, Michal Marek, Nick Desaulniers,
Tom Rix
On Fri, May 27, 2022 at 07:01:55PM +0900, Masahiro Yamada wrote:
> When CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT is enabled, objtool for
> multi-object modules is postponed until the objects are linked together.
>
> Make sure to re-run objtool and re-link multi-object modules when
> objtool is updated.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> Reviewed-by: Kees Cook <keescook@chromium.org>
> Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
> Tested-by: Nathan Chancellor <nathan@kernel.org>
> Reviewed-by: Nicolas Schier <n.schier@avm.de>
> Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
Hi Masahiro,
Is it also possible to rebuild vmlinux.o when objtool changes, for
CONFIG_LTO_CLANG, CONFIG_X86_KERNEL_IBT, or CONFIG_NOINSTR_VALIDATION?
That doesn't seem to be happening.
--
Josh
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH v7 8/8] kbuild: rebuild multi-object modules when objtool is updated
2022-05-27 15:50 ` Josh Poimboeuf
@ 2022-05-27 17:41 ` Masahiro Yamada
0 siblings, 0 replies; 18+ messages in thread
From: Masahiro Yamada @ 2022-05-27 17:41 UTC (permalink / raw)
To: Josh Poimboeuf
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Peter Zijlstra, Josh Poimboeuf, clang-built-linux, Helge Deller,
Parisc List, Kees Cook, Nathan Chancellor, Nicolas Schier,
Sedat Dilek, Michal Marek, Nick Desaulniers, Tom Rix
On Sat, May 28, 2022 at 12:50 AM Josh Poimboeuf <jpoimboe@kernel.org> wrote:
>
> On Fri, May 27, 2022 at 07:01:55PM +0900, Masahiro Yamada wrote:
> > When CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT is enabled, objtool for
> > multi-object modules is postponed until the objects are linked together.
> >
> > Make sure to re-run objtool and re-link multi-object modules when
> > objtool is updated.
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > Reviewed-by: Kees Cook <keescook@chromium.org>
> > Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
> > Tested-by: Nathan Chancellor <nathan@kernel.org>
> > Reviewed-by: Nicolas Schier <n.schier@avm.de>
> > Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
>
> Hi Masahiro,
>
> Is it also possible to rebuild vmlinux.o when objtool changes, for
> CONFIG_LTO_CLANG, CONFIG_X86_KERNEL_IBT, or CONFIG_NOINSTR_VALIDATION?
> That doesn't seem to be happening.
>
> --
> Josh
Yes.
Actually, it is already on my TODO list,
but it requires more effort.
I will aim for v5.20.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 18+ messages in thread