* [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
@ 2022-05-13 11:39 Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 01/10] modpost: extract symbol versions from *.cmd files Masahiro Yamada
` (10 more replies)
0 siblings, 11 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-13 11:39 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, Masahiro Yamada
This is the third batch of cleanups in this development cycle.
Changes in v6:
- Fix false-positive warnings when CONFIG_TRIM_UNUSED_KSYMS=y
Changes in v5:
- Fix the build error when CONFIG_DEBUG_INFO_BTF=y (reported by Nathan)
- Clean up arch/m68k/include/asm/export.h (Nick)
- Keep gen_symversions (and will be removed by a later patch)
- Add more comments in the script
Changes in v4:
- Rename .vmlinux-symver.c to .vmlinux.export.c
because I notice this approach is useful for further cleanups,
not only for modversioning but also for overall EXPORT_SYMBOL.
- New patch
- New.
Resent of my previous submission.
https://lore.kernel.org/all/20210831074004.3195284-10-masahiroy@kernel.org/
- New
Resent of my previous submission
https://lore.kernel.org/linux-kbuild/20210831074004.3195284-11-masahiroy@kernel.org/
Changes in v3:
- New patch
Changes in v2:
- Simplify the implementation (parse .cmd files after ELF)
- New patch
- replace the chain of $(if ...) with $(and )
- New patch
- New patch
Masahiro Yamada (10):
modpost: extract symbol versions from *.cmd files
kbuild: link symbol CRCs at final link, removing
CONFIG_MODULE_REL_CRCS
kbuild: stop merging *.symversions
genksyms: adjust the output format to modpost
kbuild: do not create *.prelink.o for Clang LTO or IBT
kbuild: check static EXPORT_SYMBOL* by script instead of modpost
kbuild: make built-in.a rule robust against too long argument error
kbuild: make *.mod rule robust against too long argument error
kbuild: add cmd_and_savecmd macro
kbuild: rebuild multi-object modules when objtool is updated
arch/m68k/include/asm/Kbuild | 1 +
arch/m68k/include/asm/export.h | 2 -
arch/powerpc/Kconfig | 1 -
arch/s390/Kconfig | 1 -
arch/um/Kconfig | 1 -
include/asm-generic/export.h | 22 ++-
include/linux/export-internal.h | 16 +++
include/linux/export.h | 30 ++--
init/Kconfig | 4 -
kernel/module.c | 10 +-
scripts/Kbuild.include | 10 +-
scripts/Makefile.build | 134 ++++++------------
scripts/Makefile.lib | 7 -
scripts/Makefile.modfinal | 5 +-
scripts/Makefile.modpost | 9 +-
scripts/check-local-export | 64 +++++++++
scripts/genksyms/genksyms.c | 18 +--
scripts/link-vmlinux.sh | 33 ++---
scripts/mod/modpost.c | 236 +++++++++++++++++++++-----------
19 files changed, 320 insertions(+), 284 deletions(-)
delete mode 100644 arch/m68k/include/asm/export.h
create mode 100644 include/linux/export-internal.h
create mode 100755 scripts/check-local-export
--
2.32.0
^ permalink raw reply [flat|nested] 39+ messages in thread
* [PATCH v6 01/10] modpost: extract symbol versions from *.cmd files
2022-05-13 11:39 [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
@ 2022-05-13 11:39 ` Masahiro Yamada
2022-05-28 22:47 ` Guenter Roeck
2022-05-13 11:39 ` [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS Masahiro Yamada
` (9 subsequent siblings)
10 siblings, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-13 11:39 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, Masahiro Yamada
Currently, CONFIG_MODVERSIONS needs extra link to embed the symbol
versions into ELF objects. Then, modpost extracts the version CRCs
from them.
The following figures show how it currently works, and how I am trying
to change it.
Current implementation
======================
|----------|
embed CRC -------------------------->| final |
$(CC) $(LD) / |---------| | link for |
-----> *.o -------> *.o -->| modpost | | vmlinux |
/ / | |-- *.mod.c -->| or |
/ genksyms / |---------| | module |
*.c ------> *.symversions |----------|
Genksyms outputs the calculated CRCs in the form of linker script
(*.symversions), which is used by $(LD) to update the object.
If CONFIG_LTO_CLANG=y, the build process is much more complex. Embedding
the CRCs is postponed until the LLVM bitcode is converted into ELF,
creating another intermediate *.prelink.o.
However, this complexity is unneeded. There is no reason why we must
embed version CRCs in objects so early.
There is final link stage for vmlinux (scripts/link-vmlinux.sh) and
modules (scripts/Makefile.modfinal). We can link CRCs at the very last
moment.
New implementation
==================
|----------|
--------------------------------------->| final |
$(CC) / |---------| | link for |
-----> *.o ---->| | | vmlinux |
/ | modpost |--- .vmlinux.export.c -->| or |
/ genksyms | |--- *.mod.c ------------>| module |
*.c ------> *.cmd -->|---------| |----------|
Pass the symbol versions to modpost as separate text data, which are
available in *.cmd files.
This commit changes modpost to extract CRCs from *.cmd files instead of
from ELF objects.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
---
Changes in v6:
- Fix false-positive warnings when CONFIG_TRIM_UNUSED_KSYMS=y
Changes in v2:
- Simplify the implementation (parse .cmd files after ELF)
scripts/mod/modpost.c | 179 +++++++++++++++++++++++++++++++-----------
1 file changed, 131 insertions(+), 48 deletions(-)
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index fc5db1f73cf1..213a130d1635 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -381,19 +381,10 @@ static struct symbol *sym_add_exported(const char *name, struct module *mod,
return s;
}
-static void sym_set_crc(const char *name, unsigned int crc)
+static void sym_set_crc(struct symbol *sym, unsigned int crc)
{
- struct symbol *s = find_symbol(name);
-
- /*
- * Ignore stand-alone __crc_*, which might be auto-generated symbols
- * such as __*_veneer in ARM ELF.
- */
- if (!s)
- return;
-
- s->crc = crc;
- s->crc_valid = true;
+ sym->crc = crc;
+ sym->crc_valid = true;
}
static void *grab_file(const char *filename, size_t *size)
@@ -616,33 +607,6 @@ static int ignore_undef_symbol(struct elf_info *info, const char *symname)
return 0;
}
-static void handle_modversion(const struct module *mod,
- const struct elf_info *info,
- const Elf_Sym *sym, const char *symname)
-{
- unsigned int crc;
-
- if (sym->st_shndx == SHN_UNDEF) {
- warn("EXPORT symbol \"%s\" [%s%s] version generation failed, symbol will not be versioned.\n"
- "Is \"%s\" prototyped in <asm/asm-prototypes.h>?\n",
- symname, mod->name, mod->is_vmlinux ? "" : ".ko",
- symname);
-
- return;
- }
-
- if (sym->st_shndx == SHN_ABS) {
- crc = sym->st_value;
- } else {
- unsigned int *crcp;
-
- /* symbol points to the CRC in the ELF object */
- crcp = sym_get_data(info, sym);
- crc = TO_NATIVE(*crcp);
- }
- sym_set_crc(symname, crc);
-}
-
static void handle_symbol(struct module *mod, struct elf_info *info,
const Elf_Sym *sym, const char *symname)
{
@@ -760,6 +724,104 @@ static char *remove_dot(char *s)
return s;
}
+/*
+ * The CRCs are recorded in .*.cmd files in the form of:
+ * #SYMVER <name> <crc>
+ */
+static void extract_crcs_for_object(const char *object, struct module *mod)
+{
+ char cmd_file[PATH_MAX];
+ char *buf, *p;
+ const char *base;
+ int dirlen, ret;
+
+ base = strrchr(object, '/');
+ if (base) {
+ base++;
+ dirlen = base - object;
+ } else {
+ dirlen = 0;
+ base = object;
+ }
+
+ ret = snprintf(cmd_file, sizeof(cmd_file), "%.*s.%s.cmd",
+ dirlen, object, base);
+ if (ret >= sizeof(cmd_file)) {
+ error("%s: too long path was truncated\n", cmd_file);
+ return;
+ }
+
+ buf = read_text_file(cmd_file);
+ p = buf;
+
+ while ((p = strstr(p, "\n#SYMVER "))) {
+ char *name;
+ size_t namelen;
+ unsigned int crc;
+ struct symbol *sym;
+
+ name = p + strlen("\n#SYMVER ");
+
+ p = strchr(name, ' ');
+ if (!p)
+ break;
+
+ namelen = p - name;
+ p++;
+
+ if (!isdigit(*p))
+ continue; /* skip this line */
+
+ crc = strtol(p, &p, 0);
+ if (*p != '\n')
+ continue; /* skip this line */
+
+ name[namelen] = '\0';
+
+ /*
+ * sym_find_with_module() may return NULL here.
+ * It typically occurs when CONFIG_TRIM_UNUSED_KSYMS=y.
+ * Since commit e1327a127703, genksyms calculates CRCs of all
+ * symbols, including trimmed ones. Ignore orphan CRCs.
+ */
+ sym = sym_find_with_module(name, mod);
+ if (sym)
+ sym_set_crc(sym, crc);
+ }
+
+ free(buf);
+}
+
+/*
+ * The symbol versions (CRC) are recorded in the .*.cmd files.
+ * Parse them to retrieve CRCs for the current module.
+ */
+static void mod_set_crcs(struct module *mod)
+{
+ char objlist[PATH_MAX];
+ char *buf, *p, *obj;
+ int ret;
+
+ if (mod->is_vmlinux) {
+ strcpy(objlist, ".vmlinux.objs");
+ } else {
+ /* objects for a module are listed in the *.mod file. */
+ ret = snprintf(objlist, sizeof(objlist), "%s.mod", mod->name);
+ if (ret >= sizeof(objlist)) {
+ error("%s: too long path was truncated\n", objlist);
+ return;
+ }
+ }
+
+ buf = read_text_file(objlist);
+ p = buf;
+
+ while ((obj = strsep(&p, "\n")) && obj[0])
+ extract_crcs_for_object(obj, mod);
+
+ free(buf);
+}
+
static void read_symbols(const char *modname)
{
const char *symname;
@@ -820,9 +882,6 @@ static void read_symbols(const char *modname)
if (strstarts(symname, "__kstrtabns_"))
sym_update_namespace(symname + strlen("__kstrtabns_"),
sym_get_data(&info, sym));
- if (strstarts(symname, "__crc_"))
- handle_modversion(mod, &info, sym,
- symname + strlen("__crc_"));
}
// check for static EXPORT_SYMBOL_* functions && global vars
@@ -850,12 +909,17 @@ static void read_symbols(const char *modname)
parse_elf_finish(&info);
- /* Our trick to get versioning for module struct etc. - it's
- * never passed as an argument to an exported function, so
- * the automatic versioning doesn't pick it up, but it's really
- * important anyhow */
- if (modversions)
+ if (modversions) {
+ /*
+ * Our trick to get versioning for module struct etc. - it's
+ * never passed as an argument to an exported function, so
+ * the automatic versioning doesn't pick it up, but it's really
+ * important anyhow.
+ */
sym_add_unresolved("module_layout", mod, false);
+
+ mod_set_crcs(mod);
+ }
}
static void read_symbols_from_files(const char *filename)
@@ -1012,6 +1076,23 @@ static void add_header(struct buffer *b, struct module *mod)
buf_printf(b, "\nMODULE_INFO(staging, \"Y\");\n");
}
+static void check_symversions(struct module *mod)
+{
+ struct symbol *sym;
+
+ if (!modversions)
+ return;
+
+ list_for_each_entry(sym, &mod->exported_symbols, list) {
+ if (!sym->crc_valid) {
+ warn("EXPORT symbol \"%s\" [%s%s] version generation failed, symbol will not be versioned.\n"
+ "Is \"%s\" prototyped in <asm/asm-prototypes.h>?\n",
+ sym->name, mod->name, mod->is_vmlinux ? "" : ".ko",
+ sym->name);
+ }
+ }
+}
+
/**
* Record CRCs for unresolved symbols
**/
@@ -1227,7 +1308,7 @@ static void read_dump(const char *fname)
}
s = sym_add_exported(symname, mod, gpl_only);
s->is_static = false;
- sym_set_crc(symname, crc);
+ sym_set_crc(s, crc);
sym_update_namespace(symname, namespace);
}
free(buf);
@@ -1353,6 +1434,8 @@ int main(int argc, char **argv)
if (mod->from_dump)
continue;
+ check_symversions(mod);
+
if (!mod->is_vmlinux)
write_mod_c_file(mod);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS
2022-05-13 11:39 [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 01/10] modpost: extract symbol versions from *.cmd files Masahiro Yamada
@ 2022-05-13 11:39 ` Masahiro Yamada
2022-08-20 10:02 ` Build/boot problem with 7b4537199a4a (Re: [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS) Christophe Leroy
2022-05-13 11:39 ` [PATCH v6 03/10] kbuild: stop merging *.symversions Masahiro Yamada
` (8 subsequent siblings)
10 siblings, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-13 11:39 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, Masahiro Yamada
include/{linux,asm-generic}/export.h defines a weak symbol, __crc_*
as a placeholder.
Genksyms writes the version CRCs into the linker script, which will be
used for filling the __crc_* symbols. The linker script format depends
on CONFIG_MODULE_REL_CRCS. If it is enabled, __crc_* holds the offset
to the reference of CRC.
It is time to get rid of this complexity.
Now that modpost parses text files (.*.cmd) to collect all the CRCs,
it can generate C code that will be linked to the vmlinux or modules.
Generate a new C file, .vmlinux.export.c, which contains the CRCs of
symbols exported by vmlinux. It is compiled and linked to vmlinux in
scripts/link-vmlinux.sh.
Put the CRCs of symbols exported by modules into the existing *.mod.c
files. No additional build step is needed for modules. As before,
*.mod.c are compiled and linked to *.ko in scripts/Makefile.modfinal.
No linker magic is used here. The new C implementation works in the
same way, whether CONFIG_RELOCATABLE is enabled or not.
CONFIG_MODULE_REL_CRCS is no longer needed.
Previously, Kbuild invoked additional $(LD) to update the CRCs in
objects, but this step is unneeded too.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Nicolas Schier <nicolas@fjasle.eu>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
---
(no changes since v5)
Changes in v5:
- Fix the build error when CONFIG_DEBUG_INFO_BTF=y (reported by Nathan)
- Clean up arch/m68k/include/asm/export.h (Nick)
- Keep gen_symversions (and will be removed by a later patch)
Changes in v4:
- Rename .vmlinux-symver.c to .vmlinux.export.c
because I notice this approach is useful for further cleanups,
not only for modversioning but also for overall EXPORT_SYMBOL.
Changes in v3:
- New patch
arch/m68k/include/asm/Kbuild | 1 +
arch/m68k/include/asm/export.h | 2 --
arch/powerpc/Kconfig | 1 -
arch/s390/Kconfig | 1 -
arch/um/Kconfig | 1 -
include/asm-generic/export.h | 22 ++++++++--------------
include/linux/export-internal.h | 16 ++++++++++++++++
include/linux/export.h | 30 ++++++++----------------------
init/Kconfig | 4 ----
kernel/module.c | 10 +---------
scripts/Makefile.build | 27 ++++-----------------------
scripts/genksyms/genksyms.c | 18 ++++--------------
scripts/link-vmlinux.sh | 14 +++++++++++++-
scripts/mod/modpost.c | 28 ++++++++++++++++++++++++----
14 files changed, 79 insertions(+), 96 deletions(-)
delete mode 100644 arch/m68k/include/asm/export.h
create mode 100644 include/linux/export-internal.h
diff --git a/arch/m68k/include/asm/Kbuild b/arch/m68k/include/asm/Kbuild
index 0dbf9c5c6fae..1b720299deb1 100644
--- a/arch/m68k/include/asm/Kbuild
+++ b/arch/m68k/include/asm/Kbuild
@@ -1,5 +1,6 @@
# SPDX-License-Identifier: GPL-2.0
generated-y += syscall_table.h
+generic-y += export.h
generic-y += extable.h
generic-y += kvm_para.h
generic-y += mcs_spinlock.h
diff --git a/arch/m68k/include/asm/export.h b/arch/m68k/include/asm/export.h
deleted file mode 100644
index b53008b67ce1..000000000000
--- a/arch/m68k/include/asm/export.h
+++ /dev/null
@@ -1,2 +0,0 @@
-#define KCRC_ALIGN 2
-#include <asm-generic/export.h>
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 174edabb74fa..a4e8dd889e29 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -566,7 +566,6 @@ config RELOCATABLE
bool "Build a relocatable kernel"
depends on PPC64 || (FLATMEM && (44x || FSL_BOOKE))
select NONSTATIC_KERNEL
- select MODULE_REL_CRCS if MODVERSIONS
help
This builds a kernel image that is capable of running at the
location the kernel is loaded at. For ppc32, there is no any
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index 77b5a03de13a..aa5848004c76 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -567,7 +567,6 @@ endchoice
config RELOCATABLE
bool "Build a relocatable kernel"
- select MODULE_REL_CRCS if MODVERSIONS
default y
help
This builds a kernel image that retains relocation information
diff --git a/arch/um/Kconfig b/arch/um/Kconfig
index 4d398b80aea8..e8983d098e73 100644
--- a/arch/um/Kconfig
+++ b/arch/um/Kconfig
@@ -106,7 +106,6 @@ config LD_SCRIPT_DYN
bool
default y
depends on !LD_SCRIPT_STATIC
- select MODULE_REL_CRCS if MODVERSIONS
config LD_SCRIPT_DYN_RPATH
bool "set rpath in the binary" if EXPERT
diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
index 07a36a874dca..5e4b1f2369d2 100644
--- a/include/asm-generic/export.h
+++ b/include/asm-generic/export.h
@@ -2,6 +2,14 @@
#ifndef __ASM_GENERIC_EXPORT_H
#define __ASM_GENERIC_EXPORT_H
+/*
+ * This comment block is used by fixdep. Please do not remove.
+ *
+ * When CONFIG_MODVERSIONS is changed from n to y, all source files having
+ * EXPORT_SYMBOL variants must be re-compiled because genksyms is run as a
+ * side effect of the *.o build rule.
+ */
+
#ifndef KSYM_FUNC
#define KSYM_FUNC(x) x
#endif
@@ -12,9 +20,6 @@
#else
#define KSYM_ALIGN 4
#endif
-#ifndef KCRC_ALIGN
-#define KCRC_ALIGN 4
-#endif
.macro __put, val, name
#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
@@ -43,17 +48,6 @@ __ksymtab_\name:
__kstrtab_\name:
.asciz "\name"
.previous
-#ifdef CONFIG_MODVERSIONS
- .section ___kcrctab\sec+\name,"a"
- .balign KCRC_ALIGN
-#if defined(CONFIG_MODULE_REL_CRCS)
- .long __crc_\name - .
-#else
- .long __crc_\name
-#endif
- .weak __crc_\name
- .previous
-#endif
#endif
.endm
diff --git a/include/linux/export-internal.h b/include/linux/export-internal.h
new file mode 100644
index 000000000000..77175d561058
--- /dev/null
+++ b/include/linux/export-internal.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Please do not include this explicitly.
+ * This is used by C files generated by modpost.
+ */
+
+#ifndef __LINUX_EXPORT_INTERNAL_H__
+#define __LINUX_EXPORT_INTERNAL_H__
+
+#include <linux/compiler.h>
+#include <linux/types.h>
+
+#define SYMBOL_CRC(sym, crc, sec) \
+ u32 __section("___kcrctab" sec "+" #sym) __crc_##sym = crc
+
+#endif /* __LINUX_EXPORT_INTERNAL_H__ */
diff --git a/include/linux/export.h b/include/linux/export.h
index 27d848712b90..565c5ffcb26f 100644
--- a/include/linux/export.h
+++ b/include/linux/export.h
@@ -11,6 +11,14 @@
* hackers place grumpy comments in header files.
*/
+/*
+ * This comment block is used by fixdep. Please do not remove.
+ *
+ * When CONFIG_MODVERSIONS is changed from n to y, all source files having
+ * EXPORT_SYMBOL variants must be re-compiled because genksyms is run as a
+ * side effect of the *.o build rule.
+ */
+
#ifndef __ASSEMBLY__
#ifdef MODULE
extern struct module __this_module;
@@ -19,26 +27,6 @@ extern struct module __this_module;
#define THIS_MODULE ((struct module *)0)
#endif
-#ifdef CONFIG_MODVERSIONS
-/* Mark the CRC weak since genksyms apparently decides not to
- * generate a checksums for some symbols */
-#if defined(CONFIG_MODULE_REL_CRCS)
-#define __CRC_SYMBOL(sym, sec) \
- asm(" .section \"___kcrctab" sec "+" #sym "\", \"a\" \n" \
- " .weak __crc_" #sym " \n" \
- " .long __crc_" #sym " - . \n" \
- " .previous \n")
-#else
-#define __CRC_SYMBOL(sym, sec) \
- asm(" .section \"___kcrctab" sec "+" #sym "\", \"a\" \n" \
- " .weak __crc_" #sym " \n" \
- " .long __crc_" #sym " \n" \
- " .previous \n")
-#endif
-#else
-#define __CRC_SYMBOL(sym, sec)
-#endif
-
#ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
#include <linux/compiler.h>
/*
@@ -85,7 +73,6 @@ struct kernel_symbol {
/*
* For every exported symbol, do the following:
*
- * - If applicable, place a CRC entry in the __kcrctab section.
* - Put the name of the symbol and namespace (empty string "" for none) in
* __ksymtab_strings.
* - Place a struct kernel_symbol entry in the __ksymtab section.
@@ -98,7 +85,6 @@ struct kernel_symbol {
extern typeof(sym) sym; \
extern const char __kstrtab_##sym[]; \
extern const char __kstrtabns_##sym[]; \
- __CRC_SYMBOL(sym, sec); \
asm(" .section \"__ksymtab_strings\",\"aMS\",%progbits,1 \n" \
"__kstrtab_" #sym ": \n" \
" .asciz \"" #sym "\" \n" \
diff --git a/init/Kconfig b/init/Kconfig
index ddcbefe535e9..f5b14318dfcb 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2136,10 +2136,6 @@ config ASM_MODVERSIONS
assembly. This can be enabled only when the target architecture
supports it.
-config MODULE_REL_CRCS
- bool
- depends on MODVERSIONS
-
config MODULE_SRCVERSION_ALL
bool "Source checksum for all modules"
help
diff --git a/kernel/module.c b/kernel/module.c
index 6cea788fd965..c9e2342da28e 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -1231,11 +1231,6 @@ static int try_to_force_load(struct module *mod, const char *reason)
#ifdef CONFIG_MODVERSIONS
-static u32 resolve_rel_crc(const s32 *crc)
-{
- return *(u32 *)((void *)crc + *crc);
-}
-
static int check_version(const struct load_info *info,
const char *symname,
struct module *mod,
@@ -1264,10 +1259,7 @@ static int check_version(const struct load_info *info,
if (strcmp(versions[i].name, symname) != 0)
continue;
- if (IS_ENABLED(CONFIG_MODULE_REL_CRCS))
- crcval = resolve_rel_crc(crc);
- else
- crcval = *crc;
+ crcval = *crc;
if (versions[i].crc == crcval)
return 1;
pr_debug("Found checksum %X vs module %lX\n",
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index a1023868775f..ddd9080fc028 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -128,7 +128,6 @@ $(obj)/%.i: $(src)/%.c FORCE
genksyms = scripts/genksyms/genksyms \
$(if $(1), -T $(2)) \
- $(if $(CONFIG_MODULE_REL_CRCS), -R) \
$(if $(KBUILD_PRESERVE), -p) \
-r $(or $(wildcard $(2:.symtypes=.symref)), /dev/null)
@@ -162,19 +161,11 @@ ifdef CONFIG_MODVERSIONS
# o if <file>.o doesn't contain a __ksymtab version, i.e. does
# not export symbols, it's done.
# o otherwise, we calculate symbol versions using the good old
-# genksyms on the preprocessed source and postprocess them in a way
-# that they are usable as a linker script
-# o generate .tmp_<file>.o from <file>.o using the linker to
-# replace the unresolved symbols __crc_exported_symbol with
-# the actual value of the checksum generated by genksyms
-# o remove .tmp_<file>.o to <file>.o
+# genksyms on the preprocessed source and dump them into the .cmd file.
+# o modpost will extract versions from that file and create *.c files that will
+# be compiled and linked to the kernel and/or modules.
-# Generate .o.symversions files for each .o with exported symbols, and link these
-# to the kernel and/or modules at the end.
-
-genksyms_format_rel_crc := [^_]*__crc_\([^ ]*\) = \.; LONG(\([^)]*\)).*
-genksyms_format_normal := __crc_\(.*\) = \(.*\);
-genksyms_format := $(if $(CONFIG_MODULE_REL_CRCS),$(genksyms_format_rel_crc),$(genksyms_format_normal))
+genksyms_format := __crc_\(.*\) = \(.*\);
gen_symversions = \
if $(NM) $@ 2>/dev/null | grep -q __ksymtab; then \
@@ -188,12 +179,6 @@ gen_symversions = \
cmd_gen_symversions_c = $(call gen_symversions,c)
-cmd_modversions = \
- if [ -r $@.symversions ]; then \
- $(LD) $(KBUILD_LDFLAGS) -r -o $(@D)/.tmp_$(@F) $@ \
- -T $@.symversions; \
- mv -f $(@D)/.tmp_$(@F) $@; \
- fi
endif
ifdef CONFIG_FTRACE_MCOUNT_USE_RECORDMCOUNT
@@ -273,7 +258,6 @@ define rule_cc_o_c
$(call cmd,checkdoc)
$(call cmd,gen_objtooldep)
$(call cmd,gen_symversions_c)
- $(if $(CONFIG_LTO_CLANG),,$(call cmd,modversions))
$(call cmd,record_mcount)
endef
@@ -282,7 +266,6 @@ define rule_as_o_S
$(call cmd,gen_ksymdeps)
$(call cmd,gen_objtooldep)
$(call cmd,gen_symversions_S)
- $(call cmd,modversions)
endef
# Built-in and composite module parts
@@ -296,8 +279,6 @@ ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
quiet_cmd_cc_prelink_modules = LD [M] $@
cmd_cc_prelink_modules = \
$(LD) $(ld_flags) -r -o $@ \
- $(shell [ -s $(@:.prelink.o=.o.symversions) ] && \
- echo -T $(@:.prelink.o=.o.symversions)) \
--whole-archive $(filter-out FORCE,$^) \
$(cmd_objtool)
diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c
index 4827c5abe5b7..67b23cc0df0f 100644
--- a/scripts/genksyms/genksyms.c
+++ b/scripts/genksyms/genksyms.c
@@ -33,7 +33,7 @@ char *cur_filename;
int in_source_file;
static int flag_debug, flag_dump_defs, flag_reference, flag_dump_types,
- flag_preserve, flag_warnings, flag_rel_crcs;
+ flag_preserve, flag_warnings;
static int errors;
static int nsyms;
@@ -680,11 +680,7 @@ void export_symbol(const char *name)
if (flag_dump_defs)
fputs(">\n", debugfile);
- /* Used as a linker script. */
- printf(!flag_rel_crcs ? "__crc_%s = 0x%08lx;\n" :
- "SECTIONS { .rodata : ALIGN(4) { "
- "__crc_%s = .; LONG(0x%08lx); } }\n",
- name, crc);
+ printf("__crc_%s = 0x%08lx;\n", name, crc);
}
}
@@ -733,7 +729,6 @@ static void genksyms_usage(void)
" -q, --quiet Disable warnings (default)\n"
" -h, --help Print this message\n"
" -V, --version Print the release version\n"
- " -R, --relative-crc Emit section relative symbol CRCs\n"
#else /* __GNU_LIBRARY__ */
" -s Select symbol prefix\n"
" -d Increment the debug level (repeatable)\n"
@@ -745,7 +740,6 @@ static void genksyms_usage(void)
" -q Disable warnings (default)\n"
" -h Print this message\n"
" -V Print the release version\n"
- " -R Emit section relative symbol CRCs\n"
#endif /* __GNU_LIBRARY__ */
, stderr);
}
@@ -766,14 +760,13 @@ int main(int argc, char **argv)
{"preserve", 0, 0, 'p'},
{"version", 0, 0, 'V'},
{"help", 0, 0, 'h'},
- {"relative-crc", 0, 0, 'R'},
{0, 0, 0, 0}
};
- while ((o = getopt_long(argc, argv, "s:dwqVDr:T:phR",
+ while ((o = getopt_long(argc, argv, "s:dwqVDr:T:ph",
&long_opts[0], NULL)) != EOF)
#else /* __GNU_LIBRARY__ */
- while ((o = getopt(argc, argv, "s:dwqVDr:T:phR")) != EOF)
+ while ((o = getopt(argc, argv, "s:dwqVDr:T:ph")) != EOF)
#endif /* __GNU_LIBRARY__ */
switch (o) {
case 'd':
@@ -813,9 +806,6 @@ int main(int argc, char **argv)
case 'h':
genksyms_usage();
return 0;
- case 'R':
- flag_rel_crcs = 1;
- break;
default:
genksyms_usage();
return 1;
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index eceb3ee7ec06..7313cbd755df 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -90,7 +90,6 @@ modpost_link()
if is_enabled CONFIG_MODVERSIONS; then
gen_symversions
- lds="${lds} -T .tmp_symversions.lds"
fi
# This might take a while, so indicate that we're doing
@@ -183,6 +182,10 @@ vmlinux_link()
libs="${KBUILD_VMLINUX_LIBS}"
fi
+ if is_enabled CONFIG_MODULES; then
+ objs="${objs} .vmlinux.export.o"
+ fi
+
if [ "${SRCARCH}" = "um" ]; then
wl=-Wl,
ld="${CC}"
@@ -312,6 +315,7 @@ cleanup()
rm -f vmlinux.o
rm -f .vmlinux.d
rm -f .vmlinux.objs
+ rm -f .vmlinux.export.c
}
# Use "make V=1" to debug this script
@@ -363,6 +367,14 @@ info GEN modules.builtin
tr '\0' '\n' < modules.builtin.modinfo | sed -n 's/^[[:alnum:]:_]*\.file=//p' |
tr ' ' '\n' | uniq | sed -e 's:^:kernel/:' -e 's/$/.ko/' > modules.builtin
+if is_enabled CONFIG_MODULES; then
+ info CC .vmlinux.export.c
+ ${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} \
+ ${KBUILD_CPPFLAGS} ${KBUILD_CFLAGS} \
+ ${KBUILD_CFLAGS_KERNEL} ${CFLAGS_KERNEL} \
+ -c -o .vmlinux.export.o .vmlinux.export.c
+fi
+
btf_vmlinux_bin_o=""
if is_enabled CONFIG_DEBUG_INFO_BTF; then
btf_vmlinux_bin_o=.btf.vmlinux.bin.o
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 213a130d1635..2313bdae91bf 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -1042,6 +1042,7 @@ static void add_header(struct buffer *b, struct module *mod)
buf_printf(b, "#define INCLUDE_VERMAGIC\n");
buf_printf(b, "#include <linux/build-salt.h>\n");
buf_printf(b, "#include <linux/elfnote-lto.h>\n");
+ buf_printf(b, "#include <linux/export-internal.h>\n");
buf_printf(b, "#include <linux/vermagic.h>\n");
buf_printf(b, "#include <linux/compiler.h>\n");
buf_printf(b, "\n");
@@ -1076,20 +1077,26 @@ static void add_header(struct buffer *b, struct module *mod)
buf_printf(b, "\nMODULE_INFO(staging, \"Y\");\n");
}
-static void check_symversions(struct module *mod)
+static void add_exported_symbols(struct buffer *buf, struct module *mod)
{
struct symbol *sym;
if (!modversions)
return;
+ /* record CRCs for exported symbols */
+ buf_printf(buf, "\n");
list_for_each_entry(sym, &mod->exported_symbols, list) {
if (!sym->crc_valid) {
warn("EXPORT symbol \"%s\" [%s%s] version generation failed, symbol will not be versioned.\n"
"Is \"%s\" prototyped in <asm/asm-prototypes.h>?\n",
sym->name, mod->name, mod->is_vmlinux ? "" : ".ko",
sym->name);
+ continue;
}
+
+ buf_printf(buf, "SYMBOL_CRC(%s, 0x%08x, \"%s\");\n",
+ sym->name, sym->crc, sym->is_gpl_only ? "_gpl" : "");
}
}
@@ -1226,6 +1233,18 @@ static void write_if_changed(struct buffer *b, const char *fname)
write_buf(b, fname);
}
+static void write_vmlinux_export_c_file(struct module *mod)
+{
+ struct buffer buf = { };
+
+ buf_printf(&buf,
+ "#include <linux/export-internal.h>\n");
+
+ add_exported_symbols(&buf, mod);
+ write_if_changed(&buf, ".vmlinux.export.c");
+ free(buf.p);
+}
+
/* do sanity checks, and generate *.mod.c file */
static void write_mod_c_file(struct module *mod)
{
@@ -1237,6 +1256,7 @@ static void write_mod_c_file(struct module *mod)
check_exports(mod);
add_header(&buf, mod);
+ add_exported_symbols(&buf, mod);
add_versions(&buf, mod);
add_depends(&buf, mod);
add_moddevtable(&buf, mod);
@@ -1434,9 +1454,9 @@ int main(int argc, char **argv)
if (mod->from_dump)
continue;
- check_symversions(mod);
-
- if (!mod->is_vmlinux)
+ if (mod->is_vmlinux)
+ write_vmlinux_export_c_file(mod);
+ else
write_mod_c_file(mod);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v6 03/10] kbuild: stop merging *.symversions
2022-05-13 11:39 [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 01/10] modpost: extract symbol versions from *.cmd files Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS Masahiro Yamada
@ 2022-05-13 11:39 ` Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 04/10] genksyms: adjust the output format to modpost Masahiro Yamada
` (7 subsequent siblings)
10 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-13 11:39 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, Masahiro Yamada
Now modpost reads symbol versions from .*.cmd files.
The merged *.symversions are no longer needed.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
---
(no changes since v1)
scripts/Makefile.build | 21 ++-------------------
scripts/link-vmlinux.sh | 19 -------------------
2 files changed, 2 insertions(+), 38 deletions(-)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index ddd9080fc028..dff9220135c4 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -390,17 +390,6 @@ $(obj)/%.asn1.c $(obj)/%.asn1.h: $(src)/%.asn1 $(objtree)/scripts/asn1_compiler
$(subdir-builtin): $(obj)/%/built-in.a: $(obj)/% ;
$(subdir-modorder): $(obj)/%/modules.order: $(obj)/% ;
-# combine symversions for later processing
-ifeq ($(CONFIG_LTO_CLANG) $(CONFIG_MODVERSIONS),y y)
- cmd_update_lto_symversions = \
- rm -f $@.symversions \
- $(foreach n, $(filter-out FORCE,$^), \
- $(if $(shell test -s $(n).symversions && echo y), \
- ; cat $(n).symversions >> $@.symversions))
-else
- cmd_update_lto_symversions = echo >/dev/null
-endif
-
#
# Rule to compile a set of .o files into one .a file (without symbol table)
#
@@ -408,11 +397,8 @@ endif
quiet_cmd_ar_builtin = AR $@
cmd_ar_builtin = rm -f $@; $(AR) cDPrST $@ $(real-prereqs)
-quiet_cmd_ar_and_symver = AR $@
- cmd_ar_and_symver = $(cmd_update_lto_symversions); $(cmd_ar_builtin)
-
$(obj)/built-in.a: $(real-obj-y) FORCE
- $(call if_changed,ar_and_symver)
+ $(call if_changed,ar_builtin)
#
# Rule to create modules.order file
@@ -432,16 +418,13 @@ $(obj)/modules.order: $(obj-m) FORCE
#
# Rule to compile a set of .o files into one .a file (with symbol table)
#
-quiet_cmd_ar_lib = AR $@
- cmd_ar_lib = $(cmd_update_lto_symversions); $(cmd_ar)
$(obj)/lib.a: $(lib-y) FORCE
- $(call if_changed,ar_lib)
+ $(call if_changed,ar)
ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
quiet_cmd_link_multi-m = AR [M] $@
cmd_link_multi-m = \
- $(cmd_update_lto_symversions); \
rm -f $@; \
$(AR) cDPrsT $@ @$(patsubst %.o,%.mod,$@)
else
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 7313cbd755df..47e5336d0c75 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -56,20 +56,6 @@ gen_initcalls()
> .tmp_initcalls.lds
}
-# If CONFIG_LTO_CLANG is selected, collect generated symbol versions into
-# .tmp_symversions.lds
-gen_symversions()
-{
- info GEN .tmp_symversions.lds
- rm -f .tmp_symversions.lds
-
- for o in ${KBUILD_VMLINUX_OBJS} ${KBUILD_VMLINUX_LIBS}; do
- if [ -f ${o}.symversions ]; then
- cat ${o}.symversions >> .tmp_symversions.lds
- fi
- done
-}
-
# Link of vmlinux.o used for section mismatch analysis
# ${1} output file
modpost_link()
@@ -88,10 +74,6 @@ modpost_link()
gen_initcalls
lds="-T .tmp_initcalls.lds"
- if is_enabled CONFIG_MODVERSIONS; then
- gen_symversions
- fi
-
# This might take a while, so indicate that we're doing
# an LTO link
info LTO ${1}
@@ -307,7 +289,6 @@ cleanup()
rm -f .btf.*
rm -f .tmp_System.map
rm -f .tmp_initcalls.lds
- rm -f .tmp_symversions.lds
rm -f .tmp_vmlinux*
rm -f System.map
rm -f vmlinux
--
2.32.0
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v6 04/10] genksyms: adjust the output format to modpost
2022-05-13 11:39 [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
` (2 preceding siblings ...)
2022-05-13 11:39 ` [PATCH v6 03/10] kbuild: stop merging *.symversions Masahiro Yamada
@ 2022-05-13 11:39 ` Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 05/10] kbuild: do not create *.prelink.o for Clang LTO or IBT Masahiro Yamada
` (6 subsequent siblings)
10 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-13 11:39 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, Masahiro Yamada
Make genksyms output symbol versions in the format modpost expects,
so the 'sed' is unneeded.
This commit makes *.symversions completely unneeded.
I will keep *.symversions in .gitignore and 'make clean' for a while.
Otherwise, 'git status' might be surprising.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
---
(no changes since v2)
Changes in v2:
- New patch
scripts/Makefile.build | 6 ------
scripts/genksyms/genksyms.c | 2 +-
2 files changed, 1 insertion(+), 7 deletions(-)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index dff9220135c4..461998a2ad2b 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -165,16 +165,10 @@ ifdef CONFIG_MODVERSIONS
# o modpost will extract versions from that file and create *.c files that will
# be compiled and linked to the kernel and/or modules.
-genksyms_format := __crc_\(.*\) = \(.*\);
-
gen_symversions = \
if $(NM) $@ 2>/dev/null | grep -q __ksymtab; then \
$(call cmd_gensymtypes_$(1),$(KBUILD_SYMTYPES),$(@:.o=.symtypes)) \
- > $@.symversions; \
- sed -n 's/$(genksyms_format)/$(pound)SYMVER \1 \2/p' $@.symversions \
>> $(dot-target).cmd; \
- else \
- rm -f $@.symversions; \
fi
cmd_gen_symversions_c = $(call gen_symversions,c)
diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c
index 67b23cc0df0f..f5dfdb9d80e9 100644
--- a/scripts/genksyms/genksyms.c
+++ b/scripts/genksyms/genksyms.c
@@ -680,7 +680,7 @@ void export_symbol(const char *name)
if (flag_dump_defs)
fputs(">\n", debugfile);
- printf("__crc_%s = 0x%08lx;\n", name, crc);
+ printf("#SYMVER %s 0x%08lx\n", name, crc);
}
}
--
2.32.0
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v6 05/10] kbuild: do not create *.prelink.o for Clang LTO or IBT
2022-05-13 11:39 [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
` (3 preceding siblings ...)
2022-05-13 11:39 ` [PATCH v6 04/10] genksyms: adjust the output format to modpost Masahiro Yamada
@ 2022-05-13 11:39 ` Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 06/10] kbuild: check static EXPORT_SYMBOL* by script instead of modpost Masahiro Yamada
` (5 subsequent siblings)
10 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-13 11:39 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, Masahiro Yamada
When CONFIG_LTO_CLANG=y, additional intermediate *.prelink.o is created
for each module. Also, objtool is postponed until LLVM IR is converted
to ELF.
CONFIG_X86_KERNEL_IBT works in a similar way to postpone objtool until
objects are merged together.
This commit stops generating *.prelink.o, so the build flow will look
similar with/without LTO.
The following figures show how the LTO build currently works, and
how this commit is changing it.
Current build flow
==================
[1] single-object module
$(LD)
$(CC) +objtool $(LD)
foo.c --------------------> foo.o -----> foo.prelink.o -----> foo.ko
(LLVM IR) (ELF) |
|
foo.mod.o --/
[2] multi-object module
$(LD)
$(CC) $(AR) +objtool $(LD)
foo1.c -----> foo1.o -----> foo.o -----> foo.prelink.o -----> foo.ko
| (archive) (ELF) |
foo2.c -----> foo2.o --/ |
(LLVM IR) foo.mod.o --/
One confusion is foo.o in multi-object module is an archive despite of
its suffix.
New build flow
==============
[1] single-object module
Since there is only one object, there is no need to keep the LLVM IR.
Use $(CC)+$(LD) to generate an ELF object in one build rule. When LTO
is disabled, $(LD) is unneeded because $(CC) produces an ELF object.
$(CC)+$(LD)+objtool $(LD)
foo.c ----------------------------> foo.o ---------> foo.ko
(ELF) |
|
foo.mod.o --/
[2] multi-object module
Previously, $(AR) was used to combine LLVM bitcode into an archive,
but there was no technical reason to do so. Use $(LD) to merge them
into a single ELF object.
$(LD)
$(CC) +objtool $(LD)
foo1.c ---------> foo1.o ---------> foo.o ---------> foo.ko
| (ELF) |
foo2.c ---------> foo2.o ----/ |
(LLVM IR) foo.mod.o --/
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
---
(no changes since v2)
Changes in v2:
- replace the chain of $(if ...) with $(and )
scripts/Kbuild.include | 4 +++
scripts/Makefile.build | 58 ++++++++++++---------------------------
scripts/Makefile.lib | 7 -----
scripts/Makefile.modfinal | 5 ++--
scripts/Makefile.modpost | 9 ++----
scripts/mod/modpost.c | 7 -----
6 files changed, 25 insertions(+), 65 deletions(-)
diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index 3514c2149e9d..455a0a6ce12d 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -15,6 +15,10 @@ pound := \#
# Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o
dot-target = $(dir $@).$(notdir $@)
+###
+# Name of target with a '.tmp_' as filename prefix. foo/bar.o => foo/.tmp_bar.o
+tmp-target = $(dir $@).tmp_$(notdir $@)
+
###
# The temporary file to save gcc -MMD generated dependencies must not
# contain a comma
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 461998a2ad2b..838ea5e83174 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -88,10 +88,6 @@ endif
targets-for-modules := $(foreach x, o mod $(if $(CONFIG_TRIM_UNUSED_KSYMS), usyms), \
$(patsubst %.o, %.$x, $(filter %.o, $(obj-m))))
-ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
-targets-for-modules += $(patsubst %.o, %.prelink.o, $(filter %.o, $(obj-m)))
-endif
-
ifdef need-modorder
targets-for-modules += $(obj)/modules.order
endif
@@ -152,8 +148,16 @@ $(obj)/%.ll: $(src)/%.c FORCE
# The C file is compiled and updated dependency information is generated.
# (See cmd_cc_o_c + relevant part of rule_cc_o_c)
+is-single-obj-m = $(and $(part-of-module),$(filter $@, $(obj-m)),y)
+
+ifdef CONFIG_LTO_CLANG
+cmd_ld_single_m = $(if $(is-single-obj-m), ; $(LD) $(ld_flags) -r -o $(tmp-target) $@; mv $(tmp-target) $@)
+endif
+
quiet_cmd_cc_o_c = CC $(quiet_modtag) $@
- cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $< $(cmd_objtool)
+ cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $< \
+ $(cmd_ld_single_m) \
+ $(cmd_objtool)
ifdef CONFIG_MODVERSIONS
# When module versioning is enabled the following steps are executed:
@@ -224,21 +228,16 @@ cmd_gen_objtooldep = $(if $(objtool-enabled), { echo ; echo '$@: $$(wildcard $(o
endif # CONFIG_STACK_VALIDATION
-ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
-
-# Skip objtool for LLVM bitcode
-$(obj)/%.o: objtool-enabled :=
-
-else
# 'OBJECT_FILES_NON_STANDARD := y': skip objtool checking for a directory
# 'OBJECT_FILES_NON_STANDARD_foo.o := 'y': skip objtool checking for a file
# 'OBJECT_FILES_NON_STANDARD_foo.o := 'n': override directory skip for a file
-$(obj)/%.o: objtool-enabled = $(if $(filter-out y%, \
- $(OBJECT_FILES_NON_STANDARD_$(basetarget).o)$(OBJECT_FILES_NON_STANDARD)n),y)
+is-standard-object = $(if $(filter-out y%, $(OBJECT_FILES_NON_STANDARD_$(basetarget).o)$(OBJECT_FILES_NON_STANDARD)n),y)
-endif
+delay-objtool := $(or $(CONFIG_LTO_CLANG),$(CONFIG_X86_KERNEL_IBT))
+
+$(obj)/%.o: objtool-enabled = $(if $(is-standard-object),$(if $(delay-objtool),$(is-single-obj-m),y))
ifdef CONFIG_TRIM_UNUSED_KSYMS
cmd_gen_ksymdeps = \
@@ -267,24 +266,6 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE
$(call if_changed_rule,cc_o_c)
$(call cmd,force_checksrc)
-ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
-# Module .o files may contain LLVM bitcode, compile them into native code
-# before ELF processing
-quiet_cmd_cc_prelink_modules = LD [M] $@
- cmd_cc_prelink_modules = \
- $(LD) $(ld_flags) -r -o $@ \
- --whole-archive $(filter-out FORCE,$^) \
- $(cmd_objtool)
-
-# objtool was skipped for LLVM bitcode, run it now that we have compiled
-# modules into native code
-$(obj)/%.prelink.o: objtool-enabled = y
-$(obj)/%.prelink.o: part-of-module := y
-
-$(obj)/%.prelink.o: $(obj)/%.o FORCE
- $(call if_changed,cc_prelink_modules)
-endif
-
cmd_mod = echo $(addprefix $(obj)/, $(call real-search, $*.o, .o, -objs -y -m)) | \
$(AWK) -v RS='( |\n)' '!x[$$0]++' > $@
@@ -294,7 +275,7 @@ $(obj)/%.mod: FORCE
# List module undefined symbols
cmd_undefined_syms = $(NM) $< | sed -n 's/^ *U //p' > $@
-$(obj)/%.usyms: $(obj)/%$(mod-prelink-ext).o FORCE
+$(obj)/%.usyms: $(obj)/%.o FORCE
$(call if_changed,undefined_syms)
quiet_cmd_cc_lst_c = MKLST $@
@@ -416,16 +397,11 @@ $(obj)/modules.order: $(obj-m) FORCE
$(obj)/lib.a: $(lib-y) FORCE
$(call if_changed,ar)
-ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
-quiet_cmd_link_multi-m = AR [M] $@
-cmd_link_multi-m = \
- rm -f $@; \
- $(AR) cDPrsT $@ @$(patsubst %.o,%.mod,$@)
-else
quiet_cmd_link_multi-m = LD [M] $@
- cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@)
-endif
+ cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool)
+$(multi-obj-m): objtool-enabled := $(delay-objtool)
+$(multi-obj-m): part-of-module := y
$(multi-obj-m): %.o: %.mod FORCE
$(call if_changed,link_multi-m)
$(call multi_depend, $(multi-obj-m), .o, -objs -y -m)
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 0453a1904646..f75138385449 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -225,13 +225,6 @@ dtc_cpp_flags = -Wp,-MMD,$(depfile).pre.tmp -nostdinc \
$(addprefix -I,$(DTC_INCLUDE)) \
-undef -D__DTS__
-ifneq ($(CONFIG_LTO_CLANG)$(CONFIG_X86_KERNEL_IBT),)
-# With CONFIG_LTO_CLANG, .o files in modules might be LLVM bitcode, so we
-# need to run LTO to compile them into native code (.lto.o) before further
-# processing.
-mod-prelink-ext := .prelink
-endif
-
# Useful for describing the dependency of composite objects
# Usage:
# $(call multi_depend, multi_used_targets, suffix_to_remove, suffix_to_add)
diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
index 7f39599e9fae..35100e981f4a 100644
--- a/scripts/Makefile.modfinal
+++ b/scripts/Makefile.modfinal
@@ -9,7 +9,7 @@ __modfinal:
include include/config/auto.conf
include $(srctree)/scripts/Kbuild.include
-# for c_flags and mod-prelink-ext
+# for c_flags
include $(srctree)/scripts/Makefile.lib
# find all modules listed in modules.order
@@ -54,9 +54,8 @@ if_changed_except = $(if $(call newer_prereqs_except,$(2))$(cmd-check), \
$(cmd); \
printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:)
-
# Re-generate module BTFs if either module's .ko or vmlinux changed
-$(modules): %.ko: %$(mod-prelink-ext).o %.mod.o scripts/module.lds $(if $(KBUILD_BUILTIN),vmlinux) FORCE
+$(modules): %.ko: %.o %.mod.o scripts/module.lds $(if $(KBUILD_BUILTIN),vmlinux) FORCE
+$(call if_changed_except,ld_ko_o,vmlinux)
ifdef CONFIG_DEBUG_INFO_BTF_MODULES
+$(if $(newer-prereqs),$(call cmd,btf_ko))
diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
index 48585c4d04ad..f2ce411acd59 100644
--- a/scripts/Makefile.modpost
+++ b/scripts/Makefile.modpost
@@ -41,9 +41,6 @@ __modpost:
include include/config/auto.conf
include $(srctree)/scripts/Kbuild.include
-# for mod-prelink-ext
-include $(srctree)/scripts/Makefile.lib
-
MODPOST = scripts/mod/modpost \
$(if $(CONFIG_MODVERSIONS),-m) \
$(if $(CONFIG_MODULE_SRCVERSION_ALL),-a) \
@@ -118,8 +115,6 @@ $(input-symdump):
@echo >&2 ' Modules may not have dependencies or modversions.'
@echo >&2 ' You may get many unresolved symbol warnings.'
-modules := $(sort $(shell cat $(MODORDER)))
-
# KBUILD_MODPOST_WARN can be set to avoid error out in case of undefined symbols
ifneq ($(KBUILD_MODPOST_WARN)$(filter-out $(existing-input-symdump), $(input-symdump)),)
MODPOST += -w
@@ -128,9 +123,9 @@ endif
# Read out modules.order to pass in modpost.
# Otherwise, allmodconfig would fail with "Argument list too long".
quiet_cmd_modpost = MODPOST $@
- cmd_modpost = sed 's/\.ko$$/$(mod-prelink-ext)\.o/' $< | $(MODPOST) -T -
+ cmd_modpost = sed 's/ko$$/o/' $< | $(MODPOST) -T -
-$(output-symdump): $(MODORDER) $(input-symdump) $(modules:.ko=$(mod-prelink-ext).o) FORCE
+$(output-symdump): $(MODORDER) $(input-symdump) FORCE
$(call if_changed,modpost)
targets += $(output-symdump)
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 2313bdae91bf..27c45427d989 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -716,10 +716,6 @@ static char *remove_dot(char *s)
size_t m = strspn(s + n + 1, "0123456789");
if (m && (s[n + m] == '.' || s[n + m] == 0))
s[n] = 0;
-
- /* strip trailing .prelink */
- if (strends(s, ".prelink"))
- s[strlen(s) - 8] = '\0';
}
return s;
}
@@ -841,9 +837,6 @@ static void read_symbols(const char *modname)
/* strip trailing .o */
tmp = NOFAIL(strdup(modname));
tmp[strlen(tmp) - 2] = '\0';
- /* strip trailing .prelink */
- if (strends(tmp, ".prelink"))
- tmp[strlen(tmp) - 8] = '\0';
mod = new_module(tmp);
free(tmp);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v6 06/10] kbuild: check static EXPORT_SYMBOL* by script instead of modpost
2022-05-13 11:39 [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
` (4 preceding siblings ...)
2022-05-13 11:39 ` [PATCH v6 05/10] kbuild: do not create *.prelink.o for Clang LTO or IBT Masahiro Yamada
@ 2022-05-13 11:39 ` Masahiro Yamada
2022-05-25 8:31 ` Guenter Roeck
2022-06-07 14:22 ` Jon Hunter
2022-05-13 11:39 ` [PATCH v6 07/10] kbuild: make built-in.a rule robust against too long argument error Masahiro Yamada
` (4 subsequent siblings)
10 siblings, 2 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-13 11:39 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, Masahiro Yamada
The 'static' specifier and EXPORT_SYMBOL() are an odd combination.
Commit 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL*
functions") tried to detect it, but this check has false negatives.
Here is the sample code.
Makefile:
obj-y += foo1.o foo2.o
foo1.c:
#include <linux/export.h>
static void foo(void) {}
EXPORT_SYMBOL(foo);
foo2.c:
void foo(void) {}
foo1.c exports the static symbol 'foo', but modpost cannot catch it
because it is fooled by foo2.c, which has a global symbol with the
same name.
s->is_static is cleared if a global symbol with the same name is found
somewhere, but EXPORT_SYMBOL() and the global symbol do not necessarily
belong to the same compilation unit.
This check should be done per compilation unit, but I do not know how
to do it in modpost. modpost runs against vmlinux.o or modules, which
merges multiple objects, then forgets their origin.
It is true modpost gets access to the lists of all the member objects
(.vmlinux.objs and *.mod), but modpost cannot parse individual objects
because they may not be ELF but LLVM IR when CONFIG_LTO_CLANG=y.
Add a simple bash script to parse the output from ${NM}. This works for
CONFIG_LTO_CLANG=y because llvm-nm can dump symbols of LLVM IR files.
Revert 15bfc2348d54.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Nathan Chancellor <nathan@kernel.org>
---
(no changes since v5)
Changes in v5:
- Add more comments in the script
Changes in v4:
- New patch
scripts/Makefile.build | 4 +++
scripts/check-local-export | 64 ++++++++++++++++++++++++++++++++++++++
scripts/mod/modpost.c | 28 +----------------
3 files changed, 69 insertions(+), 27 deletions(-)
create mode 100755 scripts/check-local-export
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 838ea5e83174..c2a173b3fd60 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -244,9 +244,12 @@ cmd_gen_ksymdeps = \
$(CONFIG_SHELL) $(srctree)/scripts/gen_ksymdeps.sh $@ >> $(dot-target).cmd
endif
+cmd_check_local_export = $(srctree)/scripts/check-local-export $@
+
define rule_cc_o_c
$(call cmd_and_fixdep,cc_o_c)
$(call cmd,gen_ksymdeps)
+ $(call cmd,check_local_export)
$(call cmd,checksrc)
$(call cmd,checkdoc)
$(call cmd,gen_objtooldep)
@@ -257,6 +260,7 @@ endef
define rule_as_o_S
$(call cmd_and_fixdep,as_o_S)
$(call cmd,gen_ksymdeps)
+ $(call cmd,check_local_export)
$(call cmd,gen_objtooldep)
$(call cmd,gen_symversions_S)
endef
diff --git a/scripts/check-local-export b/scripts/check-local-export
new file mode 100755
index 000000000000..829e0591c0be
--- /dev/null
+++ b/scripts/check-local-export
@@ -0,0 +1,64 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2022 Masahiro Yamada <masahiroy@kernel.org>
+#
+# Exit with error if a local exported symbol is found.
+# EXPORT_SYMBOL should be used for global symbols.
+
+set -e
+
+declare -A symbol_types
+declare -a export_symbols
+
+exit_code=0
+
+while read value type name
+do
+ # Exceptional case for Clang LTO.
+ # llvm-nm outputs this:
+ # "---------------- t"
+ # Skip this line because the name is empty.
+ if [[ ${value} = -* && -z ${name} ]]; then
+ continue
+ fi
+
+ # For undefined symbols, the first field (value) is empty.
+ # The outout looks like this:
+ # " U _printk"
+ # Shift the fields.
+ if [[ -z ${name} ]]; then
+ name=${type}
+ type=${value}
+ fi
+
+ # save (name, type) in the associative array
+ symbol_types[${name}]=${type}
+
+ # append the exported symbol to the array
+ if [[ ${name} == __ksymtab_* ]]; then
+ export_symbols+=(${name#__ksymtab_})
+ fi
+
+ # If there is no symbol in the object, ${NM} (both GNU nm and llvm-nm)
+ # shows 'no symbols' diagnostic (but exits with 0). It is harmless and
+ # hidden by '2>/dev/null'. However, it suppresses real error messages
+ # as well. Add a hand-crafted error message here.
+ #
+ # Use -q instead of 2>/dev/null when we upgrade the minimum version of
+ # binutils to 2.37, llvm to 13.0.0.
+done < <(${NM} ${1} 2>/dev/null || { echo "${0}: ${NM} failed" >&2; false; } )
+
+# Catch error in the process substitution
+wait $!
+
+for name in "${export_symbols[@]}"
+do
+ # nm(3) says "If lowercase, the symbol is usually local"
+ if [[ ${symbol_types[$name]} =~ [a-z] ]]; then
+ echo "$@: error: local symbol '${name}' was exported" >&2
+ exit_code=1
+ fi
+done
+
+exit ${exit_code}
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 27c45427d989..2328522f8a78 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -212,7 +212,6 @@ struct symbol {
unsigned int crc;
bool crc_valid;
bool weak;
- bool is_static; /* true if symbol is not global */
bool is_gpl_only; /* exported by EXPORT_SYMBOL_GPL */
char name[];
};
@@ -242,7 +241,7 @@ static struct symbol *alloc_symbol(const char *name)
memset(s, 0, sizeof(*s));
strcpy(s->name, name);
- s->is_static = true;
+
return s;
}
@@ -877,20 +876,6 @@ static void read_symbols(const char *modname)
sym_get_data(&info, sym));
}
- // check for static EXPORT_SYMBOL_* functions && global vars
- for (sym = info.symtab_start; sym < info.symtab_stop; sym++) {
- unsigned char bind = ELF_ST_BIND(sym->st_info);
-
- if (bind == STB_GLOBAL || bind == STB_WEAK) {
- struct symbol *s =
- find_symbol(remove_dot(info.strtab +
- sym->st_name));
-
- if (s)
- s->is_static = false;
- }
- }
-
check_sec_ref(mod, modname, &info);
if (!mod->is_vmlinux) {
@@ -1320,7 +1305,6 @@ static void read_dump(const char *fname)
mod->from_dump = true;
}
s = sym_add_exported(symname, mod, gpl_only);
- s->is_static = false;
sym_set_crc(s, crc);
sym_update_namespace(symname, namespace);
}
@@ -1385,7 +1369,6 @@ int main(int argc, char **argv)
char *missing_namespace_deps = NULL;
char *dump_write = NULL, *files_source = NULL;
int opt;
- int n;
LIST_HEAD(dump_lists);
struct dump_list *dl, *dl2;
@@ -1461,15 +1444,6 @@ int main(int argc, char **argv)
if (sec_mismatch_count && !sec_mismatch_warn_only)
error("Section mismatches detected.\n"
"Set CONFIG_SECTION_MISMATCH_WARN_ONLY=y to allow them.\n");
- for (n = 0; n < SYMBOL_HASH_SIZE; n++) {
- struct symbol *s;
-
- for (s = symbolhash[n]; s; s = s->next) {
- if (s->is_static)
- error("\"%s\" [%s] is a static EXPORT_SYMBOL\n",
- s->name, s->module->name);
- }
- }
if (nr_unresolved > MAX_UNRESOLVED_REPORTS)
warn("suppressed %u unresolved symbol warnings because there were too many)\n",
--
2.32.0
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v6 07/10] kbuild: make built-in.a rule robust against too long argument error
2022-05-13 11:39 [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
` (5 preceding siblings ...)
2022-05-13 11:39 ` [PATCH v6 06/10] kbuild: check static EXPORT_SYMBOL* by script instead of modpost Masahiro Yamada
@ 2022-05-13 11:39 ` Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 08/10] kbuild: make *.mod " Masahiro Yamada
` (3 subsequent siblings)
10 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-13 11:39 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, Masahiro Yamada
Kbuild runs at the top of objtree instead of changing the working
directory to subdirectories. I think this design is nice overall but
some commands have a scalability issue.
The build command of built-in.a is one of them whose length scales with:
O(D * N)
Here, D is the length of the directory path (i.e. $(obj)/ prefix),
N is the number of objects in the Makefile, O() is the big O notation.
The deeper directory the Makefile directory is located, the more easily
it will hit the too long argument error.
We can make it better. Trim the $(obj)/ by Make's builtin function, and
restore it by a shell command (sed).
With this, the command length scales with:
O(D + N)
In-tree modules still have some room to the limit (ARG_MAX=2097152),
but this is more future-proof for big modules in a deep directory.
For example, you can build i915 as builtin (CONFIG_DRM_I915=y) and
compare drivers/gpu/drm/i915/.built-in.a.cmd with/without this commit.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
---
(no changes since v2)
Changes in v2:
- New patch
scripts/Makefile.build | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index c2a173b3fd60..8f1a355df7aa 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -374,7 +374,10 @@ $(subdir-modorder): $(obj)/%/modules.order: $(obj)/% ;
#
quiet_cmd_ar_builtin = AR $@
- cmd_ar_builtin = rm -f $@; $(AR) cDPrST $@ $(real-prereqs)
+ cmd_ar_builtin = rm -f $@; \
+ echo $(patsubst $(obj)/%,%,$(real-prereqs)) | \
+ sed -E 's:([^ ]+):$(obj)/\1:g' | \
+ xargs $(AR) cDPrST $@
$(obj)/built-in.a: $(real-obj-y) FORCE
$(call if_changed,ar_builtin)
--
2.32.0
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v6 08/10] kbuild: make *.mod rule robust against too long argument error
2022-05-13 11:39 [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
` (6 preceding siblings ...)
2022-05-13 11:39 ` [PATCH v6 07/10] kbuild: make built-in.a rule robust against too long argument error Masahiro Yamada
@ 2022-05-13 11:39 ` Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 09/10] kbuild: add cmd_and_savecmd macro Masahiro Yamada
` (2 subsequent siblings)
10 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-13 11:39 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, Masahiro Yamada
Like built-in.a, the command length of the *.mod rule scales with
the depth of the directory times the number of objects in the Makefile.
Add $(obj)/ by the shell command (awk) instead of by Make's builtin
function.
In-tree modules still have some room to the limit (ARG_MAX=2097152),
but this is more future-proof for big modules in a deep directory.
For example, you can build i915 as a module (CONFIG_DRM_I915=m) and
compare drivers/gpu/drm/i915/.i915.mod.cmd with/without this commit.
The issue is more critical for external modules because the M= path
can be very long as Jeff Johnson reported before [1].
[1] https://lore.kernel.org/linux-kbuild/4c02050c4e95e4cb8cc04282695f8404@codeaurora.org/
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nathan Chancellor <nathan@kernel.org>
---
(no changes since v2)
Changes in v2:
- New patch
scripts/Makefile.build | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 8f1a355df7aa..f546b5f1f33f 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -270,8 +270,8 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE
$(call if_changed_rule,cc_o_c)
$(call cmd,force_checksrc)
-cmd_mod = echo $(addprefix $(obj)/, $(call real-search, $*.o, .o, -objs -y -m)) | \
- $(AWK) -v RS='( |\n)' '!x[$$0]++' > $@
+cmd_mod = echo $(call real-search, $*.o, .o, -objs -y -m) | \
+ $(AWK) -v RS='( |\n)' '!x[$$0]++ { print("$(obj)/"$$0) }' > $@
$(obj)/%.mod: FORCE
$(call if_changed,mod)
--
2.32.0
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v6 09/10] kbuild: add cmd_and_savecmd macro
2022-05-13 11:39 [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
` (7 preceding siblings ...)
2022-05-13 11:39 ` [PATCH v6 08/10] kbuild: make *.mod " Masahiro Yamada
@ 2022-05-13 11:39 ` Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 10/10] kbuild: rebuild multi-object modules when objtool is updated Masahiro Yamada
2022-05-13 12:20 ` [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
10 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-13 11:39 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, Masahiro Yamada, Kees Cook
Separate out the command execution part of if_changed, as we did
for if_changed_dep.
This allows us to reuse it in if_changed_rule.
define rule_foo
$(call cmd_and_savecmd,foo)
$(call cmd,bar)
endef
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
---
(no changes since v4)
Changes in v4:
- New.
Resent of my previous submission.
https://lore.kernel.org/all/20210831074004.3195284-10-masahiroy@kernel.org/
scripts/Kbuild.include | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
index 455a0a6ce12d..ece44b735061 100644
--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -142,9 +142,11 @@ check-FORCE = $(if $(filter FORCE, $^),,$(warning FORCE prerequisite is missing)
if-changed-cond = $(newer-prereqs)$(cmd-check)$(check-FORCE)
# Execute command if command has changed or prerequisite(s) are updated.
-if_changed = $(if $(if-changed-cond), \
+if_changed = $(if $(if-changed-cond),$(cmd_and_savecmd),@:)
+
+cmd_and_savecmd = \
$(cmd); \
- printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:)
+ printf '%s\n' 'cmd_$@ := $(make-cmd)' > $(dot-target).cmd
# Execute the command and also postprocess generated .d dependencies file.
if_changed_dep = $(if $(if-changed-cond),$(cmd_and_fixdep),@:)
--
2.32.0
^ permalink raw reply related [flat|nested] 39+ messages in thread
* [PATCH v6 10/10] kbuild: rebuild multi-object modules when objtool is updated
2022-05-13 11:39 [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
` (8 preceding siblings ...)
2022-05-13 11:39 ` [PATCH v6 09/10] kbuild: add cmd_and_savecmd macro Masahiro Yamada
@ 2022-05-13 11:39 ` Masahiro Yamada
2022-05-13 12:20 ` [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
10 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-13 11:39 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, Masahiro Yamada, Kees Cook,
Josh Poimboeuf
When CONFIG_LTO_CLANG or CONFIG_X86_KERNEL_IBT is enabled, objtool for
multi-object modules is postponed until the objects are linked together.
Make sure to re-run objtool and re-link multi-object modules when
objtool is updated.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Tested-by: Nathan Chancellor <nathan@kernel.org>
---
(no changes since v4)
Changes in v4:
- New
Resent of my previous submission
https://lore.kernel.org/linux-kbuild/20210831074004.3195284-11-masahiroy@kernel.org/
scripts/Makefile.build | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index f546b5f1f33f..4e6902e099e8 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -404,13 +404,18 @@ $(obj)/modules.order: $(obj-m) FORCE
$(obj)/lib.a: $(lib-y) FORCE
$(call if_changed,ar)
-quiet_cmd_link_multi-m = LD [M] $@
- cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool)
+quiet_cmd_ld_multi_m = LD [M] $@
+ cmd_ld_multi_m = $(LD) $(ld_flags) -r -o $@ @$(patsubst %.o,%.mod,$@) $(cmd_objtool)
+
+define rule_ld_multi_m
+ $(call cmd_and_savecmd,ld_multi_m)
+ $(call cmd,gen_objtooldep)
+endef
$(multi-obj-m): objtool-enabled := $(delay-objtool)
$(multi-obj-m): part-of-module := y
$(multi-obj-m): %.o: %.mod FORCE
- $(call if_changed,link_multi-m)
+ $(call if_changed_rule,ld_multi_m)
$(call multi_depend, $(multi-obj-m), .o, -objs -y -m)
targets := $(filter-out $(PHONY), $(targets))
--
2.32.0
^ permalink raw reply related [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
2022-05-13 11:39 [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
` (9 preceding siblings ...)
2022-05-13 11:39 ` [PATCH v6 10/10] kbuild: rebuild multi-object modules when objtool is updated Masahiro Yamada
@ 2022-05-13 12:20 ` Masahiro Yamada
[not found] ` <CA+icZUUWww3fXvjQcefgFuq=tPO6+FYDbHE2E5PmL-BSJg4+cw@mail.gmail.com>
[not found] ` <2c496d24174e63b27ec047f383df6700@matoro.tk>
10 siblings, 2 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-13 12:20 UTC (permalink / raw)
To: Linux Kbuild mailing list
Cc: Linux Kernel Mailing List, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, clang-built-linux,
Ard Biesheuvel, Sami Tolvanen
On Fri, May 13, 2022 at 8:42 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
>
> This is the third batch of cleanups in this development cycle.
>
This series is available at
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
lto-cleanup-v6
>
> Changes in v6:
> - Fix false-positive warnings when CONFIG_TRIM_UNUSED_KSYMS=y
>
> Changes in v5:
> - Fix the build error when CONFIG_DEBUG_INFO_BTF=y (reported by Nathan)
> - Clean up arch/m68k/include/asm/export.h (Nick)
> - Keep gen_symversions (and will be removed by a later patch)
> - Add more comments in the script
>
> Changes in v4:
> - Rename .vmlinux-symver.c to .vmlinux.export.c
> because I notice this approach is useful for further cleanups,
> not only for modversioning but also for overall EXPORT_SYMBOL.
> - New patch
> - New.
> Resent of my previous submission.
> https://lore.kernel.org/all/20210831074004.3195284-10-masahiroy@kernel.org/
> - New
> Resent of my previous submission
> https://lore.kernel.org/linux-kbuild/20210831074004.3195284-11-masahiroy@kernel.org/
>
> Changes in v3:
> - New patch
>
> Changes in v2:
> - Simplify the implementation (parse .cmd files after ELF)
> - New patch
> - replace the chain of $(if ...) with $(and )
> - New patch
> - New patch
>
> Masahiro Yamada (10):
> modpost: extract symbol versions from *.cmd files
> kbuild: link symbol CRCs at final link, removing
> CONFIG_MODULE_REL_CRCS
> kbuild: stop merging *.symversions
> genksyms: adjust the output format to modpost
> kbuild: do not create *.prelink.o for Clang LTO or IBT
> kbuild: check static EXPORT_SYMBOL* by script instead of modpost
> kbuild: make built-in.a rule robust against too long argument error
> kbuild: make *.mod rule robust against too long argument error
> kbuild: add cmd_and_savecmd macro
> kbuild: rebuild multi-object modules when objtool is updated
>
> arch/m68k/include/asm/Kbuild | 1 +
> arch/m68k/include/asm/export.h | 2 -
> arch/powerpc/Kconfig | 1 -
> arch/s390/Kconfig | 1 -
> arch/um/Kconfig | 1 -
> include/asm-generic/export.h | 22 ++-
> include/linux/export-internal.h | 16 +++
> include/linux/export.h | 30 ++--
> init/Kconfig | 4 -
> kernel/module.c | 10 +-
> scripts/Kbuild.include | 10 +-
> scripts/Makefile.build | 134 ++++++------------
> scripts/Makefile.lib | 7 -
> scripts/Makefile.modfinal | 5 +-
> scripts/Makefile.modpost | 9 +-
> scripts/check-local-export | 64 +++++++++
> scripts/genksyms/genksyms.c | 18 +--
> scripts/link-vmlinux.sh | 33 ++---
> scripts/mod/modpost.c | 236 +++++++++++++++++++++-----------
> 19 files changed, 320 insertions(+), 284 deletions(-)
> delete mode 100644 arch/m68k/include/asm/export.h
> create mode 100644 include/linux/export-internal.h
> create mode 100755 scripts/check-local-export
>
> --
> 2.32.0
>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
[not found] ` <CA+icZUUWww3fXvjQcefgFuq=tPO6+FYDbHE2E5PmL-BSJg4+cw@mail.gmail.com>
@ 2022-05-22 6:49 ` Masahiro Yamada
2022-05-22 9:45 ` Sedat Dilek
0 siblings, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-22 6:49 UTC (permalink / raw)
To: Sedat Dilek
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
On Sun, May 22, 2022 at 10:45 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Fri, May 13, 2022 at 4:31 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > On Fri, May 13, 2022 at 8:42 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > >
> > > This is the third batch of cleanups in this development cycle.
> > >
> >
> >
> > This series is available at
> > git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> > lto-cleanup-v6
> >
>
> Hi Masahiro,
>
> I cloned the repository on top of latest Linus Git.
>
> Not able to boot in Quemu - Not able to boot on bare metal.
>
> $ grep module_layout log_quemu-5.18.0-rc7-2-amd64-clang14-lto.txt
> 366:[ 2.173265] floppy: disagrees about version of symbol module_layout
> 367:[ 2.198746] scsi_common: disagrees about version of symbol module_layout
> 368:[ 2.205573] i2c_piix4: disagrees about version of symbol module_layout
> 369:[ 2.210610] psmouse: disagrees about version of symbol module_layout
> 370:[ 2.225138] scsi_common: disagrees about version of symbol module_layout
> 371:[ 2.235536] scsi_common: disagrees about version of symbol module_layout
> 375:Begin: Running /scripts/local-premount ... [ 2.298555]
> crc32c_intel: disagrees about version of symbol module_layout
> 376:[ 2.303335] crc32c_generic: disagrees about version of symbol
> module_layout
> 377:[ 2.306667] libcrc32c: disagrees about version of symbol module_layout
>
> Infos: LLVM-14 + CONFIG_LTO_CLANG_THIN=y
>
> My linux-config and qemu-log are attached.
>
Thanks for your testing.
I was also able to reproduce this issue.
The problematic parts are:
[ 2.298555] crc32c_intel: disagrees about version of symbol module_layout
[ 2.303335] crc32c_generic: disagrees about version of symbol module_layout
[ 2.306667] libcrc32c: disagrees about version of symbol module_layout
When CONFIG_LTO_CLANG_THIN=y,
I cannot see any __crc_* symbols in "nm vmlinux".
Perhaps, LTO might have discarded all the __crc_* symbols
from vmlinux, but I am still checking the details...
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
2022-05-22 6:49 ` Masahiro Yamada
@ 2022-05-22 9:45 ` Sedat Dilek
2022-05-22 14:15 ` Sedat Dilek
0 siblings, 1 reply; 39+ messages in thread
From: Sedat Dilek @ 2022-05-22 9:45 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
On Sun, May 22, 2022 at 8:50 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Sun, May 22, 2022 at 10:45 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >
> > On Fri, May 13, 2022 at 4:31 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > On Fri, May 13, 2022 at 8:42 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > >
> > > >
> > > > This is the third batch of cleanups in this development cycle.
> > > >
> > >
> > >
> > > This series is available at
> > > git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> > > lto-cleanup-v6
> > >
> >
> > Hi Masahiro,
> >
> > I cloned the repository on top of latest Linus Git.
> >
> > Not able to boot in Quemu - Not able to boot on bare metal.
> >
> > $ grep module_layout log_quemu-5.18.0-rc7-2-amd64-clang14-lto.txt
> > 366:[ 2.173265] floppy: disagrees about version of symbol module_layout
> > 367:[ 2.198746] scsi_common: disagrees about version of symbol module_layout
> > 368:[ 2.205573] i2c_piix4: disagrees about version of symbol module_layout
> > 369:[ 2.210610] psmouse: disagrees about version of symbol module_layout
> > 370:[ 2.225138] scsi_common: disagrees about version of symbol module_layout
> > 371:[ 2.235536] scsi_common: disagrees about version of symbol module_layout
> > 375:Begin: Running /scripts/local-premount ... [ 2.298555]
> > crc32c_intel: disagrees about version of symbol module_layout
> > 376:[ 2.303335] crc32c_generic: disagrees about version of symbol
> > module_layout
> > 377:[ 2.306667] libcrc32c: disagrees about version of symbol module_layout
> >
> > Infos: LLVM-14 + CONFIG_LTO_CLANG_THIN=y
> >
> > My linux-config and qemu-log are attached.
> >
>
>
> Thanks for your testing.
>
> I was also able to reproduce this issue.
>
>
> The problematic parts are:
>
> [ 2.298555] crc32c_intel: disagrees about version of symbol module_layout
> [ 2.303335] crc32c_generic: disagrees about version of symbol module_layout
> [ 2.306667] libcrc32c: disagrees about version of symbol module_layout
>
>
>
> When CONFIG_LTO_CLANG_THIN=y,
> I cannot see any __crc_* symbols in "nm vmlinux".
>
> Perhaps, LTO might have discarded all the __crc_* symbols
> from vmlinux, but I am still checking the details...
>
Thanks for taking care.
Just for the records:
$ grep CONFIG_MODVERSIONS /boot/config-5.18.0-rc7-2-amd64-clang14-lto
CONFIG_MODVERSIONS=y
-Sedat-
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
2022-05-22 9:45 ` Sedat Dilek
@ 2022-05-22 14:15 ` Sedat Dilek
2022-05-22 16:09 ` Masahiro Yamada
0 siblings, 1 reply; 39+ messages in thread
From: Sedat Dilek @ 2022-05-22 14:15 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
On Sun, May 22, 2022 at 11:45 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Sun, May 22, 2022 at 8:50 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > On Sun, May 22, 2022 at 10:45 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > >
> > > On Fri, May 13, 2022 at 4:31 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > >
> > > > On Fri, May 13, 2022 at 8:42 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > > >
> > > > >
> > > > > This is the third batch of cleanups in this development cycle.
> > > > >
> > > >
> > > >
> > > > This series is available at
> > > > git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> > > > lto-cleanup-v6
> > > >
> > >
> > > Hi Masahiro,
> > >
> > > I cloned the repository on top of latest Linus Git.
> > >
> > > Not able to boot in Quemu - Not able to boot on bare metal.
> > >
> > > $ grep module_layout log_quemu-5.18.0-rc7-2-amd64-clang14-lto.txt
> > > 366:[ 2.173265] floppy: disagrees about version of symbol module_layout
> > > 367:[ 2.198746] scsi_common: disagrees about version of symbol module_layout
> > > 368:[ 2.205573] i2c_piix4: disagrees about version of symbol module_layout
> > > 369:[ 2.210610] psmouse: disagrees about version of symbol module_layout
> > > 370:[ 2.225138] scsi_common: disagrees about version of symbol module_layout
> > > 371:[ 2.235536] scsi_common: disagrees about version of symbol module_layout
> > > 375:Begin: Running /scripts/local-premount ... [ 2.298555]
> > > crc32c_intel: disagrees about version of symbol module_layout
> > > 376:[ 2.303335] crc32c_generic: disagrees about version of symbol
> > > module_layout
> > > 377:[ 2.306667] libcrc32c: disagrees about version of symbol module_layout
> > >
> > > Infos: LLVM-14 + CONFIG_LTO_CLANG_THIN=y
> > >
> > > My linux-config and qemu-log are attached.
> > >
> >
> >
> > Thanks for your testing.
> >
> > I was also able to reproduce this issue.
> >
> >
> > The problematic parts are:
> >
> > [ 2.298555] crc32c_intel: disagrees about version of symbol module_layout
> > [ 2.303335] crc32c_generic: disagrees about version of symbol module_layout
> > [ 2.306667] libcrc32c: disagrees about version of symbol module_layout
> >
> >
> >
> > When CONFIG_LTO_CLANG_THIN=y,
> > I cannot see any __crc_* symbols in "nm vmlinux".
> >
> > Perhaps, LTO might have discarded all the __crc_* symbols
> > from vmlinux, but I am still checking the details...
> >
>
> Thanks for taking care.
>
> Just for the records:
>
> $ grep CONFIG_MODVERSIONS /boot/config-5.18.0-rc7-2-amd64-clang14-lto
> CONFIG_MODVERSIONS=y
>
Did not try CONFIG_MODVERSIONS=n.
We have a new file:
[ include/linux/export-internal.h ]
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Please do not include this explicitly.
* This is used by C files generated by modpost.
*/
#ifndef __LINUX_EXPORT_INTERNAL_H__
#define __LINUX_EXPORT_INTERNAL_H__
#include <linux/compiler.h>
#include <linux/types.h>
#define SYMBOL_CRC(sym, crc, sec) \
u32 __section("___kcrctab" sec "+" #sym) __crc_##sym = crc
#endif /* __LINUX_EXPORT_INTERNAL_H__ */
But we discard __kcrctab in scripts/module.lds.S file.
Maybe we need:
$ git diff scripts/module.lds.S
diff --git a/scripts/module.lds.S b/scripts/module.lds.S
index 1d0e1e4dc3d2..c04b596c364b 100644
--- a/scripts/module.lds.S
+++ b/scripts/module.lds.S
@@ -21,8 +21,6 @@ SECTIONS {
__ksymtab 0 : { *(SORT(___ksymtab+*)) }
__ksymtab_gpl 0 : { *(SORT(___ksymtab_gpl+*)) }
- __kcrctab 0 : { *(SORT(___kcrctab+*)) }
- __kcrctab_gpl 0 : { *(SORT(___kcrctab_gpl+*)) }
.ctors 0 : ALIGN(8) { *(SORT(.ctors.*)) *(.ctors) }
.init_array 0 : ALIGN(8) { *(SORT(.init_array.*))
*(.init_array) }
Or even?
$ git diff scripts/kallsyms.c
diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
index 8caabddf817c..fb3601fe8aa3 100644
--- a/scripts/kallsyms.c
+++ b/scripts/kallsyms.c
@@ -109,7 +109,6 @@ static bool is_ignored_symbol(const char *name, char type)
static const char * const ignored_prefixes[] = {
"$", /* local symbols for ARM, MIPS, etc. */
".L", /* local labels,
.LBB,.Ltmpxxx,.L__unnamed_xx,.LASANPC, etc. */
- "__crc_", /* modversions */
"__efistub_", /* arm64 EFI stub namespace */
"__kvm_nvhe_", /* arm64 non-VHE KVM namespace */
"__AArch64ADRPThunk_", /* arm64 lld */
- Sedat -
^ permalink raw reply related [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
2022-05-22 14:15 ` Sedat Dilek
@ 2022-05-22 16:09 ` Masahiro Yamada
2022-05-22 16:18 ` Sedat Dilek
0 siblings, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-22 16:09 UTC (permalink / raw)
To: Sedat Dilek
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
On Sun, May 22, 2022 at 11:16 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Sun, May 22, 2022 at 11:45 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >
> > On Sun, May 22, 2022 at 8:50 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > >
> > > On Sun, May 22, 2022 at 10:45 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > > >
> > > > On Fri, May 13, 2022 at 4:31 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > > >
> > > > > On Fri, May 13, 2022 at 8:42 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > > > >
> > > > > >
> > > > > > This is the third batch of cleanups in this development cycle.
> > > > > >
> > > > >
> > > > >
> > > > > This series is available at
> > > > > git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> > > > > lto-cleanup-v6
> > > > >
> > > >
> > > > Hi Masahiro,
> > > >
> > > > I cloned the repository on top of latest Linus Git.
> > > >
> > > > Not able to boot in Quemu - Not able to boot on bare metal.
> > > >
> > > > $ grep module_layout log_quemu-5.18.0-rc7-2-amd64-clang14-lto.txt
> > > > 366:[ 2.173265] floppy: disagrees about version of symbol module_layout
> > > > 367:[ 2.198746] scsi_common: disagrees about version of symbol module_layout
> > > > 368:[ 2.205573] i2c_piix4: disagrees about version of symbol module_layout
> > > > 369:[ 2.210610] psmouse: disagrees about version of symbol module_layout
> > > > 370:[ 2.225138] scsi_common: disagrees about version of symbol module_layout
> > > > 371:[ 2.235536] scsi_common: disagrees about version of symbol module_layout
> > > > 375:Begin: Running /scripts/local-premount ... [ 2.298555]
> > > > crc32c_intel: disagrees about version of symbol module_layout
> > > > 376:[ 2.303335] crc32c_generic: disagrees about version of symbol
> > > > module_layout
> > > > 377:[ 2.306667] libcrc32c: disagrees about version of symbol module_layout
> > > >
> > > > Infos: LLVM-14 + CONFIG_LTO_CLANG_THIN=y
> > > >
> > > > My linux-config and qemu-log are attached.
> > > >
> > >
> > >
> > > Thanks for your testing.
> > >
> > > I was also able to reproduce this issue.
> > >
> > >
> > > The problematic parts are:
> > >
> > > [ 2.298555] crc32c_intel: disagrees about version of symbol module_layout
> > > [ 2.303335] crc32c_generic: disagrees about version of symbol module_layout
> > > [ 2.306667] libcrc32c: disagrees about version of symbol module_layout
> > >
> > >
> > >
> > > When CONFIG_LTO_CLANG_THIN=y,
> > > I cannot see any __crc_* symbols in "nm vmlinux".
> > >
> > > Perhaps, LTO might have discarded all the __crc_* symbols
> > > from vmlinux, but I am still checking the details...
> > >
> >
> > Thanks for taking care.
> >
> > Just for the records:
> >
> > $ grep CONFIG_MODVERSIONS /boot/config-5.18.0-rc7-2-amd64-clang14-lto
> > CONFIG_MODVERSIONS=y
> >
>
> Did not try CONFIG_MODVERSIONS=n.
>
> We have a new file:
>
> [ include/linux/export-internal.h ]
> /* SPDX-License-Identifier: GPL-2.0-only */
> /*
> * Please do not include this explicitly.
> * This is used by C files generated by modpost.
> */
>
> #ifndef __LINUX_EXPORT_INTERNAL_H__
> #define __LINUX_EXPORT_INTERNAL_H__
>
> #include <linux/compiler.h>
> #include <linux/types.h>
>
> #define SYMBOL_CRC(sym, crc, sec) \
> u32 __section("___kcrctab" sec "+" #sym) __crc_##sym = crc
>
> #endif /* __LINUX_EXPORT_INTERNAL_H__ */
>
> But we discard __kcrctab in scripts/module.lds.S file.
No.
scripts/module.lds.S keeps __kcrctab.
The discarded sections are specified a few lines above:
/DISCARD/ : {
*(.discard)
*(.discard.*)
SANITIZER_DISCARDS
}
> Maybe we need:
No.
The problem is __crc_* symbols are dropped from vmlinux
when CONFIG_LTO_CLANG=y.
Please try this fixup:
https://lore.kernel.org/linux-kbuild/20220522160117.599023-1-masahiroy@kernel.org/T/#u
>
> $ git diff scripts/module.lds.S
> diff --git a/scripts/module.lds.S b/scripts/module.lds.S
> index 1d0e1e4dc3d2..c04b596c364b 100644
> --- a/scripts/module.lds.S
> +++ b/scripts/module.lds.S
> @@ -21,8 +21,6 @@ SECTIONS {
>
> __ksymtab 0 : { *(SORT(___ksymtab+*)) }
> __ksymtab_gpl 0 : { *(SORT(___ksymtab_gpl+*)) }
> - __kcrctab 0 : { *(SORT(___kcrctab+*)) }
> - __kcrctab_gpl 0 : { *(SORT(___kcrctab_gpl+*)) }
>
> .ctors 0 : ALIGN(8) { *(SORT(.ctors.*)) *(.ctors) }
> .init_array 0 : ALIGN(8) { *(SORT(.init_array.*))
> *(.init_array) }
>
> Or even?
>
> $ git diff scripts/kallsyms.c
> diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
> index 8caabddf817c..fb3601fe8aa3 100644
> --- a/scripts/kallsyms.c
> +++ b/scripts/kallsyms.c
> @@ -109,7 +109,6 @@ static bool is_ignored_symbol(const char *name, char type)
> static const char * const ignored_prefixes[] = {
> "$", /* local symbols for ARM, MIPS, etc. */
> ".L", /* local labels,
> .LBB,.Ltmpxxx,.L__unnamed_xx,.LASANPC, etc. */
> - "__crc_", /* modversions */
> "__efistub_", /* arm64 EFI stub namespace */
> "__kvm_nvhe_", /* arm64 non-VHE KVM namespace */
> "__AArch64ADRPThunk_", /* arm64 lld */
>
> - Sedat -
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
2022-05-22 16:09 ` Masahiro Yamada
@ 2022-05-22 16:18 ` Sedat Dilek
2022-05-22 22:34 ` Sedat Dilek
0 siblings, 1 reply; 39+ messages in thread
From: Sedat Dilek @ 2022-05-22 16:18 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
On Sun, May 22, 2022 at 6:10 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Sun, May 22, 2022 at 11:16 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> >
> > On Sun, May 22, 2022 at 11:45 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > >
> > > On Sun, May 22, 2022 at 8:50 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > >
> > > > On Sun, May 22, 2022 at 10:45 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > > > >
> > > > > On Fri, May 13, 2022 at 4:31 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > > > >
> > > > > > On Fri, May 13, 2022 at 8:42 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > > > > >
> > > > > > >
> > > > > > > This is the third batch of cleanups in this development cycle.
> > > > > > >
> > > > > >
> > > > > >
> > > > > > This series is available at
> > > > > > git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> > > > > > lto-cleanup-v6
> > > > > >
> > > > >
> > > > > Hi Masahiro,
> > > > >
> > > > > I cloned the repository on top of latest Linus Git.
> > > > >
> > > > > Not able to boot in Quemu - Not able to boot on bare metal.
> > > > >
> > > > > $ grep module_layout log_quemu-5.18.0-rc7-2-amd64-clang14-lto.txt
> > > > > 366:[ 2.173265] floppy: disagrees about version of symbol module_layout
> > > > > 367:[ 2.198746] scsi_common: disagrees about version of symbol module_layout
> > > > > 368:[ 2.205573] i2c_piix4: disagrees about version of symbol module_layout
> > > > > 369:[ 2.210610] psmouse: disagrees about version of symbol module_layout
> > > > > 370:[ 2.225138] scsi_common: disagrees about version of symbol module_layout
> > > > > 371:[ 2.235536] scsi_common: disagrees about version of symbol module_layout
> > > > > 375:Begin: Running /scripts/local-premount ... [ 2.298555]
> > > > > crc32c_intel: disagrees about version of symbol module_layout
> > > > > 376:[ 2.303335] crc32c_generic: disagrees about version of symbol
> > > > > module_layout
> > > > > 377:[ 2.306667] libcrc32c: disagrees about version of symbol module_layout
> > > > >
> > > > > Infos: LLVM-14 + CONFIG_LTO_CLANG_THIN=y
> > > > >
> > > > > My linux-config and qemu-log are attached.
> > > > >
> > > >
> > > >
> > > > Thanks for your testing.
> > > >
> > > > I was also able to reproduce this issue.
> > > >
> > > >
> > > > The problematic parts are:
> > > >
> > > > [ 2.298555] crc32c_intel: disagrees about version of symbol module_layout
> > > > [ 2.303335] crc32c_generic: disagrees about version of symbol module_layout
> > > > [ 2.306667] libcrc32c: disagrees about version of symbol module_layout
> > > >
> > > >
> > > >
> > > > When CONFIG_LTO_CLANG_THIN=y,
> > > > I cannot see any __crc_* symbols in "nm vmlinux".
> > > >
> > > > Perhaps, LTO might have discarded all the __crc_* symbols
> > > > from vmlinux, but I am still checking the details...
> > > >
> > >
> > > Thanks for taking care.
> > >
> > > Just for the records:
> > >
> > > $ grep CONFIG_MODVERSIONS /boot/config-5.18.0-rc7-2-amd64-clang14-lto
> > > CONFIG_MODVERSIONS=y
> > >
> >
> > Did not try CONFIG_MODVERSIONS=n.
> >
> > We have a new file:
> >
> > [ include/linux/export-internal.h ]
> > /* SPDX-License-Identifier: GPL-2.0-only */
> > /*
> > * Please do not include this explicitly.
> > * This is used by C files generated by modpost.
> > */
> >
> > #ifndef __LINUX_EXPORT_INTERNAL_H__
> > #define __LINUX_EXPORT_INTERNAL_H__
> >
> > #include <linux/compiler.h>
> > #include <linux/types.h>
> >
> > #define SYMBOL_CRC(sym, crc, sec) \
> > u32 __section("___kcrctab" sec "+" #sym) __crc_##sym = crc
> >
> > #endif /* __LINUX_EXPORT_INTERNAL_H__ */
> >
> > But we discard __kcrctab in scripts/module.lds.S file.
>
>
> No.
> scripts/module.lds.S keeps __kcrctab.
>
>
> The discarded sections are specified a few lines above:
>
> /DISCARD/ : {
> *(.discard)
> *(.discard.*)
> SANITIZER_DISCARDS
> }
>
>
>
>
>
>
>
>
>
> > Maybe we need:
>
>
> No.
>
> The problem is __crc_* symbols are dropped from vmlinux
> when CONFIG_LTO_CLANG=y.
>
>
> Please try this fixup:
> https://lore.kernel.org/linux-kbuild/20220522160117.599023-1-masahiroy@kernel.org/T/#u
>
Thanks!
Will give it a try - will report in a few hours.
-Sedat-
>
>
>
>
>
> >
> > $ git diff scripts/module.lds.S
> > diff --git a/scripts/module.lds.S b/scripts/module.lds.S
> > index 1d0e1e4dc3d2..c04b596c364b 100644
> > --- a/scripts/module.lds.S
> > +++ b/scripts/module.lds.S
> > @@ -21,8 +21,6 @@ SECTIONS {
> >
> > __ksymtab 0 : { *(SORT(___ksymtab+*)) }
> > __ksymtab_gpl 0 : { *(SORT(___ksymtab_gpl+*)) }
> > - __kcrctab 0 : { *(SORT(___kcrctab+*)) }
> > - __kcrctab_gpl 0 : { *(SORT(___kcrctab_gpl+*)) }
> >
> > .ctors 0 : ALIGN(8) { *(SORT(.ctors.*)) *(.ctors) }
> > .init_array 0 : ALIGN(8) { *(SORT(.init_array.*))
> > *(.init_array) }
> >
> > Or even?
> >
> > $ git diff scripts/kallsyms.c
> > diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
> > index 8caabddf817c..fb3601fe8aa3 100644
> > --- a/scripts/kallsyms.c
> > +++ b/scripts/kallsyms.c
> > @@ -109,7 +109,6 @@ static bool is_ignored_symbol(const char *name, char type)
> > static const char * const ignored_prefixes[] = {
> > "$", /* local symbols for ARM, MIPS, etc. */
> > ".L", /* local labels,
> > .LBB,.Ltmpxxx,.L__unnamed_xx,.LASANPC, etc. */
> > - "__crc_", /* modversions */
> > "__efistub_", /* arm64 EFI stub namespace */
> > "__kvm_nvhe_", /* arm64 non-VHE KVM namespace */
> > "__AArch64ADRPThunk_", /* arm64 lld */
> >
> > - Sedat -
>
>
>
> --
> Best Regards
> Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
2022-05-22 16:18 ` Sedat Dilek
@ 2022-05-22 22:34 ` Sedat Dilek
0 siblings, 0 replies; 39+ messages in thread
From: Sedat Dilek @ 2022-05-22 22:34 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
On Sun, May 22, 2022 at 6:18 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Sun, May 22, 2022 at 6:10 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > On Sun, May 22, 2022 at 11:16 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > >
> > > On Sun, May 22, 2022 at 11:45 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > > >
> > > > On Sun, May 22, 2022 at 8:50 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > > >
> > > > > On Sun, May 22, 2022 at 10:45 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
> > > > > >
> > > > > > On Fri, May 13, 2022 at 4:31 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > > > > >
> > > > > > > On Fri, May 13, 2022 at 8:42 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > This is the third batch of cleanups in this development cycle.
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > This series is available at
> > > > > > > git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> > > > > > > lto-cleanup-v6
> > > > > > >
> > > > > >
> > > > > > Hi Masahiro,
> > > > > >
> > > > > > I cloned the repository on top of latest Linus Git.
> > > > > >
> > > > > > Not able to boot in Quemu - Not able to boot on bare metal.
> > > > > >
> > > > > > $ grep module_layout log_quemu-5.18.0-rc7-2-amd64-clang14-lto.txt
> > > > > > 366:[ 2.173265] floppy: disagrees about version of symbol module_layout
> > > > > > 367:[ 2.198746] scsi_common: disagrees about version of symbol module_layout
> > > > > > 368:[ 2.205573] i2c_piix4: disagrees about version of symbol module_layout
> > > > > > 369:[ 2.210610] psmouse: disagrees about version of symbol module_layout
> > > > > > 370:[ 2.225138] scsi_common: disagrees about version of symbol module_layout
> > > > > > 371:[ 2.235536] scsi_common: disagrees about version of symbol module_layout
> > > > > > 375:Begin: Running /scripts/local-premount ... [ 2.298555]
> > > > > > crc32c_intel: disagrees about version of symbol module_layout
> > > > > > 376:[ 2.303335] crc32c_generic: disagrees about version of symbol
> > > > > > module_layout
> > > > > > 377:[ 2.306667] libcrc32c: disagrees about version of symbol module_layout
> > > > > >
> > > > > > Infos: LLVM-14 + CONFIG_LTO_CLANG_THIN=y
> > > > > >
> > > > > > My linux-config and qemu-log are attached.
> > > > > >
> > > > >
> > > > >
> > > > > Thanks for your testing.
> > > > >
> > > > > I was also able to reproduce this issue.
> > > > >
> > > > >
> > > > > The problematic parts are:
> > > > >
> > > > > [ 2.298555] crc32c_intel: disagrees about version of symbol module_layout
> > > > > [ 2.303335] crc32c_generic: disagrees about version of symbol module_layout
> > > > > [ 2.306667] libcrc32c: disagrees about version of symbol module_layout
> > > > >
> > > > >
> > > > >
> > > > > When CONFIG_LTO_CLANG_THIN=y,
> > > > > I cannot see any __crc_* symbols in "nm vmlinux".
> > > > >
> > > > > Perhaps, LTO might have discarded all the __crc_* symbols
> > > > > from vmlinux, but I am still checking the details...
> > > > >
> > > >
> > > > Thanks for taking care.
> > > >
> > > > Just for the records:
> > > >
> > > > $ grep CONFIG_MODVERSIONS /boot/config-5.18.0-rc7-2-amd64-clang14-lto
> > > > CONFIG_MODVERSIONS=y
> > > >
> > >
> > > Did not try CONFIG_MODVERSIONS=n.
> > >
> > > We have a new file:
> > >
> > > [ include/linux/export-internal.h ]
> > > /* SPDX-License-Identifier: GPL-2.0-only */
> > > /*
> > > * Please do not include this explicitly.
> > > * This is used by C files generated by modpost.
> > > */
> > >
> > > #ifndef __LINUX_EXPORT_INTERNAL_H__
> > > #define __LINUX_EXPORT_INTERNAL_H__
> > >
> > > #include <linux/compiler.h>
> > > #include <linux/types.h>
> > >
> > > #define SYMBOL_CRC(sym, crc, sec) \
> > > u32 __section("___kcrctab" sec "+" #sym) __crc_##sym = crc
> > >
> > > #endif /* __LINUX_EXPORT_INTERNAL_H__ */
> > >
> > > But we discard __kcrctab in scripts/module.lds.S file.
> >
> >
> > No.
> > scripts/module.lds.S keeps __kcrctab.
> >
> >
> > The discarded sections are specified a few lines above:
> >
> > /DISCARD/ : {
> > *(.discard)
> > *(.discard.*)
> > SANITIZER_DISCARDS
> > }
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > > Maybe we need:
> >
> >
> > No.
> >
> > The problem is __crc_* symbols are dropped from vmlinux
> > when CONFIG_LTO_CLANG=y.
> >
> >
> > Please try this fixup:
> > https://lore.kernel.org/linux-kbuild/20220522160117.599023-1-masahiroy@kernel.org/T/#u
> >
>
> Thanks!
>
> Will give it a try - will report in a few hours.
>
> -Sedat-
I was able to build and boot on bare metal.
Feel free to add my...
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM-14 (x86-64)
-Sedat-
>
> >
> >
> >
> >
> >
> > >
> > > $ git diff scripts/module.lds.S
> > > diff --git a/scripts/module.lds.S b/scripts/module.lds.S
> > > index 1d0e1e4dc3d2..c04b596c364b 100644
> > > --- a/scripts/module.lds.S
> > > +++ b/scripts/module.lds.S
> > > @@ -21,8 +21,6 @@ SECTIONS {
> > >
> > > __ksymtab 0 : { *(SORT(___ksymtab+*)) }
> > > __ksymtab_gpl 0 : { *(SORT(___ksymtab_gpl+*)) }
> > > - __kcrctab 0 : { *(SORT(___kcrctab+*)) }
> > > - __kcrctab_gpl 0 : { *(SORT(___kcrctab_gpl+*)) }
> > >
> > > .ctors 0 : ALIGN(8) { *(SORT(.ctors.*)) *(.ctors) }
> > > .init_array 0 : ALIGN(8) { *(SORT(.init_array.*))
> > > *(.init_array) }
> > >
> > > Or even?
> > >
> > > $ git diff scripts/kallsyms.c
> > > diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
> > > index 8caabddf817c..fb3601fe8aa3 100644
> > > --- a/scripts/kallsyms.c
> > > +++ b/scripts/kallsyms.c
> > > @@ -109,7 +109,6 @@ static bool is_ignored_symbol(const char *name, char type)
> > > static const char * const ignored_prefixes[] = {
> > > "$", /* local symbols for ARM, MIPS, etc. */
> > > ".L", /* local labels,
> > > .LBB,.Ltmpxxx,.L__unnamed_xx,.LASANPC, etc. */
> > > - "__crc_", /* modversions */
> > > "__efistub_", /* arm64 EFI stub namespace */
> > > "__kvm_nvhe_", /* arm64 non-VHE KVM namespace */
> > > "__AArch64ADRPThunk_", /* arm64 lld */
> > >
> > > - Sedat -
> >
> >
> >
> > --
> > Best Regards
> > Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 06/10] kbuild: check static EXPORT_SYMBOL* by script instead of modpost
2022-05-13 11:39 ` [PATCH v6 06/10] kbuild: check static EXPORT_SYMBOL* by script instead of modpost Masahiro Yamada
@ 2022-05-25 8:31 ` Guenter Roeck
2022-05-25 10:51 ` Masahiro Yamada
2022-06-07 14:22 ` Jon Hunter
1 sibling, 1 reply; 39+ messages in thread
From: Guenter Roeck @ 2022-05-25 8:31 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen
On Fri, May 13, 2022 at 08:39:26PM +0900, Masahiro Yamada wrote:
> The 'static' specifier and EXPORT_SYMBOL() are an odd combination.
>
> Commit 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL*
> functions") tried to detect it, but this check has false negatives.
>
> Here is the sample code.
>
> Makefile:
>
> obj-y += foo1.o foo2.o
>
> foo1.c:
>
> #include <linux/export.h>
> static void foo(void) {}
> EXPORT_SYMBOL(foo);
>
> foo2.c:
>
> void foo(void) {}
>
> foo1.c exports the static symbol 'foo', but modpost cannot catch it
> because it is fooled by foo2.c, which has a global symbol with the
> same name.
>
> s->is_static is cleared if a global symbol with the same name is found
> somewhere, but EXPORT_SYMBOL() and the global symbol do not necessarily
> belong to the same compilation unit.
>
> This check should be done per compilation unit, but I do not know how
> to do it in modpost. modpost runs against vmlinux.o or modules, which
> merges multiple objects, then forgets their origin.
>
> It is true modpost gets access to the lists of all the member objects
> (.vmlinux.objs and *.mod), but modpost cannot parse individual objects
> because they may not be ELF but LLVM IR when CONFIG_LTO_CLANG=y.
>
> Add a simple bash script to parse the output from ${NM}. This works for
> CONFIG_LTO_CLANG=y because llvm-nm can dump symbols of LLVM IR files.
>
On parisc builds, this patch results in:
Building parisc:allnoconfig ... failed
--------------
Error log:
scripts/check-local-export: sh /opt/buildbot/slave/next-next/build/arch/parisc/nm failed
Guenter
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 06/10] kbuild: check static EXPORT_SYMBOL* by script instead of modpost
2022-05-25 8:31 ` Guenter Roeck
@ 2022-05-25 10:51 ` Masahiro Yamada
0 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-25 10:51 UTC (permalink / raw)
To: Guenter Roeck
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
On Wed, May 25, 2022 at 5:31 PM Guenter Roeck <linux@roeck-us.net> wrote:
>
> On Fri, May 13, 2022 at 08:39:26PM +0900, Masahiro Yamada wrote:
> > The 'static' specifier and EXPORT_SYMBOL() are an odd combination.
> >
> > Commit 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL*
> > functions") tried to detect it, but this check has false negatives.
> >
> > Here is the sample code.
> >
> > Makefile:
> >
> > obj-y += foo1.o foo2.o
> >
> > foo1.c:
> >
> > #include <linux/export.h>
> > static void foo(void) {}
> > EXPORT_SYMBOL(foo);
> >
> > foo2.c:
> >
> > void foo(void) {}
> >
> > foo1.c exports the static symbol 'foo', but modpost cannot catch it
> > because it is fooled by foo2.c, which has a global symbol with the
> > same name.
> >
> > s->is_static is cleared if a global symbol with the same name is found
> > somewhere, but EXPORT_SYMBOL() and the global symbol do not necessarily
> > belong to the same compilation unit.
> >
> > This check should be done per compilation unit, but I do not know how
> > to do it in modpost. modpost runs against vmlinux.o or modules, which
> > merges multiple objects, then forgets their origin.
> >
> > It is true modpost gets access to the lists of all the member objects
> > (.vmlinux.objs and *.mod), but modpost cannot parse individual objects
> > because they may not be ELF but LLVM IR when CONFIG_LTO_CLANG=y.
> >
> > Add a simple bash script to parse the output from ${NM}. This works for
> > CONFIG_LTO_CLANG=y because llvm-nm can dump symbols of LLVM IR files.
> >
>
> On parisc builds, this patch results in:
>
> Building parisc:allnoconfig ... failed
> --------------
> Error log:
> scripts/check-local-export: sh /opt/buildbot/slave/next-next/build/arch/parisc/nm failed
>
> Guenter
Thanks for the report.
parisc overrides NM:
NM = sh $(srctree)/arch/parisc/nm
I will fix the script to return the correct exit code.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 01/10] modpost: extract symbol versions from *.cmd files
2022-05-13 11:39 ` [PATCH v6 01/10] modpost: extract symbol versions from *.cmd files Masahiro Yamada
@ 2022-05-28 22:47 ` Guenter Roeck
2022-05-29 4:27 ` Masahiro Yamada
0 siblings, 1 reply; 39+ messages in thread
From: Guenter Roeck @ 2022-05-28 22:47 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen
Hi,
On Fri, May 13, 2022 at 08:39:21PM +0900, Masahiro Yamada wrote:
> Currently, CONFIG_MODVERSIONS needs extra link to embed the symbol
> versions into ELF objects. Then, modpost extracts the version CRCs
> from them.
>
[ ... ]
> This commit changes modpost to extract CRCs from *.cmd files instead of
> from ELF objects.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
> Tested-by: Nathan Chancellor <nathan@kernel.org>
> Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
This patch results in
._muldi3_di.o.cmd: No such file or directory
when building parisc64:defconfig, and
._divsi3.o.cmd: No such file or directory
when building csky:allmodconfig.
Reverting this patch (and the subsequent kbuild patches to avoid
conflicts) fixes the problem for both architectures. In case it
helps, the complete build log when rebuilding an image for csky
is as follows.
#
# No change to .config
#
HOSTCC scripts/mod/modpost.o
HOSTLD scripts/mod/modpost
CALL scripts/atomic/check-atomics.sh
CALL scripts/checksyscalls.sh
CHK include/generated/compile.h
SO2S arch/csky/kernel/vdso/vdso-syms.S
AS arch/csky/kernel/vdso/vdso-syms.o
AR arch/csky/kernel/vdso/built-in.a
AR arch/csky/kernel/built-in.a
CHK kernel/kheaders_data.tar.xz
GEN .version
CHK include/generated/compile.h
UPD include/generated/compile.h
CC init/version.o
AR init/built-in.a
LD vmlinux.o
MODPOST vmlinux.symvers
._divsi3.o.cmd: No such file or directory
make[1]: *** [scripts/Makefile.modpost:59: vmlinux.symvers] Error 1
make: *** [Makefile:1159: vmlinux] Error 2
This was seen with gcc 11.2 and 11.3.
Guenter
---
bisect on mainline:
# bad: [9d004b2f4fea97cde123e7f1939b80e77bf2e695] Merge tag 'cxl-for-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl
# good: [4b0986a3613c92f4ec1bdc7f60ec66fea135991f] Linux 5.18
git bisect start 'HEAD' 'v5.18'
# good: [86c87bea6b42100c67418af690919c44de6ede6e] Merge tag 'devicetree-for-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux
git bisect good 86c87bea6b42100c67418af690919c44de6ede6e
# good: [c011dd537ffe47462051930413fed07dbdc80313] Merge tag 'arm-soc-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
git bisect good c011dd537ffe47462051930413fed07dbdc80313
# bad: [df202b452fe6c6d6f1351bad485e2367ef1e644e] Merge tag 'kbuild-v5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
git bisect bad df202b452fe6c6d6f1351bad485e2367ef1e644e
# good: [d4dcdc53c492a7b9fa9031cb85e238b21208ada2] Merge tag 'qcom-arm64-for-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/qcom/linux into arm/dt
git bisect good d4dcdc53c492a7b9fa9031cb85e238b21208ada2
# good: [ae862183285cbb2ef9032770d98ffa9becffe9d5] Merge tag 'arm-dt-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
git bisect good ae862183285cbb2ef9032770d98ffa9becffe9d5
# good: [cc3c470ae4ad758b8ddad825ab199f7eaa8b0a9e] Merge tag 'arm-drivers-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
git bisect good cc3c470ae4ad758b8ddad825ab199f7eaa8b0a9e
# good: [ecf0aa5317b0ad6bb015128a5b763c954fd58708] Merge tag 'arm-multiplatform-5.19-1' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
git bisect good ecf0aa5317b0ad6bb015128a5b763c954fd58708
# good: [4484054816cab940fc2fde23fa989174fec889d0] modpost: use doubly linked list for dump_lists
git bisect good 4484054816cab940fc2fde23fa989174fec889d0
# good: [16477cdfefdb494235a675cc80563d736991d833] Merge tag 'asm-generic-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/asm-generic
git bisect good 16477cdfefdb494235a675cc80563d736991d833
# good: [a44abaca0e196cfeef2374ed663b97daa1ad112a] modpost: move *.mod.c generation to write_mod_c_files()
git bisect good a44abaca0e196cfeef2374ed663b97daa1ad112a
# good: [69c4cc99bbcbf3ef2e1901b569954e9226180840] modpost: add sym_find_with_module() helper
git bisect good 69c4cc99bbcbf3ef2e1901b569954e9226180840
# bad: [7b4537199a4a8480b8c3ba37a2d44765ce76cd9b] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS
git bisect bad 7b4537199a4a8480b8c3ba37a2d44765ce76cd9b
# bad: [f292d875d0dc700b3af0bef04c5abc1dc7b3b62c] modpost: extract symbol versions from *.cmd files
git bisect bad f292d875d0dc700b3af0bef04c5abc1dc7b3b62c
# first bad commit: [f292d875d0dc700b3af0bef04c5abc1dc7b3b62c] modpost: extract symbol versions from *.cmd files
---
bisect on kbuild-5.19:
# bad: [5ce2176b81f77366bd02c27509b83049f0020544] genksyms: adjust the output format to modpost
# good: [3123109284176b1532874591f7c81f3837bbdc17] Linux 5.18-rc1
git bisect start 'HEAD' 'v5.18-rc1'
# good: [70ddb48db4aaddd3c2a7d8802463e15b21ce8525] modpost: move struct namespace_list to modpost.c
git bisect good 70ddb48db4aaddd3c2a7d8802463e15b21ce8525
# good: [e76cc48d8e6df5d949284132981db73d2dd8c6b5] modpost: make sym_add_exported() always allocate a new symbol
git bisect good e76cc48d8e6df5d949284132981db73d2dd8c6b5
# good: [78e9e56af3858bf2c52c065daa6c8bee0d72048c] kbuild: record symbol versions in *.cmd files
git bisect good 78e9e56af3858bf2c52c065daa6c8bee0d72048c
# good: [69c4cc99bbcbf3ef2e1901b569954e9226180840] modpost: add sym_find_with_module() helper
git bisect good 69c4cc99bbcbf3ef2e1901b569954e9226180840
# bad: [7b4537199a4a8480b8c3ba37a2d44765ce76cd9b] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS
git bisect bad 7b4537199a4a8480b8c3ba37a2d44765ce76cd9b
# bad: [f292d875d0dc700b3af0bef04c5abc1dc7b3b62c] modpost: extract symbol versions from *.cmd files
git bisect bad f292d875d0dc700b3af0bef04c5abc1dc7b3b62c
# first bad commit: [f292d875d0dc700b3af0bef04c5abc1dc7b3b62c] modpost: extract symbol versions from *.cmd files
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 01/10] modpost: extract symbol versions from *.cmd files
2022-05-28 22:47 ` Guenter Roeck
@ 2022-05-29 4:27 ` Masahiro Yamada
0 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-05-29 4:27 UTC (permalink / raw)
To: Guenter Roeck
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
On Sun, May 29, 2022 at 7:47 AM Guenter Roeck <linux@roeck-us.net> wrote:
>
> Hi,
>
> On Fri, May 13, 2022 at 08:39:21PM +0900, Masahiro Yamada wrote:
> > Currently, CONFIG_MODVERSIONS needs extra link to embed the symbol
> > versions into ELF objects. Then, modpost extracts the version CRCs
> > from them.
> >
> [ ... ]
> > This commit changes modpost to extract CRCs from *.cmd files instead of
> > from ELF objects.
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
> > Tested-by: Nathan Chancellor <nathan@kernel.org>
> > Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
>
> This patch results in
>
> ._muldi3_di.o.cmd: No such file or directory
>
> when building parisc64:defconfig, and
>
> ._divsi3.o.cmd: No such file or directory
>
> when building csky:allmodconfig.
>
> Reverting this patch (and the subsequent kbuild patches to avoid
> conflicts) fixes the problem for both architectures. In case it
> helps, the complete build log when rebuilding an image for csky
> is as follows.
Thank you for the report!
This commit has been in the tree for a while
but I did not get this kind of report from the 0-day bot.
The wide test coverage from your system is very helpful.
I just submitted a fix.
Thank you.
> #
> # No change to .config
> #
> HOSTCC scripts/mod/modpost.o
> HOSTLD scripts/mod/modpost
> CALL scripts/atomic/check-atomics.sh
> CALL scripts/checksyscalls.sh
> CHK include/generated/compile.h
> SO2S arch/csky/kernel/vdso/vdso-syms.S
> AS arch/csky/kernel/vdso/vdso-syms.o
> AR arch/csky/kernel/vdso/built-in.a
> AR arch/csky/kernel/built-in.a
> CHK kernel/kheaders_data.tar.xz
> GEN .version
> CHK include/generated/compile.h
> UPD include/generated/compile.h
> CC init/version.o
> AR init/built-in.a
> LD vmlinux.o
> MODPOST vmlinux.symvers
> ._divsi3.o.cmd: No such file or directory
> make[1]: *** [scripts/Makefile.modpost:59: vmlinux.symvers] Error 1
> make: *** [Makefile:1159: vmlinux] Error 2
>
> This was seen with gcc 11.2 and 11.3.
>
> Guenter
>
> ---
> bisect on mainline:
>
> # bad: [9d004b2f4fea97cde123e7f1939b80e77bf2e695] Merge tag 'cxl-for-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl
> # good: [4b0986a3613c92f4ec1bdc7f60ec66fea135991f] Linux 5.18
> git bisect start 'HEAD' 'v5.18'
> # good: [86c87bea6b42100c67418af690919c44de6ede6e] Merge tag 'devicetree-for-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux
> git bisect good 86c87bea6b42100c67418af690919c44de6ede6e
> # good: [c011dd537ffe47462051930413fed07dbdc80313] Merge tag 'arm-soc-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
> git bisect good c011dd537ffe47462051930413fed07dbdc80313
> # bad: [df202b452fe6c6d6f1351bad485e2367ef1e644e] Merge tag 'kbuild-v5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
> git bisect bad df202b452fe6c6d6f1351bad485e2367ef1e644e
> # good: [d4dcdc53c492a7b9fa9031cb85e238b21208ada2] Merge tag 'qcom-arm64-for-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/qcom/linux into arm/dt
> git bisect good d4dcdc53c492a7b9fa9031cb85e238b21208ada2
> # good: [ae862183285cbb2ef9032770d98ffa9becffe9d5] Merge tag 'arm-dt-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
> git bisect good ae862183285cbb2ef9032770d98ffa9becffe9d5
> # good: [cc3c470ae4ad758b8ddad825ab199f7eaa8b0a9e] Merge tag 'arm-drivers-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
> git bisect good cc3c470ae4ad758b8ddad825ab199f7eaa8b0a9e
> # good: [ecf0aa5317b0ad6bb015128a5b763c954fd58708] Merge tag 'arm-multiplatform-5.19-1' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
> git bisect good ecf0aa5317b0ad6bb015128a5b763c954fd58708
> # good: [4484054816cab940fc2fde23fa989174fec889d0] modpost: use doubly linked list for dump_lists
> git bisect good 4484054816cab940fc2fde23fa989174fec889d0
> # good: [16477cdfefdb494235a675cc80563d736991d833] Merge tag 'asm-generic-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/asm-generic
> git bisect good 16477cdfefdb494235a675cc80563d736991d833
> # good: [a44abaca0e196cfeef2374ed663b97daa1ad112a] modpost: move *.mod.c generation to write_mod_c_files()
> git bisect good a44abaca0e196cfeef2374ed663b97daa1ad112a
> # good: [69c4cc99bbcbf3ef2e1901b569954e9226180840] modpost: add sym_find_with_module() helper
> git bisect good 69c4cc99bbcbf3ef2e1901b569954e9226180840
> # bad: [7b4537199a4a8480b8c3ba37a2d44765ce76cd9b] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS
> git bisect bad 7b4537199a4a8480b8c3ba37a2d44765ce76cd9b
> # bad: [f292d875d0dc700b3af0bef04c5abc1dc7b3b62c] modpost: extract symbol versions from *.cmd files
> git bisect bad f292d875d0dc700b3af0bef04c5abc1dc7b3b62c
> # first bad commit: [f292d875d0dc700b3af0bef04c5abc1dc7b3b62c] modpost: extract symbol versions from *.cmd files
>
> ---
> bisect on kbuild-5.19:
>
> # bad: [5ce2176b81f77366bd02c27509b83049f0020544] genksyms: adjust the output format to modpost
> # good: [3123109284176b1532874591f7c81f3837bbdc17] Linux 5.18-rc1
> git bisect start 'HEAD' 'v5.18-rc1'
> # good: [70ddb48db4aaddd3c2a7d8802463e15b21ce8525] modpost: move struct namespace_list to modpost.c
> git bisect good 70ddb48db4aaddd3c2a7d8802463e15b21ce8525
> # good: [e76cc48d8e6df5d949284132981db73d2dd8c6b5] modpost: make sym_add_exported() always allocate a new symbol
> git bisect good e76cc48d8e6df5d949284132981db73d2dd8c6b5
> # good: [78e9e56af3858bf2c52c065daa6c8bee0d72048c] kbuild: record symbol versions in *.cmd files
> git bisect good 78e9e56af3858bf2c52c065daa6c8bee0d72048c
> # good: [69c4cc99bbcbf3ef2e1901b569954e9226180840] modpost: add sym_find_with_module() helper
> git bisect good 69c4cc99bbcbf3ef2e1901b569954e9226180840
> # bad: [7b4537199a4a8480b8c3ba37a2d44765ce76cd9b] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS
> git bisect bad 7b4537199a4a8480b8c3ba37a2d44765ce76cd9b
> # bad: [f292d875d0dc700b3af0bef04c5abc1dc7b3b62c] modpost: extract symbol versions from *.cmd files
> git bisect bad f292d875d0dc700b3af0bef04c5abc1dc7b3b62c
> # first bad commit: [f292d875d0dc700b3af0bef04c5abc1dc7b3b62c] modpost: extract symbol versions from *.cmd files
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 06/10] kbuild: check static EXPORT_SYMBOL* by script instead of modpost
2022-05-13 11:39 ` [PATCH v6 06/10] kbuild: check static EXPORT_SYMBOL* by script instead of modpost Masahiro Yamada
2022-05-25 8:31 ` Guenter Roeck
@ 2022-06-07 14:22 ` Jon Hunter
2022-06-07 14:25 ` Nathan Chancellor
1 sibling, 1 reply; 39+ messages in thread
From: Jon Hunter @ 2022-06-07 14:22 UTC (permalink / raw)
To: Masahiro Yamada, linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, linux-tegra
On 13/05/2022 12:39, Masahiro Yamada wrote:
> The 'static' specifier and EXPORT_SYMBOL() are an odd combination.
>
> Commit 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL*
> functions") tried to detect it, but this check has false negatives.
>
> Here is the sample code.
>
> Makefile:
>
> obj-y += foo1.o foo2.o
>
> foo1.c:
>
> #include <linux/export.h>
> static void foo(void) {}
> EXPORT_SYMBOL(foo);
>
> foo2.c:
>
> void foo(void) {}
>
> foo1.c exports the static symbol 'foo', but modpost cannot catch it
> because it is fooled by foo2.c, which has a global symbol with the
> same name.
>
> s->is_static is cleared if a global symbol with the same name is found
> somewhere, but EXPORT_SYMBOL() and the global symbol do not necessarily
> belong to the same compilation unit.
>
> This check should be done per compilation unit, but I do not know how
> to do it in modpost. modpost runs against vmlinux.o or modules, which
> merges multiple objects, then forgets their origin.
>
> It is true modpost gets access to the lists of all the member objects
> (.vmlinux.objs and *.mod), but modpost cannot parse individual objects
> because they may not be ELF but LLVM IR when CONFIG_LTO_CLANG=y.
>
> Add a simple bash script to parse the output from ${NM}. This works for
> CONFIG_LTO_CLANG=y because llvm-nm can dump symbols of LLVM IR files.
>
> Revert 15bfc2348d54.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
> Tested-by: Nathan Chancellor <nathan@kernel.org>
One some older build machines this is causing some builds (ARM/ARM64)
to fail ...
/dvs/git/dirty/git-master_l4t-upstream/kernel/scripts/check-local-export: line 54: wait: pid 48433 is not a child of this shell
/dvs/git/dirty/git-master_l4t-upstream/kernel/scripts/Makefile.build:250: recipe for target 'scripts/mod/empty.o' failed
make[2]: *** [scripts/mod/empty.o] Error 127
make[2]: *** Deleting file 'scripts/mod/empty.o'
make[2]: *** Waiting for unfinished jobs....
/dvs/git/dirty/git-master_l4t-upstream/kernel/Makefile:1285: recipe for target 'prepare0' failed
make[1]: *** [prepare0] Error 2
make[1]: Leaving directory '/dvs/git/dirty/git-master_l4t-upstream/artifacts/linux/arm64-defconfig-jetson'
Makefile:228: recipe for target '__sub-make' failed
make: *** [__sub-make] Error 2
Any ideas?
Thanks
Jon
--
nvpublic
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 06/10] kbuild: check static EXPORT_SYMBOL* by script instead of modpost
2022-06-07 14:22 ` Jon Hunter
@ 2022-06-07 14:25 ` Nathan Chancellor
2022-06-07 14:34 ` Jon Hunter
0 siblings, 1 reply; 39+ messages in thread
From: Nathan Chancellor @ 2022-06-07 14:25 UTC (permalink / raw)
To: Jon Hunter
Cc: Masahiro Yamada, linux-kbuild, linux-kernel, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, linux-tegra
On Tue, Jun 07, 2022 at 03:22:21PM +0100, Jon Hunter wrote:
>
> On 13/05/2022 12:39, Masahiro Yamada wrote:
> > The 'static' specifier and EXPORT_SYMBOL() are an odd combination.
> >
> > Commit 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL*
> > functions") tried to detect it, but this check has false negatives.
> >
> > Here is the sample code.
> >
> > Makefile:
> >
> > obj-y += foo1.o foo2.o
> >
> > foo1.c:
> >
> > #include <linux/export.h>
> > static void foo(void) {}
> > EXPORT_SYMBOL(foo);
> >
> > foo2.c:
> >
> > void foo(void) {}
> >
> > foo1.c exports the static symbol 'foo', but modpost cannot catch it
> > because it is fooled by foo2.c, which has a global symbol with the
> > same name.
> >
> > s->is_static is cleared if a global symbol with the same name is found
> > somewhere, but EXPORT_SYMBOL() and the global symbol do not necessarily
> > belong to the same compilation unit.
> >
> > This check should be done per compilation unit, but I do not know how
> > to do it in modpost. modpost runs against vmlinux.o or modules, which
> > merges multiple objects, then forgets their origin.
> >
> > It is true modpost gets access to the lists of all the member objects
> > (.vmlinux.objs and *.mod), but modpost cannot parse individual objects
> > because they may not be ELF but LLVM IR when CONFIG_LTO_CLANG=y.
> >
> > Add a simple bash script to parse the output from ${NM}. This works for
> > CONFIG_LTO_CLANG=y because llvm-nm can dump symbols of LLVM IR files.
> >
> > Revert 15bfc2348d54.
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
> > Tested-by: Nathan Chancellor <nathan@kernel.org>
>
>
> One some older build machines this is causing some builds (ARM/ARM64)
> to fail ...
>
> /dvs/git/dirty/git-master_l4t-upstream/kernel/scripts/check-local-export: line 54: wait: pid 48433 is not a child of this shell
> /dvs/git/dirty/git-master_l4t-upstream/kernel/scripts/Makefile.build:250: recipe for target 'scripts/mod/empty.o' failed
> make[2]: *** [scripts/mod/empty.o] Error 127
> make[2]: *** Deleting file 'scripts/mod/empty.o'
> make[2]: *** Waiting for unfinished jobs....
> /dvs/git/dirty/git-master_l4t-upstream/kernel/Makefile:1285: recipe for target 'prepare0' failed
> make[1]: *** [prepare0] Error 2
> make[1]: Leaving directory '/dvs/git/dirty/git-master_l4t-upstream/artifacts/linux/arm64-defconfig-jetson'
> Makefile:228: recipe for target '__sub-make' failed
> make: *** [__sub-make] Error 2
>
> Any ideas?
https://lore.kernel.org/20220607084317.211785-1-masahiroy@kernel.org/
should resolve it if you wanted to give it a test.
Cheers,
Nathan
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 06/10] kbuild: check static EXPORT_SYMBOL* by script instead of modpost
2022-06-07 14:25 ` Nathan Chancellor
@ 2022-06-07 14:34 ` Jon Hunter
0 siblings, 0 replies; 39+ messages in thread
From: Jon Hunter @ 2022-06-07 14:34 UTC (permalink / raw)
To: Nathan Chancellor
Cc: Masahiro Yamada, linux-kbuild, linux-kernel, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, linux-tegra
On 07/06/2022 15:25, Nathan Chancellor wrote:
> On Tue, Jun 07, 2022 at 03:22:21PM +0100, Jon Hunter wrote:
>>
>> On 13/05/2022 12:39, Masahiro Yamada wrote:
>>> The 'static' specifier and EXPORT_SYMBOL() are an odd combination.
>>>
>>> Commit 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL*
>>> functions") tried to detect it, but this check has false negatives.
>>>
>>> Here is the sample code.
>>>
>>> Makefile:
>>>
>>> obj-y += foo1.o foo2.o
>>>
>>> foo1.c:
>>>
>>> #include <linux/export.h>
>>> static void foo(void) {}
>>> EXPORT_SYMBOL(foo);
>>>
>>> foo2.c:
>>>
>>> void foo(void) {}
>>>
>>> foo1.c exports the static symbol 'foo', but modpost cannot catch it
>>> because it is fooled by foo2.c, which has a global symbol with the
>>> same name.
>>>
>>> s->is_static is cleared if a global symbol with the same name is found
>>> somewhere, but EXPORT_SYMBOL() and the global symbol do not necessarily
>>> belong to the same compilation unit.
>>>
>>> This check should be done per compilation unit, but I do not know how
>>> to do it in modpost. modpost runs against vmlinux.o or modules, which
>>> merges multiple objects, then forgets their origin.
>>>
>>> It is true modpost gets access to the lists of all the member objects
>>> (.vmlinux.objs and *.mod), but modpost cannot parse individual objects
>>> because they may not be ELF but LLVM IR when CONFIG_LTO_CLANG=y.
>>>
>>> Add a simple bash script to parse the output from ${NM}. This works for
>>> CONFIG_LTO_CLANG=y because llvm-nm can dump symbols of LLVM IR files.
>>>
>>> Revert 15bfc2348d54.
>>>
>>> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
>>> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
>>> Tested-by: Nathan Chancellor <nathan@kernel.org>
>>
>>
>> One some older build machines this is causing some builds (ARM/ARM64)
>> to fail ...
>>
>> /dvs/git/dirty/git-master_l4t-upstream/kernel/scripts/check-local-export: line 54: wait: pid 48433 is not a child of this shell
>> /dvs/git/dirty/git-master_l4t-upstream/kernel/scripts/Makefile.build:250: recipe for target 'scripts/mod/empty.o' failed
>> make[2]: *** [scripts/mod/empty.o] Error 127
>> make[2]: *** Deleting file 'scripts/mod/empty.o'
>> make[2]: *** Waiting for unfinished jobs....
>> /dvs/git/dirty/git-master_l4t-upstream/kernel/Makefile:1285: recipe for target 'prepare0' failed
>> make[1]: *** [prepare0] Error 2
>> make[1]: Leaving directory '/dvs/git/dirty/git-master_l4t-upstream/artifacts/linux/arm64-defconfig-jetson'
>> Makefile:228: recipe for target '__sub-make' failed
>> make: *** [__sub-make] Error 2
>>
>> Any ideas?
>
> https://lore.kernel.org/20220607084317.211785-1-masahiroy@kernel.org/
> should resolve it if you wanted to give it a test.
Thanks! Works for me.
Tested-by: Jon Hunter <jonathanh@nvidia.com>
Cheers
Jon
--
nvpublic
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
[not found] ` <2c496d24174e63b27ec047f383df6700@matoro.tk>
@ 2022-08-06 7:23 ` Sedat Dilek
2022-08-06 13:12 ` matoro
2022-08-08 5:54 ` Masahiro Yamada
1 sibling, 1 reply; 39+ messages in thread
From: Sedat Dilek @ 2022-08-06 7:23 UTC (permalink / raw)
To: matoro
Cc: Masahiro Yamada, Linux Kbuild mailing list,
Linux Kernel Mailing List, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, clang-built-linux,
Ard Biesheuvel, Sami Tolvanen
On Sat, Aug 6, 2022 at 1:40 AM matoro
<matoro_mailinglist_kernel@matoro.tk> wrote:
>
> Hi Masahiro, I'm sorry to raise this after release but this seems to be
> broken on SOME architectures. So far I have tested:
>
> Affected - sparc, alpha
> Unaffected - riscv, ia64
>
What do you mean by "release"?
Checking the kernel-version from your paste-URLs:
# Automatically generated file; DO NOT EDIT.
# Linux/sparc64 5.18.16-gentoo Kernel Configuration
# Automatically generated file; DO NOT EDIT.
# Linux/alpha 5.18.15-gentoo Kernel Configuration
# Automatically generated file; DO NOT EDIT.
# Linux/riscv 5.19.0-gentoo Kernel Configuration
# Automatically generated file; DO NOT EDIT.
# Linux/ia64 5.19.0-gentoo Kernel Configuration
Maybe you should try "Affected - sparc, alpha" with Linux v5.19.0
instead of 5.18.x :-)?
-Sedat-
> The affected systems are unable to load modules, similar to the
> previously reported issue. All module loading fails with "disagrees
> about version of symbol module_layout".
>
> Bisect blames 7b4537199a4a8480b8c3ba37a2d44765ce76cd9b, but this does
> not revert cleanly. Presumably CONFIG_MODVERSIONS=n would fix, but this
> is a pretty core feature.
>
> Unlike the issue Sedat reported, this is on a GNU toolchain, no clang
> involved.
>
> Here are the configs I am using (with make olddefconfig on upgrade to
> 5.19):
>
> Broken - sparc - https://dpaste.com/5A8F2JD6U
> Broken - alpha - https://dpaste.com/FYKK23L9X
> Working - riscv - https://dpaste.com/HV6Y4V6NT
> Working - ia64 - https://dpaste.com/HDLDNEAK4
>
> Please let me know if there's anything I can do to help track down this
> regression.
>
>
> -------- Original Message --------
> Subject: Re: [PATCH v6 00/10] kbuild: yet another series of cleanups
> (modpost, LTO, MODULE_REL_CRCS, export.h)
> Date: 2022-05-13 08:20
> From: Masahiro Yamada <masahiroy@kernel.org>
> To: Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>
>
> On Fri, May 13, 2022 at 8:42 PM Masahiro Yamada <masahiroy@kernel.org>
> wrote:
> >
> >
> > This is the third batch of cleanups in this development cycle.
> >
>
>
> This series is available at
> git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> lto-cleanup-v6
>
>
> >
> > Changes in v6:
> > - Fix false-positive warnings when CONFIG_TRIM_UNUSED_KSYMS=y
> >
> > Changes in v5:
> > - Fix the build error when CONFIG_DEBUG_INFO_BTF=y (reported by
> > Nathan)
> > - Clean up arch/m68k/include/asm/export.h (Nick)
> > - Keep gen_symversions (and will be removed by a later patch)
> > - Add more comments in the script
> >
> > Changes in v4:
> > - Rename .vmlinux-symver.c to .vmlinux.export.c
> > because I notice this approach is useful for further cleanups,
> > not only for modversioning but also for overall EXPORT_SYMBOL.
> > - New patch
> > - New.
> > Resent of my previous submission.
> >
> > https://lore.kernel.org/all/20210831074004.3195284-10-masahiroy@kernel.org/
> > - New
> > Resent of my previous submission
> >
> > https://lore.kernel.org/linux-kbuild/20210831074004.3195284-11-masahiroy@kernel.org/
> >
> > Changes in v3:
> > - New patch
> >
> > Changes in v2:
> > - Simplify the implementation (parse .cmd files after ELF)
> > - New patch
> > - replace the chain of $(if ...) with $(and )
> > - New patch
> > - New patch
> >
> > Masahiro Yamada (10):
> > modpost: extract symbol versions from *.cmd files
> > kbuild: link symbol CRCs at final link, removing
> > CONFIG_MODULE_REL_CRCS
> > kbuild: stop merging *.symversions
> > genksyms: adjust the output format to modpost
> > kbuild: do not create *.prelink.o for Clang LTO or IBT
> > kbuild: check static EXPORT_SYMBOL* by script instead of modpost
> > kbuild: make built-in.a rule robust against too long argument error
> > kbuild: make *.mod rule robust against too long argument error
> > kbuild: add cmd_and_savecmd macro
> > kbuild: rebuild multi-object modules when objtool is updated
> >
> > arch/m68k/include/asm/Kbuild | 1 +
> > arch/m68k/include/asm/export.h | 2 -
> > arch/powerpc/Kconfig | 1 -
> > arch/s390/Kconfig | 1 -
> > arch/um/Kconfig | 1 -
> > include/asm-generic/export.h | 22 ++-
> > include/linux/export-internal.h | 16 +++
> > include/linux/export.h | 30 ++--
> > init/Kconfig | 4 -
> > kernel/module.c | 10 +-
> > scripts/Kbuild.include | 10 +-
> > scripts/Makefile.build | 134 ++++++------------
> > scripts/Makefile.lib | 7 -
> > scripts/Makefile.modfinal | 5 +-
> > scripts/Makefile.modpost | 9 +-
> > scripts/check-local-export | 64 +++++++++
> > scripts/genksyms/genksyms.c | 18 +--
> > scripts/link-vmlinux.sh | 33 ++---
> > scripts/mod/modpost.c | 236 +++++++++++++++++++++-----------
> > 19 files changed, 320 insertions(+), 284 deletions(-)
> > delete mode 100644 arch/m68k/include/asm/export.h
> > create mode 100644 include/linux/export-internal.h
> > create mode 100755 scripts/check-local-export
> >
> > --
> > 2.32.0
> >
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
2022-08-06 7:23 ` Sedat Dilek
@ 2022-08-06 13:12 ` matoro
0 siblings, 0 replies; 39+ messages in thread
From: matoro @ 2022-08-06 13:12 UTC (permalink / raw)
To: sedat.dilek
Cc: Masahiro Yamada, Linux Kbuild mailing list,
Linux Kernel Mailing List, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, clang-built-linux,
Ard Biesheuvel, Sami Tolvanen
Hi Sedat - I pulled these configs from the running systems, but I used
"make olddefconfig" on these working 5.18 configs to generate the
corresponding 5.19 configs that failed to boot. I'll see if I can go
back in and repeat this process to regen the 5.19 configs (I deleted it
all after 5.19 failed to boot).
-------- Original Message --------
Subject: Re: [PATCH v6 00/10] kbuild: yet another series of cleanups
(modpost, LTO, MODULE_REL_CRCS, export.h)
Date: 2022-08-06 03:23
From: Sedat Dilek <sedat.dilek@gmail.com>
To: matoro <matoro_mailinglist_kernel@matoro.tk>
On Sat, Aug 6, 2022 at 1:40 AM matoro
<matoro_mailinglist_kernel@matoro.tk> wrote:
>
> Hi Masahiro, I'm sorry to raise this after release but this seems to be
> broken on SOME architectures. So far I have tested:
>
> Affected - sparc, alpha
> Unaffected - riscv, ia64
>
What do you mean by "release"?
Checking the kernel-version from your paste-URLs:
# Automatically generated file; DO NOT EDIT.
# Linux/sparc64 5.18.16-gentoo Kernel Configuration
# Automatically generated file; DO NOT EDIT.
# Linux/alpha 5.18.15-gentoo Kernel Configuration
# Automatically generated file; DO NOT EDIT.
# Linux/riscv 5.19.0-gentoo Kernel Configuration
# Automatically generated file; DO NOT EDIT.
# Linux/ia64 5.19.0-gentoo Kernel Configuration
Maybe you should try "Affected - sparc, alpha" with Linux v5.19.0
instead of 5.18.x :-)?
-Sedat-
> The affected systems are unable to load modules, similar to the
> previously reported issue. All module loading fails with "disagrees
> about version of symbol module_layout".
>
> Bisect blames 7b4537199a4a8480b8c3ba37a2d44765ce76cd9b, but this does
> not revert cleanly. Presumably CONFIG_MODVERSIONS=n would fix, but
> this
> is a pretty core feature.
>
> Unlike the issue Sedat reported, this is on a GNU toolchain, no clang
> involved.
>
> Here are the configs I am using (with make olddefconfig on upgrade to
> 5.19):
>
> Broken - sparc - https://dpaste.com/5A8F2JD6U
> Broken - alpha - https://dpaste.com/FYKK23L9X
> Working - riscv - https://dpaste.com/HV6Y4V6NT
> Working - ia64 - https://dpaste.com/HDLDNEAK4
>
> Please let me know if there's anything I can do to help track down this
> regression.
>
>
> -------- Original Message --------
> Subject: Re: [PATCH v6 00/10] kbuild: yet another series of cleanups
> (modpost, LTO, MODULE_REL_CRCS, export.h)
> Date: 2022-05-13 08:20
> From: Masahiro Yamada <masahiroy@kernel.org>
> To: Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>
>
> On Fri, May 13, 2022 at 8:42 PM Masahiro Yamada <masahiroy@kernel.org>
> wrote:
> >
> >
> > This is the third batch of cleanups in this development cycle.
> >
>
>
> This series is available at
> git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> lto-cleanup-v6
>
>
> >
> > Changes in v6:
> > - Fix false-positive warnings when CONFIG_TRIM_UNUSED_KSYMS=y
> >
> > Changes in v5:
> > - Fix the build error when CONFIG_DEBUG_INFO_BTF=y (reported by
> > Nathan)
> > - Clean up arch/m68k/include/asm/export.h (Nick)
> > - Keep gen_symversions (and will be removed by a later patch)
> > - Add more comments in the script
> >
> > Changes in v4:
> > - Rename .vmlinux-symver.c to .vmlinux.export.c
> > because I notice this approach is useful for further cleanups,
> > not only for modversioning but also for overall EXPORT_SYMBOL.
> > - New patch
> > - New.
> > Resent of my previous submission.
> >
> > https://lore.kernel.org/all/20210831074004.3195284-10-masahiroy@kernel.org/
> > - New
> > Resent of my previous submission
> >
> > https://lore.kernel.org/linux-kbuild/20210831074004.3195284-11-masahiroy@kernel.org/
> >
> > Changes in v3:
> > - New patch
> >
> > Changes in v2:
> > - Simplify the implementation (parse .cmd files after ELF)
> > - New patch
> > - replace the chain of $(if ...) with $(and )
> > - New patch
> > - New patch
> >
> > Masahiro Yamada (10):
> > modpost: extract symbol versions from *.cmd files
> > kbuild: link symbol CRCs at final link, removing
> > CONFIG_MODULE_REL_CRCS
> > kbuild: stop merging *.symversions
> > genksyms: adjust the output format to modpost
> > kbuild: do not create *.prelink.o for Clang LTO or IBT
> > kbuild: check static EXPORT_SYMBOL* by script instead of modpost
> > kbuild: make built-in.a rule robust against too long argument error
> > kbuild: make *.mod rule robust against too long argument error
> > kbuild: add cmd_and_savecmd macro
> > kbuild: rebuild multi-object modules when objtool is updated
> >
> > arch/m68k/include/asm/Kbuild | 1 +
> > arch/m68k/include/asm/export.h | 2 -
> > arch/powerpc/Kconfig | 1 -
> > arch/s390/Kconfig | 1 -
> > arch/um/Kconfig | 1 -
> > include/asm-generic/export.h | 22 ++-
> > include/linux/export-internal.h | 16 +++
> > include/linux/export.h | 30 ++--
> > init/Kconfig | 4 -
> > kernel/module.c | 10 +-
> > scripts/Kbuild.include | 10 +-
> > scripts/Makefile.build | 134 ++++++------------
> > scripts/Makefile.lib | 7 -
> > scripts/Makefile.modfinal | 5 +-
> > scripts/Makefile.modpost | 9 +-
> > scripts/check-local-export | 64 +++++++++
> > scripts/genksyms/genksyms.c | 18 +--
> > scripts/link-vmlinux.sh | 33 ++---
> > scripts/mod/modpost.c | 236 +++++++++++++++++++++-----------
> > 19 files changed, 320 insertions(+), 284 deletions(-)
> > delete mode 100644 arch/m68k/include/asm/export.h
> > create mode 100644 include/linux/export-internal.h
> > create mode 100755 scripts/check-local-export
> >
> > --
> > 2.32.0
> >
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
[not found] ` <2c496d24174e63b27ec047f383df6700@matoro.tk>
2022-08-06 7:23 ` Sedat Dilek
@ 2022-08-08 5:54 ` Masahiro Yamada
2022-08-08 13:26 ` matoro
1 sibling, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2022-08-08 5:54 UTC (permalink / raw)
To: matoro
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
Hi Matoro,
Thanks for the report.
I have qemu-system-sparc installed on my Ubuntu machine,
so I think I will be able to test it on sparc, at least.
How did you test these 4 architectures (sparc, alpha, riscv, ia64)?
QEMU?
Thanks
On Sat, Aug 6, 2022 at 8:39 AM matoro
<matoro_mailinglist_kernel@matoro.tk> wrote:
>
> Hi Masahiro, I'm sorry to raise this after release but this seems to be
> broken on SOME architectures. So far I have tested:
>
> Affected - sparc, alpha
> Unaffected - riscv, ia64
>
> The affected systems are unable to load modules, similar to the
> previously reported issue. All module loading fails with "disagrees
> about version of symbol module_layout".
>
> Bisect blames 7b4537199a4a8480b8c3ba37a2d44765ce76cd9b, but this does
> not revert cleanly. Presumably CONFIG_MODVERSIONS=n would fix, but this
> is a pretty core feature.
>
> Unlike the issue Sedat reported, this is on a GNU toolchain, no clang
> involved.
>
> Here are the configs I am using (with make olddefconfig on upgrade to
> 5.19):
>
> Broken - sparc - https://dpaste.com/5A8F2JD6U
> Broken - alpha - https://dpaste.com/FYKK23L9X
> Working - riscv - https://dpaste.com/HV6Y4V6NT
> Working - ia64 - https://dpaste.com/HDLDNEAK4
>
> Please let me know if there's anything I can do to help track down this
> regression.
>
>
> -------- Original Message --------
> Subject: Re: [PATCH v6 00/10] kbuild: yet another series of cleanups
> (modpost, LTO, MODULE_REL_CRCS, export.h)
> Date: 2022-05-13 08:20
> From: Masahiro Yamada <masahiroy@kernel.org>
> To: Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>
>
> On Fri, May 13, 2022 at 8:42 PM Masahiro Yamada <masahiroy@kernel.org>
> wrote:
> >
> >
> > This is the third batch of cleanups in this development cycle.
> >
>
>
> This series is available at
> git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> lto-cleanup-v6
>
>
> >
> > Changes in v6:
> > - Fix false-positive warnings when CONFIG_TRIM_UNUSED_KSYMS=y
> >
> > Changes in v5:
> > - Fix the build error when CONFIG_DEBUG_INFO_BTF=y (reported by
> > Nathan)
> > - Clean up arch/m68k/include/asm/export.h (Nick)
> > - Keep gen_symversions (and will be removed by a later patch)
> > - Add more comments in the script
> >
> > Changes in v4:
> > - Rename .vmlinux-symver.c to .vmlinux.export.c
> > because I notice this approach is useful for further cleanups,
> > not only for modversioning but also for overall EXPORT_SYMBOL.
> > - New patch
> > - New.
> > Resent of my previous submission.
> >
> > https://lore.kernel.org/all/20210831074004.3195284-10-masahiroy@kernel.org/
> > - New
> > Resent of my previous submission
> >
> > https://lore.kernel.org/linux-kbuild/20210831074004.3195284-11-masahiroy@kernel.org/
> >
> > Changes in v3:
> > - New patch
> >
> > Changes in v2:
> > - Simplify the implementation (parse .cmd files after ELF)
> > - New patch
> > - replace the chain of $(if ...) with $(and )
> > - New patch
> > - New patch
> >
> > Masahiro Yamada (10):
> > modpost: extract symbol versions from *.cmd files
> > kbuild: link symbol CRCs at final link, removing
> > CONFIG_MODULE_REL_CRCS
> > kbuild: stop merging *.symversions
> > genksyms: adjust the output format to modpost
> > kbuild: do not create *.prelink.o for Clang LTO or IBT
> > kbuild: check static EXPORT_SYMBOL* by script instead of modpost
> > kbuild: make built-in.a rule robust against too long argument error
> > kbuild: make *.mod rule robust against too long argument error
> > kbuild: add cmd_and_savecmd macro
> > kbuild: rebuild multi-object modules when objtool is updated
> >
> > arch/m68k/include/asm/Kbuild | 1 +
> > arch/m68k/include/asm/export.h | 2 -
> > arch/powerpc/Kconfig | 1 -
> > arch/s390/Kconfig | 1 -
> > arch/um/Kconfig | 1 -
> > include/asm-generic/export.h | 22 ++-
> > include/linux/export-internal.h | 16 +++
> > include/linux/export.h | 30 ++--
> > init/Kconfig | 4 -
> > kernel/module.c | 10 +-
> > scripts/Kbuild.include | 10 +-
> > scripts/Makefile.build | 134 ++++++------------
> > scripts/Makefile.lib | 7 -
> > scripts/Makefile.modfinal | 5 +-
> > scripts/Makefile.modpost | 9 +-
> > scripts/check-local-export | 64 +++++++++
> > scripts/genksyms/genksyms.c | 18 +--
> > scripts/link-vmlinux.sh | 33 ++---
> > scripts/mod/modpost.c | 236 +++++++++++++++++++++-----------
> > 19 files changed, 320 insertions(+), 284 deletions(-)
> > delete mode 100644 arch/m68k/include/asm/export.h
> > create mode 100644 include/linux/export-internal.h
> > create mode 100755 scripts/check-local-export
> >
> > --
> > 2.32.0
> >
>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
2022-08-08 5:54 ` Masahiro Yamada
@ 2022-08-08 13:26 ` matoro
2022-08-08 17:36 ` Masahiro Yamada
0 siblings, 1 reply; 39+ messages in thread
From: matoro @ 2022-08-08 13:26 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
I have real hardware for all these arches in my collection. I use it
for testing the latest kernel and toolchains on as many of the
less-popular arches as possible, exactly to find issues like this one :)
Specifically we support all of these in Gentoo. To double-check this
wasn't a config issue, I asked another user who also runs sparc to try
building 5.19 with his config (not copying mine), and he observed the
same problem. You can reach us in #gentoo-sparc on Libera.
As for testing, I make all this hardware available on an as-needed
basis. So if you can't or don't want to fiddle with qemu, just let me
know (email or IRC, same username on Libera), and I will get you direct
access to my hardware. Thanks!!
-------- Original Message --------
Subject: Re: [PATCH v6 00/10] kbuild: yet another series of cleanups
(modpost, LTO, MODULE_REL_CRCS, export.h)
Date: 2022-08-08 01:54
From: Masahiro Yamada <masahiroy@kernel.org>
To: matoro <matoro_mailinglist_kernel@matoro.tk>
Hi Matoro,
Thanks for the report.
I have qemu-system-sparc installed on my Ubuntu machine,
so I think I will be able to test it on sparc, at least.
How did you test these 4 architectures (sparc, alpha, riscv, ia64)?
QEMU?
Thanks
On Sat, Aug 6, 2022 at 8:39 AM matoro
<matoro_mailinglist_kernel@matoro.tk> wrote:
>
> Hi Masahiro, I'm sorry to raise this after release but this seems to be
> broken on SOME architectures. So far I have tested:
>
> Affected - sparc, alpha
> Unaffected - riscv, ia64
>
> The affected systems are unable to load modules, similar to the
> previously reported issue. All module loading fails with "disagrees
> about version of symbol module_layout".
>
> Bisect blames 7b4537199a4a8480b8c3ba37a2d44765ce76cd9b, but this does
> not revert cleanly. Presumably CONFIG_MODVERSIONS=n would fix, but
> this
> is a pretty core feature.
>
> Unlike the issue Sedat reported, this is on a GNU toolchain, no clang
> involved.
>
> Here are the configs I am using (with make olddefconfig on upgrade to
> 5.19):
>
> Broken - sparc - https://dpaste.com/5A8F2JD6U
> Broken - alpha - https://dpaste.com/FYKK23L9X
> Working - riscv - https://dpaste.com/HV6Y4V6NT
> Working - ia64 - https://dpaste.com/HDLDNEAK4
>
> Please let me know if there's anything I can do to help track down this
> regression.
>
>
> -------- Original Message --------
> Subject: Re: [PATCH v6 00/10] kbuild: yet another series of cleanups
> (modpost, LTO, MODULE_REL_CRCS, export.h)
> Date: 2022-05-13 08:20
> From: Masahiro Yamada <masahiroy@kernel.org>
> To: Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>
>
> On Fri, May 13, 2022 at 8:42 PM Masahiro Yamada <masahiroy@kernel.org>
> wrote:
> >
> >
> > This is the third batch of cleanups in this development cycle.
> >
>
>
> This series is available at
> git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
> lto-cleanup-v6
>
>
> >
> > Changes in v6:
> > - Fix false-positive warnings when CONFIG_TRIM_UNUSED_KSYMS=y
> >
> > Changes in v5:
> > - Fix the build error when CONFIG_DEBUG_INFO_BTF=y (reported by
> > Nathan)
> > - Clean up arch/m68k/include/asm/export.h (Nick)
> > - Keep gen_symversions (and will be removed by a later patch)
> > - Add more comments in the script
> >
> > Changes in v4:
> > - Rename .vmlinux-symver.c to .vmlinux.export.c
> > because I notice this approach is useful for further cleanups,
> > not only for modversioning but also for overall EXPORT_SYMBOL.
> > - New patch
> > - New.
> > Resent of my previous submission.
> >
> > https://lore.kernel.org/all/20210831074004.3195284-10-masahiroy@kernel.org/
> > - New
> > Resent of my previous submission
> >
> > https://lore.kernel.org/linux-kbuild/20210831074004.3195284-11-masahiroy@kernel.org/
> >
> > Changes in v3:
> > - New patch
> >
> > Changes in v2:
> > - Simplify the implementation (parse .cmd files after ELF)
> > - New patch
> > - replace the chain of $(if ...) with $(and )
> > - New patch
> > - New patch
> >
> > Masahiro Yamada (10):
> > modpost: extract symbol versions from *.cmd files
> > kbuild: link symbol CRCs at final link, removing
> > CONFIG_MODULE_REL_CRCS
> > kbuild: stop merging *.symversions
> > genksyms: adjust the output format to modpost
> > kbuild: do not create *.prelink.o for Clang LTO or IBT
> > kbuild: check static EXPORT_SYMBOL* by script instead of modpost
> > kbuild: make built-in.a rule robust against too long argument error
> > kbuild: make *.mod rule robust against too long argument error
> > kbuild: add cmd_and_savecmd macro
> > kbuild: rebuild multi-object modules when objtool is updated
> >
> > arch/m68k/include/asm/Kbuild | 1 +
> > arch/m68k/include/asm/export.h | 2 -
> > arch/powerpc/Kconfig | 1 -
> > arch/s390/Kconfig | 1 -
> > arch/um/Kconfig | 1 -
> > include/asm-generic/export.h | 22 ++-
> > include/linux/export-internal.h | 16 +++
> > include/linux/export.h | 30 ++--
> > init/Kconfig | 4 -
> > kernel/module.c | 10 +-
> > scripts/Kbuild.include | 10 +-
> > scripts/Makefile.build | 134 ++++++------------
> > scripts/Makefile.lib | 7 -
> > scripts/Makefile.modfinal | 5 +-
> > scripts/Makefile.modpost | 9 +-
> > scripts/check-local-export | 64 +++++++++
> > scripts/genksyms/genksyms.c | 18 +--
> > scripts/link-vmlinux.sh | 33 ++---
> > scripts/mod/modpost.c | 236 +++++++++++++++++++++-----------
> > 19 files changed, 320 insertions(+), 284 deletions(-)
> > delete mode 100644 arch/m68k/include/asm/export.h
> > create mode 100644 include/linux/export-internal.h
> > create mode 100755 scripts/check-local-export
> >
> > --
> > 2.32.0
> >
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
2022-08-08 13:26 ` matoro
@ 2022-08-08 17:36 ` Masahiro Yamada
2022-08-08 18:42 ` matoro
0 siblings, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2022-08-08 17:36 UTC (permalink / raw)
To: matoro
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
On Mon, Aug 8, 2022 at 10:27 PM matoro
<matoro_mailinglist_kernel@matoro.tk> wrote:
>
> I have real hardware for all these arches in my collection. I use it
> for testing the latest kernel and toolchains on as many of the
> less-popular arches as possible, exactly to find issues like this one :)
>
> Specifically we support all of these in Gentoo. To double-check this
> wasn't a config issue, I asked another user who also runs sparc to try
> building 5.19 with his config (not copying mine), and he observed the
> same problem. You can reach us in #gentoo-sparc on Libera.
>
> As for testing, I make all this hardware available on an as-needed
> basis. So if you can't or don't want to fiddle with qemu, just let me
> know (email or IRC, same username on Libera), and I will get you direct
> access to my hardware. Thanks!!
I found the root cause.
When I build the sparc kernel, I see a warning
WARNING: modpost: EXPORT symbol "_mcount" [vmlinux] version generation
failed, symbol will not be versioned.
Then, modpost missed to write out the entry.
With the following patch, you will be able to load the module.
I will send a patch with a proper commit log tomorrow. I need some sleep now.
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 29474cee10b1..337bd36a890a 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -2206,13 +2206,11 @@ static void add_exported_symbols(struct buffer
*buf, struct module *mod)
/* record CRCs for exported symbols */
buf_printf(buf, "\n");
list_for_each_entry(sym, &mod->exported_symbols, list) {
- if (!sym->crc_valid) {
+ if (!sym->crc_valid)
warn("EXPORT symbol \"%s\" [%s%s] version
generation failed, symbol will not be versioned.\n"
"Is \"%s\" prototyped in
<asm/asm-prototypes.h>?\n",
sym->name, mod->name, mod->is_vmlinux ? "" : ".ko",
sym->name);
- continue;
- }
buf_printf(buf, "SYMBOL_CRC(%s, 0x%08x, \"%s\");\n",
sym->name, sym->crc, sym->is_gpl_only ? "_gpl" : "");
--
Best Regards
Masahiro Yamada
^ permalink raw reply related [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
2022-08-08 17:36 ` Masahiro Yamada
@ 2022-08-08 18:42 ` matoro
2022-08-09 15:20 ` Masahiro Yamada
0 siblings, 1 reply; 39+ messages in thread
From: matoro @ 2022-08-08 18:42 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
That patch doesn't apply to the v5.19 stable tag, so I just manually
edited and it worked perfect! Thank you!!
Sorry I didn't mention that warning - I did see it, but it still showed
up even on the old kernel, so I thought it was irrelevant. Much
appreciated!
-------- Original Message --------
Subject: Re: [PATCH v6 00/10] kbuild: yet another series of cleanups
(modpost, LTO, MODULE_REL_CRCS, export.h)
Date: 2022-08-08 13:36
From: Masahiro Yamada <masahiroy@kernel.org>
To: matoro <matoro_mailinglist_kernel@matoro.tk>
On Mon, Aug 8, 2022 at 10:27 PM matoro
<matoro_mailinglist_kernel@matoro.tk> wrote:
>
> I have real hardware for all these arches in my collection. I use it
> for testing the latest kernel and toolchains on as many of the
> less-popular arches as possible, exactly to find issues like this one
> :)
>
> Specifically we support all of these in Gentoo. To double-check this
> wasn't a config issue, I asked another user who also runs sparc to try
> building 5.19 with his config (not copying mine), and he observed the
> same problem. You can reach us in #gentoo-sparc on Libera.
>
> As for testing, I make all this hardware available on an as-needed
> basis. So if you can't or don't want to fiddle with qemu, just let me
> know (email or IRC, same username on Libera), and I will get you direct
> access to my hardware. Thanks!!
I found the root cause.
When I build the sparc kernel, I see a warning
WARNING: modpost: EXPORT symbol "_mcount" [vmlinux] version generation
failed, symbol will not be versioned.
Then, modpost missed to write out the entry.
With the following patch, you will be able to load the module.
I will send a patch with a proper commit log tomorrow. I need some sleep
now.
diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 29474cee10b1..337bd36a890a 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -2206,13 +2206,11 @@ static void add_exported_symbols(struct buffer
*buf, struct module *mod)
/* record CRCs for exported symbols */
buf_printf(buf, "\n");
list_for_each_entry(sym, &mod->exported_symbols, list) {
- if (!sym->crc_valid) {
+ if (!sym->crc_valid)
warn("EXPORT symbol \"%s\" [%s%s] version
generation failed, symbol will not be versioned.\n"
"Is \"%s\" prototyped in
<asm/asm-prototypes.h>?\n",
sym->name, mod->name, mod->is_vmlinux ? ""
: ".ko",
sym->name);
- continue;
- }
buf_printf(buf, "SYMBOL_CRC(%s, 0x%08x, \"%s\");\n",
sym->name, sym->crc, sym->is_gpl_only ?
"_gpl" : "");
^ permalink raw reply related [flat|nested] 39+ messages in thread
* Re: [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h)
2022-08-08 18:42 ` matoro
@ 2022-08-09 15:20 ` Masahiro Yamada
0 siblings, 0 replies; 39+ messages in thread
From: Masahiro Yamada @ 2022-08-09 15:20 UTC (permalink / raw)
To: matoro
Cc: Linux Kbuild mailing list, Linux Kernel Mailing List,
Nathan Chancellor, Nick Desaulniers, Nicolas Schier,
Peter Zijlstra, linux-modules, clang-built-linux, Ard Biesheuvel,
Sami Tolvanen
On Tue, Aug 9, 2022 at 3:42 AM matoro
<matoro_mailinglist_kernel@matoro.tk> wrote:
>
> That patch doesn't apply to the v5.19 stable tag, so I just manually
> edited and it worked perfect! Thank you!!
>
> Sorry I didn't mention that warning - I did see it, but it still showed
> up even on the old kernel, so I thought it was irrelevant. Much
> appreciated!
>
> -------- Original Message --------
> Subject: Re: [PATCH v6 00/10] kbuild: yet another series of cleanups
> (modpost, LTO, MODULE_REL_CRCS, export.h)
> Date: 2022-08-08 13:36
> From: Masahiro Yamada <masahiroy@kernel.org>
> To: matoro <matoro_mailinglist_kernel@matoro.tk>
>
> On Mon, Aug 8, 2022 at 10:27 PM matoro
> <matoro_mailinglist_kernel@matoro.tk> wrote:
> >
> > I have real hardware for all these arches in my collection. I use it
> > for testing the latest kernel and toolchains on as many of the
> > less-popular arches as possible, exactly to find issues like this one
> > :)
> >
> > Specifically we support all of these in Gentoo. To double-check this
> > wasn't a config issue, I asked another user who also runs sparc to try
> > building 5.19 with his config (not copying mine), and he observed the
> > same problem. You can reach us in #gentoo-sparc on Libera.
> >
> > As for testing, I make all this hardware available on an as-needed
> > basis. So if you can't or don't want to fiddle with qemu, just let me
> > know (email or IRC, same username on Libera), and I will get you direct
> > access to my hardware. Thanks!!
>
>
> I found the root cause.
>
> When I build the sparc kernel, I see a warning
>
> WARNING: modpost: EXPORT symbol "_mcount" [vmlinux] version generation
> failed, symbol will not be versioned.
>
>
> Then, modpost missed to write out the entry.
>
> With the following patch, you will be able to load the module.
>
> I will send a patch with a proper commit log tomorrow. I need some sleep
> now.
I posted a patch:
https://patchwork.kernel.org/project/linux-kbuild/patch/20220809141117.641543-1-masahiroy@kernel.org/
If you want me to record your full name in Reported/Tested-by,
please let me know.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Build/boot problem with 7b4537199a4a (Re: [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS)
2022-05-13 11:39 ` [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS Masahiro Yamada
@ 2022-08-20 10:02 ` Christophe Leroy
2022-08-20 12:05 ` Sedat Dilek
2022-08-20 12:51 ` Masahiro Yamada
0 siblings, 2 replies; 39+ messages in thread
From: Christophe Leroy @ 2022-08-20 10:02 UTC (permalink / raw)
To: Masahiro Yamada, linux-kbuild
Cc: linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, linuxppc-dev, Kirill A . Shutemov
Hi,
Le 13/05/2022 à 13:39, Masahiro Yamada a écrit :
> include/{linux,asm-generic}/export.h defines a weak symbol, __crc_*
> as a placeholder.
>
> Genksyms writes the version CRCs into the linker script, which will be
> used for filling the __crc_* symbols. The linker script format depends
> on CONFIG_MODULE_REL_CRCS. If it is enabled, __crc_* holds the offset
> to the reference of CRC.
>
> It is time to get rid of this complexity.
>
> Now that modpost parses text files (.*.cmd) to collect all the CRCs,
> it can generate C code that will be linked to the vmlinux or modules.
>
> Generate a new C file, .vmlinux.export.c, which contains the CRCs of
> symbols exported by vmlinux. It is compiled and linked to vmlinux in
> scripts/link-vmlinux.sh.
>
> Put the CRCs of symbols exported by modules into the existing *.mod.c
> files. No additional build step is needed for modules. As before,
> *.mod.c are compiled and linked to *.ko in scripts/Makefile.modfinal.
>
> No linker magic is used here. The new C implementation works in the
> same way, whether CONFIG_RELOCATABLE is enabled or not.
> CONFIG_MODULE_REL_CRCS is no longer needed.
>
> Previously, Kbuild invoked additional $(LD) to update the CRCs in
> objects, but this step is unneeded too.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> Tested-by: Nathan Chancellor <nathan@kernel.org>
> Tested-by: Nicolas Schier <nicolas@fjasle.eu>
> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Problem with v6.0-rc1
Problem with v5.19
No problem with v5.18
Bisected to 7b4537199a4a ("kbuild: link symbol CRCs at final link,
removing CONFIG_MODULE_REL_CRCS")
The above patch leads to the following problem building
mpc85xx_defconfig + CONFIG_RELOCATABLE
LD vmlinux
SYSMAP System.map
SORTTAB vmlinux
CHKREL vmlinux
WARNING: 451 bad relocations
c0b0f26d R_PPC_UADDR32 .head.text-0x3ff9f2bc
c0b0f271 R_PPC_UADDR32 .head.text-0x3ffac300
c0b0f275 R_PPC_UADDR32 .head.text-0x3ffb0bdc
c0b0f279 R_PPC_UADDR32 .head.text-0x3fe1e080
c0b0f27d R_PPC_UADDR32 .head.text-0x3fe1df4c
c0b0f281 R_PPC_UADDR32 .head.text-0x3fe21514
c0b0f285 R_PPC_UADDR32 .head.text-0x3fe211c0
c0b0f289 R_PPC_UADDR32 .head.text-0x3ffabda0
c0b0f28d R_PPC_UADDR32 .head.text-0x3fe21258
c0b0f291 R_PPC_UADDR32 .head.text-0x3fe074d0
c0b0f295 R_PPC_UADDR32 .head.text-0x3fe07ad4
c0b0f299 R_PPC_UADDR32 .head.text-0x3fe13470
c0b0f29d R_PPC_UADDR32 .head.text-0x3fe22700
c0b0f2a1 R_PPC_UADDR32 .head.text-0x3ff4b8e0
c0b0f2a5 R_PPC_UADDR32 .head.text-0x3fe08320
c0b0f2a9 R_PPC_UADDR32 .head.text-0x3fe220dc
c0b0f2ad R_PPC_UADDR32 .head.text-0x3fe21da0
c0b0f2b1 R_PPC_UADDR32 .head.text-0x3ff89dc0
c0b0f2b5 R_PPC_UADDR32 .head.text-0x3fe16524
c0b0f2b9 R_PPC_UADDR32 .head.text-0x3fe1ef74
c0b0f2bd R_PPC_UADDR32 .head.text-0x3ff98b84
c0b0f2c1 R_PPC_UADDR32 .head.text-0x3fdef9a0
c0b0f2c5 R_PPC_UADDR32 .head.text-0x3fdf21ac
c0b0f2c9 R_PPC_UADDR32 .head.text-0x3ff993c4
...
c0b0f969 R_PPC_UADDR32 .head.text-0x3ff89dc0
c0b0f96d R_PPC_UADDR32 .head.text-0x3fe9ad40
c0b0f971 R_PPC_UADDR32 .head.text-0x3ff2eb00
c0b0f975 R_PPC_UADDR32 .head.text-0x3ff89dc0
And boot fails:
Run /init as init process
kernel tried to execute user page (0) - exploit attempt? (uid: 0)
BUG: Unable to handle kernel instruction fetch (NULL pointer?)
Faulting instruction address: 0x00000000
Oops: Kernel access of bad area, sig: 11 [#1]
BE PAGE_SIZE=4K MPC8544 DS
Modules linked in:
CPU: 0 PID: 1 Comm: init Not tainted 5.18.0-rc1-00054-g7b4537199a4a #1523
NIP: 00000000 LR: c00150e4 CTR: 00000000
REGS: c3091e10 TRAP: 0400 Not tainted (5.18.0-rc1-00054-g7b4537199a4a)
MSR: 00009000 <EE,ME> CR: 88000422 XER: 20000000
GPR00: 00004000 c3091f00 c30c8000 00000000 00000013 b7bb9f4c b7bd8f60
bfee6650
GPR08: 00000054 00000000 c0b0f26d 00000000 c13b0000 00000000 bfee6668
00000000
GPR16: 84e08000 00000000 08000000 00000064 00000000 00102000 00000001
00000001
GPR24: 00000001 00000001 b7b9c7d0 10000034 00000009 b7bd8f38 b7bd9854
b7bd8688
NIP [00000000] 0x0
LR [c00150e4] ret_from_syscall+0x0/0x28
Call Trace:
[c3091f00] [c0000af0] InstructionStorage+0x150/0x160 (unreliable)
--- interrupt: c00 at 0xb7bb28e8
NIP: b7bb28e8 LR: b7bb1384 CTR: b7bb1218
REGS: c3091f10 TRAP: 0c00 Not tainted (5.18.0-rc1-00054-g7b4537199a4a)
MSR: 0002d000 <CE,EE,PR,ME> CR: 28000422 XER: 20000000
GPR00: 0000002d bfee61f0 00000000 00000000 00000013 b7bb9f4c b7bd8f60
bfee6650
GPR08: 00000054 00000020 bfee6648 00000000 00000001 00000000 bfee6668
00000000
GPR16: 84e08000 00000000 08000000 00000064 00000000 00102000 00000001
00000001
GPR24: 00000001 00000001 b7b9c7d0 10000034 00000009 b7bd8f38 b7bd9854
b7bd8688
NIP [b7bb28e8] 0xb7bb28e8
LR [b7bb1384] 0xb7bb1384
--- interrupt: c00
Instruction dump:
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
---[ end trace 0000000000000000 ]---
Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
Christophe
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Build/boot problem with 7b4537199a4a (Re: [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS)
2022-08-20 10:02 ` Build/boot problem with 7b4537199a4a (Re: [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS) Christophe Leroy
@ 2022-08-20 12:05 ` Sedat Dilek
2022-08-20 14:10 ` Christophe Leroy
2022-08-20 12:51 ` Masahiro Yamada
1 sibling, 1 reply; 39+ messages in thread
From: Sedat Dilek @ 2022-08-20 12:05 UTC (permalink / raw)
To: Christophe Leroy
Cc: Masahiro Yamada, linux-kbuild, linux-kernel, Nathan Chancellor,
Nick Desaulniers, Nicolas Schier, Peter Zijlstra, linux-modules,
llvm, Ard Biesheuvel, Sami Tolvanen, linuxppc-dev,
Kirill A . Shutemov
On Sat, Aug 20, 2022 at 12:04 PM Christophe Leroy
<christophe.leroy@csgroup.eu> wrote:
>
> Hi,
>
> Le 13/05/2022 à 13:39, Masahiro Yamada a écrit :
> > include/{linux,asm-generic}/export.h defines a weak symbol, __crc_*
> > as a placeholder.
> >
> > Genksyms writes the version CRCs into the linker script, which will be
> > used for filling the __crc_* symbols. The linker script format depends
> > on CONFIG_MODULE_REL_CRCS. If it is enabled, __crc_* holds the offset
> > to the reference of CRC.
> >
> > It is time to get rid of this complexity.
> >
> > Now that modpost parses text files (.*.cmd) to collect all the CRCs,
> > it can generate C code that will be linked to the vmlinux or modules.
> >
> > Generate a new C file, .vmlinux.export.c, which contains the CRCs of
> > symbols exported by vmlinux. It is compiled and linked to vmlinux in
> > scripts/link-vmlinux.sh.
> >
> > Put the CRCs of symbols exported by modules into the existing *.mod.c
> > files. No additional build step is needed for modules. As before,
> > *.mod.c are compiled and linked to *.ko in scripts/Makefile.modfinal.
> >
> > No linker magic is used here. The new C implementation works in the
> > same way, whether CONFIG_RELOCATABLE is enabled or not.
> > CONFIG_MODULE_REL_CRCS is no longer needed.
> >
> > Previously, Kbuild invoked additional $(LD) to update the CRCs in
> > objects, but this step is unneeded too.
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > Tested-by: Nathan Chancellor <nathan@kernel.org>
> > Tested-by: Nicolas Schier <nicolas@fjasle.eu>
> > Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
>
> Problem with v6.0-rc1
> Problem with v5.19
> No problem with v5.18
>
> Bisected to 7b4537199a4a ("kbuild: link symbol CRCs at final link,
> removing CONFIG_MODULE_REL_CRCS")
>
What you are looking for is...
commit 7d13fd96df875a9d786ee6dcc8fec460d35d4b12
("modpost: fix module versioning when a symbol lacks valid CRC")
It's pending in kbuild.git#fixes.
-Sedat-
[1] https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=fixes&id=7d13fd96df875a9d786ee6dcc8fec460d35d4b12
[2] https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/log/?h=fixes
> The above patch leads to the following problem building
> mpc85xx_defconfig + CONFIG_RELOCATABLE
>
> LD vmlinux
> SYSMAP System.map
> SORTTAB vmlinux
> CHKREL vmlinux
> WARNING: 451 bad relocations
> c0b0f26d R_PPC_UADDR32 .head.text-0x3ff9f2bc
> c0b0f271 R_PPC_UADDR32 .head.text-0x3ffac300
> c0b0f275 R_PPC_UADDR32 .head.text-0x3ffb0bdc
> c0b0f279 R_PPC_UADDR32 .head.text-0x3fe1e080
> c0b0f27d R_PPC_UADDR32 .head.text-0x3fe1df4c
> c0b0f281 R_PPC_UADDR32 .head.text-0x3fe21514
> c0b0f285 R_PPC_UADDR32 .head.text-0x3fe211c0
> c0b0f289 R_PPC_UADDR32 .head.text-0x3ffabda0
> c0b0f28d R_PPC_UADDR32 .head.text-0x3fe21258
> c0b0f291 R_PPC_UADDR32 .head.text-0x3fe074d0
> c0b0f295 R_PPC_UADDR32 .head.text-0x3fe07ad4
> c0b0f299 R_PPC_UADDR32 .head.text-0x3fe13470
> c0b0f29d R_PPC_UADDR32 .head.text-0x3fe22700
> c0b0f2a1 R_PPC_UADDR32 .head.text-0x3ff4b8e0
> c0b0f2a5 R_PPC_UADDR32 .head.text-0x3fe08320
> c0b0f2a9 R_PPC_UADDR32 .head.text-0x3fe220dc
> c0b0f2ad R_PPC_UADDR32 .head.text-0x3fe21da0
> c0b0f2b1 R_PPC_UADDR32 .head.text-0x3ff89dc0
> c0b0f2b5 R_PPC_UADDR32 .head.text-0x3fe16524
> c0b0f2b9 R_PPC_UADDR32 .head.text-0x3fe1ef74
> c0b0f2bd R_PPC_UADDR32 .head.text-0x3ff98b84
> c0b0f2c1 R_PPC_UADDR32 .head.text-0x3fdef9a0
> c0b0f2c5 R_PPC_UADDR32 .head.text-0x3fdf21ac
> c0b0f2c9 R_PPC_UADDR32 .head.text-0x3ff993c4
> ...
> c0b0f969 R_PPC_UADDR32 .head.text-0x3ff89dc0
> c0b0f96d R_PPC_UADDR32 .head.text-0x3fe9ad40
> c0b0f971 R_PPC_UADDR32 .head.text-0x3ff2eb00
> c0b0f975 R_PPC_UADDR32 .head.text-0x3ff89dc0
>
> And boot fails:
>
> Run /init as init process
> kernel tried to execute user page (0) - exploit attempt? (uid: 0)
> BUG: Unable to handle kernel instruction fetch (NULL pointer?)
> Faulting instruction address: 0x00000000
> Oops: Kernel access of bad area, sig: 11 [#1]
> BE PAGE_SIZE=4K MPC8544 DS
> Modules linked in:
> CPU: 0 PID: 1 Comm: init Not tainted 5.18.0-rc1-00054-g7b4537199a4a #1523
> NIP: 00000000 LR: c00150e4 CTR: 00000000
> REGS: c3091e10 TRAP: 0400 Not tainted (5.18.0-rc1-00054-g7b4537199a4a)
> MSR: 00009000 <EE,ME> CR: 88000422 XER: 20000000
>
> GPR00: 00004000 c3091f00 c30c8000 00000000 00000013 b7bb9f4c b7bd8f60
> bfee6650
> GPR08: 00000054 00000000 c0b0f26d 00000000 c13b0000 00000000 bfee6668
> 00000000
> GPR16: 84e08000 00000000 08000000 00000064 00000000 00102000 00000001
> 00000001
> GPR24: 00000001 00000001 b7b9c7d0 10000034 00000009 b7bd8f38 b7bd9854
> b7bd8688
> NIP [00000000] 0x0
> LR [c00150e4] ret_from_syscall+0x0/0x28
> Call Trace:
> [c3091f00] [c0000af0] InstructionStorage+0x150/0x160 (unreliable)
> --- interrupt: c00 at 0xb7bb28e8
> NIP: b7bb28e8 LR: b7bb1384 CTR: b7bb1218
> REGS: c3091f10 TRAP: 0c00 Not tainted (5.18.0-rc1-00054-g7b4537199a4a)
> MSR: 0002d000 <CE,EE,PR,ME> CR: 28000422 XER: 20000000
>
> GPR00: 0000002d bfee61f0 00000000 00000000 00000013 b7bb9f4c b7bd8f60
> bfee6650
> GPR08: 00000054 00000020 bfee6648 00000000 00000001 00000000 bfee6668
> 00000000
> GPR16: 84e08000 00000000 08000000 00000064 00000000 00102000 00000001
> 00000001
> GPR24: 00000001 00000001 b7b9c7d0 10000034 00000009 b7bd8f38 b7bd9854
> b7bd8688
> NIP [b7bb28e8] 0xb7bb28e8
> LR [b7bb1384] 0xb7bb1384
> --- interrupt: c00
> Instruction dump:
> XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
> XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
> ---[ end trace 0000000000000000 ]---
>
> Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
>
>
>
> Christophe
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Build/boot problem with 7b4537199a4a (Re: [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS)
2022-08-20 10:02 ` Build/boot problem with 7b4537199a4a (Re: [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS) Christophe Leroy
2022-08-20 12:05 ` Sedat Dilek
@ 2022-08-20 12:51 ` Masahiro Yamada
2022-08-20 14:15 ` Christophe Leroy
1 sibling, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2022-08-20 12:51 UTC (permalink / raw)
To: Christophe Leroy
Cc: linux-kbuild, linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, linuxppc-dev, Kirill A . Shutemov
On Sat, Aug 20, 2022 at 7:02 PM Christophe Leroy
<christophe.leroy@csgroup.eu> wrote:
>
> Hi,
>
> Le 13/05/2022 à 13:39, Masahiro Yamada a écrit :
> > include/{linux,asm-generic}/export.h defines a weak symbol, __crc_*
> > as a placeholder.
> >
> > Genksyms writes the version CRCs into the linker script, which will be
> > used for filling the __crc_* symbols. The linker script format depends
> > on CONFIG_MODULE_REL_CRCS. If it is enabled, __crc_* holds the offset
> > to the reference of CRC.
> >
> > It is time to get rid of this complexity.
> >
> > Now that modpost parses text files (.*.cmd) to collect all the CRCs,
> > it can generate C code that will be linked to the vmlinux or modules.
> >
> > Generate a new C file, .vmlinux.export.c, which contains the CRCs of
> > symbols exported by vmlinux. It is compiled and linked to vmlinux in
> > scripts/link-vmlinux.sh.
> >
> > Put the CRCs of symbols exported by modules into the existing *.mod.c
> > files. No additional build step is needed for modules. As before,
> > *.mod.c are compiled and linked to *.ko in scripts/Makefile.modfinal.
> >
> > No linker magic is used here. The new C implementation works in the
> > same way, whether CONFIG_RELOCATABLE is enabled or not.
> > CONFIG_MODULE_REL_CRCS is no longer needed.
> >
> > Previously, Kbuild invoked additional $(LD) to update the CRCs in
> > objects, but this step is unneeded too.
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > Tested-by: Nathan Chancellor <nathan@kernel.org>
> > Tested-by: Nicolas Schier <nicolas@fjasle.eu>
> > Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
>
> Problem with v6.0-rc1
> Problem with v5.19
> No problem with v5.18
>
> Bisected to 7b4537199a4a ("kbuild: link symbol CRCs at final link,
> removing CONFIG_MODULE_REL_CRCS")
>
> The above patch leads to the following problem building
> mpc85xx_defconfig + CONFIG_RELOCATABLE
Is this because the relocation implementation on ppc is incomplete?
(and is it the reason why relock_check.sh exists?)
arch/powerpc/kernel/reloc_32.S does not support R_PPC_UADDR32
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Build/boot problem with 7b4537199a4a (Re: [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS)
2022-08-20 12:05 ` Sedat Dilek
@ 2022-08-20 14:10 ` Christophe Leroy
0 siblings, 0 replies; 39+ messages in thread
From: Christophe Leroy @ 2022-08-20 14:10 UTC (permalink / raw)
To: sedat.dilek
Cc: Masahiro Yamada, linux-kbuild, linux-kernel, Nathan Chancellor,
Nick Desaulniers, Nicolas Schier, Peter Zijlstra, linux-modules,
llvm, Ard Biesheuvel, Sami Tolvanen, linuxppc-dev,
Kirill A . Shutemov
Le 20/08/2022 à 14:05, Sedat Dilek a écrit :
> On Sat, Aug 20, 2022 at 12:04 PM Christophe Leroy
> <christophe.leroy@csgroup.eu> wrote:
>>
>> Hi,
>>
>> Le 13/05/2022 à 13:39, Masahiro Yamada a écrit :
>>> include/{linux,asm-generic}/export.h defines a weak symbol, __crc_*
>>> as a placeholder.
>>>
>>> Genksyms writes the version CRCs into the linker script, which will be
>>> used for filling the __crc_* symbols. The linker script format depends
>>> on CONFIG_MODULE_REL_CRCS. If it is enabled, __crc_* holds the offset
>>> to the reference of CRC.
>>>
>>> It is time to get rid of this complexity.
>>>
>>> Now that modpost parses text files (.*.cmd) to collect all the CRCs,
>>> it can generate C code that will be linked to the vmlinux or modules.
>>>
>>> Generate a new C file, .vmlinux.export.c, which contains the CRCs of
>>> symbols exported by vmlinux. It is compiled and linked to vmlinux in
>>> scripts/link-vmlinux.sh.
>>>
>>> Put the CRCs of symbols exported by modules into the existing *.mod.c
>>> files. No additional build step is needed for modules. As before,
>>> *.mod.c are compiled and linked to *.ko in scripts/Makefile.modfinal.
>>>
>>> No linker magic is used here. The new C implementation works in the
>>> same way, whether CONFIG_RELOCATABLE is enabled or not.
>>> CONFIG_MODULE_REL_CRCS is no longer needed.
>>>
>>> Previously, Kbuild invoked additional $(LD) to update the CRCs in
>>> objects, but this step is unneeded too.
>>>
>>> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
>>> Tested-by: Nathan Chancellor <nathan@kernel.org>
>>> Tested-by: Nicolas Schier <nicolas@fjasle.eu>
>>> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
>>
>> Problem with v6.0-rc1
>> Problem with v5.19
>> No problem with v5.18
>>
>> Bisected to 7b4537199a4a ("kbuild: link symbol CRCs at final link,
>> removing CONFIG_MODULE_REL_CRCS")
>>
>
> What you are looking for is...
>
> commit 7d13fd96df875a9d786ee6dcc8fec460d35d4b12
> ("modpost: fix module versioning when a symbol lacks valid CRC")
>
> It's pending in kbuild.git#fixes.
>
> -Sedat-
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=fixes&id=7d13fd96df875a9d786ee6dcc8fec460d35d4b12
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/log/?h=fixes
>
That patch doesn't fix the problem.
Christophe
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Build/boot problem with 7b4537199a4a (Re: [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS)
2022-08-20 12:51 ` Masahiro Yamada
@ 2022-08-20 14:15 ` Christophe Leroy
2022-08-20 17:01 ` Masahiro Yamada
0 siblings, 1 reply; 39+ messages in thread
From: Christophe Leroy @ 2022-08-20 14:15 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, linuxppc-dev, Kirill A . Shutemov
Le 20/08/2022 à 14:51, Masahiro Yamada a écrit :
> On Sat, Aug 20, 2022 at 7:02 PM Christophe Leroy
> <christophe.leroy@csgroup.eu> wrote:
>>
>> Hi,
>>
>> Le 13/05/2022 à 13:39, Masahiro Yamada a écrit :
>>> include/{linux,asm-generic}/export.h defines a weak symbol, __crc_*
>>> as a placeholder.
>>>
>>> Genksyms writes the version CRCs into the linker script, which will be
>>> used for filling the __crc_* symbols. The linker script format depends
>>> on CONFIG_MODULE_REL_CRCS. If it is enabled, __crc_* holds the offset
>>> to the reference of CRC.
>>>
>>> It is time to get rid of this complexity.
>>>
>>> Now that modpost parses text files (.*.cmd) to collect all the CRCs,
>>> it can generate C code that will be linked to the vmlinux or modules.
>>>
>>> Generate a new C file, .vmlinux.export.c, which contains the CRCs of
>>> symbols exported by vmlinux. It is compiled and linked to vmlinux in
>>> scripts/link-vmlinux.sh.
>>>
>>> Put the CRCs of symbols exported by modules into the existing *.mod.c
>>> files. No additional build step is needed for modules. As before,
>>> *.mod.c are compiled and linked to *.ko in scripts/Makefile.modfinal.
>>>
>>> No linker magic is used here. The new C implementation works in the
>>> same way, whether CONFIG_RELOCATABLE is enabled or not.
>>> CONFIG_MODULE_REL_CRCS is no longer needed.
>>>
>>> Previously, Kbuild invoked additional $(LD) to update the CRCs in
>>> objects, but this step is unneeded too.
>>>
>>> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
>>> Tested-by: Nathan Chancellor <nathan@kernel.org>
>>> Tested-by: Nicolas Schier <nicolas@fjasle.eu>
>>> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
>>
>> Problem with v6.0-rc1
>> Problem with v5.19
>> No problem with v5.18
>>
>> Bisected to 7b4537199a4a ("kbuild: link symbol CRCs at final link,
>> removing CONFIG_MODULE_REL_CRCS")
>>
>> The above patch leads to the following problem building
>> mpc85xx_defconfig + CONFIG_RELOCATABLE
>
>
>
> Is this because the relocation implementation on ppc is incomplete?
> (and is it the reason why relock_check.sh exists?)
>
> arch/powerpc/kernel/reloc_32.S does not support R_PPC_UADDR32
>
>
Might be the reason.
Is it expected that your patch adds an unsupported relocation ?
Why was that relocation type unneeded before ?
Thanks
Christophe
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Build/boot problem with 7b4537199a4a (Re: [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS)
2022-08-20 14:15 ` Christophe Leroy
@ 2022-08-20 17:01 ` Masahiro Yamada
2022-08-20 17:51 ` Christophe Leroy
0 siblings, 1 reply; 39+ messages in thread
From: Masahiro Yamada @ 2022-08-20 17:01 UTC (permalink / raw)
To: Christophe Leroy
Cc: linux-kbuild, linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, linuxppc-dev, Kirill A . Shutemov
On Sat, Aug 20, 2022 at 11:15 PM Christophe Leroy
<christophe.leroy@csgroup.eu> wrote:
>
>
>
> Le 20/08/2022 à 14:51, Masahiro Yamada a écrit :
> > On Sat, Aug 20, 2022 at 7:02 PM Christophe Leroy
> > <christophe.leroy@csgroup.eu> wrote:
> >>
> >> Hi,
> >>
> >> Le 13/05/2022 à 13:39, Masahiro Yamada a écrit :
> >>> include/{linux,asm-generic}/export.h defines a weak symbol, __crc_*
> >>> as a placeholder.
> >>>
> >>> Genksyms writes the version CRCs into the linker script, which will be
> >>> used for filling the __crc_* symbols. The linker script format depends
> >>> on CONFIG_MODULE_REL_CRCS. If it is enabled, __crc_* holds the offset
> >>> to the reference of CRC.
> >>>
> >>> It is time to get rid of this complexity.
> >>>
> >>> Now that modpost parses text files (.*.cmd) to collect all the CRCs,
> >>> it can generate C code that will be linked to the vmlinux or modules.
> >>>
> >>> Generate a new C file, .vmlinux.export.c, which contains the CRCs of
> >>> symbols exported by vmlinux. It is compiled and linked to vmlinux in
> >>> scripts/link-vmlinux.sh.
> >>>
> >>> Put the CRCs of symbols exported by modules into the existing *.mod.c
> >>> files. No additional build step is needed for modules. As before,
> >>> *.mod.c are compiled and linked to *.ko in scripts/Makefile.modfinal.
> >>>
> >>> No linker magic is used here. The new C implementation works in the
> >>> same way, whether CONFIG_RELOCATABLE is enabled or not.
> >>> CONFIG_MODULE_REL_CRCS is no longer needed.
> >>>
> >>> Previously, Kbuild invoked additional $(LD) to update the CRCs in
> >>> objects, but this step is unneeded too.
> >>>
> >>> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> >>> Tested-by: Nathan Chancellor <nathan@kernel.org>
> >>> Tested-by: Nicolas Schier <nicolas@fjasle.eu>
> >>> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
> >>
> >> Problem with v6.0-rc1
> >> Problem with v5.19
> >> No problem with v5.18
> >>
> >> Bisected to 7b4537199a4a ("kbuild: link symbol CRCs at final link,
> >> removing CONFIG_MODULE_REL_CRCS")
> >>
> >> The above patch leads to the following problem building
> >> mpc85xx_defconfig + CONFIG_RELOCATABLE
> >
> >
> >
> > Is this because the relocation implementation on ppc is incomplete?
> > (and is it the reason why relock_check.sh exists?)
> >
> > arch/powerpc/kernel/reloc_32.S does not support R_PPC_UADDR32
> >
> >
>
> Might be the reason.
>
> Is it expected that your patch adds an unsupported relocation ?
>
> Why was that relocation type unneeded before ?
>
> Thanks
> Christophe
I posted a patch (although I believe my commit is innocent).
https://lore.kernel.org/lkml/20220820165129.1147589-1-masahiroy@kernel.org/T/#u
The relocs_check.sh warnings are gone.
Please do a boot test.
Thanks.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Build/boot problem with 7b4537199a4a (Re: [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS)
2022-08-20 17:01 ` Masahiro Yamada
@ 2022-08-20 17:51 ` Christophe Leroy
0 siblings, 0 replies; 39+ messages in thread
From: Christophe Leroy @ 2022-08-20 17:51 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, linux-kernel, Nathan Chancellor, Nick Desaulniers,
Nicolas Schier, Peter Zijlstra, linux-modules, llvm,
Ard Biesheuvel, Sami Tolvanen, linuxppc-dev, Kirill A . Shutemov
Le 20/08/2022 à 19:01, Masahiro Yamada a écrit :
> On Sat, Aug 20, 2022 at 11:15 PM Christophe Leroy
> <christophe.leroy@csgroup.eu> wrote:
>>
>>
>>
>> Le 20/08/2022 à 14:51, Masahiro Yamada a écrit :
>>> On Sat, Aug 20, 2022 at 7:02 PM Christophe Leroy
>>> <christophe.leroy@csgroup.eu> wrote:
>>>>
>>>> Hi,
>>>>
>>>> Le 13/05/2022 à 13:39, Masahiro Yamada a écrit :
>>>>> include/{linux,asm-generic}/export.h defines a weak symbol, __crc_*
>>>>> as a placeholder.
>>>>>
>>>>> Genksyms writes the version CRCs into the linker script, which will be
>>>>> used for filling the __crc_* symbols. The linker script format depends
>>>>> on CONFIG_MODULE_REL_CRCS. If it is enabled, __crc_* holds the offset
>>>>> to the reference of CRC.
>>>>>
>>>>> It is time to get rid of this complexity.
>>>>>
>>>>> Now that modpost parses text files (.*.cmd) to collect all the CRCs,
>>>>> it can generate C code that will be linked to the vmlinux or modules.
>>>>>
>>>>> Generate a new C file, .vmlinux.export.c, which contains the CRCs of
>>>>> symbols exported by vmlinux. It is compiled and linked to vmlinux in
>>>>> scripts/link-vmlinux.sh.
>>>>>
>>>>> Put the CRCs of symbols exported by modules into the existing *.mod.c
>>>>> files. No additional build step is needed for modules. As before,
>>>>> *.mod.c are compiled and linked to *.ko in scripts/Makefile.modfinal.
>>>>>
>>>>> No linker magic is used here. The new C implementation works in the
>>>>> same way, whether CONFIG_RELOCATABLE is enabled or not.
>>>>> CONFIG_MODULE_REL_CRCS is no longer needed.
>>>>>
>>>>> Previously, Kbuild invoked additional $(LD) to update the CRCs in
>>>>> objects, but this step is unneeded too.
>>>>>
>>>>> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
>>>>> Tested-by: Nathan Chancellor <nathan@kernel.org>
>>>>> Tested-by: Nicolas Schier <nicolas@fjasle.eu>
>>>>> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
>>>>
>>>> Problem with v6.0-rc1
>>>> Problem with v5.19
>>>> No problem with v5.18
>>>>
>>>> Bisected to 7b4537199a4a ("kbuild: link symbol CRCs at final link,
>>>> removing CONFIG_MODULE_REL_CRCS")
>>>>
>>>> The above patch leads to the following problem building
>>>> mpc85xx_defconfig + CONFIG_RELOCATABLE
>>>
>>>
>>>
>>> Is this because the relocation implementation on ppc is incomplete?
>>> (and is it the reason why relock_check.sh exists?)
>>>
>>> arch/powerpc/kernel/reloc_32.S does not support R_PPC_UADDR32
>>>
>>>
>>
>> Might be the reason.
>>
>> Is it expected that your patch adds an unsupported relocation ?
>>
>> Why was that relocation type unneeded before ?
>>
>> Thanks
>> Christophe
>
>
> I posted a patch (although I believe my commit is innocent).
>
> https://lore.kernel.org/lkml/20220820165129.1147589-1-masahiroy@kernel.org/T/#u
>
> The relocs_check.sh warnings are gone.
> Please do a boot test.
> Thanks.
>
Yes it works, many Thanks.
The fixes tag should probably be c857c43b34ec ("powerpc: Don't use a
function descriptor for system call table")
Christophe
^ permalink raw reply [flat|nested] 39+ messages in thread
end of thread, other threads:[~2022-08-20 17:51 UTC | newest]
Thread overview: 39+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-13 11:39 [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 01/10] modpost: extract symbol versions from *.cmd files Masahiro Yamada
2022-05-28 22:47 ` Guenter Roeck
2022-05-29 4:27 ` Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS Masahiro Yamada
2022-08-20 10:02 ` Build/boot problem with 7b4537199a4a (Re: [PATCH v6 02/10] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS) Christophe Leroy
2022-08-20 12:05 ` Sedat Dilek
2022-08-20 14:10 ` Christophe Leroy
2022-08-20 12:51 ` Masahiro Yamada
2022-08-20 14:15 ` Christophe Leroy
2022-08-20 17:01 ` Masahiro Yamada
2022-08-20 17:51 ` Christophe Leroy
2022-05-13 11:39 ` [PATCH v6 03/10] kbuild: stop merging *.symversions Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 04/10] genksyms: adjust the output format to modpost Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 05/10] kbuild: do not create *.prelink.o for Clang LTO or IBT Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 06/10] kbuild: check static EXPORT_SYMBOL* by script instead of modpost Masahiro Yamada
2022-05-25 8:31 ` Guenter Roeck
2022-05-25 10:51 ` Masahiro Yamada
2022-06-07 14:22 ` Jon Hunter
2022-06-07 14:25 ` Nathan Chancellor
2022-06-07 14:34 ` Jon Hunter
2022-05-13 11:39 ` [PATCH v6 07/10] kbuild: make built-in.a rule robust against too long argument error Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 08/10] kbuild: make *.mod " Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 09/10] kbuild: add cmd_and_savecmd macro Masahiro Yamada
2022-05-13 11:39 ` [PATCH v6 10/10] kbuild: rebuild multi-object modules when objtool is updated Masahiro Yamada
2022-05-13 12:20 ` [PATCH v6 00/10] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada
[not found] ` <CA+icZUUWww3fXvjQcefgFuq=tPO6+FYDbHE2E5PmL-BSJg4+cw@mail.gmail.com>
2022-05-22 6:49 ` Masahiro Yamada
2022-05-22 9:45 ` Sedat Dilek
2022-05-22 14:15 ` Sedat Dilek
2022-05-22 16:09 ` Masahiro Yamada
2022-05-22 16:18 ` Sedat Dilek
2022-05-22 22:34 ` Sedat Dilek
[not found] ` <2c496d24174e63b27ec047f383df6700@matoro.tk>
2022-08-06 7:23 ` Sedat Dilek
2022-08-06 13:12 ` matoro
2022-08-08 5:54 ` Masahiro Yamada
2022-08-08 13:26 ` matoro
2022-08-08 17:36 ` Masahiro Yamada
2022-08-08 18:42 ` matoro
2022-08-09 15:20 ` Masahiro Yamada
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).