* [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR
@ 2019-07-17 6:17 Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 01/12] kbuild: do not create empty modules.order in the prepare stage Masahiro Yamada
` (11 more replies)
0 siblings, 12 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:17 UTC (permalink / raw)
To: linux-kbuild
Cc: Joe Lawrence, Masahiro Yamada, James E.J. Bottomley,
Jonathan Corbet, Martin K. Petersen, Michal Marek, Shuah Khan,
Thomas Renninger, linux-doc, linux-kernel, linux-pm, linux-scsi
This series kills the long standing MODVERDIR.
Since MODVERDIR has a flat structure, it cannot avoid a race
condition when somebody introduces a module name conflict.
Kbuild now reads modules.order to get the list of all modules.
The post-processing/installation stages will be more robust
and simpler.
Masahiro Yamada (12):
kbuild: do not create empty modules.order in the prepare stage
kbuild: get rid of kernel/ prefix from in-tree modules.{order,builtin}
kbuild: remove duplication from modules.order in sub-directories
scsi: remove pointless $(MODVERDIR)/$(obj)/53c700.ver
kbuild: modinst: read modules.order instead of $(MODVERDIR)/*.mod
kbuild: modsign: read modules.order instead of $(MODVERDIR)/*.mod
kbuild: modpost: read modules.order instead of $(MODVERDIR)/*.mod
kbuild: export_report: read modules.order instead of
.tmp_versions/*.mod
kbuild: create *.mod with full directory path and remove MODVERDIR
kbuild: remove the first line of *.mod files
kbuild: remove 'prepare1' target
kbuild: split out *.mod out of {single,multi}-used-m rules
.gitignore | 1 +
Documentation/dontdiff | 1 +
Makefile | 36 ++++++-------------
drivers/scsi/Makefile | 2 +-
lib/Kconfig.debug | 12 +------
scripts/Makefile.build | 40 +++++++++-------------
scripts/Makefile.modbuiltin | 2 +-
scripts/Makefile.modinst | 5 +--
scripts/Makefile.modpost | 19 +++++-----
scripts/Makefile.modsign | 3 +-
scripts/adjust_autoksyms.sh | 14 +++-----
scripts/export_report.pl | 11 +++---
scripts/mod/sumversion.c | 23 +++----------
scripts/modules-check.sh | 2 +-
scripts/package/mkspec | 2 +-
tools/power/cpupower/debug/kernel/Makefile | 4 +--
16 files changed, 62 insertions(+), 115 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v3 01/12] kbuild: do not create empty modules.order in the prepare stage
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
@ 2019-07-17 6:17 ` Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 02/12] kbuild: get rid of kernel/ prefix from in-tree modules.{order,builtin} Masahiro Yamada
` (10 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:17 UTC (permalink / raw)
To: linux-kbuild; +Cc: Joe Lawrence, Masahiro Yamada, Michal Marek, linux-kernel
Currently, $(objtree)/modules.order is touched in two places.
In the 'prepare0' rule, scripts/Makefile.build creates an empty
modules.order while processing 'obj=.'
In the 'modules' rule, the top-level Makefile overwrites it with
the correct list of modules.
While this might be a good side-effect that modules.order is made
empty every time (probably this is not intended functionality),
I personally do not like this behavior.
Create modules.order only when it is sensible to do so.
This avoids creating the following pointless files:
scripts/basic/modules.order
scripts/dtc/modules.order
scripts/gcc-plugins/modules.order
scripts/genksyms/modules.order
scripts/mod/modules.order
scripts/modules.order
scripts/selinux/genheaders/modules.order
scripts/selinux/mdp/modules.order
scripts/selinux/modules.order
Going forward, $(objtree)/modules.order lists the modules that
was built in the last successful build.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2:
- inverted the logic "preparing" -> need-modorder
Makefile | 4 ++--
scripts/Makefile.build | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/Makefile b/Makefile
index e43285786102..b74a6e9cefc8 100644
--- a/Makefile
+++ b/Makefile
@@ -1072,7 +1072,7 @@ $(sort $(vmlinux-deps)): $(vmlinux-dirs) ;
PHONY += $(vmlinux-dirs)
$(vmlinux-dirs): prepare
- $(Q)$(MAKE) $(build)=$@ need-builtin=1
+ $(Q)$(MAKE) $(build)=$@ need-builtin=1 need-modorder=1
filechk_kernel.release = \
echo "$(KERNELVERSION)$$($(CONFIG_SHELL) $(srctree)/scripts/setlocalversion $(srctree))"
@@ -1616,7 +1616,7 @@ $(objtree)/Module.symvers:
module-dirs := $(addprefix _module_,$(KBUILD_EXTMOD))
PHONY += $(module-dirs) modules
$(module-dirs): prepare $(objtree)/Module.symvers
- $(Q)$(MAKE) $(build)=$(patsubst _module_%,%,$@)
+ $(Q)$(MAKE) $(build)=$(patsubst _module_%,%,$@) need-modorder=1
modules: $(module-dirs)
@$(kecho) ' Building modules, stage 2.';
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 5829ccbc7dd0..631bb89524de 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -63,7 +63,7 @@ ifneq ($(strip $(real-obj-y) $(need-builtin)),)
builtin-target := $(obj)/built-in.a
endif
-ifdef CONFIG_MODULES
+ifeq ($(CONFIG_MODULES)$(need-modorder),y1)
modorder-target := $(obj)/modules.order
endif
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 02/12] kbuild: get rid of kernel/ prefix from in-tree modules.{order,builtin}
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 01/12] kbuild: do not create empty modules.order in the prepare stage Masahiro Yamada
@ 2019-07-17 6:17 ` Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 03/12] kbuild: remove duplication from modules.order in sub-directories Masahiro Yamada
` (9 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:17 UTC (permalink / raw)
To: linux-kbuild; +Cc: Joe Lawrence, Masahiro Yamada, Michal Marek, linux-kernel
Removing the 'kernel/' prefix will make our life easier because we can
simply do 'cat modules.order' to get all built modules with full paths.
Currently, we parse the first line of '*.mod' files in $(MODVERDIR).
Since we have duplicated functionality here, I plan to remove MODVERDIR
entirely.
In fact, modules.order is generated also for external modules in a
broken format. It adds the 'kernel/' prefix to the absolute path of
the module, like this:
kernel//path/to/your/external/module/foo.ko
This is fine for now since modules.order is not used for external
modules. However, I want to sanitize the format everywhere towards
the goal of removing MODVERDIR.
We cannot change the format of installed module.{order,builtin}.
So, 'make modules_install' will add the 'kernel/' prefix while copying
them to $(MODLIB)/.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
Makefile | 4 ++--
scripts/Makefile.build | 2 +-
scripts/Makefile.modbuiltin | 2 +-
scripts/modules-check.sh | 2 +-
4 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/Makefile b/Makefile
index b74a6e9cefc8..396cd5e525d1 100644
--- a/Makefile
+++ b/Makefile
@@ -1329,8 +1329,8 @@ _modinst_:
rm -f $(MODLIB)/build ; \
ln -s $(CURDIR) $(MODLIB)/build ; \
fi
- @cp -f $(objtree)/modules.order $(MODLIB)/
- @cp -f $(objtree)/modules.builtin $(MODLIB)/
+ @sed 's:^:kernel/:' modules.order > $(MODLIB)/modules.order
+ @sed 's:^:kernel/:' modules.builtin > $(MODLIB)/modules.builtin
@cp -f $(objtree)/modules.builtin.modinfo $(MODLIB)/
$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modinst
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 631bb89524de..10e92a27ec20 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -426,7 +426,7 @@ endif # builtin-target
modorder-cmds = \
$(foreach m, $(modorder), \
$(if $(filter %/modules.order, $m), \
- cat $m;, echo kernel/$m;))
+ cat $m;, echo $m;))
$(modorder-target): $(subdir-ym) FORCE
$(Q)(cat /dev/null; $(modorder-cmds)) > $@
diff --git a/scripts/Makefile.modbuiltin b/scripts/Makefile.modbuiltin
index 50a9990760f3..7d4711b88656 100644
--- a/scripts/Makefile.modbuiltin
+++ b/scripts/Makefile.modbuiltin
@@ -40,7 +40,7 @@ __modbuiltin: $(modbuiltin-target) $(subdir-ym)
@:
$(modbuiltin-target): $(subdir-ym) FORCE
- $(Q)(for m in $(modbuiltin-mods); do echo kernel/$$m; done; \
+ $(Q)(for m in $(modbuiltin-mods); do echo $$m; done; \
cat /dev/null $(modbuiltin-subdirs)) > $@
PHONY += FORCE
diff --git a/scripts/modules-check.sh b/scripts/modules-check.sh
index 39e8cb36ba19..f51f446707b8 100755
--- a/scripts/modules-check.sh
+++ b/scripts/modules-check.sh
@@ -9,7 +9,7 @@ check_same_name_modules()
for m in $(sed 's:.*/::' modules.order | sort | uniq -d)
do
echo "warning: same module names found:" >&2
- sed -n "/\/$m/s:^kernel/: :p" modules.order >&2
+ sed -n "/\/$m/s:^: :p" modules.order >&2
done
}
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 03/12] kbuild: remove duplication from modules.order in sub-directories
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 01/12] kbuild: do not create empty modules.order in the prepare stage Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 02/12] kbuild: get rid of kernel/ prefix from in-tree modules.{order,builtin} Masahiro Yamada
@ 2019-07-17 6:17 ` Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 04/12] scsi: remove pointless $(MODVERDIR)/$(obj)/53c700.ver Masahiro Yamada
` (8 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:17 UTC (permalink / raw)
To: linux-kbuild; +Cc: Joe Lawrence, Masahiro Yamada, Michal Marek, linux-kernel
Currently, only the top-level modules.order drops duplicated entries.
The modules.order files in sub-directories potentially contain
duplication. To list out the paths of all modules, I want to use
modules.order instead of parsing *.mod files in $(MODVERDIR).
To achieve this, I want to rip off duplication from modules.order
of external modules too.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
scripts/Makefile.build | 9 +++------
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 10e92a27ec20..be32a3752de4 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -423,13 +423,10 @@ endif # builtin-target
#
# Create commands to either record .ko file or cat modules.order from
# a subdirectory
-modorder-cmds = \
- $(foreach m, $(modorder), \
- $(if $(filter %/modules.order, $m), \
- cat $m;, echo $m;))
-
$(modorder-target): $(subdir-ym) FORCE
- $(Q)(cat /dev/null; $(modorder-cmds)) > $@
+ $(Q){ $(foreach m, $(modorder), \
+ $(if $(filter %/modules.order, $m), cat $m, echo $m);) :; } \
+ | $(AWK) '!x[$$0]++' - > $@
#
# Rule to compile a set of .o files into one .a file (with symbol table)
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 04/12] scsi: remove pointless $(MODVERDIR)/$(obj)/53c700.ver
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
` (2 preceding siblings ...)
2019-07-17 6:17 ` [PATCH v3 03/12] kbuild: remove duplication from modules.order in sub-directories Masahiro Yamada
@ 2019-07-17 6:17 ` Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 05/12] kbuild: modinst: read modules.order instead of $(MODVERDIR)/*.mod Masahiro Yamada
` (7 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:17 UTC (permalink / raw)
To: linux-kbuild
Cc: Joe Lawrence, Masahiro Yamada, James E.J. Bottomley,
Martin K. Petersen, linux-kernel, linux-scsi
Nothing depends on this, so it is dead code.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
drivers/scsi/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/scsi/Makefile b/drivers/scsi/Makefile
index aeda53901064..c00e3dd57990 100644
--- a/drivers/scsi/Makefile
+++ b/drivers/scsi/Makefile
@@ -185,7 +185,7 @@ zalon7xx-objs := zalon.o ncr53c8xx.o
# Files generated that shall be removed upon make clean
clean-files := 53c700_d.h 53c700_u.h scsi_devinfo_tbl.c
-$(obj)/53c700.o $(MODVERDIR)/$(obj)/53c700.ver: $(obj)/53c700_d.h
+$(obj)/53c700.o: $(obj)/53c700_d.h
$(obj)/scsi_sysfs.o: $(obj)/scsi_devinfo_tbl.c
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 05/12] kbuild: modinst: read modules.order instead of $(MODVERDIR)/*.mod
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
` (3 preceding siblings ...)
2019-07-17 6:17 ` [PATCH v3 04/12] scsi: remove pointless $(MODVERDIR)/$(obj)/53c700.ver Masahiro Yamada
@ 2019-07-17 6:17 ` Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 06/12] kbuild: modsign: " Masahiro Yamada
` (6 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:17 UTC (permalink / raw)
To: linux-kbuild; +Cc: Joe Lawrence, Masahiro Yamada, Michal Marek, linux-kernel
Towards the goal of removing MODVERDIR, read out modules.order to get
the list of modules to be installed. This is simpler than parsing *.mod
files in $(MODVERDIR).
For external modules, $(KBUILD_EXTMOD)/modules.order should be read.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
scripts/Makefile.modinst | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
index 0dae402661f3..5a4579e76485 100644
--- a/scripts/Makefile.modinst
+++ b/scripts/Makefile.modinst
@@ -8,10 +8,7 @@ __modinst:
include scripts/Kbuild.include
-#
-
-__modules := $(sort $(shell grep -h '\.ko$$' /dev/null $(wildcard $(MODVERDIR)/*.mod)))
-modules := $(patsubst %.o,%.ko,$(wildcard $(__modules:.ko=.o)))
+modules := $(sort $(shell cat $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/)modules.order))
PHONY += $(modules)
__modinst: $(modules)
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 06/12] kbuild: modsign: read modules.order instead of $(MODVERDIR)/*.mod
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
` (4 preceding siblings ...)
2019-07-17 6:17 ` [PATCH v3 05/12] kbuild: modinst: read modules.order instead of $(MODVERDIR)/*.mod Masahiro Yamada
@ 2019-07-17 6:17 ` Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 07/12] kbuild: modpost: " Masahiro Yamada
` (5 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:17 UTC (permalink / raw)
To: linux-kbuild; +Cc: Joe Lawrence, Masahiro Yamada, Michal Marek, linux-kernel
Towards the goal of removing MODVERDIR, read out modules.order to get
the list of modules to be signed. This is simpler than parsing *.mod
files in $(MODVERDIR).
The modules_sign target is only supported for in-kernel modules.
So, this commit does not take care of external modules.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
scripts/Makefile.modsign | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/scripts/Makefile.modsign b/scripts/Makefile.modsign
index da56aa78d245..d7325cefe709 100644
--- a/scripts/Makefile.modsign
+++ b/scripts/Makefile.modsign
@@ -8,8 +8,7 @@ __modsign:
include scripts/Kbuild.include
-__modules := $(sort $(shell grep -h '\.ko$$' /dev/null $(wildcard $(MODVERDIR)/*.mod)))
-modules := $(patsubst %.o,%.ko,$(wildcard $(__modules:.ko=.o)))
+modules := $(sort $(shell cat modules.order))
PHONY += $(modules)
__modsign: $(modules)
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 07/12] kbuild: modpost: read modules.order instead of $(MODVERDIR)/*.mod
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
` (5 preceding siblings ...)
2019-07-17 6:17 ` [PATCH v3 06/12] kbuild: modsign: " Masahiro Yamada
@ 2019-07-17 6:17 ` Masahiro Yamada
2019-07-25 8:39 ` Jan Kiszka
2019-07-17 6:17 ` [PATCH v3 08/12] kbuild: export_report: read modules.order instead of .tmp_versions/*.mod Masahiro Yamada
` (4 subsequent siblings)
11 siblings, 1 reply; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:17 UTC (permalink / raw)
To: linux-kbuild; +Cc: Joe Lawrence, Masahiro Yamada, Michal Marek, linux-kernel
Towards the goal of removing MODVERDIR, read out modules.order to get
the list of modules to be processed. This is simpler than parsing *.mod
files in $(MODVERDIR).
For external modules, $(KBUILD_EXTMOD)/modules.order should be read.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3:
- Add ifdef CONFIG_MODULES to avoid warning
Changes in v2: None
scripts/Makefile.modpost | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
index fec6ec2ffa47..5841508ffca9 100644
--- a/scripts/Makefile.modpost
+++ b/scripts/Makefile.modpost
@@ -8,9 +8,10 @@
# b) A <module>.o file which is the .o files above linked together
# c) A <module>.mod file in $(MODVERDIR)/, listing the name of the
# the preliminary <module>.o file, plus all .o files
+# d) modules.order, which lists all the modules
# Stage 2 is handled by this file and does the following
-# 1) Find all modules from the files listed in $(MODVERDIR)/
+# 1) Find all modules listed in modules.order
# 2) modpost is then used to
# 3) create one <module>.mod.c file pr. module
# 4) create one Module.symvers file with CRC for all exported symbols
@@ -60,10 +61,12 @@ include scripts/Makefile.lib
kernelsymfile := $(objtree)/Module.symvers
modulesymfile := $(firstword $(KBUILD_EXTMOD))/Module.symvers
-# Step 1), find all modules listed in $(MODVERDIR)/
-MODLISTCMD := find $(MODVERDIR) -name '*.mod' | xargs -r grep -h '\.ko$$' | sort -u
-__modules := $(shell $(MODLISTCMD))
-modules := $(patsubst %.o,%.ko, $(wildcard $(__modules:.ko=.o)))
+modorder := $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/)modules.order
+
+# Step 1), find all modules listed in modules.order
+ifdef CONFIG_MODULES
+modules := $(sort $(shell cat $(modorder)))
+endif
# Stop after building .o files if NOFINAL is set. Makes compile tests quicker
_modpost: $(if $(KBUILD_MODPOST_NOFINAL), $(modules:.ko:.o),$(modules))
@@ -84,7 +87,7 @@ MODPOST_OPT=$(subst -i,-n,$(filter -i,$(MAKEFLAGS)))
# We can go over command line length here, so be careful.
quiet_cmd_modpost = MODPOST $(words $(filter-out vmlinux FORCE, $^)) modules
- cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$$/.o/' | $(modpost) $(MODPOST_OPT) -s -T -
+ cmd_modpost = sed 's/ko$$/o/' $(modorder) | $(modpost) $(MODPOST_OPT) -s -T -
PHONY += __modpost
__modpost: $(modules:.ko=.o) FORCE
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 08/12] kbuild: export_report: read modules.order instead of .tmp_versions/*.mod
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
` (6 preceding siblings ...)
2019-07-17 6:17 ` [PATCH v3 07/12] kbuild: modpost: " Masahiro Yamada
@ 2019-07-17 6:17 ` Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 09/12] kbuild: create *.mod with full directory path and remove MODVERDIR Masahiro Yamada
` (3 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:17 UTC (permalink / raw)
To: linux-kbuild; +Cc: Joe Lawrence, Masahiro Yamada, linux-kernel
Towards the goal of removing MODVERDIR aka .tmp_versions, read out
modules.order to get the list of modules to be processed. This is
simpler than parsing *.mod files in .tmp_versions.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3:
- New patch
Changes in v2: None
scripts/export_report.pl | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/scripts/export_report.pl b/scripts/export_report.pl
index 0f604f62f067..7d3030d03a25 100755
--- a/scripts/export_report.pl
+++ b/scripts/export_report.pl
@@ -52,13 +52,12 @@ sub usage {
sub collectcfiles {
my @file;
- while (<.tmp_versions/*.mod>) {
- open my $fh, '<', $_ or die "cannot open $_: $!\n";
- push (@file,
- grep s/\.ko/.mod.c/, # change the suffix
- grep m/.+\.ko/, # find the .ko path
- <$fh>); # lines in opened file
+ open my $fh, '< modules.order' or die "cannot open modules.order: $!\n";
+ while (<$fh>) {
+ s/\.ko$/.mod.c/;
+ push (@file, $_)
}
+ close($fh);
chomp @file;
return @file;
}
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 09/12] kbuild: create *.mod with full directory path and remove MODVERDIR
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
` (7 preceding siblings ...)
2019-07-17 6:17 ` [PATCH v3 08/12] kbuild: export_report: read modules.order instead of .tmp_versions/*.mod Masahiro Yamada
@ 2019-07-17 6:17 ` Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 10/12] kbuild: remove the first line of *.mod files Masahiro Yamada
` (2 subsequent siblings)
11 siblings, 0 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:17 UTC (permalink / raw)
To: linux-kbuild
Cc: Joe Lawrence, Masahiro Yamada, Jonathan Corbet, Michal Marek,
Shuah Khan, Thomas Renninger, linux-doc, linux-kernel, linux-pm
While descending directories, Kbuild produces objects for modules,
but do not link final *.ko files; it is done in the modpost.
To keep track of modules, Kbuild creates a *.mod file in $(MODVERDIR)
for every module it is building. Some post-processing steps read the
necessary information from *.mod files. This avoids descending into
directories again. This mechanism was introduced in 2003 or so.
Later, commit 551559e13af1 ("kbuild: implement modules.order") added
modules.order. So, we can simply read it out to know all the modules
with directory paths. This is easier than parsing the first line of
*.mod files.
$(MODVERDIR) has a flat directory structure, that is, *.mod files
are named only with base names. This is based on the assumption that
the module name is unique across the tree. This assumption is really
fragile.
Stephen Rothwell reported a race condition caused by a module name
conflict:
https://lkml.org/lkml/2019/5/13/991
In parallel building, two different threads could write to the same
$(MODVERDIR)/*.mod simultaneously.
Non-unique module names are the source of all kind of troubles, hence
commit 3a48a91901c5 ("kbuild: check uniqueness of module names")
introduced a new checker script.
However, it is still fragile in the build system point of view because
this race happens before scripts/modules-check.sh is invoked. If it
happens again, the modpost will emit unclear error messages.
To fix this issue completely, create *.mod with full directory path
so that two threads never attempt to write to the same file.
$(MODVERDIR) is no longer needed.
Since modules with directory paths are listed in modules.order, Kbuild
is still able to find *.mod files without additional descending.
I also killed cmd_secanalysis; scripts/mod/sumversion.c computes MD4 hash
for modules with MODULE_VERSION(). When CONFIG_DEBUG_SECTION_MISMATCH=y,
it occurs not only in the modpost stage, but also during directory
descending, where sumversion.c may parse stale *.mod files. It would emit
'No such file or directory' warning when an object consisting a module is
renamed, or when a single-obj module is turned into a multi-obj module or
vice versa.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
---
Changes in v3:
- Fix build error of allnoconfig
- Remove cmd_secanalysis
- Fix up comment in scripts/adjust_autoksyms.sh
- Fix up tools/power/cpupower/debug/kernel/Makefile
Changes in v2:
- Remove -r of xargs, which is a GNU extension
- Add '--' for extra safety
.gitignore | 1 +
Documentation/dontdiff | 1 +
Makefile | 20 +++-----------------
lib/Kconfig.debug | 12 +-----------
scripts/Makefile.build | 15 +++------------
scripts/Makefile.modpost | 4 ++--
scripts/adjust_autoksyms.sh | 14 +++++---------
scripts/mod/sumversion.c | 16 +++-------------
scripts/package/mkspec | 2 +-
tools/power/cpupower/debug/kernel/Makefile | 4 ++--
10 files changed, 22 insertions(+), 67 deletions(-)
diff --git a/.gitignore b/.gitignore
index 7587ef56b92d..8f5422cba6e2 100644
--- a/.gitignore
+++ b/.gitignore
@@ -30,6 +30,7 @@
*.lz4
*.lzma
*.lzo
+*.mod
*.mod.c
*.o
*.o.*
diff --git a/Documentation/dontdiff b/Documentation/dontdiff
index 5eba889ea84d..9f4392876099 100644
--- a/Documentation/dontdiff
+++ b/Documentation/dontdiff
@@ -30,6 +30,7 @@
*.lzo
*.mo
*.moc
+*.mod
*.mod.c
*.o
*.o.*
diff --git a/Makefile b/Makefile
index 396cd5e525d1..9ad9f8d1130d 100644
--- a/Makefile
+++ b/Makefile
@@ -486,11 +486,6 @@ export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE
export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL
export KBUILD_ARFLAGS
-# When compiling out-of-tree modules, put MODVERDIR in the module
-# tree rather than in the kernel tree. The kernel tree might
-# even be read-only.
-export MODVERDIR := $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/).tmp_versions
-
# Files to ignore in find ... statements
export RCS_FIND_IGNORE := \( -name SCCS -o -name BitKeeper -o -name .svn -o \
@@ -1029,8 +1024,8 @@ vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_OBJS) $(KBUILD_VMLINUX_LIBS)
# Recurse until adjust_autoksyms.sh is satisfied
PHONY += autoksyms_recursive
-autoksyms_recursive: $(vmlinux-deps)
ifdef CONFIG_TRIM_UNUSED_KSYMS
+autoksyms_recursive: $(vmlinux-deps) modules.order
$(Q)$(CONFIG_SHELL) $(srctree)/scripts/adjust_autoksyms.sh \
"$(MAKE) -f $(srctree)/Makefile vmlinux"
endif
@@ -1113,7 +1108,6 @@ endif
prepare1: prepare3 outputmakefile asm-generic $(version_h) $(autoksyms_h) \
include/generated/utsrelease.h
- $(cmd_crmodverdir)
archprepare: archheaders archscripts prepare1 scripts
@@ -1371,7 +1365,7 @@ endif # CONFIG_MODULES
# make distclean Remove editor backup files, patch leftover files and the like
# Directories & files removed with 'make clean'
-CLEAN_DIRS += $(MODVERDIR) include/ksym
+CLEAN_DIRS += include/ksym
CLEAN_FILES += modules.builtin.modinfo
# Directories & files removed with 'make mrproper'
@@ -1641,7 +1635,6 @@ PHONY += $(clean-dirs) clean
$(clean-dirs):
$(Q)$(MAKE) $(clean)=$(patsubst _clean_%,%,$@)
-clean: rm-dirs := $(MODVERDIR)
clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers
PHONY += help
@@ -1655,8 +1648,6 @@ help:
@echo ''
PHONY += prepare
-prepare:
- $(cmd_crmodverdir)
endif # KBUILD_EXTMOD
clean: $(clean-dirs)
@@ -1667,7 +1658,7 @@ clean: $(clean-dirs)
-o -name '*.ko.*' \
-o -name '*.dtb' -o -name '*.dtb.S' -o -name '*.dt.yaml' \
-o -name '*.dwo' -o -name '*.lst' \
- -o -name '*.su' \
+ -o -name '*.su' -o -name '*.mod' \
-o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \
-o -name '*.lex.c' -o -name '*.tab.[ch]' \
-o -name '*.asn1.[ch]' \
@@ -1796,11 +1787,6 @@ quiet_cmd_depmod = DEPMOD $(KERNELRELEASE)
cmd_depmod = $(CONFIG_SHELL) $(srctree)/scripts/depmod.sh $(DEPMOD) \
$(KERNELRELEASE)
-# Create temporary dir for module support files
-# clean it up only when building all modules
-cmd_crmodverdir = $(Q)mkdir -p $(MODVERDIR) \
- $(if $(KBUILD_MODULES),; rm -f $(MODVERDIR)/*)
-
# read saved command lines for existing targets
existing-targets := $(wildcard $(sort $(targets)))
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 4ac4ca21a30a..cde5675340ba 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -353,23 +353,13 @@ config DEBUG_SECTION_MISMATCH
which results in the code/data being placed in specific sections.
The section mismatch analysis is always performed after a full
kernel build, and enabling this option causes the following
- additional steps to occur:
+ additional step to occur:
- Add the option -fno-inline-functions-called-once to gcc commands.
When inlining a function annotated with __init in a non-init
function, we would lose the section information and thus
the analysis would not catch the illegal reference.
This option tells gcc to inline less (but it does result in
a larger kernel).
- - Run the section mismatch analysis for each module/built-in.a file.
- When we run the section mismatch analysis on vmlinux.o, we
- lose valuable information about where the mismatch was
- introduced.
- Running the analysis for each module/built-in.a file
- tells where the mismatch happens much closer to the
- source. The drawback is that the same mismatch is
- reported at least twice.
- - Enable verbose reporting from modpost in order to help resolve
- the section mismatches that are reported.
config SECTION_MISMATCH_WARN_ONLY
bool "Make section mismatch errors non-fatal"
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index be32a3752de4..c6dfcc028f56 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -67,8 +67,6 @@ ifeq ($(CONFIG_MODULES)$(need-modorder),y1)
modorder-target := $(obj)/modules.order
endif
-# We keep a list of all modules in $(MODVERDIR)
-
__build: $(if $(KBUILD_BUILTIN),$(builtin-target) $(lib-target) $(extra-y)) \
$(if $(KBUILD_MODULES),$(obj-m) $(modorder-target)) \
$(subdir-ym) $(always)
@@ -87,11 +85,6 @@ ifneq ($(KBUILD_ENABLE_EXTRA_GCC_CHECKS),)
cmd_checkdoc = $(srctree)/scripts/kernel-doc -none $<
endif
-# Do section mismatch analysis for each module/built-in.a
-ifdef CONFIG_DEBUG_SECTION_MISMATCH
- cmd_secanalysis = ; scripts/mod/modpost $@
-endif
-
# Compile C sources (.c)
# ---------------------------------------------------------------------------
@@ -278,13 +271,11 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) $(objtool_dep) FORCE
$(call cmd,force_checksrc)
$(call if_changed_rule,cc_o_c)
-# Single-part modules are special since we need to mark them in $(MODVERDIR)
-
$(single-used-m): $(obj)/%.o: $(src)/%.c $(recordmcount_source) $(objtool_dep) FORCE
$(call cmd,force_checksrc)
$(call if_changed_rule,cc_o_c)
@{ echo $(@:.o=.ko); echo $@; \
- $(cmd_undef_syms); } > $(MODVERDIR)/$(@F:.o=.mod)
+ $(cmd_undef_syms); } > $(patsubst %.o,%.mod,$@)
quiet_cmd_cc_lst_c = MKLST $@
cmd_cc_lst_c = $(CC) $(c_flags) -g -c -o $*.o $< && \
@@ -461,12 +452,12 @@ endif
# module is turned into a multi object module, $^ will contain header file
# dependencies recorded in the .*.cmd file.
quiet_cmd_link_multi-m = LD [M] $@
-cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ $(filter %.o,$^) $(cmd_secanalysis)
+ cmd_link_multi-m = $(LD) $(ld_flags) -r -o $@ $(filter %.o,$^)
$(multi-used-m): FORCE
$(call if_changed,link_multi-m)
@{ echo $(@:.o=.ko); echo $(filter %.o,$^); \
- $(cmd_undef_syms); } > $(MODVERDIR)/$(@F:.o=.mod)
+ $(cmd_undef_syms); } > $(patsubst %.o,%.mod,$@)
$(call multi_depend, $(multi-used-m), .o, -objs -y -m)
targets += $(multi-used-m)
diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
index 5841508ffca9..6b19c1a4eae5 100644
--- a/scripts/Makefile.modpost
+++ b/scripts/Makefile.modpost
@@ -6,8 +6,8 @@
# Stage one of module building created the following:
# a) The individual .o files used for the module
# b) A <module>.o file which is the .o files above linked together
-# c) A <module>.mod file in $(MODVERDIR)/, listing the name of the
-# the preliminary <module>.o file, plus all .o files
+# c) A <module>.mod file, listing the name of the preliminary <module>.o file,
+# plus all .o files
# d) modules.order, which lists all the modules
# Stage 2 is handled by this file and does the following
diff --git a/scripts/adjust_autoksyms.sh b/scripts/adjust_autoksyms.sh
index aab4e299d7a2..2e4a7320bfb4 100755
--- a/scripts/adjust_autoksyms.sh
+++ b/scripts/adjust_autoksyms.sh
@@ -8,8 +8,7 @@
#
# Create/update the include/generated/autoksyms.h file from the list
-# of all module's needed symbols as recorded on the third line of
-# .tmp_versions/*.mod files.
+# of all module's needed symbols as recorded on the third line of *.mod files.
#
# For each symbol being added or removed, the corresponding dependency
# file's timestamp is updated to force a rebuild of the affected source
@@ -47,13 +46,10 @@ cat > "$new_ksyms_file" << EOT
*/
EOT
-[ "$(ls -A "$MODVERDIR")" ] &&
-for mod in "$MODVERDIR"/*.mod; do
- sed -n -e '3{s/ /\n/g;/^$/!p;}' "$mod"
-done | sort -u |
-while read sym; do
- echo "#define __KSYM_${sym} 1"
-done >> "$new_ksyms_file"
+sed 's/ko$/mod/' modules.order |
+xargs -n1 sed -n -e '3{s/ /\n/g;/^$/!p;}' -- |
+sort -u |
+sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$new_ksyms_file"
# Special case for modversions (see modpost.c)
if [ -n "$CONFIG_MODVERSIONS" ]; then
diff --git a/scripts/mod/sumversion.c b/scripts/mod/sumversion.c
index 0f6dcb4011a8..166f3fa247a9 100644
--- a/scripts/mod/sumversion.c
+++ b/scripts/mod/sumversion.c
@@ -396,21 +396,11 @@ void get_src_version(const char *modname, char sum[], unsigned sumlen)
unsigned long len;
struct md4_ctx md;
char *sources, *end, *fname;
- const char *basename;
char filelist[PATH_MAX + 1];
- char *modverdir = getenv("MODVERDIR");
- if (!modverdir)
- modverdir = ".";
-
- /* Source files for module are in .tmp_versions/modname.mod,
- after the first line. */
- if (strrchr(modname, '/'))
- basename = strrchr(modname, '/') + 1;
- else
- basename = modname;
- snprintf(filelist, sizeof(filelist), "%s/%.*s.mod", modverdir,
- (int) strlen(basename) - 2, basename);
+ /* objects for a module are listed in the second line of *.mod file. */
+ snprintf(filelist, sizeof(filelist), "%.*smod",
+ (int)strlen(modname) - 1, modname);
file = grab_file(filelist, &len);
if (!file)
diff --git a/scripts/package/mkspec b/scripts/package/mkspec
index 2d29df4a0a53..8640c278f1aa 100755
--- a/scripts/package/mkspec
+++ b/scripts/package/mkspec
@@ -29,7 +29,7 @@ fi
PROVIDES="$PROVIDES kernel-$KERNELRELEASE"
__KERNELRELEASE=$(echo $KERNELRELEASE | sed -e "s/-/_/g")
-EXCLUDES="$RCS_TAR_IGNORE --exclude=.tmp_versions --exclude=*vmlinux* \
+EXCLUDES="$RCS_TAR_IGNORE --exclude=*vmlinux* --exclude=*.mod \
--exclude=*.o --exclude=*.ko --exclude=*.cmd --exclude=Documentation \
--exclude=.config.old --exclude=.missing-syscalls.d --exclude=*.s"
diff --git a/tools/power/cpupower/debug/kernel/Makefile b/tools/power/cpupower/debug/kernel/Makefile
index c23e5a6ceb7e..7b5c43684be1 100644
--- a/tools/power/cpupower/debug/kernel/Makefile
+++ b/tools/power/cpupower/debug/kernel/Makefile
@@ -12,8 +12,8 @@ default:
$(MAKE) -C $(KDIR) M=$(CURDIR)
clean:
- - rm -rf *.o *.ko .tmp-versions .*.cmd .*.mod.* *.mod.c
- - rm -rf .tmp_versions* Module.symvers modules.order
+ - rm -rf *.o *.ko .*.cmd .*.mod.* *.mod.c
+ - rm -rf Module.symvers modules.order
install: default
install -d $(KMISC)
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 10/12] kbuild: remove the first line of *.mod files
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
` (8 preceding siblings ...)
2019-07-17 6:17 ` [PATCH v3 09/12] kbuild: create *.mod with full directory path and remove MODVERDIR Masahiro Yamada
@ 2019-07-17 6:17 ` Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 11/12] kbuild: remove 'prepare1' target Masahiro Yamada
2019-07-17 6:18 ` [PATCH v3 12/12] kbuild: split out *.mod out of {single,multi}-used-m rules Masahiro Yamada
11 siblings, 0 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:17 UTC (permalink / raw)
To: linux-kbuild; +Cc: Joe Lawrence, Masahiro Yamada, Michal Marek, linux-kernel
The current format of *.mod is like this:
line 1: directory path to the .ko file
line 2: a list of objects linked into this module
line 3: unresolved symbols (only when CONFIG_TRIM_UNUSED_KSYMS=y)
Now that *.mod and *.ko are created in the same directory, the line 1
provides no valuable information. It can be derived by replacing the
extension .mod with .ko. In fact, nobody uses the first line any more.
Cut down the first line.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
scripts/Makefile.build | 4 ++--
scripts/adjust_autoksyms.sh | 4 ++--
scripts/mod/sumversion.c | 9 ++-------
3 files changed, 6 insertions(+), 11 deletions(-)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index c6dfcc028f56..9f37f203882f 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -274,7 +274,7 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) $(objtool_dep) FORCE
$(single-used-m): $(obj)/%.o: $(src)/%.c $(recordmcount_source) $(objtool_dep) FORCE
$(call cmd,force_checksrc)
$(call if_changed_rule,cc_o_c)
- @{ echo $(@:.o=.ko); echo $@; \
+ @{ echo $@; \
$(cmd_undef_syms); } > $(patsubst %.o,%.mod,$@)
quiet_cmd_cc_lst_c = MKLST $@
@@ -456,7 +456,7 @@ quiet_cmd_link_multi-m = LD [M] $@
$(multi-used-m): FORCE
$(call if_changed,link_multi-m)
- @{ echo $(@:.o=.ko); echo $(filter %.o,$^); \
+ @{ echo $(filter %.o,$^); \
$(cmd_undef_syms); } > $(patsubst %.o,%.mod,$@)
$(call multi_depend, $(multi-used-m), .o, -objs -y -m)
diff --git a/scripts/adjust_autoksyms.sh b/scripts/adjust_autoksyms.sh
index 2e4a7320bfb4..a904bf1f5e67 100755
--- a/scripts/adjust_autoksyms.sh
+++ b/scripts/adjust_autoksyms.sh
@@ -8,7 +8,7 @@
#
# Create/update the include/generated/autoksyms.h file from the list
-# of all module's needed symbols as recorded on the third line of *.mod files.
+# of all module's needed symbols as recorded on the second line of *.mod files.
#
# For each symbol being added or removed, the corresponding dependency
# file's timestamp is updated to force a rebuild of the affected source
@@ -47,7 +47,7 @@ cat > "$new_ksyms_file" << EOT
EOT
sed 's/ko$/mod/' modules.order |
-xargs -n1 sed -n -e '3{s/ /\n/g;/^$/!p;}' -- |
+xargs -n1 sed -n -e '2{s/ /\n/g;/^$/!p;}' -- |
sort -u |
sed -e 's/\(.*\)/#define __KSYM_\1 1/' >> "$new_ksyms_file"
diff --git a/scripts/mod/sumversion.c b/scripts/mod/sumversion.c
index 166f3fa247a9..63062024ce0e 100644
--- a/scripts/mod/sumversion.c
+++ b/scripts/mod/sumversion.c
@@ -398,7 +398,7 @@ void get_src_version(const char *modname, char sum[], unsigned sumlen)
char *sources, *end, *fname;
char filelist[PATH_MAX + 1];
- /* objects for a module are listed in the second line of *.mod file. */
+ /* objects for a module are listed in the first line of *.mod file. */
snprintf(filelist, sizeof(filelist), "%.*smod",
(int)strlen(modname) - 1, modname);
@@ -407,13 +407,8 @@ void get_src_version(const char *modname, char sum[], unsigned sumlen)
/* not a module or .mod file missing - ignore */
return;
- sources = strchr(file, '\n');
- if (!sources) {
- warn("malformed versions file for %s\n", modname);
- goto release;
- }
+ sources = file;
- sources++;
end = strchr(sources, '\n');
if (!end) {
warn("bad ending versions file for %s\n", modname);
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 11/12] kbuild: remove 'prepare1' target
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
` (9 preceding siblings ...)
2019-07-17 6:17 ` [PATCH v3 10/12] kbuild: remove the first line of *.mod files Masahiro Yamada
@ 2019-07-17 6:17 ` Masahiro Yamada
2019-07-17 6:18 ` [PATCH v3 12/12] kbuild: split out *.mod out of {single,multi}-used-m rules Masahiro Yamada
11 siblings, 0 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:17 UTC (permalink / raw)
To: linux-kbuild; +Cc: Joe Lawrence, Masahiro Yamada, Michal Marek, linux-kernel
Now that there is no rule for 'prepare1', it can go away.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
Makefile | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/Makefile b/Makefile
index 9ad9f8d1130d..14458ab3d6a8 100644
--- a/Makefile
+++ b/Makefile
@@ -1089,7 +1089,7 @@ scripts: scripts_basic scripts_dtc
# archprepare is used in arch Makefiles and when processed asm symlink,
# version.h and scripts_basic is processed / created.
-PHONY += prepare archprepare prepare1 prepare3
+PHONY += prepare archprepare prepare3
# prepare3 is used to check if we are building in a separate output directory,
# and if so do:
@@ -1106,10 +1106,8 @@ ifdef building_out_of_srctree
fi;
endif
-prepare1: prepare3 outputmakefile asm-generic $(version_h) $(autoksyms_h) \
- include/generated/utsrelease.h
-
-archprepare: archheaders archscripts prepare1 scripts
+archprepare: archheaders archscripts scripts prepare3 outputmakefile \
+ asm-generic $(version_h) $(autoksyms_h) include/generated/utsrelease.h
prepare0: archprepare
$(Q)$(MAKE) $(build)=scripts/mod
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH v3 12/12] kbuild: split out *.mod out of {single,multi}-used-m rules
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
` (10 preceding siblings ...)
2019-07-17 6:17 ` [PATCH v3 11/12] kbuild: remove 'prepare1' target Masahiro Yamada
@ 2019-07-17 6:18 ` Masahiro Yamada
11 siblings, 0 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-17 6:18 UTC (permalink / raw)
To: linux-kbuild; +Cc: Joe Lawrence, Masahiro Yamada, Michal Marek, linux-kernel
Currently, *.mod is created as a side-effect of obj-m.
Split out *.mod as a dedicated build rule, which allows to unify
the %.c -> %.o rule, and remove the single-used-m rule.
This also makes the incremental build of allmodconfig faster because
it saves $(NM) invocation when there is no change in the module.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
---
Changes in v3: None
Changes in v2: None
scripts/Makefile.build | 22 +++++++++++++---------
1 file changed, 13 insertions(+), 9 deletions(-)
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 9f37f203882f..0d434d0afc0b 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -67,8 +67,10 @@ ifeq ($(CONFIG_MODULES)$(need-modorder),y1)
modorder-target := $(obj)/modules.order
endif
+mod-targets := $(patsubst %.o, %.mod, $(obj-m))
+
__build: $(if $(KBUILD_BUILTIN),$(builtin-target) $(lib-target) $(extra-y)) \
- $(if $(KBUILD_MODULES),$(obj-m) $(modorder-target)) \
+ $(if $(KBUILD_MODULES),$(obj-m) $(mod-targets) $(modorder-target)) \
$(subdir-ym) $(always)
@:
@@ -261,7 +263,7 @@ endef
# List module undefined symbols (or empty line if not enabled)
ifdef CONFIG_TRIM_UNUSED_KSYMS
-cmd_undef_syms = $(NM) $@ | sed -n 's/^ *U //p' | xargs echo
+cmd_undef_syms = $(NM) $< | sed -n 's/^ *U //p' | xargs echo
else
cmd_undef_syms = echo
endif
@@ -271,11 +273,15 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) $(objtool_dep) FORCE
$(call cmd,force_checksrc)
$(call if_changed_rule,cc_o_c)
-$(single-used-m): $(obj)/%.o: $(src)/%.c $(recordmcount_source) $(objtool_dep) FORCE
- $(call cmd,force_checksrc)
- $(call if_changed_rule,cc_o_c)
- @{ echo $@; \
- $(cmd_undef_syms); } > $(patsubst %.o,%.mod,$@)
+cmd_mod = { \
+ echo $(if $($*-objs)$($*-y)$($*-m), $(addprefix $(obj)/, $($*-objs) $($*-y) $($*-m)), $(@:.mod=.o)); \
+ $(cmd_undef_syms); \
+ } > $@
+
+$(obj)/%.mod: $(obj)/%.o FORCE
+ $(call if_changed,mod)
+
+targets += $(mod-targets)
quiet_cmd_cc_lst_c = MKLST $@
cmd_cc_lst_c = $(CC) $(c_flags) -g -c -o $*.o $< && \
@@ -456,8 +462,6 @@ quiet_cmd_link_multi-m = LD [M] $@
$(multi-used-m): FORCE
$(call if_changed,link_multi-m)
- @{ echo $(filter %.o,$^); \
- $(cmd_undef_syms); } > $(patsubst %.o,%.mod,$@)
$(call multi_depend, $(multi-used-m), .o, -objs -y -m)
targets += $(multi-used-m)
--
2.17.1
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH v3 07/12] kbuild: modpost: read modules.order instead of $(MODVERDIR)/*.mod
2019-07-17 6:17 ` [PATCH v3 07/12] kbuild: modpost: " Masahiro Yamada
@ 2019-07-25 8:39 ` Jan Kiszka
2019-07-25 15:18 ` Masahiro Yamada
0 siblings, 1 reply; 17+ messages in thread
From: Jan Kiszka @ 2019-07-25 8:39 UTC (permalink / raw)
To: Masahiro Yamada, linux-kbuild; +Cc: Joe Lawrence, Michal Marek, linux-kernel
On 17.07.19 08:17, Masahiro Yamada wrote:
> Towards the goal of removing MODVERDIR, read out modules.order to get
> the list of modules to be processed. This is simpler than parsing *.mod
> files in $(MODVERDIR).
>
> For external modules, $(KBUILD_EXTMOD)/modules.order should be read.
>
> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> ---
>
> Changes in v3:
> - Add ifdef CONFIG_MODULES to avoid warning
>
> Changes in v2: None
>
> scripts/Makefile.modpost | 15 +++++++++------
> 1 file changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
> index fec6ec2ffa47..5841508ffca9 100644
> --- a/scripts/Makefile.modpost
> +++ b/scripts/Makefile.modpost
> @@ -8,9 +8,10 @@
> # b) A <module>.o file which is the .o files above linked together
> # c) A <module>.mod file in $(MODVERDIR)/, listing the name of the
> # the preliminary <module>.o file, plus all .o files
> +# d) modules.order, which lists all the modules
>
> # Stage 2 is handled by this file and does the following
> -# 1) Find all modules from the files listed in $(MODVERDIR)/
> +# 1) Find all modules listed in modules.order
> # 2) modpost is then used to
> # 3) create one <module>.mod.c file pr. module
> # 4) create one Module.symvers file with CRC for all exported symbols
> @@ -60,10 +61,12 @@ include scripts/Makefile.lib
> kernelsymfile := $(objtree)/Module.symvers
> modulesymfile := $(firstword $(KBUILD_EXTMOD))/Module.symvers
>
> -# Step 1), find all modules listed in $(MODVERDIR)/
> -MODLISTCMD := find $(MODVERDIR) -name '*.mod' | xargs -r grep -h '\.ko$$' | sort -u
> -__modules := $(shell $(MODLISTCMD))
> -modules := $(patsubst %.o,%.ko, $(wildcard $(__modules:.ko=.o)))
> +modorder := $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/)modules.order
> +
> +# Step 1), find all modules listed in modules.order
> +ifdef CONFIG_MODULES
> +modules := $(sort $(shell cat $(modorder)))
> +endif
>
> # Stop after building .o files if NOFINAL is set. Makes compile tests quicker
> _modpost: $(if $(KBUILD_MODPOST_NOFINAL), $(modules:.ko:.o),$(modules))
> @@ -84,7 +87,7 @@ MODPOST_OPT=$(subst -i,-n,$(filter -i,$(MAKEFLAGS)))
>
> # We can go over command line length here, so be careful.
> quiet_cmd_modpost = MODPOST $(words $(filter-out vmlinux FORCE, $^)) modules
> - cmd_modpost = $(MODLISTCMD) | sed 's/\.ko$$/.o/' | $(modpost) $(MODPOST_OPT) -s -T -
> + cmd_modpost = sed 's/ko$$/o/' $(modorder) | $(modpost) $(MODPOST_OPT) -s -T -
>
> PHONY += __modpost
> __modpost: $(modules:.ko=.o) FORCE
>
This affects also external modules builds: I have patterns here that do
[Makefile]
subdir-y := some-module
[some-module/Makefile]
obj-m := some-module.o
and since this patch, the final some-module.ko is no longer built. Am I missing
something in the kbuild makefiles, or is this a regression?
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v3 07/12] kbuild: modpost: read modules.order instead of $(MODVERDIR)/*.mod
2019-07-25 8:39 ` Jan Kiszka
@ 2019-07-25 15:18 ` Masahiro Yamada
2019-07-25 15:27 ` Jan Kiszka
0 siblings, 1 reply; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-25 15:18 UTC (permalink / raw)
To: Jan Kiszka
Cc: Linux Kbuild mailing list, Joe Lawrence, Michal Marek,
Linux Kernel Mailing List
Hi Jan,
On Thu, Jul 25, 2019 at 5:39 PM Jan Kiszka <jan.kiszka@siemens.com> wrote:
> >
>
> This affects also external modules builds: I have patterns here that do
>
> [Makefile]
> subdir-y := some-module
>
> [some-module/Makefile]
> obj-m := some-module.o
>
> and since this patch, the final some-module.ko is no longer built. Am I missing
> something in the kbuild makefiles, or is this a regression?
Thanks for the report.
Interesting. I have never imagined that Makefiles were written like that.
I just wrote a fix-up, but I have not determined to apply it.
https://patchwork.kernel.org/patch/11059033/
It is easy to fixup your Makefile, though.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v3 07/12] kbuild: modpost: read modules.order instead of $(MODVERDIR)/*.mod
2019-07-25 15:18 ` Masahiro Yamada
@ 2019-07-25 15:27 ` Jan Kiszka
2019-07-25 15:49 ` Masahiro Yamada
0 siblings, 1 reply; 17+ messages in thread
From: Jan Kiszka @ 2019-07-25 15:27 UTC (permalink / raw)
To: Masahiro Yamada
Cc: Linux Kbuild mailing list, Joe Lawrence, Michal Marek,
Linux Kernel Mailing List
On 25.07.19 17:18, Masahiro Yamada wrote:
> Hi Jan,
>
> On Thu, Jul 25, 2019 at 5:39 PM Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>>
>>
>> This affects also external modules builds: I have patterns here that do
>>
>> [Makefile]
>> subdir-y := some-module
>>
>> [some-module/Makefile]
>> obj-m := some-module.o
>>
>> and since this patch, the final some-module.ko is no longer built. Am I missing
>> something in the kbuild makefiles, or is this a regression?
>
> Thanks for the report.
> Interesting. I have never imagined that Makefiles were written like that.
>
> I just wrote a fix-up, but I have not determined to apply it.
> https://patchwork.kernel.org/patch/11059033/
>
> It is easy to fixup your Makefile, though.
Thanks for addressing this quickly! I'm happy to adjust our code [1]. Is the
suggested pattern usable with recent stable kernels as well, say down to 4.4 at
least?
Jan
[1] https://github.com/siemens/jailhouse/blob/master/Kbuild#L54
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v3 07/12] kbuild: modpost: read modules.order instead of $(MODVERDIR)/*.mod
2019-07-25 15:27 ` Jan Kiszka
@ 2019-07-25 15:49 ` Masahiro Yamada
0 siblings, 0 replies; 17+ messages in thread
From: Masahiro Yamada @ 2019-07-25 15:49 UTC (permalink / raw)
To: Jan Kiszka
Cc: Linux Kbuild mailing list, Joe Lawrence, Michal Marek,
Linux Kernel Mailing List
On Fri, Jul 26, 2019 at 12:27 AM Jan Kiszka <jan.kiszka@siemens.com> wrote:
>
> On 25.07.19 17:18, Masahiro Yamada wrote:
> > Hi Jan,
> >
> > On Thu, Jul 25, 2019 at 5:39 PM Jan Kiszka <jan.kiszka@siemens.com> wrote:
> >>>
> >>
> >> This affects also external modules builds: I have patterns here that do
> >>
> >> [Makefile]
> >> subdir-y := some-module
> >>
> >> [some-module/Makefile]
> >> obj-m := some-module.o
> >>
> >> and since this patch, the final some-module.ko is no longer built. Am I missing
> >> something in the kbuild makefiles, or is this a regression?
> >
> > Thanks for the report.
> > Interesting. I have never imagined that Makefiles were written like that.
> >
> > I just wrote a fix-up, but I have not determined to apply it.
> > https://patchwork.kernel.org/patch/11059033/
> >
> > It is easy to fixup your Makefile, though.
>
> Thanks for addressing this quickly! I'm happy to adjust our code [1]. Is the
> suggested pattern usable with recent stable kernels as well, say down to 4.4 at
> least?
Yes, should work for any kernel version.
> Jan
>
> [1] https://github.com/siemens/jailhouse/blob/master/Kbuild#L54
Maybe,
obj-m := driver/ hypervisor/ configs/ inmates/ tools/
and
$(patsubst %/,$(obj)/%,$(obj-m)): $(GEN_CONFIG_MK)
Not tested at all.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2019-07-25 15:50 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-17 6:17 [PATCH v3 00/12] kbuild: create *.mod with directory path and remove MODVERDIR Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 01/12] kbuild: do not create empty modules.order in the prepare stage Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 02/12] kbuild: get rid of kernel/ prefix from in-tree modules.{order,builtin} Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 03/12] kbuild: remove duplication from modules.order in sub-directories Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 04/12] scsi: remove pointless $(MODVERDIR)/$(obj)/53c700.ver Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 05/12] kbuild: modinst: read modules.order instead of $(MODVERDIR)/*.mod Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 06/12] kbuild: modsign: " Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 07/12] kbuild: modpost: " Masahiro Yamada
2019-07-25 8:39 ` Jan Kiszka
2019-07-25 15:18 ` Masahiro Yamada
2019-07-25 15:27 ` Jan Kiszka
2019-07-25 15:49 ` Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 08/12] kbuild: export_report: read modules.order instead of .tmp_versions/*.mod Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 09/12] kbuild: create *.mod with full directory path and remove MODVERDIR Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 10/12] kbuild: remove the first line of *.mod files Masahiro Yamada
2019-07-17 6:17 ` [PATCH v3 11/12] kbuild: remove 'prepare1' target Masahiro Yamada
2019-07-17 6:18 ` [PATCH v3 12/12] kbuild: split out *.mod out of {single,multi}-used-m rules Masahiro Yamada
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).