All of lore.kernel.org
 help / color / mirror / Atom feed
From: Masahiro Yamada <masahiroy@kernel.org>
To: linux-kbuild@vger.kernel.org
Cc: Masahiro Yamada <masahiroy@kernel.org>,
	Michal Marek <michal.lkml@markovi.net>,
	Nick Desaulniers <ndesaulniers@google.com>,
	linux-kernel@vger.kernel.org
Subject: [PATCH v2 07/13] kbuild: do not create built-in.a.symversions or lib.a.symversions
Date: Tue, 31 Aug 2021 16:39:58 +0900	[thread overview]
Message-ID: <20210831074004.3195284-8-masahiroy@kernel.org> (raw)
In-Reply-To: <20210831074004.3195284-1-masahiroy@kernel.org>

Merge all *.o.symversions in scripts/link-vmlinux.sh instead of
incrementally merging them in the unit of built-in.a or lib.a.

This is a preparation for further code cleanups.

The initial patch version was implemented in a shell script, but it
was slow due to the slowness of the 'cat' command [1]. This version
was implemented in Perl.

[1]: https://lore.kernel.org/lkml/CAK7LNATyNAu6sa-TT9JXy=BXr5d2Q5K-sp-mVXXtJDuJyi6_bA@mail.gmail.com/

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 scripts/Makefile.build   | 10 ++------
 scripts/link-vmlinux.sh  |  9 ++-----
 scripts/merge-symvers.pl | 52 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 56 insertions(+), 15 deletions(-)
 create mode 100644 scripts/merge-symvers.pl

diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index afc906cd7256..3ad1b1227371 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -434,11 +434,8 @@ endif
 quiet_cmd_ar_builtin = AR      $@
       cmd_ar_builtin = rm -f $@; $(AR) cDPrST $@ $(real-prereqs)
 
-quiet_cmd_ar_and_symver = AR      $@
-      cmd_ar_and_symver = $(cmd_update_lto_symversions); $(cmd_ar_builtin)
-
 $(obj)/built-in.a: $(real-obj-y) FORCE
-	$(call if_changed,ar_and_symver)
+	$(call if_changed,ar_builtin)
 
 #
 # Rule to create modules.order file
@@ -458,11 +455,8 @@ $(obj)/modules.order: $(obj-m) FORCE
 #
 # Rule to compile a set of .o files into one .a file (with symbol table)
 #
-quiet_cmd_ar_lib = AR      $@
-      cmd_ar_lib = $(cmd_update_lto_symversions); $(cmd_ar)
-
 $(obj)/lib.a: $(lib-y) FORCE
-	$(call if_changed,ar_lib)
+	$(call if_changed,ar)
 
 # NOTE:
 # Do not replace $(filter %.o,^) with $(real-prereqs). When a single object
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index d74cee5c4326..0cc6a03f2cb1 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -57,13 +57,8 @@ gen_initcalls()
 gen_symversions()
 {
 	info GEN .tmp_symversions.lds
-	rm -f .tmp_symversions.lds
-
-	for o in ${KBUILD_VMLINUX_OBJS} ${KBUILD_VMLINUX_LIBS}; do
-		if [ -f ${o}.symversions ]; then
-			cat ${o}.symversions >> .tmp_symversions.lds
-		fi
-	done
+	${PERL} scripts/merge-symvers.pl -a ${AR} -o .tmp_symversions.lds \
+		${KBUILD_VMLINUX_OBJS} ${KBUILD_VMLINUX_LIBS}
 }
 
 # Link of vmlinux.o used for section mismatch analysis
diff --git a/scripts/merge-symvers.pl b/scripts/merge-symvers.pl
new file mode 100644
index 000000000000..0bd092d24eff
--- /dev/null
+++ b/scripts/merge-symvers.pl
@@ -0,0 +1,52 @@
+#!/usr/bin/env perl
+# SPDX-License-Identifier: GPL-2.0-only
+
+use autodie;
+use strict;
+use warnings;
+use Getopt::Long 'GetOptions';
+
+my $ar;
+my $output;
+
+GetOptions(
+	'a|ar=s' => \$ar,
+	'o|output=s'  => \$output,
+);
+
+# Collect all objects
+my @objects;
+
+foreach (@ARGV) {
+	if (/\.o$/) {
+		# Some objects (head-y) are linked to vmlinux directly.
+		push(@objects, $_);
+	} elsif (/\.a$/) {
+		# Most of built-in objects are contained in built-in.a or lib.a.
+		# Use 'ar -t' to get the list of the contained objects.
+		$_ = `$ar -t $_`;
+		push(@objects, split(/\n/));
+	} else {
+		die "$_: unknown file type\n";
+	}
+}
+
+open(my $out_fh, '>', "$output");
+
+foreach (@objects) {
+	# The symbol CRCs for foo/bar/baz.o is output to foo/bar/baz.o.symversions
+	s/(.*)/$1.symversions/;
+
+	if (! -e $_) {
+		# .symversions does not exist if the object does not contain
+		# EXPORT_SYMBOL at all. Skip it.
+		next;
+	}
+
+	open(my $in_fh, '<', "$_");
+	# Concatenate all the existing *.symversions files.
+	print $out_fh do { local $/; <$in_fh> };
+	close $in_fh;
+}
+
+close $out_fh;
-- 
2.30.2


  parent reply	other threads:[~2021-08-31  7:40 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-31  7:39 [PATCH v2 00/13] kbuild: second round of Clang LTO refactoring Masahiro Yamada
2021-08-31  7:39 ` [PATCH v2 01/13] kbuild: move objtool_args back to scripts/Makefile.build Masahiro Yamada
2021-09-04 18:18   ` Josh Poimboeuf
2021-08-31  7:39 ` [PATCH v2 02/13] kbuild: rename __objtool_obj to objtool Masahiro Yamada
2021-08-31 17:16   ` Nick Desaulniers
2021-08-31 17:29   ` Kees Cook
2021-09-04 18:19   ` Josh Poimboeuf
2021-08-31  7:39 ` [PATCH v2 03/13] kbuild: store the objtool command in *.cmd files Masahiro Yamada
2021-08-31 17:23   ` Nick Desaulniers
2021-08-31 17:30   ` Kees Cook
2021-09-04 18:04   ` Josh Poimboeuf
2021-09-04 18:45     ` Josh Poimboeuf
2021-09-08 16:12       ` Masahiro Yamada
2021-08-31  7:39 ` [PATCH v2 04/13] kbuild: factor out OBJECT_FILES_NON_STANDARD check into a macro Masahiro Yamada
2021-08-31 17:25   ` Nick Desaulniers
2021-08-31 17:31   ` Kees Cook
2021-09-04 18:59   ` Josh Poimboeuf
2021-08-31  7:39 ` [PATCH v2 05/13] kbuild: detect objtool update without using .SECONDEXPANSION Masahiro Yamada
2021-08-31 17:32   ` Kees Cook
2021-09-03  0:43     ` Masahiro Yamada
2021-08-31 17:33   ` Nick Desaulniers
2021-09-02 23:42     ` Masahiro Yamada
2021-09-04 19:04   ` Josh Poimboeuf
2021-08-31  7:39 ` [PATCH v2 06/13] kbuild: reuse $(cmd_objtool) for cmd_cc_lto_link_modules Masahiro Yamada
2021-08-31 17:35   ` Kees Cook
2021-09-03  0:39     ` Masahiro Yamada
2021-09-03  1:49       ` Kees Cook
2021-09-04 19:11   ` Josh Poimboeuf
2021-08-31  7:39 ` Masahiro Yamada [this message]
2021-08-31 17:36   ` [PATCH v2 07/13] kbuild: do not create built-in.a.symversions or lib.a.symversions Kees Cook
2021-08-31  7:39 ` [PATCH v2 08/13] kbuild: build modules in the same way with/without Clang LTO Masahiro Yamada
2021-08-31 17:39   ` Kees Cook
2021-08-31 17:46   ` Nick Desaulniers
2021-08-31  7:40 ` [PATCH v2 09/13] kbuild: add cmd_and_savecmd macro Masahiro Yamada
2021-08-31 17:39   ` Kees Cook
2021-08-31  7:40 ` [PATCH v2 10/13] kbuild: rebuild modules when objtool is updated for CONFIG_LTO_CLANG Masahiro Yamada
2021-08-31 17:40   ` Kees Cook
2021-09-04 19:13   ` Josh Poimboeuf
2021-08-31  7:40 ` [PATCH v2 11/13] kbuild: always postpone CRC links for module versioning Masahiro Yamada
2021-09-01 12:00   ` kernel test robot
2021-08-31  7:40 ` [PATCH v2 12/13] kbuild: merge cmd_modversions_c and cmd_modversions_S Masahiro Yamada
2021-08-31  7:40 ` [PATCH v2 13/13] kbuild: merge cmd_ar_builtin and cmd_ar_module Masahiro Yamada

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210831074004.3195284-8-masahiroy@kernel.org \
    --to=masahiroy@kernel.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=michal.lkml@markovi.net \
    --cc=ndesaulniers@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.