From: Masahiro Yamada <masahiroy@kernel.org> To: linux-kbuild@vger.kernel.org Cc: linux-s390@vger.kernel.org, Nicolas Schier <nicolas@fjasle.eu>, Peter Zijlstra <peterz@infradead.org>, Masahiro Yamada <masahiroy@kernel.org>, Nick Desaulniers <ndesaulniers@google.com>, linux-kernel@vger.kernel.org, Nathan Chancellor <nathan@kernel.org>, clang-built-linux@googlegroups.com, Sami Tolvanen <samitolvanen@google.com>, linuxppc-dev@lists.ozlabs.org, Ard Biesheuvel <ardb@kernel.org>, linux-modules@vger.kernel.org Subject: [PATCH v4 11/14] kbuild: make built-in.a rule robust against too long argument error Date: Mon, 9 May 2022 04:06:28 +0900 [thread overview] Message-ID: <20220508190631.2386038-12-masahiroy@kernel.org> (raw) In-Reply-To: <20220508190631.2386038-1-masahiroy@kernel.org> Kbuild runs at the top of objtree instead of changing the working directory to subdirectories. I think this design is nice overall but some commands have a scalability issue. The build command of built-in.a is one of them whose length scales with: O(D * N) Here, D is the length of the directory path (i.e. $(obj)/ prefix), N is the number of objects in the Makefile, O() is the big O notation. The deeper directory the Makefile directory is located, the more easily it will hit the too long argument error. We can make it better. Trim the $(obj)/ by Make's builtin function, and restore it by a shell command (sed). With this, the command length scales with: O(D + N) In-tree modules still have some room to the limit (ARG_MAX=2097152), but this is more future-proof for big modules in a deep directory. For example, you can build i915 as builtin (CONFIG_DRM_I915=y) and compare drivers/gpu/drm/i915/.built-in.a.cmd with/without this commit. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu> Tested-by: Nathan Chancellor <nathan@kernel.org> --- (no changes since v2) Changes in v2: - New patch scripts/Makefile.build | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/Makefile.build b/scripts/Makefile.build index c2a173b3fd60..8f1a355df7aa 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -374,7 +374,10 @@ $(subdir-modorder): $(obj)/%/modules.order: $(obj)/% ; # quiet_cmd_ar_builtin = AR $@ - cmd_ar_builtin = rm -f $@; $(AR) cDPrST $@ $(real-prereqs) + cmd_ar_builtin = rm -f $@; \ + echo $(patsubst $(obj)/%,%,$(real-prereqs)) | \ + sed -E 's:([^ ]+):$(obj)/\1:g' | \ + xargs $(AR) cDPrST $@ $(obj)/built-in.a: $(real-obj-y) FORCE $(call if_changed,ar_builtin) -- 2.32.0
WARNING: multiple messages have this Message-ID (diff)
From: Masahiro Yamada <masahiroy@kernel.org> To: linux-kbuild@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Nathan Chancellor <nathan@kernel.org>, Nick Desaulniers <ndesaulniers@google.com>, Nicolas Schier <nicolas@fjasle.eu>, Peter Zijlstra <peterz@infradead.org>, linux-modules@vger.kernel.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, clang-built-linux@googlegroups.com, Ard Biesheuvel <ardb@kernel.org>, Sami Tolvanen <samitolvanen@google.com>, Masahiro Yamada <masahiroy@kernel.org> Subject: [PATCH v4 11/14] kbuild: make built-in.a rule robust against too long argument error Date: Mon, 9 May 2022 04:06:28 +0900 [thread overview] Message-ID: <20220508190631.2386038-12-masahiroy@kernel.org> (raw) In-Reply-To: <20220508190631.2386038-1-masahiroy@kernel.org> Kbuild runs at the top of objtree instead of changing the working directory to subdirectories. I think this design is nice overall but some commands have a scalability issue. The build command of built-in.a is one of them whose length scales with: O(D * N) Here, D is the length of the directory path (i.e. $(obj)/ prefix), N is the number of objects in the Makefile, O() is the big O notation. The deeper directory the Makefile directory is located, the more easily it will hit the too long argument error. We can make it better. Trim the $(obj)/ by Make's builtin function, and restore it by a shell command (sed). With this, the command length scales with: O(D + N) In-tree modules still have some room to the limit (ARG_MAX=2097152), but this is more future-proof for big modules in a deep directory. For example, you can build i915 as builtin (CONFIG_DRM_I915=y) and compare drivers/gpu/drm/i915/.built-in.a.cmd with/without this commit. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu> Tested-by: Nathan Chancellor <nathan@kernel.org> --- (no changes since v2) Changes in v2: - New patch scripts/Makefile.build | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/Makefile.build b/scripts/Makefile.build index c2a173b3fd60..8f1a355df7aa 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -374,7 +374,10 @@ $(subdir-modorder): $(obj)/%/modules.order: $(obj)/% ; # quiet_cmd_ar_builtin = AR $@ - cmd_ar_builtin = rm -f $@; $(AR) cDPrST $@ $(real-prereqs) + cmd_ar_builtin = rm -f $@; \ + echo $(patsubst $(obj)/%,%,$(real-prereqs)) | \ + sed -E 's:([^ ]+):$(obj)/\1:g' | \ + xargs $(AR) cDPrST $@ $(obj)/built-in.a: $(real-obj-y) FORCE $(call if_changed,ar_builtin) -- 2.32.0
next prev parent reply other threads:[~2022-05-08 19:16 UTC|newest] Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-05-08 19:06 [PATCH v4 00/14] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-08 19:06 ` [PATCH v4 01/14] modpost: remove left-over cross_compile declaration Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-09 17:08 ` Nick Desaulniers 2022-05-09 17:08 ` Nick Desaulniers 2022-05-12 4:35 ` Masahiro Yamada 2022-05-12 4:35 ` Masahiro Yamada 2022-05-08 19:06 ` [PATCH v4 02/14] modpost: change the license of EXPORT_SYMBOL to bool type Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-12 4:36 ` Masahiro Yamada 2022-05-12 4:36 ` Masahiro Yamada 2022-05-08 19:06 ` [PATCH v4 03/14] modpost: split the section mismatch checks into section-check.c Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-09 17:19 ` Nick Desaulniers 2022-05-09 17:19 ` Nick Desaulniers 2022-05-10 6:55 ` Masahiro Yamada 2022-05-10 6:55 ` Masahiro Yamada 2022-05-11 18:47 ` Nick Desaulniers 2022-05-11 18:47 ` Nick Desaulniers 2022-05-11 19:27 ` Masahiro Yamada 2022-05-11 19:27 ` Masahiro Yamada 2022-05-11 19:51 ` Jeff Johnson 2022-05-11 19:51 ` Jeff Johnson 2022-05-11 20:01 ` Nick Desaulniers 2022-05-11 20:01 ` Nick Desaulniers 2022-05-08 19:06 ` [PATCH v4 04/14] modpost: add sym_find_with_module() helper Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-08 19:06 ` [PATCH v4 05/14] modpost: extract symbol versions from *.cmd files Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-09 21:52 ` Sami Tolvanen 2022-05-09 21:52 ` Sami Tolvanen 2022-05-08 19:06 ` [PATCH v4 06/14] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-09 17:50 ` Nick Desaulniers 2022-05-09 17:50 ` Nick Desaulniers 2022-05-10 13:03 ` Masahiro Yamada 2022-05-10 13:03 ` Masahiro Yamada 2022-05-08 19:06 ` [PATCH v4 07/14] kbuild: stop merging *.symversions Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-08 19:06 ` [PATCH v4 08/14] genksyms: adjust the output format to modpost Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-08 19:06 ` [PATCH v4 09/14] kbuild: do not create *.prelink.o for Clang LTO or IBT Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-09 23:13 ` Sami Tolvanen 2022-05-09 23:13 ` Sami Tolvanen 2022-05-08 19:06 ` [PATCH v4 10/14] kbuild: check static EXPORT_SYMBOL* by script instead of modpost Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-09 18:05 ` Nick Desaulniers 2022-05-09 18:05 ` Nick Desaulniers 2022-05-10 13:53 ` Masahiro Yamada 2022-05-10 13:53 ` Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada [this message] 2022-05-08 19:06 ` [PATCH v4 11/14] kbuild: make built-in.a rule robust against too long argument error Masahiro Yamada 2022-05-08 19:06 ` [PATCH v4 12/14] kbuild: make *.mod " Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-08 19:06 ` [PATCH v4 13/14] kbuild: add cmd_and_savecmd macro Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-08 19:06 ` [PATCH v4 14/14] kbuild: rebuild multi-object modules when objtool is updated Masahiro Yamada 2022-05-08 19:06 ` Masahiro Yamada 2022-05-09 4:24 ` [PATCH v4 00/14] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS, export.h) Masahiro Yamada 2022-05-09 4:24 ` Masahiro Yamada 2022-05-09 22:13 ` Nathan Chancellor 2022-05-09 22:13 ` Nathan Chancellor 2022-05-10 6:55 ` Masahiro Yamada 2022-05-10 6:55 ` Masahiro Yamada 2022-05-09 22:19 ` Nathan Chancellor 2022-05-09 22:19 ` Nathan Chancellor
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220508190631.2386038-12-masahiroy@kernel.org \ --to=masahiroy@kernel.org \ --cc=ardb@kernel.org \ --cc=clang-built-linux@googlegroups.com \ --cc=linux-kbuild@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-modules@vger.kernel.org \ --cc=linux-s390@vger.kernel.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=nathan@kernel.org \ --cc=ndesaulniers@google.com \ --cc=nicolas@fjasle.eu \ --cc=peterz@infradead.org \ --cc=samitolvanen@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.