All of lore.kernel.org
 help / color / mirror / Atom feed
From: Nicolas Schier <nicolas@fjasle.eu>
To: Masahiro Yamada <masahiroy@kernel.org>
Cc: linux-kbuild@vger.kernel.org, clang-built-linux@googlegroups.com,
	linux-kernel@vger.kernel.org, Ard Biesheuvel <ardb@kernel.org>,
	Luis Chamberlain <mcgrof@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	linuxppc-dev@lists.ozlabs.org, linux-um@lists.infradead.org,
	linux-s390@vger.kernel.org,
	Nick Desaulniers <ndesaulniers@google.com>,
	Sami Tolvanen <samitolvanen@google.com>,
	Kees Cook <keescook@chromium.org>
Subject: Re: [PATCH v3 15/15] kbuild: make *.mod rule robust against too long argument error
Date: Thu, 5 May 2022 22:31:28 +0200	[thread overview]
Message-ID: <YnQ0IHNnQTIsxOxq@fjasle.eu> (raw)
In-Reply-To: <20220505072244.1155033-16-masahiroy@kernel.org>

On Thu, May 05, 2022 at 04:22:44PM +0900 Masahiro Yamada wrote:
> Like built-in.a, the command length of the *.mod rule scales with
> the depth of the directory times the number of objects in the Makefile.
> 
> Add $(obj)/ by the shell command (awk) instead of by Make's builtin
> function.
> 
> In-tree modules still have some room to the limit (ARG_MAX=2097152),
> but this is more future-proof for big modules in a deep directory.
> 
> For example, you can build i915 as a module (CONFIG_DRM_I915=m) and
> compare drivers/gpu/drm/i915/.i915.mod.cmd with/without this commit.
> 
> The issue is more critical for external modules because the M= path
> can be very long as Jeff Johnson reported before [1].
> 
> [1] https://lore.kernel.org/linux-kbuild/4c02050c4e95e4cb8cc04282695f8404@codeaurora.org/
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
> 
> (no changes since v2)
> 
> Changes in v2:
>   - New patch
> 
>  scripts/Makefile.build | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index cea48762299c..e7b3f329d443 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -266,8 +266,8 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE
>  	$(call if_changed_rule,cc_o_c)
>  	$(call cmd,force_checksrc)
>  
> -cmd_mod = echo $(addprefix $(obj)/, $(call real-search, $*.o, .o, -objs -y -m)) | \
> -	$(AWK) -v RS='( |\n)' '!x[$$0]++' > $@
> +cmd_mod = echo $(call real-search, $*.o, .o, -objs -y -m) | \
> +	$(AWK) -v RS='( |\n)' '!x[$$0]++ { print("$(obj)/"$$0) }' > $@
>  
>  $(obj)/%.mod: FORCE
>  	$(call if_changed,mod)
> -- 
> 2.32.0

Thanks!  At work, some colleagues of mine stumbled over that problem, too.

Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

WARNING: multiple messages have this Message-ID (diff)
From: Nicolas Schier <nicolas@fjasle.eu>
To: Masahiro Yamada <masahiroy@kernel.org>
Cc: linux-s390@vger.kernel.org, Kees Cook <keescook@chromium.org>,
	linux-kbuild@vger.kernel.org,
	Peter Zijlstra <peterz@infradead.org>,
	Nick Desaulniers <ndesaulniers@google.com>,
	linux-um@lists.infradead.org, linux-kernel@vger.kernel.org,
	clang-built-linux@googlegroups.com,
	Luis Chamberlain <mcgrof@kernel.org>,
	Sami Tolvanen <samitolvanen@google.com>,
	linuxppc-dev@lists.ozlabs.org, Ard Biesheuvel <ardb@kernel.org>
Subject: Re: [PATCH v3 15/15] kbuild: make *.mod rule robust against too long argument error
Date: Thu, 5 May 2022 22:31:28 +0200	[thread overview]
Message-ID: <YnQ0IHNnQTIsxOxq@fjasle.eu> (raw)
In-Reply-To: <20220505072244.1155033-16-masahiroy@kernel.org>

On Thu, May 05, 2022 at 04:22:44PM +0900 Masahiro Yamada wrote:
> Like built-in.a, the command length of the *.mod rule scales with
> the depth of the directory times the number of objects in the Makefile.
> 
> Add $(obj)/ by the shell command (awk) instead of by Make's builtin
> function.
> 
> In-tree modules still have some room to the limit (ARG_MAX=2097152),
> but this is more future-proof for big modules in a deep directory.
> 
> For example, you can build i915 as a module (CONFIG_DRM_I915=m) and
> compare drivers/gpu/drm/i915/.i915.mod.cmd with/without this commit.
> 
> The issue is more critical for external modules because the M= path
> can be very long as Jeff Johnson reported before [1].
> 
> [1] https://lore.kernel.org/linux-kbuild/4c02050c4e95e4cb8cc04282695f8404@codeaurora.org/
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
> 
> (no changes since v2)
> 
> Changes in v2:
>   - New patch
> 
>  scripts/Makefile.build | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index cea48762299c..e7b3f329d443 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -266,8 +266,8 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE
>  	$(call if_changed_rule,cc_o_c)
>  	$(call cmd,force_checksrc)
>  
> -cmd_mod = echo $(addprefix $(obj)/, $(call real-search, $*.o, .o, -objs -y -m)) | \
> -	$(AWK) -v RS='( |\n)' '!x[$$0]++' > $@
> +cmd_mod = echo $(call real-search, $*.o, .o, -objs -y -m) | \
> +	$(AWK) -v RS='( |\n)' '!x[$$0]++ { print("$(obj)/"$$0) }' > $@
>  
>  $(obj)/%.mod: FORCE
>  	$(call if_changed,mod)
> -- 
> 2.32.0

Thanks!  At work, some colleagues of mine stumbled over that problem, too.

Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

WARNING: multiple messages have this Message-ID (diff)
From: Nicolas Schier <nicolas@fjasle.eu>
To: Masahiro Yamada <masahiroy@kernel.org>
Cc: linux-kbuild@vger.kernel.org, clang-built-linux@googlegroups.com,
	linux-kernel@vger.kernel.org, Ard Biesheuvel <ardb@kernel.org>,
	Luis Chamberlain <mcgrof@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	linuxppc-dev@lists.ozlabs.org, linux-um@lists.infradead.org,
	linux-s390@vger.kernel.org,
	Nick Desaulniers <ndesaulniers@google.com>,
	Sami Tolvanen <samitolvanen@google.com>,
	Kees Cook <keescook@chromium.org>
Subject: Re: [PATCH v3 15/15] kbuild: make *.mod rule robust against too long argument error
Date: Thu, 5 May 2022 22:31:28 +0200	[thread overview]
Message-ID: <YnQ0IHNnQTIsxOxq@fjasle.eu> (raw)
In-Reply-To: <20220505072244.1155033-16-masahiroy@kernel.org>

On Thu, May 05, 2022 at 04:22:44PM +0900 Masahiro Yamada wrote:
> Like built-in.a, the command length of the *.mod rule scales with
> the depth of the directory times the number of objects in the Makefile.
> 
> Add $(obj)/ by the shell command (awk) instead of by Make's builtin
> function.
> 
> In-tree modules still have some room to the limit (ARG_MAX=2097152),
> but this is more future-proof for big modules in a deep directory.
> 
> For example, you can build i915 as a module (CONFIG_DRM_I915=m) and
> compare drivers/gpu/drm/i915/.i915.mod.cmd with/without this commit.
> 
> The issue is more critical for external modules because the M= path
> can be very long as Jeff Johnson reported before [1].
> 
> [1] https://lore.kernel.org/linux-kbuild/4c02050c4e95e4cb8cc04282695f8404@codeaurora.org/
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
> 
> (no changes since v2)
> 
> Changes in v2:
>   - New patch
> 
>  scripts/Makefile.build | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/Makefile.build b/scripts/Makefile.build
> index cea48762299c..e7b3f329d443 100644
> --- a/scripts/Makefile.build
> +++ b/scripts/Makefile.build
> @@ -266,8 +266,8 @@ $(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE
>  	$(call if_changed_rule,cc_o_c)
>  	$(call cmd,force_checksrc)
>  
> -cmd_mod = echo $(addprefix $(obj)/, $(call real-search, $*.o, .o, -objs -y -m)) | \
> -	$(AWK) -v RS='( |\n)' '!x[$$0]++' > $@
> +cmd_mod = echo $(call real-search, $*.o, .o, -objs -y -m) | \
> +	$(AWK) -v RS='( |\n)' '!x[$$0]++ { print("$(obj)/"$$0) }' > $@
>  
>  $(obj)/%.mod: FORCE
>  	$(call if_changed,mod)
> -- 
> 2.32.0

Thanks!  At work, some colleagues of mine stumbled over that problem, too.

Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>

_______________________________________________
linux-um mailing list
linux-um@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-um


  reply	other threads:[~2022-05-05 20:38 UTC|newest]

Thread overview: 93+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-05-05  7:22 [PATCH v3 00/15] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS) Masahiro Yamada
2022-05-05  7:22 ` Masahiro Yamada
2022-05-05  7:22 ` Masahiro Yamada
2022-05-05  7:22 ` [PATCH v3 01/15] modpost: mitigate false-negatives for static EXPORT_SYMBOL checks Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05 19:25   ` Nicolas Schier
2022-05-05 19:25     ` Nicolas Schier
2022-05-05 19:25     ` Nicolas Schier
2022-05-05  7:22 ` [PATCH v3 02/15] modpost: change the license of EXPORT_SYMBOL to bool type Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05 13:48   ` Masahiro Yamada
2022-05-05 13:48     ` Masahiro Yamada
2022-05-05 13:48     ` Masahiro Yamada
2022-05-05 19:53     ` Nicolas Schier
2022-05-05 19:53       ` Nicolas Schier
2022-05-05 19:53       ` Nicolas Schier
2022-05-05  7:22 ` [PATCH v3 03/15] modpost: merge add_{intree_flag,retpoline,staging_flag} to add_header Masahiro Yamada
2022-05-05  7:22   ` [PATCH v3 03/15] modpost: merge add_{intree_flag, retpoline, staging_flag} " Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05 19:58   ` [PATCH v3 03/15] modpost: merge add_{intree_flag,retpoline,staging_flag} " Nicolas Schier
2022-05-05 19:58     ` Nicolas Schier
2022-05-05 19:58     ` Nicolas Schier
2022-05-05  7:22 ` [PATCH v3 04/15] modpost: move *.mod.c generation to write_mod_c_files() Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05 20:06   ` Nicolas Schier
2022-05-05 20:06     ` Nicolas Schier
2022-05-05 20:06     ` Nicolas Schier
2022-05-05  7:22 ` [PATCH v3 05/15] kbuild: generate a list of objects in vmlinux Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22 ` [PATCH v3 06/15] kbuild: record symbol versions in *.cmd files Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22 ` [PATCH v3 07/15] modpost: extract symbol versions from " Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05 20:09   ` Nicolas Schier
2022-05-05 20:09     ` Nicolas Schier
2022-05-05 20:09     ` Nicolas Schier
2022-05-05  7:22 ` [PATCH v3 08/15] kbuild: link symbol CRCs at final link, removing CONFIG_MODULE_REL_CRCS Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22 ` [PATCH v3 09/15] kbuild: stop merging *.symversions Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05 20:19   ` Nicolas Schier
2022-05-05 20:19     ` Nicolas Schier
2022-05-05 20:19     ` Nicolas Schier
2022-05-05  7:22 ` [PATCH v3 10/15] genksyms: adjust the output format to modpost Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05 20:22   ` Nicolas Schier
2022-05-05 20:22     ` Nicolas Schier
2022-05-05 20:22     ` Nicolas Schier
2022-05-05  7:22 ` [PATCH v3 11/15] kbuild: do not create *.prelink.o for Clang LTO or IBT Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22 ` [PATCH v3 12/15] modpost: simplify the ->is_static initialization Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05 20:27   ` Nicolas Schier
2022-05-05 20:27     ` Nicolas Schier
2022-05-05 20:27     ` Nicolas Schier
2022-05-05  7:22 ` [PATCH v3 13/15] modpost: use hlist for hash table implementation Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22 ` [PATCH v3 14/15] kbuild: make built-in.a rule robust against too long argument error Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05 20:29   ` Nicolas Schier
2022-05-05 20:29     ` Nicolas Schier
2022-05-05 20:29     ` Nicolas Schier
2022-05-05 20:31   ` Nick Desaulniers
2022-05-05 20:31     ` Nick Desaulniers
2022-05-05 20:31     ` Nick Desaulniers
2022-05-05  7:22 ` [PATCH v3 15/15] kbuild: make *.mod " Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05  7:22   ` Masahiro Yamada
2022-05-05 20:31   ` Nicolas Schier [this message]
2022-05-05 20:31     ` Nicolas Schier
2022-05-05 20:31     ` Nicolas Schier
2022-05-05 16:49 ` [PATCH v3 00/15] kbuild: yet another series of cleanups (modpost, LTO, MODULE_REL_CRCS) Masahiro Yamada
2022-05-05 16:49   ` Masahiro Yamada
2022-05-05 16:49   ` Masahiro Yamada
2022-05-06 22:45 ` Nathan Chancellor
2022-05-06 22:45   ` Nathan Chancellor
2022-05-06 22:45   ` Nathan Chancellor
2022-05-08 18:28 ` Masahiro Yamada
2022-05-08 18:28   ` Masahiro Yamada
2022-05-08 18:28   ` Masahiro Yamada

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YnQ0IHNnQTIsxOxq@fjasle.eu \
    --to=nicolas@fjasle.eu \
    --cc=ardb@kernel.org \
    --cc=clang-built-linux@googlegroups.com \
    --cc=keescook@chromium.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-s390@vger.kernel.org \
    --cc=linux-um@lists.infradead.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=masahiroy@kernel.org \
    --cc=mcgrof@kernel.org \
    --cc=ndesaulniers@google.com \
    --cc=peterz@infradead.org \
    --cc=samitolvanen@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.