* [PATCH 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory
@ 2021-07-12 11:10 Tor Vic
2021-07-12 17:22 ` Kees Cook
2021-07-13 10:45 ` Masahiro Yamada
0 siblings, 2 replies; 6+ messages in thread
From: Tor Vic @ 2021-07-12 11:10 UTC (permalink / raw)
To: linux-kernel, masahiroy, Nathan Chancellor, ndesaulniers, keescook
Cc: linux-kbuild, clang-built-linux
On some distros and configurations, it might be useful to allow for
specifying a directory where Clang stores its ThinLTO cache.
More specifically, when building the VirtualBox extramodules on Arch with
its proper 'makepkg' build system and DKMS, against an already installed
ThinLTO kernel, the build fails because it tries to create the ThinLTO
cache in a directory that is not user-writable.
A similar problem has been reported with openSUSE's OBS build system.
Add a Kconfig option that allows users to choose a directory in which
Clang's ThinLTO can store its cache.
Link: https://github.com/ClangBuiltLinux/linux/issues/1104
Signed-off-by: Tor Vic <torvic9@mailbox.org>
---
Makefile | 5 +++--
arch/Kconfig | 10 ++++++++++
scripts/Makefile.lib | 4 ++++
scripts/Makefile.modfinal | 4 ++++
4 files changed, 21 insertions(+), 2 deletions(-)
diff --git a/Makefile b/Makefile
index c3f9bd191b89..472bc8bfff03 100644
--- a/Makefile
+++ b/Makefile
@@ -932,7 +932,8 @@ endif
ifdef CONFIG_LTO_CLANG
ifdef CONFIG_LTO_CLANG_THIN
CC_FLAGS_LTO := -flto=thin -fsplit-lto-unit
-KBUILD_LDFLAGS += --thinlto-cache-dir=$(extmod_prefix).thinlto-cache
+export thinlto-dir = $(if
$(CONFIG_LTO_CLANG_THIN_CACHEDIR),$(CONFIG_LTO_CLANG_THIN_CACHEDIR)/)
+KBUILD_LDFLAGS +=
--thinlto-cache-dir=$(thinlto-dir)$(extmod_prefix).thinlto-cache
else
CC_FLAGS_LTO := -flto
endif
@@ -1728,7 +1729,7 @@ PHONY += compile_commands.json
clean-dirs := $(KBUILD_EXTMOD)
clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers
$(KBUILD_EXTMOD)/modules.nsdeps \
- $(KBUILD_EXTMOD)/compile_commands.json $(KBUILD_EXTMOD)/.thinlto-cache
+ $(KBUILD_EXTMOD)/compile_commands.json
$(thinlto-dir)$(KBUILD_EXTMOD)/.thinlto-cache
PHONY += help
help:
diff --git a/arch/Kconfig b/arch/Kconfig
index 129df498a8e1..19e4d140e12a 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -696,6 +696,16 @@ config LTO_CLANG_THIN
https://clang.llvm.org/docs/ThinLTO.html
If unsure, say Y.
+
+config LTO_CLANG_THIN_CACHEDIR
+ string "Clang ThinLTO cache directory"
+ depends on LTO_CLANG_THIN
+ default ""
+ help
+ This option allows users to choose a directory that stores
+ Clang's ThinLTO cache.
+ Leave empty for default.
+
endchoice
config ARCH_SUPPORTS_CFI_CLANG
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 10950559b223..bca87a6aa35b 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -197,6 +197,10 @@ endif
part-of-module = $(if $(filter $(basename $@).o, $(real-obj-m)),y)
quiet_modtag = $(if $(part-of-module),[M], )
+ifdef CONFIG_LTO_CLANG_THIN
+KBUILD_LDFLAGS +=
--thinlto-cache-dir=$(thinlto-dir)$(extmod-prefix).thinlto-cache
+endif
+
modkern_cflags = \
$(if $(part-of-module), \
$(KBUILD_CFLAGS_MODULE) $(CFLAGS_MODULE), \
diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
index 5e9b8057fb24..ab0d72e21318 100644
--- a/scripts/Makefile.modfinal
+++ b/scripts/Makefile.modfinal
@@ -35,6 +35,10 @@ ifdef CONFIG_LTO_CLANG
# avoid a second slow LTO link
prelink-ext := .lto
+ifdef CONFIG_LTO_CLANG_THIN
+KBUILD_LDFLAGS +=
--thinlto-cache-dir=$(thinlto-dir)$(extmod-prefix).thinlto-cache
+endif # CONFIG_LTO_CLANG_THIN
+
# ELF processing was skipped earlier because we didn't have native code,
# so let's now process the prelinked binary before we link the module.
--
2.32.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory
2021-07-12 11:10 [PATCH 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory Tor Vic
@ 2021-07-12 17:22 ` Kees Cook
2021-07-13 11:52 ` torvic9
2021-07-13 10:45 ` Masahiro Yamada
1 sibling, 1 reply; 6+ messages in thread
From: Kees Cook @ 2021-07-12 17:22 UTC (permalink / raw)
To: Tor Vic
Cc: linux-kernel, masahiroy, Nathan Chancellor, ndesaulniers,
linux-kbuild, clang-built-linux
On Mon, Jul 12, 2021 at 11:10:04AM +0000, Tor Vic wrote:
> On some distros and configurations, it might be useful to allow for
> specifying a directory where Clang stores its ThinLTO cache.
>
> More specifically, when building the VirtualBox extramodules on Arch with
> its proper 'makepkg' build system and DKMS, against an already installed
> ThinLTO kernel, the build fails because it tries to create the ThinLTO
> cache in a directory that is not user-writable.
>
> A similar problem has been reported with openSUSE's OBS build system.
>
> Add a Kconfig option that allows users to choose a directory in which
> Clang's ThinLTO can store its cache.
Ah-ha, good idea. Thanks! Question below...
>
> Link: https://github.com/ClangBuiltLinux/linux/issues/1104
> Signed-off-by: Tor Vic <torvic9@mailbox.org>
> ---
> Makefile | 5 +++--
> arch/Kconfig | 10 ++++++++++
> scripts/Makefile.lib | 4 ++++
> scripts/Makefile.modfinal | 4 ++++
> 4 files changed, 21 insertions(+), 2 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index c3f9bd191b89..472bc8bfff03 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -932,7 +932,8 @@ endif
> ifdef CONFIG_LTO_CLANG
> ifdef CONFIG_LTO_CLANG_THIN
> CC_FLAGS_LTO := -flto=thin -fsplit-lto-unit
> -KBUILD_LDFLAGS += --thinlto-cache-dir=$(extmod_prefix).thinlto-cache
> +export thinlto-dir = $(if
> $(CONFIG_LTO_CLANG_THIN_CACHEDIR),$(CONFIG_LTO_CLANG_THIN_CACHEDIR)/)
> +KBUILD_LDFLAGS +=
> --thinlto-cache-dir=$(thinlto-dir)$(extmod_prefix).thinlto-cache
> else
> CC_FLAGS_LTO := -flto
> endif
> @@ -1728,7 +1729,7 @@ PHONY += compile_commands.json
>
> clean-dirs := $(KBUILD_EXTMOD)
> clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers
> $(KBUILD_EXTMOD)/modules.nsdeps \
> - $(KBUILD_EXTMOD)/compile_commands.json $(KBUILD_EXTMOD)/.thinlto-cache
> + $(KBUILD_EXTMOD)/compile_commands.json
> $(thinlto-dir)$(KBUILD_EXTMOD)/.thinlto-cache
>
> PHONY += help
> help:
> diff --git a/arch/Kconfig b/arch/Kconfig
> index 129df498a8e1..19e4d140e12a 100644
> --- a/arch/Kconfig
> +++ b/arch/Kconfig
> @@ -696,6 +696,16 @@ config LTO_CLANG_THIN
> https://clang.llvm.org/docs/ThinLTO.html
>
> If unsure, say Y.
> +
> +config LTO_CLANG_THIN_CACHEDIR
> + string "Clang ThinLTO cache directory"
> + depends on LTO_CLANG_THIN
> + default ""
> + help
> + This option allows users to choose a directory that stores
> + Clang's ThinLTO cache.
> + Leave empty for default.
> +
> endchoice
>
> config ARCH_SUPPORTS_CFI_CLANG
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 10950559b223..bca87a6aa35b 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -197,6 +197,10 @@ endif
> part-of-module = $(if $(filter $(basename $@).o, $(real-obj-m)),y)
> quiet_modtag = $(if $(part-of-module),[M], )
>
> +ifdef CONFIG_LTO_CLANG_THIN
> +KBUILD_LDFLAGS +=
> --thinlto-cache-dir=$(thinlto-dir)$(extmod-prefix).thinlto-cache
> +endif
> +
> modkern_cflags = \
> $(if $(part-of-module), \
> $(KBUILD_CFLAGS_MODULE) $(CFLAGS_MODULE), \
> diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
> index 5e9b8057fb24..ab0d72e21318 100644
> --- a/scripts/Makefile.modfinal
> +++ b/scripts/Makefile.modfinal
> @@ -35,6 +35,10 @@ ifdef CONFIG_LTO_CLANG
> # avoid a second slow LTO link
> prelink-ext := .lto
>
> +ifdef CONFIG_LTO_CLANG_THIN
> +KBUILD_LDFLAGS +=
> --thinlto-cache-dir=$(thinlto-dir)$(extmod-prefix).thinlto-cache
> +endif # CONFIG_LTO_CLANG_THIN
> +
> # ELF processing was skipped earlier because we didn't have native code,
> # so let's now process the prelinked binary before we link the module.
Why are these changes needed in Makefile.lib and Makefile.modfinal?
Isn't KBUILD_LDFLAGS already populated from the top-level Makefile?
--
Kees Cook
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory
2021-07-12 11:10 [PATCH 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory Tor Vic
2021-07-12 17:22 ` Kees Cook
@ 2021-07-13 10:45 ` Masahiro Yamada
2021-07-13 12:38 ` torvic9
2021-07-13 12:43 ` torvic9
1 sibling, 2 replies; 6+ messages in thread
From: Masahiro Yamada @ 2021-07-13 10:45 UTC (permalink / raw)
To: Tor Vic
Cc: linux-kernel, Nathan Chancellor, ndesaulniers, Kees Cook,
Linux Kbuild mailing list, clang-built-linux
On Mon, Jul 12, 2021 at 8:10 PM Tor Vic <torvic9@mailbox.org> wrote:
>
> On some distros and configurations, it might be useful to allow for
> specifying a directory where Clang stores its ThinLTO cache.
>
> More specifically, when building the VirtualBox extramodules on Arch with
> its proper 'makepkg' build system and DKMS, against an already installed
> ThinLTO kernel, the build fails because it tries to create the ThinLTO
> cache in a directory that is not user-writable.
>
> A similar problem has been reported with openSUSE's OBS build system.
>
> Add a Kconfig option that allows users to choose a directory in which
> Clang's ThinLTO can store its cache.
>
> Link: https://github.com/ClangBuiltLinux/linux/issues/1104
> Signed-off-by: Tor Vic <torvic9@mailbox.org>
I do not understand.
Currently, .thinlto-cache is created in the same directory
as the other objects. (right under $(KBUILD_EXTMOD))
If you build in a read-only directory, you cannot put
any build artifact (not limited to the thinlto cache) there.
Why did changing the location of .thinlto-cache
solve your problem?
> ---
> Makefile | 5 +++--
> arch/Kconfig | 10 ++++++++++
> scripts/Makefile.lib | 4 ++++
> scripts/Makefile.modfinal | 4 ++++
> 4 files changed, 21 insertions(+), 2 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index c3f9bd191b89..472bc8bfff03 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -932,7 +932,8 @@ endif
> ifdef CONFIG_LTO_CLANG
> ifdef CONFIG_LTO_CLANG_THIN
> CC_FLAGS_LTO := -flto=thin -fsplit-lto-unit
> -KBUILD_LDFLAGS += --thinlto-cache-dir=$(extmod_prefix).thinlto-cache
> +export thinlto-dir = $(if
> $(CONFIG_LTO_CLANG_THIN_CACHEDIR),$(CONFIG_LTO_CLANG_THIN_CACHEDIR)/)
> +KBUILD_LDFLAGS +=
> --thinlto-cache-dir=$(thinlto-dir)$(extmod_prefix).thinlto-cache
> else
> CC_FLAGS_LTO := -flto
> endif
> @@ -1728,7 +1729,7 @@ PHONY += compile_commands.json
>
> clean-dirs := $(KBUILD_EXTMOD)
> clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers
> $(KBUILD_EXTMOD)/modules.nsdeps \
> - $(KBUILD_EXTMOD)/compile_commands.json $(KBUILD_EXTMOD)/.thinlto-cache
> + $(KBUILD_EXTMOD)/compile_commands.json
> $(thinlto-dir)$(KBUILD_EXTMOD)/.thinlto-cache
>
> PHONY += help
> help:
> diff --git a/arch/Kconfig b/arch/Kconfig
> index 129df498a8e1..19e4d140e12a 100644
> --- a/arch/Kconfig
> +++ b/arch/Kconfig
> @@ -696,6 +696,16 @@ config LTO_CLANG_THIN
> https://clang.llvm.org/docs/ThinLTO.html
>
> If unsure, say Y.
> +
> +config LTO_CLANG_THIN_CACHEDIR
> + string "Clang ThinLTO cache directory"
> + depends on LTO_CLANG_THIN
> + default ""
> + help
> + This option allows users to choose a directory that stores
> + Clang's ThinLTO cache.
> + Leave empty for default.
> +
> endchoice
>
> config ARCH_SUPPORTS_CFI_CLANG
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index 10950559b223..bca87a6aa35b 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -197,6 +197,10 @@ endif
> part-of-module = $(if $(filter $(basename $@).o, $(real-obj-m)),y)
> quiet_modtag = $(if $(part-of-module),[M], )
>
> +ifdef CONFIG_LTO_CLANG_THIN
> +KBUILD_LDFLAGS +=
> --thinlto-cache-dir=$(thinlto-dir)$(extmod-prefix).thinlto-cache
> +endif
> +
> modkern_cflags = \
> $(if $(part-of-module), \
> $(KBUILD_CFLAGS_MODULE) $(CFLAGS_MODULE), \
> diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
> index 5e9b8057fb24..ab0d72e21318 100644
> --- a/scripts/Makefile.modfinal
> +++ b/scripts/Makefile.modfinal
> @@ -35,6 +35,10 @@ ifdef CONFIG_LTO_CLANG
> # avoid a second slow LTO link
> prelink-ext := .lto
>
> +ifdef CONFIG_LTO_CLANG_THIN
> +KBUILD_LDFLAGS +=
> --thinlto-cache-dir=$(thinlto-dir)$(extmod-prefix).thinlto-cache
> +endif # CONFIG_LTO_CLANG_THIN
> +
> # ELF processing was skipped earlier because we didn't have native code,
> # so let's now process the prelinked binary before we link the module.
>
> --
> 2.32.0
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory
2021-07-12 17:22 ` Kees Cook
@ 2021-07-13 11:52 ` torvic9
0 siblings, 0 replies; 6+ messages in thread
From: torvic9 @ 2021-07-13 11:52 UTC (permalink / raw)
To: Kees Cook
Cc: linux-kernel, masahiroy, Nathan Chancellor, ndesaulniers,
linux-kbuild, clang-built-linux
> Kees Cook <keescook@chromium.org> hat am 12.07.2021 19:22 geschrieben:
>
>
> On Mon, Jul 12, 2021 at 11:10:04AM +0000, Tor Vic wrote:
> > On some distros and configurations, it might be useful to allow for
> > specifying a directory where Clang stores its ThinLTO cache.
> >
> > More specifically, when building the VirtualBox extramodules on Arch with
> > its proper 'makepkg' build system and DKMS, against an already installed
> > ThinLTO kernel, the build fails because it tries to create the ThinLTO
> > cache in a directory that is not user-writable.
> >
> > A similar problem has been reported with openSUSE's OBS build system.
> >
> > Add a Kconfig option that allows users to choose a directory in which
> > Clang's ThinLTO can store its cache.
>
> Ah-ha, good idea. Thanks! Question below...
>
> >
> > Link: https://github.com/ClangBuiltLinux/linux/issues/1104
> > Signed-off-by: Tor Vic <torvic9@mailbox.org>
> > ---
> > Makefile | 5 +++--
> > arch/Kconfig | 10 ++++++++++
> > scripts/Makefile.lib | 4 ++++
> > scripts/Makefile.modfinal | 4 ++++
> > 4 files changed, 21 insertions(+), 2 deletions(-)
> >
> > diff --git a/Makefile b/Makefile
> > index c3f9bd191b89..472bc8bfff03 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -932,7 +932,8 @@ endif
> > ifdef CONFIG_LTO_CLANG
> > ifdef CONFIG_LTO_CLANG_THIN
> > CC_FLAGS_LTO := -flto=thin -fsplit-lto-unit
> > -KBUILD_LDFLAGS += --thinlto-cache-dir=$(extmod_prefix).thinlto-cache
> > +export thinlto-dir = $(if
> > $(CONFIG_LTO_CLANG_THIN_CACHEDIR),$(CONFIG_LTO_CLANG_THIN_CACHEDIR)/)
> > +KBUILD_LDFLAGS +=
> > --thinlto-cache-dir=$(thinlto-dir)$(extmod_prefix).thinlto-cache
> > else
> > CC_FLAGS_LTO := -flto
> > endif
> > @@ -1728,7 +1729,7 @@ PHONY += compile_commands.json
> >
> > clean-dirs := $(KBUILD_EXTMOD)
> > clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers
> > $(KBUILD_EXTMOD)/modules.nsdeps \
> > - $(KBUILD_EXTMOD)/compile_commands.json $(KBUILD_EXTMOD)/.thinlto-cache
> > + $(KBUILD_EXTMOD)/compile_commands.json
> > $(thinlto-dir)$(KBUILD_EXTMOD)/.thinlto-cache
> >
> > PHONY += help
> > help:
> > diff --git a/arch/Kconfig b/arch/Kconfig
> > index 129df498a8e1..19e4d140e12a 100644
> > --- a/arch/Kconfig
> > +++ b/arch/Kconfig
> > @@ -696,6 +696,16 @@ config LTO_CLANG_THIN
> > https://clang.llvm.org/docs/ThinLTO.html
> >
> > If unsure, say Y.
> > +
> > +config LTO_CLANG_THIN_CACHEDIR
> > + string "Clang ThinLTO cache directory"
> > + depends on LTO_CLANG_THIN
> > + default ""
> > + help
> > + This option allows users to choose a directory that stores
> > + Clang's ThinLTO cache.
> > + Leave empty for default.
> > +
> > endchoice
> >
> > config ARCH_SUPPORTS_CFI_CLANG
> > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > index 10950559b223..bca87a6aa35b 100644
> > --- a/scripts/Makefile.lib
> > +++ b/scripts/Makefile.lib
> > @@ -197,6 +197,10 @@ endif
> > part-of-module = $(if $(filter $(basename $@).o, $(real-obj-m)),y)
> > quiet_modtag = $(if $(part-of-module),[M], )
> >
> > +ifdef CONFIG_LTO_CLANG_THIN
> > +KBUILD_LDFLAGS +=
> > --thinlto-cache-dir=$(thinlto-dir)$(extmod-prefix).thinlto-cache
> > +endif
> > +
> > modkern_cflags = \
> > $(if $(part-of-module), \
> > $(KBUILD_CFLAGS_MODULE) $(CFLAGS_MODULE), \
> > diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
> > index 5e9b8057fb24..ab0d72e21318 100644
> > --- a/scripts/Makefile.modfinal
> > +++ b/scripts/Makefile.modfinal
> > @@ -35,6 +35,10 @@ ifdef CONFIG_LTO_CLANG
> > # avoid a second slow LTO link
> > prelink-ext := .lto
> >
> > +ifdef CONFIG_LTO_CLANG_THIN
> > +KBUILD_LDFLAGS +=
> > --thinlto-cache-dir=$(thinlto-dir)$(extmod-prefix).thinlto-cache
> > +endif # CONFIG_LTO_CLANG_THIN
> > +
> > # ELF processing was skipped earlier because we didn't have native code,
> > # so let's now process the prelinked binary before we link the module.
>
> Why are these changes needed in Makefile.lib and Makefile.modfinal?
> Isn't KBUILD_LDFLAGS already populated from the top-level Makefile?
Hi Kees,
I think you are right.
It seems that the changes to scripts/Makefile.{lib,modfinal} are not needed.
I'll do some more testing and report back/send a v2.
>
> --
> Kees Cook
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory
2021-07-13 10:45 ` Masahiro Yamada
@ 2021-07-13 12:38 ` torvic9
2021-07-13 12:43 ` torvic9
1 sibling, 0 replies; 6+ messages in thread
From: torvic9 @ 2021-07-13 12:38 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kernel, Nathan Chancellor, ndesaulniers, Kees Cook,
Linux Kbuild mailing list, clang-built-linux
> Masahiro Yamada <masahiroy@kernel.org> hat am 13.07.2021 12:45 geschrieben:
>
>
> On Mon, Jul 12, 2021 at 8:10 PM Tor Vic <torvic9@mailbox.org> wrote:
> >
> > On some distros and configurations, it might be useful to allow for
> > specifying a directory where Clang stores its ThinLTO cache.
> >
> > More specifically, when building the VirtualBox extramodules on Arch with
> > its proper 'makepkg' build system and DKMS, against an already installed
> > ThinLTO kernel, the build fails because it tries to create the ThinLTO
> > cache in a directory that is not user-writable.
> >
> > A similar problem has been reported with openSUSE's OBS build system.
> >
> > Add a Kconfig option that allows users to choose a directory in which
> > Clang's ThinLTO can store its cache.
> >
> > Link: https://github.com/ClangBuiltLinux/linux/issues/1104
> > Signed-off-by: Tor Vic <torvic9@mailbox.org>
>
>
> I do not understand.
>
> Currently, .thinlto-cache is created in the same directory
> as the other objects. (right under $(KBUILD_EXTMOD))
>
> If you build in a read-only directory, you cannot put
> any build artifact (not limited to the thinlto cache) there.
>
> Why did changing the location of .thinlto-cache
> solve your problem?
>
I'm still not 100% what is going on, but, on Arch using DKMS,
I get the following error message:
Error: Permission denied
LLVM ERROR: ThinLTO: Can't get a temporary file
>
>
> > ---
> > Makefile | 5 +++--
> > arch/Kconfig | 10 ++++++++++
> > scripts/Makefile.lib | 4 ++++
> > scripts/Makefile.modfinal | 4 ++++
> > 4 files changed, 21 insertions(+), 2 deletions(-)
> >
> > diff --git a/Makefile b/Makefile
> > index c3f9bd191b89..472bc8bfff03 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -932,7 +932,8 @@ endif
> > ifdef CONFIG_LTO_CLANG
> > ifdef CONFIG_LTO_CLANG_THIN
> > CC_FLAGS_LTO := -flto=thin -fsplit-lto-unit
> > -KBUILD_LDFLAGS += --thinlto-cache-dir=$(extmod_prefix).thinlto-cache
> > +export thinlto-dir = $(if
> > $(CONFIG_LTO_CLANG_THIN_CACHEDIR),$(CONFIG_LTO_CLANG_THIN_CACHEDIR)/)
> > +KBUILD_LDFLAGS +=
> > --thinlto-cache-dir=$(thinlto-dir)$(extmod_prefix).thinlto-cache
> > else
> > CC_FLAGS_LTO := -flto
> > endif
> > @@ -1728,7 +1729,7 @@ PHONY += compile_commands.json
> >
> > clean-dirs := $(KBUILD_EXTMOD)
> > clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers
> > $(KBUILD_EXTMOD)/modules.nsdeps \
> > - $(KBUILD_EXTMOD)/compile_commands.json $(KBUILD_EXTMOD)/.thinlto-cache
> > + $(KBUILD_EXTMOD)/compile_commands.json
> > $(thinlto-dir)$(KBUILD_EXTMOD)/.thinlto-cache
> >
> > PHONY += help
> > help:
> > diff --git a/arch/Kconfig b/arch/Kconfig
> > index 129df498a8e1..19e4d140e12a 100644
> > --- a/arch/Kconfig
> > +++ b/arch/Kconfig
> > @@ -696,6 +696,16 @@ config LTO_CLANG_THIN
> > https://clang.llvm.org/docs/ThinLTO.html
> >
> > If unsure, say Y.
> > +
> > +config LTO_CLANG_THIN_CACHEDIR
> > + string "Clang ThinLTO cache directory"
> > + depends on LTO_CLANG_THIN
> > + default ""
> > + help
> > + This option allows users to choose a directory that stores
> > + Clang's ThinLTO cache.
> > + Leave empty for default.
> > +
> > endchoice
> >
> > config ARCH_SUPPORTS_CFI_CLANG
> > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > index 10950559b223..bca87a6aa35b 100644
> > --- a/scripts/Makefile.lib
> > +++ b/scripts/Makefile.lib
> > @@ -197,6 +197,10 @@ endif
> > part-of-module = $(if $(filter $(basename $@).o, $(real-obj-m)),y)
> > quiet_modtag = $(if $(part-of-module),[M], )
> >
> > +ifdef CONFIG_LTO_CLANG_THIN
> > +KBUILD_LDFLAGS +=
> > --thinlto-cache-dir=$(thinlto-dir)$(extmod-prefix).thinlto-cache
> > +endif
> > +
> > modkern_cflags = \
> > $(if $(part-of-module), \
> > $(KBUILD_CFLAGS_MODULE) $(CFLAGS_MODULE), \
> > diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
> > index 5e9b8057fb24..ab0d72e21318 100644
> > --- a/scripts/Makefile.modfinal
> > +++ b/scripts/Makefile.modfinal
> > @@ -35,6 +35,10 @@ ifdef CONFIG_LTO_CLANG
> > # avoid a second slow LTO link
> > prelink-ext := .lto
> >
> > +ifdef CONFIG_LTO_CLANG_THIN
> > +KBUILD_LDFLAGS +=
> > --thinlto-cache-dir=$(thinlto-dir)$(extmod-prefix).thinlto-cache
> > +endif # CONFIG_LTO_CLANG_THIN
> > +
> > # ELF processing was skipped earlier because we didn't have native code,
> > # so let's now process the prelinked binary before we link the module.
> >
> > --
> > 2.32.0
>
>
>
> --
> Best Regards
> Masahiro Yamada
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory
2021-07-13 10:45 ` Masahiro Yamada
2021-07-13 12:38 ` torvic9
@ 2021-07-13 12:43 ` torvic9
1 sibling, 0 replies; 6+ messages in thread
From: torvic9 @ 2021-07-13 12:43 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kernel, Nathan Chancellor, ndesaulniers, Kees Cook,
Linux Kbuild mailing list, clang-built-linux
> Masahiro Yamada <masahiroy@kernel.org> hat am 13.07.2021 12:45 geschrieben:
>
>
> On Mon, Jul 12, 2021 at 8:10 PM Tor Vic <torvic9@mailbox.org> wrote:
> >
> > On some distros and configurations, it might be useful to allow for
> > specifying a directory where Clang stores its ThinLTO cache.
> >
> > More specifically, when building the VirtualBox extramodules on Arch with
> > its proper 'makepkg' build system and DKMS, against an already installed
> > ThinLTO kernel, the build fails because it tries to create the ThinLTO
> > cache in a directory that is not user-writable.
> >
> > A similar problem has been reported with openSUSE's OBS build system.
> >
> > Add a Kconfig option that allows users to choose a directory in which
> > Clang's ThinLTO can store its cache.
> >
> > Link: https://github.com/ClangBuiltLinux/linux/issues/1104
> > Signed-off-by: Tor Vic <torvic9@mailbox.org>
>
>
> I do not understand.
>
> Currently, .thinlto-cache is created in the same directory
> as the other objects. (right under $(KBUILD_EXTMOD))
>
> If you build in a read-only directory, you cannot put
> any build artifact (not limited to the thinlto cache) there.
>
> Why did changing the location of .thinlto-cache
> solve your problem?
>
Sorry, I accidentally sent the message before completing it...
So, when building an out-of-tree module on Arch using DKMS,
I get the following error message:
Error: Permission denied
LLVM ERROR: ThinLTO: Can't get a temporary file
The reason for this seems to be that DKMS looks for the cache inside
the installed kernel's modules/headers folder, which on Arch is in
/usr/lib/modules/$kernelversion/build, and this folder is of course not
writeable by the user.
With the patch, a user-writable directory can be selected and DKMS seems
to pick it up.
But yes, I also have more questions than answers unfortunately...
Maybe it is more of an issue with DKMS itself than with the kernel, in
any way, DKMS needs a few fixes for Clang-built kernels, see [1].
[1] https://github.com/dell/dkms/issues/124
>
>
> > ---
> > Makefile | 5 +++--
> > arch/Kconfig | 10 ++++++++++
> > scripts/Makefile.lib | 4 ++++
> > scripts/Makefile.modfinal | 4 ++++
> > 4 files changed, 21 insertions(+), 2 deletions(-)
> >
> > diff --git a/Makefile b/Makefile
> > index c3f9bd191b89..472bc8bfff03 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -932,7 +932,8 @@ endif
> > ifdef CONFIG_LTO_CLANG
> > ifdef CONFIG_LTO_CLANG_THIN
> > CC_FLAGS_LTO := -flto=thin -fsplit-lto-unit
> > -KBUILD_LDFLAGS += --thinlto-cache-dir=$(extmod_prefix).thinlto-cache
> > +export thinlto-dir = $(if
> > $(CONFIG_LTO_CLANG_THIN_CACHEDIR),$(CONFIG_LTO_CLANG_THIN_CACHEDIR)/)
> > +KBUILD_LDFLAGS +=
> > --thinlto-cache-dir=$(thinlto-dir)$(extmod_prefix).thinlto-cache
> > else
> > CC_FLAGS_LTO := -flto
> > endif
> > @@ -1728,7 +1729,7 @@ PHONY += compile_commands.json
> >
> > clean-dirs := $(KBUILD_EXTMOD)
> > clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers
> > $(KBUILD_EXTMOD)/modules.nsdeps \
> > - $(KBUILD_EXTMOD)/compile_commands.json $(KBUILD_EXTMOD)/.thinlto-cache
> > + $(KBUILD_EXTMOD)/compile_commands.json
> > $(thinlto-dir)$(KBUILD_EXTMOD)/.thinlto-cache
> >
> > PHONY += help
> > help:
> > diff --git a/arch/Kconfig b/arch/Kconfig
> > index 129df498a8e1..19e4d140e12a 100644
> > --- a/arch/Kconfig
> > +++ b/arch/Kconfig
> > @@ -696,6 +696,16 @@ config LTO_CLANG_THIN
> > https://clang.llvm.org/docs/ThinLTO.html
> >
> > If unsure, say Y.
> > +
> > +config LTO_CLANG_THIN_CACHEDIR
> > + string "Clang ThinLTO cache directory"
> > + depends on LTO_CLANG_THIN
> > + default ""
> > + help
> > + This option allows users to choose a directory that stores
> > + Clang's ThinLTO cache.
> > + Leave empty for default.
> > +
> > endchoice
> >
> > config ARCH_SUPPORTS_CFI_CLANG
> > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > index 10950559b223..bca87a6aa35b 100644
> > --- a/scripts/Makefile.lib
> > +++ b/scripts/Makefile.lib
> > @@ -197,6 +197,10 @@ endif
> > part-of-module = $(if $(filter $(basename $@).o, $(real-obj-m)),y)
> > quiet_modtag = $(if $(part-of-module),[M], )
> >
> > +ifdef CONFIG_LTO_CLANG_THIN
> > +KBUILD_LDFLAGS +=
> > --thinlto-cache-dir=$(thinlto-dir)$(extmod-prefix).thinlto-cache
> > +endif
> > +
> > modkern_cflags = \
> > $(if $(part-of-module), \
> > $(KBUILD_CFLAGS_MODULE) $(CFLAGS_MODULE), \
> > diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
> > index 5e9b8057fb24..ab0d72e21318 100644
> > --- a/scripts/Makefile.modfinal
> > +++ b/scripts/Makefile.modfinal
> > @@ -35,6 +35,10 @@ ifdef CONFIG_LTO_CLANG
> > # avoid a second slow LTO link
> > prelink-ext := .lto
> >
> > +ifdef CONFIG_LTO_CLANG_THIN
> > +KBUILD_LDFLAGS +=
> > --thinlto-cache-dir=$(thinlto-dir)$(extmod-prefix).thinlto-cache
> > +endif # CONFIG_LTO_CLANG_THIN
> > +
> > # ELF processing was skipped earlier because we didn't have native code,
> > # so let's now process the prelinked binary before we link the module.
> >
> > --
> > 2.32.0
>
>
>
> --
> Best Regards
> Masahiro Yamada
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-07-13 12:43 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-12 11:10 [PATCH 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory Tor Vic
2021-07-12 17:22 ` Kees Cook
2021-07-13 11:52 ` torvic9
2021-07-13 10:45 ` Masahiro Yamada
2021-07-13 12:38 ` torvic9
2021-07-13 12:43 ` torvic9
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.