* [PATCH v2 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory
@ 2021-07-13 16:41 Tor Vic
2021-07-14 2:46 ` Masahiro Yamada
0 siblings, 1 reply; 3+ messages in thread
From: Tor Vic @ 2021-07-13 16:41 UTC (permalink / raw)
To: linux-kernel, masahiroy, Nathan Chancellor, ndesaulniers, keescook
Cc: linux-kbuild, clang-built-linux
On some distros and configurations, it might be useful to allow for
specifying a directory where Clang stores its ThinLTO cache.
More specifically, when building the VirtualBox extramodules on Arch with
its proper 'makepkg' build system and DKMS, against an already installed
ThinLTO kernel, the build fails because it tries to create the ThinLTO
cache in a directory that is not user-writable.
A similar problem has been reported with openSUSE's OBS build system.
Add a Kconfig option that allows users to choose a directory in which
Clang's ThinLTO can store its cache.
Link: https://github.com/ClangBuiltLinux/linux/issues/1104
Signed-off-by: Tor Vic <torvic9@mailbox.org>
---
Changes from v1 to v2: remove unneeded changes in scripts/Makefile
Makefile | 5 +++--
arch/Kconfig | 10 ++++++++++
2 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/Makefile b/Makefile
index c3f9bd191b89..472bc8bfff03 100644
--- a/Makefile
+++ b/Makefile
@@ -932,7 +932,8 @@ endif
ifdef CONFIG_LTO_CLANG
ifdef CONFIG_LTO_CLANG_THIN
CC_FLAGS_LTO := -flto=thin -fsplit-lto-unit
-KBUILD_LDFLAGS += --thinlto-cache-dir=$(extmod_prefix).thinlto-cache
+export thinlto-dir = $(if
$(CONFIG_LTO_CLANG_THIN_CACHEDIR),$(CONFIG_LTO_CLANG_THIN_CACHEDIR)/)
+KBUILD_LDFLAGS +=
--thinlto-cache-dir=$(thinlto-dir)$(extmod_prefix).thinlto-cache
else
CC_FLAGS_LTO := -flto
endif
@@ -1728,7 +1729,7 @@ PHONY += compile_commands.json
clean-dirs := $(KBUILD_EXTMOD)
clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers
$(KBUILD_EXTMOD)/modules.nsdeps \
- $(KBUILD_EXTMOD)/compile_commands.json $(KBUILD_EXTMOD)/.thinlto-cache
+ $(KBUILD_EXTMOD)/compile_commands.json
$(thinlto-dir)$(KBUILD_EXTMOD)/.thinlto-cache
PHONY += help
help:
diff --git a/arch/Kconfig b/arch/Kconfig
index 129df498a8e1..19e4d140e12a 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -696,6 +696,16 @@ config LTO_CLANG_THIN
https://clang.llvm.org/docs/ThinLTO.html
If unsure, say Y.
+
+config LTO_CLANG_THIN_CACHEDIR
+ string "Clang ThinLTO cache directory"
+ depends on LTO_CLANG_THIN
+ default ""
+ help
+ This option allows users to choose a directory that stores
+ Clang's ThinLTO cache.
+ Leave empty for default.
+
endchoice
config ARCH_SUPPORTS_CFI_CLANG
--
2.32.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH v2 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory
2021-07-13 16:41 [PATCH v2 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory Tor Vic
@ 2021-07-14 2:46 ` Masahiro Yamada
2021-07-14 6:12 ` Tor Vic
0 siblings, 1 reply; 3+ messages in thread
From: Masahiro Yamada @ 2021-07-14 2:46 UTC (permalink / raw)
To: Tor Vic
Cc: linux-kernel, Nathan Chancellor, ndesaulniers, Kees Cook,
Linux Kbuild mailing list, clang-built-linux
On Wed, Jul 14, 2021 at 1:41 AM Tor Vic <torvic9@mailbox.org> wrote:
>
> On some distros and configurations, it might be useful to allow for
> specifying a directory where Clang stores its ThinLTO cache.
>
> More specifically, when building the VirtualBox extramodules on Arch with
> its proper 'makepkg' build system and DKMS, against an already installed
> ThinLTO kernel, the build fails because it tries to create the ThinLTO
> cache in a directory that is not user-writable.
Again, I do not understand.
Is this fixing the root cause?
To me, it looks like
"Anyway, this worked for me" patch.
Besides, 'make clean' will never clean up the
cache directory.
> A similar problem has been reported with openSUSE's OBS build system.
>
> Add a Kconfig option that allows users to choose a directory in which
> Clang's ThinLTO can store its cache.
>
> Link: https://github.com/ClangBuiltLinux/linux/issues/1104
> Signed-off-by: Tor Vic <torvic9@mailbox.org>
> ---
> Changes from v1 to v2: remove unneeded changes in scripts/Makefile
>
> Makefile | 5 +++--
> arch/Kconfig | 10 ++++++++++
> 2 files changed, 13 insertions(+), 2 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index c3f9bd191b89..472bc8bfff03 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -932,7 +932,8 @@ endif
> ifdef CONFIG_LTO_CLANG
> ifdef CONFIG_LTO_CLANG_THIN
> CC_FLAGS_LTO := -flto=thin -fsplit-lto-unit
> -KBUILD_LDFLAGS += --thinlto-cache-dir=$(extmod_prefix).thinlto-cache
> +export thinlto-dir = $(if
> $(CONFIG_LTO_CLANG_THIN_CACHEDIR),$(CONFIG_LTO_CLANG_THIN_CACHEDIR)/)
> +KBUILD_LDFLAGS +=
> --thinlto-cache-dir=$(thinlto-dir)$(extmod_prefix).thinlto-cache
> else
> CC_FLAGS_LTO := -flto
> endif
> @@ -1728,7 +1729,7 @@ PHONY += compile_commands.json
>
> clean-dirs := $(KBUILD_EXTMOD)
> clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers
> $(KBUILD_EXTMOD)/modules.nsdeps \
> - $(KBUILD_EXTMOD)/compile_commands.json $(KBUILD_EXTMOD)/.thinlto-cache
> + $(KBUILD_EXTMOD)/compile_commands.json
> $(thinlto-dir)$(KBUILD_EXTMOD)/.thinlto-cache
>
> PHONY += help
> help:
> diff --git a/arch/Kconfig b/arch/Kconfig
> index 129df498a8e1..19e4d140e12a 100644
> --- a/arch/Kconfig
> +++ b/arch/Kconfig
> @@ -696,6 +696,16 @@ config LTO_CLANG_THIN
> https://clang.llvm.org/docs/ThinLTO.html
>
> If unsure, say Y.
> +
> +config LTO_CLANG_THIN_CACHEDIR
> + string "Clang ThinLTO cache directory"
> + depends on LTO_CLANG_THIN
> + default ""
> + help
> + This option allows users to choose a directory that stores
> + Clang's ThinLTO cache.
> + Leave empty for default.
> +
> endchoice
>
> config ARCH_SUPPORTS_CFI_CLANG
> --
> 2.32.0
>
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH v2 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory
2021-07-14 2:46 ` Masahiro Yamada
@ 2021-07-14 6:12 ` Tor Vic
0 siblings, 0 replies; 3+ messages in thread
From: Tor Vic @ 2021-07-14 6:12 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kernel, Nathan Chancellor, ndesaulniers, Kees Cook,
Linux Kbuild mailing list, clang-built-linux
On 14.07.21 02:46, Masahiro Yamada wrote:
> On Wed, Jul 14, 2021 at 1:41 AM Tor Vic <torvic9@mailbox.org> wrote:
>>
>> On some distros and configurations, it might be useful to allow for
>> specifying a directory where Clang stores its ThinLTO cache.
>>
>> More specifically, when building the VirtualBox extramodules on Arch with
>> its proper 'makepkg' build system and DKMS, against an already installed
>> ThinLTO kernel, the build fails because it tries to create the ThinLTO
>> cache in a directory that is not user-writable.
>
>
> Again, I do not understand.
Example:
1. build and install a kernel+headers with Arch's own makepkg/pacman tools.
2. at the end of the install, dkms is automatically called to rebuild
any out-of-tree modules like e.g. VirtualBox.
3. dkms fails because it tries to store the thinlto cache in a
directory that it not user-writable (very possibly in the installed
kernel headers directory under /usr/lib).
4. By explicitly selecting a place for the cache, e.g. /tmp, the dkms
module rebuild succeeds.
>
> Is this fixing the root cause?
If only I knew. I have very limited skills, and no IT or coding
background. It's possible that this can be fixed also on dkms but
I'm not sure.
Currently, dkms does not do a compiler check (it only seems to know
gcc) and it will fail unless explicitly told to use clang. e.g.
by specifying 'CC=clang LD=ld.lld" etc.
This has been reported to them.
>
> To me, it looks like
> "Anyway, this worked for me" patch.
At least one other person reported success by moving the cache to
another place (see link).
Admittedly, it looks the same way to me, I just wanted to get things
to work and it's likely that there are better solutions to this
problem, but AFAIR no one came forward.
>
> Besides, 'make clean' will never clean up the
> cache directory.
I can have a look at it again.
>
>
>
>
>
>> A similar problem has been reported with openSUSE's OBS build system.
>>
>> Add a Kconfig option that allows users to choose a directory in which
>> Clang's ThinLTO can store its cache.
>>
>> Link: https://github.com/ClangBuiltLinux/linux/issues/1104
>> Signed-off-by: Tor Vic <torvic9@mailbox.org>
>> ---
>> Changes from v1 to v2: remove unneeded changes in scripts/Makefile
>>
>> Makefile | 5 +++--
>> arch/Kconfig | 10 ++++++++++
>> 2 files changed, 13 insertions(+), 2 deletions(-)
>>
>> diff --git a/Makefile b/Makefile
>> index c3f9bd191b89..472bc8bfff03 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -932,7 +932,8 @@ endif
>> ifdef CONFIG_LTO_CLANG
>> ifdef CONFIG_LTO_CLANG_THIN
>> CC_FLAGS_LTO := -flto=thin -fsplit-lto-unit
>> -KBUILD_LDFLAGS += --thinlto-cache-dir=$(extmod_prefix).thinlto-cache
>> +export thinlto-dir = $(if
>> $(CONFIG_LTO_CLANG_THIN_CACHEDIR),$(CONFIG_LTO_CLANG_THIN_CACHEDIR)/)
>> +KBUILD_LDFLAGS +=
>> --thinlto-cache-dir=$(thinlto-dir)$(extmod_prefix).thinlto-cache
>> else
>> CC_FLAGS_LTO := -flto
>> endif
>> @@ -1728,7 +1729,7 @@ PHONY += compile_commands.json
>>
>> clean-dirs := $(KBUILD_EXTMOD)
>> clean: rm-files := $(KBUILD_EXTMOD)/Module.symvers
>> $(KBUILD_EXTMOD)/modules.nsdeps \
>> - $(KBUILD_EXTMOD)/compile_commands.json $(KBUILD_EXTMOD)/.thinlto-cache
>> + $(KBUILD_EXTMOD)/compile_commands.json
>> $(thinlto-dir)$(KBUILD_EXTMOD)/.thinlto-cache
>>
>> PHONY += help
>> help:
>> diff --git a/arch/Kconfig b/arch/Kconfig
>> index 129df498a8e1..19e4d140e12a 100644
>> --- a/arch/Kconfig
>> +++ b/arch/Kconfig
>> @@ -696,6 +696,16 @@ config LTO_CLANG_THIN
>> https://clang.llvm.org/docs/ThinLTO.html
>>
>> If unsure, say Y.
>> +
>> +config LTO_CLANG_THIN_CACHEDIR
>> + string "Clang ThinLTO cache directory"
>> + depends on LTO_CLANG_THIN
>> + default ""
>> + help
>> + This option allows users to choose a directory that stores
>> + Clang's ThinLTO cache.
>> + Leave empty for default.
>> +
>> endchoice
>>
>> config ARCH_SUPPORTS_CFI_CLANG
>> --
>> 2.32.0
>>
>
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-07-14 6:18 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-13 16:41 [PATCH v2 1/1] Kbuild, clang: add option for choosing a ThinLTO cache directory Tor Vic
2021-07-14 2:46 ` Masahiro Yamada
2021-07-14 6:12 ` Tor Vic
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).