From: Arnd Bergmann <arnd@kernel.org> To: Fangrui Song <maskray@google.com> Cc: Sami Tolvanen <samitolvanen@google.com>, Masahiro Yamada <masahiroy@kernel.org>, Steven Rostedt <rostedt@goodmis.org>, Will Deacon <will@kernel.org>, Josh Poimboeuf <jpoimboe@redhat.com>, Peter Zijlstra <peterz@infradead.org>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, "Paul E. McKenney" <paulmck@kernel.org>, Kees Cook <keescook@chromium.org>, Nick Desaulniers <ndesaulniers@google.com>, clang-built-linux <clang-built-linux@googlegroups.com>, Kernel Hardening <kernel-hardening@lists.openwall.com>, linux-arch <linux-arch@vger.kernel.org>, Linux ARM <linux-arm-kernel@lists.infradead.org>, Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, linux-pci <linux-pci@vger.kernel.org> Subject: Re: [PATCH v8 00/16] Add support for Clang LTO Date: Wed, 9 Dec 2020 10:19:44 +0100 [thread overview] Message-ID: <CAK8P3a1p8yKMKXB1cUpy-5PjehJGPX0SPNEx4VsMmqgOeR6fZg@mail.gmail.com> (raw) In-Reply-To: <20201209045554.fxlzrmxknakl2gdr@google.com> On Wed, Dec 9, 2020 at 5:56 AM 'Fangrui Song' via Clang Built Linux <clang-built-linux@googlegroups.com> wrote: > On 2020-12-08, 'Sami Tolvanen' via Clang Built Linux wrote: > >On Tue, Dec 8, 2020 at 4:15 AM Arnd Bergmann <arnd@kernel.org> wrote: > >> So far I have > >> not managed to get a working build out of it, the main problem so far being > >> that it is really slow to build because the link stage only uses one CPU. > >> These are the other issues I've seen so far: > > ld.lld ThinLTO uses the number of (physical cores enabled by affinity) by default. Ah, I see. Do you know if it's also possible to do something like -flto=jobserver to integrate better with the kernel build system? I tend to run multiple builds under a top-level makefile with 'make -j30' in order to use 30 of the 32 threads and leave the scheduling to jobserver instead of the kernel. If the linker itself is multithreaded but the jobserver thinks it is a single thread, could end up with 30 concurrent linkers each trying to use 16 cores. > >> - CONFIG_CPU_BIG_ENDIAN doesn't seem to work with lld, and LTO > >> doesn't work with ld.bfd. > >> I've added a CPU_LITTLE_ENDIAN dependency to > >> ARCH_SUPPORTS_LTO_CLANG{,THIN} > > > >Ah, good point. I'll fix this in v9. > > Full/Thin LTO should work with GNU ld and gold with LLVMgold.so built from > llvm-project (https://llvm.org/docs/GoldPlugin.html ). You'll need to make sure > that LLVMgold.so is newer than clang. (Newer clang may introduce bitcode > attributes which are unrecognizable by older LLVMgold.so/ld.lld) The current patch series requires LLD: config HAS_LTO_CLANG def_bool y depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD Is this something we should change then, or try to keep it simple with the current approach, leaving LTO disabled for big-endian builds and hosts without a working lld? Arnd
WARNING: multiple messages have this Message-ID (diff)
From: Arnd Bergmann <arnd@kernel.org> To: Fangrui Song <maskray@google.com> Cc: linux-arch <linux-arch@vger.kernel.org>, Kees Cook <keescook@chromium.org>, "Paul E. McKenney" <paulmck@kernel.org>, Kernel Hardening <kernel-hardening@lists.openwall.com>, Peter Zijlstra <peterz@infradead.org>, Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Masahiro Yamada <masahiroy@kernel.org>, Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>, Nick Desaulniers <ndesaulniers@google.com>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Steven Rostedt <rostedt@goodmis.org>, clang-built-linux <clang-built-linux@googlegroups.com>, linux-pci <linux-pci@vger.kernel.org>, Sami Tolvanen <samitolvanen@google.com>, Josh Poimboeuf <jpoimboe@redhat.com>, Will Deacon <will@kernel.org>, Linux ARM <linux-arm-kernel@lists.infradead.org> Subject: Re: [PATCH v8 00/16] Add support for Clang LTO Date: Wed, 9 Dec 2020 10:19:44 +0100 [thread overview] Message-ID: <CAK8P3a1p8yKMKXB1cUpy-5PjehJGPX0SPNEx4VsMmqgOeR6fZg@mail.gmail.com> (raw) In-Reply-To: <20201209045554.fxlzrmxknakl2gdr@google.com> On Wed, Dec 9, 2020 at 5:56 AM 'Fangrui Song' via Clang Built Linux <clang-built-linux@googlegroups.com> wrote: > On 2020-12-08, 'Sami Tolvanen' via Clang Built Linux wrote: > >On Tue, Dec 8, 2020 at 4:15 AM Arnd Bergmann <arnd@kernel.org> wrote: > >> So far I have > >> not managed to get a working build out of it, the main problem so far being > >> that it is really slow to build because the link stage only uses one CPU. > >> These are the other issues I've seen so far: > > ld.lld ThinLTO uses the number of (physical cores enabled by affinity) by default. Ah, I see. Do you know if it's also possible to do something like -flto=jobserver to integrate better with the kernel build system? I tend to run multiple builds under a top-level makefile with 'make -j30' in order to use 30 of the 32 threads and leave the scheduling to jobserver instead of the kernel. If the linker itself is multithreaded but the jobserver thinks it is a single thread, could end up with 30 concurrent linkers each trying to use 16 cores. > >> - CONFIG_CPU_BIG_ENDIAN doesn't seem to work with lld, and LTO > >> doesn't work with ld.bfd. > >> I've added a CPU_LITTLE_ENDIAN dependency to > >> ARCH_SUPPORTS_LTO_CLANG{,THIN} > > > >Ah, good point. I'll fix this in v9. > > Full/Thin LTO should work with GNU ld and gold with LLVMgold.so built from > llvm-project (https://llvm.org/docs/GoldPlugin.html ). You'll need to make sure > that LLVMgold.so is newer than clang. (Newer clang may introduce bitcode > attributes which are unrecognizable by older LLVMgold.so/ld.lld) The current patch series requires LLD: config HAS_LTO_CLANG def_bool y depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD Is this something we should change then, or try to keep it simple with the current approach, leaving LTO disabled for big-endian builds and hosts without a working lld? Arnd _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2020-12-09 9:21 UTC|newest] Thread overview: 141+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-12-01 21:36 [PATCH v8 00/16] Add support for Clang LTO Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` [PATCH v8 01/16] tracing: move function tracer options to Kconfig Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:47 ` Steven Rostedt 2020-12-01 21:47 ` Steven Rostedt 2020-12-01 21:36 ` [PATCH v8 02/16] kbuild: add support for Clang LTO Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-02 2:59 ` Masahiro Yamada 2020-12-02 2:59 ` Masahiro Yamada 2020-12-03 0:07 ` Nick Desaulniers 2020-12-03 0:07 ` Nick Desaulniers 2020-12-03 0:07 ` Nick Desaulniers 2020-12-01 21:36 ` [PATCH v8 03/16] kbuild: lto: fix module versioning Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` [PATCH v8 04/16] kbuild: lto: limit inlining Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` [PATCH v8 05/16] kbuild: lto: merge module sections Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` [PATCH v8 06/16] kbuild: lto: remove duplicate dependencies from .mod files Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` [PATCH v8 07/16] init: lto: ensure initcall ordering Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` [PATCH v8 08/16] init: lto: fix PREL32 relocations Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:36 ` Sami Tolvanen 2020-12-01 21:37 ` [PATCH v8 09/16] PCI: Fix PREL32 relocations for LTO Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` [PATCH v8 10/16] modpost: lto: strip .lto from module names Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` [PATCH v8 11/16] scripts/mod: disable LTO for empty.c Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` [PATCH v8 12/16] efi/libstub: disable LTO Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` [PATCH v8 13/16] drivers/misc/lkdtm: disable LTO for rodata.o Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` [PATCH v8 14/16] arm64: vdso: disable LTO Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` [PATCH v8 15/16] arm64: disable recordmcount with DYNAMIC_FTRACE_WITH_REGS Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` [PATCH v8 16/16] arm64: allow LTO to be selected Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-01 21:37 ` Sami Tolvanen 2020-12-03 0:01 ` [PATCH v8 00/16] Add support for Clang LTO Nick Desaulniers 2020-12-03 0:01 ` Nick Desaulniers 2020-12-03 0:01 ` Nick Desaulniers 2020-12-03 11:26 ` Will Deacon 2020-12-03 11:26 ` Will Deacon 2020-12-03 17:07 ` Sami Tolvanen 2020-12-03 17:07 ` Sami Tolvanen 2020-12-03 17:07 ` Sami Tolvanen 2020-12-03 18:21 ` Nathan Chancellor 2020-12-03 18:21 ` Nathan Chancellor 2020-12-03 18:22 ` Will Deacon 2020-12-03 18:22 ` Will Deacon 2020-12-03 22:32 ` Nick Desaulniers 2020-12-03 22:32 ` Nick Desaulniers 2020-12-03 22:32 ` Nick Desaulniers 2020-12-04 9:35 ` Will Deacon 2020-12-04 9:35 ` Will Deacon 2020-12-04 22:52 ` Sami Tolvanen 2020-12-04 22:52 ` Sami Tolvanen 2020-12-04 22:52 ` Sami Tolvanen 2020-12-06 6:50 ` Nathan Chancellor 2020-12-06 6:50 ` Nathan Chancellor 2020-12-06 20:09 ` Sami Tolvanen 2020-12-06 20:09 ` Sami Tolvanen 2020-12-06 20:09 ` Sami Tolvanen 2020-12-08 0:46 ` Nathan Chancellor 2020-12-08 0:46 ` Nathan Chancellor 2020-12-08 12:15 ` Arnd Bergmann 2020-12-08 12:15 ` Arnd Bergmann 2020-12-08 12:15 ` Arnd Bergmann 2020-12-08 13:54 ` Arnd Bergmann 2020-12-08 13:54 ` Arnd Bergmann 2020-12-08 13:54 ` Arnd Bergmann 2020-12-08 16:53 ` Sami Tolvanen 2020-12-08 16:53 ` Sami Tolvanen 2020-12-08 16:53 ` Sami Tolvanen 2020-12-08 16:56 ` Arnd Bergmann 2020-12-08 16:56 ` Arnd Bergmann 2020-12-08 16:56 ` Arnd Bergmann 2020-12-08 16:43 ` Sami Tolvanen 2020-12-08 16:43 ` Sami Tolvanen 2020-12-08 16:43 ` Sami Tolvanen 2020-12-08 20:59 ` Arnd Bergmann 2020-12-08 20:59 ` Arnd Bergmann 2020-12-08 21:02 ` Arnd Bergmann 2020-12-08 21:02 ` Arnd Bergmann 2020-12-09 5:23 ` Fāng-ruì Sòng 2020-12-09 5:23 ` Fāng-ruì Sòng 2020-12-09 5:23 ` Fāng-ruì Sòng 2020-12-09 9:07 ` Arnd Bergmann 2020-12-09 9:07 ` Arnd Bergmann 2020-12-09 9:07 ` Arnd Bergmann 2020-12-09 9:59 ` Arnd Bergmann 2020-12-09 9:59 ` Arnd Bergmann 2020-12-09 16:09 ` Sami Tolvanen 2020-12-09 16:09 ` Sami Tolvanen 2020-12-09 16:09 ` Sami Tolvanen 2020-12-09 19:24 ` Arnd Bergmann 2020-12-09 19:24 ` Arnd Bergmann 2020-12-09 19:24 ` Arnd Bergmann 2020-12-08 21:09 ` Nick Desaulniers 2020-12-08 21:09 ` Nick Desaulniers 2020-12-08 21:09 ` Nick Desaulniers 2020-12-08 22:20 ` Arnd Bergmann 2020-12-08 22:20 ` Arnd Bergmann 2020-12-08 22:20 ` Arnd Bergmann 2020-12-09 16:11 ` Sami Tolvanen 2020-12-09 16:11 ` Sami Tolvanen 2020-12-09 16:11 ` Sami Tolvanen 2020-12-09 4:55 ` Fangrui Song 2020-12-09 4:55 ` Fangrui Song 2020-12-09 9:19 ` Arnd Bergmann [this message] 2020-12-09 9:19 ` Arnd Bergmann 2020-12-09 9:19 ` Arnd Bergmann 2020-12-09 12:35 ` Arnd Bergmann 2020-12-09 12:35 ` Arnd Bergmann 2020-12-09 12:35 ` Arnd Bergmann 2020-12-09 16:25 ` Sami Tolvanen 2020-12-09 16:25 ` Sami Tolvanen 2020-12-09 16:25 ` Sami Tolvanen 2020-12-09 17:51 ` Arnd Bergmann 2020-12-09 17:51 ` Arnd Bergmann 2020-12-09 17:51 ` Arnd Bergmann
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=CAK8P3a1p8yKMKXB1cUpy-5PjehJGPX0SPNEx4VsMmqgOeR6fZg@mail.gmail.com \ --to=arnd@kernel.org \ --cc=clang-built-linux@googlegroups.com \ --cc=gregkh@linuxfoundation.org \ --cc=jpoimboe@redhat.com \ --cc=keescook@chromium.org \ --cc=kernel-hardening@lists.openwall.com \ --cc=linux-arch@vger.kernel.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kbuild@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-pci@vger.kernel.org \ --cc=masahiroy@kernel.org \ --cc=maskray@google.com \ --cc=ndesaulniers@google.com \ --cc=paulmck@kernel.org \ --cc=peterz@infradead.org \ --cc=rostedt@goodmis.org \ --cc=samitolvanen@google.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.