[PATCH v3] RISC-V: build: Allow LTO to be selected

[PATCH v3] RISC-V: build: Allow LTO to be selected

[Nathan Chancellor]

From: Wende Tan <twd2.me@gmail.com>

Allow LTO to be selected for RISC-V, only when LLD >= 14, since there is an
issue [1] in prior LLD versions that prevents LLD to generate proper
machine code for RISC-V when writing `nop`s.

I have tested enabling LTO for `defconfig`. The LLD took ~2m21s and ~3GiB
on our Intel Xeon Gold 6140 server and produced an 18MiB Image. The image
can boot to shell using an archriscv rootfs on QEMU.

I have also tested it for `allyesconfig` without COMPILE_TEST, FTRACE,
KASAN, and GCOV. The LLD took ~7h03m and ~335GiB on the server,
successfully producing a 1.7GiB Image. Unfortunately, we cannot boot this
image because the `create_kernel_page_table()` -> `alloc_pmd_early()` ->
`BUG_ON()` logic limits the image to be < 1GiB. Maybe we can fix it in a
separate patch further.

Disable LTO for arch/riscv/kernel/pi, as llvm-objcopy expects an ELF
object file when manipulating the files in that subfolder, rather than
LLVM bitcode.

[1] https://github.com/llvm/llvm-project/issues/50505, resolved by LLVM
    commit e63455d5e0e5 ("[MC] Use local MCSubtargetInfo in writeNops")

Tested-by: Wende Tan <twd2.me@gmail.com>
Signed-off-by: Wende Tan <twd2.me@gmail.com>
Co-developed-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
---
NOTE: I tested LLVM 14 through 18 with defconfig + full/thin LTO and
allmodconfig + thin LTO. allmodconfig + thin LTO with LLVM 15 and 16
shows

  ld.lld: error: section size decrease is too large

when linking vmlinux. This appears to be resolved in LLVM 17 with
https://github.com/llvm/llvm-project/commit/9d37ea95df1b84cca9b5e954d8964c976a5e303e
(I did not bisect but the commit message lines up with the issue). I
kept the existing version check because defconfig worked fine but we may
want to bump it to 17.0.0 if randconfigs trip over this.

Changes in v3:
- Disable LTO in arch/riscv/kernel/pi/Makefile, which was added to the
  kernel after the submission of v2. This change matches arm64.
- Link to v2: https://lore.kernel.org/r/20220512205545.992288-1-twd2.me@gmail.com/

Changes in v2:
- Some textual changes suggested by Nick.
- Drop the changes to `arch/riscv/Makefile`, since the LLVM issue is
  filed and resolved.
- Drop the unnecessary changes to `arch/riscv/kernel/vdso/Makefile`.
- Link to v1: https://lore.kernel.org/r/20210719205208.1023221-1-twd2.me@gmail.com/
---
 arch/riscv/Kconfig            | 3 +++
 arch/riscv/kernel/pi/Makefile | 3 +++
 2 files changed, 6 insertions(+)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index d607ab0f7c6d..523640f7441e 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -46,6 +46,9 @@ config RISCV
 	select ARCH_SUPPORTS_CFI_CLANG
 	select ARCH_SUPPORTS_DEBUG_PAGEALLOC if MMU
 	select ARCH_SUPPORTS_HUGETLBFS if MMU
+	# LLD >= 14: https://github.com/llvm/llvm-project/issues/50505
+	select ARCH_SUPPORTS_LTO_CLANG if LLD_VERSION >= 140000
+	select ARCH_SUPPORTS_LTO_CLANG_THIN if LLD_VERSION >= 140000
 	select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU
 	select ARCH_SUPPORTS_PER_VMA_LOCK if MMU
 	select ARCH_USE_MEMTEST
diff --git a/arch/riscv/kernel/pi/Makefile b/arch/riscv/kernel/pi/Makefile
index 07915dc9279e..b75f150b923d 100644
--- a/arch/riscv/kernel/pi/Makefile
+++ b/arch/riscv/kernel/pi/Makefile
@@ -9,6 +9,9 @@ KBUILD_CFLAGS	:= $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) -fpie \
 		   -fno-asynchronous-unwind-tables -fno-unwind-tables \
 		   $(call cc-option,-fno-addrsig)
 
+# Disable LTO
+KBUILD_CFLAGS	:= $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS))
+
 KBUILD_CFLAGS	+= -mcmodel=medany
 
 CFLAGS_cmdline_early.o += -D__NO_FORTIFY

---
base-commit: 8a749fd1a8720d4619c91c8b6e7528c0a355c0aa
change-id: 20231003-riscv-lto-f013beed8587

Best regards,
-- 
Nathan Chancellor <nathan@kernel.org>


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: build: Allow LTO to be selected

[twd2]

+CC MaskRay

Thanks!
Wende

On 2023/10/4 3:16, Nathan Chancellor wrote:
> From: Wende Tan <twd2.me@gmail.com>
>
> Allow LTO to be selected for RISC-V, only when LLD >= 14, since there is an
> issue [1] in prior LLD versions that prevents LLD to generate proper
> machine code for RISC-V when writing `nop`s.
>
> I have tested enabling LTO for `defconfig`. The LLD took ~2m21s and ~3GiB
> on our Intel Xeon Gold 6140 server and produced an 18MiB Image. The image
> can boot to shell using an archriscv rootfs on QEMU.
>
> I have also tested it for `allyesconfig` without COMPILE_TEST, FTRACE,
> KASAN, and GCOV. The LLD took ~7h03m and ~335GiB on the server,
> successfully producing a 1.7GiB Image. Unfortunately, we cannot boot this
> image because the `create_kernel_page_table()` -> `alloc_pmd_early()` ->
> `BUG_ON()` logic limits the image to be < 1GiB. Maybe we can fix it in a
> separate patch further.
>
> Disable LTO for arch/riscv/kernel/pi, as llvm-objcopy expects an ELF
> object file when manipulating the files in that subfolder, rather than
> LLVM bitcode.
>
> [1] https://github.com/llvm/llvm-project/issues/50505, resolved by LLVM
>     commit e63455d5e0e5 ("[MC] Use local MCSubtargetInfo in writeNops")
>
> Tested-by: Wende Tan <twd2.me@gmail.com>
> Signed-off-by: Wende Tan <twd2.me@gmail.com>
> Co-developed-by: Nathan Chancellor <nathan@kernel.org>
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> ---
> NOTE: I tested LLVM 14 through 18 with defconfig + full/thin LTO and
> allmodconfig + thin LTO. allmodconfig + thin LTO with LLVM 15 and 16
> shows
>
>   ld.lld: error: section size decrease is too large
>
> when linking vmlinux. This appears to be resolved in LLVM 17 with
> https://github.com/llvm/llvm-project/commit/9d37ea95df1b84cca9b5e954d8964c976a5e303e
> (I did not bisect but the commit message lines up with the issue). I
> kept the existing version check because defconfig worked fine but we may
> want to bump it to 17.0.0 if randconfigs trip over this.
>
> Changes in v3:
> - Disable LTO in arch/riscv/kernel/pi/Makefile, which was added to the
>   kernel after the submission of v2. This change matches arm64.
> - Link to v2: https://lore.kernel.org/r/20220512205545.992288-1-twd2.me@gmail.com/
>
> Changes in v2:
> - Some textual changes suggested by Nick.
> - Drop the changes to `arch/riscv/Makefile`, since the LLVM issue is
>   filed and resolved.
> - Drop the unnecessary changes to `arch/riscv/kernel/vdso/Makefile`.
> - Link to v1: https://lore.kernel.org/r/20210719205208.1023221-1-twd2.me@gmail.com/
> ---
>  arch/riscv/Kconfig            | 3 +++
>  arch/riscv/kernel/pi/Makefile | 3 +++
>  2 files changed, 6 insertions(+)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index d607ab0f7c6d..523640f7441e 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -46,6 +46,9 @@ config RISCV
>  	select ARCH_SUPPORTS_CFI_CLANG
>  	select ARCH_SUPPORTS_DEBUG_PAGEALLOC if MMU
>  	select ARCH_SUPPORTS_HUGETLBFS if MMU
> +	# LLD >= 14: https://github.com/llvm/llvm-project/issues/50505
> +	select ARCH_SUPPORTS_LTO_CLANG if LLD_VERSION >= 140000
> +	select ARCH_SUPPORTS_LTO_CLANG_THIN if LLD_VERSION >= 140000
>  	select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU
>  	select ARCH_SUPPORTS_PER_VMA_LOCK if MMU
>  	select ARCH_USE_MEMTEST
> diff --git a/arch/riscv/kernel/pi/Makefile b/arch/riscv/kernel/pi/Makefile
> index 07915dc9279e..b75f150b923d 100644
> --- a/arch/riscv/kernel/pi/Makefile
> +++ b/arch/riscv/kernel/pi/Makefile
> @@ -9,6 +9,9 @@ KBUILD_CFLAGS	:= $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) -fpie \
>  		   -fno-asynchronous-unwind-tables -fno-unwind-tables \
>  		   $(call cc-option,-fno-addrsig)
>  
> +# Disable LTO
> +KBUILD_CFLAGS	:= $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS))
> +
>  KBUILD_CFLAGS	+= -mcmodel=medany
>  
>  CFLAGS_cmdline_early.o += -D__NO_FORTIFY
>
> ---
> base-commit: 8a749fd1a8720d4619c91c8b6e7528c0a355c0aa
> change-id: 20231003-riscv-lto-f013beed8587
>
> Best regards,

-- 
Best wishes!
Sincerely,
twd2
https://twd2.me


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: build: Allow LTO to be selected

[Alexandre Ghiti]

Hi Nathan,

On 03/10/2023 21:16, Nathan Chancellor wrote:
> From: Wende Tan <twd2.me@gmail.com>
>
> Allow LTO to be selected for RISC-V, only when LLD >= 14, since there is an
> issue [1] in prior LLD versions that prevents LLD to generate proper
> machine code for RISC-V when writing `nop`s.
>
> I have tested enabling LTO for `defconfig`. The LLD took ~2m21s and ~3GiB
> on our Intel Xeon Gold 6140 server and produced an 18MiB Image. The image
> can boot to shell using an archriscv rootfs on QEMU.
>
> I have also tested it for `allyesconfig` without COMPILE_TEST, FTRACE,
> KASAN, and GCOV. The LLD took ~7h03m and ~335GiB on the server,
> successfully producing a 1.7GiB Image. Unfortunately, we cannot boot this
> image because the `create_kernel_page_table()` -> `alloc_pmd_early()` ->
> `BUG_ON()` logic limits the image to be < 1GiB. Maybe we can fix it in a
> separate patch further.


Yes, I see 2 solutions:

- either use PUD mappings, which is not that easy because we'd need the 
physical and virtual addresses to both be aligned on 1GB, which is very 
unlikely (the kernel is generally loaded at 0x8020_0000). We could use a 
relocatable kernel to offset the virtual kernel address though.

- or use a second early_pmd, which would make sense as we reserve 2GB 
for the kernel mapping anyway, but I'm not sure it would work without a 
bunch of modifications.

Let me know if that would help and I'll give it a try (unless you want 
to take care of that of course!).

Alex


>
> Disable LTO for arch/riscv/kernel/pi, as llvm-objcopy expects an ELF
> object file when manipulating the files in that subfolder, rather than
> LLVM bitcode.
>
> [1] https://github.com/llvm/llvm-project/issues/50505, resolved by LLVM
>      commit e63455d5e0e5 ("[MC] Use local MCSubtargetInfo in writeNops")
>
> Tested-by: Wende Tan <twd2.me@gmail.com>
> Signed-off-by: Wende Tan <twd2.me@gmail.com>
> Co-developed-by: Nathan Chancellor <nathan@kernel.org>
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> ---
> NOTE: I tested LLVM 14 through 18 with defconfig + full/thin LTO and
> allmodconfig + thin LTO. allmodconfig + thin LTO with LLVM 15 and 16
> shows
>
>    ld.lld: error: section size decrease is too large
>
> when linking vmlinux. This appears to be resolved in LLVM 17 with
> https://github.com/llvm/llvm-project/commit/9d37ea95df1b84cca9b5e954d8964c976a5e303e
> (I did not bisect but the commit message lines up with the issue). I
> kept the existing version check because defconfig worked fine but we may
> want to bump it to 17.0.0 if randconfigs trip over this.
>
> Changes in v3:
> - Disable LTO in arch/riscv/kernel/pi/Makefile, which was added to the
>    kernel after the submission of v2. This change matches arm64.
> - Link to v2: https://lore.kernel.org/r/20220512205545.992288-1-twd2.me@gmail.com/
>
> Changes in v2:
> - Some textual changes suggested by Nick.
> - Drop the changes to `arch/riscv/Makefile`, since the LLVM issue is
>    filed and resolved.
> - Drop the unnecessary changes to `arch/riscv/kernel/vdso/Makefile`.
> - Link to v1: https://lore.kernel.org/r/20210719205208.1023221-1-twd2.me@gmail.com/
> ---
>   arch/riscv/Kconfig            | 3 +++
>   arch/riscv/kernel/pi/Makefile | 3 +++
>   2 files changed, 6 insertions(+)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index d607ab0f7c6d..523640f7441e 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -46,6 +46,9 @@ config RISCV
>   	select ARCH_SUPPORTS_CFI_CLANG
>   	select ARCH_SUPPORTS_DEBUG_PAGEALLOC if MMU
>   	select ARCH_SUPPORTS_HUGETLBFS if MMU
> +	# LLD >= 14: https://github.com/llvm/llvm-project/issues/50505
> +	select ARCH_SUPPORTS_LTO_CLANG if LLD_VERSION >= 140000
> +	select ARCH_SUPPORTS_LTO_CLANG_THIN if LLD_VERSION >= 140000
>   	select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU
>   	select ARCH_SUPPORTS_PER_VMA_LOCK if MMU
>   	select ARCH_USE_MEMTEST
> diff --git a/arch/riscv/kernel/pi/Makefile b/arch/riscv/kernel/pi/Makefile
> index 07915dc9279e..b75f150b923d 100644
> --- a/arch/riscv/kernel/pi/Makefile
> +++ b/arch/riscv/kernel/pi/Makefile
> @@ -9,6 +9,9 @@ KBUILD_CFLAGS	:= $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) -fpie \
>   		   -fno-asynchronous-unwind-tables -fno-unwind-tables \
>   		   $(call cc-option,-fno-addrsig)
>   
> +# Disable LTO
> +KBUILD_CFLAGS	:= $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS))
> +
>   KBUILD_CFLAGS	+= -mcmodel=medany
>   
>   CFLAGS_cmdline_early.o += -D__NO_FORTIFY
>
> ---
> base-commit: 8a749fd1a8720d4619c91c8b6e7528c0a355c0aa
> change-id: 20231003-riscv-lto-f013beed8587
>
> Best regards,

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: build: Allow LTO to be selected

[Nathan Chancellor]

Hi Alex,

On Wed, Oct 04, 2023 at 11:51:26AM +0200, Alexandre Ghiti wrote:
> Hi Nathan,
> 
> On 03/10/2023 21:16, Nathan Chancellor wrote:
> > From: Wende Tan <twd2.me@gmail.com>
> > 
> > Allow LTO to be selected for RISC-V, only when LLD >= 14, since there is an
> > issue [1] in prior LLD versions that prevents LLD to generate proper
> > machine code for RISC-V when writing `nop`s.
> > 
> > I have tested enabling LTO for `defconfig`. The LLD took ~2m21s and ~3GiB
> > on our Intel Xeon Gold 6140 server and produced an 18MiB Image. The image
> > can boot to shell using an archriscv rootfs on QEMU.
> > 
> > I have also tested it for `allyesconfig` without COMPILE_TEST, FTRACE,
> > KASAN, and GCOV. The LLD took ~7h03m and ~335GiB on the server,
> > successfully producing a 1.7GiB Image. Unfortunately, we cannot boot this
> > image because the `create_kernel_page_table()` -> `alloc_pmd_early()` ->
> > `BUG_ON()` logic limits the image to be < 1GiB. Maybe we can fix it in a
> > separate patch further.
> 
> 
> Yes, I see 2 solutions:
> 
> - either use PUD mappings, which is not that easy because we'd need the
> physical and virtual addresses to both be aligned on 1GB, which is very
> unlikely (the kernel is generally loaded at 0x8020_0000). We could use a
> relocatable kernel to offset the virtual kernel address though.
> 
> - or use a second early_pmd, which would make sense as we reserve 2GB for
> the kernel mapping anyway, but I'm not sure it would work without a bunch of
> modifications.
> 
> Let me know if that would help and I'll give it a try (unless you want to
> take care of that of course!).

Wende was the one who made that comment in the commit message, not me,
so I won't be looking into this further, as I have little skin in this
game :)

In my humble opinion, allyesconfig is not indicative of any real world
scenario when it comes to kernel configuration or size, so when things
don't work with it, I tend to think more along the lines of "don't do
that then". A distributor is generally either going to distribute a
kernel for one device (in which case, they may build every driver that
they need for that platform into the kernel and turn every other driver
off, which won't result in a large kernel size) or for multiple devices
(in which case, they would want to want to build with modules as much as
possible, which also won't result in a large kernel size). Neither of
those scenarios is really represented by allyesconfig.

Furthermore, I am not sure that is really an LTO specific problem, it
sounds like a generic "kernel is larger than 1GB" problem to me, which
LTO could potentially show easier due to more inlining and such.

Cheers,
Nathan

> > Disable LTO for arch/riscv/kernel/pi, as llvm-objcopy expects an ELF
> > object file when manipulating the files in that subfolder, rather than
> > LLVM bitcode.
> > 
> > [1] https://github.com/llvm/llvm-project/issues/50505, resolved by LLVM
> >      commit e63455d5e0e5 ("[MC] Use local MCSubtargetInfo in writeNops")
> > 
> > Tested-by: Wende Tan <twd2.me@gmail.com>
> > Signed-off-by: Wende Tan <twd2.me@gmail.com>
> > Co-developed-by: Nathan Chancellor <nathan@kernel.org>
> > Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> > ---
> > NOTE: I tested LLVM 14 through 18 with defconfig + full/thin LTO and
> > allmodconfig + thin LTO. allmodconfig + thin LTO with LLVM 15 and 16
> > shows
> > 
> >    ld.lld: error: section size decrease is too large
> > 
> > when linking vmlinux. This appears to be resolved in LLVM 17 with
> > https://github.com/llvm/llvm-project/commit/9d37ea95df1b84cca9b5e954d8964c976a5e303e
> > (I did not bisect but the commit message lines up with the issue). I
> > kept the existing version check because defconfig worked fine but we may
> > want to bump it to 17.0.0 if randconfigs trip over this.
> > 
> > Changes in v3:
> > - Disable LTO in arch/riscv/kernel/pi/Makefile, which was added to the
> >    kernel after the submission of v2. This change matches arm64.
> > - Link to v2: https://lore.kernel.org/r/20220512205545.992288-1-twd2.me@gmail.com/
> > 
> > Changes in v2:
> > - Some textual changes suggested by Nick.
> > - Drop the changes to `arch/riscv/Makefile`, since the LLVM issue is
> >    filed and resolved.
> > - Drop the unnecessary changes to `arch/riscv/kernel/vdso/Makefile`.
> > - Link to v1: https://lore.kernel.org/r/20210719205208.1023221-1-twd2.me@gmail.com/
> > ---
> >   arch/riscv/Kconfig            | 3 +++
> >   arch/riscv/kernel/pi/Makefile | 3 +++
> >   2 files changed, 6 insertions(+)
> > 
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index d607ab0f7c6d..523640f7441e 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -46,6 +46,9 @@ config RISCV
> >   	select ARCH_SUPPORTS_CFI_CLANG
> >   	select ARCH_SUPPORTS_DEBUG_PAGEALLOC if MMU
> >   	select ARCH_SUPPORTS_HUGETLBFS if MMU
> > +	# LLD >= 14: https://github.com/llvm/llvm-project/issues/50505
> > +	select ARCH_SUPPORTS_LTO_CLANG if LLD_VERSION >= 140000
> > +	select ARCH_SUPPORTS_LTO_CLANG_THIN if LLD_VERSION >= 140000
> >   	select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU
> >   	select ARCH_SUPPORTS_PER_VMA_LOCK if MMU
> >   	select ARCH_USE_MEMTEST
> > diff --git a/arch/riscv/kernel/pi/Makefile b/arch/riscv/kernel/pi/Makefile
> > index 07915dc9279e..b75f150b923d 100644
> > --- a/arch/riscv/kernel/pi/Makefile
> > +++ b/arch/riscv/kernel/pi/Makefile
> > @@ -9,6 +9,9 @@ KBUILD_CFLAGS	:= $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) -fpie \
> >   		   -fno-asynchronous-unwind-tables -fno-unwind-tables \
> >   		   $(call cc-option,-fno-addrsig)
> > +# Disable LTO
> > +KBUILD_CFLAGS	:= $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS))
> > +
> >   KBUILD_CFLAGS	+= -mcmodel=medany
> >   CFLAGS_cmdline_early.o += -D__NO_FORTIFY
> > 
> > ---
> > base-commit: 8a749fd1a8720d4619c91c8b6e7528c0a355c0aa
> > change-id: 20231003-riscv-lto-f013beed8587
> > 
> > Best regards,

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: build: Allow LTO to be selected

[Nick Desaulniers]

On Tue, Oct 3, 2023 at 12:17 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> From: Wende Tan <twd2.me@gmail.com>
>
> Allow LTO to be selected for RISC-V, only when LLD >= 14, since there is an
> issue [1] in prior LLD versions that prevents LLD to generate proper
> machine code for RISC-V when writing `nop`s.
>
> I have tested enabling LTO for `defconfig`. The LLD took ~2m21s and ~3GiB
> on our Intel Xeon Gold 6140 server and produced an 18MiB Image. The image
> can boot to shell using an archriscv rootfs on QEMU.
>
> I have also tested it for `allyesconfig` without COMPILE_TEST, FTRACE,
> KASAN, and GCOV. The LLD took ~7h03m and ~335GiB on the server,
> successfully producing a 1.7GiB Image. Unfortunately, we cannot boot this
> image because the `create_kernel_page_table()` -> `alloc_pmd_early()` ->
> `BUG_ON()` logic limits the image to be < 1GiB. Maybe we can fix it in a
> separate patch further.
>
> Disable LTO for arch/riscv/kernel/pi, as llvm-objcopy expects an ELF
> object file when manipulating the files in that subfolder, rather than
> LLVM bitcode.
>
> [1] https://github.com/llvm/llvm-project/issues/50505, resolved by LLVM
>     commit e63455d5e0e5 ("[MC] Use local MCSubtargetInfo in writeNops")
>
> Tested-by: Wende Tan <twd2.me@gmail.com>
> Signed-off-by: Wende Tan <twd2.me@gmail.com>
> Co-developed-by: Nathan Chancellor <nathan@kernel.org>
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>

Perhaps worth a v4 due to linker relaxation resulting in boot failures
you observed:
https://github.com/ClangBuiltLinux/linux/issues/1942 ?

-- 
Thanks,
~Nick Desaulniers

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: build: Allow LTO to be selected

[Nathan Chancellor]

On Tue, Oct 10, 2023 at 10:03:15AM -0700, Nick Desaulniers wrote:
> On Tue, Oct 3, 2023 at 12:17 PM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > From: Wende Tan <twd2.me@gmail.com>
> >
> > Allow LTO to be selected for RISC-V, only when LLD >= 14, since there is an
> > issue [1] in prior LLD versions that prevents LLD to generate proper
> > machine code for RISC-V when writing `nop`s.
> >
> > I have tested enabling LTO for `defconfig`. The LLD took ~2m21s and ~3GiB
> > on our Intel Xeon Gold 6140 server and produced an 18MiB Image. The image
> > can boot to shell using an archriscv rootfs on QEMU.
> >
> > I have also tested it for `allyesconfig` without COMPILE_TEST, FTRACE,
> > KASAN, and GCOV. The LLD took ~7h03m and ~335GiB on the server,
> > successfully producing a 1.7GiB Image. Unfortunately, we cannot boot this
> > image because the `create_kernel_page_table()` -> `alloc_pmd_early()` ->
> > `BUG_ON()` logic limits the image to be < 1GiB. Maybe we can fix it in a
> > separate patch further.
> >
> > Disable LTO for arch/riscv/kernel/pi, as llvm-objcopy expects an ELF
> > object file when manipulating the files in that subfolder, rather than
> > LLVM bitcode.
> >
> > [1] https://github.com/llvm/llvm-project/issues/50505, resolved by LLVM
> >     commit e63455d5e0e5 ("[MC] Use local MCSubtargetInfo in writeNops")
> >
> > Tested-by: Wende Tan <twd2.me@gmail.com>
> > Signed-off-by: Wende Tan <twd2.me@gmail.com>
> > Co-developed-by: Nathan Chancellor <nathan@kernel.org>
> > Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> 
> Perhaps worth a v4 due to linker relaxation resulting in boot failures
> you observed:
> https://github.com/ClangBuiltLinux/linux/issues/1942 ?

Yes, agreed. RISC-V patchwork folks, consider marking this as "Changes
Requested" or whatever other status there is that indicates that I need
to send a v4 working around that issue, at least until we better
understand the issue.

Cheers,
Natha

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] RISC-V: build: Allow LTO to be selected

[Conor Dooley]

[-- Attachment #1.1: Type: text/plain, Size: 2138 bytes --]

On Tue, Oct 10, 2023 at 12:52:47PM -0700, Nathan Chancellor wrote:
> On Tue, Oct 10, 2023 at 10:03:15AM -0700, Nick Desaulniers wrote:
> > On Tue, Oct 3, 2023 at 12:17 PM Nathan Chancellor <nathan@kernel.org> wrote:
> > >
> > > From: Wende Tan <twd2.me@gmail.com>
> > >
> > > Allow LTO to be selected for RISC-V, only when LLD >= 14, since there is an
> > > issue [1] in prior LLD versions that prevents LLD to generate proper
> > > machine code for RISC-V when writing `nop`s.
> > >
> > > I have tested enabling LTO for `defconfig`. The LLD took ~2m21s and ~3GiB
> > > on our Intel Xeon Gold 6140 server and produced an 18MiB Image. The image
> > > can boot to shell using an archriscv rootfs on QEMU.
> > >
> > > I have also tested it for `allyesconfig` without COMPILE_TEST, FTRACE,
> > > KASAN, and GCOV. The LLD took ~7h03m and ~335GiB on the server,
> > > successfully producing a 1.7GiB Image. Unfortunately, we cannot boot this
> > > image because the `create_kernel_page_table()` -> `alloc_pmd_early()` ->
> > > `BUG_ON()` logic limits the image to be < 1GiB. Maybe we can fix it in a
> > > separate patch further.
> > >
> > > Disable LTO for arch/riscv/kernel/pi, as llvm-objcopy expects an ELF
> > > object file when manipulating the files in that subfolder, rather than
> > > LLVM bitcode.
> > >
> > > [1] https://github.com/llvm/llvm-project/issues/50505, resolved by LLVM
> > >     commit e63455d5e0e5 ("[MC] Use local MCSubtargetInfo in writeNops")
> > >
> > > Tested-by: Wende Tan <twd2.me@gmail.com>
> > > Signed-off-by: Wende Tan <twd2.me@gmail.com>
> > > Co-developed-by: Nathan Chancellor <nathan@kernel.org>
> > > Signed-off-by: Nathan Chancellor <nathan@kernel.org>
> > 
> > Perhaps worth a v4 due to linker relaxation resulting in boot failures
> > you observed:
> > https://github.com/ClangBuiltLinux/linux/issues/1942 ?
> 
> Yes, agreed. RISC-V patchwork folks, consider marking this as "Changes
> Requested" or whatever other status there is that indicates that I need
> to send a v4 working around that issue, at least until we better
> understand the issue.

Done.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 161 bytes --]

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv