[PATCH] arm64: Make CPU_BIG_ENDIAN depend on !LD_IS_LLD

[PATCH] arm64: Make CPU_BIG_ENDIAN depend on !LD_IS_LLD

[Nathan Chancellor]

Similar to commit 28187dc8ebd9 ("ARM: 9025/1: Kconfig: CPU_BIG_ENDIAN
depends on !LD_IS_LLD"), ld.lld does not support aarch64 big endian,
leading to the following build error when CONFIG_CPU_BIG_ENDIAN is
selected:

ld.lld: error: unknown emulation: aarch64linuxb

There are not currently plans to implement big endian support for
aarch64 in ld.lld but if it should be supported in the future, this
symbol can depend on the version that first supports it. In the
meantime, prevent this symbol from being selected to avoid these type
of build errors.

While we are here, the indentation of this symbol used spaces since its
introduction in commit a872013d6d03 ("arm64: kconfig: allow
CPU_BIG_ENDIAN to be selected"). Change it to tabs to be consistent with
kernel coding style.

Link: https://github.com/ClangBuiltLinux/linux/issues/380
Link: https://github.com/ClangBuiltLinux/linux/issues/1288
Reported-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
---
 arch/arm64/Kconfig | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index f39568b28ec1..5ddf9fe7b4ee 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -952,8 +952,9 @@ choice
 	  that is selected here.
 
 config CPU_BIG_ENDIAN
-       bool "Build big-endian kernel"
-       help
+	bool "Build big-endian kernel"
+	depends on !LD_IS_LLD
+	help
 	  Say Y if you plan on running a kernel with a big-endian userspace.
 
 config CPU_LITTLE_ENDIAN

base-commit: 88bb507a74ea7d75fa49edd421eaa710a7d80598
-- 
2.30.0

Re: [PATCH] arm64: Make CPU_BIG_ENDIAN depend on !LD_IS_LLD

[Arnd Bergmann]

On Tue, Feb 2, 2021 at 3:25 AM Nathan Chancellor <nathan@kernel.org> wrote:
>
> Similar to commit 28187dc8ebd9 ("ARM: 9025/1: Kconfig: CPU_BIG_ENDIAN
> depends on !LD_IS_LLD"), ld.lld does not support aarch64 big endian,
> leading to the following build error when CONFIG_CPU_BIG_ENDIAN is
> selected:
>
> ld.lld: error: unknown emulation: aarch64linuxb

While this is the original error message I reported, I think it would be
better to explain that lld actually does support linking big-endian
kernels but they don't boot, for unknown reasons.

I can send a patch to address the build error and mark big-endian
as "depends on !LD_IS_LLD || COMPILE_TEST" to reflect that
and help with randconfig testing.

        Arnd

Re: [PATCH] arm64: Make CPU_BIG_ENDIAN depend on !LD_IS_LLD

[Nathan Chancellor]

On Tue, Feb 02, 2021 at 09:04:34AM +0100, Arnd Bergmann wrote:
> On Tue, Feb 2, 2021 at 3:25 AM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > Similar to commit 28187dc8ebd9 ("ARM: 9025/1: Kconfig: CPU_BIG_ENDIAN
> > depends on !LD_IS_LLD"), ld.lld does not support aarch64 big endian,
> > leading to the following build error when CONFIG_CPU_BIG_ENDIAN is
> > selected:
> >
> > ld.lld: error: unknown emulation: aarch64linuxb
> 
> While this is the original error message I reported, I think it would be
> better to explain that lld actually does support linking big-endian
> kernels but they don't boot, for unknown reasons.

That statement seems to contradict what Peter Smith says:

https://github.com/ClangBuiltLinux/linux/issues/1288#issuecomment-770693582

https://reviews.llvm.org/D58655#1410282

> I can send a patch to address the build error and mark big-endian
> as "depends on !LD_IS_LLD || COMPILE_TEST" to reflect that
> and help with randconfig testing.

I have no strong opinion on handling this though.

Cheers,
Nathan

Re: [PATCH] arm64: Make CPU_BIG_ENDIAN depend on !LD_IS_LLD

[Arnd Bergmann]

On Tue, Feb 2, 2021 at 8:51 PM Nathan Chancellor <nathan@kernel.org> wrote:
> On Tue, Feb 02, 2021 at 09:04:34AM +0100, Arnd Bergmann wrote:
> > On Tue, Feb 2, 2021 at 3:25 AM Nathan Chancellor <nathan@kernel.org> wrote:
> > >
> > > Similar to commit 28187dc8ebd9 ("ARM: 9025/1: Kconfig: CPU_BIG_ENDIAN
> > > depends on !LD_IS_LLD"), ld.lld does not support aarch64 big endian,
> > > leading to the following build error when CONFIG_CPU_BIG_ENDIAN is
> > > selected:
> > >
> > > ld.lld: error: unknown emulation: aarch64linuxb
> >
> > While this is the original error message I reported, I think it would be
> > better to explain that lld actually does support linking big-endian
> > kernels but they don't boot, for unknown reasons.
>
> That statement seems to contradict what Peter Smith says:
>
> https://github.com/ClangBuiltLinux/linux/issues/1288#issuecomment-770693582
>
> https://reviews.llvm.org/D58655#1410282

Right, adding Peter to Cc for clarification. Peter, what I can tell from
experiments is that lld behaves just you describe it should, and (aside from
the command like flag handling) like ld.bfd, so I'm not sure if there are
any known deficiencies or just minor bugs when dealing with the kernel.

FWIW, I have now built a big-endian musl C library and a helloworld
binary, which I linked with ld.lld and successfully tested using
qemu-aarch64_be-static

      Arnd

> > I can send a patch to address the build error and mark big-endian
> > as "depends on !LD_IS_LLD || COMPILE_TEST" to reflect that
> > and help with randconfig testing.
>
> I have no strong opinion on handling this though.

RE: [PATCH] arm64: Make CPU_BIG_ENDIAN depend on !LD_IS_LLD

[Peter Smith]

> -----Original Message-----
> From: Arnd Bergmann <arnd@kernel.org>
> Sent: 02 February 2021 20:44
> To: Nathan Chancellor <nathan@kernel.org>
> Cc: Catalin Marinas <Catalin.Marinas@arm.com>; Will Deacon
> <will@kernel.org>; Linux ARM <linux-arm-kernel@lists.infradead.org>; linux-
> kernel@vger.kernel.org; Nick Desaulniers <ndesaulniers@google.com>;
> clang-built-linux <clang-built-linux@googlegroups.com>; Arnd Bergmann
> <arnd@arndb.de>; Peter Smith <Peter.Smith@arm.com>
> Subject: Re: [PATCH] arm64: Make CPU_BIG_ENDIAN depend on !LD_IS_LLD
> 
> On Tue, Feb 2, 2021 at 8:51 PM Nathan Chancellor <nathan@kernel.org>
> wrote:
> > On Tue, Feb 02, 2021 at 09:04:34AM +0100, Arnd Bergmann wrote:
> > > On Tue, Feb 2, 2021 at 3:25 AM Nathan Chancellor <nathan@kernel.org>
> wrote:
> > > >
> > > > Similar to commit 28187dc8ebd9 ("ARM: 9025/1: Kconfig:
> > > > CPU_BIG_ENDIAN depends on !LD_IS_LLD"), ld.lld does not support
> > > > aarch64 big endian, leading to the following build error when
> > > > CONFIG_CPU_BIG_ENDIAN is
> > > > selected:
> > > >
> > > > ld.lld: error: unknown emulation: aarch64linuxb
> > >
> > > While this is the original error message I reported, I think it
> > > would be better to explain that lld actually does support linking
> > > big-endian kernels but they don't boot, for unknown reasons.
> >
> > That statement seems to contradict what Peter Smith says:
> >
> > https://github.com/ClangBuiltLinux/linux/issues/1288#issuecomment-7706
> > 93582
> >
> > https://reviews.llvm.org/D58655#1410282
> 
> Right, adding Peter to Cc for clarification. Peter, what I can tell from
> experiments is that lld behaves just you describe it should, and (aside from
> the command like flag handling) like ld.bfd, so I'm not sure if there are any
> known deficiencies or just minor bugs when dealing with the kernel.
> 

If an emulation (-m) is present then current LLD maps all known Arm and AArch emulations to little endian
https://github.com/llvm/llvm-project/blob/main/lld/ELF/Driver.cpp#L147 

In this case we'll get an error message with a big-endian object.
ld.lld: error: be.o is incompatible with aarch64linux

If there are no emulations present then LLD will auto-detect big-endian from the first object file.
https://github.com/llvm/llvm-project/blob/main/lld/ELF/Driver.cpp#L1467

This will switch the generic LLD write functions to write big-endian. This will mean that any write to instructions will need to use explicit write32le and any data relocations will need to use write32/write64 as appropriate and not use the explicit le versions. The relocations for instructions look to be using explicit le writes as we'd want them to. I've spotted at least one case of using write64le for what looks like data
https://github.com/llvm/llvm-project/blob/main/lld/ELF/Arch/AArch64.cpp#L198

I've not know anyone try LLD on big-endian before so there are no known deficiencies. I expect that without an active search and tests there will be some problems. I think every write will need looking at, if it is to instructions it needs to explicitly use le, if to data (that could be either little or big endian) it must not force little endian.

Peter


> FWIW, I have now built a big-endian musl C library and a helloworld binary,
> which I linked with ld.lld and successfully tested using qemu-aarch64_be-
> static
> 
>       Arnd
> 
> > > I can send a patch to address the build error and mark big-endian as
> > > "depends on !LD_IS_LLD || COMPILE_TEST" to reflect that and help
> > > with randconfig testing.
> >
> > I have no strong opinion on handling this though.

Re: [PATCH] arm64: Make CPU_BIG_ENDIAN depend on !LD_IS_LLD

[Nathan Chancellor]

On Mon, Feb 01, 2021 at 07:24:42PM -0700, Nathan Chancellor wrote:
> Similar to commit 28187dc8ebd9 ("ARM: 9025/1: Kconfig: CPU_BIG_ENDIAN
> depends on !LD_IS_LLD"), ld.lld does not support aarch64 big endian,
> leading to the following build error when CONFIG_CPU_BIG_ENDIAN is
> selected:
> 
> ld.lld: error: unknown emulation: aarch64linuxb
> 
> There are not currently plans to implement big endian support for
> aarch64 in ld.lld but if it should be supported in the future, this
> symbol can depend on the version that first supports it. In the
> meantime, prevent this symbol from being selected to avoid these type
> of build errors.
> 
> While we are here, the indentation of this symbol used spaces since its
> introduction in commit a872013d6d03 ("arm64: kconfig: allow
> CPU_BIG_ENDIAN to be selected"). Change it to tabs to be consistent with
> kernel coding style.
> 
> Link: https://github.com/ClangBuiltLinux/linux/issues/380
> Link: https://github.com/ClangBuiltLinux/linux/issues/1288
> Reported-by: Arnd Bergmann <arnd@arndb.de>
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>

Fangrui has pushed proper aarch64linuxb and elf64-bigaarch64 for ld.lld:

https://reviews.llvm.org/D96211
https://reviews.llvm.org/D96188

This patch is needed for OUTPUT_FORMAT in the vDSO to work properly:

https://reviews.llvm.org/D96214

Once these are all merged into LLVM 13, I will update this to be:

depends on !LD_IS_LLD || LLD_VERSION >= 130000

Cheers,
Nathan

> ---
>  arch/arm64/Kconfig | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index f39568b28ec1..5ddf9fe7b4ee 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -952,8 +952,9 @@ choice
>  	  that is selected here.
>  
>  config CPU_BIG_ENDIAN
> -       bool "Build big-endian kernel"
> -       help
> +	bool "Build big-endian kernel"
> +	depends on !LD_IS_LLD
> +	help
>  	  Say Y if you plan on running a kernel with a big-endian userspace.
>  
>  config CPU_LITTLE_ENDIAN
> 
> base-commit: 88bb507a74ea7d75fa49edd421eaa710a7d80598
> -- 
> 2.30.0
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2] arm64: Make CPU_BIG_ENDIAN depend on ld.bfd or ld.lld 13.0.0+

[Nathan Chancellor]

Similar to commit 28187dc8ebd9 ("ARM: 9025/1: Kconfig: CPU_BIG_ENDIAN
depends on !LD_IS_LLD"), ld.lld prior to 13.0.0 does not properly
support aarch64 big endian, leading to the following build error when
CONFIG_CPU_BIG_ENDIAN is selected:

ld.lld: error: unknown emulation: aarch64linuxb

This has been resolved in LLVM 13. To avoid errors like this, only allow
CONFIG_CPU_BIG_ENDIAN to be selected if using ld.bfd or ld.lld 13.0.0
and newer.

While we are here, the indentation of this symbol used spaces since its
introduction in commit a872013d6d03 ("arm64: kconfig: allow
CPU_BIG_ENDIAN to be selected"). Change it to tabs to be consistent with
kernel coding style.

Link: https://github.com/ClangBuiltLinux/linux/issues/380
Link: https://github.com/ClangBuiltLinux/linux/issues/1288
Link: https://github.com/llvm/llvm-project/commit/7605a9a009b5fa3bdac07e3131c8d82f6d08feb7
Link: https://github.com/llvm/llvm-project/commit/eea34aae2e74e9b6fbdd5b95f479bc7f397bf387
Reported-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
---

v1 -> v2:

* Conditionally disable CONFIG_CPU_BIG_ENDIAN for older versions of
  ld.lld now that proper support has been added to ld.lld 13+ (thanks to
  the quick work of Fangrui Song).

 arch/arm64/Kconfig | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index f39568b28ec1..912da4e2ab59 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -952,8 +952,9 @@ choice
 	  that is selected here.
 
 config CPU_BIG_ENDIAN
-       bool "Build big-endian kernel"
-       help
+	bool "Build big-endian kernel"
+	depends on !LD_IS_LLD || LLD_VERSION >= 130000
+	help
 	  Say Y if you plan on running a kernel with a big-endian userspace.
 
 config CPU_LITTLE_ENDIAN

base-commit: 92bf22614b21a2706f4993b278017e437f7785b3
-- 
2.30.1

Re: [PATCH v2] arm64: Make CPU_BIG_ENDIAN depend on ld.bfd or ld.lld 13.0.0+

[Nick Desaulniers]

On Mon, Feb 8, 2021 at 4:58 PM Nathan Chancellor <nathan@kernel.org> wrote:
>
> Similar to commit 28187dc8ebd9 ("ARM: 9025/1: Kconfig: CPU_BIG_ENDIAN
> depends on !LD_IS_LLD"), ld.lld prior to 13.0.0 does not properly
> support aarch64 big endian, leading to the following build error when
> CONFIG_CPU_BIG_ENDIAN is selected:
>
> ld.lld: error: unknown emulation: aarch64linuxb
>
> This has been resolved in LLVM 13. To avoid errors like this, only allow
> CONFIG_CPU_BIG_ENDIAN to be selected if using ld.bfd or ld.lld 13.0.0
> and newer.
>
> While we are here, the indentation of this symbol used spaces since its
> introduction in commit a872013d6d03 ("arm64: kconfig: allow
> CPU_BIG_ENDIAN to be selected"). Change it to tabs to be consistent with
> kernel coding style.
>
> Link: https://github.com/ClangBuiltLinux/linux/issues/380
> Link: https://github.com/ClangBuiltLinux/linux/issues/1288
> Link: https://github.com/llvm/llvm-project/commit/7605a9a009b5fa3bdac07e3131c8d82f6d08feb7
> Link: https://github.com/llvm/llvm-project/commit/eea34aae2e74e9b6fbdd5b95f479bc7f397bf387
> Reported-by: Arnd Bergmann <arnd@arndb.de>
> Signed-off-by: Nathan Chancellor <nathan@kernel.org>

LGTM, thanks for implementing BE support for aarch64 in LLD, Fangrui!
(And Nathan for the kernel patch+testing)
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>

> ---
>
> v1 -> v2:
>
> * Conditionally disable CONFIG_CPU_BIG_ENDIAN for older versions of
>   ld.lld now that proper support has been added to ld.lld 13+ (thanks to
>   the quick work of Fangrui Song).
>
>  arch/arm64/Kconfig | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index f39568b28ec1..912da4e2ab59 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -952,8 +952,9 @@ choice
>           that is selected here.
>
>  config CPU_BIG_ENDIAN
> -       bool "Build big-endian kernel"
> -       help
> +       bool "Build big-endian kernel"
> +       depends on !LD_IS_LLD || LLD_VERSION >= 130000
> +       help
>           Say Y if you plan on running a kernel with a big-endian userspace.
>
>  config CPU_LITTLE_ENDIAN
>
> base-commit: 92bf22614b21a2706f4993b278017e437f7785b3
> --
> 2.30.1
>


-- 
Thanks,
~Nick Desaulniers

Re: [PATCH v2] arm64: Make CPU_BIG_ENDIAN depend on ld.bfd or ld.lld 13.0.0+

[Will Deacon]

On Mon, 8 Feb 2021 17:57:20 -0700, Nathan Chancellor wrote:
> Similar to commit 28187dc8ebd9 ("ARM: 9025/1: Kconfig: CPU_BIG_ENDIAN
> depends on !LD_IS_LLD"), ld.lld prior to 13.0.0 does not properly
> support aarch64 big endian, leading to the following build error when
> CONFIG_CPU_BIG_ENDIAN is selected:
> 
> ld.lld: error: unknown emulation: aarch64linuxb
> 
> [...]

Applied to arm64 (for-next/misc), thanks!

[1/1] arm64: Make CPU_BIG_ENDIAN depend on ld.bfd or ld.lld 13.0.0+
      https://git.kernel.org/arm64/c/e9c6deee00e9

Cheers,
-- 
Will

https://fixes.arm64.dev
https://next.arm64.dev
https://will.arm64.dev