CONFIG_ARCH_SUPPORTS_INT128 for AArch64

CONFIG_ARCH_SUPPORTS_INT128 for AArch64

[Jason A. Donenfeld]

Hi folks,

Older versions of gcc are horrible for int128_t on aarch64, emitting a
__multi3 call, which is slow and defeats the purpose. However, more
recent versions of gcc (since 5, IIRC), do the right thing and produce
the proper inlined instructions.

Do we have the infrastructure available to test for this, and
conditionally set CONFIG_ARCH_SUPPORTS_INT128 if we do?

Thanks,
Jason

Re: CONFIG_ARCH_SUPPORTS_INT128 for AArch64

[Mark Rutland]

On Tue, Oct 31, 2017 at 11:17:52AM +0100, Jason A. Donenfeld wrote:
> Hi folks,
> 
> Older versions of gcc are horrible for int128_t on aarch64, emitting a
> __multi3 call, which is slow and defeats the purpose. However, more
> recent versions of gcc (since 5, IIRC), do the right thing and produce
> the proper inlined instructions.
> 
> Do we have the infrastructure available to test for this, and
> conditionally set CONFIG_ARCH_SUPPORTS_INT128 if we do?

Judging by Documentation/kbuild/makefiles.txt, we could do something
like the below in the arm64 Makefile:

cflags-y += $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128)

Note that I havent tested this at all.

Thanks,

Re: CONFIG_ARCH_SUPPORTS_INT128 for AArch64

[Jason A. Donenfeld]

Hi Mark,

On Tue, Oct 31, 2017 at 11:43 AM, Mark Rutland <mark.rutland@arm.com> wrote:
> Judging by Documentation/kbuild/makefiles.txt, we could do something
> like the below in the arm64 Makefile:
>
> cflags-y += $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128)

Okay, in that case, I'll re-research the gcc commit that made int128
work properly on aarch64, confirm the release version, and then submit
a patch doing this.

Jason

[PATCH] arm64: support __int128 on gcc 5+

[Jason A. Donenfeld]

Versions of gcc prior to gcc 5 emitted a __multi3 function call when
dealing with TI types, resulting in failures when trying to link to
libgcc, and more generally, horrible performance. However, since gcc 5,
the compiler supports actually emitting fast instructions, which means
we can at long last enable this option and receive the speedups.

The gcc commit that added proper Aarch64 support is:
https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d

This commit appears to be part of the gcc 5 release.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
---
 arch/arm64/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 939b310913cf..1f8a0fec6998 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -53,6 +53,8 @@ KBUILD_AFLAGS	+= $(lseinstr) $(brokengasinst)
 KBUILD_CFLAGS	+= $(call cc-option,-mabi=lp64)
 KBUILD_AFLAGS	+= $(call cc-option,-mabi=lp64)
 
+KBUILD_CFLAGS	+= $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128)
+
 ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
 KBUILD_CPPFLAGS	+= -mbig-endian
 CHECKFLAGS	+= -D__AARCH64EB__
-- 
2.14.2

Re: [PATCH] arm64: support __int128 on gcc 5+

[Will Deacon]

On Tue, Oct 31, 2017 at 12:43:19PM +0100, Jason A. Donenfeld wrote:
> Versions of gcc prior to gcc 5 emitted a __multi3 function call when
> dealing with TI types, resulting in failures when trying to link to
> libgcc, and more generally, horrible performance. However, since gcc 5,
> the compiler supports actually emitting fast instructions, which means
> we can at long last enable this option and receive the speedups.
> 
> The gcc commit that added proper Aarch64 support is:
> https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d
> 
> This commit appears to be part of the gcc 5 release.
> 
> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
> ---
>  arch/arm64/Makefile | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 939b310913cf..1f8a0fec6998 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -53,6 +53,8 @@ KBUILD_AFLAGS	+= $(lseinstr) $(brokengasinst)
>  KBUILD_CFLAGS	+= $(call cc-option,-mabi=lp64)
>  KBUILD_AFLAGS	+= $(call cc-option,-mabi=lp64)
>  
> +KBUILD_CFLAGS	+= $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128)
> +
>  ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
>  KBUILD_CPPFLAGS	+= -mbig-endian
>  CHECKFLAGS	+= -D__AARCH64EB__

Which code in the kernel actually uses 128-bit types directly? I know we
have some unfortunate occurences in our headers (including uapi) for the
vector registers, but I thought we generally used asm or copy routines to
access those.

Are you seeing a performance issue without this?

Will

Re: [PATCH] arm64: support __int128 on gcc 5+

[Jason A. Donenfeld]

Hi Will,

On Tue, Oct 31, 2017 at 12:51 PM, Will Deacon <will.deacon@arm.com> wrote:
> Which code in the kernel actually uses 128-bit types directly? I know we
> have some unfortunate occurences in our headers (including uapi) for the
> vector registers, but I thought we generally used asm or copy routines to
> access those.

math64.h provides it, and various things throughout use those
functions. Notably, the scheduler and kvm use those the __int128
functions. There's also an elliptic curve implementation that uses it,
which makes a big difference. And soon I'll be adding an
implementation of curve25519 that will make heavy use of these
instructions for significant speedups as well.

Generally, adding this CONFIG_ARCH key is a hint for current and
future bits of code, so that they can use the faster 128-bit
implementations when available. Not providing it when it's there and
available would be silly. This wasn't originally added to the
architecture, because when the Kconfig symbol was added, gcc didn't
have sane support for it on aarch64. But now it does, so let's support
it.

Jason

Re: [PATCH] arm64: support __int128 on gcc 5+

[Will Deacon]

Hi Jason,

On Tue, Oct 31, 2017 at 12:57:29PM +0100, Jason A. Donenfeld wrote:
> On Tue, Oct 31, 2017 at 12:51 PM, Will Deacon <will.deacon@arm.com> wrote:
> > Which code in the kernel actually uses 128-bit types directly? I know we
> > have some unfortunate occurences in our headers (including uapi) for the
> > vector registers, but I thought we generally used asm or copy routines to
> > access those.
> 
> math64.h provides it, and various things throughout use those
> functions. Notably, the scheduler and kvm use those the __int128
> functions. There's also an elliptic curve implementation that uses it,
> which makes a big difference. And soon I'll be adding an
> implementation of curve25519 that will make heavy use of these
> instructions for significant speedups as well.

Thanks for the info. It does indeed look like mul_u64_u64_shr improves
pretty drastically with this patch, so that sounds good to me. We don't
ever call that function on arm64, but hey.

> Generally, adding this CONFIG_ARCH key is a hint for current and
> future bits of code, so that they can use the faster 128-bit
> implementations when available. Not providing it when it's there and
> available would be silly. This wasn't originally added to the
> architecture, because when the Kconfig symbol was added, gcc didn't
> have sane support for it on aarch64. But now it does, so let's support
> it.

Sure, I'll wait for your patch that matches the relevant compiler versions.

Will

Re: [PATCH] arm64: support __int128 on gcc 5+

[Jason A. Donenfeld]

On Tue, Oct 31, 2017 at 1:17 PM, Will Deacon <will.deacon@arm.com> wrote:
>
> Sure, I'll wait for your patch that matches the relevant compiler versions.

That's the patch in this thread:
http://lists.infradead.org/pipermail/linux-arm-kernel/2017-October/539943.html

As mentioned in there, gcc does the right thing for 5 and up:
https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d

Re: [PATCH] arm64: support __int128 on gcc 5+

[Will Deacon]

Hi Jason,

On Tue, Oct 31, 2017 at 12:43:19PM +0100, Jason A. Donenfeld wrote:
> Versions of gcc prior to gcc 5 emitted a __multi3 function call when
> dealing with TI types, resulting in failures when trying to link to
> libgcc, and more generally, horrible performance. However, since gcc 5,
> the compiler supports actually emitting fast instructions, which means
> we can at long last enable this option and receive the speedups.
> 
> The gcc commit that added proper Aarch64 support is:
> https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d
> 
> This commit appears to be part of the gcc 5 release.
> 
> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
> ---
>  arch/arm64/Makefile | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 939b310913cf..1f8a0fec6998 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -53,6 +53,8 @@ KBUILD_AFLAGS	+= $(lseinstr) $(brokengasinst)
>  KBUILD_CFLAGS	+= $(call cc-option,-mabi=lp64)
>  KBUILD_AFLAGS	+= $(call cc-option,-mabi=lp64)
>  
> +KBUILD_CFLAGS	+= $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128)
> +
>  ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
>  KBUILD_CPPFLAGS	+= -mbig-endian
>  CHECKFLAGS	+= -D__AARCH64EB__

This breaks an allmodconfig build:

lib/ubsan.o: In function `get_signed_val':
ubsan.c:(.text.unlikely+0x13c): undefined reference to `__ashlti3'
ubsan.c:(.text.unlikely+0x144): undefined reference to `__ashrti3'
make: *** [vmlinux] Error 1

so I'm going to drop it for now. My toolchain is:

gcc version 7.1.1 20170707 (Linaro GCC 7.1-2017.08)

Will

[PATCH v2] arm64: support __int128 on gcc 5+

[Jason A. Donenfeld]

Versions of gcc prior to gcc 5 emitted a __multi3 function call when
dealing with TI types, resulting in failures when trying to link to
libgcc, and more generally, bad performance. However, since gcc 5,
the compiler supports actually emitting fast instructions, which means
we can at long last enable this option and receive the speedups.

The gcc commit that added proper Aarch64 support is:
https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d
This commit appears to be part of the gcc 5 release.

There are still a few instructions, such as __ashlti3 and __ashrti3,
that require libgcc, which is fine. So, we also link to libgcc for these
functions when needed, which is what several other architectures already
have been doing for a long time.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
---
 arch/arm64/Makefile | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 939b310913cf..70c7c0c1bccb 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -53,6 +53,8 @@ KBUILD_AFLAGS	+= $(lseinstr) $(brokengasinst)
 KBUILD_CFLAGS	+= $(call cc-option,-mabi=lp64)
 KBUILD_AFLAGS	+= $(call cc-option,-mabi=lp64)
 
+KBUILD_CFLAGS	+= $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128)
+
 ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
 KBUILD_CPPFLAGS	+= -mbig-endian
 CHECKFLAGS	+= -D__AARCH64EB__
@@ -108,7 +110,7 @@ core-$(CONFIG_NET) += arch/arm64/net/
 core-$(CONFIG_KVM) += arch/arm64/kvm/
 core-$(CONFIG_XEN) += arch/arm64/xen/
 core-$(CONFIG_CRYPTO) += arch/arm64/crypto/
-libs-y		:= arch/arm64/lib/ $(libs-y)
+libs-y		:= arch/arm64/lib/ $(shell $(CC) $(KBUILD_CFLAGS) -print-libgcc-file-name) $(libs-y)
 core-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
 
 # Default target when executing plain make
-- 
2.14.2

Re: [PATCH v2] arm64: support __int128 on gcc 5+

[Will Deacon]

Hi Jason,

[+Ard]

On Thu, Nov 02, 2017 at 06:43:22PM +0100, Jason A. Donenfeld wrote:
> Versions of gcc prior to gcc 5 emitted a __multi3 function call when
> dealing with TI types, resulting in failures when trying to link to
> libgcc, and more generally, bad performance. However, since gcc 5,
> the compiler supports actually emitting fast instructions, which means
> we can at long last enable this option and receive the speedups.
> 
> The gcc commit that added proper Aarch64 support is:
> https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d
> This commit appears to be part of the gcc 5 release.
> 
> There are still a few instructions, such as __ashlti3 and __ashrti3,
> that require libgcc, which is fine. So, we also link to libgcc for these
> functions when needed, which is what several other architectures already
> have been doing for a long time.
> 
> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
> ---
>  arch/arm64/Makefile | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)

We used to link against libgcc way back when, but that dependency was
removed in commit d67703a8a69e ("arm64: kill off the libgcc dependency")
and I'm really not keen to add it back. I also think that there might
be licensing concerns if you link against it and make use of GCC plugins,
but IANAL.

Shouldn't we just provide our own implementations of __ashlti3 and
__ashrti3 instead?

Cheers,

Will

> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 939b310913cf..70c7c0c1bccb 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -53,6 +53,8 @@ KBUILD_AFLAGS	+= $(lseinstr) $(brokengasinst)
>  KBUILD_CFLAGS	+= $(call cc-option,-mabi=lp64)
>  KBUILD_AFLAGS	+= $(call cc-option,-mabi=lp64)
>  
> +KBUILD_CFLAGS	+= $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128)
> +
>  ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
>  KBUILD_CPPFLAGS	+= -mbig-endian
>  CHECKFLAGS	+= -D__AARCH64EB__
> @@ -108,7 +110,7 @@ core-$(CONFIG_NET) += arch/arm64/net/
>  core-$(CONFIG_KVM) += arch/arm64/kvm/
>  core-$(CONFIG_XEN) += arch/arm64/xen/
>  core-$(CONFIG_CRYPTO) += arch/arm64/crypto/
> -libs-y		:= arch/arm64/lib/ $(libs-y)
> +libs-y		:= arch/arm64/lib/ $(shell $(CC) $(KBUILD_CFLAGS) -print-libgcc-file-name) $(libs-y)
>  core-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
>  
>  # Default target when executing plain make
> -- 
> 2.14.2
> 

Re: [PATCH v2] arm64: support __int128 on gcc 5+

[Ard Biesheuvel]

> On 3 Nov 2017, at 13:42, Will Deacon <will.deacon@arm.com> wrote:
> 
> Hi Jason,
> 
> [+Ard]
> 
>> On Thu, Nov 02, 2017 at 06:43:22PM +0100, Jason A. Donenfeld wrote:
>> Versions of gcc prior to gcc 5 emitted a __multi3 function call when
>> dealing with TI types, resulting in failures when trying to link to
>> libgcc, and more generally, bad performance. However, since gcc 5,
>> the compiler supports actually emitting fast instructions, which means
>> we can at long last enable this option and receive the speedups.
>> 
>> The gcc commit that added proper Aarch64 support is:
>> https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d
>> This commit appears to be part of the gcc 5 release.
>> 
>> There are still a few instructions, such as __ashlti3 and __ashrti3,
>> that require libgcc, which is fine. So, we also link to libgcc for these
>> functions when needed, which is what several other architectures already
>> have been doing for a long time.
>> 
>> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
>> ---
>> arch/arm64/Makefile | 4 +++-
>> 1 file changed, 3 insertions(+), 1 deletion(-)
> 
> We used to link against libgcc way back when, but that dependency was
> removed in commit d67703a8a69e ("arm64: kill off the libgcc dependency")
> and I'm really not keen to add it back. I also think that there might
> be licensing concerns if you link against it and make use of GCC plugins,
> but IANAL.
> 

Is it guaranteed that libgcc doesn’t use neon/vfp? We can’t use it in the kernel otherwise.


> Shouldn't we just provide our own implementations of __ashlti3 and
> __ashrti3 instead?
> 
> Cheers,
> 
> Will
> 
>> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
>> index 939b310913cf..70c7c0c1bccb 100644
>> --- a/arch/arm64/Makefile
>> +++ b/arch/arm64/Makefile
>> @@ -53,6 +53,8 @@ KBUILD_AFLAGS    += $(lseinstr) $(brokengasinst)
>> KBUILD_CFLAGS    += $(call cc-option,-mabi=lp64)
>> KBUILD_AFLAGS    += $(call cc-option,-mabi=lp64)
>> 
>> +KBUILD_CFLAGS    += $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128)
>> +
>> ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
>> KBUILD_CPPFLAGS    += -mbig-endian
>> CHECKFLAGS    += -D__AARCH64EB__
>> @@ -108,7 +110,7 @@ core-$(CONFIG_NET) += arch/arm64/net/
>> core-$(CONFIG_KVM) += arch/arm64/kvm/
>> core-$(CONFIG_XEN) += arch/arm64/xen/
>> core-$(CONFIG_CRYPTO) += arch/arm64/crypto/
>> -libs-y        := arch/arm64/lib/ $(libs-y)
>> +libs-y        := arch/arm64/lib/ $(shell $(CC) $(KBUILD_CFLAGS) -print-libgcc-file-name) $(libs-y)
>> core-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a
>> 
>> # Default target when executing plain make
>> -- 
>> 2.14.2
>> 

Re: [PATCH v2] arm64: support __int128 on gcc 5+

[Jason A. Donenfeld]

On Fri, Nov 3, 2017 at 2:42 PM, Will Deacon <will.deacon@arm.com> wrote:
> We used to link against libgcc way back when, but that dependency was
> removed in commit d67703a8a69e ("arm64: kill off the libgcc dependency")
> and I'm really not keen to add it back. I also think that there might
> be licensing concerns if you link against it and make use of GCC plugins,
> but IANAL.

Considering many other architectures link to it, that seems strange to me...

>
> Shouldn't we just provide our own implementations of __ashlti3 and
> __ashrti3 instead?

I just coded up an implementation. I'm testing it now and I'll have v3
for you shortly.

Jason

[PATCH v3] arm64: support __int128 on gcc 5+

[Jason A. Donenfeld]

Versions of gcc prior to gcc 5 emitted a __multi3 function call when
dealing with TI types, resulting in failures when trying to link to
libgcc, and more generally, bad performance. However, since gcc 5,
the compiler supports actually emitting fast instructions, which means
we can at long last enable this option and receive the speedups.

The gcc commit that added proper Aarch64 support is:
https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d
This commit appears to be part of the gcc 5 release.

There are still a few instructions, __ashlti3 and __ashrti3, which
require libgcc, which is fine. Rather than linking to libgcc, we
simply provide them ourselves, since they're not that complicated.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
---
Changes v2->v3:
  - We now just provide trivial implementations for the missing
    functions, rather than linking to libgcc.

 arch/arm64/Makefile      |  2 ++
 arch/arm64/lib/Makefile  |  2 +-
 arch/arm64/lib/tishift.S | 59 ++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 62 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm64/lib/tishift.S

diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 939b310913cf..1f8a0fec6998 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -53,6 +53,8 @@ KBUILD_AFLAGS	+= $(lseinstr) $(brokengasinst)
 KBUILD_CFLAGS	+= $(call cc-option,-mabi=lp64)
 KBUILD_AFLAGS	+= $(call cc-option,-mabi=lp64)
 
+KBUILD_CFLAGS	+= $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128)
+
 ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
 KBUILD_CPPFLAGS	+= -mbig-endian
 CHECKFLAGS	+= -D__AARCH64EB__
diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile
index a0abc142c92b..55bdb01f1ea6 100644
--- a/arch/arm64/lib/Makefile
+++ b/arch/arm64/lib/Makefile
@@ -2,7 +2,7 @@ lib-y		:= bitops.o clear_user.o delay.o copy_from_user.o	\
 		   copy_to_user.o copy_in_user.o copy_page.o		\
 		   clear_page.o memchr.o memcpy.o memmove.o memset.o	\
 		   memcmp.o strcmp.o strncmp.o strlen.o strnlen.o	\
-		   strchr.o strrchr.o
+		   strchr.o strrchr.o tishift.o
 
 # Tell the compiler to treat all general purpose registers (with the
 # exception of the IP registers, which are already handled by the caller
diff --git a/arch/arm64/lib/tishift.S b/arch/arm64/lib/tishift.S
new file mode 100644
index 000000000000..47b6f7ee2b11
--- /dev/null
+++ b/arch/arm64/lib/tishift.S
@@ -0,0 +1,59 @@
+/*
+ * Copyright (C) 2017 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Resreved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+
+ENTRY(__ashlti3)
+	cbz	x2, 1f
+	mov	x3, #64
+	sub	x3, x3, x2
+	cmp	x3, #0
+	b.le	2f
+	lsl	x1, x1, x2
+	lsr	x3, x0, x3
+	lsl	x2, x0, x2
+	orr	x1, x1, x3
+	mov	x0, x2
+1:
+	ret
+2:
+	neg	w1, w3
+	mov	x2, #0
+	lsl	x1, x0, x1
+	mov	x0, x2
+	ret
+ENDPROC(__ashlti3)
+
+ENTRY(__ashrti3)
+	cbz	x2, 3f
+	mov	x3, #64
+	sub	x3, x3, x2
+	cmp	x3, #0
+	b.le	4f
+	lsr	x0, x0, x2
+	lsl	x3, x1, x3
+	asr	x2, x1, x2
+	orr	x0, x0, x3
+	mov	x1, x2
+3:
+	ret
+4:
+	neg	w0, w3
+	asr	x2, x1, #63
+	asr	x0, x1, x0
+	mov	x1, x2
+	ret
+ENDPROC(__ashrti3)
-- 
2.14.2

[PATCH v4] arm64: support __int128 on gcc 5+

[Jason A. Donenfeld]

Versions of gcc prior to gcc 5 emitted a __multi3 function call when
dealing with TI types, resulting in failures when trying to link to
libgcc, and more generally, bad performance. However, since gcc 5,
the compiler supports actually emitting fast instructions, which means
we can at long last enable this option and receive the speedups.

The gcc commit that added proper Aarch64 support is:
https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d
This commit appears to be part of the gcc 5 release.

There are still a few instructions, __ashlti3 and __ashrti3, which
require libgcc, which is fine. Rather than linking to libgcc, we
simply provide them ourselves, since they're not that complicated.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
---
Changes v3->v4:
  - Typo in comment
  - Relabel second function

 arch/arm64/Makefile      |  2 ++
 arch/arm64/lib/Makefile  |  2 +-
 arch/arm64/lib/tishift.S | 59 ++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 62 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm64/lib/tishift.S

diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 939b310913cf..1f8a0fec6998 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -53,6 +53,8 @@ KBUILD_AFLAGS	+= $(lseinstr) $(brokengasinst)
 KBUILD_CFLAGS	+= $(call cc-option,-mabi=lp64)
 KBUILD_AFLAGS	+= $(call cc-option,-mabi=lp64)
 
+KBUILD_CFLAGS	+= $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128)
+
 ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
 KBUILD_CPPFLAGS	+= -mbig-endian
 CHECKFLAGS	+= -D__AARCH64EB__
diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile
index a0abc142c92b..55bdb01f1ea6 100644
--- a/arch/arm64/lib/Makefile
+++ b/arch/arm64/lib/Makefile
@@ -2,7 +2,7 @@ lib-y		:= bitops.o clear_user.o delay.o copy_from_user.o	\
 		   copy_to_user.o copy_in_user.o copy_page.o		\
 		   clear_page.o memchr.o memcpy.o memmove.o memset.o	\
 		   memcmp.o strcmp.o strncmp.o strlen.o strnlen.o	\
-		   strchr.o strrchr.o
+		   strchr.o strrchr.o tishift.o
 
 # Tell the compiler to treat all general purpose registers (with the
 # exception of the IP registers, which are already handled by the caller
diff --git a/arch/arm64/lib/tishift.S b/arch/arm64/lib/tishift.S
new file mode 100644
index 000000000000..bffe03c478a5
--- /dev/null
+++ b/arch/arm64/lib/tishift.S
@@ -0,0 +1,59 @@
+/*
+ * Copyright (C) 2017 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+
+ENTRY(__ashlti3)
+	cbz	x2, 1f
+	mov	x3, #64
+	sub	x3, x3, x2
+	cmp	x3, #0
+	b.le	2f
+	lsl	x1, x1, x2
+	lsr	x3, x0, x3
+	lsl	x2, x0, x2
+	orr	x1, x1, x3
+	mov	x0, x2
+1:
+	ret
+2:
+	neg	w1, w3
+	mov	x2, #0
+	lsl	x1, x0, x1
+	mov	x0, x2
+	ret
+ENDPROC(__ashlti3)
+
+ENTRY(__ashrti3)
+	cbz	x2, 1f
+	mov	x3, #64
+	sub	x3, x3, x2
+	cmp	x3, #0
+	b.le	2f
+	lsr	x0, x0, x2
+	lsl	x3, x1, x3
+	asr	x2, x1, x2
+	orr	x0, x0, x3
+	mov	x1, x2
+1:
+	ret
+2:
+	neg	w0, w3
+	asr	x2, x1, #63
+	asr	x0, x1, x0
+	mov	x1, x2
+	ret
+ENDPROC(__ashrti3)
-- 
2.15.0

Re: [PATCH v4] arm64: support __int128 on gcc 5+

[Catalin Marinas]

On Mon, Nov 06, 2017 at 10:31:51AM +0100, Jason A. Donenfeld wrote:
> Versions of gcc prior to gcc 5 emitted a __multi3 function call when
> dealing with TI types, resulting in failures when trying to link to
> libgcc, and more generally, bad performance. However, since gcc 5,
> the compiler supports actually emitting fast instructions, which means
> we can at long last enable this option and receive the speedups.
> 
> The gcc commit that added proper Aarch64 support is:
> https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d
> This commit appears to be part of the gcc 5 release.
> 
> There are still a few instructions, __ashlti3 and __ashrti3, which
> require libgcc, which is fine. Rather than linking to libgcc, we
> simply provide them ourselves, since they're not that complicated.
> 
> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>

According to Arnd, linux-next (with this patch included) fails to build
with gcc-7 (config: https://pastebin.com/raw/sgvPe96e).

Will is managing the upcoming merging window for arm64 but is travelling
this week. I'll wait for couple of days, he'll probably catch up with
emails but my suggestion is that we revert the patch and push it back
again into -next after the 4.15 merging window.

-- 
Catalin

Re: [PATCH v4] arm64: support __int128 on gcc 5+

[Catalin Marinas]

On Mon, Nov 06, 2017 at 03:59:18PM +0000, Catalin Marinas wrote:
> On Mon, Nov 06, 2017 at 10:31:51AM +0100, Jason A. Donenfeld wrote:
> > Versions of gcc prior to gcc 5 emitted a __multi3 function call when
> > dealing with TI types, resulting in failures when trying to link to
> > libgcc, and more generally, bad performance. However, since gcc 5,
> > the compiler supports actually emitting fast instructions, which means
> > we can at long last enable this option and receive the speedups.
> > 
> > The gcc commit that added proper Aarch64 support is:
> > https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d
> > This commit appears to be part of the gcc 5 release.
> > 
> > There are still a few instructions, __ashlti3 and __ashrti3, which
> > require libgcc, which is fine. Rather than linking to libgcc, we
> > simply provide them ourselves, since they're not that complicated.
> > 
> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
> 
> According to Arnd, linux-next (with this patch included) fails to build
> with gcc-7 (config: https://pastebin.com/raw/sgvPe96e).

Actually, it fails with gcc-6 as well (gcc (Debian 6.3.0-18) 6.3.0
20170516) with this error:

kernel/sched/fair.o: In function `__calc_delta':
fair.c:(.text+0x84c): undefined reference to `__lshrti3'
kernel/time/timekeeping.o: In function `timekeeping_resume':
timekeeping.c:(.text+0x2cac): undefined reference to `__lshrti3'

So I'm for reverting the commit and we should allow more randconfig
tests for the 4.16 merging window.

-- 
Catalin

Re: [PATCH v4] arm64: support __int128 on gcc 5+

[Ard Biesheuvel]

On 6 November 2017 at 16:51, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> Wait, please don't jump to that decision so quickly.
>
> Are you sure the fail is for v4? Will mentioned this with v1, which is whay
> this current v4 is supposed to fix up.
>

It appears your v4 adds __ashlti3() and __ashrti3, whereas the error
is about __lshrti3() being undefined.


> On Nov 7, 2017 01:15, "Catalin Marinas" <catalin.marinas@arm.com> wrote:
>
> On Mon, Nov 06, 2017 at 03:59:18PM +0000, Catalin Marinas wrote:
>> On Mon, Nov 06, 2017 at 10:31:51AM +0100, Jason A. Donenfeld wrote:
>> > Versions of gcc prior to gcc 5 emitted a __multi3 function call when
>> > dealing with TI types, resulting in failures when trying to link to
>> > libgcc, and more generally, bad performance. However, since gcc 5,
>> > the compiler supports actually emitting fast instructions, which means
>> > we can at long last enable this option and receive the speedups.
>> >
>> > The gcc commit that added proper Aarch64 support is:
>> >
>> > https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d
>> > This commit appears to be part of the gcc 5 release.
>> >
>> > There are still a few instructions, __ashlti3 and __ashrti3, which
>> > require libgcc, which is fine. Rather than linking to libgcc, we
>> > simply provide them ourselves, since they're not that complicated.
>> >
>> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
>>
>> According to Arnd, linux-next (with this patch included) fails to build
>> with gcc-7 (config: https://pastebin.com/raw/sgvPe96e).
>
> Actually, it fails with gcc-6 as well (gcc (Debian 6.3.0-18) 6.3.0
> 20170516) with this error:
>
> kernel/sched/fair.o: In function `__calc_delta':
> fair.c:(.text+0x84c): undefined reference to `__lshrti3'
> kernel/time/timekeeping.o: In function `timekeeping_resume':
> timekeeping.c:(.text+0x2cac): undefined reference to `__lshrti3'
>
> So I'm for reverting the commit and we should allow more randconfig
> tests for the 4.16 merging window.
>
> --
> Catalin
>
>

[PATCH v5] arm64: support __int128 on gcc 5+

[Jason A. Donenfeld]

Versions of gcc prior to gcc 5 emitted a __multi3 function call when
dealing with TI types, resulting in failures when trying to link to
libgcc, and more generally, bad performance. However, since gcc 5,
the compiler supports actually emitting fast instructions, which means
we can at long last enable this option and receive the speedups.

The gcc commit that added proper Aarch64 support is:
https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d
This commit appears to be part of the gcc 5 release.

There are still a few instructions, __ashlti3, __ashrti3, and __lshrti3
which require libgcc, which is fine. Rather than linking to libgcc, we
simply provide them ourselves, since they're not that complicated.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
---
Changes v4->v5:
   - Add implementation of __lshrti3. (Note that __lshlti3 is not
     a function.)

 arch/arm64/Makefile      |  2 ++
 arch/arm64/lib/Makefile  |  2 +-
 arch/arm64/lib/tishift.S | 80 ++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 83 insertions(+), 1 deletion(-)
 create mode 100644 arch/arm64/lib/tishift.S

diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 939b310913cf..1f8a0fec6998 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -53,6 +53,8 @@ KBUILD_AFLAGS	+= $(lseinstr) $(brokengasinst)
 KBUILD_CFLAGS	+= $(call cc-option,-mabi=lp64)
 KBUILD_AFLAGS	+= $(call cc-option,-mabi=lp64)
 
+KBUILD_CFLAGS	+= $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128)
+
 ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
 KBUILD_CPPFLAGS	+= -mbig-endian
 CHECKFLAGS	+= -D__AARCH64EB__
diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile
index 9a8cb96555d6..4e696f96451f 100644
--- a/arch/arm64/lib/Makefile
+++ b/arch/arm64/lib/Makefile
@@ -3,7 +3,7 @@ lib-y		:= bitops.o clear_user.o delay.o copy_from_user.o	\
 		   copy_to_user.o copy_in_user.o copy_page.o		\
 		   clear_page.o memchr.o memcpy.o memmove.o memset.o	\
 		   memcmp.o strcmp.o strncmp.o strlen.o strnlen.o	\
-		   strchr.o strrchr.o
+		   strchr.o strrchr.o tishift.o
 
 # Tell the compiler to treat all general purpose registers (with the
 # exception of the IP registers, which are already handled by the caller
diff --git a/arch/arm64/lib/tishift.S b/arch/arm64/lib/tishift.S
new file mode 100644
index 000000000000..d3db9b2cd479
--- /dev/null
+++ b/arch/arm64/lib/tishift.S
@@ -0,0 +1,80 @@
+/*
+ * Copyright (C) 2017 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+
+ENTRY(__ashlti3)
+	cbz	x2, 1f
+	mov	x3, #64
+	sub	x3, x3, x2
+	cmp	x3, #0
+	b.le	2f
+	lsl	x1, x1, x2
+	lsr	x3, x0, x3
+	lsl	x2, x0, x2
+	orr	x1, x1, x3
+	mov	x0, x2
+1:
+	ret
+2:
+	neg	w1, w3
+	mov	x2, #0
+	lsl	x1, x0, x1
+	mov	x0, x2
+	ret
+ENDPROC(__ashlti3)
+
+ENTRY(__ashrti3)
+	cbz	x2, 1f
+	mov	x3, #64
+	sub	x3, x3, x2
+	cmp	x3, #0
+	b.le	2f
+	lsr	x0, x0, x2
+	lsl	x3, x1, x3
+	asr	x2, x1, x2
+	orr	x0, x0, x3
+	mov	x1, x2
+1:
+	ret
+2:
+	neg	w0, w3
+	asr	x2, x1, #63
+	asr	x0, x1, x0
+	mov	x1, x2
+	ret
+ENDPROC(__ashrti3)
+
+ENTRY(__lshrti3)
+	cbz	x2, 1f
+	mov	x3, #64
+	sub	x3, x3, x2
+	cmp	x3, #0
+	b.le	2f
+	lsr	x0, x0, x2
+	lsl	x3, x1, x3
+	lsr	x2, x1, x2
+	orr	x0, x0, x3
+	mov	x1, x2
+1:
+	ret
+2:
+	neg	w0, w3
+	mov	x2, #0
+	lsr	x0, x1, x0
+	mov	x1, x2
+	ret
+ENDPROC(__lshrti3)
-- 
2.15.0

Re: [PATCH v4] arm64: support __int128 on gcc 5+

[Jason A. Donenfeld]

On Tue, Nov 7, 2017 at 1:55 AM, Ard Biesheuvel
<ard.biesheuvel@linaro.org> wrote:
> It appears your v4 adds __ashlti3() and __ashrti3, whereas the error
> is about __lshrti3() being undefined.

Whoopsie. v5 adds the final missing function. Looks like it now
compiles for -next with the config you provided me.

Re: [PATCH v4] arm64: support __int128 on gcc 5+

[Will Deacon]

On Mon, Nov 06, 2017 at 03:59:18PM +0000, Catalin Marinas wrote:
> On Mon, Nov 06, 2017 at 10:31:51AM +0100, Jason A. Donenfeld wrote:
> > Versions of gcc prior to gcc 5 emitted a __multi3 function call when
> > dealing with TI types, resulting in failures when trying to link to
> > libgcc, and more generally, bad performance. However, since gcc 5,
> > the compiler supports actually emitting fast instructions, which means
> > we can at long last enable this option and receive the speedups.
> > 
> > The gcc commit that added proper Aarch64 support is:
> > https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=d1ae7bb994f49316f6f63e6173f2931e837a351d
> > This commit appears to be part of the gcc 5 release.
> > 
> > There are still a few instructions, __ashlti3 and __ashrti3, which
> > require libgcc, which is fine. Rather than linking to libgcc, we
> > simply provide them ourselves, since they're not that complicated.
> > 
> > Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
> 
> According to Arnd, linux-next (with this patch included) fails to build
> with gcc-7 (config: https://pastebin.com/raw/sgvPe96e).
> 
> Will is managing the upcoming merging window for arm64 but is travelling
> this week. I'll wait for couple of days, he'll probably catch up with
> emails but my suggestion is that we revert the patch and push it back
> again into -next after the 4.15 merging window.

So I pushed a fixup patch on top of for-next/core, but I suggest we
temporarily revert the- DCONFIG_ARCH_SUPPORTS_INT128 option if any other
issues crop up.

Cheers,

Will

Re: [PATCH v4] arm64: support __int128 on gcc 5+

[Jason A. Donenfeld]

Hi Will,

On Tue, Nov 7, 2017 at 11:13 AM, Will Deacon <will.deacon@arm.com> wrote:
> So I pushed a fixup patch on top of for-next/core, but I suggest we
> temporarily revert the- DCONFIG_ARCH_SUPPORTS_INT128 option if any other
> issues crop up.

The fixup looks good to me.

If there are additional problems, I'm happy to keep providing patches
until it's sorted. I just did an allyesconfig and didn't have any
problems, though, so I think we're all set now.

Jason

Re: [PATCH] arm64: support __int128 on gcc 5+

[Jason A. Donenfeld]

Hi Will,

Nice catch. I posted a v2 that addresses that.

Thanks,
Jason