[PATCH 0/3] Address compilation of eBPF related software with clang compiler on arm architecture

[PATCH 0/3] Address compilation of eBPF related software with clang compiler on arm architecture

[Pavo Banicevic]

This patchset is fixing compilation issues that are encountered in our usage of the Linux kernel.

Two patches are addressing compilation of eBPF related software with clang compiler on arm architecture.
The third patch resolves compilation of the perf tool in this specific scenario.

We are also interested in possible alternative approaches in fixing these compilation issues which could
then be incorporated into the mainline.

Ivan Khoronzhuk (2):
  arm: include: asm: swab: mask rev16 instruction for clang
  arm: include: asm: unified: mask .syntax unified for clang

Matt Redfearn (1):
  include/uapi/linux/swab: Fix potentially missing __always_inline

 arch/arm/include/asm/swab.h    | 3 +++
 arch/arm/include/asm/unified.h | 4 +++-
 include/uapi/linux/swab.h      | 2 +-
 3 files changed, 7 insertions(+), 2 deletions(-)

-- 
2.32.0

[PATCH 1/3] arm: include: asm: swab: mask rev16 instruction for clang

[Pavo Banicevic]

From: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>

The samples/bpf with clang -emit-llvm reuses linux headers to build
bpf samples, and this w/a only for samples (samples/bpf/Makefile
CLANG-bpf).

It allows to build samples/bpf for arm bpf using clang.
In another way clang -emit-llvm generates errors like:

CLANG-bpf  samples/bpf/tc_l2_redirect_kern.o
<inline asm>:1:2: error: invalid register/token name
rev16 r3, r0

This decision is arguable, probably there is another way, but
it doesn't have impact on samples/bpf, so it's easier just ignore
it for clang, at least for now.

Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
---
 arch/arm/include/asm/swab.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/arm/include/asm/swab.h b/arch/arm/include/asm/swab.h
index c6051823048b..a9fd9cd33d5e 100644
--- a/arch/arm/include/asm/swab.h
+++ b/arch/arm/include/asm/swab.h
@@ -25,8 +25,11 @@ static inline __attribute_const__ __u32 __arch_swahb32(__u32 x)
 	__asm__ ("rev16 %0, %1" : "=r" (x) : "r" (x));
 	return x;
 }
+
+#ifndef __clang__
 #define __arch_swahb32 __arch_swahb32
 #define __arch_swab16(x) ((__u16)__arch_swahb32(x))
+#endif
 
 static inline __attribute_const__ __u32 __arch_swab32(__u32 x)
 {
-- 
2.32.0

Re: [PATCH 1/3] arm: include: asm: swab: mask rev16 instruction for clang

[Yonghong Song]

On 7/27/21 7:11 AM, Pavo Banicevic wrote:
> From: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
> 
> The samples/bpf with clang -emit-llvm reuses linux headers to build
> bpf samples, and this w/a only for samples (samples/bpf/Makefile
> CLANG-bpf).
> 
> It allows to build samples/bpf for arm bpf using clang.
> In another way clang -emit-llvm generates errors like:
> 
> CLANG-bpf  samples/bpf/tc_l2_redirect_kern.o
> <inline asm>:1:2: error: invalid register/token name
> rev16 r3, r0
> 
> This decision is arguable, probably there is another way, but
> it doesn't have impact on samples/bpf, so it's easier just ignore
> it for clang, at least for now.
> 
> Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
> ---
>   arch/arm/include/asm/swab.h | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/arch/arm/include/asm/swab.h b/arch/arm/include/asm/swab.h
> index c6051823048b..a9fd9cd33d5e 100644
> --- a/arch/arm/include/asm/swab.h
> +++ b/arch/arm/include/asm/swab.h
> @@ -25,8 +25,11 @@ static inline __attribute_const__ __u32 __arch_swahb32(__u32 x)
>   	__asm__ ("rev16 %0, %1" : "=r" (x) : "r" (x));
>   	return x;
>   }
> +
> +#ifndef __clang__
>   #define __arch_swahb32 __arch_swahb32
>   #define __arch_swab16(x) ((__u16)__arch_swahb32(x))
> +#endif

What if the kernel is compiled with clang compiler?
Does this still work?

To workaround samples/bpf issue, I think you can use
__BPF_TRACING__ macro.

>   
>   static inline __attribute_const__ __u32 __arch_swab32(__u32 x)
>   {
> 

Re: [PATCH 1/3] arm: include: asm: swab: mask rev16 instruction for clang

[Nick Desaulniers]

On Tue, Jul 27, 2021 at 7:12 AM Pavo Banicevic
<pavo.banicevic@sartura.hr> wrote:
>
> From: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
>
> The samples/bpf with clang -emit-llvm reuses linux headers to build
> bpf samples, and this w/a only for samples (samples/bpf/Makefile
> CLANG-bpf).
>
> It allows to build samples/bpf for arm bpf using clang.
> In another way clang -emit-llvm generates errors like:
>
> CLANG-bpf  samples/bpf/tc_l2_redirect_kern.o
> <inline asm>:1:2: error: invalid register/token name
> rev16 r3, r0
>
> This decision is arguable, probably there is another way, but
> it doesn't have impact on samples/bpf, so it's easier just ignore
> it for clang, at least for now.

NACK

The way to fix these is to sort out the header includes, not turning
off arbitrary things that are used by the actual kernel build for 32b
ARM.

>
> Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
> ---
>  arch/arm/include/asm/swab.h | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/arch/arm/include/asm/swab.h b/arch/arm/include/asm/swab.h
> index c6051823048b..a9fd9cd33d5e 100644
> --- a/arch/arm/include/asm/swab.h
> +++ b/arch/arm/include/asm/swab.h
> @@ -25,8 +25,11 @@ static inline __attribute_const__ __u32 __arch_swahb32(__u32 x)
>         __asm__ ("rev16 %0, %1" : "=r" (x) : "r" (x));
>         return x;
>  }
> +
> +#ifndef __clang__
>  #define __arch_swahb32 __arch_swahb32
>  #define __arch_swab16(x) ((__u16)__arch_swahb32(x))
> +#endif
>
>  static inline __attribute_const__ __u32 __arch_swab32(__u32 x)
>  {
> --
> 2.32.0
>


-- 
Thanks,
~Nick Desaulniers

Re: [PATCH 1/3] arm: include: asm: swab: mask rev16 instruction for clang

[Andrii Nakryiko]

On Tue, Jul 27, 2021 at 10:53 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Tue, Jul 27, 2021 at 7:12 AM Pavo Banicevic
> <pavo.banicevic@sartura.hr> wrote:
> >
> > From: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
> >
> > The samples/bpf with clang -emit-llvm reuses linux headers to build
> > bpf samples, and this w/a only for samples (samples/bpf/Makefile
> > CLANG-bpf).
> >
> > It allows to build samples/bpf for arm bpf using clang.
> > In another way clang -emit-llvm generates errors like:
> >
> > CLANG-bpf  samples/bpf/tc_l2_redirect_kern.o
> > <inline asm>:1:2: error: invalid register/token name
> > rev16 r3, r0
> >
> > This decision is arguable, probably there is another way, but
> > it doesn't have impact on samples/bpf, so it's easier just ignore
> > it for clang, at least for now.
>
> NACK
>
> The way to fix these is to sort out the header includes, not turning
> off arbitrary things that are used by the actual kernel build for 32b
> ARM.

Would it be too horrible to just get rid of `clang -emit-llvm` and use
vmlinux.h (we don't need to do CO-RE, btw, just generate vmlinux.h
from the matching kernel)? Kumar has already started moving in that
direction in his recent patch set ([0]). Would that get rid of all
these issues?

  [0] https://patchwork.kernel.org/project/netdevbpf/list/?series=519281&state=*


>
> >
> > Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
> > ---
> >  arch/arm/include/asm/swab.h | 3 +++
> >  1 file changed, 3 insertions(+)
> >
> > diff --git a/arch/arm/include/asm/swab.h b/arch/arm/include/asm/swab.h
> > index c6051823048b..a9fd9cd33d5e 100644
> > --- a/arch/arm/include/asm/swab.h
> > +++ b/arch/arm/include/asm/swab.h
> > @@ -25,8 +25,11 @@ static inline __attribute_const__ __u32 __arch_swahb32(__u32 x)
> >         __asm__ ("rev16 %0, %1" : "=r" (x) : "r" (x));
> >         return x;
> >  }
> > +
> > +#ifndef __clang__
> >  #define __arch_swahb32 __arch_swahb32
> >  #define __arch_swab16(x) ((__u16)__arch_swahb32(x))
> > +#endif
> >
> >  static inline __attribute_const__ __u32 __arch_swab32(__u32 x)
> >  {
> > --
> > 2.32.0
> >
>
>
> --
> Thanks,
> ~Nick Desaulniers

[PATCH 2/3] arm: include: asm: unified: mask .syntax unified for clang

[Pavo Banicevic]

From: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>

The samples/bpf reuses linux headers, with clang -emit-llvm,
so this w/a is only for samples/bpf (samples/bpf/Makefile CLANG-bpf).

It allows to build samples/bpf for arm on target board.
In another way clang -emit-llvm generates errors like:

<inline asm>:1:1: error: unknown directive
.syntax unified

I have verified it on clang 5, 6, 7, 8, 9, 10
as on native platform as for cross-compiling. This decision is
arguable, but it doesn't have impact on samples/bpf so it's easier
just ignore it for clang, at least for now...

Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
---
 arch/arm/include/asm/unified.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/arm/include/asm/unified.h b/arch/arm/include/asm/unified.h
index 1e2c3eb04353..8718f313e7c4 100644
--- a/arch/arm/include/asm/unified.h
+++ b/arch/arm/include/asm/unified.h
@@ -11,7 +11,9 @@
 #if defined(__ASSEMBLY__)
 	.syntax unified
 #else
-__asm__(".syntax unified");
+
+#ifndef __clang__
+	__asm__(".syntax unified");
 #endif
 
 #ifdef CONFIG_CPU_V7M
-- 
2.32.0

Re: [PATCH 2/3] arm: include: asm: unified: mask .syntax unified for clang

[Nick Desaulniers]

On Tue, Jul 27, 2021 at 7:12 AM Pavo Banicevic
<pavo.banicevic@sartura.hr> wrote:
>
> From: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
>
> The samples/bpf reuses linux headers, with clang -emit-llvm,
> so this w/a is only for samples/bpf (samples/bpf/Makefile CLANG-bpf).
>
> It allows to build samples/bpf for arm on target board.
> In another way clang -emit-llvm generates errors like:
>
> <inline asm>:1:1: error: unknown directive
> .syntax unified
>
> I have verified it on clang 5, 6, 7, 8, 9, 10
> as on native platform as for cross-compiling. This decision is
> arguable, but it doesn't have impact on samples/bpf so it's easier
> just ignore it for clang, at least for now...

Did you test ARCH=arm kernel builds with Clang with this series applied?

>
> Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
> ---
>  arch/arm/include/asm/unified.h | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm/include/asm/unified.h b/arch/arm/include/asm/unified.h
> index 1e2c3eb04353..8718f313e7c4 100644
> --- a/arch/arm/include/asm/unified.h
> +++ b/arch/arm/include/asm/unified.h
> @@ -11,7 +11,9 @@
>  #if defined(__ASSEMBLY__)
>         .syntax unified
>  #else
> -__asm__(".syntax unified");
> +
> +#ifndef __clang__
> +       __asm__(".syntax unified");
>  #endif
>
>  #ifdef CONFIG_CPU_V7M
> --
> 2.32.0
>


-- 
Thanks,
~Nick Desaulniers

[PATCH 3/3] include/uapi/linux/swab: Fix potentially missing __always_inline

[Pavo Banicevic]

From: Matt Redfearn <matt.redfearn@mips.com>

Commit bc27fb68aaad ("include/uapi/linux/byteorder, swab: force inlining
of some byteswap operations") added __always_inline to swab functions
and commit 283d75737837 ("uapi/linux/stddef.h: Provide __always_inline to
userspace headers") added a definition of __always_inline for use in
exported headers when the kernel's compiler.h is not available.

However, since swab.h does not include stddef.h, if the header soup does
not indirectly include it, the definition of __always_inline is missing,
resulting in a compilation failure, which was observed compiling the
perf tool using exported headers containing this commit:

In file included from /usr/include/linux/byteorder/little_endian.h:12:0,
                 from /usr/include/asm/byteorder.h:14,
                 from tools/include/uapi/linux/perf_event.h:20,
                 from perf.h:8,
                 from builtin-bench.c:18:
/usr/include/linux/swab.h:160:8: error: unknown type name `__always_inline'
 static __always_inline __u16 __swab16p(const __u16 *p)

Fix this by replacing the inclusion of linux/compiler.h with
linux/stddef.h to ensure that we pick up that definition if required,
without relying on it's indirect inclusion. compiler.h is then included
indirectly, via stddef.h.

Fixes: 283d75737837 ("uapi/linux/stddef.h: Provide __always_inline to userspace headers")

Signed-off-by: Matt Redfearn <matt.redfearn@mips.com>
---
 include/uapi/linux/swab.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/uapi/linux/swab.h b/include/uapi/linux/swab.h
index 7272f85d6d6a..3736f2fe1541 100644
--- a/include/uapi/linux/swab.h
+++ b/include/uapi/linux/swab.h
@@ -3,7 +3,7 @@
 #define _UAPI_LINUX_SWAB_H
 
 #include <linux/types.h>
-#include <linux/compiler.h>
+#include <linux/stddef.h>
 #include <asm/bitsperlong.h>
 #include <asm/swab.h>
 
-- 
2.32.0

Re: [PATCH 3/3] include/uapi/linux/swab: Fix potentially missing __always_inline

[Petr Vaněk]

On Tue, Jul 27, 2021 at 04:11:19PM +0200, Pavo Banicevic wrote:
> From: Matt Redfearn <matt.redfearn@mips.com>
> 
> Commit bc27fb68aaad ("include/uapi/linux/byteorder, swab: force inlining
> of some byteswap operations") added __always_inline to swab functions
> and commit 283d75737837 ("uapi/linux/stddef.h: Provide __always_inline to
> userspace headers") added a definition of __always_inline for use in
> exported headers when the kernel's compiler.h is not available.
> 
> However, since swab.h does not include stddef.h, if the header soup does
> not indirectly include it, the definition of __always_inline is missing,
> resulting in a compilation failure, which was observed compiling the
> perf tool using exported headers containing this commit:
> 
> In file included from /usr/include/linux/byteorder/little_endian.h:12:0,
>                  from /usr/include/asm/byteorder.h:14,
>                  from tools/include/uapi/linux/perf_event.h:20,
>                  from perf.h:8,
>                  from builtin-bench.c:18:
> /usr/include/linux/swab.h:160:8: error: unknown type name `__always_inline'
>  static __always_inline __u16 __swab16p(const __u16 *p)
> 
> Fix this by replacing the inclusion of linux/compiler.h with
> linux/stddef.h to ensure that we pick up that definition if required,
> without relying on it's indirect inclusion. compiler.h is then included
> indirectly, via stddef.h.
> 
> Fixes: 283d75737837 ("uapi/linux/stddef.h: Provide __always_inline to userspace headers")
> 
> Signed-off-by: Matt Redfearn <matt.redfearn@mips.com>
> ---

I use this patch in order to fix __always_inline issue for kernels
5.12+, see https://lore.kernel.org/lkml/YPGXXt6Z3O1W0AYS@arkam/ .
I believe this is the correct solution.

Reviewed-by: Petr Vaněk <arkamar@atlas.cz>