[PATCH RFC 0/3] bpf/arm64/mips: Avoid inline asm in BPF

[PATCH RFC 0/3] bpf/arm64/mips: Avoid inline asm in BPF

[David Daney]

To build samples/bpf on MIPS we need to avoid some inline asm that
causes llvm to fail.

Looking at the code, it seems that arm64 had the same problem and
avoided it by defining the header guard symbol.  This approach does
not scale, so I invented a preprocessor define to identify the case of
building with a BPF target that can be used instead.

It is an RFC at this point as I haven't yet tested the arm64 change,
and I wanted to see if others think this is the proper way to handle
avoidance of inline asm.

David Daney (3):
  arm64: Gate inclusion of asm/sysreg.h by __EMITTING_BPF__
  samples/bpf: Add define __EMITTING_BPF__ when building BPF
  MIPS: Include file changes to enable building BPF code with llvm

 arch/arm64/include/asm/sysreg.h   | 4 +++-
 arch/mips/Makefile                | 1 +
 arch/mips/cavium-octeon/Platform  | 3 +++
 arch/mips/include/asm/checksum.h  | 2 +-
 arch/mips/include/uapi/asm/swab.h | 2 +-
 samples/bpf/Makefile              | 8 ++++----
 6 files changed, 13 insertions(+), 7 deletions(-)

-- 
2.11.0

[PATCH RFC 1/3] arm64: Gate inclusion of asm/sysreg.h by __EMITTING_BPF__

[David Daney]

Compilation to eBPF chokes on the inline asm in asm/sysreg.h, so don't
include it when compiling to a BPF target.

Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/arm64/include/asm/sysreg.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 15c142ce991c..faa8f853e369 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -20,6 +20,8 @@
 #ifndef __ASM_SYSREG_H
 #define __ASM_SYSREG_H
 
+#ifndef __EMITTING_BPF__
+
 #include <linux/stringify.h>
 
 /*
@@ -502,5 +504,5 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
 }
 
 #endif
-
+#endif  /* __EMITTING_BPF__ */
 #endif	/* __ASM_SYSREG_H */
-- 
2.11.0

[PATCH RFC 2/3] samples/bpf: Add define __EMITTING_BPF__ when building BPF

[David Daney]

... this allows gating of inline assembly code that causes llvm to
fail when emitting BPF.

Signed-off-by: David Daney <david.daney@cavium.com>
---
 samples/bpf/Makefile | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index a0561dc762fe..4979e6b56662 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -193,12 +193,12 @@ $(src)/*.c: verify_target_bpf
 
 $(obj)/tracex5_kern.o: $(obj)/syscall_nrs.h
 
-# asm/sysreg.h - inline assembly used by it is incompatible with llvm.
-# But, there is no easy way to fix it, so just exclude it since it is
-# useless for BPF samples.
+# __EMITTING_BPF__ used to exclude inline assembly, which cannot be
+# emitted in BPF code.
 $(obj)/%.o: $(src)/%.c
 	$(CLANG) $(NOSTDINC_FLAGS) $(LINUXINCLUDE) $(EXTRA_CFLAGS) \
-		-D__KERNEL__ -D__ASM_SYSREG_H -Wno-unused-value -Wno-pointer-sign \
+		-D__KERNEL__ -D__EMITTING_BPF__ \
+		-Wno-unused-value -Wno-pointer-sign \
 		-Wno-compare-distinct-pointer-types \
 		-Wno-gnu-variable-sized-type-not-at-end \
 		-Wno-address-of-packed-member -Wno-tautological-compare \
-- 
2.11.0

[PATCH RFC 3/3] MIPS: Include file changes to enable building BPF code with llvm

[David Daney]

When building for the eBPF target archecture. Inline asm cannot be
used as MIPS instructions are fundamentally incompatible with eBPF
bytecode.  The preprocessor symbol __EMITTING_BPF__ is used to gate
the inclusion of inline asm in constructs used the by the BPF
programs.

Also make the Makefile symbol LINUXINCLUDE contain the
asm/mach-MACHINE directory so that the BPF compilation process can
pull in the necessary include files.

Signed-off-by: David Daney <david.daney@cavium.com>
---
 arch/mips/Makefile                | 1 +
 arch/mips/cavium-octeon/Platform  | 3 +++
 arch/mips/include/asm/checksum.h  | 2 +-
 arch/mips/include/uapi/asm/swab.h | 2 +-
 4 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/mips/Makefile b/arch/mips/Makefile
index 02a1787c888c..ca968415597f 100644
--- a/arch/mips/Makefile
+++ b/arch/mips/Makefile
@@ -247,6 +247,7 @@ entry-y				= 0x$(shell $(NM) vmlinux 2>/dev/null \
 					| grep "\bkernel_entry\b" | cut -f1 -d \ )
 
 cflags-y			+= -I$(srctree)/arch/mips/include/asm/mach-generic
+LINUXINCLUDE			+= -I$(srctree)/arch/mips/include/asm/mach-generic
 drivers-$(CONFIG_PCI)		+= arch/mips/pci/
 
 #
diff --git a/arch/mips/cavium-octeon/Platform b/arch/mips/cavium-octeon/Platform
index 45be853700e6..9ef3e4074099 100644
--- a/arch/mips/cavium-octeon/Platform
+++ b/arch/mips/cavium-octeon/Platform
@@ -4,4 +4,7 @@
 platform-$(CONFIG_CAVIUM_OCTEON_SOC)	+= cavium-octeon/
 cflags-$(CONFIG_CAVIUM_OCTEON_SOC)	+=				\
 		-I$(srctree)/arch/mips/include/asm/mach-cavium-octeon
+ifdef CONFIG_CAVIUM_OCTEON_SOC
+LINUXINCLUDE	+= -I$(srctree)/arch/mips/include/asm/mach-cavium-octeon
+endif
 load-$(CONFIG_CAVIUM_OCTEON_SOC)	+= 0xffffffff81100000
diff --git a/arch/mips/include/asm/checksum.h b/arch/mips/include/asm/checksum.h
index 77cad232a1c6..f8fff2ced216 100644
--- a/arch/mips/include/asm/checksum.h
+++ b/arch/mips/include/asm/checksum.h
@@ -12,7 +12,7 @@
 #ifndef _ASM_CHECKSUM_H
 #define _ASM_CHECKSUM_H
 
-#ifdef CONFIG_GENERIC_CSUM
+#if defined(CONFIG_GENERIC_CSUM) || defined(__EMITTING_BPF__)
 #include <asm-generic/checksum.h>
 #else
 
diff --git a/arch/mips/include/uapi/asm/swab.h b/arch/mips/include/uapi/asm/swab.h
index 23cd9b118c9e..42ed70015c70 100644
--- a/arch/mips/include/uapi/asm/swab.h
+++ b/arch/mips/include/uapi/asm/swab.h
@@ -13,7 +13,7 @@
 
 #define __SWAB_64_THRU_32__
 
-#if !defined(__mips16) &&					\
+#if !defined(__mips16) && !defined(__EMITTING_BPF__) &&			\
 	((defined(__mips_isa_rev) && (__mips_isa_rev >= 2)) ||	\
 	 defined(_MIPS_ARCH_LOONGSON3A))
 
-- 
2.11.0

Re: [PATCH RFC 2/3] samples/bpf: Add define __EMITTING_BPF__ when building BPF

[Daniel Borkmann]

On 06/16/2017 12:35 AM, David Daney wrote:
> ... this allows gating of inline assembly code that causes llvm to
> fail when emitting BPF.
>
> Signed-off-by: David Daney <david.daney@cavium.com>

I don't have a better idea at the moment, perhaps there could be
a clang rewrite plugin that would ignore all inline assembly code
since this is never used from BPF progs. Hmm. Really ugly that
we have to add this __EMITTING_BPF__ into arch asm files, but I
don't have a better idea for an immediate workaround right now ...
I would really prefer if we could avoid just for the sake of the
kernel samples going down the road of adding a !defined(__EMITTING_BPF__)
into a uapi asm header for mips, though. Is this coming from
networking sample code or rather tracing?

> ---
>   samples/bpf/Makefile | 8 ++++----
>   1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
> index a0561dc762fe..4979e6b56662 100644
> --- a/samples/bpf/Makefile
> +++ b/samples/bpf/Makefile
> @@ -193,12 +193,12 @@ $(src)/*.c: verify_target_bpf
>
>   $(obj)/tracex5_kern.o: $(obj)/syscall_nrs.h
>
> -# asm/sysreg.h - inline assembly used by it is incompatible with llvm.
> -# But, there is no easy way to fix it, so just exclude it since it is
> -# useless for BPF samples.
> +# __EMITTING_BPF__ used to exclude inline assembly, which cannot be
> +# emitted in BPF code.
>   $(obj)/%.o: $(src)/%.c
>   	$(CLANG) $(NOSTDINC_FLAGS) $(LINUXINCLUDE) $(EXTRA_CFLAGS) \
> -		-D__KERNEL__ -D__ASM_SYSREG_H -Wno-unused-value -Wno-pointer-sign \
> +		-D__KERNEL__ -D__EMITTING_BPF__ \
> +		-Wno-unused-value -Wno-pointer-sign \
>   		-Wno-compare-distinct-pointer-types \
>   		-Wno-gnu-variable-sized-type-not-at-end \
>   		-Wno-address-of-packed-member -Wno-tautological-compare \
>

Re: [PATCH RFC 2/3] samples/bpf: Add define __EMITTING_BPF__ when building BPF

[David Miller]

From: Daniel Borkmann <daniel@iogearbox.net>
Date: Fri, 16 Jun 2017 12:24:06 +0200

> On 06/16/2017 12:35 AM, David Daney wrote:
>> ... this allows gating of inline assembly code that causes llvm to
>> fail when emitting BPF.
>>
>> Signed-off-by: David Daney <david.daney@cavium.com>
> 
> I don't have a better idea at the moment, perhaps there could be
> a clang rewrite plugin that would ignore all inline assembly code
> since this is never used from BPF progs. Hmm. Really ugly that
> we have to add this __EMITTING_BPF__ into arch asm files, but I
> don't have a better idea for an immediate workaround right now ...
> I would really prefer if we could avoid just for the sake of the
> kernel samples going down the road of adding a
> !defined(__EMITTING_BPF__)
> into a uapi asm header for mips, though. Is this coming from
> networking sample code or rather tracing?

The problem is that we include the arch include/asm files at all.

When we build bpf stuff, we should have a set of asm/ files
specifically for builds targetting BPF.

Let's just fix this right and stop putting all of these hacks in.

Thank you.

Re: [PATCH RFC 2/3] samples/bpf: Add define __EMITTING_BPF__ when building BPF

[Andy Gospodarek]

On Thu, Jun 15, 2017 at 03:35:42PM -0700, David Daney wrote:
> ... this allows gating of inline assembly code that causes llvm to
> fail when emitting BPF.

I floated essentially the same patch in Feb without much luck:

http://lists.infradead.org/pipermail/linux-arm-kernel/2017-March/492758.html

Many of the folks on the cc-list here had objections then and I suspect they
unfortunately still object.  I like the fact that this fix allowed
architectures that are currently problematic to move forward before there is a
possibly mythical "fix in llvm" to address this problem.

When talking about this in one of the IOVisor calls it was also discussed that
this needs to be fixed for tracing, so there are more than just the BPF
use-case where this is important.

I wasn't sure there was buy-in from the ARM developers, but my thought had been
that a cleaner solution to this would be to reorganize sysreg.h into multiple
files.  The inline assembly would be the only thing in sysreg-asm.h (that was
actually the only thing originally in sysreg.h) and the rest of the code would
be in sysreg.h.

That is not what Dave suggested, but it would be a good starting point for a
custom asm/ layer for BPF/tracing.  I'm with Dave and think a specialized set
of asm/ files for tracing/BPF to avoid these issues all-together and let arch
developers to do whatever they want in their code.

> Signed-off-by: David Daney <david.daney@cavium.com>
> ---
>  samples/bpf/Makefile | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
> index a0561dc762fe..4979e6b56662 100644
> --- a/samples/bpf/Makefile
> +++ b/samples/bpf/Makefile
> @@ -193,12 +193,12 @@ $(src)/*.c: verify_target_bpf
>  
>  $(obj)/tracex5_kern.o: $(obj)/syscall_nrs.h
>  
> -# asm/sysreg.h - inline assembly used by it is incompatible with llvm.
> -# But, there is no easy way to fix it, so just exclude it since it is
> -# useless for BPF samples.
> +# __EMITTING_BPF__ used to exclude inline assembly, which cannot be
> +# emitted in BPF code.
>  $(obj)/%.o: $(src)/%.c
>  	$(CLANG) $(NOSTDINC_FLAGS) $(LINUXINCLUDE) $(EXTRA_CFLAGS) \
> -		-D__KERNEL__ -D__ASM_SYSREG_H -Wno-unused-value -Wno-pointer-sign \
> +		-D__KERNEL__ -D__EMITTING_BPF__ \
> +		-Wno-unused-value -Wno-pointer-sign \
>  		-Wno-compare-distinct-pointer-types \
>  		-Wno-gnu-variable-sized-type-not-at-end \
>  		-Wno-address-of-packed-member -Wno-tautological-compare \