linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/6] eBPF JIT for PPC64
@ 2016-06-07 13:32 Naveen N. Rao
  2016-06-07 13:32 ` [PATCH 1/6] ppc: bpf/jit: Fix/enhance 32-bit Load Immediate implementation Naveen N. Rao
                   ` (6 more replies)
  0 siblings, 7 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-07 13:32 UTC (permalink / raw)
  To: linux-kernel, linuxppc-dev, netdev, mpe
  Cc: Matt Evans, Denis Kirjanov, Paul Mackerras, Alexei Starovoitov,
	Daniel Borkmann, David S. Miller, Ananth N Mavinakayanahalli

Implement extended BPF JIT for ppc64. We retain the classic BPF JIT for
ppc32 and move ppc64 BE/LE to use the new JIT. Classic BPF filters will
be converted to extended BPF (see convert_filter()) and JIT'ed with the
new compiler.

Most of the existing macros are retained and fixed/enhanced where
appropriate. Patches 1-4 are geared towards this.

Patch 5 breaks out the classic BPF JIT specifics into a separate
bpf_jit32.h header file, while retaining all the generic instruction
macros in bpf_jit.h.

Patch 6 implements eBPF JIT for ppc64.

Since the RFC patchset [1], powerpc JIT has now gained support for skb
access helpers and now passes all tests in test_bpf.ko. Review comments
on the RFC patches have been addressed (use of an ABI macro [2] and use
of bpf_jit_binary_alloc()), along with a few other generic fixes and
updates.

Prominent TODOs:
 - implement BPF tail calls
 - support for BPF constant blinding

Please note that patch [2] is a pre-requisite for this patchset, and is
not yet upstream.


- Naveen

[1] http://thread.gmane.org/gmane.linux.kernel/2188694
[2] http://thread.gmane.org/gmane.linux.ports.ppc.embedded/96514


Naveen N. Rao (6):
  ppc: bpf/jit: Fix/enhance 32-bit Load Immediate implementation
  ppc: bpf/jit: Optimize 64-bit Immediate loads
  ppc: bpf/jit: Introduce rotate immediate instructions
  ppc: bpf/jit: A few cleanups
  ppc: bpf/jit: Isolate classic BPF JIT specifics into a separate header
  ppc: ebpf/jit: Implement JIT compiler for extended BPF

 arch/powerpc/Kconfig                  |   3 +-
 arch/powerpc/include/asm/asm-compat.h |   2 +
 arch/powerpc/include/asm/ppc-opcode.h |  22 +-
 arch/powerpc/net/Makefile             |   4 +
 arch/powerpc/net/bpf_jit.h            | 235 ++++-----
 arch/powerpc/net/bpf_jit32.h          | 139 +++++
 arch/powerpc/net/bpf_jit64.h          | 102 ++++
 arch/powerpc/net/bpf_jit_asm.S        |   2 +-
 arch/powerpc/net/bpf_jit_asm64.S      | 180 +++++++
 arch/powerpc/net/bpf_jit_comp.c       |  10 +-
 arch/powerpc/net/bpf_jit_comp64.c     | 956 ++++++++++++++++++++++++++++++++++
 11 files changed, 1504 insertions(+), 151 deletions(-)
 create mode 100644 arch/powerpc/net/bpf_jit32.h
 create mode 100644 arch/powerpc/net/bpf_jit64.h
 create mode 100644 arch/powerpc/net/bpf_jit_asm64.S
 create mode 100644 arch/powerpc/net/bpf_jit_comp64.c

-- 
2.8.2

^ permalink raw reply	[flat|nested] 34+ messages in thread

* [PATCH 1/6] ppc: bpf/jit: Fix/enhance 32-bit Load Immediate implementation
  2016-06-07 13:32 [PATCH 0/6] eBPF JIT for PPC64 Naveen N. Rao
@ 2016-06-07 13:32 ` Naveen N. Rao
  2016-06-07 13:32 ` [PATCH 2/6] ppc: bpf/jit: Optimize 64-bit Immediate loads Naveen N. Rao
                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-07 13:32 UTC (permalink / raw)
  To: linux-kernel, linuxppc-dev, netdev, mpe
  Cc: Matt Evans, Denis Kirjanov, Paul Mackerras, Alexei Starovoitov,
	Daniel Borkmann, David S. Miller, Ananth N Mavinakayanahalli

The existing LI32() macro can sometimes result in a sign-extended 32-bit
load that does not clear the top 32-bits properly. As an example,
loading 0x7fffffff results in the register containing
0xffffffff7fffffff. While this does not impact classic BPF JIT
implementation (since that only uses the lower word for all operations),
we would like to share this macro between classic BPF JIT and extended
BPF JIT, wherein the entire 64-bit value in the register matters. Fix
this by first doing a shifted LI followed by ORI.

An additional optimization is with loading values between -32768 to -1,
where we now only need a single LI.

The new implementation now generates the same or less number of
instructions.

Cc: Matt Evans <matt@ozlabs.org>
Cc: Denis Kirjanov <kda@linux-powerpc.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
---
 arch/powerpc/net/bpf_jit.h | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h
index 889fd19..a9882db 100644
--- a/arch/powerpc/net/bpf_jit.h
+++ b/arch/powerpc/net/bpf_jit.h
@@ -232,10 +232,17 @@ DECLARE_LOAD_FUNC(sk_load_byte_msh);
 					     (((cond) & 0x3ff) << 16) |	      \
 					     (((dest) - (ctx->idx * 4)) &     \
 					      0xfffc))
-#define PPC_LI32(d, i)		do { PPC_LI(d, IMM_L(i));		      \
-		if ((u32)(uintptr_t)(i) >= 32768) {			      \
-			PPC_ADDIS(d, d, IMM_HA(i));			      \
+/* Sign-extended 32-bit immediate load */
+#define PPC_LI32(d, i)		do {					      \
+		if ((int)(uintptr_t)(i) >= -32768 &&			      \
+				(int)(uintptr_t)(i) < 32768)		      \
+			PPC_LI(d, i);					      \
+		else {							      \
+			PPC_LIS(d, IMM_H(i));				      \
+			if (IMM_L(i))					      \
+				PPC_ORI(d, d, IMM_L(i));		      \
 		} } while(0)
+
 #define PPC_LI64(d, i)		do {					      \
 		if (!((uintptr_t)(i) & 0xffffffff00000000ULL))		      \
 			PPC_LI32(d, i);					      \
-- 
2.8.2

^ permalink raw reply related	[flat|nested] 34+ messages in thread

* [PATCH 2/6] ppc: bpf/jit: Optimize 64-bit Immediate loads
  2016-06-07 13:32 [PATCH 0/6] eBPF JIT for PPC64 Naveen N. Rao
  2016-06-07 13:32 ` [PATCH 1/6] ppc: bpf/jit: Fix/enhance 32-bit Load Immediate implementation Naveen N. Rao
@ 2016-06-07 13:32 ` Naveen N. Rao
  2016-06-07 13:32 ` [PATCH 3/6] ppc: bpf/jit: Introduce rotate immediate instructions Naveen N. Rao
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-07 13:32 UTC (permalink / raw)
  To: linux-kernel, linuxppc-dev, netdev, mpe
  Cc: Matt Evans, Denis Kirjanov, Paul Mackerras, Alexei Starovoitov,
	Daniel Borkmann, David S. Miller, Ananth N Mavinakayanahalli

Similar to the LI32() optimization, if the value can be represented
in 32-bits, use LI32(). Also handle loading a few specific forms of
immediate values in an optimum manner.

Cc: Matt Evans <matt@ozlabs.org>
Cc: Denis Kirjanov <kda@linux-powerpc.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
---
 arch/powerpc/net/bpf_jit.h | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h
index a9882db..4c1e055 100644
--- a/arch/powerpc/net/bpf_jit.h
+++ b/arch/powerpc/net/bpf_jit.h
@@ -244,20 +244,25 @@ DECLARE_LOAD_FUNC(sk_load_byte_msh);
 		} } while(0)
 
 #define PPC_LI64(d, i)		do {					      \
-		if (!((uintptr_t)(i) & 0xffffffff00000000ULL))		      \
+		if ((long)(i) >= -2147483648 &&				      \
+				(long)(i) < 2147483648)			      \
 			PPC_LI32(d, i);					      \
 		else {							      \
-			PPC_LIS(d, ((uintptr_t)(i) >> 48));		      \
-			if ((uintptr_t)(i) & 0x0000ffff00000000ULL)	      \
-				PPC_ORI(d, d,				      \
-					((uintptr_t)(i) >> 32) & 0xffff);     \
+			if (!((uintptr_t)(i) & 0xffff800000000000ULL))	      \
+				PPC_LI(d, ((uintptr_t)(i) >> 32) & 0xffff);   \
+			else {						      \
+				PPC_LIS(d, ((uintptr_t)(i) >> 48));	      \
+				if ((uintptr_t)(i) & 0x0000ffff00000000ULL)   \
+					PPC_ORI(d, d,			      \
+					  ((uintptr_t)(i) >> 32) & 0xffff);   \
+			}						      \
 			PPC_SLDI(d, d, 32);				      \
 			if ((uintptr_t)(i) & 0x00000000ffff0000ULL)	      \
 				PPC_ORIS(d, d,				      \
 					 ((uintptr_t)(i) >> 16) & 0xffff);    \
 			if ((uintptr_t)(i) & 0x000000000000ffffULL)	      \
 				PPC_ORI(d, d, (uintptr_t)(i) & 0xffff);	      \
-		} } while (0);
+		} } while (0)
 
 #ifdef CONFIG_PPC64
 #define PPC_FUNC_ADDR(d,i) do { PPC_LI64(d, i); } while(0)
-- 
2.8.2

^ permalink raw reply related	[flat|nested] 34+ messages in thread

* [PATCH 3/6] ppc: bpf/jit: Introduce rotate immediate instructions
  2016-06-07 13:32 [PATCH 0/6] eBPF JIT for PPC64 Naveen N. Rao
  2016-06-07 13:32 ` [PATCH 1/6] ppc: bpf/jit: Fix/enhance 32-bit Load Immediate implementation Naveen N. Rao
  2016-06-07 13:32 ` [PATCH 2/6] ppc: bpf/jit: Optimize 64-bit Immediate loads Naveen N. Rao
@ 2016-06-07 13:32 ` Naveen N. Rao
  2016-06-07 13:32 ` [PATCH 4/6] ppc: bpf/jit: A few cleanups Naveen N. Rao
                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-07 13:32 UTC (permalink / raw)
  To: linux-kernel, linuxppc-dev, netdev, mpe
  Cc: Matt Evans, Denis Kirjanov, Paul Mackerras, Alexei Starovoitov,
	Daniel Borkmann, David S. Miller, Ananth N Mavinakayanahalli

Since we will be using the rotate immediate instructions for extended
BPF JIT, let's introduce macros for the same. And since the shift
immediate operations use the rotate immediate instructions, let's redo
those macros to use the newly introduced instructions.

Cc: Matt Evans <matt@ozlabs.org>
Cc: Denis Kirjanov <kda@linux-powerpc.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
---
 arch/powerpc/include/asm/ppc-opcode.h |  2 ++
 arch/powerpc/net/bpf_jit.h            | 20 +++++++++++---------
 2 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/arch/powerpc/include/asm/ppc-opcode.h b/arch/powerpc/include/asm/ppc-opcode.h
index 1d035c1..fd8d640 100644
--- a/arch/powerpc/include/asm/ppc-opcode.h
+++ b/arch/powerpc/include/asm/ppc-opcode.h
@@ -272,6 +272,8 @@
 #define __PPC_SH(s)	__PPC_WS(s)
 #define __PPC_MB(s)	(((s) & 0x1f) << 6)
 #define __PPC_ME(s)	(((s) & 0x1f) << 1)
+#define __PPC_MB64(s)	(__PPC_MB(s) | ((s) & 0x20))
+#define __PPC_ME64(s)	__PPC_MB64(s)
 #define __PPC_BI(s)	(((s) & 0x1f) << 16)
 #define __PPC_CT(t)	(((t) & 0x0f) << 21)
 
diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h
index 4c1e055..95d0e38 100644
--- a/arch/powerpc/net/bpf_jit.h
+++ b/arch/powerpc/net/bpf_jit.h
@@ -210,18 +210,20 @@ DECLARE_LOAD_FUNC(sk_load_byte_msh);
 				     ___PPC_RS(a) | ___PPC_RB(s))
 #define PPC_SRW(d, a, s)	EMIT(PPC_INST_SRW | ___PPC_RA(d) |	      \
 				     ___PPC_RS(a) | ___PPC_RB(s))
+#define PPC_RLWINM(d, a, i, mb, me)	EMIT(PPC_INST_RLWINM | ___PPC_RA(d) | \
+					___PPC_RS(a) | __PPC_SH(i) |	      \
+					__PPC_MB(mb) | __PPC_ME(me))
+#define PPC_RLDICR(d, a, i, me)		EMIT(PPC_INST_RLDICR | ___PPC_RA(d) | \
+					___PPC_RS(a) | __PPC_SH(i) |	      \
+					__PPC_ME64(me) | (((i) & 0x20) >> 4))
+
 /* slwi = rlwinm Rx, Ry, n, 0, 31-n */
-#define PPC_SLWI(d, a, i)	EMIT(PPC_INST_RLWINM | ___PPC_RA(d) |	      \
-				     ___PPC_RS(a) | __PPC_SH(i) |	      \
-				     __PPC_MB(0) | __PPC_ME(31-(i)))
+#define PPC_SLWI(d, a, i)	PPC_RLWINM(d, a, i, 0, 31-(i))
 /* srwi = rlwinm Rx, Ry, 32-n, n, 31 */
-#define PPC_SRWI(d, a, i)	EMIT(PPC_INST_RLWINM | ___PPC_RA(d) |	      \
-				     ___PPC_RS(a) | __PPC_SH(32-(i)) |	      \
-				     __PPC_MB(i) | __PPC_ME(31))
+#define PPC_SRWI(d, a, i)	PPC_RLWINM(d, a, 32-(i), i, 31)
 /* sldi = rldicr Rx, Ry, n, 63-n */
-#define PPC_SLDI(d, a, i)	EMIT(PPC_INST_RLDICR | ___PPC_RA(d) |	      \
-				     ___PPC_RS(a) | __PPC_SH(i) |	      \
-				     __PPC_MB(63-(i)) | (((i) & 0x20) >> 4))
+#define PPC_SLDI(d, a, i)	PPC_RLDICR(d, a, i, 63-(i))
+
 #define PPC_NEG(d, a)		EMIT(PPC_INST_NEG | ___PPC_RT(d) | ___PPC_RA(a))
 
 /* Long jump; (unconditional 'branch') */
-- 
2.8.2

^ permalink raw reply related	[flat|nested] 34+ messages in thread

* [PATCH 4/6] ppc: bpf/jit: A few cleanups
  2016-06-07 13:32 [PATCH 0/6] eBPF JIT for PPC64 Naveen N. Rao
                   ` (2 preceding siblings ...)
  2016-06-07 13:32 ` [PATCH 3/6] ppc: bpf/jit: Introduce rotate immediate instructions Naveen N. Rao
@ 2016-06-07 13:32 ` Naveen N. Rao
  2016-06-07 13:32 ` [PATCH 5/6] ppc: bpf/jit: Isolate classic BPF JIT specifics into a separate header Naveen N. Rao
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-07 13:32 UTC (permalink / raw)
  To: linux-kernel, linuxppc-dev, netdev, mpe
  Cc: Matt Evans, Denis Kirjanov, Paul Mackerras, Alexei Starovoitov,
	Daniel Borkmann, David S. Miller, Ananth N Mavinakayanahalli

1. Per the ISA, ADDIS actually uses RT, rather than RS. Though
the result is the same, make the usage clear.
2. The multiply instruction used is a 32-bit multiply. Rename PPC_MUL()
to PPC_MULW() to make the same clear.
3. PPC_STW[U] take the entire 16-bit immediate value and do not require
word-alignment, per the ISA. Change the macros to use IMM_L().
4. A few white-space cleanups to satisfy checkpatch.pl.

Cc: Matt Evans <matt@ozlabs.org>
Cc: Denis Kirjanov <kda@linux-powerpc.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
---
 arch/powerpc/net/bpf_jit.h      | 13 +++++++------
 arch/powerpc/net/bpf_jit_comp.c |  8 ++++----
 2 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h
index 95d0e38..9041d3f 100644
--- a/arch/powerpc/net/bpf_jit.h
+++ b/arch/powerpc/net/bpf_jit.h
@@ -83,7 +83,7 @@ DECLARE_LOAD_FUNC(sk_load_byte_msh);
  */
 #define IMM_H(i)		((uintptr_t)(i)>>16)
 #define IMM_HA(i)		(((uintptr_t)(i)>>16) +			      \
-				 (((uintptr_t)(i) & 0x8000) >> 15))
+					(((uintptr_t)(i) & 0x8000) >> 15))
 #define IMM_L(i)		((uintptr_t)(i) & 0xffff)
 
 #define PLANT_INSTR(d, idx, instr)					      \
@@ -99,16 +99,16 @@ DECLARE_LOAD_FUNC(sk_load_byte_msh);
 #define PPC_MR(d, a)		PPC_OR(d, a, a)
 #define PPC_LI(r, i)		PPC_ADDI(r, 0, i)
 #define PPC_ADDIS(d, a, i)	EMIT(PPC_INST_ADDIS |			      \
-				     ___PPC_RS(d) | ___PPC_RA(a) | IMM_L(i))
+				     ___PPC_RT(d) | ___PPC_RA(a) | IMM_L(i))
 #define PPC_LIS(r, i)		PPC_ADDIS(r, 0, i)
 #define PPC_STD(r, base, i)	EMIT(PPC_INST_STD | ___PPC_RS(r) |	      \
 				     ___PPC_RA(base) | ((i) & 0xfffc))
 #define PPC_STDU(r, base, i)	EMIT(PPC_INST_STDU | ___PPC_RS(r) |	      \
 				     ___PPC_RA(base) | ((i) & 0xfffc))
 #define PPC_STW(r, base, i)	EMIT(PPC_INST_STW | ___PPC_RS(r) |	      \
-				     ___PPC_RA(base) | ((i) & 0xfffc))
+				     ___PPC_RA(base) | IMM_L(i))
 #define PPC_STWU(r, base, i)	EMIT(PPC_INST_STWU | ___PPC_RS(r) |	      \
-				     ___PPC_RA(base) | ((i) & 0xfffc))
+				     ___PPC_RA(base) | IMM_L(i))
 
 #define PPC_LBZ(r, base, i)	EMIT(PPC_INST_LBZ | ___PPC_RT(r) |	      \
 				     ___PPC_RA(base) | IMM_L(i))
@@ -174,13 +174,14 @@ DECLARE_LOAD_FUNC(sk_load_byte_msh);
 #define PPC_CMPWI(a, i)		EMIT(PPC_INST_CMPWI | ___PPC_RA(a) | IMM_L(i))
 #define PPC_CMPDI(a, i)		EMIT(PPC_INST_CMPDI | ___PPC_RA(a) | IMM_L(i))
 #define PPC_CMPLWI(a, i)	EMIT(PPC_INST_CMPLWI | ___PPC_RA(a) | IMM_L(i))
-#define PPC_CMPLW(a, b)		EMIT(PPC_INST_CMPLW | ___PPC_RA(a) | ___PPC_RB(b))
+#define PPC_CMPLW(a, b)		EMIT(PPC_INST_CMPLW | ___PPC_RA(a) |	      \
+					___PPC_RB(b))
 
 #define PPC_SUB(d, a, b)	EMIT(PPC_INST_SUB | ___PPC_RT(d) |	      \
 				     ___PPC_RB(a) | ___PPC_RA(b))
 #define PPC_ADD(d, a, b)	EMIT(PPC_INST_ADD | ___PPC_RT(d) |	      \
 				     ___PPC_RA(a) | ___PPC_RB(b))
-#define PPC_MUL(d, a, b)	EMIT(PPC_INST_MULLW | ___PPC_RT(d) |	      \
+#define PPC_MULW(d, a, b)	EMIT(PPC_INST_MULLW | ___PPC_RT(d) |	      \
 				     ___PPC_RA(a) | ___PPC_RB(b))
 #define PPC_MULHWU(d, a, b)	EMIT(PPC_INST_MULHWU | ___PPC_RT(d) |	      \
 				     ___PPC_RA(a) | ___PPC_RB(b))
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 2d66a84..6012aac 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -161,14 +161,14 @@ static int bpf_jit_build_body(struct bpf_prog *fp, u32 *image,
 			break;
 		case BPF_ALU | BPF_MUL | BPF_X: /* A *= X; */
 			ctx->seen |= SEEN_XREG;
-			PPC_MUL(r_A, r_A, r_X);
+			PPC_MULW(r_A, r_A, r_X);
 			break;
 		case BPF_ALU | BPF_MUL | BPF_K: /* A *= K */
 			if (K < 32768)
 				PPC_MULI(r_A, r_A, K);
 			else {
 				PPC_LI32(r_scratch1, K);
-				PPC_MUL(r_A, r_A, r_scratch1);
+				PPC_MULW(r_A, r_A, r_scratch1);
 			}
 			break;
 		case BPF_ALU | BPF_MOD | BPF_X: /* A %= X; */
@@ -184,7 +184,7 @@ static int bpf_jit_build_body(struct bpf_prog *fp, u32 *image,
 			}
 			if (code == (BPF_ALU | BPF_MOD | BPF_X)) {
 				PPC_DIVWU(r_scratch1, r_A, r_X);
-				PPC_MUL(r_scratch1, r_X, r_scratch1);
+				PPC_MULW(r_scratch1, r_X, r_scratch1);
 				PPC_SUB(r_A, r_A, r_scratch1);
 			} else {
 				PPC_DIVWU(r_A, r_A, r_X);
@@ -193,7 +193,7 @@ static int bpf_jit_build_body(struct bpf_prog *fp, u32 *image,
 		case BPF_ALU | BPF_MOD | BPF_K: /* A %= K; */
 			PPC_LI32(r_scratch2, K);
 			PPC_DIVWU(r_scratch1, r_A, r_scratch2);
-			PPC_MUL(r_scratch1, r_scratch2, r_scratch1);
+			PPC_MULW(r_scratch1, r_scratch2, r_scratch1);
 			PPC_SUB(r_A, r_A, r_scratch1);
 			break;
 		case BPF_ALU | BPF_DIV | BPF_K: /* A /= K */
-- 
2.8.2

^ permalink raw reply related	[flat|nested] 34+ messages in thread

* [PATCH 5/6] ppc: bpf/jit: Isolate classic BPF JIT specifics into a separate header
  2016-06-07 13:32 [PATCH 0/6] eBPF JIT for PPC64 Naveen N. Rao
                   ` (3 preceding siblings ...)
  2016-06-07 13:32 ` [PATCH 4/6] ppc: bpf/jit: A few cleanups Naveen N. Rao
@ 2016-06-07 13:32 ` Naveen N. Rao
  2016-06-07 13:32 ` [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF Naveen N. Rao
  2016-06-11  5:47 ` [PATCH 0/6] eBPF JIT for PPC64 David Miller
  6 siblings, 0 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-07 13:32 UTC (permalink / raw)
  To: linux-kernel, linuxppc-dev, netdev, mpe
  Cc: Matt Evans, Denis Kirjanov, Paul Mackerras, Alexei Starovoitov,
	Daniel Borkmann, David S. Miller, Ananth N Mavinakayanahalli

Break out classic BPF JIT specifics into a separate header in
preparation for eBPF JIT implementation. Note that ppc32 will still need
the classic BPF JIT.

Cc: Matt Evans <matt@ozlabs.org>
Cc: Denis Kirjanov <kda@linux-powerpc.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
---
 arch/powerpc/net/bpf_jit.h      | 121 +---------------------------------
 arch/powerpc/net/bpf_jit32.h    | 139 ++++++++++++++++++++++++++++++++++++++++
 arch/powerpc/net/bpf_jit_asm.S  |   2 +-
 arch/powerpc/net/bpf_jit_comp.c |   2 +-
 4 files changed, 143 insertions(+), 121 deletions(-)
 create mode 100644 arch/powerpc/net/bpf_jit32.h

diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h
index 9041d3f..313cfaf 100644
--- a/arch/powerpc/net/bpf_jit.h
+++ b/arch/powerpc/net/bpf_jit.h
@@ -1,4 +1,5 @@
-/* bpf_jit.h: BPF JIT compiler for PPC64
+/*
+ * bpf_jit.h: BPF JIT compiler for PPC
  *
  * Copyright 2011 Matt Evans <matt@ozlabs.org>, IBM Corporation
  *
@@ -10,66 +11,8 @@
 #ifndef _BPF_JIT_H
 #define _BPF_JIT_H
 
-#ifdef CONFIG_PPC64
-#define BPF_PPC_STACK_R3_OFF	48
-#define BPF_PPC_STACK_LOCALS	32
-#define BPF_PPC_STACK_BASIC	(48+64)
-#define BPF_PPC_STACK_SAVE	(18*8)
-#define BPF_PPC_STACKFRAME	(BPF_PPC_STACK_BASIC+BPF_PPC_STACK_LOCALS+ \
-				 BPF_PPC_STACK_SAVE)
-#define BPF_PPC_SLOWPATH_FRAME	(48+64)
-#else
-#define BPF_PPC_STACK_R3_OFF	24
-#define BPF_PPC_STACK_LOCALS	16
-#define BPF_PPC_STACK_BASIC	(24+32)
-#define BPF_PPC_STACK_SAVE	(18*4)
-#define BPF_PPC_STACKFRAME	(BPF_PPC_STACK_BASIC+BPF_PPC_STACK_LOCALS+ \
-				 BPF_PPC_STACK_SAVE)
-#define BPF_PPC_SLOWPATH_FRAME	(24+32)
-#endif
-
-#define REG_SZ         (BITS_PER_LONG/8)
-
-/*
- * Generated code register usage:
- *
- * As normal PPC C ABI (e.g. r1=sp, r2=TOC), with:
- *
- * skb		r3	(Entry parameter)
- * A register	r4
- * X register	r5
- * addr param	r6
- * r7-r10	scratch
- * skb->data	r14
- * skb headlen	r15	(skb->len - skb->data_len)
- * m[0]		r16
- * m[...]	...
- * m[15]	r31
- */
-#define r_skb		3
-#define r_ret		3
-#define r_A		4
-#define r_X		5
-#define r_addr		6
-#define r_scratch1	7
-#define r_scratch2	8
-#define r_D		14
-#define r_HL		15
-#define r_M		16
-
 #ifndef __ASSEMBLY__
 
-/*
- * Assembly helpers from arch/powerpc/net/bpf_jit.S:
- */
-#define DECLARE_LOAD_FUNC(func)	\
-	extern u8 func[], func##_negative_offset[], func##_positive_offset[]
-
-DECLARE_LOAD_FUNC(sk_load_word);
-DECLARE_LOAD_FUNC(sk_load_half);
-DECLARE_LOAD_FUNC(sk_load_byte);
-DECLARE_LOAD_FUNC(sk_load_byte_msh);
-
 #ifdef CONFIG_PPC64
 #define FUNCTION_DESCR_SIZE	24
 #else
@@ -131,46 +74,6 @@ DECLARE_LOAD_FUNC(sk_load_byte_msh);
 #define PPC_BPF_STLU(r, base, i) do { PPC_STWU(r, base, i); } while(0)
 #endif
 
-/* Convenience helpers for the above with 'far' offsets: */
-#define PPC_LBZ_OFFS(r, base, i) do { if ((i) < 32768) PPC_LBZ(r, base, i);   \
-		else {	PPC_ADDIS(r, base, IMM_HA(i));			      \
-			PPC_LBZ(r, r, IMM_L(i)); } } while(0)
-
-#define PPC_LD_OFFS(r, base, i) do { if ((i) < 32768) PPC_LD(r, base, i);     \
-		else {	PPC_ADDIS(r, base, IMM_HA(i));			      \
-			PPC_LD(r, r, IMM_L(i)); } } while(0)
-
-#define PPC_LWZ_OFFS(r, base, i) do { if ((i) < 32768) PPC_LWZ(r, base, i);   \
-		else {	PPC_ADDIS(r, base, IMM_HA(i));			      \
-			PPC_LWZ(r, r, IMM_L(i)); } } while(0)
-
-#define PPC_LHZ_OFFS(r, base, i) do { if ((i) < 32768) PPC_LHZ(r, base, i);   \
-		else {	PPC_ADDIS(r, base, IMM_HA(i));			      \
-			PPC_LHZ(r, r, IMM_L(i)); } } while(0)
-
-#ifdef CONFIG_PPC64
-#define PPC_LL_OFFS(r, base, i) do { PPC_LD_OFFS(r, base, i); } while(0)
-#else
-#define PPC_LL_OFFS(r, base, i) do { PPC_LWZ_OFFS(r, base, i); } while(0)
-#endif
-
-#ifdef CONFIG_SMP
-#ifdef CONFIG_PPC64
-#define PPC_BPF_LOAD_CPU(r)		\
-	do { BUILD_BUG_ON(FIELD_SIZEOF(struct paca_struct, paca_index) != 2);	\
-		PPC_LHZ_OFFS(r, 13, offsetof(struct paca_struct, paca_index));		\
-	} while (0)
-#else
-#define PPC_BPF_LOAD_CPU(r)     \
-	do { BUILD_BUG_ON(FIELD_SIZEOF(struct thread_info, cpu) != 4);			\
-		PPC_LHZ_OFFS(r, (1 & ~(THREAD_SIZE - 1)),							\
-				offsetof(struct thread_info, cpu));							\
-	} while(0)
-#endif
-#else
-#define PPC_BPF_LOAD_CPU(r) do { PPC_LI(r, 0); } while(0)
-#endif
-
 #define PPC_CMPWI(a, i)		EMIT(PPC_INST_CMPWI | ___PPC_RA(a) | IMM_L(i))
 #define PPC_CMPDI(a, i)		EMIT(PPC_INST_CMPDI | ___PPC_RA(a) | IMM_L(i))
 #define PPC_CMPLWI(a, i)	EMIT(PPC_INST_CMPLWI | ___PPC_RA(a) | IMM_L(i))
@@ -273,14 +176,6 @@ DECLARE_LOAD_FUNC(sk_load_byte_msh);
 #define PPC_FUNC_ADDR(d,i) do { PPC_LI32(d, i); } while(0)
 #endif
 
-#define PPC_LHBRX_OFFS(r, base, i) \
-		do { PPC_LI32(r, i); PPC_LHBRX(r, r, base); } while(0)
-#ifdef __LITTLE_ENDIAN__
-#define PPC_NTOHS_OFFS(r, base, i)	PPC_LHBRX_OFFS(r, base, i)
-#else
-#define PPC_NTOHS_OFFS(r, base, i)	PPC_LHZ_OFFS(r, base, i)
-#endif
-
 static inline bool is_nearbranch(int offset)
 {
 	return (offset < 32768) && (offset >= -32768);
@@ -317,18 +212,6 @@ static inline bool is_nearbranch(int offset)
 #define COND_NE		(CR0_EQ | COND_CMP_FALSE)
 #define COND_LT		(CR0_LT | COND_CMP_TRUE)
 
-#define SEEN_DATAREF 0x10000 /* might call external helpers */
-#define SEEN_XREG    0x20000 /* X reg is used */
-#define SEEN_MEM     0x40000 /* SEEN_MEM+(1<<n) = use mem[n] for temporary
-			      * storage */
-#define SEEN_MEM_MSK 0x0ffff
-
-struct codegen_context {
-	unsigned int seen;
-	unsigned int idx;
-	int pc_ret0; /* bpf index of first RET #0 instruction (if any) */
-};
-
 #endif
 
 #endif
diff --git a/arch/powerpc/net/bpf_jit32.h b/arch/powerpc/net/bpf_jit32.h
new file mode 100644
index 0000000..a8cd7e2
--- /dev/null
+++ b/arch/powerpc/net/bpf_jit32.h
@@ -0,0 +1,139 @@
+/*
+ * bpf_jit32.h: BPF JIT compiler for PPC
+ *
+ * Copyright 2011 Matt Evans <matt@ozlabs.org>, IBM Corporation
+ *
+ * Split from bpf_jit.h
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2
+ * of the License.
+ */
+#ifndef _BPF_JIT32_H
+#define _BPF_JIT32_H
+
+#include "bpf_jit.h"
+
+#ifdef CONFIG_PPC64
+#define BPF_PPC_STACK_R3_OFF	48
+#define BPF_PPC_STACK_LOCALS	32
+#define BPF_PPC_STACK_BASIC	(48+64)
+#define BPF_PPC_STACK_SAVE	(18*8)
+#define BPF_PPC_STACKFRAME	(BPF_PPC_STACK_BASIC+BPF_PPC_STACK_LOCALS+ \
+				 BPF_PPC_STACK_SAVE)
+#define BPF_PPC_SLOWPATH_FRAME	(48+64)
+#else
+#define BPF_PPC_STACK_R3_OFF	24
+#define BPF_PPC_STACK_LOCALS	16
+#define BPF_PPC_STACK_BASIC	(24+32)
+#define BPF_PPC_STACK_SAVE	(18*4)
+#define BPF_PPC_STACKFRAME	(BPF_PPC_STACK_BASIC+BPF_PPC_STACK_LOCALS+ \
+				 BPF_PPC_STACK_SAVE)
+#define BPF_PPC_SLOWPATH_FRAME	(24+32)
+#endif
+
+#define REG_SZ         (BITS_PER_LONG/8)
+
+/*
+ * Generated code register usage:
+ *
+ * As normal PPC C ABI (e.g. r1=sp, r2=TOC), with:
+ *
+ * skb		r3	(Entry parameter)
+ * A register	r4
+ * X register	r5
+ * addr param	r6
+ * r7-r10	scratch
+ * skb->data	r14
+ * skb headlen	r15	(skb->len - skb->data_len)
+ * m[0]		r16
+ * m[...]	...
+ * m[15]	r31
+ */
+#define r_skb		3
+#define r_ret		3
+#define r_A		4
+#define r_X		5
+#define r_addr		6
+#define r_scratch1	7
+#define r_scratch2	8
+#define r_D		14
+#define r_HL		15
+#define r_M		16
+
+#ifndef __ASSEMBLY__
+
+/*
+ * Assembly helpers from arch/powerpc/net/bpf_jit.S:
+ */
+#define DECLARE_LOAD_FUNC(func)	\
+	extern u8 func[], func##_negative_offset[], func##_positive_offset[]
+
+DECLARE_LOAD_FUNC(sk_load_word);
+DECLARE_LOAD_FUNC(sk_load_half);
+DECLARE_LOAD_FUNC(sk_load_byte);
+DECLARE_LOAD_FUNC(sk_load_byte_msh);
+
+#define PPC_LBZ_OFFS(r, base, i) do { if ((i) < 32768) PPC_LBZ(r, base, i);   \
+		else {	PPC_ADDIS(r, base, IMM_HA(i));			      \
+			PPC_LBZ(r, r, IMM_L(i)); } } while(0)
+
+#define PPC_LD_OFFS(r, base, i) do { if ((i) < 32768) PPC_LD(r, base, i);     \
+		else {	PPC_ADDIS(r, base, IMM_HA(i));			      \
+			PPC_LD(r, r, IMM_L(i)); } } while(0)
+
+#define PPC_LWZ_OFFS(r, base, i) do { if ((i) < 32768) PPC_LWZ(r, base, i);   \
+		else {	PPC_ADDIS(r, base, IMM_HA(i));			      \
+			PPC_LWZ(r, r, IMM_L(i)); } } while(0)
+
+#define PPC_LHZ_OFFS(r, base, i) do { if ((i) < 32768) PPC_LHZ(r, base, i);   \
+		else {	PPC_ADDIS(r, base, IMM_HA(i));			      \
+			PPC_LHZ(r, r, IMM_L(i)); } } while(0)
+
+#ifdef CONFIG_PPC64
+#define PPC_LL_OFFS(r, base, i) do { PPC_LD_OFFS(r, base, i); } while(0)
+#else
+#define PPC_LL_OFFS(r, base, i) do { PPC_LWZ_OFFS(r, base, i); } while(0)
+#endif
+
+#ifdef CONFIG_SMP
+#ifdef CONFIG_PPC64
+#define PPC_BPF_LOAD_CPU(r)		\
+	do { BUILD_BUG_ON(FIELD_SIZEOF(struct paca_struct, paca_index) != 2);	\
+		PPC_LHZ_OFFS(r, 13, offsetof(struct paca_struct, paca_index));	\
+	} while (0)
+#else
+#define PPC_BPF_LOAD_CPU(r)     \
+	do { BUILD_BUG_ON(FIELD_SIZEOF(struct thread_info, cpu) != 4);		\
+		PPC_LHZ_OFFS(r, (1 & ~(THREAD_SIZE - 1)),			\
+				offsetof(struct thread_info, cpu));		\
+	} while(0)
+#endif
+#else
+#define PPC_BPF_LOAD_CPU(r) do { PPC_LI(r, 0); } while(0)
+#endif
+
+#define PPC_LHBRX_OFFS(r, base, i) \
+		do { PPC_LI32(r, i); PPC_LHBRX(r, r, base); } while(0)
+#ifdef __LITTLE_ENDIAN__
+#define PPC_NTOHS_OFFS(r, base, i)	PPC_LHBRX_OFFS(r, base, i)
+#else
+#define PPC_NTOHS_OFFS(r, base, i)	PPC_LHZ_OFFS(r, base, i)
+#endif
+
+#define SEEN_DATAREF 0x10000 /* might call external helpers */
+#define SEEN_XREG    0x20000 /* X reg is used */
+#define SEEN_MEM     0x40000 /* SEEN_MEM+(1<<n) = use mem[n] for temporary
+			      * storage */
+#define SEEN_MEM_MSK 0x0ffff
+
+struct codegen_context {
+	unsigned int seen;
+	unsigned int idx;
+	int pc_ret0; /* bpf index of first RET #0 instruction (if any) */
+};
+
+#endif
+
+#endif
diff --git a/arch/powerpc/net/bpf_jit_asm.S b/arch/powerpc/net/bpf_jit_asm.S
index 8ff5a3b..3dd9c43 100644
--- a/arch/powerpc/net/bpf_jit_asm.S
+++ b/arch/powerpc/net/bpf_jit_asm.S
@@ -10,7 +10,7 @@
  */
 
 #include <asm/ppc_asm.h>
-#include "bpf_jit.h"
+#include "bpf_jit32.h"
 
 /*
  * All of these routines are called directly from generated code,
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 6012aac..7e706f3 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -16,7 +16,7 @@
 #include <linux/filter.h>
 #include <linux/if_vlan.h>
 
-#include "bpf_jit.h"
+#include "bpf_jit32.h"
 
 int bpf_jit_enable __read_mostly;
 
-- 
2.8.2

^ permalink raw reply related	[flat|nested] 34+ messages in thread

* [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-07 13:32 [PATCH 0/6] eBPF JIT for PPC64 Naveen N. Rao
                   ` (4 preceding siblings ...)
  2016-06-07 13:32 ` [PATCH 5/6] ppc: bpf/jit: Isolate classic BPF JIT specifics into a separate header Naveen N. Rao
@ 2016-06-07 13:32 ` Naveen N. Rao
  2016-06-07 22:56   ` Alexei Starovoitov
                     ` (3 more replies)
  2016-06-11  5:47 ` [PATCH 0/6] eBPF JIT for PPC64 David Miller
  6 siblings, 4 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-07 13:32 UTC (permalink / raw)
  To: linux-kernel, linuxppc-dev, netdev, mpe
  Cc: Matt Evans, Denis Kirjanov, Paul Mackerras, Alexei Starovoitov,
	Daniel Borkmann, David S. Miller, Ananth N Mavinakayanahalli

PPC64 eBPF JIT compiler.

Enable with:
echo 1 > /proc/sys/net/core/bpf_jit_enable
or
echo 2 > /proc/sys/net/core/bpf_jit_enable

... to see the generated JIT code. This can further be processed with
tools/net/bpf_jit_disasm.

With CONFIG_TEST_BPF=m and 'modprobe test_bpf':
test_bpf: Summary: 305 PASSED, 0 FAILED, [297/297 JIT'ed]

... on both ppc64 BE and LE.

The details of the approach are documented through various comments in
the code.

Cc: Matt Evans <matt@ozlabs.org>
Cc: Denis Kirjanov <kda@linux-powerpc.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
---
 arch/powerpc/Kconfig                  |   3 +-
 arch/powerpc/include/asm/asm-compat.h |   2 +
 arch/powerpc/include/asm/ppc-opcode.h |  20 +-
 arch/powerpc/net/Makefile             |   4 +
 arch/powerpc/net/bpf_jit.h            |  53 +-
 arch/powerpc/net/bpf_jit64.h          | 102 ++++
 arch/powerpc/net/bpf_jit_asm64.S      | 180 +++++++
 arch/powerpc/net/bpf_jit_comp64.c     | 956 ++++++++++++++++++++++++++++++++++
 8 files changed, 1317 insertions(+), 3 deletions(-)
 create mode 100644 arch/powerpc/net/bpf_jit64.h
 create mode 100644 arch/powerpc/net/bpf_jit_asm64.S
 create mode 100644 arch/powerpc/net/bpf_jit_comp64.c

diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index 01f7464..ee82f9a 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -128,7 +128,8 @@ config PPC
 	select IRQ_FORCED_THREADING
 	select HAVE_RCU_TABLE_FREE if SMP
 	select HAVE_SYSCALL_TRACEPOINTS
-	select HAVE_CBPF_JIT
+	select HAVE_CBPF_JIT if !PPC64
+	select HAVE_EBPF_JIT if PPC64
 	select HAVE_ARCH_JUMP_LABEL
 	select ARCH_HAVE_NMI_SAFE_CMPXCHG
 	select ARCH_HAS_GCOV_PROFILE_ALL
diff --git a/arch/powerpc/include/asm/asm-compat.h b/arch/powerpc/include/asm/asm-compat.h
index dc85dcb..cee3aa0 100644
--- a/arch/powerpc/include/asm/asm-compat.h
+++ b/arch/powerpc/include/asm/asm-compat.h
@@ -36,11 +36,13 @@
 #define PPC_MIN_STKFRM	112
 
 #ifdef __BIG_ENDIAN__
+#define LHZX_BE	stringify_in_c(lhzx)
 #define LWZX_BE	stringify_in_c(lwzx)
 #define LDX_BE	stringify_in_c(ldx)
 #define STWX_BE	stringify_in_c(stwx)
 #define STDX_BE	stringify_in_c(stdx)
 #else
+#define LHZX_BE	stringify_in_c(lhbrx)
 #define LWZX_BE	stringify_in_c(lwbrx)
 #define LDX_BE	stringify_in_c(ldbrx)
 #define STWX_BE	stringify_in_c(stwbrx)
diff --git a/arch/powerpc/include/asm/ppc-opcode.h b/arch/powerpc/include/asm/ppc-opcode.h
index fd8d640..6a77d130 100644
--- a/arch/powerpc/include/asm/ppc-opcode.h
+++ b/arch/powerpc/include/asm/ppc-opcode.h
@@ -142,9 +142,11 @@
 #define PPC_INST_ISEL			0x7c00001e
 #define PPC_INST_ISEL_MASK		0xfc00003e
 #define PPC_INST_LDARX			0x7c0000a8
+#define PPC_INST_STDCX			0x7c0001ad
 #define PPC_INST_LSWI			0x7c0004aa
 #define PPC_INST_LSWX			0x7c00042a
 #define PPC_INST_LWARX			0x7c000028
+#define PPC_INST_STWCX			0x7c00012d
 #define PPC_INST_LWSYNC			0x7c2004ac
 #define PPC_INST_SYNC			0x7c0004ac
 #define PPC_INST_SYNC_MASK		0xfc0007fe
@@ -211,8 +213,11 @@
 #define PPC_INST_LBZ			0x88000000
 #define PPC_INST_LD			0xe8000000
 #define PPC_INST_LHZ			0xa0000000
-#define PPC_INST_LHBRX			0x7c00062c
 #define PPC_INST_LWZ			0x80000000
+#define PPC_INST_LHBRX			0x7c00062c
+#define PPC_INST_LDBRX			0x7c000428
+#define PPC_INST_STB			0x98000000
+#define PPC_INST_STH			0xb0000000
 #define PPC_INST_STD			0xf8000000
 #define PPC_INST_STDU			0xf8000001
 #define PPC_INST_STW			0x90000000
@@ -221,22 +226,34 @@
 #define PPC_INST_MTLR			0x7c0803a6
 #define PPC_INST_CMPWI			0x2c000000
 #define PPC_INST_CMPDI			0x2c200000
+#define PPC_INST_CMPW			0x7c000000
+#define PPC_INST_CMPD			0x7c200000
 #define PPC_INST_CMPLW			0x7c000040
+#define PPC_INST_CMPLD			0x7c200040
 #define PPC_INST_CMPLWI			0x28000000
+#define PPC_INST_CMPLDI			0x28200000
 #define PPC_INST_ADDI			0x38000000
 #define PPC_INST_ADDIS			0x3c000000
 #define PPC_INST_ADD			0x7c000214
 #define PPC_INST_SUB			0x7c000050
 #define PPC_INST_BLR			0x4e800020
 #define PPC_INST_BLRL			0x4e800021
+#define PPC_INST_MULLD			0x7c0001d2
 #define PPC_INST_MULLW			0x7c0001d6
 #define PPC_INST_MULHWU			0x7c000016
 #define PPC_INST_MULLI			0x1c000000
 #define PPC_INST_DIVWU			0x7c000396
+#define PPC_INST_DIVD			0x7c0003d2
 #define PPC_INST_RLWINM			0x54000000
+#define PPC_INST_RLWIMI			0x50000000
+#define PPC_INST_RLDICL			0x78000000
 #define PPC_INST_RLDICR			0x78000004
 #define PPC_INST_SLW			0x7c000030
+#define PPC_INST_SLD			0x7c000036
 #define PPC_INST_SRW			0x7c000430
+#define PPC_INST_SRD			0x7c000436
+#define PPC_INST_SRAD			0x7c000634
+#define PPC_INST_SRADI			0x7c000674
 #define PPC_INST_AND			0x7c000038
 #define PPC_INST_ANDDOT			0x7c000039
 #define PPC_INST_OR			0x7c000378
@@ -247,6 +264,7 @@
 #define PPC_INST_XORI			0x68000000
 #define PPC_INST_XORIS			0x6c000000
 #define PPC_INST_NEG			0x7c0000d0
+#define PPC_INST_EXTSW			0x7c0007b4
 #define PPC_INST_BRANCH			0x48000000
 #define PPC_INST_BRANCH_COND		0x40800000
 #define PPC_INST_LBZCIX			0x7c0006aa
diff --git a/arch/powerpc/net/Makefile b/arch/powerpc/net/Makefile
index 1306a58..c1ff16a 100644
--- a/arch/powerpc/net/Makefile
+++ b/arch/powerpc/net/Makefile
@@ -1,4 +1,8 @@
 #
 # Arch-specific network modules
 #
+ifeq ($(CONFIG_PPC64),y)
+obj-$(CONFIG_BPF_JIT) += bpf_jit_asm64.o bpf_jit_comp64.o
+else
 obj-$(CONFIG_BPF_JIT) += bpf_jit_asm.o bpf_jit_comp.o
+endif
diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h
index 313cfaf..d5301b6 100644
--- a/arch/powerpc/net/bpf_jit.h
+++ b/arch/powerpc/net/bpf_jit.h
@@ -2,6 +2,7 @@
  * bpf_jit.h: BPF JIT compiler for PPC
  *
  * Copyright 2011 Matt Evans <matt@ozlabs.org>, IBM Corporation
+ * 	     2016 Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License
@@ -13,7 +14,9 @@
 
 #ifndef __ASSEMBLY__
 
-#ifdef CONFIG_PPC64
+#include <asm/types.h>
+
+#ifdef PPC64_ELF_ABI_v1
 #define FUNCTION_DESCR_SIZE	24
 #else
 #define FUNCTION_DESCR_SIZE	0
@@ -52,6 +55,10 @@
 				     ___PPC_RA(base) | IMM_L(i))
 #define PPC_STWU(r, base, i)	EMIT(PPC_INST_STWU | ___PPC_RS(r) |	      \
 				     ___PPC_RA(base) | IMM_L(i))
+#define PPC_STH(r, base, i)	EMIT(PPC_INST_STH | ___PPC_RS(r) |	      \
+				     ___PPC_RA(base) | IMM_L(i))
+#define PPC_STB(r, base, i)	EMIT(PPC_INST_STB | ___PPC_RS(r) |	      \
+				     ___PPC_RA(base) | IMM_L(i))
 
 #define PPC_LBZ(r, base, i)	EMIT(PPC_INST_LBZ | ___PPC_RT(r) |	      \
 				     ___PPC_RA(base) | IMM_L(i))
@@ -63,6 +70,19 @@
 				     ___PPC_RA(base) | IMM_L(i))
 #define PPC_LHBRX(r, base, b)	EMIT(PPC_INST_LHBRX | ___PPC_RT(r) |	      \
 				     ___PPC_RA(base) | ___PPC_RB(b))
+#define PPC_LDBRX(r, base, b)	EMIT(PPC_INST_LDBRX | ___PPC_RT(r) |	      \
+				     ___PPC_RA(base) | ___PPC_RB(b))
+
+#define PPC_BPF_LDARX(t, a, b, eh) EMIT(PPC_INST_LDARX | ___PPC_RT(t) |	      \
+					___PPC_RA(a) | ___PPC_RB(b) |	      \
+					__PPC_EH(eh))
+#define PPC_BPF_LWARX(t, a, b, eh) EMIT(PPC_INST_LWARX | ___PPC_RT(t) |	      \
+					___PPC_RA(a) | ___PPC_RB(b) |	      \
+					__PPC_EH(eh))
+#define PPC_BPF_STWCX(s, a, b)	EMIT(PPC_INST_STWCX | ___PPC_RS(s) |	      \
+					___PPC_RA(a) | ___PPC_RB(b))
+#define PPC_BPF_STDCX(s, a, b)	EMIT(PPC_INST_STDCX | ___PPC_RS(s) |	      \
+					___PPC_RA(a) | ___PPC_RB(b))
 
 #ifdef CONFIG_PPC64
 #define PPC_BPF_LL(r, base, i) do { PPC_LD(r, base, i); } while(0)
@@ -76,14 +96,23 @@
 
 #define PPC_CMPWI(a, i)		EMIT(PPC_INST_CMPWI | ___PPC_RA(a) | IMM_L(i))
 #define PPC_CMPDI(a, i)		EMIT(PPC_INST_CMPDI | ___PPC_RA(a) | IMM_L(i))
+#define PPC_CMPW(a, b)		EMIT(PPC_INST_CMPW | ___PPC_RA(a) |	      \
+					___PPC_RB(b))
+#define PPC_CMPD(a, b)		EMIT(PPC_INST_CMPD | ___PPC_RA(a) |	      \
+					___PPC_RB(b))
 #define PPC_CMPLWI(a, i)	EMIT(PPC_INST_CMPLWI | ___PPC_RA(a) | IMM_L(i))
+#define PPC_CMPLDI(a, i)	EMIT(PPC_INST_CMPLDI | ___PPC_RA(a) | IMM_L(i))
 #define PPC_CMPLW(a, b)		EMIT(PPC_INST_CMPLW | ___PPC_RA(a) |	      \
 					___PPC_RB(b))
+#define PPC_CMPLD(a, b)		EMIT(PPC_INST_CMPLD | ___PPC_RA(a) |	      \
+					___PPC_RB(b))
 
 #define PPC_SUB(d, a, b)	EMIT(PPC_INST_SUB | ___PPC_RT(d) |	      \
 				     ___PPC_RB(a) | ___PPC_RA(b))
 #define PPC_ADD(d, a, b)	EMIT(PPC_INST_ADD | ___PPC_RT(d) |	      \
 				     ___PPC_RA(a) | ___PPC_RB(b))
+#define PPC_MULD(d, a, b)	EMIT(PPC_INST_MULLD | ___PPC_RT(d) |	      \
+				     ___PPC_RA(a) | ___PPC_RB(b))
 #define PPC_MULW(d, a, b)	EMIT(PPC_INST_MULLW | ___PPC_RT(d) |	      \
 				     ___PPC_RA(a) | ___PPC_RB(b))
 #define PPC_MULHWU(d, a, b)	EMIT(PPC_INST_MULHWU | ___PPC_RT(d) |	      \
@@ -92,6 +121,8 @@
 				     ___PPC_RA(a) | IMM_L(i))
 #define PPC_DIVWU(d, a, b)	EMIT(PPC_INST_DIVWU | ___PPC_RT(d) |	      \
 				     ___PPC_RA(a) | ___PPC_RB(b))
+#define PPC_DIVD(d, a, b)	EMIT(PPC_INST_DIVD | ___PPC_RT(d) |	      \
+				     ___PPC_RA(a) | ___PPC_RB(b))
 #define PPC_AND(d, a, b)	EMIT(PPC_INST_AND | ___PPC_RA(d) |	      \
 				     ___PPC_RS(a) | ___PPC_RB(b))
 #define PPC_ANDI(d, a, i)	EMIT(PPC_INST_ANDI | ___PPC_RA(d) |	      \
@@ -100,6 +131,7 @@
 				     ___PPC_RS(a) | ___PPC_RB(b))
 #define PPC_OR(d, a, b)		EMIT(PPC_INST_OR | ___PPC_RA(d) |	      \
 				     ___PPC_RS(a) | ___PPC_RB(b))
+#define PPC_MR(d, a)		PPC_OR(d, a, a)
 #define PPC_ORI(d, a, i)	EMIT(PPC_INST_ORI | ___PPC_RA(d) |	      \
 				     ___PPC_RS(a) | IMM_L(i))
 #define PPC_ORIS(d, a, i)	EMIT(PPC_INST_ORIS | ___PPC_RA(d) |	      \
@@ -110,13 +142,30 @@
 				     ___PPC_RS(a) | IMM_L(i))
 #define PPC_XORIS(d, a, i)	EMIT(PPC_INST_XORIS | ___PPC_RA(d) |	      \
 				     ___PPC_RS(a) | IMM_L(i))
+#define PPC_EXTSW(d, a)		EMIT(PPC_INST_EXTSW | ___PPC_RA(d) |	      \
+				     ___PPC_RS(a))
 #define PPC_SLW(d, a, s)	EMIT(PPC_INST_SLW | ___PPC_RA(d) |	      \
 				     ___PPC_RS(a) | ___PPC_RB(s))
+#define PPC_SLD(d, a, s)	EMIT(PPC_INST_SLD | ___PPC_RA(d) |	      \
+				     ___PPC_RS(a) | ___PPC_RB(s))
 #define PPC_SRW(d, a, s)	EMIT(PPC_INST_SRW | ___PPC_RA(d) |	      \
 				     ___PPC_RS(a) | ___PPC_RB(s))
+#define PPC_SRD(d, a, s)	EMIT(PPC_INST_SRD | ___PPC_RA(d) |	      \
+				     ___PPC_RS(a) | ___PPC_RB(s))
+#define PPC_SRAD(d, a, s)	EMIT(PPC_INST_SRAD | ___PPC_RA(d) |	      \
+				     ___PPC_RS(a) | ___PPC_RB(s))
+#define PPC_SRADI(d, a, i)	EMIT(PPC_INST_SRADI | ___PPC_RA(d) |	      \
+				     ___PPC_RS(a) | __PPC_SH(i) |             \
+				     (((i) & 0x20) >> 4))
 #define PPC_RLWINM(d, a, i, mb, me)	EMIT(PPC_INST_RLWINM | ___PPC_RA(d) | \
 					___PPC_RS(a) | __PPC_SH(i) |	      \
 					__PPC_MB(mb) | __PPC_ME(me))
+#define PPC_RLWIMI(d, a, i, mb, me)	EMIT(PPC_INST_RLWIMI | ___PPC_RA(d) | \
+					___PPC_RS(a) | __PPC_SH(i) |	      \
+					__PPC_MB(mb) | __PPC_ME(me))
+#define PPC_RLDICL(d, a, i, mb)		EMIT(PPC_INST_RLDICL | ___PPC_RA(d) | \
+					___PPC_RS(a) | __PPC_SH(i) |	      \
+					__PPC_MB64(mb) | (((i) & 0x20) >> 4))
 #define PPC_RLDICR(d, a, i, me)		EMIT(PPC_INST_RLDICR | ___PPC_RA(d) | \
 					___PPC_RS(a) | __PPC_SH(i) |	      \
 					__PPC_ME64(me) | (((i) & 0x20) >> 4))
@@ -127,6 +176,8 @@
 #define PPC_SRWI(d, a, i)	PPC_RLWINM(d, a, 32-(i), i, 31)
 /* sldi = rldicr Rx, Ry, n, 63-n */
 #define PPC_SLDI(d, a, i)	PPC_RLDICR(d, a, i, 63-(i))
+/* sldi = rldicl Rx, Ry, 64-n, n */
+#define PPC_SRDI(d, a, i)	PPC_RLDICL(d, a, 64-(i), i)
 
 #define PPC_NEG(d, a)		EMIT(PPC_INST_NEG | ___PPC_RT(d) | ___PPC_RA(a))
 
diff --git a/arch/powerpc/net/bpf_jit64.h b/arch/powerpc/net/bpf_jit64.h
new file mode 100644
index 0000000..5046d6f
--- /dev/null
+++ b/arch/powerpc/net/bpf_jit64.h
@@ -0,0 +1,102 @@
+/*
+ * bpf_jit64.h: BPF JIT compiler for PPC64
+ *
+ * Copyright 2016 Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
+ *		  IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2
+ * of the License.
+ */
+#ifndef _BPF_JIT64_H
+#define _BPF_JIT64_H
+
+#include "bpf_jit.h"
+
+/*
+ * Stack layout:
+ *
+ *		[	prev sp		] <-------------
+ *		[   nv gpr save area	] 8*8		|
+ * fp (r31) -->	[   ebpf stack space	] 512		|
+ *		[  local/tmp var space	] 16		|
+ *		[     frame header	] 32/112	|
+ * sp (r1) --->	[    stack pointer	] --------------
+ */
+
+/* for bpf JIT code internal usage */
+#define BPF_PPC_STACK_LOCALS	16
+/* for gpr non volatile registers BPG_REG_6 to 10, plus skb cache registers */
+#define BPF_PPC_STACK_SAVE	(8*8)
+/* Ensure this is quadword aligned */
+#define BPF_PPC_STACKFRAME	(STACK_FRAME_MIN_SIZE + BPF_PPC_STACK_LOCALS + \
+				 MAX_BPF_STACK + BPF_PPC_STACK_SAVE)
+
+#ifndef __ASSEMBLY__
+
+/* BPF register usage */
+#define SKB_HLEN_REG	(MAX_BPF_REG + 0)
+#define SKB_DATA_REG	(MAX_BPF_REG + 1)
+#define TMP_REG_1	(MAX_BPF_REG + 2)
+#define TMP_REG_2	(MAX_BPF_REG + 3)
+
+/* BPF to ppc register mappings */
+static const int b2p[] = {
+	/* function return value */
+	[BPF_REG_0] = 8,
+	/* function arguments */
+	[BPF_REG_1] = 3,
+	[BPF_REG_2] = 4,
+	[BPF_REG_3] = 5,
+	[BPF_REG_4] = 6,
+	[BPF_REG_5] = 7,
+	/* non volatile registers */
+	[BPF_REG_6] = 27,
+	[BPF_REG_7] = 28,
+	[BPF_REG_8] = 29,
+	[BPF_REG_9] = 30,
+	/* frame pointer aka BPF_REG_10 */
+	[BPF_REG_FP] = 31,
+	/* eBPF jit internal registers */
+	[SKB_HLEN_REG] = 25,
+	[SKB_DATA_REG] = 26,
+	[TMP_REG_1] = 9,
+	[TMP_REG_2] = 10
+};
+
+/* Assembly helpers */
+#define DECLARE_LOAD_FUNC(func)	u64 func(u64 r3, u64 r4);			\
+				u64 func##_negative_offset(u64 r3, u64 r4);	\
+				u64 func##_positive_offset(u64 r3, u64 r4);
+
+DECLARE_LOAD_FUNC(sk_load_word);
+DECLARE_LOAD_FUNC(sk_load_half);
+DECLARE_LOAD_FUNC(sk_load_byte);
+
+#define CHOOSE_LOAD_FUNC(imm, func)						\
+			(imm < 0 ?						\
+			(imm >= SKF_LL_OFF ? func##_negative_offset : func) :	\
+			func##_positive_offset)
+
+#define SEEN_FUNC	0x1000 /* might call external helpers */
+#define SEEN_STACK	0x2000 /* uses BPF stack */
+#define SEEN_SKB	0x4000 /* uses sk_buff */
+
+struct codegen_context {
+	/*
+	 * This is used to track register usage as well
+	 * as calls to external helpers.
+	 * - register usage is tracked with corresponding
+	 *   bits (r3-r10 and r25-r31)
+	 * - rest of the bits can be used to track other
+	 *   things -- for now, we use bits 16 to 23
+	 *   encoded in SEEN_* macros above
+	 */
+	unsigned int seen;
+	unsigned int idx;
+};
+
+#endif /* !__ASSEMBLY__ */
+
+#endif
diff --git a/arch/powerpc/net/bpf_jit_asm64.S b/arch/powerpc/net/bpf_jit_asm64.S
new file mode 100644
index 0000000..7e4c514
--- /dev/null
+++ b/arch/powerpc/net/bpf_jit_asm64.S
@@ -0,0 +1,180 @@
+/*
+ * bpf_jit_asm64.S: Packet/header access helper functions
+ * for PPC64 BPF compiler.
+ *
+ * Copyright 2016, Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
+ * 		   IBM Corporation
+ *
+ * Based on bpf_jit_asm.S by Matt Evans
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2
+ * of the License.
+ */
+
+#include <asm/ppc_asm.h>
+#include <asm/ptrace.h>
+#include "bpf_jit64.h"
+
+/*
+ * All of these routines are called directly from generated code,
+ * with the below register usage:
+ * r27		skb pointer (ctx)
+ * r25		skb header length
+ * r26		skb->data pointer
+ * r4		offset
+ *
+ * Result is passed back in:
+ * r8		data read in host endian format (accumulator)
+ *
+ * r9 is used as a temporary register
+ */
+
+#define r_skb	r27
+#define r_hlen	r25
+#define r_data	r26
+#define r_off	r4
+#define r_val	r8
+#define r_tmp	r9
+
+_GLOBAL_TOC(sk_load_word)
+	cmpdi	r_off, 0
+	blt	bpf_slow_path_word_neg
+	b	sk_load_word_positive_offset
+
+_GLOBAL_TOC(sk_load_word_positive_offset)
+	/* Are we accessing past headlen? */
+	subi	r_tmp, r_hlen, 4
+	cmpd	r_tmp, r_off
+	blt	bpf_slow_path_word
+	/* Nope, just hitting the header.  cr0 here is eq or gt! */
+	LWZX_BE	r_val, r_data, r_off
+	blr	/* Return success, cr0 != LT */
+
+_GLOBAL_TOC(sk_load_half)
+	cmpdi	r_off, 0
+	blt	bpf_slow_path_half_neg
+	b	sk_load_half_positive_offset
+
+_GLOBAL_TOC(sk_load_half_positive_offset)
+	subi	r_tmp, r_hlen, 2
+	cmpd	r_tmp, r_off
+	blt	bpf_slow_path_half
+	LHZX_BE	r_val, r_data, r_off
+	blr
+
+_GLOBAL_TOC(sk_load_byte)
+	cmpdi	r_off, 0
+	blt	bpf_slow_path_byte_neg
+	b	sk_load_byte_positive_offset
+
+_GLOBAL_TOC(sk_load_byte_positive_offset)
+	cmpd	r_hlen, r_off
+	ble	bpf_slow_path_byte
+	lbzx	r_val, r_data, r_off
+	blr
+
+/*
+ * Call out to skb_copy_bits:
+ * Allocate a new stack frame here to remain ABI-compliant in
+ * stashing LR.
+ */
+#define bpf_slow_path_common(SIZE)					\
+	mflr	r0;							\
+	std	r0, PPC_LR_STKOFF(r1);					\
+	stdu	r1, -(STACK_FRAME_MIN_SIZE + BPF_PPC_STACK_LOCALS)(r1);	\
+	mr	r3, r_skb;						\
+	/* r4 = r_off as passed */					\
+	addi	r5, r1, STACK_FRAME_MIN_SIZE;				\
+	li	r6, SIZE;						\
+	bl	skb_copy_bits;						\
+	nop;								\
+	/* save r5 */							\
+	addi	r5, r1, STACK_FRAME_MIN_SIZE;				\
+	/* r3 = 0 on success */						\
+	addi	r1, r1, STACK_FRAME_MIN_SIZE + BPF_PPC_STACK_LOCALS;	\
+	ld	r0, PPC_LR_STKOFF(r1);					\
+	mtlr	r0;							\
+	cmpdi	r3, 0;							\
+	blt	bpf_error;	/* cr0 = LT */
+
+bpf_slow_path_word:
+	bpf_slow_path_common(4)
+	/* Data value is on stack, and cr0 != LT */
+	LWZX_BE	r_val, 0, r5
+	blr
+
+bpf_slow_path_half:
+	bpf_slow_path_common(2)
+	LHZX_BE	r_val, 0, r5
+	blr
+
+bpf_slow_path_byte:
+	bpf_slow_path_common(1)
+	lbzx	r_val, 0, r5
+	blr
+
+/*
+ * Call out to bpf_internal_load_pointer_neg_helper
+ */
+#define sk_negative_common(SIZE)				\
+	mflr	r0;						\
+	std	r0, PPC_LR_STKOFF(r1);				\
+	stdu	r1, -STACK_FRAME_MIN_SIZE(r1);			\
+	mr	r3, r_skb;					\
+	/* r4 = r_off, as passed */				\
+	li	r5, SIZE;					\
+	bl	bpf_internal_load_pointer_neg_helper;		\
+	nop;							\
+	addi	r1, r1, STACK_FRAME_MIN_SIZE;			\
+	ld	r0, PPC_LR_STKOFF(r1);				\
+	mtlr	r0;						\
+	/* R3 != 0 on success */				\
+	cmpldi	r3, 0;						\
+	beq	bpf_error_slow;	/* cr0 = EQ */
+
+bpf_slow_path_word_neg:
+	lis     r_tmp, -32	/* SKF_LL_OFF */
+	cmpd	r_off, r_tmp	/* addr < SKF_* */
+	blt	bpf_error	/* cr0 = LT */
+	b	sk_load_word_negative_offset
+
+_GLOBAL_TOC(sk_load_word_negative_offset)
+	sk_negative_common(4)
+	LWZX_BE	r_val, 0, r3
+	blr
+
+bpf_slow_path_half_neg:
+	lis     r_tmp, -32	/* SKF_LL_OFF */
+	cmpd	r_off, r_tmp	/* addr < SKF_* */
+	blt	bpf_error	/* cr0 = LT */
+	b	sk_load_half_negative_offset
+
+_GLOBAL_TOC(sk_load_half_negative_offset)
+	sk_negative_common(2)
+	LHZX_BE	r_val, 0, r3
+	blr
+
+bpf_slow_path_byte_neg:
+	lis     r_tmp, -32	/* SKF_LL_OFF */
+	cmpd	r_off, r_tmp	/* addr < SKF_* */
+	blt	bpf_error	/* cr0 = LT */
+	b	sk_load_byte_negative_offset
+
+_GLOBAL_TOC(sk_load_byte_negative_offset)
+	sk_negative_common(1)
+	lbzx	r_val, 0, r3
+	blr
+
+bpf_error_slow:
+	/* fabricate a cr0 = lt */
+	li	r_tmp, -1
+	cmpdi	r_tmp, 0
+bpf_error:
+	/*
+	 * Entered with cr0 = lt
+	 * Generated code will 'blt epilogue', returning 0.
+	 */
+	li	r_val, 0
+	blr
diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
new file mode 100644
index 0000000..954ff53
--- /dev/null
+++ b/arch/powerpc/net/bpf_jit_comp64.c
@@ -0,0 +1,956 @@
+/*
+ * bpf_jit_comp64.c: eBPF JIT compiler
+ *
+ * Copyright 2016 Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
+ *		  IBM Corporation
+ *
+ * Based on the powerpc classic BPF JIT compiler by Matt Evans
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2
+ * of the License.
+ */
+#include <linux/moduleloader.h>
+#include <asm/cacheflush.h>
+#include <linux/netdevice.h>
+#include <linux/filter.h>
+#include <linux/if_vlan.h>
+
+#include "bpf_jit64.h"
+
+int bpf_jit_enable __read_mostly;
+
+static void bpf_jit_fill_ill_insns(void *area, unsigned int size)
+{
+	int *p = area;
+
+	/* Fill whole space with trap instructions */
+	while (p < (int *)((char *)area + size))
+		*p++ = BREAKPOINT_INSTRUCTION;
+}
+
+static inline void bpf_flush_icache(void *start, void *end)
+{
+	smp_wmb();
+	flush_icache_range((unsigned long)start, (unsigned long)end);
+}
+
+static inline bool bpf_is_seen_register(struct codegen_context *ctx, int i)
+{
+	return (ctx->seen & (1 << (31 - b2p[i])));
+}
+
+static inline void bpf_set_seen_register(struct codegen_context *ctx, int i)
+{
+	ctx->seen |= (1 << (31 - b2p[i]));
+}
+
+static inline bool bpf_has_stack_frame(struct codegen_context *ctx)
+{
+	/*
+	 * We only need a stack frame if:
+	 * - we call other functions (kernel helpers), or
+	 * - the bpf program uses its stack area
+	 * The latter condition is deduced from the usage of BPF_REG_FP
+	 */
+	return ctx->seen & SEEN_FUNC || bpf_is_seen_register(ctx, BPF_REG_FP);
+}
+
+static void bpf_jit_emit_skb_loads(u32 *image, struct codegen_context *ctx)
+{
+	/*
+	 * Load skb->len and skb->data_len
+	 * r3 points to skb
+	 */
+	PPC_LWZ(b2p[SKB_HLEN_REG], 3, offsetof(struct sk_buff, len));
+	PPC_LWZ(b2p[TMP_REG_1], 3, offsetof(struct sk_buff, data_len));
+	/* header_len = len - data_len */
+	PPC_SUB(b2p[SKB_HLEN_REG], b2p[SKB_HLEN_REG], b2p[TMP_REG_1]);
+
+	/* skb->data pointer */
+	PPC_BPF_LL(b2p[SKB_DATA_REG], 3, offsetof(struct sk_buff, data));
+}
+
+static void bpf_jit_emit_func_call(u32 *image, struct codegen_context *ctx, u64 func)
+{
+#ifdef PPC64_ELF_ABI_v1
+	/* func points to the function descriptor */
+	PPC_LI64(b2p[TMP_REG_2], func);
+	/* Load actual entry point from function descriptor */
+	PPC_BPF_LL(b2p[TMP_REG_1], b2p[TMP_REG_2], 0);
+	/* ... and move it to LR */
+	PPC_MTLR(b2p[TMP_REG_1]);
+	/*
+	 * Load TOC from function descriptor at offset 8.
+	 * We can clobber r2 since we get called through a
+	 * function pointer (so caller will save/restore r2)
+	 * and since we don't use a TOC ourself.
+	 */
+	PPC_BPF_LL(2, b2p[TMP_REG_2], 8);
+#else
+	/* We can clobber r12 */
+	PPC_FUNC_ADDR(12, func);
+	PPC_MTLR(12);
+#endif
+	PPC_BLRL();
+}
+
+static void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx)
+{
+	int i;
+	bool new_stack_frame = bpf_has_stack_frame(ctx);
+
+	if (new_stack_frame) {
+		/*
+		 * We need a stack frame, but we don't necessarily need to
+		 * save/restore LR unless we call other functions
+		 */
+		if (ctx->seen & SEEN_FUNC) {
+			EMIT(PPC_INST_MFLR | __PPC_RT(R0));
+			PPC_BPF_STL(0, 1, PPC_LR_STKOFF);
+		}
+
+		PPC_BPF_STLU(1, 1, -BPF_PPC_STACKFRAME);
+	}
+
+	/*
+	 * Back up non-volatile regs -- BPF registers 6-10
+	 * If we haven't created our own stack frame, we save these
+	 * in the protected zone below the previous stack frame
+	 */
+	for (i = BPF_REG_6; i <= BPF_REG_10; i++)
+		if (bpf_is_seen_register(ctx, i))
+			PPC_BPF_STL(b2p[i], 1,
+				(new_stack_frame ? BPF_PPC_STACKFRAME : 0) -
+					(8 * (32 - b2p[i])));
+
+	/*
+	 * Save additional non-volatile regs if we cache skb
+	 * Also, setup skb data
+	 */
+	if (ctx->seen & SEEN_SKB) {
+		PPC_BPF_STL(b2p[SKB_HLEN_REG], 1,
+			BPF_PPC_STACKFRAME - (8 * (32 - b2p[SKB_HLEN_REG])));
+		PPC_BPF_STL(b2p[SKB_DATA_REG], 1,
+			BPF_PPC_STACKFRAME - (8 * (32 - b2p[SKB_DATA_REG])));
+		bpf_jit_emit_skb_loads(image, ctx);
+	}
+
+	/* Setup frame pointer to point to the bpf stack area */
+	if (bpf_is_seen_register(ctx, BPF_REG_FP))
+		PPC_ADDI(b2p[BPF_REG_FP], 1,
+				BPF_PPC_STACKFRAME - BPF_PPC_STACK_SAVE);
+}
+
+static void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx)
+{
+	int i;
+	bool new_stack_frame = bpf_has_stack_frame(ctx);
+
+	/* Move result to r3 */
+	PPC_MR(3, b2p[BPF_REG_0]);
+
+	/* Restore NVRs */
+	for (i = BPF_REG_6; i <= BPF_REG_10; i++)
+		if (bpf_is_seen_register(ctx, i))
+			PPC_BPF_LL(b2p[i], 1,
+				(new_stack_frame ? BPF_PPC_STACKFRAME : 0) -
+					(8 * (32 - b2p[i])));
+
+	/* Restore non-volatile registers used for skb cache */
+	if (ctx->seen & SEEN_SKB) {
+		PPC_BPF_LL(b2p[SKB_HLEN_REG], 1,
+			BPF_PPC_STACKFRAME - (8 * (32 - b2p[SKB_HLEN_REG])));
+		PPC_BPF_LL(b2p[SKB_DATA_REG], 1,
+			BPF_PPC_STACKFRAME - (8 * (32 - b2p[SKB_DATA_REG])));
+	}
+
+	/* Tear down our stack frame */
+	if (new_stack_frame) {
+		PPC_ADDI(1, 1, BPF_PPC_STACKFRAME);
+		if (ctx->seen & SEEN_FUNC) {
+			PPC_BPF_LL(0, 1, PPC_LR_STKOFF);
+			PPC_MTLR(0);
+		}
+	}
+
+	PPC_BLR();
+}
+
+/* Assemble the body code between the prologue & epilogue */
+static int bpf_jit_build_body(struct bpf_prog *fp, u32 *image,
+			      struct codegen_context *ctx,
+			      u32 *addrs)
+{
+	const struct bpf_insn *insn = fp->insnsi;
+	int flen = fp->len;
+	int i;
+
+	/* Start of epilogue code - will only be valid 2nd pass onwards */
+	u32 exit_addr = addrs[flen];
+
+	for (i = 0; i < flen; i++) {
+		u32 code = insn[i].code;
+		u32 dst_reg = b2p[insn[i].dst_reg];
+		u32 src_reg = b2p[insn[i].src_reg];
+		s16 off = insn[i].off;
+		s32 imm = insn[i].imm;
+		u64 imm64;
+		u8 *func;
+		u32 true_cond;
+		int stack_local_off;
+
+		/*
+		 * addrs[] maps a BPF bytecode address into a real offset from
+		 * the start of the body code.
+		 */
+		addrs[i] = ctx->idx * 4;
+
+		/*
+		 * As an optimization, we note down which non-volatile registers
+		 * are used so that we can only save/restore those in our
+		 * prologue and epilogue. We do this here regardless of whether
+		 * the actual BPF instruction uses src/dst registers or not
+		 * (for instance, BPF_CALL does not use them). The expectation
+		 * is that those instructions will have src_reg/dst_reg set to
+		 * 0. Even otherwise, we just lose some prologue/epilogue
+		 * optimization but everything else should work without
+		 * any issues.
+		 */
+		if (dst_reg >= 24 && dst_reg <= 31)
+			bpf_set_seen_register(ctx, insn[i].dst_reg);
+		if (src_reg >= 24 && src_reg <= 31)
+			bpf_set_seen_register(ctx, insn[i].src_reg);
+
+		switch (code) {
+		/*
+		 * Arithmetic operations: ADD/SUB/MUL/DIV/MOD/NEG
+		 */
+		case BPF_ALU | BPF_ADD | BPF_X: /* (u32) dst += (u32) src */
+		case BPF_ALU64 | BPF_ADD | BPF_X: /* dst += src */
+			PPC_ADD(dst_reg, dst_reg, src_reg);
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_SUB | BPF_X: /* (u32) dst -= (u32) src */
+		case BPF_ALU64 | BPF_SUB | BPF_X: /* dst -= src */
+			PPC_SUB(dst_reg, dst_reg, src_reg);
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_ADD | BPF_K: /* (u32) dst += (u32) imm */
+		case BPF_ALU | BPF_SUB | BPF_K: /* (u32) dst -= (u32) imm */
+		case BPF_ALU64 | BPF_ADD | BPF_K: /* dst += imm */
+		case BPF_ALU64 | BPF_SUB | BPF_K: /* dst -= imm */
+			if (BPF_OP(code) == BPF_SUB)
+				imm = -imm;
+			if (imm) {
+				if (imm >= -32768 && imm < 32768)
+					PPC_ADDI(dst_reg, dst_reg, IMM_L(imm));
+				else {
+					PPC_LI32(b2p[TMP_REG_1], imm);
+					PPC_ADD(dst_reg, dst_reg, b2p[TMP_REG_1]);
+				}
+			}
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_MUL | BPF_X: /* (u32) dst *= (u32) src */
+		case BPF_ALU64 | BPF_MUL | BPF_X: /* dst *= src */
+			if (BPF_CLASS(code) == BPF_ALU)
+				PPC_MULW(dst_reg, dst_reg, src_reg);
+			else
+				PPC_MULD(dst_reg, dst_reg, src_reg);
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_MUL | BPF_K: /* (u32) dst *= (u32) imm */
+		case BPF_ALU64 | BPF_MUL | BPF_K: /* dst *= imm */
+			if (imm >= -32768 && imm < 32768)
+				PPC_MULI(dst_reg, dst_reg, IMM_L(imm));
+			else {
+				PPC_LI32(b2p[TMP_REG_1], imm);
+				if (BPF_CLASS(code) == BPF_ALU)
+					PPC_MULW(dst_reg, dst_reg,
+							b2p[TMP_REG_1]);
+				else
+					PPC_MULD(dst_reg, dst_reg,
+							b2p[TMP_REG_1]);
+			}
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_DIV | BPF_X: /* (u32) dst /= (u32) src */
+		case BPF_ALU | BPF_MOD | BPF_X: /* (u32) dst %= (u32) src */
+			PPC_CMPWI(src_reg, 0);
+			PPC_BCC_SHORT(COND_NE, (ctx->idx * 4) + 12);
+			PPC_LI(b2p[BPF_REG_0], 0);
+			PPC_JMP(exit_addr);
+			if (BPF_OP(code) == BPF_MOD) {
+				PPC_DIVWU(b2p[TMP_REG_1], dst_reg, src_reg);
+				PPC_MULW(b2p[TMP_REG_1], src_reg,
+						b2p[TMP_REG_1]);
+				PPC_SUB(dst_reg, dst_reg, b2p[TMP_REG_1]);
+			} else
+				PPC_DIVWU(dst_reg, dst_reg, src_reg);
+			goto bpf_alu32_trunc;
+		case BPF_ALU64 | BPF_DIV | BPF_X: /* dst /= src */
+		case BPF_ALU64 | BPF_MOD | BPF_X: /* dst %= src */
+			PPC_CMPDI(src_reg, 0);
+			PPC_BCC_SHORT(COND_NE, (ctx->idx * 4) + 12);
+			PPC_LI(b2p[BPF_REG_0], 0);
+			PPC_JMP(exit_addr);
+			if (BPF_OP(code) == BPF_MOD) {
+				PPC_DIVD(b2p[TMP_REG_1], dst_reg, src_reg);
+				PPC_MULD(b2p[TMP_REG_1], src_reg,
+						b2p[TMP_REG_1]);
+				PPC_SUB(dst_reg, dst_reg, b2p[TMP_REG_1]);
+			} else
+				PPC_DIVD(dst_reg, dst_reg, src_reg);
+			break;
+		case BPF_ALU | BPF_MOD | BPF_K: /* (u32) dst %= (u32) imm */
+		case BPF_ALU | BPF_DIV | BPF_K: /* (u32) dst /= (u32) imm */
+		case BPF_ALU64 | BPF_MOD | BPF_K: /* dst %= imm */
+		case BPF_ALU64 | BPF_DIV | BPF_K: /* dst /= imm */
+			if (imm == 0)
+				return -EINVAL;
+			else if (imm == 1)
+				goto bpf_alu32_trunc;
+
+			PPC_LI32(b2p[TMP_REG_1], imm);
+			switch (BPF_CLASS(code)) {
+			case BPF_ALU:
+				if (BPF_OP(code) == BPF_MOD) {
+					PPC_DIVWU(b2p[TMP_REG_2], dst_reg,
+							b2p[TMP_REG_1]);
+					PPC_MULW(b2p[TMP_REG_1],
+							b2p[TMP_REG_1],
+							b2p[TMP_REG_2]);
+					PPC_SUB(dst_reg, dst_reg,
+							b2p[TMP_REG_1]);
+				} else
+					PPC_DIVWU(dst_reg, dst_reg,
+							b2p[TMP_REG_1]);
+				break;
+			case BPF_ALU64:
+				if (BPF_OP(code) == BPF_MOD) {
+					PPC_DIVD(b2p[TMP_REG_2], dst_reg,
+							b2p[TMP_REG_1]);
+					PPC_MULD(b2p[TMP_REG_1],
+							b2p[TMP_REG_1],
+							b2p[TMP_REG_2]);
+					PPC_SUB(dst_reg, dst_reg,
+							b2p[TMP_REG_1]);
+				} else
+					PPC_DIVD(dst_reg, dst_reg,
+							b2p[TMP_REG_1]);
+				break;
+			}
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_NEG: /* (u32) dst = -dst */
+		case BPF_ALU64 | BPF_NEG: /* dst = -dst */
+			PPC_NEG(dst_reg, dst_reg);
+			goto bpf_alu32_trunc;
+
+		/*
+		 * Logical operations: AND/OR/XOR/[A]LSH/[A]RSH
+		 */
+		case BPF_ALU | BPF_AND | BPF_X: /* (u32) dst = dst & src */
+		case BPF_ALU64 | BPF_AND | BPF_X: /* dst = dst & src */
+			PPC_AND(dst_reg, dst_reg, src_reg);
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_AND | BPF_K: /* (u32) dst = dst & imm */
+		case BPF_ALU64 | BPF_AND | BPF_K: /* dst = dst & imm */
+			if (!IMM_H(imm))
+				PPC_ANDI(dst_reg, dst_reg, IMM_L(imm));
+			else {
+				/* Sign-extended */
+				PPC_LI32(b2p[TMP_REG_1], imm);
+				PPC_AND(dst_reg, dst_reg, b2p[TMP_REG_1]);
+			}
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_OR | BPF_X: /* dst = (u32) dst | (u32) src */
+		case BPF_ALU64 | BPF_OR | BPF_X: /* dst = dst | src */
+			PPC_OR(dst_reg, dst_reg, src_reg);
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_OR | BPF_K:/* dst = (u32) dst | (u32) imm */
+		case BPF_ALU64 | BPF_OR | BPF_K:/* dst = dst | imm */
+			if (imm < 0 && BPF_CLASS(code) == BPF_ALU64) {
+				/* Sign-extended */
+				PPC_LI32(b2p[TMP_REG_1], imm);
+				PPC_OR(dst_reg, dst_reg, b2p[TMP_REG_1]);
+			} else {
+				if (IMM_L(imm))
+					PPC_ORI(dst_reg, dst_reg, IMM_L(imm));
+				if (IMM_H(imm))
+					PPC_ORIS(dst_reg, dst_reg, IMM_H(imm));
+			}
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_XOR | BPF_X: /* (u32) dst ^= src */
+		case BPF_ALU64 | BPF_XOR | BPF_X: /* dst ^= src */
+			PPC_XOR(dst_reg, dst_reg, src_reg);
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_XOR | BPF_K: /* (u32) dst ^= (u32) imm */
+		case BPF_ALU64 | BPF_XOR | BPF_K: /* dst ^= imm */
+			if (imm < 0 && BPF_CLASS(code) == BPF_ALU64) {
+				/* Sign-extended */
+				PPC_LI32(b2p[TMP_REG_1], imm);
+				PPC_XOR(dst_reg, dst_reg, b2p[TMP_REG_1]);
+			} else {
+				if (IMM_L(imm))
+					PPC_XORI(dst_reg, dst_reg, IMM_L(imm));
+				if (IMM_H(imm))
+					PPC_XORIS(dst_reg, dst_reg, IMM_H(imm));
+			}
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_LSH | BPF_X: /* (u32) dst <<= (u32) src */
+			/* slw clears top 32 bits */
+			PPC_SLW(dst_reg, dst_reg, src_reg);
+			break;
+		case BPF_ALU64 | BPF_LSH | BPF_X: /* dst <<= src; */
+			PPC_SLD(dst_reg, dst_reg, src_reg);
+			break;
+		case BPF_ALU | BPF_LSH | BPF_K: /* (u32) dst <<== (u32) imm */
+			/* with imm 0, we still need to clear top 32 bits */
+			PPC_SLWI(dst_reg, dst_reg, imm);
+			break;
+		case BPF_ALU64 | BPF_LSH | BPF_K: /* dst <<== imm */
+			if (imm != 0)
+				PPC_SLDI(dst_reg, dst_reg, imm);
+			break;
+		case BPF_ALU | BPF_RSH | BPF_X: /* (u32) dst >>= (u32) src */
+			PPC_SRW(dst_reg, dst_reg, src_reg);
+			break;
+		case BPF_ALU64 | BPF_RSH | BPF_X: /* dst >>= src */
+			PPC_SRD(dst_reg, dst_reg, src_reg);
+			break;
+		case BPF_ALU | BPF_RSH | BPF_K: /* (u32) dst >>= (u32) imm */
+			PPC_SRWI(dst_reg, dst_reg, imm);
+			break;
+		case BPF_ALU64 | BPF_RSH | BPF_K: /* dst >>= imm */
+			if (imm != 0)
+				PPC_SRDI(dst_reg, dst_reg, imm);
+			break;
+		case BPF_ALU64 | BPF_ARSH | BPF_X: /* (s64) dst >>= src */
+			PPC_SRAD(dst_reg, dst_reg, src_reg);
+			break;
+		case BPF_ALU64 | BPF_ARSH | BPF_K: /* (s64) dst >>= imm */
+			if (imm != 0)
+				PPC_SRADI(dst_reg, dst_reg, imm);
+			break;
+
+		/*
+		 * MOV
+		 */
+		case BPF_ALU | BPF_MOV | BPF_X: /* (u32) dst = src */
+		case BPF_ALU64 | BPF_MOV | BPF_X: /* dst = src */
+			PPC_MR(dst_reg, src_reg);
+			goto bpf_alu32_trunc;
+		case BPF_ALU | BPF_MOV | BPF_K: /* (u32) dst = imm */
+		case BPF_ALU64 | BPF_MOV | BPF_K: /* dst = (s64) imm */
+			PPC_LI32(dst_reg, imm);
+			if (imm < 0)
+				goto bpf_alu32_trunc;
+			break;
+
+bpf_alu32_trunc:
+		/* Truncate to 32-bits */
+		if (BPF_CLASS(code) == BPF_ALU)
+			PPC_RLWINM(dst_reg, dst_reg, 0, 0, 31);
+		break;
+
+		/*
+		 * BPF_FROM_BE/LE
+		 */
+		case BPF_ALU | BPF_END | BPF_FROM_LE:
+		case BPF_ALU | BPF_END | BPF_FROM_BE:
+#ifdef __BIG_ENDIAN__
+			if (BPF_SRC(code) == BPF_FROM_BE)
+				goto emit_clear;
+#else /* !__BIG_ENDIAN__ */
+			if (BPF_SRC(code) == BPF_FROM_LE)
+				goto emit_clear;
+#endif
+			switch (imm) {
+			case 16:
+				/* Rotate 8 bits left & mask with 0x0000ff00 */
+				PPC_RLWINM(b2p[TMP_REG_1], dst_reg, 8, 16, 23);
+				/* Rotate 8 bits right & insert LSB to reg */
+				PPC_RLWIMI(b2p[TMP_REG_1], dst_reg, 24, 24, 31);
+				/* Move result back to dst_reg */
+				PPC_MR(dst_reg, b2p[TMP_REG_1]);
+				break;
+			case 32:
+				/*
+				 * Rotate word left by 8 bits:
+				 * 2 bytes are already in their final position
+				 * -- byte 2 and 4 (of bytes 1, 2, 3 and 4)
+				 */
+				PPC_RLWINM(b2p[TMP_REG_1], dst_reg, 8, 0, 31);
+				/* Rotate 24 bits and insert byte 1 */
+				PPC_RLWIMI(b2p[TMP_REG_1], dst_reg, 24, 0, 7);
+				/* Rotate 24 bits and insert byte 3 */
+				PPC_RLWIMI(b2p[TMP_REG_1], dst_reg, 24, 16, 23);
+				PPC_MR(dst_reg, b2p[TMP_REG_1]);
+				break;
+			case 64:
+				/*
+				 * Way easier and faster(?) to store the value
+				 * into stack and then use ldbrx
+				 *
+				 * First, determine where in stack we can store
+				 * this:
+				 * - if we have allotted a stack frame, then we
+				 *   will utilize the area set aside by
+				 *   BPF_PPC_STACK_LOCALS
+				 * - else, we use the area beneath the NV GPR
+				 *   save area
+				 *
+				 * ctx->seen will be reliable in pass2, but
+				 * the instructions generated will remain the
+				 * same across all passes
+				 */
+				if (bpf_has_stack_frame(ctx))
+					stack_local_off = STACK_FRAME_MIN_SIZE;
+				else
+					stack_local_off = -(BPF_PPC_STACK_SAVE + 8);
+
+				PPC_STD(dst_reg, 1, stack_local_off);
+				PPC_ADDI(b2p[TMP_REG_1], 1, stack_local_off);
+				PPC_LDBRX(dst_reg, 0, b2p[TMP_REG_1]);
+				break;
+			}
+			break;
+
+emit_clear:
+			switch (imm) {
+			case 16:
+				/* zero-extend 16 bits into 64 bits */
+				PPC_RLDICL(dst_reg, dst_reg, 0, 48);
+				break;
+			case 32:
+				/* zero-extend 32 bits into 64 bits */
+				PPC_RLDICL(dst_reg, dst_reg, 0, 32);
+				break;
+			case 64:
+				/* nop */
+				break;
+			}
+			break;
+
+		/*
+		 * BPF_ST(X)
+		 */
+		case BPF_STX | BPF_MEM | BPF_B: /* *(u8 *)(dst + off) = src */
+		case BPF_ST | BPF_MEM | BPF_B: /* *(u8 *)(dst + off) = imm */
+			if (BPF_CLASS(code) == BPF_ST) {
+				PPC_LI(b2p[TMP_REG_1], imm);
+				src_reg = b2p[TMP_REG_1];
+			}
+			PPC_STB(src_reg, dst_reg, off);
+			break;
+		case BPF_STX | BPF_MEM | BPF_H: /* (u16 *)(dst + off) = src */
+		case BPF_ST | BPF_MEM | BPF_H: /* (u16 *)(dst + off) = imm */
+			if (BPF_CLASS(code) == BPF_ST) {
+				PPC_LI(b2p[TMP_REG_1], imm);
+				src_reg = b2p[TMP_REG_1];
+			}
+			PPC_STH(src_reg, dst_reg, off);
+			break;
+		case BPF_STX | BPF_MEM | BPF_W: /* *(u32 *)(dst + off) = src */
+		case BPF_ST | BPF_MEM | BPF_W: /* *(u32 *)(dst + off) = imm */
+			if (BPF_CLASS(code) == BPF_ST) {
+				PPC_LI32(b2p[TMP_REG_1], imm);
+				src_reg = b2p[TMP_REG_1];
+			}
+			PPC_STW(src_reg, dst_reg, off);
+			break;
+		case BPF_STX | BPF_MEM | BPF_DW: /* (u64 *)(dst + off) = src */
+		case BPF_ST | BPF_MEM | BPF_DW: /* *(u64 *)(dst + off) = imm */
+			if (BPF_CLASS(code) == BPF_ST) {
+				PPC_LI32(b2p[TMP_REG_1], imm);
+				src_reg = b2p[TMP_REG_1];
+			}
+			PPC_STD(src_reg, dst_reg, off);
+			break;
+
+		/*
+		 * BPF_STX XADD (atomic_add)
+		 */
+		/* *(u32 *)(dst + off) += src */
+		case BPF_STX | BPF_XADD | BPF_W:
+			/* Get EA into TMP_REG_1 */
+			PPC_ADDI(b2p[TMP_REG_1], dst_reg, off);
+			/* error if EA is not word-aligned */
+			PPC_ANDI(b2p[TMP_REG_2], b2p[TMP_REG_1], 0x03);
+			PPC_BCC_SHORT(COND_EQ, (ctx->idx * 4) + 12);
+			PPC_LI(b2p[BPF_REG_0], 0);
+			PPC_JMP(exit_addr);
+			/* load value from memory into TMP_REG_2 */
+			PPC_BPF_LWARX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1], 0);
+			/* add value from src_reg into this */
+			PPC_ADD(b2p[TMP_REG_2], b2p[TMP_REG_2], src_reg);
+			/* store result back */
+			PPC_BPF_STWCX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1]);
+			/* we're done if this succeeded */
+			PPC_BCC_SHORT(COND_EQ, (ctx->idx * 4) + (7*4));
+			/* otherwise, let's try once more */
+			PPC_BPF_LWARX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1], 0);
+			PPC_ADD(b2p[TMP_REG_2], b2p[TMP_REG_2], src_reg);
+			PPC_BPF_STWCX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1]);
+			/* exit if the store was not successful */
+			PPC_LI(b2p[BPF_REG_0], 0);
+			PPC_BCC(COND_NE, exit_addr);
+			break;
+		/* *(u64 *)(dst + off) += src */
+		case BPF_STX | BPF_XADD | BPF_DW:
+			PPC_ADDI(b2p[TMP_REG_1], dst_reg, off);
+			/* error if EA is not doubleword-aligned */
+			PPC_ANDI(b2p[TMP_REG_2], b2p[TMP_REG_1], 0x07);
+			PPC_BCC_SHORT(COND_EQ, (ctx->idx * 4) + (3*4));
+			PPC_LI(b2p[BPF_REG_0], 0);
+			PPC_JMP(exit_addr);
+			PPC_BPF_LDARX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1], 0);
+			PPC_ADD(b2p[TMP_REG_2], b2p[TMP_REG_2], src_reg);
+			PPC_BPF_STDCX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1]);
+			PPC_BCC_SHORT(COND_EQ, (ctx->idx * 4) + (7*4));
+			PPC_BPF_LDARX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1], 0);
+			PPC_ADD(b2p[TMP_REG_2], b2p[TMP_REG_2], src_reg);
+			PPC_BPF_STDCX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1]);
+			PPC_LI(b2p[BPF_REG_0], 0);
+			PPC_BCC(COND_NE, exit_addr);
+			break;
+
+		/*
+		 * BPF_LDX
+		 */
+		/* dst = *(u8 *)(ul) (src + off) */
+		case BPF_LDX | BPF_MEM | BPF_B:
+			PPC_LBZ(dst_reg, src_reg, off);
+			break;
+		/* dst = *(u16 *)(ul) (src + off) */
+		case BPF_LDX | BPF_MEM | BPF_H:
+			PPC_LHZ(dst_reg, src_reg, off);
+			break;
+		/* dst = *(u32 *)(ul) (src + off) */
+		case BPF_LDX | BPF_MEM | BPF_W:
+			PPC_LWZ(dst_reg, src_reg, off);
+			break;
+		/* dst = *(u64 *)(ul) (src + off) */
+		case BPF_LDX | BPF_MEM | BPF_DW:
+			PPC_LD(dst_reg, src_reg, off);
+			break;
+
+		/*
+		 * Doubleword load
+		 * 16 byte instruction that uses two 'struct bpf_insn'
+		 */
+		case BPF_LD | BPF_IMM | BPF_DW: /* dst = (u64) imm */
+			imm64 = ((u64)(u32) insn[i].imm) |
+				    (((u64)(u32) insn[i+1].imm) << 32);
+			/* Adjust for two bpf instructions */
+			addrs[++i] = ctx->idx * 4;
+			PPC_LI64(dst_reg, imm64);
+			break;
+
+		/*
+		 * Return/Exit
+		 */
+		case BPF_JMP | BPF_EXIT:
+			/*
+			 * If this isn't the very last instruction, branch to
+			 * the epilogue. If we _are_ the last instruction,
+			 * we'll just fall through to the epilogue.
+			 */
+			if (i != flen - 1)
+				PPC_JMP(exit_addr);
+			/* else fall through to the epilogue */
+			break;
+
+		/*
+		 * Call kernel helper
+		 */
+		case BPF_JMP | BPF_CALL:
+			ctx->seen |= SEEN_FUNC;
+			func = (u8 *) __bpf_call_base + imm;
+
+			/* Save skb pointer if we need to re-cache skb data */
+			if (bpf_helper_changes_skb_data(func))
+				PPC_BPF_STL(3, 1, STACK_FRAME_MIN_SIZE);
+
+			bpf_jit_emit_func_call(image, ctx, (u64)func);
+
+			/* move return value from r3 to BPF_REG_0 */
+			PPC_MR(b2p[BPF_REG_0], 3);
+
+			/* refresh skb cache */
+			if (bpf_helper_changes_skb_data(func)) {
+				/* reload skb pointer to r3 */
+				PPC_BPF_LL(3, 1, STACK_FRAME_MIN_SIZE);
+				bpf_jit_emit_skb_loads(image, ctx);
+			}
+			break;
+
+		/*
+		 * Jumps and branches
+		 */
+		case BPF_JMP | BPF_JA:
+			PPC_JMP(addrs[i + 1 + off]);
+			break;
+
+		case BPF_JMP | BPF_JGT | BPF_K:
+		case BPF_JMP | BPF_JGT | BPF_X:
+		case BPF_JMP | BPF_JSGT | BPF_K:
+		case BPF_JMP | BPF_JSGT | BPF_X:
+			true_cond = COND_GT;
+			goto cond_branch;
+		case BPF_JMP | BPF_JGE | BPF_K:
+		case BPF_JMP | BPF_JGE | BPF_X:
+		case BPF_JMP | BPF_JSGE | BPF_K:
+		case BPF_JMP | BPF_JSGE | BPF_X:
+			true_cond = COND_GE;
+			goto cond_branch;
+		case BPF_JMP | BPF_JEQ | BPF_K:
+		case BPF_JMP | BPF_JEQ | BPF_X:
+			true_cond = COND_EQ;
+			goto cond_branch;
+		case BPF_JMP | BPF_JNE | BPF_K:
+		case BPF_JMP | BPF_JNE | BPF_X:
+			true_cond = COND_NE;
+			goto cond_branch;
+		case BPF_JMP | BPF_JSET | BPF_K:
+		case BPF_JMP | BPF_JSET | BPF_X:
+			true_cond = COND_NE;
+			/* Fall through */
+
+cond_branch:
+			switch (code) {
+			case BPF_JMP | BPF_JGT | BPF_X:
+			case BPF_JMP | BPF_JGE | BPF_X:
+			case BPF_JMP | BPF_JEQ | BPF_X:
+			case BPF_JMP | BPF_JNE | BPF_X:
+				/* unsigned comparison */
+				PPC_CMPLD(dst_reg, src_reg);
+				break;
+			case BPF_JMP | BPF_JSGT | BPF_X:
+			case BPF_JMP | BPF_JSGE | BPF_X:
+				/* signed comparison */
+				PPC_CMPD(dst_reg, src_reg);
+				break;
+			case BPF_JMP | BPF_JSET | BPF_X:
+				PPC_AND_DOT(b2p[TMP_REG_1], dst_reg, src_reg);
+				break;
+			case BPF_JMP | BPF_JNE | BPF_K:
+			case BPF_JMP | BPF_JEQ | BPF_K:
+			case BPF_JMP | BPF_JGT | BPF_K:
+			case BPF_JMP | BPF_JGE | BPF_K:
+				/*
+				 * Need sign-extended load, so only positive
+				 * values can be used as imm in cmpldi
+				 */
+				if (imm >= 0 && imm < 32768)
+					PPC_CMPLDI(dst_reg, imm);
+				else {
+					/* sign-extending load */
+					PPC_LI32(b2p[TMP_REG_1], imm);
+					/* ... but unsigned comparison */
+					PPC_CMPLD(dst_reg, b2p[TMP_REG_1]);
+				}
+				break;
+			case BPF_JMP | BPF_JSGT | BPF_K:
+			case BPF_JMP | BPF_JSGE | BPF_K:
+				/*
+				 * signed comparison, so any 16-bit value
+				 * can be used in cmpdi
+				 */
+				if (imm >= -32768 && imm < 32768)
+					PPC_CMPDI(dst_reg, imm);
+				else {
+					PPC_LI32(b2p[TMP_REG_1], imm);
+					PPC_CMPD(dst_reg, b2p[TMP_REG_1]);
+				}
+				break;
+			case BPF_JMP | BPF_JSET | BPF_K:
+				/* andi does not sign-extend the immediate */
+				if (imm >= 0 && imm < 32768)
+					/* PPC_ANDI is _only/always_ dot-form */
+					PPC_ANDI(b2p[TMP_REG_1], dst_reg, imm);
+				else {
+					PPC_LI32(b2p[TMP_REG_1], imm);
+					PPC_AND_DOT(b2p[TMP_REG_1], dst_reg,
+						    b2p[TMP_REG_1]);
+				}
+				break;
+			}
+			PPC_BCC(true_cond, addrs[i + 1 + off]);
+			break;
+
+		/*
+		 * Loads from packet header/data
+		 * Assume 32-bit input value in imm and X (src_reg)
+		 */
+
+		/* Absolute loads */
+		case BPF_LD | BPF_W | BPF_ABS:
+			func = (u8 *)CHOOSE_LOAD_FUNC(imm, sk_load_word);
+			goto common_load_abs;
+		case BPF_LD | BPF_H | BPF_ABS:
+			func = (u8 *)CHOOSE_LOAD_FUNC(imm, sk_load_half);
+			goto common_load_abs;
+		case BPF_LD | BPF_B | BPF_ABS:
+			func = (u8 *)CHOOSE_LOAD_FUNC(imm, sk_load_byte);
+common_load_abs:
+			/*
+			 * Load from [imm]
+			 * Load into r4, which can just be passed onto
+			 *  skb load helpers as the second parameter
+			 */
+			PPC_LI32(4, imm);
+			goto common_load;
+
+		/* Indirect loads */
+		case BPF_LD | BPF_W | BPF_IND:
+			func = (u8 *)sk_load_word;
+			goto common_load_ind;
+		case BPF_LD | BPF_H | BPF_IND:
+			func = (u8 *)sk_load_half;
+			goto common_load_ind;
+		case BPF_LD | BPF_B | BPF_IND:
+			func = (u8 *)sk_load_byte;
+common_load_ind:
+			/*
+			 * Load from [src_reg + imm]
+			 * Treat src_reg as a 32-bit value
+			 */
+			PPC_EXTSW(4, src_reg);
+			if (imm) {
+				if (imm >= -32768 && imm < 32768)
+					PPC_ADDI(4, 4, IMM_L(imm));
+				else {
+					PPC_LI32(b2p[TMP_REG_1], imm);
+					PPC_ADD(4, 4, b2p[TMP_REG_1]);
+				}
+			}
+
+common_load:
+			ctx->seen |= SEEN_SKB;
+			ctx->seen |= SEEN_FUNC;
+			bpf_jit_emit_func_call(image, ctx, (u64)func);
+
+			/*
+			 * Helper returns 'lt' condition on error, and an
+			 * appropriate return value in BPF_REG_0
+			 */
+			PPC_BCC(COND_LT, exit_addr);
+			break;
+
+		/*
+		 * TODO: Tail call
+		 */
+		case BPF_JMP | BPF_CALL | BPF_X:
+
+		default:
+			/*
+			 * The filter contains something cruel & unusual.
+			 * We don't handle it, but also there shouldn't be
+			 * anything missing from our list.
+			 */
+			pr_err_ratelimited("eBPF filter opcode %04x (@%d) unsupported\n",
+					code, i);
+			return -ENOTSUPP;
+		}
+	}
+
+	/* Set end-of-body-code address for exit. */
+	addrs[i] = ctx->idx * 4;
+
+	return 0;
+}
+
+void bpf_jit_compile(struct bpf_prog *fp) { }
+
+struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
+{
+	u32 proglen;
+	u32 alloclen;
+	u32 *image = NULL;
+	u32 *code_base;
+	u32 *addrs;
+	struct codegen_context cgctx;
+	int pass;
+	int flen;
+	struct bpf_binary_header *bpf_hdr;
+
+	if (!bpf_jit_enable)
+		return fp;
+
+	if (!fp || !fp->len)
+		return fp;
+
+	flen = fp->len;
+	addrs = kzalloc((flen+1) * sizeof(*addrs), GFP_KERNEL);
+	if (addrs == NULL)
+		return fp;
+
+	cgctx.idx = 0;
+	cgctx.seen = 0;
+	/* Scouting faux-generate pass 0 */
+	if (bpf_jit_build_body(fp, 0, &cgctx, addrs))
+		/* We hit something illegal or unsupported. */
+		goto out;
+
+	/*
+	 * Pretend to build prologue, given the features we've seen.  This will
+	 * update ctgtx.idx as it pretends to output instructions, then we can
+	 * calculate total size from idx.
+	 */
+	bpf_jit_build_prologue(0, &cgctx);
+	bpf_jit_build_epilogue(0, &cgctx);
+
+	proglen = cgctx.idx * 4;
+	alloclen = proglen + FUNCTION_DESCR_SIZE;
+
+	bpf_hdr = bpf_jit_binary_alloc(alloclen, (u8 **)&image, 4,
+			bpf_jit_fill_ill_insns);
+	if (!bpf_hdr)
+		goto out;
+
+	code_base = image + (FUNCTION_DESCR_SIZE/4);
+
+	/* Code generation passes 1-2 */
+	for (pass = 1; pass < 3; pass++) {
+		/* Now build the prologue, body code & epilogue for real. */
+		cgctx.idx = 0;
+		bpf_jit_build_prologue(code_base, &cgctx);
+		bpf_jit_build_body(fp, code_base, &cgctx, addrs);
+		bpf_jit_build_epilogue(code_base, &cgctx);
+
+		if (bpf_jit_enable > 1)
+			pr_info("Pass %d: shrink = %d, seen = 0x%x\n", pass,
+				proglen - (cgctx.idx * 4), cgctx.seen);
+	}
+
+	if (bpf_jit_enable > 1)
+		/*
+		 * Note that we output the base address of the code_base
+		 * rather than image, since opcodes are in code_base.
+		 */
+		bpf_jit_dump(flen, proglen, pass, code_base);
+
+	if (image) {
+		bpf_flush_icache(bpf_hdr, image + alloclen);
+#ifdef PPC64_ELF_ABI_v1
+		/* Function descriptor nastiness: Address + TOC */
+		((u64 *)image)[0] = (u64)code_base;
+		((u64 *)image)[1] = local_paca->kernel_toc;
+#endif
+		fp->bpf_func = (void *)image;
+		fp->jited = 1;
+	}
+
+out:
+	kfree(addrs);
+	return fp;
+}
+
+void bpf_jit_free(struct bpf_prog *fp)
+{
+	unsigned long addr = (unsigned long)fp->bpf_func & PAGE_MASK;
+	struct bpf_binary_header *bpf_hdr = (void *)addr;
+
+	if (fp->jited)
+		bpf_jit_binary_free(bpf_hdr);
+
+	bpf_prog_unlock_free(fp);
+}
-- 
2.8.2

^ permalink raw reply related	[flat|nested] 34+ messages in thread

* Re: [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-07 13:32 ` [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF Naveen N. Rao
@ 2016-06-07 22:56   ` Alexei Starovoitov
  2016-06-08 17:17     ` Naveen N. Rao
  2016-06-09  3:19   ` Nilay Vaish
                     ` (2 subsequent siblings)
  3 siblings, 1 reply; 34+ messages in thread
From: Alexei Starovoitov @ 2016-06-07 22:56 UTC (permalink / raw)
  To: Naveen N. Rao
  Cc: linux-kernel, linuxppc-dev, netdev, mpe, Matt Evans,
	Denis Kirjanov, Paul Mackerras, Alexei Starovoitov,
	Daniel Borkmann, David S. Miller, Ananth N Mavinakayanahalli

On Tue, Jun 07, 2016 at 07:02:23PM +0530, Naveen N. Rao wrote:
> PPC64 eBPF JIT compiler.
> 
> Enable with:
> echo 1 > /proc/sys/net/core/bpf_jit_enable
> or
> echo 2 > /proc/sys/net/core/bpf_jit_enable
> 
> ... to see the generated JIT code. This can further be processed with
> tools/net/bpf_jit_disasm.
> 
> With CONFIG_TEST_BPF=m and 'modprobe test_bpf':
> test_bpf: Summary: 305 PASSED, 0 FAILED, [297/297 JIT'ed]
> 
> ... on both ppc64 BE and LE.

Nice. That's even better than on x64 which cannot jit one test:
test_bpf: #262 BPF_MAXINSNS: Jump, gap, jump, ... jited:0 168 PASS
which was designed specifically to hit x64 jit pass limit.
ppc jit has predicatble number of passes and doesn't have this problem
as expected. Great.

> The details of the approach are documented through various comments in
> the code.
> 
> Cc: Matt Evans <matt@ozlabs.org>
> Cc: Denis Kirjanov <kda@linux-powerpc.org>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Alexei Starovoitov <ast@fb.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
> Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
> ---
>  arch/powerpc/Kconfig                  |   3 +-
>  arch/powerpc/include/asm/asm-compat.h |   2 +
>  arch/powerpc/include/asm/ppc-opcode.h |  20 +-
>  arch/powerpc/net/Makefile             |   4 +
>  arch/powerpc/net/bpf_jit.h            |  53 +-
>  arch/powerpc/net/bpf_jit64.h          | 102 ++++
>  arch/powerpc/net/bpf_jit_asm64.S      | 180 +++++++
>  arch/powerpc/net/bpf_jit_comp64.c     | 956 ++++++++++++++++++++++++++++++++++
>  8 files changed, 1317 insertions(+), 3 deletions(-)
>  create mode 100644 arch/powerpc/net/bpf_jit64.h
>  create mode 100644 arch/powerpc/net/bpf_jit_asm64.S
>  create mode 100644 arch/powerpc/net/bpf_jit_comp64.c

don't see any issues with the code.
Thank you for working on this.

Acked-by: Alexei Starovoitov <ast@kernel.org>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-07 22:56   ` Alexei Starovoitov
@ 2016-06-08 17:17     ` Naveen N. Rao
  0 siblings, 0 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-08 17:17 UTC (permalink / raw)
  To: Alexei Starovoitov
  Cc: Daniel Borkmann, Matt Evans, Alexei Starovoitov, netdev,
	Denis Kirjanov, linux-kernel, Paul Mackerras, linuxppc-dev,
	David S. Miller, Ananth N Mavinakayanahalli

On 2016/06/07 03:56PM, Alexei Starovoitov wrote:
> On Tue, Jun 07, 2016 at 07:02:23PM +0530, Naveen N. Rao wrote:
> > PPC64 eBPF JIT compiler.
> > 
> > Enable with:
> > echo 1 > /proc/sys/net/core/bpf_jit_enable
> > or
> > echo 2 > /proc/sys/net/core/bpf_jit_enable
> > 
> > ... to see the generated JIT code. This can further be processed with
> > tools/net/bpf_jit_disasm.
> > 
> > With CONFIG_TEST_BPF=m and 'modprobe test_bpf':
> > test_bpf: Summary: 305 PASSED, 0 FAILED, [297/297 JIT'ed]
> > 
> > ... on both ppc64 BE and LE.
> 
> Nice. That's even better than on x64 which cannot jit one test:
> test_bpf: #262 BPF_MAXINSNS: Jump, gap, jump, ... jited:0 168 PASS
> which was designed specifically to hit x64 jit pass limit.
> ppc jit has predicatble number of passes and doesn't have this problem
> as expected. Great.

Yes, that's thanks to the clever handling of conditional branches by 
Matt -- we always emit 2 instructions for this reason (encoded in 
PPC_BCC() macro).

> 
> > The details of the approach are documented through various comments in
> > the code.
> > 
> > Cc: Matt Evans <matt@ozlabs.org>
> > Cc: Denis Kirjanov <kda@linux-powerpc.org>
> > Cc: Michael Ellerman <mpe@ellerman.id.au>
> > Cc: Paul Mackerras <paulus@samba.org>
> > Cc: Alexei Starovoitov <ast@fb.com>
> > Cc: Daniel Borkmann <daniel@iogearbox.net>
> > Cc: "David S. Miller" <davem@davemloft.net>
> > Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
> > Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
> > ---
> >  arch/powerpc/Kconfig                  |   3 +-
> >  arch/powerpc/include/asm/asm-compat.h |   2 +
> >  arch/powerpc/include/asm/ppc-opcode.h |  20 +-
> >  arch/powerpc/net/Makefile             |   4 +
> >  arch/powerpc/net/bpf_jit.h            |  53 +-
> >  arch/powerpc/net/bpf_jit64.h          | 102 ++++
> >  arch/powerpc/net/bpf_jit_asm64.S      | 180 +++++++
> >  arch/powerpc/net/bpf_jit_comp64.c     | 956 ++++++++++++++++++++++++++++++++++
> >  8 files changed, 1317 insertions(+), 3 deletions(-)
> >  create mode 100644 arch/powerpc/net/bpf_jit64.h
> >  create mode 100644 arch/powerpc/net/bpf_jit_asm64.S
> >  create mode 100644 arch/powerpc/net/bpf_jit_comp64.c
> 
> don't see any issues with the code.
> Thank you for working on this.
> 
> Acked-by: Alexei Starovoitov <ast@kernel.org>

Thanks, Alexei!


Regards,
Naveen

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-07 13:32 ` [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF Naveen N. Rao
  2016-06-07 22:56   ` Alexei Starovoitov
@ 2016-06-09  3:19   ` Nilay Vaish
  2016-06-09  6:07     ` Naveen N. Rao
  2016-06-17 12:53   ` [6/6] " Michael Ellerman
  2016-06-22 10:37   ` [PATCH 6/6] " Michael Ellerman
  3 siblings, 1 reply; 34+ messages in thread
From: Nilay Vaish @ 2016-06-09  3:19 UTC (permalink / raw)
  To: Naveen N. Rao
  Cc: Linux Kernel list, linuxppc-dev, netdev, mpe, Matt Evans,
	Denis Kirjanov, Paul Mackerras, Alexei Starovoitov,
	Daniel Borkmann, David S. Miller, Ananth N Mavinakayanahalli

Naveen, can you point out where in the patch you update the variable:
idx, a member of codegen_contex structure?  Somehow I am unable to
figure it out.  I can only see that we set it to 0 in the
bpf_int_jit_compile function.  Since all your test cases pass, I am
clearly overlooking something.

Thanks
Nilay

On 7 June 2016 at 08:32, Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com> wrote:
> PPC64 eBPF JIT compiler.
>
> Enable with:
> echo 1 > /proc/sys/net/core/bpf_jit_enable
> or
> echo 2 > /proc/sys/net/core/bpf_jit_enable
>
> ... to see the generated JIT code. This can further be processed with
> tools/net/bpf_jit_disasm.
>
> With CONFIG_TEST_BPF=m and 'modprobe test_bpf':
> test_bpf: Summary: 305 PASSED, 0 FAILED, [297/297 JIT'ed]
>
> ... on both ppc64 BE and LE.
>
> The details of the approach are documented through various comments in
> the code.
>
> Cc: Matt Evans <matt@ozlabs.org>
> Cc: Denis Kirjanov <kda@linux-powerpc.org>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Paul Mackerras <paulus@samba.org>
> Cc: Alexei Starovoitov <ast@fb.com>
> Cc: Daniel Borkmann <daniel@iogearbox.net>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
> Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
> ---
>  arch/powerpc/Kconfig                  |   3 +-
>  arch/powerpc/include/asm/asm-compat.h |   2 +
>  arch/powerpc/include/asm/ppc-opcode.h |  20 +-
>  arch/powerpc/net/Makefile             |   4 +
>  arch/powerpc/net/bpf_jit.h            |  53 +-
>  arch/powerpc/net/bpf_jit64.h          | 102 ++++
>  arch/powerpc/net/bpf_jit_asm64.S      | 180 +++++++
>  arch/powerpc/net/bpf_jit_comp64.c     | 956 ++++++++++++++++++++++++++++++++++
>  8 files changed, 1317 insertions(+), 3 deletions(-)
>  create mode 100644 arch/powerpc/net/bpf_jit64.h
>  create mode 100644 arch/powerpc/net/bpf_jit_asm64.S
>  create mode 100644 arch/powerpc/net/bpf_jit_comp64.c
>
> diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
> index 01f7464..ee82f9a 100644
> --- a/arch/powerpc/Kconfig
> +++ b/arch/powerpc/Kconfig
> @@ -128,7 +128,8 @@ config PPC
>         select IRQ_FORCED_THREADING
>         select HAVE_RCU_TABLE_FREE if SMP
>         select HAVE_SYSCALL_TRACEPOINTS
> -       select HAVE_CBPF_JIT
> +       select HAVE_CBPF_JIT if !PPC64
> +       select HAVE_EBPF_JIT if PPC64
>         select HAVE_ARCH_JUMP_LABEL
>         select ARCH_HAVE_NMI_SAFE_CMPXCHG
>         select ARCH_HAS_GCOV_PROFILE_ALL
> diff --git a/arch/powerpc/include/asm/asm-compat.h b/arch/powerpc/include/asm/asm-compat.h
> index dc85dcb..cee3aa0 100644
> --- a/arch/powerpc/include/asm/asm-compat.h
> +++ b/arch/powerpc/include/asm/asm-compat.h
> @@ -36,11 +36,13 @@
>  #define PPC_MIN_STKFRM 112
>
>  #ifdef __BIG_ENDIAN__
> +#define LHZX_BE        stringify_in_c(lhzx)
>  #define LWZX_BE        stringify_in_c(lwzx)
>  #define LDX_BE stringify_in_c(ldx)
>  #define STWX_BE        stringify_in_c(stwx)
>  #define STDX_BE        stringify_in_c(stdx)
>  #else
> +#define LHZX_BE        stringify_in_c(lhbrx)
>  #define LWZX_BE        stringify_in_c(lwbrx)
>  #define LDX_BE stringify_in_c(ldbrx)
>  #define STWX_BE        stringify_in_c(stwbrx)
> diff --git a/arch/powerpc/include/asm/ppc-opcode.h b/arch/powerpc/include/asm/ppc-opcode.h
> index fd8d640..6a77d130 100644
> --- a/arch/powerpc/include/asm/ppc-opcode.h
> +++ b/arch/powerpc/include/asm/ppc-opcode.h
> @@ -142,9 +142,11 @@
>  #define PPC_INST_ISEL                  0x7c00001e
>  #define PPC_INST_ISEL_MASK             0xfc00003e
>  #define PPC_INST_LDARX                 0x7c0000a8
> +#define PPC_INST_STDCX                 0x7c0001ad
>  #define PPC_INST_LSWI                  0x7c0004aa
>  #define PPC_INST_LSWX                  0x7c00042a
>  #define PPC_INST_LWARX                 0x7c000028
> +#define PPC_INST_STWCX                 0x7c00012d
>  #define PPC_INST_LWSYNC                        0x7c2004ac
>  #define PPC_INST_SYNC                  0x7c0004ac
>  #define PPC_INST_SYNC_MASK             0xfc0007fe
> @@ -211,8 +213,11 @@
>  #define PPC_INST_LBZ                   0x88000000
>  #define PPC_INST_LD                    0xe8000000
>  #define PPC_INST_LHZ                   0xa0000000
> -#define PPC_INST_LHBRX                 0x7c00062c
>  #define PPC_INST_LWZ                   0x80000000
> +#define PPC_INST_LHBRX                 0x7c00062c
> +#define PPC_INST_LDBRX                 0x7c000428
> +#define PPC_INST_STB                   0x98000000
> +#define PPC_INST_STH                   0xb0000000
>  #define PPC_INST_STD                   0xf8000000
>  #define PPC_INST_STDU                  0xf8000001
>  #define PPC_INST_STW                   0x90000000
> @@ -221,22 +226,34 @@
>  #define PPC_INST_MTLR                  0x7c0803a6
>  #define PPC_INST_CMPWI                 0x2c000000
>  #define PPC_INST_CMPDI                 0x2c200000
> +#define PPC_INST_CMPW                  0x7c000000
> +#define PPC_INST_CMPD                  0x7c200000
>  #define PPC_INST_CMPLW                 0x7c000040
> +#define PPC_INST_CMPLD                 0x7c200040
>  #define PPC_INST_CMPLWI                        0x28000000
> +#define PPC_INST_CMPLDI                        0x28200000
>  #define PPC_INST_ADDI                  0x38000000
>  #define PPC_INST_ADDIS                 0x3c000000
>  #define PPC_INST_ADD                   0x7c000214
>  #define PPC_INST_SUB                   0x7c000050
>  #define PPC_INST_BLR                   0x4e800020
>  #define PPC_INST_BLRL                  0x4e800021
> +#define PPC_INST_MULLD                 0x7c0001d2
>  #define PPC_INST_MULLW                 0x7c0001d6
>  #define PPC_INST_MULHWU                        0x7c000016
>  #define PPC_INST_MULLI                 0x1c000000
>  #define PPC_INST_DIVWU                 0x7c000396
> +#define PPC_INST_DIVD                  0x7c0003d2
>  #define PPC_INST_RLWINM                        0x54000000
> +#define PPC_INST_RLWIMI                        0x50000000
> +#define PPC_INST_RLDICL                        0x78000000
>  #define PPC_INST_RLDICR                        0x78000004
>  #define PPC_INST_SLW                   0x7c000030
> +#define PPC_INST_SLD                   0x7c000036
>  #define PPC_INST_SRW                   0x7c000430
> +#define PPC_INST_SRD                   0x7c000436
> +#define PPC_INST_SRAD                  0x7c000634
> +#define PPC_INST_SRADI                 0x7c000674
>  #define PPC_INST_AND                   0x7c000038
>  #define PPC_INST_ANDDOT                        0x7c000039
>  #define PPC_INST_OR                    0x7c000378
> @@ -247,6 +264,7 @@
>  #define PPC_INST_XORI                  0x68000000
>  #define PPC_INST_XORIS                 0x6c000000
>  #define PPC_INST_NEG                   0x7c0000d0
> +#define PPC_INST_EXTSW                 0x7c0007b4
>  #define PPC_INST_BRANCH                        0x48000000
>  #define PPC_INST_BRANCH_COND           0x40800000
>  #define PPC_INST_LBZCIX                        0x7c0006aa
> diff --git a/arch/powerpc/net/Makefile b/arch/powerpc/net/Makefile
> index 1306a58..c1ff16a 100644
> --- a/arch/powerpc/net/Makefile
> +++ b/arch/powerpc/net/Makefile
> @@ -1,4 +1,8 @@
>  #
>  # Arch-specific network modules
>  #
> +ifeq ($(CONFIG_PPC64),y)
> +obj-$(CONFIG_BPF_JIT) += bpf_jit_asm64.o bpf_jit_comp64.o
> +else
>  obj-$(CONFIG_BPF_JIT) += bpf_jit_asm.o bpf_jit_comp.o
> +endif
> diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h
> index 313cfaf..d5301b6 100644
> --- a/arch/powerpc/net/bpf_jit.h
> +++ b/arch/powerpc/net/bpf_jit.h
> @@ -2,6 +2,7 @@
>   * bpf_jit.h: BPF JIT compiler for PPC
>   *
>   * Copyright 2011 Matt Evans <matt@ozlabs.org>, IBM Corporation
> + *          2016 Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
>   *
>   * This program is free software; you can redistribute it and/or
>   * modify it under the terms of the GNU General Public License
> @@ -13,7 +14,9 @@
>
>  #ifndef __ASSEMBLY__
>
> -#ifdef CONFIG_PPC64
> +#include <asm/types.h>
> +
> +#ifdef PPC64_ELF_ABI_v1
>  #define FUNCTION_DESCR_SIZE    24
>  #else
>  #define FUNCTION_DESCR_SIZE    0
> @@ -52,6 +55,10 @@
>                                      ___PPC_RA(base) | IMM_L(i))
>  #define PPC_STWU(r, base, i)   EMIT(PPC_INST_STWU | ___PPC_RS(r) |           \
>                                      ___PPC_RA(base) | IMM_L(i))
> +#define PPC_STH(r, base, i)    EMIT(PPC_INST_STH | ___PPC_RS(r) |            \
> +                                    ___PPC_RA(base) | IMM_L(i))
> +#define PPC_STB(r, base, i)    EMIT(PPC_INST_STB | ___PPC_RS(r) |            \
> +                                    ___PPC_RA(base) | IMM_L(i))
>
>  #define PPC_LBZ(r, base, i)    EMIT(PPC_INST_LBZ | ___PPC_RT(r) |            \
>                                      ___PPC_RA(base) | IMM_L(i))
> @@ -63,6 +70,19 @@
>                                      ___PPC_RA(base) | IMM_L(i))
>  #define PPC_LHBRX(r, base, b)  EMIT(PPC_INST_LHBRX | ___PPC_RT(r) |          \
>                                      ___PPC_RA(base) | ___PPC_RB(b))
> +#define PPC_LDBRX(r, base, b)  EMIT(PPC_INST_LDBRX | ___PPC_RT(r) |          \
> +                                    ___PPC_RA(base) | ___PPC_RB(b))
> +
> +#define PPC_BPF_LDARX(t, a, b, eh) EMIT(PPC_INST_LDARX | ___PPC_RT(t) |              \
> +                                       ___PPC_RA(a) | ___PPC_RB(b) |         \
> +                                       __PPC_EH(eh))
> +#define PPC_BPF_LWARX(t, a, b, eh) EMIT(PPC_INST_LWARX | ___PPC_RT(t) |              \
> +                                       ___PPC_RA(a) | ___PPC_RB(b) |         \
> +                                       __PPC_EH(eh))
> +#define PPC_BPF_STWCX(s, a, b) EMIT(PPC_INST_STWCX | ___PPC_RS(s) |          \
> +                                       ___PPC_RA(a) | ___PPC_RB(b))
> +#define PPC_BPF_STDCX(s, a, b) EMIT(PPC_INST_STDCX | ___PPC_RS(s) |          \
> +                                       ___PPC_RA(a) | ___PPC_RB(b))
>
>  #ifdef CONFIG_PPC64
>  #define PPC_BPF_LL(r, base, i) do { PPC_LD(r, base, i); } while(0)
> @@ -76,14 +96,23 @@
>
>  #define PPC_CMPWI(a, i)                EMIT(PPC_INST_CMPWI | ___PPC_RA(a) | IMM_L(i))
>  #define PPC_CMPDI(a, i)                EMIT(PPC_INST_CMPDI | ___PPC_RA(a) | IMM_L(i))
> +#define PPC_CMPW(a, b)         EMIT(PPC_INST_CMPW | ___PPC_RA(a) |           \
> +                                       ___PPC_RB(b))
> +#define PPC_CMPD(a, b)         EMIT(PPC_INST_CMPD | ___PPC_RA(a) |           \
> +                                       ___PPC_RB(b))
>  #define PPC_CMPLWI(a, i)       EMIT(PPC_INST_CMPLWI | ___PPC_RA(a) | IMM_L(i))
> +#define PPC_CMPLDI(a, i)       EMIT(PPC_INST_CMPLDI | ___PPC_RA(a) | IMM_L(i))
>  #define PPC_CMPLW(a, b)                EMIT(PPC_INST_CMPLW | ___PPC_RA(a) |          \
>                                         ___PPC_RB(b))
> +#define PPC_CMPLD(a, b)                EMIT(PPC_INST_CMPLD | ___PPC_RA(a) |          \
> +                                       ___PPC_RB(b))
>
>  #define PPC_SUB(d, a, b)       EMIT(PPC_INST_SUB | ___PPC_RT(d) |            \
>                                      ___PPC_RB(a) | ___PPC_RA(b))
>  #define PPC_ADD(d, a, b)       EMIT(PPC_INST_ADD | ___PPC_RT(d) |            \
>                                      ___PPC_RA(a) | ___PPC_RB(b))
> +#define PPC_MULD(d, a, b)      EMIT(PPC_INST_MULLD | ___PPC_RT(d) |          \
> +                                    ___PPC_RA(a) | ___PPC_RB(b))
>  #define PPC_MULW(d, a, b)      EMIT(PPC_INST_MULLW | ___PPC_RT(d) |          \
>                                      ___PPC_RA(a) | ___PPC_RB(b))
>  #define PPC_MULHWU(d, a, b)    EMIT(PPC_INST_MULHWU | ___PPC_RT(d) |         \
> @@ -92,6 +121,8 @@
>                                      ___PPC_RA(a) | IMM_L(i))
>  #define PPC_DIVWU(d, a, b)     EMIT(PPC_INST_DIVWU | ___PPC_RT(d) |          \
>                                      ___PPC_RA(a) | ___PPC_RB(b))
> +#define PPC_DIVD(d, a, b)      EMIT(PPC_INST_DIVD | ___PPC_RT(d) |           \
> +                                    ___PPC_RA(a) | ___PPC_RB(b))
>  #define PPC_AND(d, a, b)       EMIT(PPC_INST_AND | ___PPC_RA(d) |            \
>                                      ___PPC_RS(a) | ___PPC_RB(b))
>  #define PPC_ANDI(d, a, i)      EMIT(PPC_INST_ANDI | ___PPC_RA(d) |           \
> @@ -100,6 +131,7 @@
>                                      ___PPC_RS(a) | ___PPC_RB(b))
>  #define PPC_OR(d, a, b)                EMIT(PPC_INST_OR | ___PPC_RA(d) |             \
>                                      ___PPC_RS(a) | ___PPC_RB(b))
> +#define PPC_MR(d, a)           PPC_OR(d, a, a)
>  #define PPC_ORI(d, a, i)       EMIT(PPC_INST_ORI | ___PPC_RA(d) |            \
>                                      ___PPC_RS(a) | IMM_L(i))
>  #define PPC_ORIS(d, a, i)      EMIT(PPC_INST_ORIS | ___PPC_RA(d) |           \
> @@ -110,13 +142,30 @@
>                                      ___PPC_RS(a) | IMM_L(i))
>  #define PPC_XORIS(d, a, i)     EMIT(PPC_INST_XORIS | ___PPC_RA(d) |          \
>                                      ___PPC_RS(a) | IMM_L(i))
> +#define PPC_EXTSW(d, a)                EMIT(PPC_INST_EXTSW | ___PPC_RA(d) |          \
> +                                    ___PPC_RS(a))
>  #define PPC_SLW(d, a, s)       EMIT(PPC_INST_SLW | ___PPC_RA(d) |            \
>                                      ___PPC_RS(a) | ___PPC_RB(s))
> +#define PPC_SLD(d, a, s)       EMIT(PPC_INST_SLD | ___PPC_RA(d) |            \
> +                                    ___PPC_RS(a) | ___PPC_RB(s))
>  #define PPC_SRW(d, a, s)       EMIT(PPC_INST_SRW | ___PPC_RA(d) |            \
>                                      ___PPC_RS(a) | ___PPC_RB(s))
> +#define PPC_SRD(d, a, s)       EMIT(PPC_INST_SRD | ___PPC_RA(d) |            \
> +                                    ___PPC_RS(a) | ___PPC_RB(s))
> +#define PPC_SRAD(d, a, s)      EMIT(PPC_INST_SRAD | ___PPC_RA(d) |           \
> +                                    ___PPC_RS(a) | ___PPC_RB(s))
> +#define PPC_SRADI(d, a, i)     EMIT(PPC_INST_SRADI | ___PPC_RA(d) |          \
> +                                    ___PPC_RS(a) | __PPC_SH(i) |             \
> +                                    (((i) & 0x20) >> 4))
>  #define PPC_RLWINM(d, a, i, mb, me)    EMIT(PPC_INST_RLWINM | ___PPC_RA(d) | \
>                                         ___PPC_RS(a) | __PPC_SH(i) |          \
>                                         __PPC_MB(mb) | __PPC_ME(me))
> +#define PPC_RLWIMI(d, a, i, mb, me)    EMIT(PPC_INST_RLWIMI | ___PPC_RA(d) | \
> +                                       ___PPC_RS(a) | __PPC_SH(i) |          \
> +                                       __PPC_MB(mb) | __PPC_ME(me))
> +#define PPC_RLDICL(d, a, i, mb)                EMIT(PPC_INST_RLDICL | ___PPC_RA(d) | \
> +                                       ___PPC_RS(a) | __PPC_SH(i) |          \
> +                                       __PPC_MB64(mb) | (((i) & 0x20) >> 4))
>  #define PPC_RLDICR(d, a, i, me)                EMIT(PPC_INST_RLDICR | ___PPC_RA(d) | \
>                                         ___PPC_RS(a) | __PPC_SH(i) |          \
>                                         __PPC_ME64(me) | (((i) & 0x20) >> 4))
> @@ -127,6 +176,8 @@
>  #define PPC_SRWI(d, a, i)      PPC_RLWINM(d, a, 32-(i), i, 31)
>  /* sldi = rldicr Rx, Ry, n, 63-n */
>  #define PPC_SLDI(d, a, i)      PPC_RLDICR(d, a, i, 63-(i))
> +/* sldi = rldicl Rx, Ry, 64-n, n */
> +#define PPC_SRDI(d, a, i)      PPC_RLDICL(d, a, 64-(i), i)
>
>  #define PPC_NEG(d, a)          EMIT(PPC_INST_NEG | ___PPC_RT(d) | ___PPC_RA(a))
>
> diff --git a/arch/powerpc/net/bpf_jit64.h b/arch/powerpc/net/bpf_jit64.h
> new file mode 100644
> index 0000000..5046d6f
> --- /dev/null
> +++ b/arch/powerpc/net/bpf_jit64.h
> @@ -0,0 +1,102 @@
> +/*
> + * bpf_jit64.h: BPF JIT compiler for PPC64
> + *
> + * Copyright 2016 Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
> + *               IBM Corporation
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; version 2
> + * of the License.
> + */
> +#ifndef _BPF_JIT64_H
> +#define _BPF_JIT64_H
> +
> +#include "bpf_jit.h"
> +
> +/*
> + * Stack layout:
> + *
> + *             [       prev sp         ] <-------------
> + *             [   nv gpr save area    ] 8*8           |
> + * fp (r31) -->        [   ebpf stack space    ] 512           |
> + *             [  local/tmp var space  ] 16            |
> + *             [     frame header      ] 32/112        |
> + * sp (r1) --->        [    stack pointer      ] --------------
> + */
> +
> +/* for bpf JIT code internal usage */
> +#define BPF_PPC_STACK_LOCALS   16
> +/* for gpr non volatile registers BPG_REG_6 to 10, plus skb cache registers */
> +#define BPF_PPC_STACK_SAVE     (8*8)
> +/* Ensure this is quadword aligned */
> +#define BPF_PPC_STACKFRAME     (STACK_FRAME_MIN_SIZE + BPF_PPC_STACK_LOCALS + \
> +                                MAX_BPF_STACK + BPF_PPC_STACK_SAVE)
> +
> +#ifndef __ASSEMBLY__
> +
> +/* BPF register usage */
> +#define SKB_HLEN_REG   (MAX_BPF_REG + 0)
> +#define SKB_DATA_REG   (MAX_BPF_REG + 1)
> +#define TMP_REG_1      (MAX_BPF_REG + 2)
> +#define TMP_REG_2      (MAX_BPF_REG + 3)
> +
> +/* BPF to ppc register mappings */
> +static const int b2p[] = {
> +       /* function return value */
> +       [BPF_REG_0] = 8,
> +       /* function arguments */
> +       [BPF_REG_1] = 3,
> +       [BPF_REG_2] = 4,
> +       [BPF_REG_3] = 5,
> +       [BPF_REG_4] = 6,
> +       [BPF_REG_5] = 7,
> +       /* non volatile registers */
> +       [BPF_REG_6] = 27,
> +       [BPF_REG_7] = 28,
> +       [BPF_REG_8] = 29,
> +       [BPF_REG_9] = 30,
> +       /* frame pointer aka BPF_REG_10 */
> +       [BPF_REG_FP] = 31,
> +       /* eBPF jit internal registers */
> +       [SKB_HLEN_REG] = 25,
> +       [SKB_DATA_REG] = 26,
> +       [TMP_REG_1] = 9,
> +       [TMP_REG_2] = 10
> +};
> +
> +/* Assembly helpers */
> +#define DECLARE_LOAD_FUNC(func)        u64 func(u64 r3, u64 r4);                       \
> +                               u64 func##_negative_offset(u64 r3, u64 r4);     \
> +                               u64 func##_positive_offset(u64 r3, u64 r4);
> +
> +DECLARE_LOAD_FUNC(sk_load_word);
> +DECLARE_LOAD_FUNC(sk_load_half);
> +DECLARE_LOAD_FUNC(sk_load_byte);
> +
> +#define CHOOSE_LOAD_FUNC(imm, func)                                            \
> +                       (imm < 0 ?                                              \
> +                       (imm >= SKF_LL_OFF ? func##_negative_offset : func) :   \
> +                       func##_positive_offset)
> +
> +#define SEEN_FUNC      0x1000 /* might call external helpers */
> +#define SEEN_STACK     0x2000 /* uses BPF stack */
> +#define SEEN_SKB       0x4000 /* uses sk_buff */
> +
> +struct codegen_context {
> +       /*
> +        * This is used to track register usage as well
> +        * as calls to external helpers.
> +        * - register usage is tracked with corresponding
> +        *   bits (r3-r10 and r25-r31)
> +        * - rest of the bits can be used to track other
> +        *   things -- for now, we use bits 16 to 23
> +        *   encoded in SEEN_* macros above
> +        */
> +       unsigned int seen;
> +       unsigned int idx;
> +};
> +
> +#endif /* !__ASSEMBLY__ */
> +
> +#endif
> diff --git a/arch/powerpc/net/bpf_jit_asm64.S b/arch/powerpc/net/bpf_jit_asm64.S
> new file mode 100644
> index 0000000..7e4c514
> --- /dev/null
> +++ b/arch/powerpc/net/bpf_jit_asm64.S
> @@ -0,0 +1,180 @@
> +/*
> + * bpf_jit_asm64.S: Packet/header access helper functions
> + * for PPC64 BPF compiler.
> + *
> + * Copyright 2016, Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
> + *                IBM Corporation
> + *
> + * Based on bpf_jit_asm.S by Matt Evans
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; version 2
> + * of the License.
> + */
> +
> +#include <asm/ppc_asm.h>
> +#include <asm/ptrace.h>
> +#include "bpf_jit64.h"
> +
> +/*
> + * All of these routines are called directly from generated code,
> + * with the below register usage:
> + * r27         skb pointer (ctx)
> + * r25         skb header length
> + * r26         skb->data pointer
> + * r4          offset
> + *
> + * Result is passed back in:
> + * r8          data read in host endian format (accumulator)
> + *
> + * r9 is used as a temporary register
> + */
> +
> +#define r_skb  r27
> +#define r_hlen r25
> +#define r_data r26
> +#define r_off  r4
> +#define r_val  r8
> +#define r_tmp  r9
> +
> +_GLOBAL_TOC(sk_load_word)
> +       cmpdi   r_off, 0
> +       blt     bpf_slow_path_word_neg
> +       b       sk_load_word_positive_offset
> +
> +_GLOBAL_TOC(sk_load_word_positive_offset)
> +       /* Are we accessing past headlen? */
> +       subi    r_tmp, r_hlen, 4
> +       cmpd    r_tmp, r_off
> +       blt     bpf_slow_path_word
> +       /* Nope, just hitting the header.  cr0 here is eq or gt! */
> +       LWZX_BE r_val, r_data, r_off
> +       blr     /* Return success, cr0 != LT */
> +
> +_GLOBAL_TOC(sk_load_half)
> +       cmpdi   r_off, 0
> +       blt     bpf_slow_path_half_neg
> +       b       sk_load_half_positive_offset
> +
> +_GLOBAL_TOC(sk_load_half_positive_offset)
> +       subi    r_tmp, r_hlen, 2
> +       cmpd    r_tmp, r_off
> +       blt     bpf_slow_path_half
> +       LHZX_BE r_val, r_data, r_off
> +       blr
> +
> +_GLOBAL_TOC(sk_load_byte)
> +       cmpdi   r_off, 0
> +       blt     bpf_slow_path_byte_neg
> +       b       sk_load_byte_positive_offset
> +
> +_GLOBAL_TOC(sk_load_byte_positive_offset)
> +       cmpd    r_hlen, r_off
> +       ble     bpf_slow_path_byte
> +       lbzx    r_val, r_data, r_off
> +       blr
> +
> +/*
> + * Call out to skb_copy_bits:
> + * Allocate a new stack frame here to remain ABI-compliant in
> + * stashing LR.
> + */
> +#define bpf_slow_path_common(SIZE)                                     \
> +       mflr    r0;                                                     \
> +       std     r0, PPC_LR_STKOFF(r1);                                  \
> +       stdu    r1, -(STACK_FRAME_MIN_SIZE + BPF_PPC_STACK_LOCALS)(r1); \
> +       mr      r3, r_skb;                                              \
> +       /* r4 = r_off as passed */                                      \
> +       addi    r5, r1, STACK_FRAME_MIN_SIZE;                           \
> +       li      r6, SIZE;                                               \
> +       bl      skb_copy_bits;                                          \
> +       nop;                                                            \
> +       /* save r5 */                                                   \
> +       addi    r5, r1, STACK_FRAME_MIN_SIZE;                           \
> +       /* r3 = 0 on success */                                         \
> +       addi    r1, r1, STACK_FRAME_MIN_SIZE + BPF_PPC_STACK_LOCALS;    \
> +       ld      r0, PPC_LR_STKOFF(r1);                                  \
> +       mtlr    r0;                                                     \
> +       cmpdi   r3, 0;                                                  \
> +       blt     bpf_error;      /* cr0 = LT */
> +
> +bpf_slow_path_word:
> +       bpf_slow_path_common(4)
> +       /* Data value is on stack, and cr0 != LT */
> +       LWZX_BE r_val, 0, r5
> +       blr
> +
> +bpf_slow_path_half:
> +       bpf_slow_path_common(2)
> +       LHZX_BE r_val, 0, r5
> +       blr
> +
> +bpf_slow_path_byte:
> +       bpf_slow_path_common(1)
> +       lbzx    r_val, 0, r5
> +       blr
> +
> +/*
> + * Call out to bpf_internal_load_pointer_neg_helper
> + */
> +#define sk_negative_common(SIZE)                               \
> +       mflr    r0;                                             \
> +       std     r0, PPC_LR_STKOFF(r1);                          \
> +       stdu    r1, -STACK_FRAME_MIN_SIZE(r1);                  \
> +       mr      r3, r_skb;                                      \
> +       /* r4 = r_off, as passed */                             \
> +       li      r5, SIZE;                                       \
> +       bl      bpf_internal_load_pointer_neg_helper;           \
> +       nop;                                                    \
> +       addi    r1, r1, STACK_FRAME_MIN_SIZE;                   \
> +       ld      r0, PPC_LR_STKOFF(r1);                          \
> +       mtlr    r0;                                             \
> +       /* R3 != 0 on success */                                \
> +       cmpldi  r3, 0;                                          \
> +       beq     bpf_error_slow; /* cr0 = EQ */
> +
> +bpf_slow_path_word_neg:
> +       lis     r_tmp, -32      /* SKF_LL_OFF */
> +       cmpd    r_off, r_tmp    /* addr < SKF_* */
> +       blt     bpf_error       /* cr0 = LT */
> +       b       sk_load_word_negative_offset
> +
> +_GLOBAL_TOC(sk_load_word_negative_offset)
> +       sk_negative_common(4)
> +       LWZX_BE r_val, 0, r3
> +       blr
> +
> +bpf_slow_path_half_neg:
> +       lis     r_tmp, -32      /* SKF_LL_OFF */
> +       cmpd    r_off, r_tmp    /* addr < SKF_* */
> +       blt     bpf_error       /* cr0 = LT */
> +       b       sk_load_half_negative_offset
> +
> +_GLOBAL_TOC(sk_load_half_negative_offset)
> +       sk_negative_common(2)
> +       LHZX_BE r_val, 0, r3
> +       blr
> +
> +bpf_slow_path_byte_neg:
> +       lis     r_tmp, -32      /* SKF_LL_OFF */
> +       cmpd    r_off, r_tmp    /* addr < SKF_* */
> +       blt     bpf_error       /* cr0 = LT */
> +       b       sk_load_byte_negative_offset
> +
> +_GLOBAL_TOC(sk_load_byte_negative_offset)
> +       sk_negative_common(1)
> +       lbzx    r_val, 0, r3
> +       blr
> +
> +bpf_error_slow:
> +       /* fabricate a cr0 = lt */
> +       li      r_tmp, -1
> +       cmpdi   r_tmp, 0
> +bpf_error:
> +       /*
> +        * Entered with cr0 = lt
> +        * Generated code will 'blt epilogue', returning 0.
> +        */
> +       li      r_val, 0
> +       blr
> diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
> new file mode 100644
> index 0000000..954ff53
> --- /dev/null
> +++ b/arch/powerpc/net/bpf_jit_comp64.c
> @@ -0,0 +1,956 @@
> +/*
> + * bpf_jit_comp64.c: eBPF JIT compiler
> + *
> + * Copyright 2016 Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
> + *               IBM Corporation
> + *
> + * Based on the powerpc classic BPF JIT compiler by Matt Evans
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; version 2
> + * of the License.
> + */
> +#include <linux/moduleloader.h>
> +#include <asm/cacheflush.h>
> +#include <linux/netdevice.h>
> +#include <linux/filter.h>
> +#include <linux/if_vlan.h>
> +
> +#include "bpf_jit64.h"
> +
> +int bpf_jit_enable __read_mostly;
> +
> +static void bpf_jit_fill_ill_insns(void *area, unsigned int size)
> +{
> +       int *p = area;
> +
> +       /* Fill whole space with trap instructions */
> +       while (p < (int *)((char *)area + size))
> +               *p++ = BREAKPOINT_INSTRUCTION;
> +}
> +
> +static inline void bpf_flush_icache(void *start, void *end)
> +{
> +       smp_wmb();
> +       flush_icache_range((unsigned long)start, (unsigned long)end);
> +}
> +
> +static inline bool bpf_is_seen_register(struct codegen_context *ctx, int i)
> +{
> +       return (ctx->seen & (1 << (31 - b2p[i])));
> +}
> +
> +static inline void bpf_set_seen_register(struct codegen_context *ctx, int i)
> +{
> +       ctx->seen |= (1 << (31 - b2p[i]));
> +}
> +
> +static inline bool bpf_has_stack_frame(struct codegen_context *ctx)
> +{
> +       /*
> +        * We only need a stack frame if:
> +        * - we call other functions (kernel helpers), or
> +        * - the bpf program uses its stack area
> +        * The latter condition is deduced from the usage of BPF_REG_FP
> +        */
> +       return ctx->seen & SEEN_FUNC || bpf_is_seen_register(ctx, BPF_REG_FP);
> +}
> +
> +static void bpf_jit_emit_skb_loads(u32 *image, struct codegen_context *ctx)
> +{
> +       /*
> +        * Load skb->len and skb->data_len
> +        * r3 points to skb
> +        */
> +       PPC_LWZ(b2p[SKB_HLEN_REG], 3, offsetof(struct sk_buff, len));
> +       PPC_LWZ(b2p[TMP_REG_1], 3, offsetof(struct sk_buff, data_len));
> +       /* header_len = len - data_len */
> +       PPC_SUB(b2p[SKB_HLEN_REG], b2p[SKB_HLEN_REG], b2p[TMP_REG_1]);
> +
> +       /* skb->data pointer */
> +       PPC_BPF_LL(b2p[SKB_DATA_REG], 3, offsetof(struct sk_buff, data));
> +}
> +
> +static void bpf_jit_emit_func_call(u32 *image, struct codegen_context *ctx, u64 func)
> +{
> +#ifdef PPC64_ELF_ABI_v1
> +       /* func points to the function descriptor */
> +       PPC_LI64(b2p[TMP_REG_2], func);
> +       /* Load actual entry point from function descriptor */
> +       PPC_BPF_LL(b2p[TMP_REG_1], b2p[TMP_REG_2], 0);
> +       /* ... and move it to LR */
> +       PPC_MTLR(b2p[TMP_REG_1]);
> +       /*
> +        * Load TOC from function descriptor at offset 8.
> +        * We can clobber r2 since we get called through a
> +        * function pointer (so caller will save/restore r2)
> +        * and since we don't use a TOC ourself.
> +        */
> +       PPC_BPF_LL(2, b2p[TMP_REG_2], 8);
> +#else
> +       /* We can clobber r12 */
> +       PPC_FUNC_ADDR(12, func);
> +       PPC_MTLR(12);
> +#endif
> +       PPC_BLRL();
> +}
> +
> +static void bpf_jit_build_prologue(u32 *image, struct codegen_context *ctx)
> +{
> +       int i;
> +       bool new_stack_frame = bpf_has_stack_frame(ctx);
> +
> +       if (new_stack_frame) {
> +               /*
> +                * We need a stack frame, but we don't necessarily need to
> +                * save/restore LR unless we call other functions
> +                */
> +               if (ctx->seen & SEEN_FUNC) {
> +                       EMIT(PPC_INST_MFLR | __PPC_RT(R0));
> +                       PPC_BPF_STL(0, 1, PPC_LR_STKOFF);
> +               }
> +
> +               PPC_BPF_STLU(1, 1, -BPF_PPC_STACKFRAME);
> +       }
> +
> +       /*
> +        * Back up non-volatile regs -- BPF registers 6-10
> +        * If we haven't created our own stack frame, we save these
> +        * in the protected zone below the previous stack frame
> +        */
> +       for (i = BPF_REG_6; i <= BPF_REG_10; i++)
> +               if (bpf_is_seen_register(ctx, i))
> +                       PPC_BPF_STL(b2p[i], 1,
> +                               (new_stack_frame ? BPF_PPC_STACKFRAME : 0) -
> +                                       (8 * (32 - b2p[i])));
> +
> +       /*
> +        * Save additional non-volatile regs if we cache skb
> +        * Also, setup skb data
> +        */
> +       if (ctx->seen & SEEN_SKB) {
> +               PPC_BPF_STL(b2p[SKB_HLEN_REG], 1,
> +                       BPF_PPC_STACKFRAME - (8 * (32 - b2p[SKB_HLEN_REG])));
> +               PPC_BPF_STL(b2p[SKB_DATA_REG], 1,
> +                       BPF_PPC_STACKFRAME - (8 * (32 - b2p[SKB_DATA_REG])));
> +               bpf_jit_emit_skb_loads(image, ctx);
> +       }
> +
> +       /* Setup frame pointer to point to the bpf stack area */
> +       if (bpf_is_seen_register(ctx, BPF_REG_FP))
> +               PPC_ADDI(b2p[BPF_REG_FP], 1,
> +                               BPF_PPC_STACKFRAME - BPF_PPC_STACK_SAVE);
> +}
> +
> +static void bpf_jit_build_epilogue(u32 *image, struct codegen_context *ctx)
> +{
> +       int i;
> +       bool new_stack_frame = bpf_has_stack_frame(ctx);
> +
> +       /* Move result to r3 */
> +       PPC_MR(3, b2p[BPF_REG_0]);
> +
> +       /* Restore NVRs */
> +       for (i = BPF_REG_6; i <= BPF_REG_10; i++)
> +               if (bpf_is_seen_register(ctx, i))
> +                       PPC_BPF_LL(b2p[i], 1,
> +                               (new_stack_frame ? BPF_PPC_STACKFRAME : 0) -
> +                                       (8 * (32 - b2p[i])));
> +
> +       /* Restore non-volatile registers used for skb cache */
> +       if (ctx->seen & SEEN_SKB) {
> +               PPC_BPF_LL(b2p[SKB_HLEN_REG], 1,
> +                       BPF_PPC_STACKFRAME - (8 * (32 - b2p[SKB_HLEN_REG])));
> +               PPC_BPF_LL(b2p[SKB_DATA_REG], 1,
> +                       BPF_PPC_STACKFRAME - (8 * (32 - b2p[SKB_DATA_REG])));
> +       }
> +
> +       /* Tear down our stack frame */
> +       if (new_stack_frame) {
> +               PPC_ADDI(1, 1, BPF_PPC_STACKFRAME);
> +               if (ctx->seen & SEEN_FUNC) {
> +                       PPC_BPF_LL(0, 1, PPC_LR_STKOFF);
> +                       PPC_MTLR(0);
> +               }
> +       }
> +
> +       PPC_BLR();
> +}
> +
> +/* Assemble the body code between the prologue & epilogue */
> +static int bpf_jit_build_body(struct bpf_prog *fp, u32 *image,
> +                             struct codegen_context *ctx,
> +                             u32 *addrs)
> +{
> +       const struct bpf_insn *insn = fp->insnsi;
> +       int flen = fp->len;
> +       int i;
> +
> +       /* Start of epilogue code - will only be valid 2nd pass onwards */
> +       u32 exit_addr = addrs[flen];
> +
> +       for (i = 0; i < flen; i++) {
> +               u32 code = insn[i].code;
> +               u32 dst_reg = b2p[insn[i].dst_reg];
> +               u32 src_reg = b2p[insn[i].src_reg];
> +               s16 off = insn[i].off;
> +               s32 imm = insn[i].imm;
> +               u64 imm64;
> +               u8 *func;
> +               u32 true_cond;
> +               int stack_local_off;
> +
> +               /*
> +                * addrs[] maps a BPF bytecode address into a real offset from
> +                * the start of the body code.
> +                */
> +               addrs[i] = ctx->idx * 4;
> +
> +               /*
> +                * As an optimization, we note down which non-volatile registers
> +                * are used so that we can only save/restore those in our
> +                * prologue and epilogue. We do this here regardless of whether
> +                * the actual BPF instruction uses src/dst registers or not
> +                * (for instance, BPF_CALL does not use them). The expectation
> +                * is that those instructions will have src_reg/dst_reg set to
> +                * 0. Even otherwise, we just lose some prologue/epilogue
> +                * optimization but everything else should work without
> +                * any issues.
> +                */
> +               if (dst_reg >= 24 && dst_reg <= 31)
> +                       bpf_set_seen_register(ctx, insn[i].dst_reg);
> +               if (src_reg >= 24 && src_reg <= 31)
> +                       bpf_set_seen_register(ctx, insn[i].src_reg);
> +
> +               switch (code) {
> +               /*
> +                * Arithmetic operations: ADD/SUB/MUL/DIV/MOD/NEG
> +                */
> +               case BPF_ALU | BPF_ADD | BPF_X: /* (u32) dst += (u32) src */
> +               case BPF_ALU64 | BPF_ADD | BPF_X: /* dst += src */
> +                       PPC_ADD(dst_reg, dst_reg, src_reg);
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_SUB | BPF_X: /* (u32) dst -= (u32) src */
> +               case BPF_ALU64 | BPF_SUB | BPF_X: /* dst -= src */
> +                       PPC_SUB(dst_reg, dst_reg, src_reg);
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_ADD | BPF_K: /* (u32) dst += (u32) imm */
> +               case BPF_ALU | BPF_SUB | BPF_K: /* (u32) dst -= (u32) imm */
> +               case BPF_ALU64 | BPF_ADD | BPF_K: /* dst += imm */
> +               case BPF_ALU64 | BPF_SUB | BPF_K: /* dst -= imm */
> +                       if (BPF_OP(code) == BPF_SUB)
> +                               imm = -imm;
> +                       if (imm) {
> +                               if (imm >= -32768 && imm < 32768)
> +                                       PPC_ADDI(dst_reg, dst_reg, IMM_L(imm));
> +                               else {
> +                                       PPC_LI32(b2p[TMP_REG_1], imm);
> +                                       PPC_ADD(dst_reg, dst_reg, b2p[TMP_REG_1]);
> +                               }
> +                       }
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_MUL | BPF_X: /* (u32) dst *= (u32) src */
> +               case BPF_ALU64 | BPF_MUL | BPF_X: /* dst *= src */
> +                       if (BPF_CLASS(code) == BPF_ALU)
> +                               PPC_MULW(dst_reg, dst_reg, src_reg);
> +                       else
> +                               PPC_MULD(dst_reg, dst_reg, src_reg);
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_MUL | BPF_K: /* (u32) dst *= (u32) imm */
> +               case BPF_ALU64 | BPF_MUL | BPF_K: /* dst *= imm */
> +                       if (imm >= -32768 && imm < 32768)
> +                               PPC_MULI(dst_reg, dst_reg, IMM_L(imm));
> +                       else {
> +                               PPC_LI32(b2p[TMP_REG_1], imm);
> +                               if (BPF_CLASS(code) == BPF_ALU)
> +                                       PPC_MULW(dst_reg, dst_reg,
> +                                                       b2p[TMP_REG_1]);
> +                               else
> +                                       PPC_MULD(dst_reg, dst_reg,
> +                                                       b2p[TMP_REG_1]);
> +                       }
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_DIV | BPF_X: /* (u32) dst /= (u32) src */
> +               case BPF_ALU | BPF_MOD | BPF_X: /* (u32) dst %= (u32) src */
> +                       PPC_CMPWI(src_reg, 0);
> +                       PPC_BCC_SHORT(COND_NE, (ctx->idx * 4) + 12);
> +                       PPC_LI(b2p[BPF_REG_0], 0);
> +                       PPC_JMP(exit_addr);
> +                       if (BPF_OP(code) == BPF_MOD) {
> +                               PPC_DIVWU(b2p[TMP_REG_1], dst_reg, src_reg);
> +                               PPC_MULW(b2p[TMP_REG_1], src_reg,
> +                                               b2p[TMP_REG_1]);
> +                               PPC_SUB(dst_reg, dst_reg, b2p[TMP_REG_1]);
> +                       } else
> +                               PPC_DIVWU(dst_reg, dst_reg, src_reg);
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU64 | BPF_DIV | BPF_X: /* dst /= src */
> +               case BPF_ALU64 | BPF_MOD | BPF_X: /* dst %= src */
> +                       PPC_CMPDI(src_reg, 0);
> +                       PPC_BCC_SHORT(COND_NE, (ctx->idx * 4) + 12);
> +                       PPC_LI(b2p[BPF_REG_0], 0);
> +                       PPC_JMP(exit_addr);
> +                       if (BPF_OP(code) == BPF_MOD) {
> +                               PPC_DIVD(b2p[TMP_REG_1], dst_reg, src_reg);
> +                               PPC_MULD(b2p[TMP_REG_1], src_reg,
> +                                               b2p[TMP_REG_1]);
> +                               PPC_SUB(dst_reg, dst_reg, b2p[TMP_REG_1]);
> +                       } else
> +                               PPC_DIVD(dst_reg, dst_reg, src_reg);
> +                       break;
> +               case BPF_ALU | BPF_MOD | BPF_K: /* (u32) dst %= (u32) imm */
> +               case BPF_ALU | BPF_DIV | BPF_K: /* (u32) dst /= (u32) imm */
> +               case BPF_ALU64 | BPF_MOD | BPF_K: /* dst %= imm */
> +               case BPF_ALU64 | BPF_DIV | BPF_K: /* dst /= imm */
> +                       if (imm == 0)
> +                               return -EINVAL;
> +                       else if (imm == 1)
> +                               goto bpf_alu32_trunc;
> +
> +                       PPC_LI32(b2p[TMP_REG_1], imm);
> +                       switch (BPF_CLASS(code)) {
> +                       case BPF_ALU:
> +                               if (BPF_OP(code) == BPF_MOD) {
> +                                       PPC_DIVWU(b2p[TMP_REG_2], dst_reg,
> +                                                       b2p[TMP_REG_1]);
> +                                       PPC_MULW(b2p[TMP_REG_1],
> +                                                       b2p[TMP_REG_1],
> +                                                       b2p[TMP_REG_2]);
> +                                       PPC_SUB(dst_reg, dst_reg,
> +                                                       b2p[TMP_REG_1]);
> +                               } else
> +                                       PPC_DIVWU(dst_reg, dst_reg,
> +                                                       b2p[TMP_REG_1]);
> +                               break;
> +                       case BPF_ALU64:
> +                               if (BPF_OP(code) == BPF_MOD) {
> +                                       PPC_DIVD(b2p[TMP_REG_2], dst_reg,
> +                                                       b2p[TMP_REG_1]);
> +                                       PPC_MULD(b2p[TMP_REG_1],
> +                                                       b2p[TMP_REG_1],
> +                                                       b2p[TMP_REG_2]);
> +                                       PPC_SUB(dst_reg, dst_reg,
> +                                                       b2p[TMP_REG_1]);
> +                               } else
> +                                       PPC_DIVD(dst_reg, dst_reg,
> +                                                       b2p[TMP_REG_1]);
> +                               break;
> +                       }
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_NEG: /* (u32) dst = -dst */
> +               case BPF_ALU64 | BPF_NEG: /* dst = -dst */
> +                       PPC_NEG(dst_reg, dst_reg);
> +                       goto bpf_alu32_trunc;
> +
> +               /*
> +                * Logical operations: AND/OR/XOR/[A]LSH/[A]RSH
> +                */
> +               case BPF_ALU | BPF_AND | BPF_X: /* (u32) dst = dst & src */
> +               case BPF_ALU64 | BPF_AND | BPF_X: /* dst = dst & src */
> +                       PPC_AND(dst_reg, dst_reg, src_reg);
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_AND | BPF_K: /* (u32) dst = dst & imm */
> +               case BPF_ALU64 | BPF_AND | BPF_K: /* dst = dst & imm */
> +                       if (!IMM_H(imm))
> +                               PPC_ANDI(dst_reg, dst_reg, IMM_L(imm));
> +                       else {
> +                               /* Sign-extended */
> +                               PPC_LI32(b2p[TMP_REG_1], imm);
> +                               PPC_AND(dst_reg, dst_reg, b2p[TMP_REG_1]);
> +                       }
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_OR | BPF_X: /* dst = (u32) dst | (u32) src */
> +               case BPF_ALU64 | BPF_OR | BPF_X: /* dst = dst | src */
> +                       PPC_OR(dst_reg, dst_reg, src_reg);
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_OR | BPF_K:/* dst = (u32) dst | (u32) imm */
> +               case BPF_ALU64 | BPF_OR | BPF_K:/* dst = dst | imm */
> +                       if (imm < 0 && BPF_CLASS(code) == BPF_ALU64) {
> +                               /* Sign-extended */
> +                               PPC_LI32(b2p[TMP_REG_1], imm);
> +                               PPC_OR(dst_reg, dst_reg, b2p[TMP_REG_1]);
> +                       } else {
> +                               if (IMM_L(imm))
> +                                       PPC_ORI(dst_reg, dst_reg, IMM_L(imm));
> +                               if (IMM_H(imm))
> +                                       PPC_ORIS(dst_reg, dst_reg, IMM_H(imm));
> +                       }
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_XOR | BPF_X: /* (u32) dst ^= src */
> +               case BPF_ALU64 | BPF_XOR | BPF_X: /* dst ^= src */
> +                       PPC_XOR(dst_reg, dst_reg, src_reg);
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_XOR | BPF_K: /* (u32) dst ^= (u32) imm */
> +               case BPF_ALU64 | BPF_XOR | BPF_K: /* dst ^= imm */
> +                       if (imm < 0 && BPF_CLASS(code) == BPF_ALU64) {
> +                               /* Sign-extended */
> +                               PPC_LI32(b2p[TMP_REG_1], imm);
> +                               PPC_XOR(dst_reg, dst_reg, b2p[TMP_REG_1]);
> +                       } else {
> +                               if (IMM_L(imm))
> +                                       PPC_XORI(dst_reg, dst_reg, IMM_L(imm));
> +                               if (IMM_H(imm))
> +                                       PPC_XORIS(dst_reg, dst_reg, IMM_H(imm));
> +                       }
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_LSH | BPF_X: /* (u32) dst <<= (u32) src */
> +                       /* slw clears top 32 bits */
> +                       PPC_SLW(dst_reg, dst_reg, src_reg);
> +                       break;
> +               case BPF_ALU64 | BPF_LSH | BPF_X: /* dst <<= src; */
> +                       PPC_SLD(dst_reg, dst_reg, src_reg);
> +                       break;
> +               case BPF_ALU | BPF_LSH | BPF_K: /* (u32) dst <<== (u32) imm */
> +                       /* with imm 0, we still need to clear top 32 bits */
> +                       PPC_SLWI(dst_reg, dst_reg, imm);
> +                       break;
> +               case BPF_ALU64 | BPF_LSH | BPF_K: /* dst <<== imm */
> +                       if (imm != 0)
> +                               PPC_SLDI(dst_reg, dst_reg, imm);
> +                       break;
> +               case BPF_ALU | BPF_RSH | BPF_X: /* (u32) dst >>= (u32) src */
> +                       PPC_SRW(dst_reg, dst_reg, src_reg);
> +                       break;
> +               case BPF_ALU64 | BPF_RSH | BPF_X: /* dst >>= src */
> +                       PPC_SRD(dst_reg, dst_reg, src_reg);
> +                       break;
> +               case BPF_ALU | BPF_RSH | BPF_K: /* (u32) dst >>= (u32) imm */
> +                       PPC_SRWI(dst_reg, dst_reg, imm);
> +                       break;
> +               case BPF_ALU64 | BPF_RSH | BPF_K: /* dst >>= imm */
> +                       if (imm != 0)
> +                               PPC_SRDI(dst_reg, dst_reg, imm);
> +                       break;
> +               case BPF_ALU64 | BPF_ARSH | BPF_X: /* (s64) dst >>= src */
> +                       PPC_SRAD(dst_reg, dst_reg, src_reg);
> +                       break;
> +               case BPF_ALU64 | BPF_ARSH | BPF_K: /* (s64) dst >>= imm */
> +                       if (imm != 0)
> +                               PPC_SRADI(dst_reg, dst_reg, imm);
> +                       break;
> +
> +               /*
> +                * MOV
> +                */
> +               case BPF_ALU | BPF_MOV | BPF_X: /* (u32) dst = src */
> +               case BPF_ALU64 | BPF_MOV | BPF_X: /* dst = src */
> +                       PPC_MR(dst_reg, src_reg);
> +                       goto bpf_alu32_trunc;
> +               case BPF_ALU | BPF_MOV | BPF_K: /* (u32) dst = imm */
> +               case BPF_ALU64 | BPF_MOV | BPF_K: /* dst = (s64) imm */
> +                       PPC_LI32(dst_reg, imm);
> +                       if (imm < 0)
> +                               goto bpf_alu32_trunc;
> +                       break;
> +
> +bpf_alu32_trunc:
> +               /* Truncate to 32-bits */
> +               if (BPF_CLASS(code) == BPF_ALU)
> +                       PPC_RLWINM(dst_reg, dst_reg, 0, 0, 31);
> +               break;
> +
> +               /*
> +                * BPF_FROM_BE/LE
> +                */
> +               case BPF_ALU | BPF_END | BPF_FROM_LE:
> +               case BPF_ALU | BPF_END | BPF_FROM_BE:
> +#ifdef __BIG_ENDIAN__
> +                       if (BPF_SRC(code) == BPF_FROM_BE)
> +                               goto emit_clear;
> +#else /* !__BIG_ENDIAN__ */
> +                       if (BPF_SRC(code) == BPF_FROM_LE)
> +                               goto emit_clear;
> +#endif
> +                       switch (imm) {
> +                       case 16:
> +                               /* Rotate 8 bits left & mask with 0x0000ff00 */
> +                               PPC_RLWINM(b2p[TMP_REG_1], dst_reg, 8, 16, 23);
> +                               /* Rotate 8 bits right & insert LSB to reg */
> +                               PPC_RLWIMI(b2p[TMP_REG_1], dst_reg, 24, 24, 31);
> +                               /* Move result back to dst_reg */
> +                               PPC_MR(dst_reg, b2p[TMP_REG_1]);
> +                               break;
> +                       case 32:
> +                               /*
> +                                * Rotate word left by 8 bits:
> +                                * 2 bytes are already in their final position
> +                                * -- byte 2 and 4 (of bytes 1, 2, 3 and 4)
> +                                */
> +                               PPC_RLWINM(b2p[TMP_REG_1], dst_reg, 8, 0, 31);
> +                               /* Rotate 24 bits and insert byte 1 */
> +                               PPC_RLWIMI(b2p[TMP_REG_1], dst_reg, 24, 0, 7);
> +                               /* Rotate 24 bits and insert byte 3 */
> +                               PPC_RLWIMI(b2p[TMP_REG_1], dst_reg, 24, 16, 23);
> +                               PPC_MR(dst_reg, b2p[TMP_REG_1]);
> +                               break;
> +                       case 64:
> +                               /*
> +                                * Way easier and faster(?) to store the value
> +                                * into stack and then use ldbrx
> +                                *
> +                                * First, determine where in stack we can store
> +                                * this:
> +                                * - if we have allotted a stack frame, then we
> +                                *   will utilize the area set aside by
> +                                *   BPF_PPC_STACK_LOCALS
> +                                * - else, we use the area beneath the NV GPR
> +                                *   save area
> +                                *
> +                                * ctx->seen will be reliable in pass2, but
> +                                * the instructions generated will remain the
> +                                * same across all passes
> +                                */
> +                               if (bpf_has_stack_frame(ctx))
> +                                       stack_local_off = STACK_FRAME_MIN_SIZE;
> +                               else
> +                                       stack_local_off = -(BPF_PPC_STACK_SAVE + 8);
> +
> +                               PPC_STD(dst_reg, 1, stack_local_off);
> +                               PPC_ADDI(b2p[TMP_REG_1], 1, stack_local_off);
> +                               PPC_LDBRX(dst_reg, 0, b2p[TMP_REG_1]);
> +                               break;
> +                       }
> +                       break;
> +
> +emit_clear:
> +                       switch (imm) {
> +                       case 16:
> +                               /* zero-extend 16 bits into 64 bits */
> +                               PPC_RLDICL(dst_reg, dst_reg, 0, 48);
> +                               break;
> +                       case 32:
> +                               /* zero-extend 32 bits into 64 bits */
> +                               PPC_RLDICL(dst_reg, dst_reg, 0, 32);
> +                               break;
> +                       case 64:
> +                               /* nop */
> +                               break;
> +                       }
> +                       break;
> +
> +               /*
> +                * BPF_ST(X)
> +                */
> +               case BPF_STX | BPF_MEM | BPF_B: /* *(u8 *)(dst + off) = src */
> +               case BPF_ST | BPF_MEM | BPF_B: /* *(u8 *)(dst + off) = imm */
> +                       if (BPF_CLASS(code) == BPF_ST) {
> +                               PPC_LI(b2p[TMP_REG_1], imm);
> +                               src_reg = b2p[TMP_REG_1];
> +                       }
> +                       PPC_STB(src_reg, dst_reg, off);
> +                       break;
> +               case BPF_STX | BPF_MEM | BPF_H: /* (u16 *)(dst + off) = src */
> +               case BPF_ST | BPF_MEM | BPF_H: /* (u16 *)(dst + off) = imm */
> +                       if (BPF_CLASS(code) == BPF_ST) {
> +                               PPC_LI(b2p[TMP_REG_1], imm);
> +                               src_reg = b2p[TMP_REG_1];
> +                       }
> +                       PPC_STH(src_reg, dst_reg, off);
> +                       break;
> +               case BPF_STX | BPF_MEM | BPF_W: /* *(u32 *)(dst + off) = src */
> +               case BPF_ST | BPF_MEM | BPF_W: /* *(u32 *)(dst + off) = imm */
> +                       if (BPF_CLASS(code) == BPF_ST) {
> +                               PPC_LI32(b2p[TMP_REG_1], imm);
> +                               src_reg = b2p[TMP_REG_1];
> +                       }
> +                       PPC_STW(src_reg, dst_reg, off);
> +                       break;
> +               case BPF_STX | BPF_MEM | BPF_DW: /* (u64 *)(dst + off) = src */
> +               case BPF_ST | BPF_MEM | BPF_DW: /* *(u64 *)(dst + off) = imm */
> +                       if (BPF_CLASS(code) == BPF_ST) {
> +                               PPC_LI32(b2p[TMP_REG_1], imm);
> +                               src_reg = b2p[TMP_REG_1];
> +                       }
> +                       PPC_STD(src_reg, dst_reg, off);
> +                       break;
> +
> +               /*
> +                * BPF_STX XADD (atomic_add)
> +                */
> +               /* *(u32 *)(dst + off) += src */
> +               case BPF_STX | BPF_XADD | BPF_W:
> +                       /* Get EA into TMP_REG_1 */
> +                       PPC_ADDI(b2p[TMP_REG_1], dst_reg, off);
> +                       /* error if EA is not word-aligned */
> +                       PPC_ANDI(b2p[TMP_REG_2], b2p[TMP_REG_1], 0x03);
> +                       PPC_BCC_SHORT(COND_EQ, (ctx->idx * 4) + 12);
> +                       PPC_LI(b2p[BPF_REG_0], 0);
> +                       PPC_JMP(exit_addr);
> +                       /* load value from memory into TMP_REG_2 */
> +                       PPC_BPF_LWARX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1], 0);
> +                       /* add value from src_reg into this */
> +                       PPC_ADD(b2p[TMP_REG_2], b2p[TMP_REG_2], src_reg);
> +                       /* store result back */
> +                       PPC_BPF_STWCX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1]);
> +                       /* we're done if this succeeded */
> +                       PPC_BCC_SHORT(COND_EQ, (ctx->idx * 4) + (7*4));
> +                       /* otherwise, let's try once more */
> +                       PPC_BPF_LWARX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1], 0);
> +                       PPC_ADD(b2p[TMP_REG_2], b2p[TMP_REG_2], src_reg);
> +                       PPC_BPF_STWCX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1]);
> +                       /* exit if the store was not successful */
> +                       PPC_LI(b2p[BPF_REG_0], 0);
> +                       PPC_BCC(COND_NE, exit_addr);
> +                       break;
> +               /* *(u64 *)(dst + off) += src */
> +               case BPF_STX | BPF_XADD | BPF_DW:
> +                       PPC_ADDI(b2p[TMP_REG_1], dst_reg, off);
> +                       /* error if EA is not doubleword-aligned */
> +                       PPC_ANDI(b2p[TMP_REG_2], b2p[TMP_REG_1], 0x07);
> +                       PPC_BCC_SHORT(COND_EQ, (ctx->idx * 4) + (3*4));
> +                       PPC_LI(b2p[BPF_REG_0], 0);
> +                       PPC_JMP(exit_addr);
> +                       PPC_BPF_LDARX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1], 0);
> +                       PPC_ADD(b2p[TMP_REG_2], b2p[TMP_REG_2], src_reg);
> +                       PPC_BPF_STDCX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1]);
> +                       PPC_BCC_SHORT(COND_EQ, (ctx->idx * 4) + (7*4));
> +                       PPC_BPF_LDARX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1], 0);
> +                       PPC_ADD(b2p[TMP_REG_2], b2p[TMP_REG_2], src_reg);
> +                       PPC_BPF_STDCX(b2p[TMP_REG_2], 0, b2p[TMP_REG_1]);
> +                       PPC_LI(b2p[BPF_REG_0], 0);
> +                       PPC_BCC(COND_NE, exit_addr);
> +                       break;
> +
> +               /*
> +                * BPF_LDX
> +                */
> +               /* dst = *(u8 *)(ul) (src + off) */
> +               case BPF_LDX | BPF_MEM | BPF_B:
> +                       PPC_LBZ(dst_reg, src_reg, off);
> +                       break;
> +               /* dst = *(u16 *)(ul) (src + off) */
> +               case BPF_LDX | BPF_MEM | BPF_H:
> +                       PPC_LHZ(dst_reg, src_reg, off);
> +                       break;
> +               /* dst = *(u32 *)(ul) (src + off) */
> +               case BPF_LDX | BPF_MEM | BPF_W:
> +                       PPC_LWZ(dst_reg, src_reg, off);
> +                       break;
> +               /* dst = *(u64 *)(ul) (src + off) */
> +               case BPF_LDX | BPF_MEM | BPF_DW:
> +                       PPC_LD(dst_reg, src_reg, off);
> +                       break;
> +
> +               /*
> +                * Doubleword load
> +                * 16 byte instruction that uses two 'struct bpf_insn'
> +                */
> +               case BPF_LD | BPF_IMM | BPF_DW: /* dst = (u64) imm */
> +                       imm64 = ((u64)(u32) insn[i].imm) |
> +                                   (((u64)(u32) insn[i+1].imm) << 32);
> +                       /* Adjust for two bpf instructions */
> +                       addrs[++i] = ctx->idx * 4;
> +                       PPC_LI64(dst_reg, imm64);
> +                       break;
> +
> +               /*
> +                * Return/Exit
> +                */
> +               case BPF_JMP | BPF_EXIT:
> +                       /*
> +                        * If this isn't the very last instruction, branch to
> +                        * the epilogue. If we _are_ the last instruction,
> +                        * we'll just fall through to the epilogue.
> +                        */
> +                       if (i != flen - 1)
> +                               PPC_JMP(exit_addr);
> +                       /* else fall through to the epilogue */
> +                       break;
> +
> +               /*
> +                * Call kernel helper
> +                */
> +               case BPF_JMP | BPF_CALL:
> +                       ctx->seen |= SEEN_FUNC;
> +                       func = (u8 *) __bpf_call_base + imm;
> +
> +                       /* Save skb pointer if we need to re-cache skb data */
> +                       if (bpf_helper_changes_skb_data(func))
> +                               PPC_BPF_STL(3, 1, STACK_FRAME_MIN_SIZE);
> +
> +                       bpf_jit_emit_func_call(image, ctx, (u64)func);
> +
> +                       /* move return value from r3 to BPF_REG_0 */
> +                       PPC_MR(b2p[BPF_REG_0], 3);
> +
> +                       /* refresh skb cache */
> +                       if (bpf_helper_changes_skb_data(func)) {
> +                               /* reload skb pointer to r3 */
> +                               PPC_BPF_LL(3, 1, STACK_FRAME_MIN_SIZE);
> +                               bpf_jit_emit_skb_loads(image, ctx);
> +                       }
> +                       break;
> +
> +               /*
> +                * Jumps and branches
> +                */
> +               case BPF_JMP | BPF_JA:
> +                       PPC_JMP(addrs[i + 1 + off]);
> +                       break;
> +
> +               case BPF_JMP | BPF_JGT | BPF_K:
> +               case BPF_JMP | BPF_JGT | BPF_X:
> +               case BPF_JMP | BPF_JSGT | BPF_K:
> +               case BPF_JMP | BPF_JSGT | BPF_X:
> +                       true_cond = COND_GT;
> +                       goto cond_branch;
> +               case BPF_JMP | BPF_JGE | BPF_K:
> +               case BPF_JMP | BPF_JGE | BPF_X:
> +               case BPF_JMP | BPF_JSGE | BPF_K:
> +               case BPF_JMP | BPF_JSGE | BPF_X:
> +                       true_cond = COND_GE;
> +                       goto cond_branch;
> +               case BPF_JMP | BPF_JEQ | BPF_K:
> +               case BPF_JMP | BPF_JEQ | BPF_X:
> +                       true_cond = COND_EQ;
> +                       goto cond_branch;
> +               case BPF_JMP | BPF_JNE | BPF_K:
> +               case BPF_JMP | BPF_JNE | BPF_X:
> +                       true_cond = COND_NE;
> +                       goto cond_branch;
> +               case BPF_JMP | BPF_JSET | BPF_K:
> +               case BPF_JMP | BPF_JSET | BPF_X:
> +                       true_cond = COND_NE;
> +                       /* Fall through */
> +
> +cond_branch:
> +                       switch (code) {
> +                       case BPF_JMP | BPF_JGT | BPF_X:
> +                       case BPF_JMP | BPF_JGE | BPF_X:
> +                       case BPF_JMP | BPF_JEQ | BPF_X:
> +                       case BPF_JMP | BPF_JNE | BPF_X:
> +                               /* unsigned comparison */
> +                               PPC_CMPLD(dst_reg, src_reg);
> +                               break;
> +                       case BPF_JMP | BPF_JSGT | BPF_X:
> +                       case BPF_JMP | BPF_JSGE | BPF_X:
> +                               /* signed comparison */
> +                               PPC_CMPD(dst_reg, src_reg);
> +                               break;
> +                       case BPF_JMP | BPF_JSET | BPF_X:
> +                               PPC_AND_DOT(b2p[TMP_REG_1], dst_reg, src_reg);
> +                               break;
> +                       case BPF_JMP | BPF_JNE | BPF_K:
> +                       case BPF_JMP | BPF_JEQ | BPF_K:
> +                       case BPF_JMP | BPF_JGT | BPF_K:
> +                       case BPF_JMP | BPF_JGE | BPF_K:
> +                               /*
> +                                * Need sign-extended load, so only positive
> +                                * values can be used as imm in cmpldi
> +                                */
> +                               if (imm >= 0 && imm < 32768)
> +                                       PPC_CMPLDI(dst_reg, imm);
> +                               else {
> +                                       /* sign-extending load */
> +                                       PPC_LI32(b2p[TMP_REG_1], imm);
> +                                       /* ... but unsigned comparison */
> +                                       PPC_CMPLD(dst_reg, b2p[TMP_REG_1]);
> +                               }
> +                               break;
> +                       case BPF_JMP | BPF_JSGT | BPF_K:
> +                       case BPF_JMP | BPF_JSGE | BPF_K:
> +                               /*
> +                                * signed comparison, so any 16-bit value
> +                                * can be used in cmpdi
> +                                */
> +                               if (imm >= -32768 && imm < 32768)
> +                                       PPC_CMPDI(dst_reg, imm);
> +                               else {
> +                                       PPC_LI32(b2p[TMP_REG_1], imm);
> +                                       PPC_CMPD(dst_reg, b2p[TMP_REG_1]);
> +                               }
> +                               break;
> +                       case BPF_JMP | BPF_JSET | BPF_K:
> +                               /* andi does not sign-extend the immediate */
> +                               if (imm >= 0 && imm < 32768)
> +                                       /* PPC_ANDI is _only/always_ dot-form */
> +                                       PPC_ANDI(b2p[TMP_REG_1], dst_reg, imm);
> +                               else {
> +                                       PPC_LI32(b2p[TMP_REG_1], imm);
> +                                       PPC_AND_DOT(b2p[TMP_REG_1], dst_reg,
> +                                                   b2p[TMP_REG_1]);
> +                               }
> +                               break;
> +                       }
> +                       PPC_BCC(true_cond, addrs[i + 1 + off]);
> +                       break;
> +
> +               /*
> +                * Loads from packet header/data
> +                * Assume 32-bit input value in imm and X (src_reg)
> +                */
> +
> +               /* Absolute loads */
> +               case BPF_LD | BPF_W | BPF_ABS:
> +                       func = (u8 *)CHOOSE_LOAD_FUNC(imm, sk_load_word);
> +                       goto common_load_abs;
> +               case BPF_LD | BPF_H | BPF_ABS:
> +                       func = (u8 *)CHOOSE_LOAD_FUNC(imm, sk_load_half);
> +                       goto common_load_abs;
> +               case BPF_LD | BPF_B | BPF_ABS:
> +                       func = (u8 *)CHOOSE_LOAD_FUNC(imm, sk_load_byte);
> +common_load_abs:
> +                       /*
> +                        * Load from [imm]
> +                        * Load into r4, which can just be passed onto
> +                        *  skb load helpers as the second parameter
> +                        */
> +                       PPC_LI32(4, imm);
> +                       goto common_load;
> +
> +               /* Indirect loads */
> +               case BPF_LD | BPF_W | BPF_IND:
> +                       func = (u8 *)sk_load_word;
> +                       goto common_load_ind;
> +               case BPF_LD | BPF_H | BPF_IND:
> +                       func = (u8 *)sk_load_half;
> +                       goto common_load_ind;
> +               case BPF_LD | BPF_B | BPF_IND:
> +                       func = (u8 *)sk_load_byte;
> +common_load_ind:
> +                       /*
> +                        * Load from [src_reg + imm]
> +                        * Treat src_reg as a 32-bit value
> +                        */
> +                       PPC_EXTSW(4, src_reg);
> +                       if (imm) {
> +                               if (imm >= -32768 && imm < 32768)
> +                                       PPC_ADDI(4, 4, IMM_L(imm));
> +                               else {
> +                                       PPC_LI32(b2p[TMP_REG_1], imm);
> +                                       PPC_ADD(4, 4, b2p[TMP_REG_1]);
> +                               }
> +                       }
> +
> +common_load:
> +                       ctx->seen |= SEEN_SKB;
> +                       ctx->seen |= SEEN_FUNC;
> +                       bpf_jit_emit_func_call(image, ctx, (u64)func);
> +
> +                       /*
> +                        * Helper returns 'lt' condition on error, and an
> +                        * appropriate return value in BPF_REG_0
> +                        */
> +                       PPC_BCC(COND_LT, exit_addr);
> +                       break;
> +
> +               /*
> +                * TODO: Tail call
> +                */
> +               case BPF_JMP | BPF_CALL | BPF_X:
> +
> +               default:
> +                       /*
> +                        * The filter contains something cruel & unusual.
> +                        * We don't handle it, but also there shouldn't be
> +                        * anything missing from our list.
> +                        */
> +                       pr_err_ratelimited("eBPF filter opcode %04x (@%d) unsupported\n",
> +                                       code, i);
> +                       return -ENOTSUPP;
> +               }
> +       }
> +
> +       /* Set end-of-body-code address for exit. */
> +       addrs[i] = ctx->idx * 4;
> +
> +       return 0;
> +}
> +
> +void bpf_jit_compile(struct bpf_prog *fp) { }
> +
> +struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
> +{
> +       u32 proglen;
> +       u32 alloclen;
> +       u32 *image = NULL;
> +       u32 *code_base;
> +       u32 *addrs;
> +       struct codegen_context cgctx;
> +       int pass;
> +       int flen;
> +       struct bpf_binary_header *bpf_hdr;
> +
> +       if (!bpf_jit_enable)
> +               return fp;
> +
> +       if (!fp || !fp->len)
> +               return fp;
> +
> +       flen = fp->len;
> +       addrs = kzalloc((flen+1) * sizeof(*addrs), GFP_KERNEL);
> +       if (addrs == NULL)
> +               return fp;
> +
> +       cgctx.idx = 0;
> +       cgctx.seen = 0;
> +       /* Scouting faux-generate pass 0 */
> +       if (bpf_jit_build_body(fp, 0, &cgctx, addrs))
> +               /* We hit something illegal or unsupported. */
> +               goto out;
> +
> +       /*
> +        * Pretend to build prologue, given the features we've seen.  This will
> +        * update ctgtx.idx as it pretends to output instructions, then we can
> +        * calculate total size from idx.
> +        */
> +       bpf_jit_build_prologue(0, &cgctx);
> +       bpf_jit_build_epilogue(0, &cgctx);
> +
> +       proglen = cgctx.idx * 4;
> +       alloclen = proglen + FUNCTION_DESCR_SIZE;
> +
> +       bpf_hdr = bpf_jit_binary_alloc(alloclen, (u8 **)&image, 4,
> +                       bpf_jit_fill_ill_insns);
> +       if (!bpf_hdr)
> +               goto out;
> +
> +       code_base = image + (FUNCTION_DESCR_SIZE/4);
> +
> +       /* Code generation passes 1-2 */
> +       for (pass = 1; pass < 3; pass++) {
> +               /* Now build the prologue, body code & epilogue for real. */
> +               cgctx.idx = 0;
> +               bpf_jit_build_prologue(code_base, &cgctx);
> +               bpf_jit_build_body(fp, code_base, &cgctx, addrs);
> +               bpf_jit_build_epilogue(code_base, &cgctx);
> +
> +               if (bpf_jit_enable > 1)
> +                       pr_info("Pass %d: shrink = %d, seen = 0x%x\n", pass,
> +                               proglen - (cgctx.idx * 4), cgctx.seen);
> +       }
> +
> +       if (bpf_jit_enable > 1)
> +               /*
> +                * Note that we output the base address of the code_base
> +                * rather than image, since opcodes are in code_base.
> +                */
> +               bpf_jit_dump(flen, proglen, pass, code_base);
> +
> +       if (image) {
> +               bpf_flush_icache(bpf_hdr, image + alloclen);
> +#ifdef PPC64_ELF_ABI_v1
> +               /* Function descriptor nastiness: Address + TOC */
> +               ((u64 *)image)[0] = (u64)code_base;
> +               ((u64 *)image)[1] = local_paca->kernel_toc;
> +#endif
> +               fp->bpf_func = (void *)image;
> +               fp->jited = 1;
> +       }
> +
> +out:
> +       kfree(addrs);
> +       return fp;
> +}
> +
> +void bpf_jit_free(struct bpf_prog *fp)
> +{
> +       unsigned long addr = (unsigned long)fp->bpf_func & PAGE_MASK;
> +       struct bpf_binary_header *bpf_hdr = (void *)addr;
> +
> +       if (fp->jited)
> +               bpf_jit_binary_free(bpf_hdr);
> +
> +       bpf_prog_unlock_free(fp);
> +}
> --
> 2.8.2
>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-09  3:19   ` Nilay Vaish
@ 2016-06-09  6:07     ` Naveen N. Rao
  0 siblings, 0 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-09  6:07 UTC (permalink / raw)
  To: Nilay Vaish
  Cc: Daniel Borkmann, Matt Evans, Alexei Starovoitov, netdev,
	Denis Kirjanov, Linux Kernel list, Paul Mackerras, linuxppc-dev,
	David S. Miller, Ananth N Mavinakayanahalli

On 2016/06/08 10:19PM, Nilay Vaish wrote:
> Naveen, can you point out where in the patch you update the variable:
> idx, a member of codegen_contex structure?  Somehow I am unable to
> figure it out.  I can only see that we set it to 0 in the
> bpf_int_jit_compile function.  Since all your test cases pass, I am
> clearly overlooking something.

Yes, that's being done in bpf_jit.h (see the earlier patches in the 
series). All the PPC_*() instruction macros are defined to EMIT() the 
respective powerpc instruction encoding.  EMIT() translates to 
PLANT_INSTR(), which actually increments idx.

- Naveen

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH 0/6] eBPF JIT for PPC64
  2016-06-07 13:32 [PATCH 0/6] eBPF JIT for PPC64 Naveen N. Rao
                   ` (5 preceding siblings ...)
  2016-06-07 13:32 ` [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF Naveen N. Rao
@ 2016-06-11  5:47 ` David Miller
  2016-06-13  5:40   ` Naveen N. Rao
  6 siblings, 1 reply; 34+ messages in thread
From: David Miller @ 2016-06-11  5:47 UTC (permalink / raw)
  To: naveen.n.rao
  Cc: linux-kernel, linuxppc-dev, netdev, mpe, matt, kda, paulus, ast,
	daniel, ananth

From: "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>
Date: Tue,  7 Jun 2016 19:02:17 +0530

> Please note that patch [2] is a pre-requisite for this patchset, and is
> not yet upstream.
 ...
> [1] http://thread.gmane.org/gmane.linux.kernel/2188694
> [2] http://thread.gmane.org/gmane.linux.ports.ppc.embedded/96514

Because of #2 I don't think I can take this directly into the networking
tree, right?

Therefore, how would you like this to be merged?

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH 0/6] eBPF JIT for PPC64
  2016-06-11  5:47 ` [PATCH 0/6] eBPF JIT for PPC64 David Miller
@ 2016-06-13  5:40   ` Naveen N. Rao
  2016-06-18  4:57     ` mpe
  0 siblings, 1 reply; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-13  5:40 UTC (permalink / raw)
  To: David Miller, Michael Ellerman
  Cc: linux-kernel, linuxppc-dev, netdev, matt, kda, paulus, ast,
	daniel, ananth

On 2016/06/10 10:47PM, David Miller wrote:
> From: "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>
> Date: Tue,  7 Jun 2016 19:02:17 +0530
> 
> > Please note that patch [2] is a pre-requisite for this patchset, and is
> > not yet upstream.
>  ...
> > [1] http://thread.gmane.org/gmane.linux.kernel/2188694
> > [2] http://thread.gmane.org/gmane.linux.ports.ppc.embedded/96514
> 
> Because of #2 I don't think I can take this directly into the networking
> tree, right?
> 
> Therefore, how would you like this to be merged?

Hi David,
Thanks for asking. Yes, I think it is better to take this through the 
powerpc tree as all the changes are contained within arch/powerpc, 
unless Michael Ellerman feels differently.

Michael?


Regards,
Naveen

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-07 13:32 ` [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF Naveen N. Rao
  2016-06-07 22:56   ` Alexei Starovoitov
  2016-06-09  3:19   ` Nilay Vaish
@ 2016-06-17 12:53   ` Michael Ellerman
  2016-06-17 13:00     ` [PATCH] ppc: Fix BPF JIT for ABIv2 Thadeu Lima de Souza Cascardo
  2016-06-19 17:36     ` [6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF Naveen N. Rao
  2016-06-22 10:37   ` [PATCH 6/6] " Michael Ellerman
  3 siblings, 2 replies; 34+ messages in thread
From: Michael Ellerman @ 2016-06-17 12:53 UTC (permalink / raw)
  To: Naveen N. Rao, linux-kernel, linuxppc-dev, netdev
  Cc: Matt Evans, Daniel Borkmann, Alexei Starovoitov, Denis Kirjanov,
	Paul Mackerras, David S. Miller, Ananth N Mavinakayanahalli

On Tue, 2016-07-06 at 13:32:23 UTC, "Naveen N. Rao" wrote:
> diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
> new file mode 100644
> index 0000000..954ff53
> --- /dev/null
> +++ b/arch/powerpc/net/bpf_jit_comp64.c
> @@ -0,0 +1,956 @@
...
> +
> +static void bpf_jit_fill_ill_insns(void *area, unsigned int size)
> +{
> +	int *p = area;
> +
> +	/* Fill whole space with trap instructions */
> +	while (p < (int *)((char *)area + size))
> +		*p++ = BREAKPOINT_INSTRUCTION;
> +}

This breaks the build for some configs, presumably you're missing a header:

  arch/powerpc/net/bpf_jit_comp64.c:30:10: error: 'BREAKPOINT_INSTRUCTION' undeclared (first use in this function)

http://kisskb.ellerman.id.au/kisskb/buildresult/12720611/

cheers

^ permalink raw reply	[flat|nested] 34+ messages in thread

* [PATCH] ppc: Fix BPF JIT for ABIv2
  2016-06-17 12:53   ` [6/6] " Michael Ellerman
@ 2016-06-17 13:00     ` Thadeu Lima de Souza Cascardo
  2016-06-19 17:49       ` Naveen N. Rao
  2016-06-22  5:18       ` Michael Ellerman
  2016-06-19 17:36     ` [6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF Naveen N. Rao
  1 sibling, 2 replies; 34+ messages in thread
From: Thadeu Lima de Souza Cascardo @ 2016-06-17 13:00 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: Naveen N. Rao, linux-kernel, linuxppc-dev, netdev, Matt Evans,
	Daniel Borkmann, Alexei Starovoitov, Denis Kirjanov,
	Paul Mackerras, David S. Miller, Ananth N Mavinakayanahalli

On Fri, Jun 17, 2016 at 10:53:21PM +1000, Michael Ellerman wrote:
> On Tue, 2016-07-06 at 13:32:23 UTC, "Naveen N. Rao" wrote:
> > diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
> > new file mode 100644
> > index 0000000..954ff53
> > --- /dev/null
> > +++ b/arch/powerpc/net/bpf_jit_comp64.c
> > @@ -0,0 +1,956 @@
> ...
> > +
> > +static void bpf_jit_fill_ill_insns(void *area, unsigned int size)
> > +{
> > +	int *p = area;
> > +
> > +	/* Fill whole space with trap instructions */
> > +	while (p < (int *)((char *)area + size))
> > +		*p++ = BREAKPOINT_INSTRUCTION;
> > +}
> 
> This breaks the build for some configs, presumably you're missing a header:
> 
>   arch/powerpc/net/bpf_jit_comp64.c:30:10: error: 'BREAKPOINT_INSTRUCTION' undeclared (first use in this function)
> 
> http://kisskb.ellerman.id.au/kisskb/buildresult/12720611/
> 
> cheers

Hi, Michael and Naveen.

I noticed independently that there is a problem with BPF JIT and ABIv2, and
worked out the patch below before I noticed Naveen's patchset and the latest
changes in ppc tree for a better way to check for ABI versions.

However, since the issue described below affect mainline and stable kernels,
would you consider applying it before merging your two patchsets, so that we can
more easily backport the fix?

Thanks.
Cascardo.

---
>From a984dc02b6317a1d3a3c2302385adba5227be5bd Mon Sep 17 00:00:00 2001
From: Thadeu Lima de Souza Cascardo <cascardo@redhat.com>
Date: Wed, 15 Jun 2016 13:22:12 -0300
Subject: [PATCH] ppc: Fix BPF JIT for ABIv2

ABIv2 used for ppc64le does not use function descriptors. Without this patch,
whenever BPF JIT is enabled, we get a crash as below.

[root@ibm-p8-kvm-05-guest-02 ~]# echo 2 > /proc/sys/net/core/bpf_jit_enable
[root@ibm-p8-kvm-05-guest-02 ~]# tcpdump -n -i eth0 tcp port 22
device eth0 entered promiscuous mode
Pass 1: shrink = 0, seen = 0x0
Pass 2: shrink = 0, seen = 0x0
flen=1 proglen=8 pass=3 image=d000000005bb9018 from=tcpdump pid=11387
JIT code: 00000000: 00 00 60 38 20 00 80 4e
Pass 1: shrink = 0, seen = 0x30000
Pass 2: shrink = 0, seen = 0x30000
flen=20 proglen=524 pass=3 image=d000000005bbd018 from=tcpdump pid=11387
JIT code: 00000000: a6 02 08 7c 10 00 01 f8 70 ff c1 f9 78 ff e1 f9
JIT code: 00000010: e1 fe 21 f8 7c 00 e3 80 78 00 e3 81 50 78 e7 7d
JIT code: 00000020: c8 00 c3 e9 00 00 a0 38 00 c0 e0 3c c6 07 e7 78
JIT code: 00000030: 08 00 e7 64 54 1b e7 60 a6 03 e8 7c 0c 00 c0 38
JIT code: 00000040: 21 00 80 4e b0 01 80 41 00 00 00 60 dd 86 e0 38
JIT code: 00000050: 01 00 e7 3c 40 38 04 7c 9c 00 82 40 00 00 00 60
JIT code: 00000060: 00 c0 e0 3c c6 07 e7 78 08 00 e7 64 70 1b e7 60
JIT code: 00000070: a6 03 e8 7c 14 00 c0 38 21 00 80 4e 78 01 80 41
JIT code: 00000080: 00 00 00 60 06 00 04 28 68 01 82 40 00 00 00 60
JIT code: 00000090: 00 c0 e0 3c c6 07 e7 78 08 00 e7 64 54 1b e7 60
JIT code: 000000a0: a6 03 e8 7c 36 00 c0 38 21 00 80 4e 48 01 80 41
JIT code: 000000b0: 00 00 00 60 16 00 04 28 2c 01 82 41 00 00 00 60
JIT code: 000000c0: 00 c0 e0 3c c6 07 e7 78 08 00 e7 64 54 1b e7 60
JIT code: 000000d0: a6 03 e8 7c 38 00 c0 38 21 00 80 4e 18 01 80 41
JIT code: 000000e0: 00 00 00 60 16 00 04 28 fc 00 82 41 00 00 00 60
JIT code: 000000f0: 00 01 00 48 00 08 04 28 f8 00 82 40 00 00 00 60
JIT code: 00000100: 00 c0 e0 3c c6 07 e7 78 08 00 e7 64 70 1b e7 60
JIT code: 00000110: a6 03 e8 7c 17 00 c0 38 21 00 80 4e d8 00 80 41
JIT code: 00000120: 00 00 00 60 06 00 04 28 c8 00 82 40 00 00 00 60
JIT code: 00000130: 00 c0 e0 3c c6 07 e7 78 08 00 e7 64 54 1b e7 60
JIT code: 00000140: a6 03 e8 7c 14 00 c0 38 21 00 80 4e a8 00 80 41
JIT code: 00000150: 00 00 00 60 ff 1f 87 70 98 00 82 40 00 00 00 60
JIT code: 00000160: 00 c0 e0 3c c6 07 e7 78 08 00 e7 64 88 1b e7 60
JIT code: 00000170: a6 03 e8 7c 0e 00 c0 38 21 00 80 4e 78 00 80 41
JIT code: 00000180: 00 00 00 60 00 c0 e0 3c c6 07 e7 78 08 00 e7 64
JIT code: 00000190: 4c 1b e7 60 a6 03 e8 7c 0e 00 c5 38 21 00 80 4e
JIT code: 000001a0: 54 00 80 41 00 00 00 60 16 00 04 28 38 00 82 41
JIT code: 000001b0: 00 00 00 60 00 c0 e0 3c c6 07 e7 78 08 00 e7 64
JIT code: 000001c0: 4c 1b e7 60 a6 03 e8 7c 10 00 c5 38 21 00 80 4e
JIT code: 000001d0: 24 00 80 41 00 00 00 60 16 00 04 28 14 00 82 40
JIT code: 000001e0: 00 00 00 60 ff ff 60 38 01 00 63 3c 08 00 00 48
JIT code: 000001f0: 00 00 60 38 20 01 21 38 10 00 01 e8 a6 03 08 7c
JIT code: 00000200: 70 ff c1 e9 78 ff e1 e9 20 00 80 4e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
Oops: Exception in kernel mode, sig: 4 [#1]
SMP NR_CPUS=32 NUMA pSeries
Modules linked in: virtio_balloon nfsd ip_tables x_tables autofs4 xfs libcrc32c virtio_console virtio_net virtio_pci virtio_ring virtio
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.7.0-rc3-00009-gdb06d75 #1
task: c0000004a9254500 ti: c0000004bffe4000 task.ti: c0000004a9260000
NIP: d000000005bbd000 LR: c0000000008bcad8 CTR: d000000005bbd000
REGS: c0000004bffe76d0 TRAP: 0700   Not tainted  (4.7.0-rc3-00009-gdb06d75)
MSR: 8000000000089033 <SF,EE,ME,IR,DR,RI,LE>  CR: 28002028  XER: 00000000
CFAR: c0000000008bcad4 SOFTE: 1
GPR00: c0000000008c18dc c0000004bffe7950 c000000000f19900 c0000004a9d98600
GPR04: d000000005bbb028 000000000000005c c000000002977000 000000000000004e
GPR08: 000000000000005c d000000005bbd000 0000000000000000 0000000000000700
GPR12: d000000005bbd000 c00000000fff8400 c000000000f21c38 f0000000104a3ac0
GPR16: 0000000000000000 c0000004a5b7e808 c000000002977831 c000000002977834
GPR20: c000000000f23040 0000000000000001 000000000000dd86 0000000000000000
GPR24: ffffffff80000000 000000000000005c c000000002977000 000000000000004e
GPR28: c0000004a5ede28e c000000002977000 c000000002848000 c0000004a9d98600
NIP [d000000005bbd000] 0xd000000005bbd000
LR [c0000000008bcad8] run_filter+0x48/0x90
Call Trace:
[c0000004bffe7950] [c0000004bffe7980] 0xc0000004bffe7980 (unreliable)
[c0000004bffe7970] [c0000000008c18dc] tpacket_rcv+0x13c/0x9d0
[c0000004bffe7a60] [c0000000007f060c] __netif_receive_skb_core+0x22c/0xb00
[c0000004bffe7b30] [c0000000007f3cfc] netif_receive_skb_internal+0x2c/0xc0
[c0000004bffe7b70] [c0000000007f524c] napi_gro_receive+0x11c/0x220
[c0000004bffe7bb0] [d0000000056e252c] virtnet_receive+0x2fc/0x940 [virtio_net]
[c0000004bffe7cc0] [d0000000056e2ba0] virtnet_poll+0x30/0xf0 [virtio_net]
[c0000004bffe7d00] [c0000000007f4878] net_rx_action+0x2d8/0x450
[c0000004bffe7e10] [c0000000000ae464] __do_softirq+0x164/0x3d0
[c0000004bffe7f00] [c0000000000aea68] irq_exit+0xc8/0x100
[c0000004bffe7f20] [c000000000010a98] __do_irq+0x98/0x200
[c0000004bffe7f90] [c000000000022274] call_do_irq+0x14/0x24
[c0000004a9263a00] [c000000000010c94] do_IRQ+0x94/0x110
[c0000004a9263a50] [c0000000000025d8] hardware_interrupt_common+0x158/0x180
--- interrupt: 501 at plpar_hcall_norets+0x1c/0x28
    LR = check_and_cede_processor+0x2c/0x40
[c0000004a9263d40] [c0000004a9263d90] 0xc0000004a9263d90 (unreliable)
[c0000004a9263da0] [c000000000799b90] shared_cede_loop+0x60/0x170
[c0000004a9263de0] [c0000000007974b4] cpuidle_enter_state+0xb4/0x380
[c0000004a9263e40] [c0000000000f5f84] call_cpuidle+0x44/0x80
[c0000004a9263e60] [c0000000000f64b0] cpu_startup_entry+0x380/0x450
[c0000004a9263f20] [c00000000003de94] start_secondary+0x304/0x350
[c0000004a9263f90] [c000000000008b6c] start_secondary_prolog+0x10/0x14
Instruction dump:
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
---[ end trace cc80130850d8e991 ]---

Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@redhat.com>
---
 arch/powerpc/net/bpf_jit.h      | 2 +-
 arch/powerpc/net/bpf_jit_comp.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h
index 889fd19..28b89ed 100644
--- a/arch/powerpc/net/bpf_jit.h
+++ b/arch/powerpc/net/bpf_jit.h
@@ -70,7 +70,7 @@ DECLARE_LOAD_FUNC(sk_load_half);
 DECLARE_LOAD_FUNC(sk_load_byte);
 DECLARE_LOAD_FUNC(sk_load_byte_msh);
 
-#ifdef CONFIG_PPC64
+#if defined(CONFIG_PPC64) && (!defined(_CALL_ELF) || _CALL_ELF != 2)
 #define FUNCTION_DESCR_SIZE	24
 #else
 #define FUNCTION_DESCR_SIZE	0
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 2d66a84..035b887 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -664,7 +664,7 @@ void bpf_jit_compile(struct bpf_prog *fp)
 
 	if (image) {
 		bpf_flush_icache(code_base, code_base + (proglen/4));
-#ifdef CONFIG_PPC64
+#if defined(CONFIG_PPC64) && (!defined(_CALL_ELF) || _CALL_ELF != 2)
 		/* Function descriptor nastiness: Address + TOC */
 		((u64 *)image)[0] = (u64)code_base;
 		((u64 *)image)[1] = local_paca->kernel_toc;
-- 
2.5.5

^ permalink raw reply related	[flat|nested] 34+ messages in thread

* Re: [PATCH 0/6] eBPF JIT for PPC64
  2016-06-13  5:40   ` Naveen N. Rao
@ 2016-06-18  4:57     ` mpe
  0 siblings, 0 replies; 34+ messages in thread
From: mpe @ 2016-06-18  4:57 UTC (permalink / raw)
  To: Naveen N. Rao
  Cc: David Miller, linux-kernel, linuxppc-dev, netdev, matt, kda,
	paulus, ast, daniel, ananth

On 2016-06-13 15:40, Naveen N. Rao wrote:
> On 2016/06/10 10:47PM, David Miller wrote:
>> From: "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>
>> Date: Tue,  7 Jun 2016 19:02:17 +0530
>> 
>> > Please note that patch [2] is a pre-requisite for this patchset, and is
>> > not yet upstream.
>>  ...
>> > [1] http://thread.gmane.org/gmane.linux.kernel/2188694
>> > [2] http://thread.gmane.org/gmane.linux.ports.ppc.embedded/96514
>> 
>> Because of #2 I don't think I can take this directly into the 
>> networking
>> tree, right?
>> 
>> Therefore, how would you like this to be merged?
> 
> Hi David,
> Thanks for asking. Yes, I think it is better to take this through the
> powerpc tree as all the changes are contained within arch/powerpc,
> unless Michael Ellerman feels differently.
> 
> Michael?

Yeah I was planning to take it.

I put it in my test tree last night but it broke the build for some 
configs.
Once that is fixed I'll take it via powerpc#next.

cheers

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-17 12:53   ` [6/6] " Michael Ellerman
  2016-06-17 13:00     ` [PATCH] ppc: Fix BPF JIT for ABIv2 Thadeu Lima de Souza Cascardo
@ 2016-06-19 17:36     ` Naveen N. Rao
  2016-06-20 23:38       ` Michael Ellerman
  1 sibling, 1 reply; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-19 17:36 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: linux-kernel, linuxppc-dev, netdev, Matt Evans, Daniel Borkmann,
	Alexei Starovoitov, Denis Kirjanov, Paul Mackerras,
	David S. Miller, Ananth N Mavinakayanahalli

On 2016/06/17 10:53PM, Michael Ellerman wrote:
> On Tue, 2016-07-06 at 13:32:23 UTC, "Naveen N. Rao" wrote:
> > diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
> > new file mode 100644
> > index 0000000..954ff53
> > --- /dev/null
> > +++ b/arch/powerpc/net/bpf_jit_comp64.c
> > @@ -0,0 +1,956 @@
> ...
> > +
> > +static void bpf_jit_fill_ill_insns(void *area, unsigned int size)
> > +{
> > +	int *p = area;
> > +
> > +	/* Fill whole space with trap instructions */
> > +	while (p < (int *)((char *)area + size))
> > +		*p++ = BREAKPOINT_INSTRUCTION;
> > +}
> 
> This breaks the build for some configs, presumably you're missing a header:
> 
>   arch/powerpc/net/bpf_jit_comp64.c:30:10: error: 'BREAKPOINT_INSTRUCTION' undeclared (first use in this function)
> 
> http://kisskb.ellerman.id.au/kisskb/buildresult/12720611/

Oops. Yes, I should have caught that. I need to add:

#include <asm/kprobes.h>

in bpf_jit_comp64.c

Can you please check if it resolves the build error?

Regards,
Naveen

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH] ppc: Fix BPF JIT for ABIv2
  2016-06-17 13:00     ` [PATCH] ppc: Fix BPF JIT for ABIv2 Thadeu Lima de Souza Cascardo
@ 2016-06-19 17:49       ` Naveen N. Rao
  2016-06-20 18:56         ` Thadeu Lima de Souza Cascardo
  2016-06-22  5:18       ` Michael Ellerman
  1 sibling, 1 reply; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-19 17:49 UTC (permalink / raw)
  To: Thadeu Lima de Souza Cascardo
  Cc: Michael Ellerman, linux-kernel, linuxppc-dev, netdev, Matt Evans,
	Daniel Borkmann, Alexei Starovoitov, Denis Kirjanov,
	Paul Mackerras, David S. Miller, Ananth N Mavinakayanahalli

On 2016/06/17 10:00AM, Thadeu Lima de Souza Cascardo wrote:
> On Fri, Jun 17, 2016 at 10:53:21PM +1000, Michael Ellerman wrote:
> > On Tue, 2016-07-06 at 13:32:23 UTC, "Naveen N. Rao" wrote:
> > > diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
> > > new file mode 100644
> > > index 0000000..954ff53
> > > --- /dev/null
> > > +++ b/arch/powerpc/net/bpf_jit_comp64.c
> > > @@ -0,0 +1,956 @@
> > ...
> > > +
> > > +static void bpf_jit_fill_ill_insns(void *area, unsigned int size)
> > > +{
> > > +	int *p = area;
> > > +
> > > +	/* Fill whole space with trap instructions */
> > > +	while (p < (int *)((char *)area + size))
> > > +		*p++ = BREAKPOINT_INSTRUCTION;
> > > +}
> > 
> > This breaks the build for some configs, presumably you're missing a header:
> > 
> >   arch/powerpc/net/bpf_jit_comp64.c:30:10: error: 'BREAKPOINT_INSTRUCTION' undeclared (first use in this function)
> > 
> > http://kisskb.ellerman.id.au/kisskb/buildresult/12720611/
> > 
> > cheers
> 
> Hi, Michael and Naveen.
> 
> I noticed independently that there is a problem with BPF JIT and ABIv2, and
> worked out the patch below before I noticed Naveen's patchset and the latest
> changes in ppc tree for a better way to check for ABI versions.
> 
> However, since the issue described below affect mainline and stable kernels,
> would you consider applying it before merging your two patchsets, so that we can
> more easily backport the fix?

Hi Cascardo,
Given that this has been broken on ABIv2 since forever, I didn't bother 
fixing it. But, I can see why this would be a good thing to have for 
-stable and existing distros. However, while your patch below may fix 
the crash you're seeing on ppc64le, it is not sufficient -- you'll need 
changes in bpf_jit_asm.S as well.

Regards,
Naveen

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH] ppc: Fix BPF JIT for ABIv2
  2016-06-19 17:49       ` Naveen N. Rao
@ 2016-06-20 18:56         ` Thadeu Lima de Souza Cascardo
  2016-06-21  8:58           ` Naveen N. Rao
  0 siblings, 1 reply; 34+ messages in thread
From: Thadeu Lima de Souza Cascardo @ 2016-06-20 18:56 UTC (permalink / raw)
  To: Naveen N. Rao
  Cc: Michael Ellerman, linux-kernel, linuxppc-dev, netdev, Matt Evans,
	Daniel Borkmann, Alexei Starovoitov, Denis Kirjanov,
	Paul Mackerras, David S. Miller, Ananth N Mavinakayanahalli

On Sun, Jun 19, 2016 at 11:19:14PM +0530, Naveen N. Rao wrote:
> On 2016/06/17 10:00AM, Thadeu Lima de Souza Cascardo wrote:
> > On Fri, Jun 17, 2016 at 10:53:21PM +1000, Michael Ellerman wrote:
> > > On Tue, 2016-07-06 at 13:32:23 UTC, "Naveen N. Rao" wrote:
> > > > diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
> > > > new file mode 100644
> > > > index 0000000..954ff53
> > > > --- /dev/null
> > > > +++ b/arch/powerpc/net/bpf_jit_comp64.c
> > > > @@ -0,0 +1,956 @@
> > > ...
> > > > +
> > > > +static void bpf_jit_fill_ill_insns(void *area, unsigned int size)
> > > > +{
> > > > +	int *p = area;
> > > > +
> > > > +	/* Fill whole space with trap instructions */
> > > > +	while (p < (int *)((char *)area + size))
> > > > +		*p++ = BREAKPOINT_INSTRUCTION;
> > > > +}
> > > 
> > > This breaks the build for some configs, presumably you're missing a header:
> > > 
> > >   arch/powerpc/net/bpf_jit_comp64.c:30:10: error: 'BREAKPOINT_INSTRUCTION' undeclared (first use in this function)
> > > 
> > > http://kisskb.ellerman.id.au/kisskb/buildresult/12720611/
> > > 
> > > cheers
> > 
> > Hi, Michael and Naveen.
> > 
> > I noticed independently that there is a problem with BPF JIT and ABIv2, and
> > worked out the patch below before I noticed Naveen's patchset and the latest
> > changes in ppc tree for a better way to check for ABI versions.
> > 
> > However, since the issue described below affect mainline and stable kernels,
> > would you consider applying it before merging your two patchsets, so that we can
> > more easily backport the fix?
> 
> Hi Cascardo,
> Given that this has been broken on ABIv2 since forever, I didn't bother 
> fixing it. But, I can see why this would be a good thing to have for 
> -stable and existing distros. However, while your patch below may fix 
> the crash you're seeing on ppc64le, it is not sufficient -- you'll need 
> changes in bpf_jit_asm.S as well.

Hi, Naveen.

Any tips on how to exercise possible issues there? Or what changes you think
would be sufficient?

I will see what I can find by myself, but would appreciate any help.

Regards.
Cascardo.

> 
> Regards,
> Naveen
> 

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-19 17:36     ` [6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF Naveen N. Rao
@ 2016-06-20 23:38       ` Michael Ellerman
  2016-06-21  6:58         ` Naveen N. Rao
  0 siblings, 1 reply; 34+ messages in thread
From: Michael Ellerman @ 2016-06-20 23:38 UTC (permalink / raw)
  To: Naveen N. Rao
  Cc: linux-kernel, linuxppc-dev, netdev, Matt Evans, Daniel Borkmann,
	Alexei Starovoitov, Denis Kirjanov, Paul Mackerras,
	David S. Miller, Ananth N Mavinakayanahalli

On Sun, 2016-06-19 at 23:06 +0530, Naveen N. Rao wrote:
> On 2016/06/17 10:53PM, Michael Ellerman wrote:
> > On Tue, 2016-07-06 at 13:32:23 UTC, "Naveen N. Rao" wrote:
> > > diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
> > > new file mode 100644
> > > index 0000000..954ff53
> > > --- /dev/null
> > > +++ b/arch/powerpc/net/bpf_jit_comp64.c
> > > @@ -0,0 +1,956 @@
> > ...

> > > +
> > > +static void bpf_jit_fill_ill_insns(void *area, unsigned int size)
> > > +{
> > > +	int *p = area;
> > > +
> > > +	/* Fill whole space with trap instructions */
> > > +	while (p < (int *)((char *)area + size))
> > > +		*p++ = BREAKPOINT_INSTRUCTION;
> > > +}
> > 
> > This breaks the build for some configs, presumably you're missing a header:
> > 
> >   arch/powerpc/net/bpf_jit_comp64.c:30:10: error: 'BREAKPOINT_INSTRUCTION' undeclared (first use in this function)
> > 
> > http://kisskb.ellerman.id.au/kisskb/buildresult/12720611/
> 
> Oops. Yes, I should have caught that. I need to add:
> 
> #include <asm/kprobes.h>
> 
> in bpf_jit_comp64.c
> 
> Can you please check if it resolves the build error?

Can you? :D

cheers

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-20 23:38       ` Michael Ellerman
@ 2016-06-21  6:58         ` Naveen N. Rao
  2016-06-21 11:04           ` Michael Ellerman
  0 siblings, 1 reply; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-21  6:58 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: linux-kernel, linuxppc-dev, netdev, Matt Evans, Daniel Borkmann,
	Alexei Starovoitov, Denis Kirjanov, Paul Mackerras,
	David S. Miller, Ananth N Mavinakayanahalli

On 2016/06/21 09:38AM, Michael Ellerman wrote:
> On Sun, 2016-06-19 at 23:06 +0530, Naveen N. Rao wrote:
> > On 2016/06/17 10:53PM, Michael Ellerman wrote:
> > > On Tue, 2016-07-06 at 13:32:23 UTC, "Naveen N. Rao" wrote:
> > > > diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
> > > > new file mode 100644
> > > > index 0000000..954ff53
> > > > --- /dev/null
> > > > +++ b/arch/powerpc/net/bpf_jit_comp64.c
> > > > @@ -0,0 +1,956 @@
> > > ...
> 
> > > > +
> > > > +static void bpf_jit_fill_ill_insns(void *area, unsigned int size)
> > > > +{
> > > > +	int *p = area;
> > > > +
> > > > +	/* Fill whole space with trap instructions */
> > > > +	while (p < (int *)((char *)area + size))
> > > > +		*p++ = BREAKPOINT_INSTRUCTION;
> > > > +}
> > > 
> > > This breaks the build for some configs, presumably you're missing a header:
> > > 
> > >   arch/powerpc/net/bpf_jit_comp64.c:30:10: error: 'BREAKPOINT_INSTRUCTION' undeclared (first use in this function)
> > > 
> > > http://kisskb.ellerman.id.au/kisskb/buildresult/12720611/
> > 
> > Oops. Yes, I should have caught that. I need to add:
> > 
> > #include <asm/kprobes.h>
> > 
> > in bpf_jit_comp64.c
> > 
> > Can you please check if it resolves the build error?
> 
> Can you? :D

:)
Sorry, I should have explained myself better. I did actually try your 
config and I was able to reproduce the build error. After the above 
#include, that error went away, but I saw some vdso related errors. I 
thought I was doing something wrong and needed a different setup for 
that particular kernel config, which is why I requested your help in the 
matter. I just didn't do a good job of putting across that message...

Note to self: randconfig builds *and* more time drafting emails :)

Do you want me to respin the patches?


Thanks,
Naveen

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH] ppc: Fix BPF JIT for ABIv2
  2016-06-20 18:56         ` Thadeu Lima de Souza Cascardo
@ 2016-06-21  8:58           ` Naveen N. Rao
  2016-06-21 11:15             ` Michael Ellerman
  2016-06-22  5:20             ` Michael Ellerman
  0 siblings, 2 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-21  8:58 UTC (permalink / raw)
  To: Thadeu Lima de Souza Cascardo, Michael Ellerman
  Cc: linux-kernel, linuxppc-dev, netdev, Matt Evans, Daniel Borkmann,
	Alexei Starovoitov, Denis Kirjanov, Paul Mackerras,
	David S. Miller, Ananth N Mavinakayanahalli

On 2016/06/20 03:56PM, Thadeu Lima de Souza Cascardo wrote:
> On Sun, Jun 19, 2016 at 11:19:14PM +0530, Naveen N. Rao wrote:
> > On 2016/06/17 10:00AM, Thadeu Lima de Souza Cascardo wrote:
> > > 
> > > Hi, Michael and Naveen.
> > > 
> > > I noticed independently that there is a problem with BPF JIT and ABIv2, and
> > > worked out the patch below before I noticed Naveen's patchset and the latest
> > > changes in ppc tree for a better way to check for ABI versions.
> > > 
> > > However, since the issue described below affect mainline and stable kernels,
> > > would you consider applying it before merging your two patchsets, so that we can
> > > more easily backport the fix?
> > 
> > Hi Cascardo,
> > Given that this has been broken on ABIv2 since forever, I didn't bother 
> > fixing it. But, I can see why this would be a good thing to have for 
> > -stable and existing distros. However, while your patch below may fix 
> > the crash you're seeing on ppc64le, it is not sufficient -- you'll need 
> > changes in bpf_jit_asm.S as well.
> 
> Hi, Naveen.
> 
> Any tips on how to exercise possible issues there? Or what changes you think
> would be sufficient?

The calling convention is different with ABIv2 and so we'll need changes 
in bpf_slow_path_common() and sk_negative_common().

However, rather than enabling classic JIT for ppc64le, are we better off 
just disabling it?

--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -128,7 +128,7 @@ config PPC
        select IRQ_FORCED_THREADING
        select HAVE_RCU_TABLE_FREE if SMP
        select HAVE_SYSCALL_TRACEPOINTS
-       select HAVE_CBPF_JIT
+       select HAVE_CBPF_JIT if CPU_BIG_ENDIAN
        select HAVE_ARCH_JUMP_LABEL
        select ARCH_HAVE_NMI_SAFE_CMPXCHG
        select ARCH_HAS_GCOV_PROFILE_ALL


Michael,
Let me know your thoughts on whether you intend to take this patch or 
Cascardo's patch for -stable before the eBPF patches. I can redo my 
patches accordingly.


- Naveen

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-21  6:58         ` Naveen N. Rao
@ 2016-06-21 11:04           ` Michael Ellerman
  2016-06-22  7:06             ` Naveen N. Rao
  0 siblings, 1 reply; 34+ messages in thread
From: Michael Ellerman @ 2016-06-21 11:04 UTC (permalink / raw)
  To: Naveen N. Rao
  Cc: linux-kernel, linuxppc-dev, netdev, Matt Evans, Daniel Borkmann,
	Alexei Starovoitov, Denis Kirjanov, Paul Mackerras,
	David S. Miller, Ananth N Mavinakayanahalli

On Tue, 2016-06-21 at 12:28 +0530, Naveen N. Rao wrote:
> On 2016/06/21 09:38AM, Michael Ellerman wrote:
> > On Sun, 2016-06-19 at 23:06 +0530, Naveen N. Rao wrote:
> > > 
> > > #include <asm/kprobes.h>
> > > 
> > > in bpf_jit_comp64.c
> > > 
> > > Can you please check if it resolves the build error?
> > 
> > Can you? :D
> 
> :)
> Sorry, I should have explained myself better. I did actually try your 
> config and I was able to reproduce the build error. After the above 
> #include, that error went away, but I saw some vdso related errors. I 
> thought I was doing something wrong and needed a different setup for 
> that particular kernel config, which is why I requested your help in the 
> matter. I just didn't do a good job of putting across that message...
 
Ah OK. Not sure why you're seeing VDSO errors?

> Note to self: randconfig builds *and* more time drafting emails :)

No stress. You don't need to do randconfig builds, or even build all the
arch/powerpc/ configs, just try to do a reasonable set, something like - ppc64,
powernv, pseries, pmac32, ppc64e.

I'm happy to catch the esoteric build failures.

> Do you want me to respin the patches?

No that's fine, I'll fix it up here.

cheers

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH] ppc: Fix BPF JIT for ABIv2
  2016-06-21  8:58           ` Naveen N. Rao
@ 2016-06-21 11:15             ` Michael Ellerman
  2016-06-21 14:47               ` Thadeu Lima de Souza Cascardo
  2016-06-22  5:20             ` Michael Ellerman
  1 sibling, 1 reply; 34+ messages in thread
From: Michael Ellerman @ 2016-06-21 11:15 UTC (permalink / raw)
  To: Naveen N. Rao, Thadeu Lima de Souza Cascardo
  Cc: linux-kernel, linuxppc-dev, netdev, Matt Evans, Daniel Borkmann,
	Alexei Starovoitov, Denis Kirjanov, Paul Mackerras,
	David S. Miller, Ananth N Mavinakayanahalli

On Tue, 2016-06-21 at 14:28 +0530, Naveen N. Rao wrote:
> On 2016/06/20 03:56PM, Thadeu Lima de Souza Cascardo wrote:
> > On Sun, Jun 19, 2016 at 11:19:14PM +0530, Naveen N. Rao wrote:
> > > On 2016/06/17 10:00AM, Thadeu Lima de Souza Cascardo wrote:
> > > > 
> > > > Hi, Michael and Naveen.
> > > > 
> > > > I noticed independently that there is a problem with BPF JIT and ABIv2, and
> > > > worked out the patch below before I noticed Naveen's patchset and the latest
> > > > changes in ppc tree for a better way to check for ABI versions.
> > > > 
> > > > However, since the issue described below affect mainline and stable kernels,
> > > > would you consider applying it before merging your two patchsets, so that we can
> > > > more easily backport the fix?
> > > 
> > > Hi Cascardo,
> > > Given that this has been broken on ABIv2 since forever, I didn't bother 
> > > fixing it. But, I can see why this would be a good thing to have for 
> > > -stable and existing distros. However, while your patch below may fix 
> > > the crash you're seeing on ppc64le, it is not sufficient -- you'll need 
> > > changes in bpf_jit_asm.S as well.
> > 
> > Hi, Naveen.
> > 
> > Any tips on how to exercise possible issues there? Or what changes you think
> > would be sufficient?
> 
> The calling convention is different with ABIv2 and so we'll need changes 
> in bpf_slow_path_common() and sk_negative_common().

How big would those changes be? Do we know?

How come no one reported this was broken previously? This is the first I've
heard of it being broken.

> However, rather than enabling classic JIT for ppc64le, are we better off 
> just disabling it?
> 
> --- a/arch/powerpc/Kconfig
> +++ b/arch/powerpc/Kconfig
> @@ -128,7 +128,7 @@ config PPC
>         select IRQ_FORCED_THREADING
>         select HAVE_RCU_TABLE_FREE if SMP
>         select HAVE_SYSCALL_TRACEPOINTS
> -       select HAVE_CBPF_JIT
> +       select HAVE_CBPF_JIT if CPU_BIG_ENDIAN
>         select HAVE_ARCH_JUMP_LABEL
>         select ARCH_HAVE_NMI_SAFE_CMPXCHG
>         select ARCH_HAS_GCOV_PROFILE_ALL
> 
> 
> Michael,
> Let me know your thoughts on whether you intend to take this patch or 
> Cascardo's patch for -stable before the eBPF patches. I can redo my 
> patches accordingly.

This patch sounds like the best option at the moment for something we can
backport. Unless the changes to fix it are minimal.

cheers

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH] ppc: Fix BPF JIT for ABIv2
  2016-06-21 11:15             ` Michael Ellerman
@ 2016-06-21 14:47               ` Thadeu Lima de Souza Cascardo
  2016-06-21 15:45                 ` Alexei Starovoitov
  2016-06-22  7:12                 ` Naveen N. Rao
  0 siblings, 2 replies; 34+ messages in thread
From: Thadeu Lima de Souza Cascardo @ 2016-06-21 14:47 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: Naveen N. Rao, linux-kernel, linuxppc-dev, netdev, Matt Evans,
	Daniel Borkmann, Alexei Starovoitov, Denis Kirjanov,
	Paul Mackerras, David S. Miller, Ananth N Mavinakayanahalli

On Tue, Jun 21, 2016 at 09:15:48PM +1000, Michael Ellerman wrote:
> On Tue, 2016-06-21 at 14:28 +0530, Naveen N. Rao wrote:
> > On 2016/06/20 03:56PM, Thadeu Lima de Souza Cascardo wrote:
> > > On Sun, Jun 19, 2016 at 11:19:14PM +0530, Naveen N. Rao wrote:
> > > > On 2016/06/17 10:00AM, Thadeu Lima de Souza Cascardo wrote:
> > > > > 
> > > > > Hi, Michael and Naveen.
> > > > > 
> > > > > I noticed independently that there is a problem with BPF JIT and ABIv2, and
> > > > > worked out the patch below before I noticed Naveen's patchset and the latest
> > > > > changes in ppc tree for a better way to check for ABI versions.
> > > > > 
> > > > > However, since the issue described below affect mainline and stable kernels,
> > > > > would you consider applying it before merging your two patchsets, so that we can
> > > > > more easily backport the fix?
> > > > 
> > > > Hi Cascardo,
> > > > Given that this has been broken on ABIv2 since forever, I didn't bother 
> > > > fixing it. But, I can see why this would be a good thing to have for 
> > > > -stable and existing distros. However, while your patch below may fix 
> > > > the crash you're seeing on ppc64le, it is not sufficient -- you'll need 
> > > > changes in bpf_jit_asm.S as well.
> > > 
> > > Hi, Naveen.
> > > 
> > > Any tips on how to exercise possible issues there? Or what changes you think
> > > would be sufficient?
> > 
> > The calling convention is different with ABIv2 and so we'll need changes 
> > in bpf_slow_path_common() and sk_negative_common().
> 
> How big would those changes be? Do we know?
> 
> How come no one reported this was broken previously? This is the first I've
> heard of it being broken.
> 

I just heard of it less than two weeks ago, and only could investigate it last
week, when I realized mainline was also affected.

It looks like the little-endian support for classic JIT were done before the
conversion to ABIv2. And as JIT is disabled by default, no one seems to have
exercised it.

> > However, rather than enabling classic JIT for ppc64le, are we better off 
> > just disabling it?
> > 
> > --- a/arch/powerpc/Kconfig
> > +++ b/arch/powerpc/Kconfig
> > @@ -128,7 +128,7 @@ config PPC
> >         select IRQ_FORCED_THREADING
> >         select HAVE_RCU_TABLE_FREE if SMP
> >         select HAVE_SYSCALL_TRACEPOINTS
> > -       select HAVE_CBPF_JIT
> > +       select HAVE_CBPF_JIT if CPU_BIG_ENDIAN
> >         select HAVE_ARCH_JUMP_LABEL
> >         select ARCH_HAVE_NMI_SAFE_CMPXCHG
> >         select ARCH_HAS_GCOV_PROFILE_ALL
> > 
> > 
> > Michael,
> > Let me know your thoughts on whether you intend to take this patch or 
> > Cascardo's patch for -stable before the eBPF patches. I can redo my 
> > patches accordingly.
> 
> This patch sounds like the best option at the moment for something we can
> backport. Unless the changes to fix it are minimal.
> 
> cheers
> 

With my patch only, I can run a minimal tcpdump tcp port 22 with success. It
correctly filter packets. But as pointed out, slow paths may not be taken.

I don't have strong opinions on what to apply to stable, just that it would be
nice to have something for the crash before applying all the nice changes by
Naveen.

Cascardo.

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH] ppc: Fix BPF JIT for ABIv2
  2016-06-21 14:47               ` Thadeu Lima de Souza Cascardo
@ 2016-06-21 15:45                 ` Alexei Starovoitov
  2016-06-22  4:06                   ` Michael Ellerman
  2016-06-22  7:12                 ` Naveen N. Rao
  1 sibling, 1 reply; 34+ messages in thread
From: Alexei Starovoitov @ 2016-06-21 15:45 UTC (permalink / raw)
  To: Thadeu Lima de Souza Cascardo, Michael Ellerman
  Cc: Naveen N. Rao, linux-kernel, linuxppc-dev, netdev, Matt Evans,
	Daniel Borkmann, Denis Kirjanov, Paul Mackerras, David S. Miller,
	Ananth N Mavinakayanahalli

On 6/21/16 7:47 AM, Thadeu Lima de Souza Cascardo wrote:
>>>
>>> The calling convention is different with ABIv2 and so we'll need changes
>>> in bpf_slow_path_common() and sk_negative_common().
>>
>> How big would those changes be? Do we know?
>>
>> How come no one reported this was broken previously? This is the first I've
>> heard of it being broken.
>>
>
> I just heard of it less than two weeks ago, and only could investigate it last
> week, when I realized mainline was also affected.
>
> It looks like the little-endian support for classic JIT were done before the
> conversion to ABIv2. And as JIT is disabled by default, no one seems to have
> exercised it.

it's not a surprise unfortunately. The JITs that were written before
test_bpf.ko was developed were missing corner cases. Typical tcpdump
would be fine, but fragmented packets, negative offsets and
out-out-bounds wouldn't be handled correctly.
I'd suggest to validate the stable backport with test_bpf as well.

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH] ppc: Fix BPF JIT for ABIv2
  2016-06-21 15:45                 ` Alexei Starovoitov
@ 2016-06-22  4:06                   ` Michael Ellerman
  0 siblings, 0 replies; 34+ messages in thread
From: Michael Ellerman @ 2016-06-22  4:06 UTC (permalink / raw)
  To: Alexei Starovoitov, Thadeu Lima de Souza Cascardo
  Cc: Naveen N. Rao, linux-kernel, linuxppc-dev, netdev, Matt Evans,
	Daniel Borkmann, Denis Kirjanov, Paul Mackerras, David S. Miller,
	Ananth N Mavinakayanahalli

On Tue, 2016-06-21 at 08:45 -0700, Alexei Starovoitov wrote:
> On 6/21/16 7:47 AM, Thadeu Lima de Souza Cascardo wrote:
> > > > 
> > > > The calling convention is different with ABIv2 and so we'll need changes
> > > > in bpf_slow_path_common() and sk_negative_common().
> > > 
> > > How big would those changes be? Do we know?
> > > 
> > > How come no one reported this was broken previously? This is the first I've
> > > heard of it being broken.
> > > 
> > 
> > I just heard of it less than two weeks ago, and only could investigate it last
> > week, when I realized mainline was also affected.
> > 
> > It looks like the little-endian support for classic JIT were done before the
> > conversion to ABIv2. And as JIT is disabled by default, no one seems to have
> > exercised it.
> 
> it's not a surprise unfortunately. The JITs that were written before
> test_bpf.ko was developed were missing corner cases. Typical tcpdump
> would be fine, but fragmented packets, negative offsets and
> out-out-bounds wouldn't be handled correctly.
> I'd suggest to validate the stable backport with test_bpf as well.
 
OK thanks.

I have been running seltests/net/test_bpf, but I realise now it doesn't enable
the JIT.

cheers

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH] ppc: Fix BPF JIT for ABIv2
  2016-06-17 13:00     ` [PATCH] ppc: Fix BPF JIT for ABIv2 Thadeu Lima de Souza Cascardo
  2016-06-19 17:49       ` Naveen N. Rao
@ 2016-06-22  5:18       ` Michael Ellerman
  1 sibling, 0 replies; 34+ messages in thread
From: Michael Ellerman @ 2016-06-22  5:18 UTC (permalink / raw)
  To: Thadeu Lima de Souza Cascardo
  Cc: Naveen N. Rao, linux-kernel, linuxppc-dev, netdev, Matt Evans,
	Daniel Borkmann, Alexei Starovoitov, Denis Kirjanov,
	Paul Mackerras, David S. Miller, Ananth N Mavinakayanahalli

On Fri, 2016-06-17 at 10:00 -0300, Thadeu Lima de Souza Cascardo wrote:
> From a984dc02b6317a1d3a3c2302385adba5227be5bd Mon Sep 17 00:00:00 2001
> From: Thadeu Lima de Souza Cascardo <cascardo@redhat.com>
> Date: Wed, 15 Jun 2016 13:22:12 -0300
> Subject: [PATCH] ppc: Fix BPF JIT for ABIv2
> 
> ABIv2 used for ppc64le does not use function descriptors. Without this patch,
> whenever BPF JIT is enabled, we get a crash as below.
> 
...

> diff --git a/arch/powerpc/net/bpf_jit.h b/arch/powerpc/net/bpf_jit.h
> index 889fd19..28b89ed 100644
> --- a/arch/powerpc/net/bpf_jit.h
> +++ b/arch/powerpc/net/bpf_jit.h
> @@ -70,7 +70,7 @@ DECLARE_LOAD_FUNC(sk_load_half);
>  DECLARE_LOAD_FUNC(sk_load_byte);
>  DECLARE_LOAD_FUNC(sk_load_byte_msh);
>  
> -#ifdef CONFIG_PPC64
> +#if defined(CONFIG_PPC64) && (!defined(_CALL_ELF) || _CALL_ELF != 2)
>  #define FUNCTION_DESCR_SIZE	24
>  #else
>  #define FUNCTION_DESCR_SIZE	0
> diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
> index 2d66a84..035b887 100644
> --- a/arch/powerpc/net/bpf_jit_comp.c
> +++ b/arch/powerpc/net/bpf_jit_comp.c
> @@ -664,7 +664,7 @@ void bpf_jit_compile(struct bpf_prog *fp)
>  
>  	if (image) {
>  		bpf_flush_icache(code_base, code_base + (proglen/4));
> -#ifdef CONFIG_PPC64
> +#if defined(CONFIG_PPC64) && (!defined(_CALL_ELF) || _CALL_ELF != 2)
>  		/* Function descriptor nastiness: Address + TOC */
>  		((u64 *)image)[0] = (u64)code_base;
>  		((u64 *)image)[1] = local_paca->kernel_toc;


Confirmed that even with this patch we still crash:

  # echo 1 > /proc/sys/net/core/bpf_jit_enable
  # modprobe test_bpf
  BPF filter opcode 0020 (@3) unsupported
  BPF filter opcode 0020 (@2) unsupported
  BPF filter opcode 0020 (@0) unsupported
  Unable to handle kernel paging request for data at address 0xd0000000054f65e8
  Faulting instruction address: 0xc0000000008765f8
  cpu 0x0: Vector: 300 (Data Access) at [c0000000034f3480]
      pc: c0000000008765f8: skb_copy_bits+0x158/0x330
      lr: c00000000008fb7c: bpf_slow_path_byte+0x28/0x54
      sp: c0000000034f3700
     msr: 800000010280b033
     dar: d0000000054f65e8
   dsisr: 40000000
    current = 0xc0000001f857d8d0
    paca    = 0xc000000007b80000	 softe: 0	 irq_happened: 0x01
      pid   = 2993, comm = modprobe
  Linux version 4.7.0-rc3-00055-g9497a1c1c5b4-dirty (michael@ka3.ozlabs.ibm.com) () #30 SMP Wed Jun 22 15:06:58 AEST 2016
  enter ? for help
  [c0000000034f3770] c00000000008fb7c bpf_slow_path_byte+0x28/0x54
  [c0000000034f37e0] d000000007bb004c
  [c0000000034f3900] d000000005331668 test_bpf_init+0x5fc/0x7f8 [test_bpf]
  [c0000000034f3a30] c00000000000b628 do_one_initcall+0x68/0x1d0
  [c0000000034f3af0] c0000000009beb24 do_init_module+0x90/0x240
  [c0000000034f3b80] c0000000001642bc load_module+0x206c/0x22f0
  [c0000000034f3d30] c0000000001648b0 SyS_finit_module+0x120/0x180
  [c0000000034f3e30] c000000000009260 system_call+0x38/0x108
  --- Exception: c01 (System Call) at 00003fff7ffa2db4


cheers

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH] ppc: Fix BPF JIT for ABIv2
  2016-06-21  8:58           ` Naveen N. Rao
  2016-06-21 11:15             ` Michael Ellerman
@ 2016-06-22  5:20             ` Michael Ellerman
  1 sibling, 0 replies; 34+ messages in thread
From: Michael Ellerman @ 2016-06-22  5:20 UTC (permalink / raw)
  To: Naveen N. Rao, Thadeu Lima de Souza Cascardo
  Cc: linux-kernel, linuxppc-dev, netdev, Matt Evans, Daniel Borkmann,
	Alexei Starovoitov, Denis Kirjanov, Paul Mackerras,
	David S. Miller, Ananth N Mavinakayanahalli

On Tue, 2016-06-21 at 14:28 +0530, Naveen N. Rao wrote:
> On 2016/06/20 03:56PM, Thadeu Lima de Souza Cascardo wrote:
> > On Sun, Jun 19, 2016 at 11:19:14PM +0530, Naveen N. Rao wrote:
> > > On 2016/06/17 10:00AM, Thadeu Lima de Souza Cascardo wrote:
> > > > 
> > > > Hi, Michael and Naveen.
> > > > 
> > > > I noticed independently that there is a problem with BPF JIT and ABIv2, and
> > > > worked out the patch below before I noticed Naveen's patchset and the latest
> > > > changes in ppc tree for a better way to check for ABI versions.
> > > > 
> > > > However, since the issue described below affect mainline and stable kernels,
> > > > would you consider applying it before merging your two patchsets, so that we can
> > > > more easily backport the fix?
> > > 
> > > Hi Cascardo,
> > > Given that this has been broken on ABIv2 since forever, I didn't bother 
> > > fixing it. But, I can see why this would be a good thing to have for 
> > > -stable and existing distros. However, while your patch below may fix 
> > > the crash you're seeing on ppc64le, it is not sufficient -- you'll need 
> > > changes in bpf_jit_asm.S as well.
> > 
> > Hi, Naveen.
> > 
> > Any tips on how to exercise possible issues there? Or what changes you think
> > would be sufficient?
> 
> The calling convention is different with ABIv2 and so we'll need changes 
> in bpf_slow_path_common() and sk_negative_common().
> 
> However, rather than enabling classic JIT for ppc64le, are we better off 
> just disabling it?
> 
> --- a/arch/powerpc/Kconfig
> +++ b/arch/powerpc/Kconfig
> @@ -128,7 +128,7 @@ config PPC
>         select IRQ_FORCED_THREADING
>         select HAVE_RCU_TABLE_FREE if SMP
>         select HAVE_SYSCALL_TRACEPOINTS
> -       select HAVE_CBPF_JIT
> +       select HAVE_CBPF_JIT if CPU_BIG_ENDIAN
>         select HAVE_ARCH_JUMP_LABEL
>         select ARCH_HAVE_NMI_SAFE_CMPXCHG
>         select ARCH_HAS_GCOV_PROFILE_ALL
> 
> 
> Michael,
> Let me know your thoughts on whether you intend to take this patch or 
> Cascardo's patch for -stable before the eBPF patches. I can redo my 
> patches accordingly.

Can one of you send me a proper version of this patch, with change log and
sign-off etc.

cheers

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-21 11:04           ` Michael Ellerman
@ 2016-06-22  7:06             ` Naveen N. Rao
  0 siblings, 0 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-22  7:06 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: linux-kernel, linuxppc-dev, netdev, Matt Evans, Daniel Borkmann,
	Alexei Starovoitov, Denis Kirjanov, Paul Mackerras,
	David S. Miller, Ananth N Mavinakayanahalli

On 2016/06/21 09:04PM, Michael Ellerman wrote:
> On Tue, 2016-06-21 at 12:28 +0530, Naveen N. Rao wrote:
> > On 2016/06/21 09:38AM, Michael Ellerman wrote:
> > > On Sun, 2016-06-19 at 23:06 +0530, Naveen N. Rao wrote:
> > > > 
> > > > #include <asm/kprobes.h>
> > > > 
> > > > in bpf_jit_comp64.c
> > > > 
> > > > Can you please check if it resolves the build error?
> > > 
> > > Can you? :D
> > 
> > :)
> > Sorry, I should have explained myself better. I did actually try your 
> > config and I was able to reproduce the build error. After the above 
> > #include, that error went away, but I saw some vdso related errors. I 
> > thought I was doing something wrong and needed a different setup for 
> > that particular kernel config, which is why I requested your help in the 
> > matter. I just didn't do a good job of putting across that message...
> 
> Ah OK. Not sure why you're seeing VDSO errors?

'Cause I wasn't paying attention. I tried your .config on a LE machine.  
It works fine on BE, as it should.

> 
> > Note to self: randconfig builds *and* more time drafting emails :)
> 
> No stress. You don't need to do randconfig builds, or even build all the
> arch/powerpc/ configs, just try to do a reasonable set, something like - ppc64,
> powernv, pseries, pmac32, ppc64e.

Ok, will do.

> 
> I'm happy to catch the esoteric build failures.
> 
> > Do you want me to respin the patches?
> 
> No that's fine, I'll fix it up here.

Thanks,
Naveen

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH] ppc: Fix BPF JIT for ABIv2
  2016-06-21 14:47               ` Thadeu Lima de Souza Cascardo
  2016-06-21 15:45                 ` Alexei Starovoitov
@ 2016-06-22  7:12                 ` Naveen N. Rao
  2016-06-22 14:57                   ` Naveen N. Rao
  1 sibling, 1 reply; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-22  7:12 UTC (permalink / raw)
  To: Thadeu Lima de Souza Cascardo
  Cc: Michael Ellerman, Matt Evans, Daniel Borkmann,
	Alexei Starovoitov, netdev, Denis Kirjanov, linux-kernel,
	Paul Mackerras, linuxppc-dev, David S. Miller,
	Ananth N Mavinakayanahalli

On 2016/06/21 11:47AM, Thadeu Lima de Souza Cascardo wrote:
> On Tue, Jun 21, 2016 at 09:15:48PM +1000, Michael Ellerman wrote:
> > On Tue, 2016-06-21 at 14:28 +0530, Naveen N. Rao wrote:
> > > On 2016/06/20 03:56PM, Thadeu Lima de Souza Cascardo wrote:
> > > > On Sun, Jun 19, 2016 at 11:19:14PM +0530, Naveen N. Rao wrote:
> > > > > On 2016/06/17 10:00AM, Thadeu Lima de Souza Cascardo wrote:
> > > > > > 
> > > > > > Hi, Michael and Naveen.
> > > > > > 
> > > > > > I noticed independently that there is a problem with BPF JIT and ABIv2, and
> > > > > > worked out the patch below before I noticed Naveen's patchset and the latest
> > > > > > changes in ppc tree for a better way to check for ABI versions.
> > > > > > 
> > > > > > However, since the issue described below affect mainline and stable kernels,
> > > > > > would you consider applying it before merging your two patchsets, so that we can
> > > > > > more easily backport the fix?
> > > > > 
> > > > > Hi Cascardo,
> > > > > Given that this has been broken on ABIv2 since forever, I didn't bother 
> > > > > fixing it. But, I can see why this would be a good thing to have for 
> > > > > -stable and existing distros. However, while your patch below may fix 
> > > > > the crash you're seeing on ppc64le, it is not sufficient -- you'll need 
> > > > > changes in bpf_jit_asm.S as well.
> > > > 
> > > > Hi, Naveen.
> > > > 
> > > > Any tips on how to exercise possible issues there? Or what changes you think
> > > > would be sufficient?
> > > 
> > > The calling convention is different with ABIv2 and so we'll need changes 
> > > in bpf_slow_path_common() and sk_negative_common().
> > 
> > How big would those changes be? Do we know?

I don't think it'd be that much -- I will take a stab at this today.

> > 
> > How come no one reported this was broken previously? This is the first I've
> > heard of it being broken.
> > 
> 
> I just heard of it less than two weeks ago, and only could investigate it last
> week, when I realized mainline was also affected.
> 
> It looks like the little-endian support for classic JIT were done before the
> conversion to ABIv2. And as JIT is disabled by default, no one seems to have
> exercised it.

Yes, my thoughts too. I didn't previously think much about this as JIT 
wouldn't be enabled by default. It's interesting though that no one else 
reported this as an issue before.

> 
> > > However, rather than enabling classic JIT for ppc64le, are we better off 
> > > just disabling it?
> > > 
> > > --- a/arch/powerpc/Kconfig
> > > +++ b/arch/powerpc/Kconfig
> > > @@ -128,7 +128,7 @@ config PPC
> > >         select IRQ_FORCED_THREADING
> > >         select HAVE_RCU_TABLE_FREE if SMP
> > >         select HAVE_SYSCALL_TRACEPOINTS
> > > -       select HAVE_CBPF_JIT
> > > +       select HAVE_CBPF_JIT if CPU_BIG_ENDIAN
> > >         select HAVE_ARCH_JUMP_LABEL
> > >         select ARCH_HAVE_NMI_SAFE_CMPXCHG
> > >         select ARCH_HAS_GCOV_PROFILE_ALL
> > > 
> > > 
> > > Michael,
> > > Let me know your thoughts on whether you intend to take this patch or 
> > > Cascardo's patch for -stable before the eBPF patches. I can redo my 
> > > patches accordingly.
> > 
> > This patch sounds like the best option at the moment for something we can
> > backport. Unless the changes to fix it are minimal.

Right -- I will take a look today to see what changes would be needed.

- Naveen

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-07 13:32 ` [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF Naveen N. Rao
                     ` (2 preceding siblings ...)
  2016-06-17 12:53   ` [6/6] " Michael Ellerman
@ 2016-06-22 10:37   ` Michael Ellerman
  2016-06-22 12:32     ` Naveen N. Rao
  3 siblings, 1 reply; 34+ messages in thread
From: Michael Ellerman @ 2016-06-22 10:37 UTC (permalink / raw)
  To: Naveen N. Rao, linux-kernel, linuxppc-dev, netdev
  Cc: Matt Evans, Denis Kirjanov, Paul Mackerras, Alexei Starovoitov,
	Daniel Borkmann, David S. Miller, Ananth N Mavinakayanahalli

On Tue, 2016-06-07 at 19:02 +0530, Naveen N. Rao wrote:

> PPC64 eBPF JIT compiler.
> 
> Enable with:
> echo 1 > /proc/sys/net/core/bpf_jit_enable
> or
> echo 2 > /proc/sys/net/core/bpf_jit_enable
> 
> ... to see the generated JIT code. This can further be processed with
> tools/net/bpf_jit_disasm.
> 
> With CONFIG_TEST_BPF=m and 'modprobe test_bpf':
> test_bpf: Summary: 305 PASSED, 0 FAILED, [297/297 JIT'ed]
> 
> ... on both ppc64 BE and LE.
> 
> The details of the approach are documented through various comments in
> the code.

This is crashing for me on a Cell machine, not sure why at a glance:


test_bpf: #250 JMP_JSET_X: if (0x3 & 0xffffffff) return 1 jited:1 14 PASS
test_bpf: #251 JMP_JA: Jump, gap, jump, ... jited:1 15 PASS
test_bpf: #252 BPF_MAXINSNS: Maximum possible literals 
Unable to handle kernel paging request for data at address 0xd000000007b20000
Faulting instruction address: 0xc000000000667b6c
cpu 0x0: Vector: 300 (Data Access) at [c0000007f83bf3a0]
    pc: c000000000667b6c: .flush_icache_range+0x3c/0x84
    lr: c000000000082354: .bpf_int_jit_compile+0x1fc/0x2c8
    sp: c0000007f83bf620
   msr: 900000000200b032
   dar: d000000007b20000
 dsisr: 40000000
  current = 0xc0000007f8249580
  paca    = 0xc00000000fff0000	 softe: 0	 irq_happened: 0x01
    pid   = 1822, comm = insmod
Linux version 4.7.0-rc3-00061-g007c99b9d8c1 (michael@ka3.ozlabs.ibm.com) (gcc version 6.1.0 (GCC) ) #3 SMP Wed Jun 22 19:22:23 AEST 2016
enter ? for help
[link register   ] c000000000082354 .bpf_int_jit_compile+0x1fc/0x2c8
[c0000007f83bf620] c0000000000822fc .bpf_int_jit_compile+0x1a4/0x2c8 (unreliable)
[c0000007f83bf700] c00000000013cda4 .bpf_prog_select_runtime+0x24/0x108
[c0000007f83bf780] c000000000548918 .bpf_prepare_filter+0x9b0/0x9e8
[c0000007f83bf830] c0000000005489d4 .bpf_prog_create+0x84/0xd0
[c0000007f83bf8c0] d000000003b21158 .test_bpf_init+0x28c/0x83c [test_bpf]
[c0000007f83bfa00] c00000000000a7b4 .do_one_initcall+0x5c/0x1c0
[c0000007f83bfae0] c000000000669058 .do_init_module+0x80/0x21c
[c0000007f83bfb80] c00000000011e3a0 .load_module+0x2028/0x23a8
[c0000007f83bfd20] c00000000011e898 .SyS_init_module+0x178/0x1b0
[c0000007f83bfe30] c000000000009220 system_call+0x38/0x110
--- Exception: c01 (System Call) at 000000000ff5e0c4
SP (ffde0960) is in userspace
0:mon> r
R00 = 000000000000c01c   R16 = 0000000000000000
R01 = c0000007f83bf620   R17 = 00000000024000c0
R02 = c00000000094ce00   R18 = 0000000000000000
R03 = d000000007b10000   R19 = d000000003c32df0
R04 = d000000007b40338   R20 = c00000000072b488
R05 = 000000000000007f   R21 = d000000007b10000
R06 = d000000007b20000   R22 = c00000000098184c
R07 = 0000000000000080   R23 = 0000000000000000
R08 = 0000000000000607   R24 = 00000000000300e0
R09 = 0000000000000007   R25 = 000000000000c020
R10 = c000000000861ee0   R26 = d000000007b10270
R11 = c0000000006755f8   R27 = c0000007fe0e0000
R12 = d000000007b10270   R28 = 0000000000002003
R13 = c00000000fff0000   R29 = c0000007f83bf690
R14 = d000000003c32d61   R30 = 0000000000000003
R15 = 0000000000000000   R31 = d000000007ae0000
pc  = c000000000667b6c .flush_icache_range+0x3c/0x84
lr  = c000000000082354 .bpf_int_jit_compile+0x1fc/0x2c8
msr = 900000000200b032   cr  = 44000248
ctr = 0000000000000407   xer = 0000000020000000   trap =  300
dar = d000000007b20000   dsisr = 40000000
0:mon> S
msr  = 9000000000001032  sprg0= 0000000000008001
pvr  = 0000000000703000  sprg1= c00000000fff0000
dec  = 000000009f2d8ba4  sprg2= c00000000fff0000
sp   = c0000007f83bed30  sprg3= 0000000000000000
toc  = c00000000094ce00  dar  = d000000007b20000
0:mon> u
SLB contents of cpu 0x0
00 c000000008000000 0000af32f5079500 256M ESID=c00000000  VSID=    af32f5079 LLP:100 
01 d000000008000000 0000836935091510 256M ESID=d00000000  VSID=    836935091 LLP:110 
02 c0000007f8000000 0000b52186c20500 256M ESID=c0000007f  VSID=    b52186c20 LLP:100 
03 c0000003f0000000 0000b224435e0500
04 c0000007f0000000 0000b52186c20500
05 c0000003f0000000 0000b224435e0500
06 c0000007f0000000 0000b52186c20500
07 c0000003f0000000 0000b224435e0500
08 c0000007f0000000 0000b52186c20500
09 c0000003f0000000 0000b224435e0500
10 c0000007f0000000 0000b52186c20500
11 c0000003f0000000 0000b224435e0500
12 c0000007f0000000 0000b52186c20500
13 c0000003f0000000 0000b224435e0500
14 c0000007f0000000 0000b52186c20500
15 c0000003f0000000 0000b224435e0500
16 c0000007f0000000 0000b52186c20500
17 c000000078000000 0000af86a8668500 256M ESID=c00000007  VSID=    af86a8668 LLP:100 
18 c0000003f0000000 0000b224435e0500
19 c0000007f0000000 0000b52186c20500
20 c0000003f0000000 0000b224435e0500
21 c0000007f0000000 0000b52186c20500
22 c0000003f0000000 0000b224435e0500
23 c0000007f0000000 0000b52186c20500
24 c0000003f0000000 0000b224435e0500
25 c0000007f0000000 0000b52186c20500
26 c0000003f0000000 0000b224435e0500
27 c0000007f0000000 0000b52186c20500
28 c0000003f0000000 0000b224435e0500
29 c0000007f0000000 0000b52186c20500
30 c0000003f0000000 0000b224435e0500
31 c0000007f0000000 0000b52186c20500
32 c0000003f0000000 0000b224435e0500
33 c0000007f0000000 0000b52186c20500
34 c0000003f0000000 0000b224435e0500
35 c0000007f0000000 0000b52186c20500
36 c0000003f0000000 0000b224435e0500
37 c0000007f0000000 0000b52186c20500
38 c0000003f0000000 0000b224435e0500
39 c0000007f0000000 0000b52186c20500
40 c0000003f0000000 0000b224435e0500
41 c0000007f0000000 0000b52186c20500
42 c0000003f0000000 0000b224435e0500
43 c0000007f0000000 0000b52186c20500
44 c0000003f0000000 0000b224435e0500
45 c0000007f0000000 0000b52186c20500
46 c0000003f0000000 0000b224435e0500
47 c0000007f0000000 0000b52186c20500
48 c0000003f0000000 0000b224435e0500
49 d000080088000000 00007e4fa575c510 256M ESID=d00008008  VSID=    7e4fa575c LLP:110 
50 c0000007f0000000 0000b52186c20500
51 c0000003f0000000 0000b224435e0500
52 c0000007f0000000 0000b52186c20500
53 c0000003f0000000 0000b224435e0500
54 c0000007f0000000 0000b52186c20500
55 c0000003f0000000 0000b224435e0500
56 c0000007f0000000 0000b52186c20500
57 c0000003f8000000 0000b224435e0500 256M ESID=c0000003f  VSID=    b224435e0 LLP:100 
58 f000000008000000 00002bd5b50c1500 256M ESID=f00000000  VSID=    2bd5b50c1 LLP:100 
59 c0000007f0000000 0000b52186c20500
60 c0000003f0000000 0000b224435e0500
61 c0000007f0000000 0000b52186c20500
62 c0000003f0000000 0000b224435e0500
63 c0000007f0000000 0000b52186c20500
0:mon> 



cheers

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF
  2016-06-22 10:37   ` [PATCH 6/6] " Michael Ellerman
@ 2016-06-22 12:32     ` Naveen N. Rao
  0 siblings, 0 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-22 12:32 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: linux-kernel, linuxppc-dev, netdev, Matt Evans, Denis Kirjanov,
	Paul Mackerras, Alexei Starovoitov, Daniel Borkmann,
	David S. Miller, Ananth N Mavinakayanahalli

On 2016/06/22 08:37PM, Michael Ellerman wrote:
> On Tue, 2016-06-07 at 19:02 +0530, Naveen N. Rao wrote:
> 
> > PPC64 eBPF JIT compiler.
> > 
> > Enable with:
> > echo 1 > /proc/sys/net/core/bpf_jit_enable
> > or
> > echo 2 > /proc/sys/net/core/bpf_jit_enable
> > 
> > ... to see the generated JIT code. This can further be processed with
> > tools/net/bpf_jit_disasm.
> > 
> > With CONFIG_TEST_BPF=m and 'modprobe test_bpf':
> > test_bpf: Summary: 305 PASSED, 0 FAILED, [297/297 JIT'ed]
> > 
> > ... on both ppc64 BE and LE.
> > 
> > The details of the approach are documented through various comments in
> > the code.
> 
> This is crashing for me on a Cell machine, not sure why at a glance:
> 
> 
> test_bpf: #250 JMP_JSET_X: if (0x3 & 0xffffffff) return 1 jited:1 14 PASS
> test_bpf: #251 JMP_JA: Jump, gap, jump, ... jited:1 15 PASS
> test_bpf: #252 BPF_MAXINSNS: Maximum possible literals 
> Unable to handle kernel paging request for data at address 0xd000000007b20000
> Faulting instruction address: 0xc000000000667b6c
> cpu 0x0: Vector: 300 (Data Access) at [c0000007f83bf3a0]
>     pc: c000000000667b6c: .flush_icache_range+0x3c/0x84
>     lr: c000000000082354: .bpf_int_jit_compile+0x1fc/0x2c8
>     sp: c0000007f83bf620
>    msr: 900000000200b032
>    dar: d000000007b20000
>  dsisr: 40000000
>   current = 0xc0000007f8249580
>   paca    = 0xc00000000fff0000	 softe: 0	 irq_happened: 0x01
>     pid   = 1822, comm = insmod
> Linux version 4.7.0-rc3-00061-g007c99b9d8c1 (michael@ka3.ozlabs.ibm.com) (gcc version 6.1.0 (GCC) ) #3 SMP Wed Jun 22 19:22:23 AEST 2016
> enter ? for help
> [link register   ] c000000000082354 .bpf_int_jit_compile+0x1fc/0x2c8
> [c0000007f83bf620] c0000000000822fc .bpf_int_jit_compile+0x1a4/0x2c8 (unreliable)
> [c0000007f83bf700] c00000000013cda4 .bpf_prog_select_runtime+0x24/0x108
> [c0000007f83bf780] c000000000548918 .bpf_prepare_filter+0x9b0/0x9e8
> [c0000007f83bf830] c0000000005489d4 .bpf_prog_create+0x84/0xd0
> [c0000007f83bf8c0] d000000003b21158 .test_bpf_init+0x28c/0x83c [test_bpf]
> [c0000007f83bfa00] c00000000000a7b4 .do_one_initcall+0x5c/0x1c0
> [c0000007f83bfae0] c000000000669058 .do_init_module+0x80/0x21c
> [c0000007f83bfb80] c00000000011e3a0 .load_module+0x2028/0x23a8
> [c0000007f83bfd20] c00000000011e898 .SyS_init_module+0x178/0x1b0
> [c0000007f83bfe30] c000000000009220 system_call+0x38/0x110
> --- Exception: c01 (System Call) at 000000000ff5e0c4
> SP (ffde0960) is in userspace
> 0:mon> r
> R00 = 000000000000c01c   R16 = 0000000000000000
> R01 = c0000007f83bf620   R17 = 00000000024000c0
> R02 = c00000000094ce00   R18 = 0000000000000000
> R03 = d000000007b10000   R19 = d000000003c32df0
> R04 = d000000007b40338   R20 = c00000000072b488

Wow. I can't actually understand why this did not trigger for me. We are 
sending incorrect values into flush_icache_range(). So the first page is 
being flushed properly, but we are faulting trying to access another 
page. Patch forthcoming.

Thanks,
Naveen
 

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH] ppc: Fix BPF JIT for ABIv2
  2016-06-22  7:12                 ` Naveen N. Rao
@ 2016-06-22 14:57                   ` Naveen N. Rao
  0 siblings, 0 replies; 34+ messages in thread
From: Naveen N. Rao @ 2016-06-22 14:57 UTC (permalink / raw)
  To: Thadeu Lima de Souza Cascardo
  Cc: Michael Ellerman, Matt Evans, Daniel Borkmann,
	Alexei Starovoitov, netdev, Denis Kirjanov, linux-kernel,
	Paul Mackerras, linuxppc-dev, David S. Miller,
	Ananth N Mavinakayanahalli

On 2016/06/22 12:42PM, Naveen N Rao wrote:
> On 2016/06/21 11:47AM, Thadeu Lima de Souza Cascardo wrote:
> > On Tue, Jun 21, 2016 at 09:15:48PM +1000, Michael Ellerman wrote:
> > > On Tue, 2016-06-21 at 14:28 +0530, Naveen N. Rao wrote:
> > > > On 2016/06/20 03:56PM, Thadeu Lima de Souza Cascardo wrote:
> > > > > On Sun, Jun 19, 2016 at 11:19:14PM +0530, Naveen N. Rao wrote:
> > > > > > On 2016/06/17 10:00AM, Thadeu Lima de Souza Cascardo wrote:
> > > > > > > 
> > > > > > > Hi, Michael and Naveen.
> > > > > > > 
> > > > > > > I noticed independently that there is a problem with BPF JIT and ABIv2, and
> > > > > > > worked out the patch below before I noticed Naveen's patchset and the latest
> > > > > > > changes in ppc tree for a better way to check for ABI versions.
> > > > > > > 
> > > > > > > However, since the issue described below affect mainline and stable kernels,
> > > > > > > would you consider applying it before merging your two patchsets, so that we can
> > > > > > > more easily backport the fix?
> > > > > > 
> > > > > > Hi Cascardo,
> > > > > > Given that this has been broken on ABIv2 since forever, I didn't bother 
> > > > > > fixing it. But, I can see why this would be a good thing to have for 
> > > > > > -stable and existing distros. However, while your patch below may fix 
> > > > > > the crash you're seeing on ppc64le, it is not sufficient -- you'll need 
> > > > > > changes in bpf_jit_asm.S as well.
> > > > > 
> > > > > Hi, Naveen.
> > > > > 
> > > > > Any tips on how to exercise possible issues there? Or what changes you think
> > > > > would be sufficient?
> > > > 
> > > > The calling convention is different with ABIv2 and so we'll need changes 
> > > > in bpf_slow_path_common() and sk_negative_common().
> > > 
> > > How big would those changes be? Do we know?

So, this does need quite a few changes:
- the skb helpers need to emit code to setup TOC and the JIT code needs 
  to be updated to setup r12.
- the slow path code needs to be changed to store r3 elsewhere on ABIv2
- the above also means we need to change the stack macros with the 
  proper ABIv2 values
- the little endian support isn't complete as well -- some of the skb 
  helpers are not using byte swap instructions.

As such, I think we should just disable classic JIT on ppc64le.


- Naveen
 

^ permalink raw reply	[flat|nested] 34+ messages in thread

end of thread, other threads:[~2016-06-22 14:58 UTC | newest]

Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-06-07 13:32 [PATCH 0/6] eBPF JIT for PPC64 Naveen N. Rao
2016-06-07 13:32 ` [PATCH 1/6] ppc: bpf/jit: Fix/enhance 32-bit Load Immediate implementation Naveen N. Rao
2016-06-07 13:32 ` [PATCH 2/6] ppc: bpf/jit: Optimize 64-bit Immediate loads Naveen N. Rao
2016-06-07 13:32 ` [PATCH 3/6] ppc: bpf/jit: Introduce rotate immediate instructions Naveen N. Rao
2016-06-07 13:32 ` [PATCH 4/6] ppc: bpf/jit: A few cleanups Naveen N. Rao
2016-06-07 13:32 ` [PATCH 5/6] ppc: bpf/jit: Isolate classic BPF JIT specifics into a separate header Naveen N. Rao
2016-06-07 13:32 ` [PATCH 6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF Naveen N. Rao
2016-06-07 22:56   ` Alexei Starovoitov
2016-06-08 17:17     ` Naveen N. Rao
2016-06-09  3:19   ` Nilay Vaish
2016-06-09  6:07     ` Naveen N. Rao
2016-06-17 12:53   ` [6/6] " Michael Ellerman
2016-06-17 13:00     ` [PATCH] ppc: Fix BPF JIT for ABIv2 Thadeu Lima de Souza Cascardo
2016-06-19 17:49       ` Naveen N. Rao
2016-06-20 18:56         ` Thadeu Lima de Souza Cascardo
2016-06-21  8:58           ` Naveen N. Rao
2016-06-21 11:15             ` Michael Ellerman
2016-06-21 14:47               ` Thadeu Lima de Souza Cascardo
2016-06-21 15:45                 ` Alexei Starovoitov
2016-06-22  4:06                   ` Michael Ellerman
2016-06-22  7:12                 ` Naveen N. Rao
2016-06-22 14:57                   ` Naveen N. Rao
2016-06-22  5:20             ` Michael Ellerman
2016-06-22  5:18       ` Michael Ellerman
2016-06-19 17:36     ` [6/6] ppc: ebpf/jit: Implement JIT compiler for extended BPF Naveen N. Rao
2016-06-20 23:38       ` Michael Ellerman
2016-06-21  6:58         ` Naveen N. Rao
2016-06-21 11:04           ` Michael Ellerman
2016-06-22  7:06             ` Naveen N. Rao
2016-06-22 10:37   ` [PATCH 6/6] " Michael Ellerman
2016-06-22 12:32     ` Naveen N. Rao
2016-06-11  5:47 ` [PATCH 0/6] eBPF JIT for PPC64 David Miller
2016-06-13  5:40   ` Naveen N. Rao
2016-06-18  4:57     ` mpe

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).