* [PATCH bpf-next v3 0/4] bpf, arm64: Optimize BPF store/load using
@ 2022-03-16 16:26 ` Xu Kuohai
0 siblings, 0 replies; 11+ messages in thread
From: Xu Kuohai @ 2022-03-16 16:26 UTC (permalink / raw)
To: bpf, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Daniel Borkmann,
Alexei Starovoitov, Zi Shen Lim, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Julien Thierry, Mark Rutland, Hou Tao, Fuad Tabba,
James Morse
The current BPF store/load instruction is translated by the JIT into two
instructions. The first instruction moves the immediate offset into a
temporary register. The second instruction uses this temporary register
to do the real store/load.
In fact, arm64 supports addressing with immediate offsets. So This series
introduces optimization that uses arm64 str/ldr instruction with immediate
offset when the offset fits.
Example of generated instuction for r2 = *(u64 *)(r1 + 0):
Without optimization:
mov x10, 0
ldr x1, [x0, x10]
With optimization:
ldr x1, [x0, 0]
For the following bpftrace command:
bpftrace -e 'kprobe:do_sys_open { printf("opening: %s\n", str(arg1)); }'
Without this series, jited code(fragment):
0: bti c
4: stp x29, x30, [sp, #-16]!
8: mov x29, sp
c: stp x19, x20, [sp, #-16]!
10: stp x21, x22, [sp, #-16]!
14: stp x25, x26, [sp, #-16]!
18: mov x25, sp
1c: mov x26, #0x0 // #0
20: bti j
24: sub sp, sp, #0x90
28: add x19, x0, #0x0
2c: mov x0, #0x0 // #0
30: mov x10, #0xffffffffffffff78 // #-136
34: str x0, [x25, x10]
38: mov x10, #0xffffffffffffff80 // #-128
3c: str x0, [x25, x10]
40: mov x10, #0xffffffffffffff88 // #-120
44: str x0, [x25, x10]
48: mov x10, #0xffffffffffffff90 // #-112
4c: str x0, [x25, x10]
50: mov x10, #0xffffffffffffff98 // #-104
54: str x0, [x25, x10]
58: mov x10, #0xffffffffffffffa0 // #-96
5c: str x0, [x25, x10]
60: mov x10, #0xffffffffffffffa8 // #-88
64: str x0, [x25, x10]
68: mov x10, #0xffffffffffffffb0 // #-80
6c: str x0, [x25, x10]
70: mov x10, #0xffffffffffffffb8 // #-72
74: str x0, [x25, x10]
78: mov x10, #0xffffffffffffffc0 // #-64
7c: str x0, [x25, x10]
80: mov x10, #0xffffffffffffffc8 // #-56
84: str x0, [x25, x10]
88: mov x10, #0xffffffffffffffd0 // #-48
8c: str x0, [x25, x10]
90: mov x10, #0xffffffffffffffd8 // #-40
94: str x0, [x25, x10]
98: mov x10, #0xffffffffffffffe0 // #-32
9c: str x0, [x25, x10]
a0: mov x10, #0xffffffffffffffe8 // #-24
a4: str x0, [x25, x10]
a8: mov x10, #0xfffffffffffffff0 // #-16
ac: str x0, [x25, x10]
b0: mov x10, #0xfffffffffffffff8 // #-8
b4: str x0, [x25, x10]
b8: mov x10, #0x8 // #8
bc: ldr x2, [x19, x10]
[...]
With this series, jited code(fragment):
0: bti c
4: stp x29, x30, [sp, #-16]!
8: mov x29, sp
c: stp x19, x20, [sp, #-16]!
10: stp x21, x22, [sp, #-16]!
14: stp x25, x26, [sp, #-16]!
18: stp x27, x28, [sp, #-16]!
1c: mov x25, sp
20: sub x27, x25, #0x88
24: mov x26, #0x0 // #0
28: bti j
2c: sub sp, sp, #0x90
30: add x19, x0, #0x0
34: mov x0, #0x0 // #0
38: str x0, [x27]
3c: str x0, [x27, #8]
40: str x0, [x27, #16]
44: str x0, [x27, #24]
48: str x0, [x27, #32]
4c: str x0, [x27, #40]
50: str x0, [x27, #48]
54: str x0, [x27, #56]
58: str x0, [x27, #64]
5c: str x0, [x27, #72]
60: str x0, [x27, #80]
64: str x0, [x27, #88]
68: str x0, [x27, #96]
6c: str x0, [x27, #104]
70: str x0, [x27, #112]
74: str x0, [x27, #120]
78: str x0, [x27, #128]
7c: ldr x2, [x19, #8]
[...]
Tested with test_bpf on both big-endian and little-endian arm64 qemu:
test_bpf: Summary: 1026 PASSED, 0 FAILED, [1014/1014 JIT'ed]
test_bpf: test_tail_calls: Summary: 8 PASSED, 0 FAILED, [8/8 JIT'ed]
test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED
v2 -> v3:
1. Split the v2 patch into 2 patches, one for arm64 instruction encoder,
the other for BPF JIT
2. Add tests for BPF_LDX/BPF_STX with different offsets
3. Adjust the offset of str/ldr(immediate) to positive number
v1 -> v2:
1. Remove macro definition that causes checkpatch to fail
2. Append result to commit message
Xu Kuohai (4):
arm64: insn: add ldr/str with immediate offset
bpf, arm64: Optimize BPF store/load using str/ldr with immediate offset
bpf/tests: Add tests for BPF_LDX/BPF_STX with different offsets
bpf, arm64: adjust the offset of str/ldr(immediate) to positive number
arch/arm64/include/asm/insn.h | 9 ++
arch/arm64/lib/insn.c | 67 ++++++--
arch/arm64/net/bpf_jit.h | 14 ++
arch/arm64/net/bpf_jit_comp.c | 212 ++++++++++++++++++++++---
lib/test_bpf.c | 285 +++++++++++++++++++++++++++++++++-
5 files changed, 549 insertions(+), 38 deletions(-)
--
2.30.2
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH bpf-next v3 0/4] bpf, arm64: Optimize BPF store/load using
@ 2022-03-16 16:26 ` Xu Kuohai
0 siblings, 0 replies; 11+ messages in thread
From: Xu Kuohai @ 2022-03-16 16:26 UTC (permalink / raw)
To: bpf, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Daniel Borkmann,
Alexei Starovoitov, Zi Shen Lim, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Julien Thierry, Mark Rutland, Hou Tao, Fuad Tabba,
James Morse
The current BPF store/load instruction is translated by the JIT into two
instructions. The first instruction moves the immediate offset into a
temporary register. The second instruction uses this temporary register
to do the real store/load.
In fact, arm64 supports addressing with immediate offsets. So This series
introduces optimization that uses arm64 str/ldr instruction with immediate
offset when the offset fits.
Example of generated instuction for r2 = *(u64 *)(r1 + 0):
Without optimization:
mov x10, 0
ldr x1, [x0, x10]
With optimization:
ldr x1, [x0, 0]
For the following bpftrace command:
bpftrace -e 'kprobe:do_sys_open { printf("opening: %s\n", str(arg1)); }'
Without this series, jited code(fragment):
0: bti c
4: stp x29, x30, [sp, #-16]!
8: mov x29, sp
c: stp x19, x20, [sp, #-16]!
10: stp x21, x22, [sp, #-16]!
14: stp x25, x26, [sp, #-16]!
18: mov x25, sp
1c: mov x26, #0x0 // #0
20: bti j
24: sub sp, sp, #0x90
28: add x19, x0, #0x0
2c: mov x0, #0x0 // #0
30: mov x10, #0xffffffffffffff78 // #-136
34: str x0, [x25, x10]
38: mov x10, #0xffffffffffffff80 // #-128
3c: str x0, [x25, x10]
40: mov x10, #0xffffffffffffff88 // #-120
44: str x0, [x25, x10]
48: mov x10, #0xffffffffffffff90 // #-112
4c: str x0, [x25, x10]
50: mov x10, #0xffffffffffffff98 // #-104
54: str x0, [x25, x10]
58: mov x10, #0xffffffffffffffa0 // #-96
5c: str x0, [x25, x10]
60: mov x10, #0xffffffffffffffa8 // #-88
64: str x0, [x25, x10]
68: mov x10, #0xffffffffffffffb0 // #-80
6c: str x0, [x25, x10]
70: mov x10, #0xffffffffffffffb8 // #-72
74: str x0, [x25, x10]
78: mov x10, #0xffffffffffffffc0 // #-64
7c: str x0, [x25, x10]
80: mov x10, #0xffffffffffffffc8 // #-56
84: str x0, [x25, x10]
88: mov x10, #0xffffffffffffffd0 // #-48
8c: str x0, [x25, x10]
90: mov x10, #0xffffffffffffffd8 // #-40
94: str x0, [x25, x10]
98: mov x10, #0xffffffffffffffe0 // #-32
9c: str x0, [x25, x10]
a0: mov x10, #0xffffffffffffffe8 // #-24
a4: str x0, [x25, x10]
a8: mov x10, #0xfffffffffffffff0 // #-16
ac: str x0, [x25, x10]
b0: mov x10, #0xfffffffffffffff8 // #-8
b4: str x0, [x25, x10]
b8: mov x10, #0x8 // #8
bc: ldr x2, [x19, x10]
[...]
With this series, jited code(fragment):
0: bti c
4: stp x29, x30, [sp, #-16]!
8: mov x29, sp
c: stp x19, x20, [sp, #-16]!
10: stp x21, x22, [sp, #-16]!
14: stp x25, x26, [sp, #-16]!
18: stp x27, x28, [sp, #-16]!
1c: mov x25, sp
20: sub x27, x25, #0x88
24: mov x26, #0x0 // #0
28: bti j
2c: sub sp, sp, #0x90
30: add x19, x0, #0x0
34: mov x0, #0x0 // #0
38: str x0, [x27]
3c: str x0, [x27, #8]
40: str x0, [x27, #16]
44: str x0, [x27, #24]
48: str x0, [x27, #32]
4c: str x0, [x27, #40]
50: str x0, [x27, #48]
54: str x0, [x27, #56]
58: str x0, [x27, #64]
5c: str x0, [x27, #72]
60: str x0, [x27, #80]
64: str x0, [x27, #88]
68: str x0, [x27, #96]
6c: str x0, [x27, #104]
70: str x0, [x27, #112]
74: str x0, [x27, #120]
78: str x0, [x27, #128]
7c: ldr x2, [x19, #8]
[...]
Tested with test_bpf on both big-endian and little-endian arm64 qemu:
test_bpf: Summary: 1026 PASSED, 0 FAILED, [1014/1014 JIT'ed]
test_bpf: test_tail_calls: Summary: 8 PASSED, 0 FAILED, [8/8 JIT'ed]
test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED
v2 -> v3:
1. Split the v2 patch into 2 patches, one for arm64 instruction encoder,
the other for BPF JIT
2. Add tests for BPF_LDX/BPF_STX with different offsets
3. Adjust the offset of str/ldr(immediate) to positive number
v1 -> v2:
1. Remove macro definition that causes checkpatch to fail
2. Append result to commit message
Xu Kuohai (4):
arm64: insn: add ldr/str with immediate offset
bpf, arm64: Optimize BPF store/load using str/ldr with immediate offset
bpf/tests: Add tests for BPF_LDX/BPF_STX with different offsets
bpf, arm64: adjust the offset of str/ldr(immediate) to positive number
arch/arm64/include/asm/insn.h | 9 ++
arch/arm64/lib/insn.c | 67 ++++++--
arch/arm64/net/bpf_jit.h | 14 ++
arch/arm64/net/bpf_jit_comp.c | 212 ++++++++++++++++++++++---
lib/test_bpf.c | 285 +++++++++++++++++++++++++++++++++-
5 files changed, 549 insertions(+), 38 deletions(-)
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH -next v3 1/4] arm64: insn: add ldr/str with immediate offset
2022-03-16 16:26 ` Xu Kuohai
@ 2022-03-16 16:26 ` Xu Kuohai
-1 siblings, 0 replies; 11+ messages in thread
From: Xu Kuohai @ 2022-03-16 16:26 UTC (permalink / raw)
To: bpf, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Daniel Borkmann,
Alexei Starovoitov, Zi Shen Lim, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Julien Thierry, Mark Rutland, Hou Tao, Fuad Tabba,
James Morse
This patch introduces ldr/str with immediate offset support to simplify
the JIT implementation of BPF LDX/STX instructions on arm64. Although
arm64 ldr/str immediate is available in pre-index, post-index and
unsigned offset forms, the unsigned offset form is sufficient for BPF,
so this patch only adds this type.
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arm64/include/asm/insn.h | 9 +++++
arch/arm64/lib/insn.c | 67 +++++++++++++++++++++++++++--------
2 files changed, 62 insertions(+), 14 deletions(-)
diff --git a/arch/arm64/include/asm/insn.h b/arch/arm64/include/asm/insn.h
index 0b6b31307e68..d507acfdf02d 100644
--- a/arch/arm64/include/asm/insn.h
+++ b/arch/arm64/include/asm/insn.h
@@ -200,6 +200,8 @@ enum aarch64_insn_size_type {
enum aarch64_insn_ldst_type {
AARCH64_INSN_LDST_LOAD_REG_OFFSET,
AARCH64_INSN_LDST_STORE_REG_OFFSET,
+ AARCH64_INSN_LDST_LOAD_IMM_OFFSET,
+ AARCH64_INSN_LDST_STORE_IMM_OFFSET,
AARCH64_INSN_LDST_LOAD_PAIR_PRE_INDEX,
AARCH64_INSN_LDST_STORE_PAIR_PRE_INDEX,
AARCH64_INSN_LDST_LOAD_PAIR_POST_INDEX,
@@ -334,6 +336,7 @@ __AARCH64_INSN_FUNCS(load_pre, 0x3FE00C00, 0x38400C00)
__AARCH64_INSN_FUNCS(store_post, 0x3FE00C00, 0x38000400)
__AARCH64_INSN_FUNCS(load_post, 0x3FE00C00, 0x38400400)
__AARCH64_INSN_FUNCS(str_reg, 0x3FE0EC00, 0x38206800)
+__AARCH64_INSN_FUNCS(str_imm, 0x3FC00000, 0x39000000)
__AARCH64_INSN_FUNCS(ldadd, 0x3F20FC00, 0x38200000)
__AARCH64_INSN_FUNCS(ldclr, 0x3F20FC00, 0x38201000)
__AARCH64_INSN_FUNCS(ldeor, 0x3F20FC00, 0x38202000)
@@ -341,6 +344,7 @@ __AARCH64_INSN_FUNCS(ldset, 0x3F20FC00, 0x38203000)
__AARCH64_INSN_FUNCS(swp, 0x3F20FC00, 0x38208000)
__AARCH64_INSN_FUNCS(cas, 0x3FA07C00, 0x08A07C00)
__AARCH64_INSN_FUNCS(ldr_reg, 0x3FE0EC00, 0x38606800)
+__AARCH64_INSN_FUNCS(ldr_imm, 0x3FC00000, 0x39400000)
__AARCH64_INSN_FUNCS(ldr_lit, 0xBF000000, 0x18000000)
__AARCH64_INSN_FUNCS(ldrsw_lit, 0xFF000000, 0x98000000)
__AARCH64_INSN_FUNCS(exclusive, 0x3F800000, 0x08000000)
@@ -500,6 +504,11 @@ u32 aarch64_insn_gen_load_store_reg(enum aarch64_insn_register reg,
enum aarch64_insn_register offset,
enum aarch64_insn_size_type size,
enum aarch64_insn_ldst_type type);
+u32 aarch64_insn_gen_load_store_imm(enum aarch64_insn_register reg,
+ enum aarch64_insn_register base,
+ unsigned int imm,
+ enum aarch64_insn_size_type size,
+ enum aarch64_insn_ldst_type type);
u32 aarch64_insn_gen_load_store_pair(enum aarch64_insn_register reg1,
enum aarch64_insn_register reg2,
enum aarch64_insn_register base,
diff --git a/arch/arm64/lib/insn.c b/arch/arm64/lib/insn.c
index 5e90887deec4..695d7368fadc 100644
--- a/arch/arm64/lib/insn.c
+++ b/arch/arm64/lib/insn.c
@@ -299,29 +299,24 @@ static u32 aarch64_insn_encode_register(enum aarch64_insn_register_type type,
return insn;
}
+static const u32 aarch64_insn_ldst_size[] = {
+ [AARCH64_INSN_SIZE_8] = 0,
+ [AARCH64_INSN_SIZE_16] = 1,
+ [AARCH64_INSN_SIZE_32] = 2,
+ [AARCH64_INSN_SIZE_64] = 3,
+};
+
static u32 aarch64_insn_encode_ldst_size(enum aarch64_insn_size_type type,
u32 insn)
{
u32 size;
- switch (type) {
- case AARCH64_INSN_SIZE_8:
- size = 0;
- break;
- case AARCH64_INSN_SIZE_16:
- size = 1;
- break;
- case AARCH64_INSN_SIZE_32:
- size = 2;
- break;
- case AARCH64_INSN_SIZE_64:
- size = 3;
- break;
- default:
+ if (type < AARCH64_INSN_SIZE_8 || type > AARCH64_INSN_SIZE_64) {
pr_err("%s: unknown size encoding %d\n", __func__, type);
return AARCH64_BREAK_FAULT;
}
+ size = aarch64_insn_ldst_size[type];
insn &= ~GENMASK(31, 30);
insn |= size << 30;
@@ -504,6 +499,50 @@ u32 aarch64_insn_gen_load_store_reg(enum aarch64_insn_register reg,
offset);
}
+u32 aarch64_insn_gen_load_store_imm(enum aarch64_insn_register reg,
+ enum aarch64_insn_register base,
+ unsigned int imm,
+ enum aarch64_insn_size_type size,
+ enum aarch64_insn_ldst_type type)
+{
+ u32 insn;
+ u32 shift;
+
+ if (size < AARCH64_INSN_SIZE_8 || size > AARCH64_INSN_SIZE_64) {
+ pr_err("%s: unknown size encoding %d\n", __func__, type);
+ return AARCH64_BREAK_FAULT;
+ }
+
+ shift = aarch64_insn_ldst_size[size];
+ if (imm & ~(BIT(12 + shift) - BIT(shift))) {
+ pr_err("%s: invalid imm: %d\n", __func__, imm);
+ return AARCH64_BREAK_FAULT;
+ }
+
+ imm >>= shift;
+
+ switch (type) {
+ case AARCH64_INSN_LDST_LOAD_IMM_OFFSET:
+ insn = aarch64_insn_get_ldr_imm_value();
+ break;
+ case AARCH64_INSN_LDST_STORE_IMM_OFFSET:
+ insn = aarch64_insn_get_str_imm_value();
+ break;
+ default:
+ pr_err("%s: unknown load/store encoding %d\n", __func__, type);
+ return AARCH64_BREAK_FAULT;
+ }
+
+ insn = aarch64_insn_encode_ldst_size(size, insn);
+
+ insn = aarch64_insn_encode_register(AARCH64_INSN_REGTYPE_RT, insn, reg);
+
+ insn = aarch64_insn_encode_register(AARCH64_INSN_REGTYPE_RN, insn,
+ base);
+
+ return aarch64_insn_encode_immediate(AARCH64_INSN_IMM_12, insn, imm);
+}
+
u32 aarch64_insn_gen_load_store_pair(enum aarch64_insn_register reg1,
enum aarch64_insn_register reg2,
enum aarch64_insn_register base,
--
2.30.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH -next v3 1/4] arm64: insn: add ldr/str with immediate offset
@ 2022-03-16 16:26 ` Xu Kuohai
0 siblings, 0 replies; 11+ messages in thread
From: Xu Kuohai @ 2022-03-16 16:26 UTC (permalink / raw)
To: bpf, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Daniel Borkmann,
Alexei Starovoitov, Zi Shen Lim, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Julien Thierry, Mark Rutland, Hou Tao, Fuad Tabba,
James Morse
This patch introduces ldr/str with immediate offset support to simplify
the JIT implementation of BPF LDX/STX instructions on arm64. Although
arm64 ldr/str immediate is available in pre-index, post-index and
unsigned offset forms, the unsigned offset form is sufficient for BPF,
so this patch only adds this type.
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arm64/include/asm/insn.h | 9 +++++
arch/arm64/lib/insn.c | 67 +++++++++++++++++++++++++++--------
2 files changed, 62 insertions(+), 14 deletions(-)
diff --git a/arch/arm64/include/asm/insn.h b/arch/arm64/include/asm/insn.h
index 0b6b31307e68..d507acfdf02d 100644
--- a/arch/arm64/include/asm/insn.h
+++ b/arch/arm64/include/asm/insn.h
@@ -200,6 +200,8 @@ enum aarch64_insn_size_type {
enum aarch64_insn_ldst_type {
AARCH64_INSN_LDST_LOAD_REG_OFFSET,
AARCH64_INSN_LDST_STORE_REG_OFFSET,
+ AARCH64_INSN_LDST_LOAD_IMM_OFFSET,
+ AARCH64_INSN_LDST_STORE_IMM_OFFSET,
AARCH64_INSN_LDST_LOAD_PAIR_PRE_INDEX,
AARCH64_INSN_LDST_STORE_PAIR_PRE_INDEX,
AARCH64_INSN_LDST_LOAD_PAIR_POST_INDEX,
@@ -334,6 +336,7 @@ __AARCH64_INSN_FUNCS(load_pre, 0x3FE00C00, 0x38400C00)
__AARCH64_INSN_FUNCS(store_post, 0x3FE00C00, 0x38000400)
__AARCH64_INSN_FUNCS(load_post, 0x3FE00C00, 0x38400400)
__AARCH64_INSN_FUNCS(str_reg, 0x3FE0EC00, 0x38206800)
+__AARCH64_INSN_FUNCS(str_imm, 0x3FC00000, 0x39000000)
__AARCH64_INSN_FUNCS(ldadd, 0x3F20FC00, 0x38200000)
__AARCH64_INSN_FUNCS(ldclr, 0x3F20FC00, 0x38201000)
__AARCH64_INSN_FUNCS(ldeor, 0x3F20FC00, 0x38202000)
@@ -341,6 +344,7 @@ __AARCH64_INSN_FUNCS(ldset, 0x3F20FC00, 0x38203000)
__AARCH64_INSN_FUNCS(swp, 0x3F20FC00, 0x38208000)
__AARCH64_INSN_FUNCS(cas, 0x3FA07C00, 0x08A07C00)
__AARCH64_INSN_FUNCS(ldr_reg, 0x3FE0EC00, 0x38606800)
+__AARCH64_INSN_FUNCS(ldr_imm, 0x3FC00000, 0x39400000)
__AARCH64_INSN_FUNCS(ldr_lit, 0xBF000000, 0x18000000)
__AARCH64_INSN_FUNCS(ldrsw_lit, 0xFF000000, 0x98000000)
__AARCH64_INSN_FUNCS(exclusive, 0x3F800000, 0x08000000)
@@ -500,6 +504,11 @@ u32 aarch64_insn_gen_load_store_reg(enum aarch64_insn_register reg,
enum aarch64_insn_register offset,
enum aarch64_insn_size_type size,
enum aarch64_insn_ldst_type type);
+u32 aarch64_insn_gen_load_store_imm(enum aarch64_insn_register reg,
+ enum aarch64_insn_register base,
+ unsigned int imm,
+ enum aarch64_insn_size_type size,
+ enum aarch64_insn_ldst_type type);
u32 aarch64_insn_gen_load_store_pair(enum aarch64_insn_register reg1,
enum aarch64_insn_register reg2,
enum aarch64_insn_register base,
diff --git a/arch/arm64/lib/insn.c b/arch/arm64/lib/insn.c
index 5e90887deec4..695d7368fadc 100644
--- a/arch/arm64/lib/insn.c
+++ b/arch/arm64/lib/insn.c
@@ -299,29 +299,24 @@ static u32 aarch64_insn_encode_register(enum aarch64_insn_register_type type,
return insn;
}
+static const u32 aarch64_insn_ldst_size[] = {
+ [AARCH64_INSN_SIZE_8] = 0,
+ [AARCH64_INSN_SIZE_16] = 1,
+ [AARCH64_INSN_SIZE_32] = 2,
+ [AARCH64_INSN_SIZE_64] = 3,
+};
+
static u32 aarch64_insn_encode_ldst_size(enum aarch64_insn_size_type type,
u32 insn)
{
u32 size;
- switch (type) {
- case AARCH64_INSN_SIZE_8:
- size = 0;
- break;
- case AARCH64_INSN_SIZE_16:
- size = 1;
- break;
- case AARCH64_INSN_SIZE_32:
- size = 2;
- break;
- case AARCH64_INSN_SIZE_64:
- size = 3;
- break;
- default:
+ if (type < AARCH64_INSN_SIZE_8 || type > AARCH64_INSN_SIZE_64) {
pr_err("%s: unknown size encoding %d\n", __func__, type);
return AARCH64_BREAK_FAULT;
}
+ size = aarch64_insn_ldst_size[type];
insn &= ~GENMASK(31, 30);
insn |= size << 30;
@@ -504,6 +499,50 @@ u32 aarch64_insn_gen_load_store_reg(enum aarch64_insn_register reg,
offset);
}
+u32 aarch64_insn_gen_load_store_imm(enum aarch64_insn_register reg,
+ enum aarch64_insn_register base,
+ unsigned int imm,
+ enum aarch64_insn_size_type size,
+ enum aarch64_insn_ldst_type type)
+{
+ u32 insn;
+ u32 shift;
+
+ if (size < AARCH64_INSN_SIZE_8 || size > AARCH64_INSN_SIZE_64) {
+ pr_err("%s: unknown size encoding %d\n", __func__, type);
+ return AARCH64_BREAK_FAULT;
+ }
+
+ shift = aarch64_insn_ldst_size[size];
+ if (imm & ~(BIT(12 + shift) - BIT(shift))) {
+ pr_err("%s: invalid imm: %d\n", __func__, imm);
+ return AARCH64_BREAK_FAULT;
+ }
+
+ imm >>= shift;
+
+ switch (type) {
+ case AARCH64_INSN_LDST_LOAD_IMM_OFFSET:
+ insn = aarch64_insn_get_ldr_imm_value();
+ break;
+ case AARCH64_INSN_LDST_STORE_IMM_OFFSET:
+ insn = aarch64_insn_get_str_imm_value();
+ break;
+ default:
+ pr_err("%s: unknown load/store encoding %d\n", __func__, type);
+ return AARCH64_BREAK_FAULT;
+ }
+
+ insn = aarch64_insn_encode_ldst_size(size, insn);
+
+ insn = aarch64_insn_encode_register(AARCH64_INSN_REGTYPE_RT, insn, reg);
+
+ insn = aarch64_insn_encode_register(AARCH64_INSN_REGTYPE_RN, insn,
+ base);
+
+ return aarch64_insn_encode_immediate(AARCH64_INSN_IMM_12, insn, imm);
+}
+
u32 aarch64_insn_gen_load_store_pair(enum aarch64_insn_register reg1,
enum aarch64_insn_register reg2,
enum aarch64_insn_register base,
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH -next v3 2/4] bpf, arm64: Optimize BPF store/load using str/ldr with immediate offset
2022-03-16 16:26 ` Xu Kuohai
@ 2022-03-16 16:26 ` Xu Kuohai
-1 siblings, 0 replies; 11+ messages in thread
From: Xu Kuohai @ 2022-03-16 16:26 UTC (permalink / raw)
To: bpf, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Daniel Borkmann,
Alexei Starovoitov, Zi Shen Lim, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Julien Thierry, Mark Rutland, Hou Tao, Fuad Tabba,
James Morse
The current BPF store/load instruction is translated by the JIT into two
instructions. The first instruction moves the immediate offset into a
temporary register. The second instruction uses this temporary register
to do the real store/load.
In fact, arm64 supports addressing with immediate offsets. So This patch
introduces optimization that uses arm64 str/ldr instruction with immediate
offset when the offset fits.
Example of generated instuction for r2 = *(u64 *)(r1 + 0):
without optimization:
mov x10, 0
ldr x1, [x0, x10]
with optimization:
ldr x1, [x0, 0]
If the offset is negative, or is not aligned correctly, or exceeds max
value, rollback to the use of temporary register.
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arm64/net/bpf_jit.h | 14 ++++
arch/arm64/net/bpf_jit_comp.c | 128 ++++++++++++++++++++++++++++++----
2 files changed, 127 insertions(+), 15 deletions(-)
diff --git a/arch/arm64/net/bpf_jit.h b/arch/arm64/net/bpf_jit.h
index dd59b5ad8fe4..3920213244f0 100644
--- a/arch/arm64/net/bpf_jit.h
+++ b/arch/arm64/net/bpf_jit.h
@@ -66,6 +66,20 @@
#define A64_STR64(Xt, Xn, Xm) A64_LS_REG(Xt, Xn, Xm, 64, STORE)
#define A64_LDR64(Xt, Xn, Xm) A64_LS_REG(Xt, Xn, Xm, 64, LOAD)
+/* Load/store register (immediate offset) */
+#define A64_LS_IMM(Rt, Rn, imm, size, type) \
+ aarch64_insn_gen_load_store_imm(Rt, Rn, imm, \
+ AARCH64_INSN_SIZE_##size, \
+ AARCH64_INSN_LDST_##type##_IMM_OFFSET)
+#define A64_STRBI(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 8, STORE)
+#define A64_LDRBI(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 8, LOAD)
+#define A64_STRHI(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 16, STORE)
+#define A64_LDRHI(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 16, LOAD)
+#define A64_STR32I(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 32, STORE)
+#define A64_LDR32I(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 32, LOAD)
+#define A64_STR64I(Xt, Xn, imm) A64_LS_IMM(Xt, Xn, imm, 64, STORE)
+#define A64_LDR64I(Xt, Xn, imm) A64_LS_IMM(Xt, Xn, imm, 64, LOAD)
+
/* Load/store register pair */
#define A64_LS_PAIR(Rt, Rt2, Rn, offset, ls, type) \
aarch64_insn_gen_load_store_pair(Rt, Rt2, Rn, offset, \
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index e850c69e128c..e56274ce001a 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -191,6 +191,47 @@ static bool is_addsub_imm(u32 imm)
return !(imm & ~0xfff) || !(imm & ~0xfff000);
}
+/*
+ * There are 3 types of AArch64 LDR/STR (immediate) instruction:
+ * Post-index, Pre-index, Unsigned offset.
+ *
+ * For BPF ldr/str, the "unsigned offset" type is sufficient.
+ *
+ * "Unsigned offset" type LDR(immediate) format:
+ *
+ * 3 2 1 0
+ * 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0
+ * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ * |x x|1 1 1 0 0 1 0 1| imm12 | Rn | Rt |
+ * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ * scale
+ *
+ * "Unsigned offset" type STR(immediate) format:
+ * 3 2 1 0
+ * 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0
+ * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ * |x x|1 1 1 0 0 1 0 0| imm12 | Rn | Rt |
+ * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ * scale
+ *
+ * The offset is calculated from imm12 and scale in the following way:
+ *
+ * offset = (u64)imm12 << scale
+ */
+static noinline bool is_lsi_offset(s16 offset, int scale)
+{
+ if (offset < 0)
+ return false;
+
+ if (offset > (0xFFF << scale))
+ return false;
+
+ if (offset & ((1 << scale) - 1))
+ return false;
+
+ return true;
+}
+
/* Tail call offset to jump into */
#if IS_ENABLED(CONFIG_ARM64_BTI_KERNEL)
#define PROLOGUE_OFFSET 8
@@ -971,19 +1012,38 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_LDX | BPF_PROBE_MEM | BPF_W:
case BPF_LDX | BPF_PROBE_MEM | BPF_H:
case BPF_LDX | BPF_PROBE_MEM | BPF_B:
- emit_a64_mov_i(1, tmp, off, ctx);
switch (BPF_SIZE(code)) {
case BPF_W:
- emit(A64_LDR32(dst, src, tmp), ctx);
+ if (is_lsi_offset(off, 2)) {
+ emit(A64_LDR32I(dst, src, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_LDR32(dst, src, tmp), ctx);
+ }
break;
case BPF_H:
- emit(A64_LDRH(dst, src, tmp), ctx);
+ if (is_lsi_offset(off, 1)) {
+ emit(A64_LDRHI(dst, src, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_LDRH(dst, src, tmp), ctx);
+ }
break;
case BPF_B:
- emit(A64_LDRB(dst, src, tmp), ctx);
+ if (is_lsi_offset(off, 0)) {
+ emit(A64_LDRBI(dst, src, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_LDRB(dst, src, tmp), ctx);
+ }
break;
case BPF_DW:
- emit(A64_LDR64(dst, src, tmp), ctx);
+ if (is_lsi_offset(off, 3)) {
+ emit(A64_LDR64I(dst, src, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_LDR64(dst, src, tmp), ctx);
+ }
break;
}
@@ -1011,20 +1071,39 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_ST | BPF_MEM | BPF_B:
case BPF_ST | BPF_MEM | BPF_DW:
/* Load imm to a register then store it */
- emit_a64_mov_i(1, tmp2, off, ctx);
emit_a64_mov_i(1, tmp, imm, ctx);
switch (BPF_SIZE(code)) {
case BPF_W:
- emit(A64_STR32(tmp, dst, tmp2), ctx);
+ if (is_lsi_offset(off, 2)) {
+ emit(A64_STR32I(tmp, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp2, off, ctx);
+ emit(A64_STR32(tmp, dst, tmp2), ctx);
+ }
break;
case BPF_H:
- emit(A64_STRH(tmp, dst, tmp2), ctx);
+ if (is_lsi_offset(off, 1)) {
+ emit(A64_STRHI(tmp, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp2, off, ctx);
+ emit(A64_STRH(tmp, dst, tmp2), ctx);
+ }
break;
case BPF_B:
- emit(A64_STRB(tmp, dst, tmp2), ctx);
+ if (is_lsi_offset(off, 0)) {
+ emit(A64_STRBI(tmp, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp2, off, ctx);
+ emit(A64_STRB(tmp, dst, tmp2), ctx);
+ }
break;
case BPF_DW:
- emit(A64_STR64(tmp, dst, tmp2), ctx);
+ if (is_lsi_offset(off, 3)) {
+ emit(A64_STR64I(tmp, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp2, off, ctx);
+ emit(A64_STR64(tmp, dst, tmp2), ctx);
+ }
break;
}
break;
@@ -1034,19 +1113,38 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_STX | BPF_MEM | BPF_H:
case BPF_STX | BPF_MEM | BPF_B:
case BPF_STX | BPF_MEM | BPF_DW:
- emit_a64_mov_i(1, tmp, off, ctx);
switch (BPF_SIZE(code)) {
case BPF_W:
- emit(A64_STR32(src, dst, tmp), ctx);
+ if (is_lsi_offset(off, 2)) {
+ emit(A64_STR32I(src, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_STR32(src, dst, tmp), ctx);
+ }
break;
case BPF_H:
- emit(A64_STRH(src, dst, tmp), ctx);
+ if (is_lsi_offset(off, 1)) {
+ emit(A64_STRHI(src, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_STRH(src, dst, tmp), ctx);
+ }
break;
case BPF_B:
- emit(A64_STRB(src, dst, tmp), ctx);
+ if (is_lsi_offset(off, 0)) {
+ emit(A64_STRBI(src, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_STRB(src, dst, tmp), ctx);
+ }
break;
case BPF_DW:
- emit(A64_STR64(src, dst, tmp), ctx);
+ if (is_lsi_offset(off, 3)) {
+ emit(A64_STR64I(src, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_STR64(src, dst, tmp), ctx);
+ }
break;
}
break;
--
2.30.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH -next v3 2/4] bpf, arm64: Optimize BPF store/load using str/ldr with immediate offset
@ 2022-03-16 16:26 ` Xu Kuohai
0 siblings, 0 replies; 11+ messages in thread
From: Xu Kuohai @ 2022-03-16 16:26 UTC (permalink / raw)
To: bpf, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Daniel Borkmann,
Alexei Starovoitov, Zi Shen Lim, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Julien Thierry, Mark Rutland, Hou Tao, Fuad Tabba,
James Morse
The current BPF store/load instruction is translated by the JIT into two
instructions. The first instruction moves the immediate offset into a
temporary register. The second instruction uses this temporary register
to do the real store/load.
In fact, arm64 supports addressing with immediate offsets. So This patch
introduces optimization that uses arm64 str/ldr instruction with immediate
offset when the offset fits.
Example of generated instuction for r2 = *(u64 *)(r1 + 0):
without optimization:
mov x10, 0
ldr x1, [x0, x10]
with optimization:
ldr x1, [x0, 0]
If the offset is negative, or is not aligned correctly, or exceeds max
value, rollback to the use of temporary register.
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arm64/net/bpf_jit.h | 14 ++++
arch/arm64/net/bpf_jit_comp.c | 128 ++++++++++++++++++++++++++++++----
2 files changed, 127 insertions(+), 15 deletions(-)
diff --git a/arch/arm64/net/bpf_jit.h b/arch/arm64/net/bpf_jit.h
index dd59b5ad8fe4..3920213244f0 100644
--- a/arch/arm64/net/bpf_jit.h
+++ b/arch/arm64/net/bpf_jit.h
@@ -66,6 +66,20 @@
#define A64_STR64(Xt, Xn, Xm) A64_LS_REG(Xt, Xn, Xm, 64, STORE)
#define A64_LDR64(Xt, Xn, Xm) A64_LS_REG(Xt, Xn, Xm, 64, LOAD)
+/* Load/store register (immediate offset) */
+#define A64_LS_IMM(Rt, Rn, imm, size, type) \
+ aarch64_insn_gen_load_store_imm(Rt, Rn, imm, \
+ AARCH64_INSN_SIZE_##size, \
+ AARCH64_INSN_LDST_##type##_IMM_OFFSET)
+#define A64_STRBI(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 8, STORE)
+#define A64_LDRBI(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 8, LOAD)
+#define A64_STRHI(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 16, STORE)
+#define A64_LDRHI(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 16, LOAD)
+#define A64_STR32I(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 32, STORE)
+#define A64_LDR32I(Wt, Xn, imm) A64_LS_IMM(Wt, Xn, imm, 32, LOAD)
+#define A64_STR64I(Xt, Xn, imm) A64_LS_IMM(Xt, Xn, imm, 64, STORE)
+#define A64_LDR64I(Xt, Xn, imm) A64_LS_IMM(Xt, Xn, imm, 64, LOAD)
+
/* Load/store register pair */
#define A64_LS_PAIR(Rt, Rt2, Rn, offset, ls, type) \
aarch64_insn_gen_load_store_pair(Rt, Rt2, Rn, offset, \
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index e850c69e128c..e56274ce001a 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -191,6 +191,47 @@ static bool is_addsub_imm(u32 imm)
return !(imm & ~0xfff) || !(imm & ~0xfff000);
}
+/*
+ * There are 3 types of AArch64 LDR/STR (immediate) instruction:
+ * Post-index, Pre-index, Unsigned offset.
+ *
+ * For BPF ldr/str, the "unsigned offset" type is sufficient.
+ *
+ * "Unsigned offset" type LDR(immediate) format:
+ *
+ * 3 2 1 0
+ * 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0
+ * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ * |x x|1 1 1 0 0 1 0 1| imm12 | Rn | Rt |
+ * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ * scale
+ *
+ * "Unsigned offset" type STR(immediate) format:
+ * 3 2 1 0
+ * 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0
+ * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ * |x x|1 1 1 0 0 1 0 0| imm12 | Rn | Rt |
+ * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ * scale
+ *
+ * The offset is calculated from imm12 and scale in the following way:
+ *
+ * offset = (u64)imm12 << scale
+ */
+static noinline bool is_lsi_offset(s16 offset, int scale)
+{
+ if (offset < 0)
+ return false;
+
+ if (offset > (0xFFF << scale))
+ return false;
+
+ if (offset & ((1 << scale) - 1))
+ return false;
+
+ return true;
+}
+
/* Tail call offset to jump into */
#if IS_ENABLED(CONFIG_ARM64_BTI_KERNEL)
#define PROLOGUE_OFFSET 8
@@ -971,19 +1012,38 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_LDX | BPF_PROBE_MEM | BPF_W:
case BPF_LDX | BPF_PROBE_MEM | BPF_H:
case BPF_LDX | BPF_PROBE_MEM | BPF_B:
- emit_a64_mov_i(1, tmp, off, ctx);
switch (BPF_SIZE(code)) {
case BPF_W:
- emit(A64_LDR32(dst, src, tmp), ctx);
+ if (is_lsi_offset(off, 2)) {
+ emit(A64_LDR32I(dst, src, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_LDR32(dst, src, tmp), ctx);
+ }
break;
case BPF_H:
- emit(A64_LDRH(dst, src, tmp), ctx);
+ if (is_lsi_offset(off, 1)) {
+ emit(A64_LDRHI(dst, src, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_LDRH(dst, src, tmp), ctx);
+ }
break;
case BPF_B:
- emit(A64_LDRB(dst, src, tmp), ctx);
+ if (is_lsi_offset(off, 0)) {
+ emit(A64_LDRBI(dst, src, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_LDRB(dst, src, tmp), ctx);
+ }
break;
case BPF_DW:
- emit(A64_LDR64(dst, src, tmp), ctx);
+ if (is_lsi_offset(off, 3)) {
+ emit(A64_LDR64I(dst, src, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_LDR64(dst, src, tmp), ctx);
+ }
break;
}
@@ -1011,20 +1071,39 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_ST | BPF_MEM | BPF_B:
case BPF_ST | BPF_MEM | BPF_DW:
/* Load imm to a register then store it */
- emit_a64_mov_i(1, tmp2, off, ctx);
emit_a64_mov_i(1, tmp, imm, ctx);
switch (BPF_SIZE(code)) {
case BPF_W:
- emit(A64_STR32(tmp, dst, tmp2), ctx);
+ if (is_lsi_offset(off, 2)) {
+ emit(A64_STR32I(tmp, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp2, off, ctx);
+ emit(A64_STR32(tmp, dst, tmp2), ctx);
+ }
break;
case BPF_H:
- emit(A64_STRH(tmp, dst, tmp2), ctx);
+ if (is_lsi_offset(off, 1)) {
+ emit(A64_STRHI(tmp, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp2, off, ctx);
+ emit(A64_STRH(tmp, dst, tmp2), ctx);
+ }
break;
case BPF_B:
- emit(A64_STRB(tmp, dst, tmp2), ctx);
+ if (is_lsi_offset(off, 0)) {
+ emit(A64_STRBI(tmp, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp2, off, ctx);
+ emit(A64_STRB(tmp, dst, tmp2), ctx);
+ }
break;
case BPF_DW:
- emit(A64_STR64(tmp, dst, tmp2), ctx);
+ if (is_lsi_offset(off, 3)) {
+ emit(A64_STR64I(tmp, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp2, off, ctx);
+ emit(A64_STR64(tmp, dst, tmp2), ctx);
+ }
break;
}
break;
@@ -1034,19 +1113,38 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_STX | BPF_MEM | BPF_H:
case BPF_STX | BPF_MEM | BPF_B:
case BPF_STX | BPF_MEM | BPF_DW:
- emit_a64_mov_i(1, tmp, off, ctx);
switch (BPF_SIZE(code)) {
case BPF_W:
- emit(A64_STR32(src, dst, tmp), ctx);
+ if (is_lsi_offset(off, 2)) {
+ emit(A64_STR32I(src, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_STR32(src, dst, tmp), ctx);
+ }
break;
case BPF_H:
- emit(A64_STRH(src, dst, tmp), ctx);
+ if (is_lsi_offset(off, 1)) {
+ emit(A64_STRHI(src, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_STRH(src, dst, tmp), ctx);
+ }
break;
case BPF_B:
- emit(A64_STRB(src, dst, tmp), ctx);
+ if (is_lsi_offset(off, 0)) {
+ emit(A64_STRBI(src, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_STRB(src, dst, tmp), ctx);
+ }
break;
case BPF_DW:
- emit(A64_STR64(src, dst, tmp), ctx);
+ if (is_lsi_offset(off, 3)) {
+ emit(A64_STR64I(src, dst, off), ctx);
+ } else {
+ emit_a64_mov_i(1, tmp, off, ctx);
+ emit(A64_STR64(src, dst, tmp), ctx);
+ }
break;
}
break;
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH -next v3 3/4] bpf/tests: Add tests for BPF_LDX/BPF_STX with different offsets
2022-03-16 16:26 ` Xu Kuohai
@ 2022-03-16 16:26 ` Xu Kuohai
-1 siblings, 0 replies; 11+ messages in thread
From: Xu Kuohai @ 2022-03-16 16:26 UTC (permalink / raw)
To: bpf, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Daniel Borkmann,
Alexei Starovoitov, Zi Shen Lim, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Julien Thierry, Mark Rutland, Hou Tao, Fuad Tabba,
James Morse
This patch adds tests to verify the behavior of BPF_LDX/BPF_STX +
BPF_B/BPF_H/BPF_W/BPF_DW with negative offset, small positive offset,
large positive offset, and misaligned offset.
Tested on both big-endian and little-endian arm64 qemu, result:
test_bpf: Summary: 1026 PASSED, 0 FAILED, [1014/1014 JIT'ed]
test_bpf: test_tail_calls: Summary: 8 PASSED, 0 FAILED, [8/8 JIT'ed]
test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
lib/test_bpf.c | 285 ++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 281 insertions(+), 4 deletions(-)
diff --git a/lib/test_bpf.c b/lib/test_bpf.c
index 0c5cb2d6436a..e6d862294f23 100644
--- a/lib/test_bpf.c
+++ b/lib/test_bpf.c
@@ -53,6 +53,7 @@
#define FLAG_EXPECTED_FAIL BIT(1)
#define FLAG_SKB_FRAG BIT(2)
#define FLAG_VERIFIER_ZEXT BIT(3)
+#define FLAG_LARGE_MEM BIT(4)
enum {
CLASSIC = BIT(6), /* Old BPF instructions only. */
@@ -7838,7 +7839,7 @@ static struct bpf_test tests[] = {
},
/* BPF_LDX_MEM B/H/W/DW */
{
- "BPF_LDX_MEM | BPF_B",
+ "BPF_LDX_MEM | BPF_B, base",
.u.insns_int = {
BPF_LD_IMM64(R1, 0x0102030405060708ULL),
BPF_LD_IMM64(R2, 0x0000000000000008ULL),
@@ -7878,7 +7879,56 @@ static struct bpf_test tests[] = {
.stack_depth = 8,
},
{
- "BPF_LDX_MEM | BPF_H",
+ "BPF_LDX_MEM | BPF_B, negative offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000000088ULL),
+ BPF_ALU64_IMM(BPF_ADD, R1, 512),
+ BPF_STX_MEM(BPF_B, R1, R2, -256),
+ BPF_LDX_MEM(BPF_B, R0, R1, -256),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_B, small positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000000088ULL),
+ BPF_STX_MEM(BPF_B, R1, R2, 256),
+ BPF_LDX_MEM(BPF_B, R0, R1, 256),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_B, large positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000000088ULL),
+ BPF_STX_MEM(BPF_B, R1, R2, 4096),
+ BPF_LDX_MEM(BPF_B, R0, R1, 4096),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 4096 + 16, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_H, base",
.u.insns_int = {
BPF_LD_IMM64(R1, 0x0102030405060708ULL),
BPF_LD_IMM64(R2, 0x0000000000000708ULL),
@@ -7918,7 +7968,72 @@ static struct bpf_test tests[] = {
.stack_depth = 8,
},
{
- "BPF_LDX_MEM | BPF_W",
+ "BPF_LDX_MEM | BPF_H, negative offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000008788ULL),
+ BPF_ALU64_IMM(BPF_ADD, R1, 512),
+ BPF_STX_MEM(BPF_H, R1, R2, -256),
+ BPF_LDX_MEM(BPF_H, R0, R1, -256),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_H, small positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000008788ULL),
+ BPF_STX_MEM(BPF_H, R1, R2, 256),
+ BPF_LDX_MEM(BPF_H, R0, R1, 256),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_H, large positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000008788ULL),
+ BPF_STX_MEM(BPF_H, R1, R2, 8192),
+ BPF_LDX_MEM(BPF_H, R0, R1, 8192),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 8192 + 16, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_H, misaligned offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000008788ULL),
+ BPF_STX_MEM(BPF_H, R1, R2, 13),
+ BPF_LDX_MEM(BPF_H, R0, R1, 13),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 32, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_W, base",
.u.insns_int = {
BPF_LD_IMM64(R1, 0x0102030405060708ULL),
BPF_LD_IMM64(R2, 0x0000000005060708ULL),
@@ -7957,6 +8072,162 @@ static struct bpf_test tests[] = {
{ { 0, 0 } },
.stack_depth = 8,
},
+ {
+ "BPF_LDX_MEM | BPF_W, negative offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000085868788ULL),
+ BPF_ALU64_IMM(BPF_ADD, R1, 512),
+ BPF_STX_MEM(BPF_W, R1, R2, -256),
+ BPF_LDX_MEM(BPF_W, R0, R1, -256),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_W, small positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000085868788ULL),
+ BPF_STX_MEM(BPF_W, R1, R2, 256),
+ BPF_LDX_MEM(BPF_W, R0, R1, 256),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_W, large positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000085868788ULL),
+ BPF_STX_MEM(BPF_W, R1, R2, 16384),
+ BPF_LDX_MEM(BPF_W, R0, R1, 16384),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 16384 + 16, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_W, misaligned positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000085868788ULL),
+ BPF_STX_MEM(BPF_W, R1, R2, 13),
+ BPF_LDX_MEM(BPF_W, R0, R1, 13),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 32, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_DW, base",
+ .u.insns_int = {
+ BPF_LD_IMM64(R1, 0x0102030405060708ULL),
+ BPF_STX_MEM(BPF_DW, R10, R1, -8),
+ BPF_LDX_MEM(BPF_DW, R0, R10, -8),
+ BPF_JMP_REG(BPF_JNE, R0, R1, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL,
+ { },
+ { { 0, 0 } },
+ .stack_depth = 8,
+ },
+ {
+ "BPF_LDX_MEM | BPF_DW, MSB set",
+ .u.insns_int = {
+ BPF_LD_IMM64(R1, 0x8182838485868788ULL),
+ BPF_STX_MEM(BPF_DW, R10, R1, -8),
+ BPF_LDX_MEM(BPF_DW, R0, R10, -8),
+ BPF_JMP_REG(BPF_JNE, R0, R1, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL,
+ { },
+ { { 0, 0 } },
+ .stack_depth = 8,
+ },
+ {
+ "BPF_LDX_MEM | BPF_DW, negative offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_ALU64_IMM(BPF_ADD, R1, 512),
+ BPF_STX_MEM(BPF_DW, R1, R2, -256),
+ BPF_LDX_MEM(BPF_DW, R0, R1, -256),
+ BPF_JMP_REG(BPF_JNE, R0, R2, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_DW, small positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_STX_MEM(BPF_DW, R1, R2, 256),
+ BPF_LDX_MEM(BPF_DW, R0, R1, 256),
+ BPF_JMP_REG(BPF_JNE, R0, R2, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 8,
+ },
+ {
+ "BPF_LDX_MEM | BPF_DW, large positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_STX_MEM(BPF_DW, R1, R2, 32768),
+ BPF_LDX_MEM(BPF_DW, R0, R1, 32768),
+ BPF_JMP_REG(BPF_JNE, R0, R2, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 32768 + 16, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_DW, misaligned positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_STX_MEM(BPF_DW, R1, R2, 13),
+ BPF_LDX_MEM(BPF_DW, R0, R1, 13),
+ BPF_JMP_REG(BPF_JNE, R0, R2, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 32, 0 } },
+ .stack_depth = 0,
+ },
/* BPF_STX_MEM B/H/W/DW */
{
"BPF_STX_MEM | BPF_B",
@@ -14094,6 +14365,9 @@ static void *generate_test_data(struct bpf_test *test, int sub)
if (test->aux & FLAG_NO_DATA)
return NULL;
+ if (test->aux & FLAG_LARGE_MEM)
+ return kmalloc(test->test[sub].data_size, GFP_KERNEL);
+
/* Test case expects an skb, so populate one. Various
* subtests generate skbs of different sizes based on
* the same data.
@@ -14137,7 +14411,10 @@ static void release_test_data(const struct bpf_test *test, void *data)
if (test->aux & FLAG_NO_DATA)
return;
- kfree_skb(data);
+ if (test->aux & FLAG_LARGE_MEM)
+ kfree(data);
+ else
+ kfree_skb(data);
}
static int filter_length(int which)
--
2.30.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH -next v3 3/4] bpf/tests: Add tests for BPF_LDX/BPF_STX with different offsets
@ 2022-03-16 16:26 ` Xu Kuohai
0 siblings, 0 replies; 11+ messages in thread
From: Xu Kuohai @ 2022-03-16 16:26 UTC (permalink / raw)
To: bpf, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Daniel Borkmann,
Alexei Starovoitov, Zi Shen Lim, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Julien Thierry, Mark Rutland, Hou Tao, Fuad Tabba,
James Morse
This patch adds tests to verify the behavior of BPF_LDX/BPF_STX +
BPF_B/BPF_H/BPF_W/BPF_DW with negative offset, small positive offset,
large positive offset, and misaligned offset.
Tested on both big-endian and little-endian arm64 qemu, result:
test_bpf: Summary: 1026 PASSED, 0 FAILED, [1014/1014 JIT'ed]
test_bpf: test_tail_calls: Summary: 8 PASSED, 0 FAILED, [8/8 JIT'ed]
test_bpf: test_skb_segment: Summary: 2 PASSED, 0 FAILED
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
lib/test_bpf.c | 285 ++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 281 insertions(+), 4 deletions(-)
diff --git a/lib/test_bpf.c b/lib/test_bpf.c
index 0c5cb2d6436a..e6d862294f23 100644
--- a/lib/test_bpf.c
+++ b/lib/test_bpf.c
@@ -53,6 +53,7 @@
#define FLAG_EXPECTED_FAIL BIT(1)
#define FLAG_SKB_FRAG BIT(2)
#define FLAG_VERIFIER_ZEXT BIT(3)
+#define FLAG_LARGE_MEM BIT(4)
enum {
CLASSIC = BIT(6), /* Old BPF instructions only. */
@@ -7838,7 +7839,7 @@ static struct bpf_test tests[] = {
},
/* BPF_LDX_MEM B/H/W/DW */
{
- "BPF_LDX_MEM | BPF_B",
+ "BPF_LDX_MEM | BPF_B, base",
.u.insns_int = {
BPF_LD_IMM64(R1, 0x0102030405060708ULL),
BPF_LD_IMM64(R2, 0x0000000000000008ULL),
@@ -7878,7 +7879,56 @@ static struct bpf_test tests[] = {
.stack_depth = 8,
},
{
- "BPF_LDX_MEM | BPF_H",
+ "BPF_LDX_MEM | BPF_B, negative offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000000088ULL),
+ BPF_ALU64_IMM(BPF_ADD, R1, 512),
+ BPF_STX_MEM(BPF_B, R1, R2, -256),
+ BPF_LDX_MEM(BPF_B, R0, R1, -256),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_B, small positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000000088ULL),
+ BPF_STX_MEM(BPF_B, R1, R2, 256),
+ BPF_LDX_MEM(BPF_B, R0, R1, 256),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_B, large positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000000088ULL),
+ BPF_STX_MEM(BPF_B, R1, R2, 4096),
+ BPF_LDX_MEM(BPF_B, R0, R1, 4096),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 4096 + 16, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_H, base",
.u.insns_int = {
BPF_LD_IMM64(R1, 0x0102030405060708ULL),
BPF_LD_IMM64(R2, 0x0000000000000708ULL),
@@ -7918,7 +7968,72 @@ static struct bpf_test tests[] = {
.stack_depth = 8,
},
{
- "BPF_LDX_MEM | BPF_W",
+ "BPF_LDX_MEM | BPF_H, negative offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000008788ULL),
+ BPF_ALU64_IMM(BPF_ADD, R1, 512),
+ BPF_STX_MEM(BPF_H, R1, R2, -256),
+ BPF_LDX_MEM(BPF_H, R0, R1, -256),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_H, small positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000008788ULL),
+ BPF_STX_MEM(BPF_H, R1, R2, 256),
+ BPF_LDX_MEM(BPF_H, R0, R1, 256),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_H, large positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000008788ULL),
+ BPF_STX_MEM(BPF_H, R1, R2, 8192),
+ BPF_LDX_MEM(BPF_H, R0, R1, 8192),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 8192 + 16, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_H, misaligned offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000000008788ULL),
+ BPF_STX_MEM(BPF_H, R1, R2, 13),
+ BPF_LDX_MEM(BPF_H, R0, R1, 13),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 32, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_W, base",
.u.insns_int = {
BPF_LD_IMM64(R1, 0x0102030405060708ULL),
BPF_LD_IMM64(R2, 0x0000000005060708ULL),
@@ -7957,6 +8072,162 @@ static struct bpf_test tests[] = {
{ { 0, 0 } },
.stack_depth = 8,
},
+ {
+ "BPF_LDX_MEM | BPF_W, negative offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000085868788ULL),
+ BPF_ALU64_IMM(BPF_ADD, R1, 512),
+ BPF_STX_MEM(BPF_W, R1, R2, -256),
+ BPF_LDX_MEM(BPF_W, R0, R1, -256),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_W, small positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000085868788ULL),
+ BPF_STX_MEM(BPF_W, R1, R2, 256),
+ BPF_LDX_MEM(BPF_W, R0, R1, 256),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_W, large positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000085868788ULL),
+ BPF_STX_MEM(BPF_W, R1, R2, 16384),
+ BPF_LDX_MEM(BPF_W, R0, R1, 16384),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 16384 + 16, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_W, misaligned positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_LD_IMM64(R3, 0x0000000085868788ULL),
+ BPF_STX_MEM(BPF_W, R1, R2, 13),
+ BPF_LDX_MEM(BPF_W, R0, R1, 13),
+ BPF_JMP_REG(BPF_JNE, R0, R3, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 32, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_DW, base",
+ .u.insns_int = {
+ BPF_LD_IMM64(R1, 0x0102030405060708ULL),
+ BPF_STX_MEM(BPF_DW, R10, R1, -8),
+ BPF_LDX_MEM(BPF_DW, R0, R10, -8),
+ BPF_JMP_REG(BPF_JNE, R0, R1, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL,
+ { },
+ { { 0, 0 } },
+ .stack_depth = 8,
+ },
+ {
+ "BPF_LDX_MEM | BPF_DW, MSB set",
+ .u.insns_int = {
+ BPF_LD_IMM64(R1, 0x8182838485868788ULL),
+ BPF_STX_MEM(BPF_DW, R10, R1, -8),
+ BPF_LDX_MEM(BPF_DW, R0, R10, -8),
+ BPF_JMP_REG(BPF_JNE, R0, R1, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL,
+ { },
+ { { 0, 0 } },
+ .stack_depth = 8,
+ },
+ {
+ "BPF_LDX_MEM | BPF_DW, negative offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_ALU64_IMM(BPF_ADD, R1, 512),
+ BPF_STX_MEM(BPF_DW, R1, R2, -256),
+ BPF_LDX_MEM(BPF_DW, R0, R1, -256),
+ BPF_JMP_REG(BPF_JNE, R0, R2, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_DW, small positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_STX_MEM(BPF_DW, R1, R2, 256),
+ BPF_LDX_MEM(BPF_DW, R0, R1, 256),
+ BPF_JMP_REG(BPF_JNE, R0, R2, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 512, 0 } },
+ .stack_depth = 8,
+ },
+ {
+ "BPF_LDX_MEM | BPF_DW, large positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_STX_MEM(BPF_DW, R1, R2, 32768),
+ BPF_LDX_MEM(BPF_DW, R0, R1, 32768),
+ BPF_JMP_REG(BPF_JNE, R0, R2, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 32768 + 16, 0 } },
+ .stack_depth = 0,
+ },
+ {
+ "BPF_LDX_MEM | BPF_DW, misaligned positive offset",
+ .u.insns_int = {
+ BPF_LD_IMM64(R2, 0x8182838485868788ULL),
+ BPF_STX_MEM(BPF_DW, R1, R2, 13),
+ BPF_LDX_MEM(BPF_DW, R0, R1, 13),
+ BPF_JMP_REG(BPF_JNE, R0, R2, 1),
+ BPF_ALU64_IMM(BPF_MOV, R0, 0),
+ BPF_EXIT_INSN(),
+ },
+ INTERNAL | FLAG_LARGE_MEM,
+ { },
+ { { 32, 0 } },
+ .stack_depth = 0,
+ },
/* BPF_STX_MEM B/H/W/DW */
{
"BPF_STX_MEM | BPF_B",
@@ -14094,6 +14365,9 @@ static void *generate_test_data(struct bpf_test *test, int sub)
if (test->aux & FLAG_NO_DATA)
return NULL;
+ if (test->aux & FLAG_LARGE_MEM)
+ return kmalloc(test->test[sub].data_size, GFP_KERNEL);
+
/* Test case expects an skb, so populate one. Various
* subtests generate skbs of different sizes based on
* the same data.
@@ -14137,7 +14411,10 @@ static void release_test_data(const struct bpf_test *test, void *data)
if (test->aux & FLAG_NO_DATA)
return;
- kfree_skb(data);
+ if (test->aux & FLAG_LARGE_MEM)
+ kfree(data);
+ else
+ kfree_skb(data);
}
static int filter_length(int which)
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH -next v3 4/4] bpf, arm64: adjust the offset of str/ldr(immediate) to positive number
2022-03-16 16:26 ` Xu Kuohai
@ 2022-03-16 16:26 ` Xu Kuohai
-1 siblings, 0 replies; 11+ messages in thread
From: Xu Kuohai @ 2022-03-16 16:26 UTC (permalink / raw)
To: bpf, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Daniel Borkmann,
Alexei Starovoitov, Zi Shen Lim, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Julien Thierry, Mark Rutland, Hou Tao, Fuad Tabba,
James Morse
The BPF STX/LDX instruction uses offset relative to the FP to address
stack space. Since the BPF_FP locates at the top of the frame, the offset
is usually a negative number. However, arm64 str/ldr immediate instruction
requires that offset be a positive number. Therefore, this patch tries to
convert the offsets.
The method is to find the negative offset furthest from the FP firstly.
Then add it to the FP, calculate a bottom position, called FPB, and then
adjust the offsets in other STR/LDX instructions relative to FPB.
FPB is saved using the callee-saved register x27 of arm64 which is not
used yet.
Before adjusting the offset, the patch checks every instruction to ensure
that the FP does not change in run-time. If the FP may change, no offset
is adjusted.
For example, for the following bpftrace command:
bpftrace -e 'kprobe:do_sys_open { printf("opening: %s\n", str(arg1)); }'
Without this patch, jited code(fragment):
0: bti c
4: stp x29, x30, [sp, #-16]!
8: mov x29, sp
c: stp x19, x20, [sp, #-16]!
10: stp x21, x22, [sp, #-16]!
14: stp x25, x26, [sp, #-16]!
18: mov x25, sp
1c: mov x26, #0x0 // #0
20: bti j
24: sub sp, sp, #0x90
28: add x19, x0, #0x0
2c: mov x0, #0x0 // #0
30: mov x10, #0xffffffffffffff78 // #-136
34: str x0, [x25, x10]
38: mov x10, #0xffffffffffffff80 // #-128
3c: str x0, [x25, x10]
40: mov x10, #0xffffffffffffff88 // #-120
44: str x0, [x25, x10]
48: mov x10, #0xffffffffffffff90 // #-112
4c: str x0, [x25, x10]
50: mov x10, #0xffffffffffffff98 // #-104
54: str x0, [x25, x10]
58: mov x10, #0xffffffffffffffa0 // #-96
5c: str x0, [x25, x10]
60: mov x10, #0xffffffffffffffa8 // #-88
64: str x0, [x25, x10]
68: mov x10, #0xffffffffffffffb0 // #-80
6c: str x0, [x25, x10]
70: mov x10, #0xffffffffffffffb8 // #-72
74: str x0, [x25, x10]
78: mov x10, #0xffffffffffffffc0 // #-64
7c: str x0, [x25, x10]
80: mov x10, #0xffffffffffffffc8 // #-56
84: str x0, [x25, x10]
88: mov x10, #0xffffffffffffffd0 // #-48
8c: str x0, [x25, x10]
90: mov x10, #0xffffffffffffffd8 // #-40
94: str x0, [x25, x10]
98: mov x10, #0xffffffffffffffe0 // #-32
9c: str x0, [x25, x10]
a0: mov x10, #0xffffffffffffffe8 // #-24
a4: str x0, [x25, x10]
a8: mov x10, #0xfffffffffffffff0 // #-16
ac: str x0, [x25, x10]
b0: mov x10, #0xfffffffffffffff8 // #-8
b4: str x0, [x25, x10]
b8: mov x10, #0x8 // #8
bc: ldr x2, [x19, x10]
[...]
With this patch, jited code(fragment):
0: bti c
4: stp x29, x30, [sp, #-16]!
8: mov x29, sp
c: stp x19, x20, [sp, #-16]!
10: stp x21, x22, [sp, #-16]!
14: stp x25, x26, [sp, #-16]!
18: stp x27, x28, [sp, #-16]!
1c: mov x25, sp
20: sub x27, x25, #0x88
24: mov x26, #0x0 // #0
28: bti j
2c: sub sp, sp, #0x90
30: add x19, x0, #0x0
34: mov x0, #0x0 // #0
38: str x0, [x27]
3c: str x0, [x27, #8]
40: str x0, [x27, #16]
44: str x0, [x27, #24]
48: str x0, [x27, #32]
4c: str x0, [x27, #40]
50: str x0, [x27, #48]
54: str x0, [x27, #56]
58: str x0, [x27, #64]
5c: str x0, [x27, #72]
60: str x0, [x27, #80]
64: str x0, [x27, #88]
68: str x0, [x27, #96]
6c: str x0, [x27, #104]
70: str x0, [x27, #112]
74: str x0, [x27, #120]
78: str x0, [x27, #128]
7c: ldr x2, [x19, #8]
[...]
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arm64/net/bpf_jit_comp.c | 84 ++++++++++++++++++++++++++++++++---
1 file changed, 79 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index e56274ce001a..9e1fd7a1e08f 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -26,6 +26,7 @@
#define TMP_REG_2 (MAX_BPF_JIT_REG + 1)
#define TCALL_CNT (MAX_BPF_JIT_REG + 2)
#define TMP_REG_3 (MAX_BPF_JIT_REG + 3)
+#define FP_BOTTOM (MAX_BPF_JIT_REG + 4)
#define check_imm(bits, imm) do { \
if ((((imm) > 0) && ((imm) >> (bits))) || \
@@ -63,6 +64,7 @@ static const int bpf2a64[] = {
[TCALL_CNT] = A64_R(26),
/* temporary register for blinding constants */
[BPF_REG_AX] = A64_R(9),
+ [FP_BOTTOM] = A64_R(27),
};
struct jit_ctx {
@@ -73,6 +75,7 @@ struct jit_ctx {
int exentry_idx;
__le32 *image;
u32 stack_size;
+ int fpb_offset;
};
static inline void emit(const u32 insn, struct jit_ctx *ctx)
@@ -234,9 +237,9 @@ static noinline bool is_lsi_offset(s16 offset, int scale)
/* Tail call offset to jump into */
#if IS_ENABLED(CONFIG_ARM64_BTI_KERNEL)
-#define PROLOGUE_OFFSET 8
+#define PROLOGUE_OFFSET 10
#else
-#define PROLOGUE_OFFSET 7
+#define PROLOGUE_OFFSET 9
#endif
static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf)
@@ -248,6 +251,7 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf)
const u8 r9 = bpf2a64[BPF_REG_9];
const u8 fp = bpf2a64[BPF_REG_FP];
const u8 tcc = bpf2a64[TCALL_CNT];
+ const u8 fpb = bpf2a64[FP_BOTTOM];
const int idx0 = ctx->idx;
int cur_offset;
@@ -286,9 +290,11 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf)
emit(A64_PUSH(r6, r7, A64_SP), ctx);
emit(A64_PUSH(r8, r9, A64_SP), ctx);
emit(A64_PUSH(fp, tcc, A64_SP), ctx);
+ emit(A64_PUSH(fpb, A64_R(28), A64_SP), ctx);
/* Set up BPF prog stack base register */
emit(A64_MOV(1, fp, A64_SP), ctx);
+ emit(A64_SUB_I(1, fpb, fp, ctx->fpb_offset), ctx);
if (!ebpf_from_cbpf) {
/* Initialize tail_call_cnt */
@@ -553,11 +559,13 @@ static void build_epilogue(struct jit_ctx *ctx)
const u8 r8 = bpf2a64[BPF_REG_8];
const u8 r9 = bpf2a64[BPF_REG_9];
const u8 fp = bpf2a64[BPF_REG_FP];
+ const u8 fpb = bpf2a64[FP_BOTTOM];
/* We're done with BPF stack */
emit(A64_ADD_I(1, A64_SP, A64_SP, ctx->stack_size), ctx);
/* Restore fs (x25) and x26 */
+ emit(A64_POP(fpb, A64_R(28), A64_SP), ctx);
emit(A64_POP(fp, A64_R(26), A64_SP), ctx);
/* Restore callee-saved register */
@@ -645,12 +653,14 @@ static int add_exception_handler(const struct bpf_insn *insn,
static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
bool extra_pass)
{
+ u8 dst = bpf2a64[insn->dst_reg];
+ u8 src = bpf2a64[insn->src_reg];
+ s16 off = insn->off;
+ const u8 fp = bpf2a64[BPF_REG_FP];
const u8 code = insn->code;
- const u8 dst = bpf2a64[insn->dst_reg];
- const u8 src = bpf2a64[insn->src_reg];
const u8 tmp = bpf2a64[TMP_REG_1];
const u8 tmp2 = bpf2a64[TMP_REG_2];
- const s16 off = insn->off;
+ const u8 fpb = bpf2a64[FP_BOTTOM];
const s32 imm = insn->imm;
const int i = insn - ctx->prog->insnsi;
const bool is64 = BPF_CLASS(code) == BPF_ALU64 ||
@@ -1012,6 +1022,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_LDX | BPF_PROBE_MEM | BPF_W:
case BPF_LDX | BPF_PROBE_MEM | BPF_H:
case BPF_LDX | BPF_PROBE_MEM | BPF_B:
+ if (ctx->fpb_offset > 0 && src == fp) {
+ src = fpb;
+ off += ctx->fpb_offset;
+ }
switch (BPF_SIZE(code)) {
case BPF_W:
if (is_lsi_offset(off, 2)) {
@@ -1070,6 +1084,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_ST | BPF_MEM | BPF_H:
case BPF_ST | BPF_MEM | BPF_B:
case BPF_ST | BPF_MEM | BPF_DW:
+ if (ctx->fpb_offset > 0 && dst == fp) {
+ dst = fpb;
+ off += ctx->fpb_offset;
+ }
/* Load imm to a register then store it */
emit_a64_mov_i(1, tmp, imm, ctx);
switch (BPF_SIZE(code)) {
@@ -1113,6 +1131,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_STX | BPF_MEM | BPF_H:
case BPF_STX | BPF_MEM | BPF_B:
case BPF_STX | BPF_MEM | BPF_DW:
+ if (ctx->fpb_offset > 0 && dst == fp) {
+ dst = fpb;
+ off += ctx->fpb_offset;
+ }
switch (BPF_SIZE(code)) {
case BPF_W:
if (is_lsi_offset(off, 2)) {
@@ -1167,6 +1189,56 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
return 0;
}
+/*
+ * Return 0 if FP may change at runtime, otherwise find the minimum negative
+ * offset to FP and converts it to positive number.
+ */
+static int find_fpb_offset(struct bpf_prog *prog)
+{
+ int i;
+ int offset = 0;
+
+ for (i = 0; i < prog->len; i++) {
+ const struct bpf_insn *insn = &prog->insnsi[i];
+
+ switch (BPF_CLASS(insn->code)) {
+ case BPF_STX:
+ case BPF_ST:
+ if (BPF_MODE(insn->code) == BPF_ATOMIC) {
+ if ((insn->imm == BPF_XCHG ||
+ insn->imm == (BPF_ADD | BPF_FETCH) ||
+ insn->imm == (BPF_AND | BPF_FETCH) ||
+ insn->imm == (BPF_OR | BPF_FETCH) ||
+ insn->imm == (BPF_XOR | BPF_FETCH)) &&
+ insn->src_reg == BPF_REG_FP) {
+ return 0;
+ }
+ }
+ if (BPF_MODE(insn->code) == BPF_MEM &&
+ insn->dst_reg == BPF_REG_FP) {
+ if (insn->off < offset)
+ offset = insn->off;
+ }
+ break;
+
+ case BPF_JMP32:
+ case BPF_JMP:
+ break;
+
+ case BPF_ALU:
+ case BPF_ALU64:
+ case BPF_LDX:
+ case BPF_LD:
+ default:
+ if (insn->dst_reg == BPF_REG_FP)
+ return 0;
+ }
+ }
+
+ /* safely be converted to a positive 'int', since insn->off is 's16' */
+ return -offset;
+}
+
static int build_body(struct jit_ctx *ctx, bool extra_pass)
{
const struct bpf_prog *prog = ctx->prog;
@@ -1288,6 +1360,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
goto out_off;
}
+ ctx.fpb_offset = find_fpb_offset(prog);
+
/*
* 1. Initial fake pass to compute ctx->idx and ctx->offset.
*
--
2.30.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH -next v3 4/4] bpf, arm64: adjust the offset of str/ldr(immediate) to positive number
@ 2022-03-16 16:26 ` Xu Kuohai
0 siblings, 0 replies; 11+ messages in thread
From: Xu Kuohai @ 2022-03-16 16:26 UTC (permalink / raw)
To: bpf, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Daniel Borkmann,
Alexei Starovoitov, Zi Shen Lim, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Julien Thierry, Mark Rutland, Hou Tao, Fuad Tabba,
James Morse
The BPF STX/LDX instruction uses offset relative to the FP to address
stack space. Since the BPF_FP locates at the top of the frame, the offset
is usually a negative number. However, arm64 str/ldr immediate instruction
requires that offset be a positive number. Therefore, this patch tries to
convert the offsets.
The method is to find the negative offset furthest from the FP firstly.
Then add it to the FP, calculate a bottom position, called FPB, and then
adjust the offsets in other STR/LDX instructions relative to FPB.
FPB is saved using the callee-saved register x27 of arm64 which is not
used yet.
Before adjusting the offset, the patch checks every instruction to ensure
that the FP does not change in run-time. If the FP may change, no offset
is adjusted.
For example, for the following bpftrace command:
bpftrace -e 'kprobe:do_sys_open { printf("opening: %s\n", str(arg1)); }'
Without this patch, jited code(fragment):
0: bti c
4: stp x29, x30, [sp, #-16]!
8: mov x29, sp
c: stp x19, x20, [sp, #-16]!
10: stp x21, x22, [sp, #-16]!
14: stp x25, x26, [sp, #-16]!
18: mov x25, sp
1c: mov x26, #0x0 // #0
20: bti j
24: sub sp, sp, #0x90
28: add x19, x0, #0x0
2c: mov x0, #0x0 // #0
30: mov x10, #0xffffffffffffff78 // #-136
34: str x0, [x25, x10]
38: mov x10, #0xffffffffffffff80 // #-128
3c: str x0, [x25, x10]
40: mov x10, #0xffffffffffffff88 // #-120
44: str x0, [x25, x10]
48: mov x10, #0xffffffffffffff90 // #-112
4c: str x0, [x25, x10]
50: mov x10, #0xffffffffffffff98 // #-104
54: str x0, [x25, x10]
58: mov x10, #0xffffffffffffffa0 // #-96
5c: str x0, [x25, x10]
60: mov x10, #0xffffffffffffffa8 // #-88
64: str x0, [x25, x10]
68: mov x10, #0xffffffffffffffb0 // #-80
6c: str x0, [x25, x10]
70: mov x10, #0xffffffffffffffb8 // #-72
74: str x0, [x25, x10]
78: mov x10, #0xffffffffffffffc0 // #-64
7c: str x0, [x25, x10]
80: mov x10, #0xffffffffffffffc8 // #-56
84: str x0, [x25, x10]
88: mov x10, #0xffffffffffffffd0 // #-48
8c: str x0, [x25, x10]
90: mov x10, #0xffffffffffffffd8 // #-40
94: str x0, [x25, x10]
98: mov x10, #0xffffffffffffffe0 // #-32
9c: str x0, [x25, x10]
a0: mov x10, #0xffffffffffffffe8 // #-24
a4: str x0, [x25, x10]
a8: mov x10, #0xfffffffffffffff0 // #-16
ac: str x0, [x25, x10]
b0: mov x10, #0xfffffffffffffff8 // #-8
b4: str x0, [x25, x10]
b8: mov x10, #0x8 // #8
bc: ldr x2, [x19, x10]
[...]
With this patch, jited code(fragment):
0: bti c
4: stp x29, x30, [sp, #-16]!
8: mov x29, sp
c: stp x19, x20, [sp, #-16]!
10: stp x21, x22, [sp, #-16]!
14: stp x25, x26, [sp, #-16]!
18: stp x27, x28, [sp, #-16]!
1c: mov x25, sp
20: sub x27, x25, #0x88
24: mov x26, #0x0 // #0
28: bti j
2c: sub sp, sp, #0x90
30: add x19, x0, #0x0
34: mov x0, #0x0 // #0
38: str x0, [x27]
3c: str x0, [x27, #8]
40: str x0, [x27, #16]
44: str x0, [x27, #24]
48: str x0, [x27, #32]
4c: str x0, [x27, #40]
50: str x0, [x27, #48]
54: str x0, [x27, #56]
58: str x0, [x27, #64]
5c: str x0, [x27, #72]
60: str x0, [x27, #80]
64: str x0, [x27, #88]
68: str x0, [x27, #96]
6c: str x0, [x27, #104]
70: str x0, [x27, #112]
74: str x0, [x27, #120]
78: str x0, [x27, #128]
7c: ldr x2, [x19, #8]
[...]
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
arch/arm64/net/bpf_jit_comp.c | 84 ++++++++++++++++++++++++++++++++---
1 file changed, 79 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index e56274ce001a..9e1fd7a1e08f 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -26,6 +26,7 @@
#define TMP_REG_2 (MAX_BPF_JIT_REG + 1)
#define TCALL_CNT (MAX_BPF_JIT_REG + 2)
#define TMP_REG_3 (MAX_BPF_JIT_REG + 3)
+#define FP_BOTTOM (MAX_BPF_JIT_REG + 4)
#define check_imm(bits, imm) do { \
if ((((imm) > 0) && ((imm) >> (bits))) || \
@@ -63,6 +64,7 @@ static const int bpf2a64[] = {
[TCALL_CNT] = A64_R(26),
/* temporary register for blinding constants */
[BPF_REG_AX] = A64_R(9),
+ [FP_BOTTOM] = A64_R(27),
};
struct jit_ctx {
@@ -73,6 +75,7 @@ struct jit_ctx {
int exentry_idx;
__le32 *image;
u32 stack_size;
+ int fpb_offset;
};
static inline void emit(const u32 insn, struct jit_ctx *ctx)
@@ -234,9 +237,9 @@ static noinline bool is_lsi_offset(s16 offset, int scale)
/* Tail call offset to jump into */
#if IS_ENABLED(CONFIG_ARM64_BTI_KERNEL)
-#define PROLOGUE_OFFSET 8
+#define PROLOGUE_OFFSET 10
#else
-#define PROLOGUE_OFFSET 7
+#define PROLOGUE_OFFSET 9
#endif
static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf)
@@ -248,6 +251,7 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf)
const u8 r9 = bpf2a64[BPF_REG_9];
const u8 fp = bpf2a64[BPF_REG_FP];
const u8 tcc = bpf2a64[TCALL_CNT];
+ const u8 fpb = bpf2a64[FP_BOTTOM];
const int idx0 = ctx->idx;
int cur_offset;
@@ -286,9 +290,11 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf)
emit(A64_PUSH(r6, r7, A64_SP), ctx);
emit(A64_PUSH(r8, r9, A64_SP), ctx);
emit(A64_PUSH(fp, tcc, A64_SP), ctx);
+ emit(A64_PUSH(fpb, A64_R(28), A64_SP), ctx);
/* Set up BPF prog stack base register */
emit(A64_MOV(1, fp, A64_SP), ctx);
+ emit(A64_SUB_I(1, fpb, fp, ctx->fpb_offset), ctx);
if (!ebpf_from_cbpf) {
/* Initialize tail_call_cnt */
@@ -553,11 +559,13 @@ static void build_epilogue(struct jit_ctx *ctx)
const u8 r8 = bpf2a64[BPF_REG_8];
const u8 r9 = bpf2a64[BPF_REG_9];
const u8 fp = bpf2a64[BPF_REG_FP];
+ const u8 fpb = bpf2a64[FP_BOTTOM];
/* We're done with BPF stack */
emit(A64_ADD_I(1, A64_SP, A64_SP, ctx->stack_size), ctx);
/* Restore fs (x25) and x26 */
+ emit(A64_POP(fpb, A64_R(28), A64_SP), ctx);
emit(A64_POP(fp, A64_R(26), A64_SP), ctx);
/* Restore callee-saved register */
@@ -645,12 +653,14 @@ static int add_exception_handler(const struct bpf_insn *insn,
static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
bool extra_pass)
{
+ u8 dst = bpf2a64[insn->dst_reg];
+ u8 src = bpf2a64[insn->src_reg];
+ s16 off = insn->off;
+ const u8 fp = bpf2a64[BPF_REG_FP];
const u8 code = insn->code;
- const u8 dst = bpf2a64[insn->dst_reg];
- const u8 src = bpf2a64[insn->src_reg];
const u8 tmp = bpf2a64[TMP_REG_1];
const u8 tmp2 = bpf2a64[TMP_REG_2];
- const s16 off = insn->off;
+ const u8 fpb = bpf2a64[FP_BOTTOM];
const s32 imm = insn->imm;
const int i = insn - ctx->prog->insnsi;
const bool is64 = BPF_CLASS(code) == BPF_ALU64 ||
@@ -1012,6 +1022,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_LDX | BPF_PROBE_MEM | BPF_W:
case BPF_LDX | BPF_PROBE_MEM | BPF_H:
case BPF_LDX | BPF_PROBE_MEM | BPF_B:
+ if (ctx->fpb_offset > 0 && src == fp) {
+ src = fpb;
+ off += ctx->fpb_offset;
+ }
switch (BPF_SIZE(code)) {
case BPF_W:
if (is_lsi_offset(off, 2)) {
@@ -1070,6 +1084,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_ST | BPF_MEM | BPF_H:
case BPF_ST | BPF_MEM | BPF_B:
case BPF_ST | BPF_MEM | BPF_DW:
+ if (ctx->fpb_offset > 0 && dst == fp) {
+ dst = fpb;
+ off += ctx->fpb_offset;
+ }
/* Load imm to a register then store it */
emit_a64_mov_i(1, tmp, imm, ctx);
switch (BPF_SIZE(code)) {
@@ -1113,6 +1131,10 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
case BPF_STX | BPF_MEM | BPF_H:
case BPF_STX | BPF_MEM | BPF_B:
case BPF_STX | BPF_MEM | BPF_DW:
+ if (ctx->fpb_offset > 0 && dst == fp) {
+ dst = fpb;
+ off += ctx->fpb_offset;
+ }
switch (BPF_SIZE(code)) {
case BPF_W:
if (is_lsi_offset(off, 2)) {
@@ -1167,6 +1189,56 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
return 0;
}
+/*
+ * Return 0 if FP may change at runtime, otherwise find the minimum negative
+ * offset to FP and converts it to positive number.
+ */
+static int find_fpb_offset(struct bpf_prog *prog)
+{
+ int i;
+ int offset = 0;
+
+ for (i = 0; i < prog->len; i++) {
+ const struct bpf_insn *insn = &prog->insnsi[i];
+
+ switch (BPF_CLASS(insn->code)) {
+ case BPF_STX:
+ case BPF_ST:
+ if (BPF_MODE(insn->code) == BPF_ATOMIC) {
+ if ((insn->imm == BPF_XCHG ||
+ insn->imm == (BPF_ADD | BPF_FETCH) ||
+ insn->imm == (BPF_AND | BPF_FETCH) ||
+ insn->imm == (BPF_OR | BPF_FETCH) ||
+ insn->imm == (BPF_XOR | BPF_FETCH)) &&
+ insn->src_reg == BPF_REG_FP) {
+ return 0;
+ }
+ }
+ if (BPF_MODE(insn->code) == BPF_MEM &&
+ insn->dst_reg == BPF_REG_FP) {
+ if (insn->off < offset)
+ offset = insn->off;
+ }
+ break;
+
+ case BPF_JMP32:
+ case BPF_JMP:
+ break;
+
+ case BPF_ALU:
+ case BPF_ALU64:
+ case BPF_LDX:
+ case BPF_LD:
+ default:
+ if (insn->dst_reg == BPF_REG_FP)
+ return 0;
+ }
+ }
+
+ /* safely be converted to a positive 'int', since insn->off is 's16' */
+ return -offset;
+}
+
static int build_body(struct jit_ctx *ctx, bool extra_pass)
{
const struct bpf_prog *prog = ctx->prog;
@@ -1288,6 +1360,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
goto out_off;
}
+ ctx.fpb_offset = find_fpb_offset(prog);
+
/*
* 1. Initial fake pass to compute ctx->idx and ctx->offset.
*
--
2.30.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH -next v3 3/4] bpf/tests: Add tests for BPF_LDX/BPF_STX with different offsets
2022-03-16 16:26 ` Xu Kuohai
(?)
@ 2022-03-17 0:47 ` kernel test robot
-1 siblings, 0 replies; 11+ messages in thread
From: kernel test robot @ 2022-03-17 0:47 UTC (permalink / raw)
To: Xu Kuohai, bpf, linux-arm-kernel
Cc: llvm, kbuild-all, Catalin Marinas, Will Deacon, Daniel Borkmann,
Alexei Starovoitov, Zi Shen Lim, Andrii Nakryiko,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
KP Singh, Julien Thierry, Mark Rutland, Hou Tao, Fuad Tabba,
James Morse
Hi Xu,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on bpf-next/master]
url: https://github.com/0day-ci/linux/commits/Xu-Kuohai/bpf-arm64-Optimize-BPF-store-load-using/20220317-001942
base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
config: hexagon-randconfig-r041-20220314 (https://download.01.org/0day-ci/archive/20220317/202203170850.fi6zWTrj-lkp@intel.com/config)
compiler: clang version 15.0.0 (https://github.com/llvm/llvm-project a6ec1e3d798f8eab43fb3a91028c6ab04e115fcb)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/abb0326abc5a5dcf7ffed75f7fd98b621cab1152
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Xu-Kuohai/bpf-arm64-Optimize-BPF-store-load-using/20220317-001942
git checkout abb0326abc5a5dcf7ffed75f7fd98b621cab1152
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=hexagon SHELL=/bin/bash lib/
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> lib/test_bpf.c:8206:32: warning: implicit conversion from 'int' to '__s16' (aka 'short') changes value from 32768 to -32768 [-Wconstant-conversion]
BPF_LDX_MEM(BPF_DW, R0, R1, 32768),
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
include/linux/filter.h:253:12: note: expanded from macro 'BPF_LDX_MEM'
.off = OFF, \
^~~
lib/test_bpf.c:8205:32: warning: implicit conversion from 'int' to '__s16' (aka 'short') changes value from 32768 to -32768 [-Wconstant-conversion]
BPF_STX_MEM(BPF_DW, R1, R2, 32768),
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
include/linux/filter.h:263:12: note: expanded from macro 'BPF_STX_MEM'
.off = OFF, \
^~~
2 warnings generated.
vim +8206 lib/test_bpf.c
3061
3062
3063 static struct bpf_test tests[] = {
3064 {
3065 "TAX",
3066 .u.insns = {
3067 BPF_STMT(BPF_LD | BPF_IMM, 1),
3068 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3069 BPF_STMT(BPF_LD | BPF_IMM, 2),
3070 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
3071 BPF_STMT(BPF_ALU | BPF_NEG, 0), /* A == -3 */
3072 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3073 BPF_STMT(BPF_LD | BPF_LEN, 0),
3074 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
3075 BPF_STMT(BPF_MISC | BPF_TAX, 0), /* X == len - 3 */
3076 BPF_STMT(BPF_LD | BPF_B | BPF_IND, 1),
3077 BPF_STMT(BPF_RET | BPF_A, 0)
3078 },
3079 CLASSIC,
3080 { 10, 20, 30, 40, 50 },
3081 { { 2, 10 }, { 3, 20 }, { 4, 30 } },
3082 },
3083 {
3084 "TXA",
3085 .u.insns = {
3086 BPF_STMT(BPF_LDX | BPF_LEN, 0),
3087 BPF_STMT(BPF_MISC | BPF_TXA, 0),
3088 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
3089 BPF_STMT(BPF_RET | BPF_A, 0) /* A == len * 2 */
3090 },
3091 CLASSIC,
3092 { 10, 20, 30, 40, 50 },
3093 { { 1, 2 }, { 3, 6 }, { 4, 8 } },
3094 },
3095 {
3096 "ADD_SUB_MUL_K",
3097 .u.insns = {
3098 BPF_STMT(BPF_LD | BPF_IMM, 1),
3099 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 2),
3100 BPF_STMT(BPF_LDX | BPF_IMM, 3),
3101 BPF_STMT(BPF_ALU | BPF_SUB | BPF_X, 0),
3102 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 0xffffffff),
3103 BPF_STMT(BPF_ALU | BPF_MUL | BPF_K, 3),
3104 BPF_STMT(BPF_RET | BPF_A, 0)
3105 },
3106 CLASSIC | FLAG_NO_DATA,
3107 { },
3108 { { 0, 0xfffffffd } }
3109 },
3110 {
3111 "DIV_MOD_KX",
3112 .u.insns = {
3113 BPF_STMT(BPF_LD | BPF_IMM, 8),
3114 BPF_STMT(BPF_ALU | BPF_DIV | BPF_K, 2),
3115 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3116 BPF_STMT(BPF_LD | BPF_IMM, 0xffffffff),
3117 BPF_STMT(BPF_ALU | BPF_DIV | BPF_X, 0),
3118 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3119 BPF_STMT(BPF_LD | BPF_IMM, 0xffffffff),
3120 BPF_STMT(BPF_ALU | BPF_DIV | BPF_K, 0x70000000),
3121 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3122 BPF_STMT(BPF_LD | BPF_IMM, 0xffffffff),
3123 BPF_STMT(BPF_ALU | BPF_MOD | BPF_X, 0),
3124 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3125 BPF_STMT(BPF_LD | BPF_IMM, 0xffffffff),
3126 BPF_STMT(BPF_ALU | BPF_MOD | BPF_K, 0x70000000),
3127 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
3128 BPF_STMT(BPF_RET | BPF_A, 0)
3129 },
3130 CLASSIC | FLAG_NO_DATA,
3131 { },
3132 { { 0, 0x20000000 } }
3133 },
3134 {
3135 "AND_OR_LSH_K",
3136 .u.insns = {
3137 BPF_STMT(BPF_LD | BPF_IMM, 0xff),
3138 BPF_STMT(BPF_ALU | BPF_AND | BPF_K, 0xf0),
3139 BPF_STMT(BPF_ALU | BPF_LSH | BPF_K, 27),
3140 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3141 BPF_STMT(BPF_LD | BPF_IMM, 0xf),
3142 BPF_STMT(BPF_ALU | BPF_OR | BPF_K, 0xf0),
3143 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
3144 BPF_STMT(BPF_RET | BPF_A, 0)
3145 },
3146 CLASSIC | FLAG_NO_DATA,
3147 { },
3148 { { 0, 0x800000ff }, { 1, 0x800000ff } },
3149 },
3150 {
3151 "LD_IMM_0",
3152 .u.insns = {
3153 BPF_STMT(BPF_LD | BPF_IMM, 0), /* ld #0 */
3154 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 0, 1, 0),
3155 BPF_STMT(BPF_RET | BPF_K, 0),
3156 BPF_STMT(BPF_RET | BPF_K, 1),
3157 },
3158 CLASSIC,
3159 { },
3160 { { 1, 1 } },
3161 },
3162 {
3163 "LD_IND",
3164 .u.insns = {
3165 BPF_STMT(BPF_LDX | BPF_LEN, 0),
3166 BPF_STMT(BPF_LD | BPF_H | BPF_IND, MAX_K),
3167 BPF_STMT(BPF_RET | BPF_K, 1)
3168 },
3169 CLASSIC,
3170 { },
3171 { { 1, 0 }, { 10, 0 }, { 60, 0 } },
3172 },
3173 {
3174 "LD_ABS",
3175 .u.insns = {
3176 BPF_STMT(BPF_LD | BPF_W | BPF_ABS, 1000),
3177 BPF_STMT(BPF_RET | BPF_K, 1)
3178 },
3179 CLASSIC,
3180 { },
3181 { { 1, 0 }, { 10, 0 }, { 60, 0 } },
3182 },
3183 {
3184 "LD_ABS_LL",
3185 .u.insns = {
3186 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, SKF_LL_OFF),
3187 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3188 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, SKF_LL_OFF + 1),
3189 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
3190 BPF_STMT(BPF_RET | BPF_A, 0)
3191 },
3192 CLASSIC,
3193 { 1, 2, 3 },
3194 { { 1, 0 }, { 2, 3 } },
3195 },
3196 {
3197 "LD_IND_LL",
3198 .u.insns = {
3199 BPF_STMT(BPF_LD | BPF_IMM, SKF_LL_OFF - 1),
3200 BPF_STMT(BPF_LDX | BPF_LEN, 0),
3201 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
3202 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3203 BPF_STMT(BPF_LD | BPF_B | BPF_IND, 0),
3204 BPF_STMT(BPF_RET | BPF_A, 0)
3205 },
3206 CLASSIC,
3207 { 1, 2, 3, 0xff },
3208 { { 1, 1 }, { 3, 3 }, { 4, 0xff } },
3209 },
3210 {
3211 "LD_ABS_NET",
3212 .u.insns = {
3213 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, SKF_NET_OFF),
3214 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3215 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, SKF_NET_OFF + 1),
3216 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
3217 BPF_STMT(BPF_RET | BPF_A, 0)
3218 },
3219 CLASSIC,
3220 { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3 },
3221 { { 15, 0 }, { 16, 3 } },
3222 },
3223 {
3224 "LD_IND_NET",
3225 .u.insns = {
3226 BPF_STMT(BPF_LD | BPF_IMM, SKF_NET_OFF - 15),
3227 BPF_STMT(BPF_LDX | BPF_LEN, 0),
3228 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
3229 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3230 BPF_STMT(BPF_LD | BPF_B | BPF_IND, 0),
3231 BPF_STMT(BPF_RET | BPF_A, 0)
3232 },
3233 CLASSIC,
3234 { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 3 },
3235 { { 14, 0 }, { 15, 1 }, { 17, 3 } },
3236 },
3237 {
3238 "LD_PKTTYPE",
3239 .u.insns = {
3240 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3241 SKF_AD_OFF + SKF_AD_PKTTYPE),
3242 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, SKB_TYPE, 1, 0),
3243 BPF_STMT(BPF_RET | BPF_K, 1),
3244 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3245 SKF_AD_OFF + SKF_AD_PKTTYPE),
3246 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, SKB_TYPE, 1, 0),
3247 BPF_STMT(BPF_RET | BPF_K, 1),
3248 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3249 SKF_AD_OFF + SKF_AD_PKTTYPE),
3250 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, SKB_TYPE, 1, 0),
3251 BPF_STMT(BPF_RET | BPF_K, 1),
3252 BPF_STMT(BPF_RET | BPF_A, 0)
3253 },
3254 CLASSIC,
3255 { },
3256 { { 1, 3 }, { 10, 3 } },
3257 },
3258 {
3259 "LD_MARK",
3260 .u.insns = {
3261 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3262 SKF_AD_OFF + SKF_AD_MARK),
3263 BPF_STMT(BPF_RET | BPF_A, 0)
3264 },
3265 CLASSIC,
3266 { },
3267 { { 1, SKB_MARK}, { 10, SKB_MARK} },
3268 },
3269 {
3270 "LD_RXHASH",
3271 .u.insns = {
3272 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3273 SKF_AD_OFF + SKF_AD_RXHASH),
3274 BPF_STMT(BPF_RET | BPF_A, 0)
3275 },
3276 CLASSIC,
3277 { },
3278 { { 1, SKB_HASH}, { 10, SKB_HASH} },
3279 },
3280 {
3281 "LD_QUEUE",
3282 .u.insns = {
3283 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3284 SKF_AD_OFF + SKF_AD_QUEUE),
3285 BPF_STMT(BPF_RET | BPF_A, 0)
3286 },
3287 CLASSIC,
3288 { },
3289 { { 1, SKB_QUEUE_MAP }, { 10, SKB_QUEUE_MAP } },
3290 },
3291 {
3292 "LD_PROTOCOL",
3293 .u.insns = {
3294 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, 1),
3295 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 20, 1, 0),
3296 BPF_STMT(BPF_RET | BPF_K, 0),
3297 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3298 SKF_AD_OFF + SKF_AD_PROTOCOL),
3299 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3300 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, 2),
3301 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 30, 1, 0),
3302 BPF_STMT(BPF_RET | BPF_K, 0),
3303 BPF_STMT(BPF_MISC | BPF_TXA, 0),
3304 BPF_STMT(BPF_RET | BPF_A, 0)
3305 },
3306 CLASSIC,
3307 { 10, 20, 30 },
3308 { { 10, ETH_P_IP }, { 100, ETH_P_IP } },
3309 },
3310 {
3311 "LD_VLAN_TAG",
3312 .u.insns = {
3313 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3314 SKF_AD_OFF + SKF_AD_VLAN_TAG),
3315 BPF_STMT(BPF_RET | BPF_A, 0)
3316 },
3317 CLASSIC,
3318 { },
3319 {
3320 { 1, SKB_VLAN_TCI },
3321 { 10, SKB_VLAN_TCI }
3322 },
3323 },
3324 {
3325 "LD_VLAN_TAG_PRESENT",
3326 .u.insns = {
3327 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3328 SKF_AD_OFF + SKF_AD_VLAN_TAG_PRESENT),
3329 BPF_STMT(BPF_RET | BPF_A, 0)
3330 },
3331 CLASSIC,
3332 { },
3333 {
3334 { 1, SKB_VLAN_PRESENT },
3335 { 10, SKB_VLAN_PRESENT }
3336 },
3337 },
3338 {
3339 "LD_IFINDEX",
3340 .u.insns = {
3341 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3342 SKF_AD_OFF + SKF_AD_IFINDEX),
3343 BPF_STMT(BPF_RET | BPF_A, 0)
3344 },
3345 CLASSIC,
3346 { },
3347 { { 1, SKB_DEV_IFINDEX }, { 10, SKB_DEV_IFINDEX } },
3348 },
3349 {
3350 "LD_HATYPE",
3351 .u.insns = {
3352 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3353 SKF_AD_OFF + SKF_AD_HATYPE),
3354 BPF_STMT(BPF_RET | BPF_A, 0)
3355 },
3356 CLASSIC,
3357 { },
3358 { { 1, SKB_DEV_TYPE }, { 10, SKB_DEV_TYPE } },
3359 },
3360 {
3361 "LD_CPU",
3362 .u.insns = {
3363 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3364 SKF_AD_OFF + SKF_AD_CPU),
3365 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3366 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3367 SKF_AD_OFF + SKF_AD_CPU),
3368 BPF_STMT(BPF_ALU | BPF_SUB | BPF_X, 0),
3369 BPF_STMT(BPF_RET | BPF_A, 0)
3370 },
3371 CLASSIC,
3372 { },
3373 { { 1, 0 }, { 10, 0 } },
3374 },
3375 {
3376 "LD_NLATTR",
3377 .u.insns = {
3378 BPF_STMT(BPF_LDX | BPF_IMM, 2),
3379 BPF_STMT(BPF_MISC | BPF_TXA, 0),
3380 BPF_STMT(BPF_LDX | BPF_IMM, 3),
3381 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3382 SKF_AD_OFF + SKF_AD_NLATTR),
3383 BPF_STMT(BPF_RET | BPF_A, 0)
3384 },
3385 CLASSIC,
3386 #ifdef __BIG_ENDIAN
3387 { 0xff, 0xff, 0, 4, 0, 2, 0, 4, 0, 3 },
3388 #else
3389 { 0xff, 0xff, 4, 0, 2, 0, 4, 0, 3, 0 },
3390 #endif
3391 { { 4, 0 }, { 20, 6 } },
3392 },
3393 {
3394 "LD_NLATTR_NEST",
3395 .u.insns = {
3396 BPF_STMT(BPF_LD | BPF_IMM, 2),
3397 BPF_STMT(BPF_LDX | BPF_IMM, 3),
3398 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3399 SKF_AD_OFF + SKF_AD_NLATTR_NEST),
3400 BPF_STMT(BPF_LD | BPF_IMM, 2),
3401 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3402 SKF_AD_OFF + SKF_AD_NLATTR_NEST),
3403 BPF_STMT(BPF_LD | BPF_IMM, 2),
3404 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3405 SKF_AD_OFF + SKF_AD_NLATTR_NEST),
3406 BPF_STMT(BPF_LD | BPF_IMM, 2),
3407 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3408 SKF_AD_OFF + SKF_AD_NLATTR_NEST),
3409 BPF_STMT(BPF_LD | BPF_IMM, 2),
3410 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3411 SKF_AD_OFF + SKF_AD_NLATTR_NEST),
3412 BPF_STMT(BPF_LD | BPF_IMM, 2),
3413 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3414 SKF_AD_OFF + SKF_AD_NLATTR_NEST),
3415 BPF_STMT(BPF_LD | BPF_IMM, 2),
3416 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3417 SKF_AD_OFF + SKF_AD_NLATTR_NEST),
3418 BPF_STMT(BPF_LD | BPF_IMM, 2),
3419 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3420 SKF_AD_OFF + SKF_AD_NLATTR_NEST),
3421 BPF_STMT(BPF_RET | BPF_A, 0)
3422 },
3423 CLASSIC,
3424 #ifdef __BIG_ENDIAN
3425 { 0xff, 0xff, 0, 12, 0, 1, 0, 4, 0, 2, 0, 4, 0, 3 },
3426 #else
3427 { 0xff, 0xff, 12, 0, 1, 0, 4, 0, 2, 0, 4, 0, 3, 0 },
3428 #endif
3429 { { 4, 0 }, { 20, 10 } },
3430 },
3431 {
3432 "LD_PAYLOAD_OFF",
3433 .u.insns = {
3434 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3435 SKF_AD_OFF + SKF_AD_PAY_OFFSET),
3436 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3437 SKF_AD_OFF + SKF_AD_PAY_OFFSET),
3438 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3439 SKF_AD_OFF + SKF_AD_PAY_OFFSET),
3440 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3441 SKF_AD_OFF + SKF_AD_PAY_OFFSET),
3442 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3443 SKF_AD_OFF + SKF_AD_PAY_OFFSET),
3444 BPF_STMT(BPF_RET | BPF_A, 0)
3445 },
3446 CLASSIC,
3447 /* 00:00:00:00:00:00 > 00:00:00:00:00:00, ethtype IPv4 (0x0800),
3448 * length 98: 127.0.0.1 > 127.0.0.1: ICMP echo request,
3449 * id 9737, seq 1, length 64
3450 */
3451 { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
3452 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
3453 0x08, 0x00,
3454 0x45, 0x00, 0x00, 0x54, 0xac, 0x8b, 0x40, 0x00, 0x40,
3455 0x01, 0x90, 0x1b, 0x7f, 0x00, 0x00, 0x01 },
3456 { { 30, 0 }, { 100, 42 } },
3457 },
3458 {
3459 "LD_ANC_XOR",
3460 .u.insns = {
3461 BPF_STMT(BPF_LD | BPF_IMM, 10),
3462 BPF_STMT(BPF_LDX | BPF_IMM, 300),
3463 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
3464 SKF_AD_OFF + SKF_AD_ALU_XOR_X),
3465 BPF_STMT(BPF_RET | BPF_A, 0)
3466 },
3467 CLASSIC,
3468 { },
3469 { { 4, 0xA ^ 300 }, { 20, 0xA ^ 300 } },
3470 },
3471 {
3472 "SPILL_FILL",
3473 .u.insns = {
3474 BPF_STMT(BPF_LDX | BPF_LEN, 0),
3475 BPF_STMT(BPF_LD | BPF_IMM, 2),
3476 BPF_STMT(BPF_ALU | BPF_RSH, 1),
3477 BPF_STMT(BPF_ALU | BPF_XOR | BPF_X, 0),
3478 BPF_STMT(BPF_ST, 1), /* M1 = 1 ^ len */
3479 BPF_STMT(BPF_ALU | BPF_XOR | BPF_K, 0x80000000),
3480 BPF_STMT(BPF_ST, 2), /* M2 = 1 ^ len ^ 0x80000000 */
3481 BPF_STMT(BPF_STX, 15), /* M3 = len */
3482 BPF_STMT(BPF_LDX | BPF_MEM, 1),
3483 BPF_STMT(BPF_LD | BPF_MEM, 2),
3484 BPF_STMT(BPF_ALU | BPF_XOR | BPF_X, 0),
3485 BPF_STMT(BPF_LDX | BPF_MEM, 15),
3486 BPF_STMT(BPF_ALU | BPF_XOR | BPF_X, 0),
3487 BPF_STMT(BPF_RET | BPF_A, 0)
3488 },
3489 CLASSIC,
3490 { },
3491 { { 1, 0x80000001 }, { 2, 0x80000002 }, { 60, 0x80000000 ^ 60 } }
3492 },
3493 {
3494 "JEQ",
3495 .u.insns = {
3496 BPF_STMT(BPF_LDX | BPF_LEN, 0),
3497 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, 2),
3498 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_X, 0, 0, 1),
3499 BPF_STMT(BPF_RET | BPF_K, 1),
3500 BPF_STMT(BPF_RET | BPF_K, MAX_K)
3501 },
3502 CLASSIC,
3503 { 3, 3, 3, 3, 3 },
3504 { { 1, 0 }, { 3, 1 }, { 4, MAX_K } },
3505 },
3506 {
3507 "JGT",
3508 .u.insns = {
3509 BPF_STMT(BPF_LDX | BPF_LEN, 0),
3510 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, 2),
3511 BPF_JUMP(BPF_JMP | BPF_JGT | BPF_X, 0, 0, 1),
3512 BPF_STMT(BPF_RET | BPF_K, 1),
3513 BPF_STMT(BPF_RET | BPF_K, MAX_K)
3514 },
3515 CLASSIC,
3516 { 4, 4, 4, 3, 3 },
3517 { { 2, 0 }, { 3, 1 }, { 4, MAX_K } },
3518 },
3519 {
3520 "JGE (jt 0), test 1",
3521 .u.insns = {
3522 BPF_STMT(BPF_LDX | BPF_LEN, 0),
3523 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, 2),
3524 BPF_JUMP(BPF_JMP | BPF_JGE | BPF_X, 0, 0, 1),
3525 BPF_STMT(BPF_RET | BPF_K, 1),
3526 BPF_STMT(BPF_RET | BPF_K, MAX_K)
3527 },
3528 CLASSIC,
3529 { 4, 4, 4, 3, 3 },
3530 { { 2, 0 }, { 3, 1 }, { 4, 1 } },
3531 },
3532 {
3533 "JGE (jt 0), test 2",
3534 .u.insns = {
3535 BPF_STMT(BPF_LDX | BPF_LEN, 0),
3536 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, 2),
3537 BPF_JUMP(BPF_JMP | BPF_JGE | BPF_X, 0, 0, 1),
3538 BPF_STMT(BPF_RET | BPF_K, 1),
3539 BPF_STMT(BPF_RET | BPF_K, MAX_K)
3540 },
3541 CLASSIC,
3542 { 4, 4, 5, 3, 3 },
3543 { { 4, 1 }, { 5, 1 }, { 6, MAX_K } },
3544 },
3545 {
3546 "JGE",
3547 .u.insns = {
3548 BPF_STMT(BPF_LDX | BPF_LEN, 0),
3549 BPF_STMT(BPF_LD | BPF_B | BPF_IND, MAX_K),
3550 BPF_JUMP(BPF_JMP | BPF_JGE | BPF_K, 1, 1, 0),
3551 BPF_STMT(BPF_RET | BPF_K, 10),
3552 BPF_JUMP(BPF_JMP | BPF_JGE | BPF_K, 2, 1, 0),
3553 BPF_STMT(BPF_RET | BPF_K, 20),
3554 BPF_JUMP(BPF_JMP | BPF_JGE | BPF_K, 3, 1, 0),
3555 BPF_STMT(BPF_RET | BPF_K, 30),
3556 BPF_JUMP(BPF_JMP | BPF_JGE | BPF_K, 4, 1, 0),
3557 BPF_STMT(BPF_RET | BPF_K, 40),
3558 BPF_STMT(BPF_RET | BPF_K, MAX_K)
3559 },
3560 CLASSIC,
3561 { 1, 2, 3, 4, 5 },
3562 { { 1, 20 }, { 3, 40 }, { 5, MAX_K } },
3563 },
3564 {
3565 "JSET",
3566 .u.insns = {
3567 BPF_JUMP(BPF_JMP | BPF_JA, 0, 0, 0),
3568 BPF_JUMP(BPF_JMP | BPF_JA, 1, 1, 1),
3569 BPF_JUMP(BPF_JMP | BPF_JA, 0, 0, 0),
3570 BPF_JUMP(BPF_JMP | BPF_JA, 0, 0, 0),
3571 BPF_STMT(BPF_LDX | BPF_LEN, 0),
3572 BPF_STMT(BPF_MISC | BPF_TXA, 0),
3573 BPF_STMT(BPF_ALU | BPF_SUB | BPF_K, 4),
3574 BPF_STMT(BPF_MISC | BPF_TAX, 0),
3575 BPF_STMT(BPF_LD | BPF_W | BPF_IND, 0),
3576 BPF_JUMP(BPF_JMP | BPF_JSET | BPF_K, 1, 0, 1),
3577 BPF_STMT(BPF_RET | BPF_K, 10),
3578 BPF_JUMP(BPF_JMP | BPF_JSET | BPF_K, 0x80000000, 0, 1),
3579 BPF_STMT(BPF_RET | BPF_K, 20),
3580 BPF_JUMP(BPF_JMP | BPF_JSET | BPF_K, 0xffffff, 1, 0),
3581 BPF_STMT(BPF_RET | BPF_K, 30),
3582 BPF_JUMP(BPF_JMP | BPF_JSET | BPF_K, 0xffffff, 1, 0),
3583 BPF_STMT(BPF_RET | BPF_K, 30),
3584 BPF_JUMP(BPF_JMP | BPF_JSET | BPF_K, 0xffffff, 1, 0),
3585 BPF_STMT(BPF_RET | BPF_K, 30),
3586 BPF_JUMP(BPF_JMP | BPF_JSET | BPF_K, 0xffffff, 1, 0),
3587 BPF_STMT(BPF_RET | BPF_K, 30),
3588 BPF_JUMP(BPF_JMP | BPF_JSET | BPF_K, 0xffffff, 1, 0),
3589 BPF_STMT(BPF_RET | BPF_K, 30),
3590 BPF_STMT(BPF_RET | BPF_K, MAX_K)
3591 },
3592 CLASSIC,
3593 { 0, 0xAA, 0x55, 1 },
3594 { { 4, 10 }, { 5, 20 }, { 6, MAX_K } },
3595 },
3596 {
3597 "tcpdump port 22",
3598 .u.insns = {
3599 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 12),
3600 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 0x86dd, 0, 8), /* IPv6 */
3601 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, 20),
3602 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 0x84, 2, 0),
3603 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 0x6, 1, 0),
3604 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 0x11, 0, 17),
3605 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 54),
3606 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 22, 14, 0),
3607 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 56),
3608 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 22, 12, 13),
3609 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 0x0800, 0, 12), /* IPv4 */
3610 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, 23),
3611 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 0x84, 2, 0),
3612 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 0x6, 1, 0),
3613 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 0x11, 0, 8),
3614 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 20),
3615 BPF_JUMP(BPF_JMP | BPF_JSET | BPF_K, 0x1fff, 6, 0),
3616 BPF_STMT(BPF_LDX | BPF_B | BPF_MSH, 14),
3617 BPF_STMT(BPF_LD | BPF_H | BPF_IND, 14),
3618 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 22, 2, 0),
3619 BPF_STMT(BPF_LD | BPF_H | BPF_IND, 16),
3620 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 22, 0, 1),
3621 BPF_STMT(BPF_RET | BPF_K, 0xffff),
3622 BPF_STMT(BPF_RET | BPF_K, 0),
3623 },
3624 CLASSIC,
3625 /* 3c:07:54:43:e5:76 > 10:bf:48:d6:43:d6, ethertype IPv4(0x0800)
3626 * length 114: 10.1.1.149.49700 > 10.1.2.10.22: Flags [P.],
3627 * seq 1305692979:1305693027, ack 3650467037, win 65535,
3628 * options [nop,nop,TS val 2502645400 ecr 3971138], length 48
3629 */
3630 { 0x10, 0xbf, 0x48, 0xd6, 0x43, 0xd6,
3631 0x3c, 0x07, 0x54, 0x43, 0xe5, 0x76,
3632 0x08, 0x00,
3633 0x45, 0x10, 0x00, 0x64, 0x75, 0xb5,
3634 0x40, 0x00, 0x40, 0x06, 0xad, 0x2e, /* IP header */
3635 0x0a, 0x01, 0x01, 0x95, /* ip src */
3636 0x0a, 0x01, 0x02, 0x0a, /* ip dst */
3637 0xc2, 0x24,
3638 0x00, 0x16 /* dst port */ },
3639 { { 10, 0 }, { 30, 0 }, { 100, 65535 } },
3640 },
3641 {
3642 "tcpdump complex",
3643 .u.insns = {
3644 /* tcpdump -nei eth0 'tcp port 22 and (((ip[2:2] -
3645 * ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0) and
3646 * (len > 115 or len < 30000000000)' -d
3647 */
3648 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 12),
3649 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 0x86dd, 30, 0),
3650 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 0x800, 0, 29),
3651 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, 23),
3652 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 0x6, 0, 27),
3653 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 20),
3654 BPF_JUMP(BPF_JMP | BPF_JSET | BPF_K, 0x1fff, 25, 0),
3655 BPF_STMT(BPF_LDX | BPF_B | BPF_MSH, 14),
3656 BPF_STMT(BPF_LD | BPF_H | BPF_IND, 14),
3657 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 22, 2, 0),
3658 BPF_STMT(BPF_LD | BPF_H | BPF_IND, 16),
3659 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, 22, 0, 20),
3660 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 16),
3661 BPF_STMT(BPF_ST, 1),
3662 BPF_STMT(BPF_LD | BPF_B | BPF_ABS, 14),
3663 BPF_STMT(BPF_ALU | BPF_AND | BPF_K, 0xf),
3664 BPF_STMT(BPF_ALU | BPF_LSH | BPF_K, 2),
3665 BPF_STMT(BPF_MISC | BPF_TAX, 0x5), /* libpcap emits K on TAX */
3666 BPF_STMT(BPF_LD | BPF_MEM, 1),
3667 BPF_STMT(BPF_ALU | BPF_SUB | BPF_X, 0),
3668 BPF_STMT(BPF_ST, 5),
3669 BPF_STMT(BPF_LDX | BPF_B | BPF_MSH, 14),
3670 BPF_STMT(BPF_LD | BPF_B | BPF_IND, 26),
3671 BPF_STMT(BPF_ALU | BPF_AND | BPF_K, 0xf0),
3672 BPF_STMT(BPF_ALU | BPF_RSH | BPF_K, 2),
3673 BPF_STMT(BPF_MISC | BPF_TAX, 0x9), /* libpcap emits K on TAX */
3674 BPF_STMT(BPF_LD | BPF_MEM, 5),
3675 BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_X, 0, 4, 0),
3676 BPF_STMT(BPF_LD | BPF_LEN, 0),
3677 BPF_JUMP(BPF_JMP | BPF_JGT | BPF_K, 0x73, 1, 0),
3678 BPF_JUMP(BPF_JMP | BPF_JGE | BPF_K, 0xfc23ac00, 1, 0),
3679 BPF_STMT(BPF_RET | BPF_K, 0xffff),
3680 BPF_STMT(BPF_RET | BPF_K, 0),
3681 },
3682 CLASSIC,
3683 { 0x10, 0xbf, 0x48, 0xd6, 0x43, 0xd6,
3684 0x3c, 0x07, 0x54, 0x43, 0xe5, 0x76,
3685 0x08, 0x00,
3686 0x45, 0x10, 0x00, 0x64, 0x75, 0xb5,
3687 0x40, 0x00, 0x40, 0x06, 0xad, 0x2e, /* IP header */
3688 0x0a, 0x01, 0x01, 0x95, /* ip src */
3689 0x0a, 0x01, 0x02, 0x0a, /* ip dst */
3690 0xc2, 0x24,
3691 0x00, 0x16 /* dst port */ },
3692 { { 10, 0 }, { 30, 0 }, { 100, 65535 } },
3693 },
3694 {
3695 "RET_A",
3696 .u.insns = {
3697 /* check that uninitialized X and A contain zeros */
3698 BPF_STMT(BPF_MISC | BPF_TXA, 0),
3699 BPF_STMT(BPF_RET | BPF_A, 0)
3700 },
3701 CLASSIC,
3702 { },
3703 { {1, 0}, {2, 0} },
3704 },
3705 {
3706 "INT: ADD trivial",
3707 .u.insns_int = {
3708 BPF_ALU64_IMM(BPF_MOV, R1, 1),
3709 BPF_ALU64_IMM(BPF_ADD, R1, 2),
3710 BPF_ALU64_IMM(BPF_MOV, R2, 3),
3711 BPF_ALU64_REG(BPF_SUB, R1, R2),
3712 BPF_ALU64_IMM(BPF_ADD, R1, -1),
3713 BPF_ALU64_IMM(BPF_MUL, R1, 3),
3714 BPF_ALU64_REG(BPF_MOV, R0, R1),
3715 BPF_EXIT_INSN(),
3716 },
3717 INTERNAL,
3718 { },
3719 { { 0, 0xfffffffd } }
3720 },
3721 {
3722 "INT: MUL_X",
3723 .u.insns_int = {
3724 BPF_ALU64_IMM(BPF_MOV, R0, -1),
3725 BPF_ALU64_IMM(BPF_MOV, R1, -1),
3726 BPF_ALU64_IMM(BPF_MOV, R2, 3),
3727 BPF_ALU64_REG(BPF_MUL, R1, R2),
3728 BPF_JMP_IMM(BPF_JEQ, R1, 0xfffffffd, 1),
3729 BPF_EXIT_INSN(),
3730 BPF_ALU64_IMM(BPF_MOV, R0, 1),
3731 BPF_EXIT_INSN(),
3732 },
3733 INTERNAL,
3734 { },
3735 { { 0, 1 } }
3736 },
3737 {
3738 "INT: MUL_X2",
3739 .u.insns_int = {
3740 BPF_ALU32_IMM(BPF_MOV, R0, -1),
3741 BPF_ALU32_IMM(BPF_MOV, R1, -1),
3742 BPF_ALU32_IMM(BPF_MOV, R2, 3),
3743 BPF_ALU64_REG(BPF_MUL, R1, R2),
3744 BPF_ALU64_IMM(BPF_RSH, R1, 8),
3745 BPF_JMP_IMM(BPF_JEQ, R1, 0x2ffffff, 1),
3746 BPF_EXIT_INSN(),
3747 BPF_ALU32_IMM(BPF_MOV, R0, 1),
3748 BPF_EXIT_INSN(),
3749 },
3750 INTERNAL,
3751 { },
3752 { { 0, 1 } }
3753 },
3754 {
3755 "INT: MUL32_X",
3756 .u.insns_int = {
3757 BPF_ALU32_IMM(BPF_MOV, R0, -1),
3758 BPF_ALU64_IMM(BPF_MOV, R1, -1),
3759 BPF_ALU32_IMM(BPF_MOV, R2, 3),
3760 BPF_ALU32_REG(BPF_MUL, R1, R2),
3761 BPF_ALU64_IMM(BPF_RSH, R1, 8),
3762 BPF_JMP_IMM(BPF_JEQ, R1, 0xffffff, 1),
3763 BPF_EXIT_INSN(),
3764 BPF_ALU32_IMM(BPF_MOV, R0, 1),
3765 BPF_EXIT_INSN(),
3766 },
3767 INTERNAL,
3768 { },
3769 { { 0, 1 } }
3770 },
3771 {
3772 /* Have to test all register combinations, since
3773 * JITing of different registers will produce
3774 * different asm code.
3775 */
3776 "INT: ADD 64-bit",
3777 .u.insns_int = {
3778 BPF_ALU64_IMM(BPF_MOV, R0, 0),
3779 BPF_ALU64_IMM(BPF_MOV, R1, 1),
3780 BPF_ALU64_IMM(BPF_MOV, R2, 2),
3781 BPF_ALU64_IMM(BPF_MOV, R3, 3),
3782 BPF_ALU64_IMM(BPF_MOV, R4, 4),
3783 BPF_ALU64_IMM(BPF_MOV, R5, 5),
3784 BPF_ALU64_IMM(BPF_MOV, R6, 6),
3785 BPF_ALU64_IMM(BPF_MOV, R7, 7),
3786 BPF_ALU64_IMM(BPF_MOV, R8, 8),
3787 BPF_ALU64_IMM(BPF_MOV, R9, 9),
3788 BPF_ALU64_IMM(BPF_ADD, R0, 20),
3789 BPF_ALU64_IMM(BPF_ADD, R1, 20),
3790 BPF_ALU64_IMM(BPF_ADD, R2, 20),
3791 BPF_ALU64_IMM(BPF_ADD, R3, 20),
3792 BPF_ALU64_IMM(BPF_ADD, R4, 20),
3793 BPF_ALU64_IMM(BPF_ADD, R5, 20),
3794 BPF_ALU64_IMM(BPF_ADD, R6, 20),
3795 BPF_ALU64_IMM(BPF_ADD, R7, 20),
3796 BPF_ALU64_IMM(BPF_ADD, R8, 20),
3797 BPF_ALU64_IMM(BPF_ADD, R9, 20),
3798 BPF_ALU64_IMM(BPF_SUB, R0, 10),
3799 BPF_ALU64_IMM(BPF_SUB, R1, 10),
3800 BPF_ALU64_IMM(BPF_SUB, R2, 10),
3801 BPF_ALU64_IMM(BPF_SUB, R3, 10),
3802 BPF_ALU64_IMM(BPF_SUB, R4, 10),
3803 BPF_ALU64_IMM(BPF_SUB, R5, 10),
3804 BPF_ALU64_IMM(BPF_SUB, R6, 10),
3805 BPF_ALU64_IMM(BPF_SUB, R7, 10),
3806 BPF_ALU64_IMM(BPF_SUB, R8, 10),
3807 BPF_ALU64_IMM(BPF_SUB, R9, 10),
3808 BPF_ALU64_REG(BPF_ADD, R0, R0),
3809 BPF_ALU64_REG(BPF_ADD, R0, R1),
3810 BPF_ALU64_REG(BPF_ADD, R0, R2),
3811 BPF_ALU64_REG(BPF_ADD, R0, R3),
3812 BPF_ALU64_REG(BPF_ADD, R0, R4),
3813 BPF_ALU64_REG(BPF_ADD, R0, R5),
3814 BPF_ALU64_REG(BPF_ADD, R0, R6),
3815 BPF_ALU64_REG(BPF_ADD, R0, R7),
3816 BPF_ALU64_REG(BPF_ADD, R0, R8),
3817 BPF_ALU64_REG(BPF_ADD, R0, R9), /* R0 == 155 */
3818 BPF_JMP_IMM(BPF_JEQ, R0, 155, 1),
3819 BPF_EXIT_INSN(),
3820 BPF_ALU64_REG(BPF_ADD, R1, R0),
3821 BPF_ALU64_REG(BPF_ADD, R1, R1),
3822 BPF_ALU64_REG(BPF_ADD, R1, R2),
3823 BPF_ALU64_REG(BPF_ADD, R1, R3),
3824 BPF_ALU64_REG(BPF_ADD, R1, R4),
3825 BPF_ALU64_REG(BPF_ADD, R1, R5),
3826 BPF_ALU64_REG(BPF_ADD, R1, R6),
3827 BPF_ALU64_REG(BPF_ADD, R1, R7),
3828 BPF_ALU64_REG(BPF_ADD, R1, R8),
3829 BPF_ALU64_REG(BPF_ADD, R1, R9), /* R1 == 456 */
3830 BPF_JMP_IMM(BPF_JEQ, R1, 456, 1),
3831 BPF_EXIT_INSN(),
3832 BPF_ALU64_REG(BPF_ADD, R2, R0),
3833 BPF_ALU64_REG(BPF_ADD, R2, R1),
3834 BPF_ALU64_REG(BPF_ADD, R2, R2),
3835 BPF_ALU64_REG(BPF_ADD, R2, R3),
3836 BPF_ALU64_REG(BPF_ADD, R2, R4),
3837 BPF_ALU64_REG(BPF_ADD, R2, R5),
3838 BPF_ALU64_REG(BPF_ADD, R2, R6),
3839 BPF_ALU64_REG(BPF_ADD, R2, R7),
3840 BPF_ALU64_REG(BPF_ADD, R2, R8),
3841 BPF_ALU64_REG(BPF_ADD, R2, R9), /* R2 == 1358 */
3842 BPF_JMP_IMM(BPF_JEQ, R2, 1358, 1),
3843 BPF_EXIT_INSN(),
3844 BPF_ALU64_REG(BPF_ADD, R3, R0),
3845 BPF_ALU64_REG(BPF_ADD, R3, R1),
3846 BPF_ALU64_REG(BPF_ADD, R3, R2),
3847 BPF_ALU64_REG(BPF_ADD, R3, R3),
3848 BPF_ALU64_REG(BPF_ADD, R3, R4),
3849 BPF_ALU64_REG(BPF_ADD, R3, R5),
3850 BPF_ALU64_REG(BPF_ADD, R3, R6),
3851 BPF_ALU64_REG(BPF_ADD, R3, R7),
3852 BPF_ALU64_REG(BPF_ADD, R3, R8),
3853 BPF_ALU64_REG(BPF_ADD, R3, R9), /* R3 == 4063 */
3854 BPF_JMP_IMM(BPF_JEQ, R3, 4063, 1),
3855 BPF_EXIT_INSN(),
3856 BPF_ALU64_REG(BPF_ADD, R4, R0),
3857 BPF_ALU64_REG(BPF_ADD, R4, R1),
3858 BPF_ALU64_REG(BPF_ADD, R4, R2),
3859 BPF_ALU64_REG(BPF_ADD, R4, R3),
3860 BPF_ALU64_REG(BPF_ADD, R4, R4),
3861 BPF_ALU64_REG(BPF_ADD, R4, R5),
3862 BPF_ALU64_REG(BPF_ADD, R4, R6),
3863 BPF_ALU64_REG(BPF_ADD, R4, R7),
3864 BPF_ALU64_REG(BPF_ADD, R4, R8),
3865 BPF_ALU64_REG(BPF_ADD, R4, R9), /* R4 == 12177 */
3866 BPF_JMP_IMM(BPF_JEQ, R4, 12177, 1),
3867 BPF_EXIT_INSN(),
3868 BPF_ALU64_REG(BPF_ADD, R5, R0),
3869 BPF_ALU64_REG(BPF_ADD, R5, R1),
3870 BPF_ALU64_REG(BPF_ADD, R5, R2),
3871 BPF_ALU64_REG(BPF_ADD, R5, R3),
3872 BPF_ALU64_REG(BPF_ADD, R5, R4),
3873 BPF_ALU64_REG(BPF_ADD, R5, R5),
3874 BPF_ALU64_REG(BPF_ADD, R5, R6),
3875 BPF_ALU64_REG(BPF_ADD, R5, R7),
3876 BPF_ALU64_REG(BPF_ADD, R5, R8),
3877 BPF_ALU64_REG(BPF_ADD, R5, R9), /* R5 == 36518 */
3878 BPF_JMP_IMM(BPF_JEQ, R5, 36518, 1),
3879 BPF_EXIT_INSN(),
3880 BPF_ALU64_REG(BPF_ADD, R6, R0),
3881 BPF_ALU64_REG(BPF_ADD, R6, R1),
3882 BPF_ALU64_REG(BPF_ADD, R6, R2),
3883 BPF_ALU64_REG(BPF_ADD, R6, R3),
3884 BPF_ALU64_REG(BPF_ADD, R6, R4),
3885 BPF_ALU64_REG(BPF_ADD, R6, R5),
3886 BPF_ALU64_REG(BPF_ADD, R6, R6),
3887 BPF_ALU64_REG(BPF_ADD, R6, R7),
3888 BPF_ALU64_REG(BPF_ADD, R6, R8),
3889 BPF_ALU64_REG(BPF_ADD, R6, R9), /* R6 == 109540 */
3890 BPF_JMP_IMM(BPF_JEQ, R6, 109540, 1),
3891 BPF_EXIT_INSN(),
3892 BPF_ALU64_REG(BPF_ADD, R7, R0),
3893 BPF_ALU64_REG(BPF_ADD, R7, R1),
3894 BPF_ALU64_REG(BPF_ADD, R7, R2),
3895 BPF_ALU64_REG(BPF_ADD, R7, R3),
3896 BPF_ALU64_REG(BPF_ADD, R7, R4),
3897 BPF_ALU64_REG(BPF_ADD, R7, R5),
3898 BPF_ALU64_REG(BPF_ADD, R7, R6),
3899 BPF_ALU64_REG(BPF_ADD, R7, R7),
3900 BPF_ALU64_REG(BPF_ADD, R7, R8),
3901 BPF_ALU64_REG(BPF_ADD, R7, R9), /* R7 == 328605 */
3902 BPF_JMP_IMM(BPF_JEQ, R7, 328605, 1),
3903 BPF_EXIT_INSN(),
3904 BPF_ALU64_REG(BPF_ADD, R8, R0),
3905 BPF_ALU64_REG(BPF_ADD, R8, R1),
3906 BPF_ALU64_REG(BPF_ADD, R8, R2),
3907 BPF_ALU64_REG(BPF_ADD, R8, R3),
3908 BPF_ALU64_REG(BPF_ADD, R8, R4),
3909 BPF_ALU64_REG(BPF_ADD, R8, R5),
3910 BPF_ALU64_REG(BPF_ADD, R8, R6),
3911 BPF_ALU64_REG(BPF_ADD, R8, R7),
3912 BPF_ALU64_REG(BPF_ADD, R8, R8),
3913 BPF_ALU64_REG(BPF_ADD, R8, R9), /* R8 == 985799 */
3914 BPF_JMP_IMM(BPF_JEQ, R8, 985799, 1),
3915 BPF_EXIT_INSN(),
3916 BPF_ALU64_REG(BPF_ADD, R9, R0),
3917 BPF_ALU64_REG(BPF_ADD, R9, R1),
3918 BPF_ALU64_REG(BPF_ADD, R9, R2),
3919 BPF_ALU64_REG(BPF_ADD, R9, R3),
3920 BPF_ALU64_REG(BPF_ADD, R9, R4),
3921 BPF_ALU64_REG(BPF_ADD, R9, R5),
3922 BPF_ALU64_REG(BPF_ADD, R9, R6),
3923 BPF_ALU64_REG(BPF_ADD, R9, R7),
3924 BPF_ALU64_REG(BPF_ADD, R9, R8),
3925 BPF_ALU64_REG(BPF_ADD, R9, R9), /* R9 == 2957380 */
3926 BPF_ALU64_REG(BPF_MOV, R0, R9),
3927 BPF_EXIT_INSN(),
3928 },
3929 INTERNAL,
3930 { },
3931 { { 0, 2957380 } }
3932 },
3933 {
3934 "INT: ADD 32-bit",
3935 .u.insns_int = {
3936 BPF_ALU32_IMM(BPF_MOV, R0, 20),
3937 BPF_ALU32_IMM(BPF_MOV, R1, 1),
3938 BPF_ALU32_IMM(BPF_MOV, R2, 2),
3939 BPF_ALU32_IMM(BPF_MOV, R3, 3),
3940 BPF_ALU32_IMM(BPF_MOV, R4, 4),
3941 BPF_ALU32_IMM(BPF_MOV, R5, 5),
3942 BPF_ALU32_IMM(BPF_MOV, R6, 6),
3943 BPF_ALU32_IMM(BPF_MOV, R7, 7),
3944 BPF_ALU32_IMM(BPF_MOV, R8, 8),
3945 BPF_ALU32_IMM(BPF_MOV, R9, 9),
3946 BPF_ALU64_IMM(BPF_ADD, R1, 10),
3947 BPF_ALU64_IMM(BPF_ADD, R2, 10),
3948 BPF_ALU64_IMM(BPF_ADD, R3, 10),
3949 BPF_ALU64_IMM(BPF_ADD, R4, 10),
3950 BPF_ALU64_IMM(BPF_ADD, R5, 10),
3951 BPF_ALU64_IMM(BPF_ADD, R6, 10),
3952 BPF_ALU64_IMM(BPF_ADD, R7, 10),
3953 BPF_ALU64_IMM(BPF_ADD, R8, 10),
3954 BPF_ALU64_IMM(BPF_ADD, R9, 10),
3955 BPF_ALU32_REG(BPF_ADD, R0, R1),
3956 BPF_ALU32_REG(BPF_ADD, R0, R2),
3957 BPF_ALU32_REG(BPF_ADD, R0, R3),
3958 BPF_ALU32_REG(BPF_ADD, R0, R4),
3959 BPF_ALU32_REG(BPF_ADD, R0, R5),
3960 BPF_ALU32_REG(BPF_ADD, R0, R6),
3961 BPF_ALU32_REG(BPF_ADD, R0, R7),
3962 BPF_ALU32_REG(BPF_ADD, R0, R8),
3963 BPF_ALU32_REG(BPF_ADD, R0, R9), /* R0 == 155 */
3964 BPF_JMP_IMM(BPF_JEQ, R0, 155, 1),
3965 BPF_EXIT_INSN(),
3966 BPF_ALU32_REG(BPF_ADD, R1, R0),
3967 BPF_ALU32_REG(BPF_ADD, R1, R1),
3968 BPF_ALU32_REG(BPF_ADD, R1, R2),
3969 BPF_ALU32_REG(BPF_ADD, R1, R3),
3970 BPF_ALU32_REG(BPF_ADD, R1, R4),
3971 BPF_ALU32_REG(BPF_ADD, R1, R5),
3972 BPF_ALU32_REG(BPF_ADD, R1, R6),
3973 BPF_ALU32_REG(BPF_ADD, R1, R7),
3974 BPF_ALU32_REG(BPF_ADD, R1, R8),
3975 BPF_ALU32_REG(BPF_ADD, R1, R9), /* R1 == 456 */
3976 BPF_JMP_IMM(BPF_JEQ, R1, 456, 1),
3977 BPF_EXIT_INSN(),
3978 BPF_ALU32_REG(BPF_ADD, R2, R0),
3979 BPF_ALU32_REG(BPF_ADD, R2, R1),
3980 BPF_ALU32_REG(BPF_ADD, R2, R2),
3981 BPF_ALU32_REG(BPF_ADD, R2, R3),
3982 BPF_ALU32_REG(BPF_ADD, R2, R4),
3983 BPF_ALU32_REG(BPF_ADD, R2, R5),
3984 BPF_ALU32_REG(BPF_ADD, R2, R6),
3985 BPF_ALU32_REG(BPF_ADD, R2, R7),
3986 BPF_ALU32_REG(BPF_ADD, R2, R8),
3987 BPF_ALU32_REG(BPF_ADD, R2, R9), /* R2 == 1358 */
3988 BPF_JMP_IMM(BPF_JEQ, R2, 1358, 1),
3989 BPF_EXIT_INSN(),
3990 BPF_ALU32_REG(BPF_ADD, R3, R0),
3991 BPF_ALU32_REG(BPF_ADD, R3, R1),
3992 BPF_ALU32_REG(BPF_ADD, R3, R2),
3993 BPF_ALU32_REG(BPF_ADD, R3, R3),
3994 BPF_ALU32_REG(BPF_ADD, R3, R4),
3995 BPF_ALU32_REG(BPF_ADD, R3, R5),
3996 BPF_ALU32_REG(BPF_ADD, R3, R6),
3997 BPF_ALU32_REG(BPF_ADD, R3, R7),
3998 BPF_ALU32_REG(BPF_ADD, R3, R8),
3999 BPF_ALU32_REG(BPF_ADD, R3, R9), /* R3 == 4063 */
4000 BPF_JMP_IMM(BPF_JEQ, R3, 4063, 1),
4001 BPF_EXIT_INSN(),
4002 BPF_ALU32_REG(BPF_ADD, R4, R0),
4003 BPF_ALU32_REG(BPF_ADD, R4, R1),
4004 BPF_ALU32_REG(BPF_ADD, R4, R2),
4005 BPF_ALU32_REG(BPF_ADD, R4, R3),
4006 BPF_ALU32_REG(BPF_ADD, R4, R4),
4007 BPF_ALU32_REG(BPF_ADD, R4, R5),
4008 BPF_ALU32_REG(BPF_ADD, R4, R6),
4009 BPF_ALU32_REG(BPF_ADD, R4, R7),
4010 BPF_ALU32_REG(BPF_ADD, R4, R8),
4011 BPF_ALU32_REG(BPF_ADD, R4, R9), /* R4 == 12177 */
4012 BPF_JMP_IMM(BPF_JEQ, R4, 12177, 1),
4013 BPF_EXIT_INSN(),
4014 BPF_ALU32_REG(BPF_ADD, R5, R0),
4015 BPF_ALU32_REG(BPF_ADD, R5, R1),
4016 BPF_ALU32_REG(BPF_ADD, R5, R2),
4017 BPF_ALU32_REG(BPF_ADD, R5, R3),
4018 BPF_ALU32_REG(BPF_ADD, R5, R4),
4019 BPF_ALU32_REG(BPF_ADD, R5, R5),
4020 BPF_ALU32_REG(BPF_ADD, R5, R6),
4021 BPF_ALU32_REG(BPF_ADD, R5, R7),
4022 BPF_ALU32_REG(BPF_ADD, R5, R8),
4023 BPF_ALU32_REG(BPF_ADD, R5, R9), /* R5 == 36518 */
4024 BPF_JMP_IMM(BPF_JEQ, R5, 36518, 1),
4025 BPF_EXIT_INSN(),
4026 BPF_ALU32_REG(BPF_ADD, R6, R0),
4027 BPF_ALU32_REG(BPF_ADD, R6, R1),
4028 BPF_ALU32_REG(BPF_ADD, R6, R2),
4029 BPF_ALU32_REG(BPF_ADD, R6, R3),
4030 BPF_ALU32_REG(BPF_ADD, R6, R4),
4031 BPF_ALU32_REG(BPF_ADD, R6, R5),
4032 BPF_ALU32_REG(BPF_ADD, R6, R6),
4033 BPF_ALU32_REG(BPF_ADD, R6, R7),
4034 BPF_ALU32_REG(BPF_ADD, R6, R8),
4035 BPF_ALU32_REG(BPF_ADD, R6, R9), /* R6 == 109540 */
4036 BPF_JMP_IMM(BPF_JEQ, R6, 109540, 1),
4037 BPF_EXIT_INSN(),
4038 BPF_ALU32_REG(BPF_ADD, R7, R0),
4039 BPF_ALU32_REG(BPF_ADD, R7, R1),
4040 BPF_ALU32_REG(BPF_ADD, R7, R2),
4041 BPF_ALU32_REG(BPF_ADD, R7, R3),
4042 BPF_ALU32_REG(BPF_ADD, R7, R4),
4043 BPF_ALU32_REG(BPF_ADD, R7, R5),
4044 BPF_ALU32_REG(BPF_ADD, R7, R6),
4045 BPF_ALU32_REG(BPF_ADD, R7, R7),
4046 BPF_ALU32_REG(BPF_ADD, R7, R8),
4047 BPF_ALU32_REG(BPF_ADD, R7, R9), /* R7 == 328605 */
4048 BPF_JMP_IMM(BPF_JEQ, R7, 328605, 1),
4049 BPF_EXIT_INSN(),
4050 BPF_ALU32_REG(BPF_ADD, R8, R0),
4051 BPF_ALU32_REG(BPF_ADD, R8, R1),
4052 BPF_ALU32_REG(BPF_ADD, R8, R2),
4053 BPF_ALU32_REG(BPF_ADD, R8, R3),
4054 BPF_ALU32_REG(BPF_ADD, R8, R4),
4055 BPF_ALU32_REG(BPF_ADD, R8, R5),
4056 BPF_ALU32_REG(BPF_ADD, R8, R6),
4057 BPF_ALU32_REG(BPF_ADD, R8, R7),
4058 BPF_ALU32_REG(BPF_ADD, R8, R8),
4059 BPF_ALU32_REG(BPF_ADD, R8, R9), /* R8 == 985799 */
4060 BPF_JMP_IMM(BPF_JEQ, R8, 985799, 1),
4061 BPF_EXIT_INSN(),
4062 BPF_ALU32_REG(BPF_ADD, R9, R0),
4063 BPF_ALU32_REG(BPF_ADD, R9, R1),
4064 BPF_ALU32_REG(BPF_ADD, R9, R2),
4065 BPF_ALU32_REG(BPF_ADD, R9, R3),
4066 BPF_ALU32_REG(BPF_ADD, R9, R4),
4067 BPF_ALU32_REG(BPF_ADD, R9, R5),
4068 BPF_ALU32_REG(BPF_ADD, R9, R6),
4069 BPF_ALU32_REG(BPF_ADD, R9, R7),
4070 BPF_ALU32_REG(BPF_ADD, R9, R8),
4071 BPF_ALU32_REG(BPF_ADD, R9, R9), /* R9 == 2957380 */
4072 BPF_ALU32_REG(BPF_MOV, R0, R9),
4073 BPF_EXIT_INSN(),
4074 },
4075 INTERNAL,
4076 { },
4077 { { 0, 2957380 } }
4078 },
4079 { /* Mainly checking JIT here. */
4080 "INT: SUB",
4081 .u.insns_int = {
4082 BPF_ALU64_IMM(BPF_MOV, R0, 0),
4083 BPF_ALU64_IMM(BPF_MOV, R1, 1),
4084 BPF_ALU64_IMM(BPF_MOV, R2, 2),
4085 BPF_ALU64_IMM(BPF_MOV, R3, 3),
4086 BPF_ALU64_IMM(BPF_MOV, R4, 4),
4087 BPF_ALU64_IMM(BPF_MOV, R5, 5),
4088 BPF_ALU64_IMM(BPF_MOV, R6, 6),
4089 BPF_ALU64_IMM(BPF_MOV, R7, 7),
4090 BPF_ALU64_IMM(BPF_MOV, R8, 8),
4091 BPF_ALU64_IMM(BPF_MOV, R9, 9),
4092 BPF_ALU64_REG(BPF_SUB, R0, R0),
4093 BPF_ALU64_REG(BPF_SUB, R0, R1),
4094 BPF_ALU64_REG(BPF_SUB, R0, R2),
4095 BPF_ALU64_REG(BPF_SUB, R0, R3),
4096 BPF_ALU64_REG(BPF_SUB, R0, R4),
4097 BPF_ALU64_REG(BPF_SUB, R0, R5),
4098 BPF_ALU64_REG(BPF_SUB, R0, R6),
4099 BPF_ALU64_REG(BPF_SUB, R0, R7),
4100 BPF_ALU64_REG(BPF_SUB, R0, R8),
4101 BPF_ALU64_REG(BPF_SUB, R0, R9),
4102 BPF_ALU64_IMM(BPF_SUB, R0, 10),
4103 BPF_JMP_IMM(BPF_JEQ, R0, -55, 1),
4104 BPF_EXIT_INSN(),
4105 BPF_ALU64_REG(BPF_SUB, R1, R0),
4106 BPF_ALU64_REG(BPF_SUB, R1, R2),
4107 BPF_ALU64_REG(BPF_SUB, R1, R3),
4108 BPF_ALU64_REG(BPF_SUB, R1, R4),
4109 BPF_ALU64_REG(BPF_SUB, R1, R5),
4110 BPF_ALU64_REG(BPF_SUB, R1, R6),
4111 BPF_ALU64_REG(BPF_SUB, R1, R7),
4112 BPF_ALU64_REG(BPF_SUB, R1, R8),
4113 BPF_ALU64_REG(BPF_SUB, R1, R9),
4114 BPF_ALU64_IMM(BPF_SUB, R1, 10),
4115 BPF_ALU64_REG(BPF_SUB, R2, R0),
4116 BPF_ALU64_REG(BPF_SUB, R2, R1),
4117 BPF_ALU64_REG(BPF_SUB, R2, R3),
4118 BPF_ALU64_REG(BPF_SUB, R2, R4),
4119 BPF_ALU64_REG(BPF_SUB, R2, R5),
4120 BPF_ALU64_REG(BPF_SUB, R2, R6),
4121 BPF_ALU64_REG(BPF_SUB, R2, R7),
4122 BPF_ALU64_REG(BPF_SUB, R2, R8),
4123 BPF_ALU64_REG(BPF_SUB, R2, R9),
4124 BPF_ALU64_IMM(BPF_SUB, R2, 10),
4125 BPF_ALU64_REG(BPF_SUB, R3, R0),
4126 BPF_ALU64_REG(BPF_SUB, R3, R1),
4127 BPF_ALU64_REG(BPF_SUB, R3, R2),
4128 BPF_ALU64_REG(BPF_SUB, R3, R4),
4129 BPF_ALU64_REG(BPF_SUB, R3, R5),
4130 BPF_ALU64_REG(BPF_SUB, R3, R6),
4131 BPF_ALU64_REG(BPF_SUB, R3, R7),
4132 BPF_ALU64_REG(BPF_SUB, R3, R8),
4133 BPF_ALU64_REG(BPF_SUB, R3, R9),
4134 BPF_ALU64_IMM(BPF_SUB, R3, 10),
4135 BPF_ALU64_REG(BPF_SUB, R4, R0),
4136 BPF_ALU64_REG(BPF_SUB, R4, R1),
4137 BPF_ALU64_REG(BPF_SUB, R4, R2),
4138 BPF_ALU64_REG(BPF_SUB, R4, R3),
4139 BPF_ALU64_REG(BPF_SUB, R4, R5),
4140 BPF_ALU64_REG(BPF_SUB, R4, R6),
4141 BPF_ALU64_REG(BPF_SUB, R4, R7),
4142 BPF_ALU64_REG(BPF_SUB, R4, R8),
4143 BPF_ALU64_REG(BPF_SUB, R4, R9),
4144 BPF_ALU64_IMM(BPF_SUB, R4, 10),
4145 BPF_ALU64_REG(BPF_SUB, R5, R0),
4146 BPF_ALU64_REG(BPF_SUB, R5, R1),
4147 BPF_ALU64_REG(BPF_SUB, R5, R2),
4148 BPF_ALU64_REG(BPF_SUB, R5, R3),
4149 BPF_ALU64_REG(BPF_SUB, R5, R4),
4150 BPF_ALU64_REG(BPF_SUB, R5, R6),
4151 BPF_ALU64_REG(BPF_SUB, R5, R7),
4152 BPF_ALU64_REG(BPF_SUB, R5, R8),
4153 BPF_ALU64_REG(BPF_SUB, R5, R9),
4154 BPF_ALU64_IMM(BPF_SUB, R5, 10),
4155 BPF_ALU64_REG(BPF_SUB, R6, R0),
4156 BPF_ALU64_REG(BPF_SUB, R6, R1),
4157 BPF_ALU64_REG(BPF_SUB, R6, R2),
4158 BPF_ALU64_REG(BPF_SUB, R6, R3),
4159 BPF_ALU64_REG(BPF_SUB, R6, R4),
4160 BPF_ALU64_REG(BPF_SUB, R6, R5),
4161 BPF_ALU64_REG(BPF_SUB, R6, R7),
4162 BPF_ALU64_REG(BPF_SUB, R6, R8),
4163 BPF_ALU64_REG(BPF_SUB, R6, R9),
4164 BPF_ALU64_IMM(BPF_SUB, R6, 10),
4165 BPF_ALU64_REG(BPF_SUB, R7, R0),
4166 BPF_ALU64_REG(BPF_SUB, R7, R1),
4167 BPF_ALU64_REG(BPF_SUB, R7, R2),
4168 BPF_ALU64_REG(BPF_SUB, R7, R3),
4169 BPF_ALU64_REG(BPF_SUB, R7, R4),
4170 BPF_ALU64_REG(BPF_SUB, R7, R5),
4171 BPF_ALU64_REG(BPF_SUB, R7, R6),
4172 BPF_ALU64_REG(BPF_SUB, R7, R8),
4173 BPF_ALU64_REG(BPF_SUB, R7, R9),
4174 BPF_ALU64_IMM(BPF_SUB, R7, 10),
4175 BPF_ALU64_REG(BPF_SUB, R8, R0),
4176 BPF_ALU64_REG(BPF_SUB, R8, R1),
4177 BPF_ALU64_REG(BPF_SUB, R8, R2),
4178 BPF_ALU64_REG(BPF_SUB, R8, R3),
4179 BPF_ALU64_REG(BPF_SUB, R8, R4),
4180 BPF_ALU64_REG(BPF_SUB, R8, R5),
4181 BPF_ALU64_REG(BPF_SUB, R8, R6),
4182 BPF_ALU64_REG(BPF_SUB, R8, R7),
4183 BPF_ALU64_REG(BPF_SUB, R8, R9),
4184 BPF_ALU64_IMM(BPF_SUB, R8, 10),
4185 BPF_ALU64_REG(BPF_SUB, R9, R0),
4186 BPF_ALU64_REG(BPF_SUB, R9, R1),
4187 BPF_ALU64_REG(BPF_SUB, R9, R2),
4188 BPF_ALU64_REG(BPF_SUB, R9, R3),
4189 BPF_ALU64_REG(BPF_SUB, R9, R4),
4190 BPF_ALU64_REG(BPF_SUB, R9, R5),
4191 BPF_ALU64_REG(BPF_SUB, R9, R6),
4192 BPF_ALU64_REG(BPF_SUB, R9, R7),
4193 BPF_ALU64_REG(BPF_SUB, R9, R8),
4194 BPF_ALU64_IMM(BPF_SUB, R9, 10),
4195 BPF_ALU64_IMM(BPF_SUB, R0, 10),
4196 BPF_ALU64_IMM(BPF_NEG, R0, 0),
4197 BPF_ALU64_REG(BPF_SUB, R0, R1),
4198 BPF_ALU64_REG(BPF_SUB, R0, R2),
4199 BPF_ALU64_REG(BPF_SUB, R0, R3),
4200 BPF_ALU64_REG(BPF_SUB, R0, R4),
4201 BPF_ALU64_REG(BPF_SUB, R0, R5),
4202 BPF_ALU64_REG(BPF_SUB, R0, R6),
4203 BPF_ALU64_REG(BPF_SUB, R0, R7),
4204 BPF_ALU64_REG(BPF_SUB, R0, R8),
4205 BPF_ALU64_REG(BPF_SUB, R0, R9),
4206 BPF_EXIT_INSN(),
4207 },
4208 INTERNAL,
4209 { },
4210 { { 0, 11 } }
4211 },
4212 { /* Mainly checking JIT here. */
4213 "INT: XOR",
4214 .u.insns_int = {
4215 BPF_ALU64_REG(BPF_SUB, R0, R0),
4216 BPF_ALU64_REG(BPF_XOR, R1, R1),
4217 BPF_JMP_REG(BPF_JEQ, R0, R1, 1),
4218 BPF_EXIT_INSN(),
4219 BPF_ALU64_IMM(BPF_MOV, R0, 10),
4220 BPF_ALU64_IMM(BPF_MOV, R1, -1),
4221 BPF_ALU64_REG(BPF_SUB, R1, R1),
4222 BPF_ALU64_REG(BPF_XOR, R2, R2),
4223 BPF_JMP_REG(BPF_JEQ, R1, R2, 1),
4224 BPF_EXIT_INSN(),
4225 BPF_ALU64_REG(BPF_SUB, R2, R2),
4226 BPF_ALU64_REG(BPF_XOR, R3, R3),
4227 BPF_ALU64_IMM(BPF_MOV, R0, 10),
4228 BPF_ALU64_IMM(BPF_MOV, R1, -1),
4229 BPF_JMP_REG(BPF_JEQ, R2, R3, 1),
4230 BPF_EXIT_INSN(),
4231 BPF_ALU64_REG(BPF_SUB, R3, R3),
4232 BPF_ALU64_REG(BPF_XOR, R4, R4),
4233 BPF_ALU64_IMM(BPF_MOV, R2, 1),
4234 BPF_ALU64_IMM(BPF_MOV, R5, -1),
4235 BPF_JMP_REG(BPF_JEQ, R3, R4, 1),
4236 BPF_EXIT_INSN(),
4237 BPF_ALU64_REG(BPF_SUB, R4, R4),
4238 BPF_ALU64_REG(BPF_XOR, R5, R5),
4239 BPF_ALU64_IMM(BPF_MOV, R3, 1),
4240 BPF_ALU64_IMM(BPF_MOV, R7, -1),
4241 BPF_JMP_REG(BPF_JEQ, R5, R4, 1),
4242 BPF_EXIT_INSN(),
4243 BPF_ALU64_IMM(BPF_MOV, R5, 1),
4244 BPF_ALU64_REG(BPF_SUB, R5, R5),
4245 BPF_ALU64_REG(BPF_XOR, R6, R6),
4246 BPF_ALU64_IMM(BPF_MOV, R1, 1),
4247 BPF_ALU64_IMM(BPF_MOV, R8, -1),
4248 BPF_JMP_REG(BPF_JEQ, R5, R6, 1),
4249 BPF_EXIT_INSN(),
4250 BPF_ALU64_REG(BPF_SUB, R6, R6),
4251 BPF_ALU64_REG(BPF_XOR, R7, R7),
4252 BPF_JMP_REG(BPF_JEQ, R7, R6, 1),
4253 BPF_EXIT_INSN(),
4254 BPF_ALU64_REG(BPF_SUB, R7, R7),
4255 BPF_ALU64_REG(BPF_XOR, R8, R8),
4256 BPF_JMP_REG(BPF_JEQ, R7, R8, 1),
4257 BPF_EXIT_INSN(),
4258 BPF_ALU64_REG(BPF_SUB, R8, R8),
4259 BPF_ALU64_REG(BPF_XOR, R9, R9),
4260 BPF_JMP_REG(BPF_JEQ, R9, R8, 1),
4261 BPF_EXIT_INSN(),
4262 BPF_ALU64_REG(BPF_SUB, R9, R9),
4263 BPF_ALU64_REG(BPF_XOR, R0, R0),
4264 BPF_JMP_REG(BPF_JEQ, R9, R0, 1),
4265 BPF_EXIT_INSN(),
4266 BPF_ALU64_REG(BPF_SUB, R1, R1),
4267 BPF_ALU64_REG(BPF_XOR, R0, R0),
4268 BPF_JMP_REG(BPF_JEQ, R9, R0, 2),
4269 BPF_ALU64_IMM(BPF_MOV, R0, 0),
4270 BPF_EXIT_INSN(),
4271 BPF_ALU64_IMM(BPF_MOV, R0, 1),
4272 BPF_EXIT_INSN(),
4273 },
4274 INTERNAL,
4275 { },
4276 { { 0, 1 } }
4277 },
4278 { /* Mainly checking JIT here. */
4279 "INT: MUL",
4280 .u.insns_int = {
4281 BPF_ALU64_IMM(BPF_MOV, R0, 11),
4282 BPF_ALU64_IMM(BPF_MOV, R1, 1),
4283 BPF_ALU64_IMM(BPF_MOV, R2, 2),
4284 BPF_ALU64_IMM(BPF_MOV, R3, 3),
4285 BPF_ALU64_IMM(BPF_MOV, R4, 4),
4286 BPF_ALU64_IMM(BPF_MOV, R5, 5),
4287 BPF_ALU64_IMM(BPF_MOV, R6, 6),
4288 BPF_ALU64_IMM(BPF_MOV, R7, 7),
4289 BPF_ALU64_IMM(BPF_MOV, R8, 8),
4290 BPF_ALU64_IMM(BPF_MOV, R9, 9),
4291 BPF_ALU64_REG(BPF_MUL, R0, R0),
4292 BPF_ALU64_REG(BPF_MUL, R0, R1),
4293 BPF_ALU64_REG(BPF_MUL, R0, R2),
4294 BPF_ALU64_REG(BPF_MUL, R0, R3),
4295 BPF_ALU64_REG(BPF_MUL, R0, R4),
4296 BPF_ALU64_REG(BPF_MUL, R0, R5),
4297 BPF_ALU64_REG(BPF_MUL, R0, R6),
4298 BPF_ALU64_REG(BPF_MUL, R0, R7),
4299 BPF_ALU64_REG(BPF_MUL, R0, R8),
4300 BPF_ALU64_REG(BPF_MUL, R0, R9),
4301 BPF_ALU64_IMM(BPF_MUL, R0, 10),
4302 BPF_JMP_IMM(BPF_JEQ, R0, 439084800, 1),
4303 BPF_EXIT_INSN(),
4304 BPF_ALU64_REG(BPF_MUL, R1, R0),
4305 BPF_ALU64_REG(BPF_MUL, R1, R2),
4306 BPF_ALU64_REG(BPF_MUL, R1, R3),
4307 BPF_ALU64_REG(BPF_MUL, R1, R4),
4308 BPF_ALU64_REG(BPF_MUL, R1, R5),
4309 BPF_ALU64_REG(BPF_MUL, R1, R6),
4310 BPF_ALU64_REG(BPF_MUL, R1, R7),
4311 BPF_ALU64_REG(BPF_MUL, R1, R8),
4312 BPF_ALU64_REG(BPF_MUL, R1, R9),
4313 BPF_ALU64_IMM(BPF_MUL, R1, 10),
4314 BPF_ALU64_REG(BPF_MOV, R2, R1),
4315 BPF_ALU64_IMM(BPF_RSH, R2, 32),
4316 BPF_JMP_IMM(BPF_JEQ, R2, 0x5a924, 1),
4317 BPF_EXIT_INSN(),
4318 BPF_ALU64_IMM(BPF_LSH, R1, 32),
4319 BPF_ALU64_IMM(BPF_ARSH, R1, 32),
4320 BPF_JMP_IMM(BPF_JEQ, R1, 0xebb90000, 1),
4321 BPF_EXIT_INSN(),
4322 BPF_ALU64_REG(BPF_MUL, R2, R0),
4323 BPF_ALU64_REG(BPF_MUL, R2, R1),
4324 BPF_ALU64_REG(BPF_MUL, R2, R3),
4325 BPF_ALU64_REG(BPF_MUL, R2, R4),
4326 BPF_ALU64_REG(BPF_MUL, R2, R5),
4327 BPF_ALU64_REG(BPF_MUL, R2, R6),
4328 BPF_ALU64_REG(BPF_MUL, R2, R7),
4329 BPF_ALU64_REG(BPF_MUL, R2, R8),
4330 BPF_ALU64_REG(BPF_MUL, R2, R9),
4331 BPF_ALU64_IMM(BPF_MUL, R2, 10),
4332 BPF_ALU64_IMM(BPF_RSH, R2, 32),
4333 BPF_ALU64_REG(BPF_MOV, R0, R2),
4334 BPF_EXIT_INSN(),
4335 },
4336 INTERNAL,
4337 { },
4338 { { 0, 0x35d97ef2 } }
4339 },
4340 { /* Mainly checking JIT here. */
4341 "MOV REG64",
4342 .u.insns_int = {
4343 BPF_LD_IMM64(R0, 0xffffffffffffffffLL),
4344 BPF_MOV64_REG(R1, R0),
4345 BPF_MOV64_REG(R2, R1),
4346 BPF_MOV64_REG(R3, R2),
4347 BPF_MOV64_REG(R4, R3),
4348 BPF_MOV64_REG(R5, R4),
4349 BPF_MOV64_REG(R6, R5),
4350 BPF_MOV64_REG(R7, R6),
4351 BPF_MOV64_REG(R8, R7),
4352 BPF_MOV64_REG(R9, R8),
4353 BPF_ALU64_IMM(BPF_MOV, R0, 0),
4354 BPF_ALU64_IMM(BPF_MOV, R1, 0),
4355 BPF_ALU64_IMM(BPF_MOV, R2, 0),
4356 BPF_ALU64_IMM(BPF_MOV, R3, 0),
4357 BPF_ALU64_IMM(BPF_MOV, R4, 0),
4358 BPF_ALU64_IMM(BPF_MOV, R5, 0),
4359 BPF_ALU64_IMM(BPF_MOV, R6, 0),
4360 BPF_ALU64_IMM(BPF_MOV, R7, 0),
4361 BPF_ALU64_IMM(BPF_MOV, R8, 0),
4362 BPF_ALU64_IMM(BPF_MOV, R9, 0),
4363 BPF_ALU64_REG(BPF_ADD, R0, R0),
4364 BPF_ALU64_REG(BPF_ADD, R0, R1),
4365 BPF_ALU64_REG(BPF_ADD, R0, R2),
4366 BPF_ALU64_REG(BPF_ADD, R0, R3),
4367 BPF_ALU64_REG(BPF_ADD, R0, R4),
4368 BPF_ALU64_REG(BPF_ADD, R0, R5),
4369 BPF_ALU64_REG(BPF_ADD, R0, R6),
4370 BPF_ALU64_REG(BPF_ADD, R0, R7),
4371 BPF_ALU64_REG(BPF_ADD, R0, R8),
4372 BPF_ALU64_REG(BPF_ADD, R0, R9),
4373 BPF_ALU64_IMM(BPF_ADD, R0, 0xfefe),
4374 BPF_EXIT_INSN(),
4375 },
4376 INTERNAL,
4377 { },
4378 { { 0, 0xfefe } }
4379 },
4380 { /* Mainly checking JIT here. */
4381 "MOV REG32",
4382 .u.insns_int = {
4383 BPF_LD_IMM64(R0, 0xffffffffffffffffLL),
4384 BPF_MOV64_REG(R1, R0),
4385 BPF_MOV64_REG(R2, R1),
4386 BPF_MOV64_REG(R3, R2),
4387 BPF_MOV64_REG(R4, R3),
4388 BPF_MOV64_REG(R5, R4),
4389 BPF_MOV64_REG(R6, R5),
4390 BPF_MOV64_REG(R7, R6),
4391 BPF_MOV64_REG(R8, R7),
4392 BPF_MOV64_REG(R9, R8),
4393 BPF_ALU32_IMM(BPF_MOV, R0, 0),
4394 BPF_ALU32_IMM(BPF_MOV, R1, 0),
4395 BPF_ALU32_IMM(BPF_MOV, R2, 0),
4396 BPF_ALU32_IMM(BPF_MOV, R3, 0),
4397 BPF_ALU32_IMM(BPF_MOV, R4, 0),
4398 BPF_ALU32_IMM(BPF_MOV, R5, 0),
4399 BPF_ALU32_IMM(BPF_MOV, R6, 0),
4400 BPF_ALU32_IMM(BPF_MOV, R7, 0),
4401 BPF_ALU32_IMM(BPF_MOV, R8, 0),
4402 BPF_ALU32_IMM(BPF_MOV, R9, 0),
4403 BPF_ALU64_REG(BPF_ADD, R0, R0),
4404 BPF_ALU64_REG(BPF_ADD, R0, R1),
4405 BPF_ALU64_REG(BPF_ADD, R0, R2),
4406 BPF_ALU64_REG(BPF_ADD, R0, R3),
4407 BPF_ALU64_REG(BPF_ADD, R0, R4),
4408 BPF_ALU64_REG(BPF_ADD, R0, R5),
4409 BPF_ALU64_REG(BPF_ADD, R0, R6),
4410 BPF_ALU64_REG(BPF_ADD, R0, R7),
4411 BPF_ALU64_REG(BPF_ADD, R0, R8),
4412 BPF_ALU64_REG(BPF_ADD, R0, R9),
4413 BPF_ALU64_IMM(BPF_ADD, R0, 0xfefe),
4414 BPF_EXIT_INSN(),
4415 },
4416 INTERNAL,
4417 { },
4418 { { 0, 0xfefe } }
4419 },
4420 { /* Mainly checking JIT here. */
4421 "LD IMM64",
4422 .u.insns_int = {
4423 BPF_LD_IMM64(R0, 0xffffffffffffffffLL),
4424 BPF_MOV64_REG(R1, R0),
4425 BPF_MOV64_REG(R2, R1),
4426 BPF_MOV64_REG(R3, R2),
4427 BPF_MOV64_REG(R4, R3),
4428 BPF_MOV64_REG(R5, R4),
4429 BPF_MOV64_REG(R6, R5),
4430 BPF_MOV64_REG(R7, R6),
4431 BPF_MOV64_REG(R8, R7),
4432 BPF_MOV64_REG(R9, R8),
4433 BPF_LD_IMM64(R0, 0x0LL),
4434 BPF_LD_IMM64(R1, 0x0LL),
4435 BPF_LD_IMM64(R2, 0x0LL),
4436 BPF_LD_IMM64(R3, 0x0LL),
4437 BPF_LD_IMM64(R4, 0x0LL),
4438 BPF_LD_IMM64(R5, 0x0LL),
4439 BPF_LD_IMM64(R6, 0x0LL),
4440 BPF_LD_IMM64(R7, 0x0LL),
4441 BPF_LD_IMM64(R8, 0x0LL),
4442 BPF_LD_IMM64(R9, 0x0LL),
4443 BPF_ALU64_REG(BPF_ADD, R0, R0),
4444 BPF_ALU64_REG(BPF_ADD, R0, R1),
4445 BPF_ALU64_REG(BPF_ADD, R0, R2),
4446 BPF_ALU64_REG(BPF_ADD, R0, R3),
4447 BPF_ALU64_REG(BPF_ADD, R0, R4),
4448 BPF_ALU64_REG(BPF_ADD, R0, R5),
4449 BPF_ALU64_REG(BPF_ADD, R0, R6),
4450 BPF_ALU64_REG(BPF_ADD, R0, R7),
4451 BPF_ALU64_REG(BPF_ADD, R0, R8),
4452 BPF_ALU64_REG(BPF_ADD, R0, R9),
4453 BPF_ALU64_IMM(BPF_ADD, R0, 0xfefe),
4454 BPF_EXIT_INSN(),
4455 },
4456 INTERNAL,
4457 { },
4458 { { 0, 0xfefe } }
4459 },
4460 {
4461 "INT: ALU MIX",
4462 .u.insns_int = {
4463 BPF_ALU64_IMM(BPF_MOV, R0, 11),
4464 BPF_ALU64_IMM(BPF_ADD, R0, -1),
4465 BPF_ALU64_IMM(BPF_MOV, R2, 2),
4466 BPF_ALU64_IMM(BPF_XOR, R2, 3),
4467 BPF_ALU64_REG(BPF_DIV, R0, R2),
4468 BPF_JMP_IMM(BPF_JEQ, R0, 10, 1),
4469 BPF_EXIT_INSN(),
4470 BPF_ALU64_IMM(BPF_MOD, R0, 3),
4471 BPF_JMP_IMM(BPF_JEQ, R0, 1, 1),
4472 BPF_EXIT_INSN(),
4473 BPF_ALU64_IMM(BPF_MOV, R0, -1),
4474 BPF_EXIT_INSN(),
4475 },
4476 INTERNAL,
4477 { },
4478 { { 0, -1 } }
4479 },
4480 {
4481 "INT: shifts by register",
4482 .u.insns_int = {
4483 BPF_MOV64_IMM(R0, -1234),
4484 BPF_MOV64_IMM(R1, 1),
4485 BPF_ALU32_REG(BPF_RSH, R0, R1),
4486 BPF_JMP_IMM(BPF_JEQ, R0, 0x7ffffd97, 1),
4487 BPF_EXIT_INSN(),
4488 BPF_MOV64_IMM(R2, 1),
4489 BPF_ALU64_REG(BPF_LSH, R0, R2),
4490 BPF_MOV32_IMM(R4, -1234),
4491 BPF_JMP_REG(BPF_JEQ, R0, R4, 1),
4492 BPF_EXIT_INSN(),
4493 BPF_ALU64_IMM(BPF_AND, R4, 63),
4494 BPF_ALU64_REG(BPF_LSH, R0, R4), /* R0 <= 46 */
4495 BPF_MOV64_IMM(R3, 47),
4496 BPF_ALU64_REG(BPF_ARSH, R0, R3),
4497 BPF_JMP_IMM(BPF_JEQ, R0, -617, 1),
4498 BPF_EXIT_INSN(),
4499 BPF_MOV64_IMM(R2, 1),
4500 BPF_ALU64_REG(BPF_LSH, R4, R2), /* R4 = 46 << 1 */
4501 BPF_JMP_IMM(BPF_JEQ, R4, 92, 1),
4502 BPF_EXIT_INSN(),
4503 BPF_MOV64_IMM(R4, 4),
4504 BPF_ALU64_REG(BPF_LSH, R4, R4), /* R4 = 4 << 4 */
4505 BPF_JMP_IMM(BPF_JEQ, R4, 64, 1),
4506 BPF_EXIT_INSN(),
4507 BPF_MOV64_IMM(R4, 5),
4508 BPF_ALU32_REG(BPF_LSH, R4, R4), /* R4 = 5 << 5 */
4509 BPF_JMP_IMM(BPF_JEQ, R4, 160, 1),
4510 BPF_EXIT_INSN(),
4511 BPF_MOV64_IMM(R0, -1),
4512 BPF_EXIT_INSN(),
4513 },
4514 INTERNAL,
4515 { },
4516 { { 0, -1 } }
4517 },
4518 #ifdef CONFIG_32BIT
4519 {
4520 "INT: 32-bit context pointer word order and zero-extension",
4521 .u.insns_int = {
4522 BPF_ALU32_IMM(BPF_MOV, R0, 0),
4523 BPF_JMP32_IMM(BPF_JEQ, R1, 0, 3),
4524 BPF_ALU64_IMM(BPF_RSH, R1, 32),
4525 BPF_JMP32_IMM(BPF_JNE, R1, 0, 1),
4526 BPF_ALU32_IMM(BPF_MOV, R0, 1),
4527 BPF_EXIT_INSN(),
4528 },
4529 INTERNAL,
4530 { },
4531 { { 0, 1 } }
4532 },
4533 #endif
4534 {
4535 "check: missing ret",
4536 .u.insns = {
4537 BPF_STMT(BPF_LD | BPF_IMM, 1),
4538 },
4539 CLASSIC | FLAG_NO_DATA | FLAG_EXPECTED_FAIL,
4540 { },
4541 { },
4542 .fill_helper = NULL,
4543 .expected_errcode = -EINVAL,
4544 },
4545 {
4546 "check: div_k_0",
4547 .u.insns = {
4548 BPF_STMT(BPF_ALU | BPF_DIV | BPF_K, 0),
4549 BPF_STMT(BPF_RET | BPF_K, 0)
4550 },
4551 CLASSIC | FLAG_NO_DATA | FLAG_EXPECTED_FAIL,
4552 { },
4553 { },
4554 .fill_helper = NULL,
4555 .expected_errcode = -EINVAL,
4556 },
4557 {
4558 "check: unknown insn",
4559 .u.insns = {
4560 /* seccomp insn, rejected in socket filter */
4561 BPF_STMT(BPF_LDX | BPF_W | BPF_ABS, 0),
4562 BPF_STMT(BPF_RET | BPF_K, 0)
4563 },
4564 CLASSIC | FLAG_EXPECTED_FAIL,
4565 { },
4566 { },
4567 .fill_helper = NULL,
4568 .expected_errcode = -EINVAL,
4569 },
4570 {
4571 "check: out of range spill/fill",
4572 .u.insns = {
4573 BPF_STMT(BPF_STX, 16),
4574 BPF_STMT(BPF_RET | BPF_K, 0)
4575 },
4576 CLASSIC | FLAG_NO_DATA | FLAG_EXPECTED_FAIL,
4577 { },
4578 { },
4579 .fill_helper = NULL,
4580 .expected_errcode = -EINVAL,
4581 },
4582 {
4583 "JUMPS + HOLES",
4584 .u.insns = {
4585 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4586 BPF_JUMP(BPF_JMP | BPF_JGE, 0, 13, 15),
4587 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4588 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4589 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4590 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4591 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4592 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4593 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4594 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4595 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4596 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4597 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4598 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4599 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4600 BPF_JUMP(BPF_JMP | BPF_JEQ, 0x90c2894d, 3, 4),
4601 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4602 BPF_JUMP(BPF_JMP | BPF_JEQ, 0x90c2894d, 1, 2),
4603 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4604 BPF_JUMP(BPF_JMP | BPF_JGE, 0, 14, 15),
4605 BPF_JUMP(BPF_JMP | BPF_JGE, 0, 13, 14),
4606 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4607 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4608 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4609 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4610 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4611 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4612 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4613 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4614 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4615 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4616 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4617 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4618 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4619 BPF_JUMP(BPF_JMP | BPF_JEQ, 0x2ac28349, 2, 3),
4620 BPF_JUMP(BPF_JMP | BPF_JEQ, 0x2ac28349, 1, 2),
4621 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4622 BPF_JUMP(BPF_JMP | BPF_JGE, 0, 14, 15),
4623 BPF_JUMP(BPF_JMP | BPF_JGE, 0, 13, 14),
4624 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4625 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4626 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4627 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4628 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4629 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4630 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4631 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4632 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4633 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4634 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4635 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4636 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4637 BPF_JUMP(BPF_JMP | BPF_JEQ, 0x90d2ff41, 2, 3),
4638 BPF_JUMP(BPF_JMP | BPF_JEQ, 0x90d2ff41, 1, 2),
4639 BPF_STMT(BPF_LD | BPF_H | BPF_ABS, 0),
4640 BPF_STMT(BPF_RET | BPF_A, 0),
4641 BPF_STMT(BPF_RET | BPF_A, 0),
4642 },
4643 CLASSIC,
4644 { 0x00, 0x1b, 0x21, 0x3c, 0x9d, 0xf8,
4645 0x90, 0xe2, 0xba, 0x0a, 0x56, 0xb4,
4646 0x08, 0x00,
4647 0x45, 0x00, 0x00, 0x28, 0x00, 0x00,
4648 0x20, 0x00, 0x40, 0x11, 0x00, 0x00, /* IP header */
4649 0xc0, 0xa8, 0x33, 0x01,
4650 0xc0, 0xa8, 0x33, 0x02,
4651 0xbb, 0xb6,
4652 0xa9, 0xfa,
4653 0x00, 0x14, 0x00, 0x00,
4654 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc,
4655 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc,
4656 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc,
4657 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc,
4658 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc,
4659 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc,
4660 0xcc, 0xcc, 0xcc, 0xcc, 0xcc, 0xcc,
4661 0xcc, 0xcc, 0xcc, 0xcc },
4662 { { 88, 0x001b } }
4663 },
4664 {
4665 "check: RET X",
4666 .u.insns = {
4667 BPF_STMT(BPF_RET | BPF_X, 0),
4668 },
4669 CLASSIC | FLAG_NO_DATA | FLAG_EXPECTED_FAIL,
4670 { },
4671 { },
4672 .fill_helper = NULL,
4673 .expected_errcode = -EINVAL,
4674 },
4675 {
4676 "check: LDX + RET X",
4677 .u.insns = {
4678 BPF_STMT(BPF_LDX | BPF_IMM, 42),
4679 BPF_STMT(BPF_RET | BPF_X, 0),
4680 },
4681 CLASSIC | FLAG_NO_DATA | FLAG_EXPECTED_FAIL,
4682 { },
4683 { },
4684 .fill_helper = NULL,
4685 .expected_errcode = -EINVAL,
4686 },
4687 { /* Mainly checking JIT here. */
4688 "M[]: alt STX + LDX",
4689 .u.insns = {
4690 BPF_STMT(BPF_LDX | BPF_IMM, 100),
4691 BPF_STMT(BPF_STX, 0),
4692 BPF_STMT(BPF_LDX | BPF_MEM, 0),
4693 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4694 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4695 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4696 BPF_STMT(BPF_STX, 1),
4697 BPF_STMT(BPF_LDX | BPF_MEM, 1),
4698 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4699 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4700 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4701 BPF_STMT(BPF_STX, 2),
4702 BPF_STMT(BPF_LDX | BPF_MEM, 2),
4703 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4704 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4705 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4706 BPF_STMT(BPF_STX, 3),
4707 BPF_STMT(BPF_LDX | BPF_MEM, 3),
4708 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4709 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4710 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4711 BPF_STMT(BPF_STX, 4),
4712 BPF_STMT(BPF_LDX | BPF_MEM, 4),
4713 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4714 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4715 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4716 BPF_STMT(BPF_STX, 5),
4717 BPF_STMT(BPF_LDX | BPF_MEM, 5),
4718 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4719 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4720 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4721 BPF_STMT(BPF_STX, 6),
4722 BPF_STMT(BPF_LDX | BPF_MEM, 6),
4723 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4724 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4725 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4726 BPF_STMT(BPF_STX, 7),
4727 BPF_STMT(BPF_LDX | BPF_MEM, 7),
4728 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4729 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4730 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4731 BPF_STMT(BPF_STX, 8),
4732 BPF_STMT(BPF_LDX | BPF_MEM, 8),
4733 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4734 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4735 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4736 BPF_STMT(BPF_STX, 9),
4737 BPF_STMT(BPF_LDX | BPF_MEM, 9),
4738 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4739 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4740 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4741 BPF_STMT(BPF_STX, 10),
4742 BPF_STMT(BPF_LDX | BPF_MEM, 10),
4743 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4744 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4745 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4746 BPF_STMT(BPF_STX, 11),
4747 BPF_STMT(BPF_LDX | BPF_MEM, 11),
4748 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4749 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4750 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4751 BPF_STMT(BPF_STX, 12),
4752 BPF_STMT(BPF_LDX | BPF_MEM, 12),
4753 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4754 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4755 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4756 BPF_STMT(BPF_STX, 13),
4757 BPF_STMT(BPF_LDX | BPF_MEM, 13),
4758 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4759 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4760 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4761 BPF_STMT(BPF_STX, 14),
4762 BPF_STMT(BPF_LDX | BPF_MEM, 14),
4763 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4764 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4765 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4766 BPF_STMT(BPF_STX, 15),
4767 BPF_STMT(BPF_LDX | BPF_MEM, 15),
4768 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4769 BPF_STMT(BPF_ALU | BPF_ADD | BPF_K, 1),
4770 BPF_STMT(BPF_MISC | BPF_TAX, 0),
4771 BPF_STMT(BPF_RET | BPF_A, 0),
4772 },
4773 CLASSIC | FLAG_NO_DATA,
4774 { },
4775 { { 0, 116 } },
4776 },
4777 { /* Mainly checking JIT here. */
4778 "M[]: full STX + full LDX",
4779 .u.insns = {
4780 BPF_STMT(BPF_LDX | BPF_IMM, 0xbadfeedb),
4781 BPF_STMT(BPF_STX, 0),
4782 BPF_STMT(BPF_LDX | BPF_IMM, 0xecabedae),
4783 BPF_STMT(BPF_STX, 1),
4784 BPF_STMT(BPF_LDX | BPF_IMM, 0xafccfeaf),
4785 BPF_STMT(BPF_STX, 2),
4786 BPF_STMT(BPF_LDX | BPF_IMM, 0xbffdcedc),
4787 BPF_STMT(BPF_STX, 3),
4788 BPF_STMT(BPF_LDX | BPF_IMM, 0xfbbbdccb),
4789 BPF_STMT(BPF_STX, 4),
4790 BPF_STMT(BPF_LDX | BPF_IMM, 0xfbabcbda),
4791 BPF_STMT(BPF_STX, 5),
4792 BPF_STMT(BPF_LDX | BPF_IMM, 0xaedecbdb),
4793 BPF_STMT(BPF_STX, 6),
4794 BPF_STMT(BPF_LDX | BPF_IMM, 0xadebbade),
4795 BPF_STMT(BPF_STX, 7),
4796 BPF_STMT(BPF_LDX | BPF_IMM, 0xfcfcfaec),
4797 BPF_STMT(BPF_STX, 8),
4798 BPF_STMT(BPF_LDX | BPF_IMM, 0xbcdddbdc),
4799 BPF_STMT(BPF_STX, 9),
4800 BPF_STMT(BPF_LDX | BPF_IMM, 0xfeefdfac),
4801 BPF_STMT(BPF_STX, 10),
4802 BPF_STMT(BPF_LDX | BPF_IMM, 0xcddcdeea),
4803 BPF_STMT(BPF_STX, 11),
4804 BPF_STMT(BPF_LDX | BPF_IMM, 0xaccfaebb),
4805 BPF_STMT(BPF_STX, 12),
4806 BPF_STMT(BPF_LDX | BPF_IMM, 0xbdcccdcf),
4807 BPF_STMT(BPF_STX, 13),
4808 BPF_STMT(BPF_LDX | BPF_IMM, 0xaaedecde),
4809 BPF_STMT(BPF_STX, 14),
4810 BPF_STMT(BPF_LDX | BPF_IMM, 0xfaeacdad),
4811 BPF_STMT(BPF_STX, 15),
4812 BPF_STMT(BPF_LDX | BPF_MEM, 0),
4813 BPF_STMT(BPF_MISC | BPF_TXA, 0),
4814 BPF_STMT(BPF_LDX | BPF_MEM, 1),
4815 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4816 BPF_STMT(BPF_LDX | BPF_MEM, 2),
4817 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4818 BPF_STMT(BPF_LDX | BPF_MEM, 3),
4819 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4820 BPF_STMT(BPF_LDX | BPF_MEM, 4),
4821 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4822 BPF_STMT(BPF_LDX | BPF_MEM, 5),
4823 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4824 BPF_STMT(BPF_LDX | BPF_MEM, 6),
4825 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4826 BPF_STMT(BPF_LDX | BPF_MEM, 7),
4827 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4828 BPF_STMT(BPF_LDX | BPF_MEM, 8),
4829 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4830 BPF_STMT(BPF_LDX | BPF_MEM, 9),
4831 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4832 BPF_STMT(BPF_LDX | BPF_MEM, 10),
4833 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4834 BPF_STMT(BPF_LDX | BPF_MEM, 11),
4835 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4836 BPF_STMT(BPF_LDX | BPF_MEM, 12),
4837 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4838 BPF_STMT(BPF_LDX | BPF_MEM, 13),
4839 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4840 BPF_STMT(BPF_LDX | BPF_MEM, 14),
4841 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4842 BPF_STMT(BPF_LDX | BPF_MEM, 15),
4843 BPF_STMT(BPF_ALU | BPF_ADD | BPF_X, 0),
4844 BPF_STMT(BPF_RET | BPF_A, 0),
4845 },
4846 CLASSIC | FLAG_NO_DATA,
4847 { },
4848 { { 0, 0x2a5a5e5 } },
4849 },
4850 {
4851 "check: SKF_AD_MAX",
4852 .u.insns = {
4853 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
4854 SKF_AD_OFF + SKF_AD_MAX),
4855 BPF_STMT(BPF_RET | BPF_A, 0),
4856 },
4857 CLASSIC | FLAG_NO_DATA | FLAG_EXPECTED_FAIL,
4858 { },
4859 { },
4860 .fill_helper = NULL,
4861 .expected_errcode = -EINVAL,
4862 },
4863 { /* Passes checker but fails during runtime. */
4864 "LD [SKF_AD_OFF-1]",
4865 .u.insns = {
4866 BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
4867 SKF_AD_OFF - 1),
4868 BPF_STMT(BPF_RET | BPF_K, 1),
4869 },
4870 CLASSIC,
4871 { },
4872 { { 1, 0 } },
4873 },
4874 {
4875 "load 64-bit immediate",
4876 .u.insns_int = {
4877 BPF_LD_IMM64(R1, 0x567800001234LL),
4878 BPF_MOV64_REG(R2, R1),
4879 BPF_MOV64_REG(R3, R2),
4880 BPF_ALU64_IMM(BPF_RSH, R2, 32),
4881 BPF_ALU64_IMM(BPF_LSH, R3, 32),
4882 BPF_ALU64_IMM(BPF_RSH, R3, 32),
4883 BPF_ALU64_IMM(BPF_MOV, R0, 0),
4884 BPF_JMP_IMM(BPF_JEQ, R2, 0x5678, 1),
4885 BPF_EXIT_INSN(),
4886 BPF_JMP_IMM(BPF_JEQ, R3, 0x1234, 1),
4887 BPF_EXIT_INSN(),
4888 BPF_LD_IMM64(R0, 0x1ffffffffLL),
4889 BPF_ALU64_IMM(BPF_RSH, R0, 32), /* R0 = 1 */
4890 BPF_EXIT_INSN(),
4891 },
4892 INTERNAL,
4893 { },
4894 { { 0, 1 } }
4895 },
4896 /* BPF_ALU | BPF_MOV | BPF_X */
4897 {
4898 "ALU_MOV_X: dst = 2",
4899 .u.insns_int = {
4900 BPF_ALU32_IMM(BPF_MOV, R1, 2),
4901 BPF_ALU32_REG(BPF_MOV, R0, R1),
4902 BPF_EXIT_INSN(),
4903 },
4904 INTERNAL,
4905 { },
4906 { { 0, 2 } },
4907 },
4908 {
4909 "ALU_MOV_X: dst = 4294967295",
4910 .u.insns_int = {
4911 BPF_ALU32_IMM(BPF_MOV, R1, 4294967295U),
4912 BPF_ALU32_REG(BPF_MOV, R0, R1),
4913 BPF_EXIT_INSN(),
4914 },
4915 INTERNAL,
4916 { },
4917 { { 0, 4294967295U } },
4918 },
4919 {
4920 "ALU64_MOV_X: dst = 2",
4921 .u.insns_int = {
4922 BPF_ALU32_IMM(BPF_MOV, R1, 2),
4923 BPF_ALU64_REG(BPF_MOV, R0, R1),
4924 BPF_EXIT_INSN(),
4925 },
4926 INTERNAL,
4927 { },
4928 { { 0, 2 } },
4929 },
4930 {
4931 "ALU64_MOV_X: dst = 4294967295",
4932 .u.insns_int = {
4933 BPF_ALU32_IMM(BPF_MOV, R1, 4294967295U),
4934 BPF_ALU64_REG(BPF_MOV, R0, R1),
4935 BPF_EXIT_INSN(),
4936 },
4937 INTERNAL,
4938 { },
4939 { { 0, 4294967295U } },
4940 },
4941 /* BPF_ALU | BPF_MOV | BPF_K */
4942 {
4943 "ALU_MOV_K: dst = 2",
4944 .u.insns_int = {
4945 BPF_ALU32_IMM(BPF_MOV, R0, 2),
4946 BPF_EXIT_INSN(),
4947 },
4948 INTERNAL,
4949 { },
4950 { { 0, 2 } },
4951 },
4952 {
4953 "ALU_MOV_K: dst = 4294967295",
4954 .u.insns_int = {
4955 BPF_ALU32_IMM(BPF_MOV, R0, 4294967295U),
4956 BPF_EXIT_INSN(),
4957 },
4958 INTERNAL,
4959 { },
4960 { { 0, 4294967295U } },
4961 },
4962 {
4963 "ALU_MOV_K: 0x0000ffffffff0000 = 0x00000000ffffffff",
4964 .u.insns_int = {
4965 BPF_LD_IMM64(R2, 0x0000ffffffff0000LL),
4966 BPF_LD_IMM64(R3, 0x00000000ffffffffLL),
4967 BPF_ALU32_IMM(BPF_MOV, R2, 0xffffffff),
4968 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
4969 BPF_MOV32_IMM(R0, 2),
4970 BPF_EXIT_INSN(),
4971 BPF_MOV32_IMM(R0, 1),
4972 BPF_EXIT_INSN(),
4973 },
4974 INTERNAL,
4975 { },
4976 { { 0, 0x1 } },
4977 },
4978 {
4979 "ALU_MOV_K: small negative",
4980 .u.insns_int = {
4981 BPF_ALU32_IMM(BPF_MOV, R0, -123),
4982 BPF_EXIT_INSN(),
4983 },
4984 INTERNAL,
4985 { },
4986 { { 0, -123 } }
4987 },
4988 {
4989 "ALU_MOV_K: small negative zero extension",
4990 .u.insns_int = {
4991 BPF_ALU32_IMM(BPF_MOV, R0, -123),
4992 BPF_ALU64_IMM(BPF_RSH, R0, 32),
4993 BPF_EXIT_INSN(),
4994 },
4995 INTERNAL,
4996 { },
4997 { { 0, 0 } }
4998 },
4999 {
5000 "ALU_MOV_K: large negative",
5001 .u.insns_int = {
5002 BPF_ALU32_IMM(BPF_MOV, R0, -123456789),
5003 BPF_EXIT_INSN(),
5004 },
5005 INTERNAL,
5006 { },
5007 { { 0, -123456789 } }
5008 },
5009 {
5010 "ALU_MOV_K: large negative zero extension",
5011 .u.insns_int = {
5012 BPF_ALU32_IMM(BPF_MOV, R0, -123456789),
5013 BPF_ALU64_IMM(BPF_RSH, R0, 32),
5014 BPF_EXIT_INSN(),
5015 },
5016 INTERNAL,
5017 { },
5018 { { 0, 0 } }
5019 },
5020 {
5021 "ALU64_MOV_K: dst = 2",
5022 .u.insns_int = {
5023 BPF_ALU64_IMM(BPF_MOV, R0, 2),
5024 BPF_EXIT_INSN(),
5025 },
5026 INTERNAL,
5027 { },
5028 { { 0, 2 } },
5029 },
5030 {
5031 "ALU64_MOV_K: dst = 2147483647",
5032 .u.insns_int = {
5033 BPF_ALU64_IMM(BPF_MOV, R0, 2147483647),
5034 BPF_EXIT_INSN(),
5035 },
5036 INTERNAL,
5037 { },
5038 { { 0, 2147483647 } },
5039 },
5040 {
5041 "ALU64_OR_K: dst = 0x0",
5042 .u.insns_int = {
5043 BPF_LD_IMM64(R2, 0x0000ffffffff0000LL),
5044 BPF_LD_IMM64(R3, 0x0),
5045 BPF_ALU64_IMM(BPF_MOV, R2, 0x0),
5046 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5047 BPF_MOV32_IMM(R0, 2),
5048 BPF_EXIT_INSN(),
5049 BPF_MOV32_IMM(R0, 1),
5050 BPF_EXIT_INSN(),
5051 },
5052 INTERNAL,
5053 { },
5054 { { 0, 0x1 } },
5055 },
5056 {
5057 "ALU64_MOV_K: dst = -1",
5058 .u.insns_int = {
5059 BPF_LD_IMM64(R2, 0x0000ffffffff0000LL),
5060 BPF_LD_IMM64(R3, 0xffffffffffffffffLL),
5061 BPF_ALU64_IMM(BPF_MOV, R2, 0xffffffff),
5062 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5063 BPF_MOV32_IMM(R0, 2),
5064 BPF_EXIT_INSN(),
5065 BPF_MOV32_IMM(R0, 1),
5066 BPF_EXIT_INSN(),
5067 },
5068 INTERNAL,
5069 { },
5070 { { 0, 0x1 } },
5071 },
5072 {
5073 "ALU64_MOV_K: small negative",
5074 .u.insns_int = {
5075 BPF_ALU64_IMM(BPF_MOV, R0, -123),
5076 BPF_EXIT_INSN(),
5077 },
5078 INTERNAL,
5079 { },
5080 { { 0, -123 } }
5081 },
5082 {
5083 "ALU64_MOV_K: small negative sign extension",
5084 .u.insns_int = {
5085 BPF_ALU64_IMM(BPF_MOV, R0, -123),
5086 BPF_ALU64_IMM(BPF_RSH, R0, 32),
5087 BPF_EXIT_INSN(),
5088 },
5089 INTERNAL,
5090 { },
5091 { { 0, 0xffffffff } }
5092 },
5093 {
5094 "ALU64_MOV_K: large negative",
5095 .u.insns_int = {
5096 BPF_ALU64_IMM(BPF_MOV, R0, -123456789),
5097 BPF_EXIT_INSN(),
5098 },
5099 INTERNAL,
5100 { },
5101 { { 0, -123456789 } }
5102 },
5103 {
5104 "ALU64_MOV_K: large negative sign extension",
5105 .u.insns_int = {
5106 BPF_ALU64_IMM(BPF_MOV, R0, -123456789),
5107 BPF_ALU64_IMM(BPF_RSH, R0, 32),
5108 BPF_EXIT_INSN(),
5109 },
5110 INTERNAL,
5111 { },
5112 { { 0, 0xffffffff } }
5113 },
5114 /* BPF_ALU | BPF_ADD | BPF_X */
5115 {
5116 "ALU_ADD_X: 1 + 2 = 3",
5117 .u.insns_int = {
5118 BPF_LD_IMM64(R0, 1),
5119 BPF_ALU32_IMM(BPF_MOV, R1, 2),
5120 BPF_ALU32_REG(BPF_ADD, R0, R1),
5121 BPF_EXIT_INSN(),
5122 },
5123 INTERNAL,
5124 { },
5125 { { 0, 3 } },
5126 },
5127 {
5128 "ALU_ADD_X: 1 + 4294967294 = 4294967295",
5129 .u.insns_int = {
5130 BPF_LD_IMM64(R0, 1),
5131 BPF_ALU32_IMM(BPF_MOV, R1, 4294967294U),
5132 BPF_ALU32_REG(BPF_ADD, R0, R1),
5133 BPF_EXIT_INSN(),
5134 },
5135 INTERNAL,
5136 { },
5137 { { 0, 4294967295U } },
5138 },
5139 {
5140 "ALU_ADD_X: 2 + 4294967294 = 0",
5141 .u.insns_int = {
5142 BPF_LD_IMM64(R0, 2),
5143 BPF_LD_IMM64(R1, 4294967294U),
5144 BPF_ALU32_REG(BPF_ADD, R0, R1),
5145 BPF_JMP_IMM(BPF_JEQ, R0, 0, 2),
5146 BPF_ALU32_IMM(BPF_MOV, R0, 0),
5147 BPF_EXIT_INSN(),
5148 BPF_ALU32_IMM(BPF_MOV, R0, 1),
5149 BPF_EXIT_INSN(),
5150 },
5151 INTERNAL,
5152 { },
5153 { { 0, 1 } },
5154 },
5155 {
5156 "ALU64_ADD_X: 1 + 2 = 3",
5157 .u.insns_int = {
5158 BPF_LD_IMM64(R0, 1),
5159 BPF_ALU32_IMM(BPF_MOV, R1, 2),
5160 BPF_ALU64_REG(BPF_ADD, R0, R1),
5161 BPF_EXIT_INSN(),
5162 },
5163 INTERNAL,
5164 { },
5165 { { 0, 3 } },
5166 },
5167 {
5168 "ALU64_ADD_X: 1 + 4294967294 = 4294967295",
5169 .u.insns_int = {
5170 BPF_LD_IMM64(R0, 1),
5171 BPF_ALU32_IMM(BPF_MOV, R1, 4294967294U),
5172 BPF_ALU64_REG(BPF_ADD, R0, R1),
5173 BPF_EXIT_INSN(),
5174 },
5175 INTERNAL,
5176 { },
5177 { { 0, 4294967295U } },
5178 },
5179 {
5180 "ALU64_ADD_X: 2 + 4294967294 = 4294967296",
5181 .u.insns_int = {
5182 BPF_LD_IMM64(R0, 2),
5183 BPF_LD_IMM64(R1, 4294967294U),
5184 BPF_LD_IMM64(R2, 4294967296ULL),
5185 BPF_ALU64_REG(BPF_ADD, R0, R1),
5186 BPF_JMP_REG(BPF_JEQ, R0, R2, 2),
5187 BPF_MOV32_IMM(R0, 0),
5188 BPF_EXIT_INSN(),
5189 BPF_MOV32_IMM(R0, 1),
5190 BPF_EXIT_INSN(),
5191 },
5192 INTERNAL,
5193 { },
5194 { { 0, 1 } },
5195 },
5196 /* BPF_ALU | BPF_ADD | BPF_K */
5197 {
5198 "ALU_ADD_K: 1 + 2 = 3",
5199 .u.insns_int = {
5200 BPF_LD_IMM64(R0, 1),
5201 BPF_ALU32_IMM(BPF_ADD, R0, 2),
5202 BPF_EXIT_INSN(),
5203 },
5204 INTERNAL,
5205 { },
5206 { { 0, 3 } },
5207 },
5208 {
5209 "ALU_ADD_K: 3 + 0 = 3",
5210 .u.insns_int = {
5211 BPF_LD_IMM64(R0, 3),
5212 BPF_ALU32_IMM(BPF_ADD, R0, 0),
5213 BPF_EXIT_INSN(),
5214 },
5215 INTERNAL,
5216 { },
5217 { { 0, 3 } },
5218 },
5219 {
5220 "ALU_ADD_K: 1 + 4294967294 = 4294967295",
5221 .u.insns_int = {
5222 BPF_LD_IMM64(R0, 1),
5223 BPF_ALU32_IMM(BPF_ADD, R0, 4294967294U),
5224 BPF_EXIT_INSN(),
5225 },
5226 INTERNAL,
5227 { },
5228 { { 0, 4294967295U } },
5229 },
5230 {
5231 "ALU_ADD_K: 4294967294 + 2 = 0",
5232 .u.insns_int = {
5233 BPF_LD_IMM64(R0, 4294967294U),
5234 BPF_ALU32_IMM(BPF_ADD, R0, 2),
5235 BPF_JMP_IMM(BPF_JEQ, R0, 0, 2),
5236 BPF_ALU32_IMM(BPF_MOV, R0, 0),
5237 BPF_EXIT_INSN(),
5238 BPF_ALU32_IMM(BPF_MOV, R0, 1),
5239 BPF_EXIT_INSN(),
5240 },
5241 INTERNAL,
5242 { },
5243 { { 0, 1 } },
5244 },
5245 {
5246 "ALU_ADD_K: 0 + (-1) = 0x00000000ffffffff",
5247 .u.insns_int = {
5248 BPF_LD_IMM64(R2, 0x0),
5249 BPF_LD_IMM64(R3, 0x00000000ffffffff),
5250 BPF_ALU32_IMM(BPF_ADD, R2, 0xffffffff),
5251 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5252 BPF_MOV32_IMM(R0, 2),
5253 BPF_EXIT_INSN(),
5254 BPF_MOV32_IMM(R0, 1),
5255 BPF_EXIT_INSN(),
5256 },
5257 INTERNAL,
5258 { },
5259 { { 0, 0x1 } },
5260 },
5261 {
5262 "ALU_ADD_K: 0 + 0xffff = 0xffff",
5263 .u.insns_int = {
5264 BPF_LD_IMM64(R2, 0x0),
5265 BPF_LD_IMM64(R3, 0xffff),
5266 BPF_ALU32_IMM(BPF_ADD, R2, 0xffff),
5267 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5268 BPF_MOV32_IMM(R0, 2),
5269 BPF_EXIT_INSN(),
5270 BPF_MOV32_IMM(R0, 1),
5271 BPF_EXIT_INSN(),
5272 },
5273 INTERNAL,
5274 { },
5275 { { 0, 0x1 } },
5276 },
5277 {
5278 "ALU_ADD_K: 0 + 0x7fffffff = 0x7fffffff",
5279 .u.insns_int = {
5280 BPF_LD_IMM64(R2, 0x0),
5281 BPF_LD_IMM64(R3, 0x7fffffff),
5282 BPF_ALU32_IMM(BPF_ADD, R2, 0x7fffffff),
5283 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5284 BPF_MOV32_IMM(R0, 2),
5285 BPF_EXIT_INSN(),
5286 BPF_MOV32_IMM(R0, 1),
5287 BPF_EXIT_INSN(),
5288 },
5289 INTERNAL,
5290 { },
5291 { { 0, 0x1 } },
5292 },
5293 {
5294 "ALU_ADD_K: 0 + 0x80000000 = 0x80000000",
5295 .u.insns_int = {
5296 BPF_LD_IMM64(R2, 0x0),
5297 BPF_LD_IMM64(R3, 0x80000000),
5298 BPF_ALU32_IMM(BPF_ADD, R2, 0x80000000),
5299 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5300 BPF_MOV32_IMM(R0, 2),
5301 BPF_EXIT_INSN(),
5302 BPF_MOV32_IMM(R0, 1),
5303 BPF_EXIT_INSN(),
5304 },
5305 INTERNAL,
5306 { },
5307 { { 0, 0x1 } },
5308 },
5309 {
5310 "ALU_ADD_K: 0 + 0x80008000 = 0x80008000",
5311 .u.insns_int = {
5312 BPF_LD_IMM64(R2, 0x0),
5313 BPF_LD_IMM64(R3, 0x80008000),
5314 BPF_ALU32_IMM(BPF_ADD, R2, 0x80008000),
5315 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5316 BPF_MOV32_IMM(R0, 2),
5317 BPF_EXIT_INSN(),
5318 BPF_MOV32_IMM(R0, 1),
5319 BPF_EXIT_INSN(),
5320 },
5321 INTERNAL,
5322 { },
5323 { { 0, 0x1 } },
5324 },
5325 {
5326 "ALU64_ADD_K: 1 + 2 = 3",
5327 .u.insns_int = {
5328 BPF_LD_IMM64(R0, 1),
5329 BPF_ALU64_IMM(BPF_ADD, R0, 2),
5330 BPF_EXIT_INSN(),
5331 },
5332 INTERNAL,
5333 { },
5334 { { 0, 3 } },
5335 },
5336 {
5337 "ALU64_ADD_K: 3 + 0 = 3",
5338 .u.insns_int = {
5339 BPF_LD_IMM64(R0, 3),
5340 BPF_ALU64_IMM(BPF_ADD, R0, 0),
5341 BPF_EXIT_INSN(),
5342 },
5343 INTERNAL,
5344 { },
5345 { { 0, 3 } },
5346 },
5347 {
5348 "ALU64_ADD_K: 1 + 2147483646 = 2147483647",
5349 .u.insns_int = {
5350 BPF_LD_IMM64(R0, 1),
5351 BPF_ALU64_IMM(BPF_ADD, R0, 2147483646),
5352 BPF_EXIT_INSN(),
5353 },
5354 INTERNAL,
5355 { },
5356 { { 0, 2147483647 } },
5357 },
5358 {
5359 "ALU64_ADD_K: 4294967294 + 2 = 4294967296",
5360 .u.insns_int = {
5361 BPF_LD_IMM64(R0, 4294967294U),
5362 BPF_LD_IMM64(R1, 4294967296ULL),
5363 BPF_ALU64_IMM(BPF_ADD, R0, 2),
5364 BPF_JMP_REG(BPF_JEQ, R0, R1, 2),
5365 BPF_ALU32_IMM(BPF_MOV, R0, 0),
5366 BPF_EXIT_INSN(),
5367 BPF_ALU32_IMM(BPF_MOV, R0, 1),
5368 BPF_EXIT_INSN(),
5369 },
5370 INTERNAL,
5371 { },
5372 { { 0, 1 } },
5373 },
5374 {
5375 "ALU64_ADD_K: 2147483646 + -2147483647 = -1",
5376 .u.insns_int = {
5377 BPF_LD_IMM64(R0, 2147483646),
5378 BPF_ALU64_IMM(BPF_ADD, R0, -2147483647),
5379 BPF_EXIT_INSN(),
5380 },
5381 INTERNAL,
5382 { },
5383 { { 0, -1 } },
5384 },
5385 {
5386 "ALU64_ADD_K: 1 + 0 = 1",
5387 .u.insns_int = {
5388 BPF_LD_IMM64(R2, 0x1),
5389 BPF_LD_IMM64(R3, 0x1),
5390 BPF_ALU64_IMM(BPF_ADD, R2, 0x0),
5391 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5392 BPF_MOV32_IMM(R0, 2),
5393 BPF_EXIT_INSN(),
5394 BPF_MOV32_IMM(R0, 1),
5395 BPF_EXIT_INSN(),
5396 },
5397 INTERNAL,
5398 { },
5399 { { 0, 0x1 } },
5400 },
5401 {
5402 "ALU64_ADD_K: 0 + (-1) = 0xffffffffffffffff",
5403 .u.insns_int = {
5404 BPF_LD_IMM64(R2, 0x0),
5405 BPF_LD_IMM64(R3, 0xffffffffffffffffLL),
5406 BPF_ALU64_IMM(BPF_ADD, R2, 0xffffffff),
5407 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5408 BPF_MOV32_IMM(R0, 2),
5409 BPF_EXIT_INSN(),
5410 BPF_MOV32_IMM(R0, 1),
5411 BPF_EXIT_INSN(),
5412 },
5413 INTERNAL,
5414 { },
5415 { { 0, 0x1 } },
5416 },
5417 {
5418 "ALU64_ADD_K: 0 + 0xffff = 0xffff",
5419 .u.insns_int = {
5420 BPF_LD_IMM64(R2, 0x0),
5421 BPF_LD_IMM64(R3, 0xffff),
5422 BPF_ALU64_IMM(BPF_ADD, R2, 0xffff),
5423 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5424 BPF_MOV32_IMM(R0, 2),
5425 BPF_EXIT_INSN(),
5426 BPF_MOV32_IMM(R0, 1),
5427 BPF_EXIT_INSN(),
5428 },
5429 INTERNAL,
5430 { },
5431 { { 0, 0x1 } },
5432 },
5433 {
5434 "ALU64_ADD_K: 0 + 0x7fffffff = 0x7fffffff",
5435 .u.insns_int = {
5436 BPF_LD_IMM64(R2, 0x0),
5437 BPF_LD_IMM64(R3, 0x7fffffff),
5438 BPF_ALU64_IMM(BPF_ADD, R2, 0x7fffffff),
5439 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5440 BPF_MOV32_IMM(R0, 2),
5441 BPF_EXIT_INSN(),
5442 BPF_MOV32_IMM(R0, 1),
5443 BPF_EXIT_INSN(),
5444 },
5445 INTERNAL,
5446 { },
5447 { { 0, 0x1 } },
5448 },
5449 {
5450 "ALU64_ADD_K: 0 + 0x80000000 = 0xffffffff80000000",
5451 .u.insns_int = {
5452 BPF_LD_IMM64(R2, 0x0),
5453 BPF_LD_IMM64(R3, 0xffffffff80000000LL),
5454 BPF_ALU64_IMM(BPF_ADD, R2, 0x80000000),
5455 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5456 BPF_MOV32_IMM(R0, 2),
5457 BPF_EXIT_INSN(),
5458 BPF_MOV32_IMM(R0, 1),
5459 BPF_EXIT_INSN(),
5460 },
5461 INTERNAL,
5462 { },
5463 { { 0, 0x1 } },
5464 },
5465 {
5466 "ALU_ADD_K: 0 + 0x80008000 = 0xffffffff80008000",
5467 .u.insns_int = {
5468 BPF_LD_IMM64(R2, 0x0),
5469 BPF_LD_IMM64(R3, 0xffffffff80008000LL),
5470 BPF_ALU64_IMM(BPF_ADD, R2, 0x80008000),
5471 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5472 BPF_MOV32_IMM(R0, 2),
5473 BPF_EXIT_INSN(),
5474 BPF_MOV32_IMM(R0, 1),
5475 BPF_EXIT_INSN(),
5476 },
5477 INTERNAL,
5478 { },
5479 { { 0, 0x1 } },
5480 },
5481 /* BPF_ALU | BPF_SUB | BPF_X */
5482 {
5483 "ALU_SUB_X: 3 - 1 = 2",
5484 .u.insns_int = {
5485 BPF_LD_IMM64(R0, 3),
5486 BPF_ALU32_IMM(BPF_MOV, R1, 1),
5487 BPF_ALU32_REG(BPF_SUB, R0, R1),
5488 BPF_EXIT_INSN(),
5489 },
5490 INTERNAL,
5491 { },
5492 { { 0, 2 } },
5493 },
5494 {
5495 "ALU_SUB_X: 4294967295 - 4294967294 = 1",
5496 .u.insns_int = {
5497 BPF_LD_IMM64(R0, 4294967295U),
5498 BPF_ALU32_IMM(BPF_MOV, R1, 4294967294U),
5499 BPF_ALU32_REG(BPF_SUB, R0, R1),
5500 BPF_EXIT_INSN(),
5501 },
5502 INTERNAL,
5503 { },
5504 { { 0, 1 } },
5505 },
5506 {
5507 "ALU64_SUB_X: 3 - 1 = 2",
5508 .u.insns_int = {
5509 BPF_LD_IMM64(R0, 3),
5510 BPF_ALU32_IMM(BPF_MOV, R1, 1),
5511 BPF_ALU64_REG(BPF_SUB, R0, R1),
5512 BPF_EXIT_INSN(),
5513 },
5514 INTERNAL,
5515 { },
5516 { { 0, 2 } },
5517 },
5518 {
5519 "ALU64_SUB_X: 4294967295 - 4294967294 = 1",
5520 .u.insns_int = {
5521 BPF_LD_IMM64(R0, 4294967295U),
5522 BPF_ALU32_IMM(BPF_MOV, R1, 4294967294U),
5523 BPF_ALU64_REG(BPF_SUB, R0, R1),
5524 BPF_EXIT_INSN(),
5525 },
5526 INTERNAL,
5527 { },
5528 { { 0, 1 } },
5529 },
5530 /* BPF_ALU | BPF_SUB | BPF_K */
5531 {
5532 "ALU_SUB_K: 3 - 1 = 2",
5533 .u.insns_int = {
5534 BPF_LD_IMM64(R0, 3),
5535 BPF_ALU32_IMM(BPF_SUB, R0, 1),
5536 BPF_EXIT_INSN(),
5537 },
5538 INTERNAL,
5539 { },
5540 { { 0, 2 } },
5541 },
5542 {
5543 "ALU_SUB_K: 3 - 0 = 3",
5544 .u.insns_int = {
5545 BPF_LD_IMM64(R0, 3),
5546 BPF_ALU32_IMM(BPF_SUB, R0, 0),
5547 BPF_EXIT_INSN(),
5548 },
5549 INTERNAL,
5550 { },
5551 { { 0, 3 } },
5552 },
5553 {
5554 "ALU_SUB_K: 4294967295 - 4294967294 = 1",
5555 .u.insns_int = {
5556 BPF_LD_IMM64(R0, 4294967295U),
5557 BPF_ALU32_IMM(BPF_SUB, R0, 4294967294U),
5558 BPF_EXIT_INSN(),
5559 },
5560 INTERNAL,
5561 { },
5562 { { 0, 1 } },
5563 },
5564 {
5565 "ALU64_SUB_K: 3 - 1 = 2",
5566 .u.insns_int = {
5567 BPF_LD_IMM64(R0, 3),
5568 BPF_ALU64_IMM(BPF_SUB, R0, 1),
5569 BPF_EXIT_INSN(),
5570 },
5571 INTERNAL,
5572 { },
5573 { { 0, 2 } },
5574 },
5575 {
5576 "ALU64_SUB_K: 3 - 0 = 3",
5577 .u.insns_int = {
5578 BPF_LD_IMM64(R0, 3),
5579 BPF_ALU64_IMM(BPF_SUB, R0, 0),
5580 BPF_EXIT_INSN(),
5581 },
5582 INTERNAL,
5583 { },
5584 { { 0, 3 } },
5585 },
5586 {
5587 "ALU64_SUB_K: 4294967294 - 4294967295 = -1",
5588 .u.insns_int = {
5589 BPF_LD_IMM64(R0, 4294967294U),
5590 BPF_ALU64_IMM(BPF_SUB, R0, 4294967295U),
5591 BPF_EXIT_INSN(),
5592 },
5593 INTERNAL,
5594 { },
5595 { { 0, -1 } },
5596 },
5597 {
5598 "ALU64_ADD_K: 2147483646 - 2147483647 = -1",
5599 .u.insns_int = {
5600 BPF_LD_IMM64(R0, 2147483646),
5601 BPF_ALU64_IMM(BPF_SUB, R0, 2147483647),
5602 BPF_EXIT_INSN(),
5603 },
5604 INTERNAL,
5605 { },
5606 { { 0, -1 } },
5607 },
5608 /* BPF_ALU | BPF_MUL | BPF_X */
5609 {
5610 "ALU_MUL_X: 2 * 3 = 6",
5611 .u.insns_int = {
5612 BPF_LD_IMM64(R0, 2),
5613 BPF_ALU32_IMM(BPF_MOV, R1, 3),
5614 BPF_ALU32_REG(BPF_MUL, R0, R1),
5615 BPF_EXIT_INSN(),
5616 },
5617 INTERNAL,
5618 { },
5619 { { 0, 6 } },
5620 },
5621 {
5622 "ALU_MUL_X: 2 * 0x7FFFFFF8 = 0xFFFFFFF0",
5623 .u.insns_int = {
5624 BPF_LD_IMM64(R0, 2),
5625 BPF_ALU32_IMM(BPF_MOV, R1, 0x7FFFFFF8),
5626 BPF_ALU32_REG(BPF_MUL, R0, R1),
5627 BPF_EXIT_INSN(),
5628 },
5629 INTERNAL,
5630 { },
5631 { { 0, 0xFFFFFFF0 } },
5632 },
5633 {
5634 "ALU_MUL_X: -1 * -1 = 1",
5635 .u.insns_int = {
5636 BPF_LD_IMM64(R0, -1),
5637 BPF_ALU32_IMM(BPF_MOV, R1, -1),
5638 BPF_ALU32_REG(BPF_MUL, R0, R1),
5639 BPF_EXIT_INSN(),
5640 },
5641 INTERNAL,
5642 { },
5643 { { 0, 1 } },
5644 },
5645 {
5646 "ALU64_MUL_X: 2 * 3 = 6",
5647 .u.insns_int = {
5648 BPF_LD_IMM64(R0, 2),
5649 BPF_ALU32_IMM(BPF_MOV, R1, 3),
5650 BPF_ALU64_REG(BPF_MUL, R0, R1),
5651 BPF_EXIT_INSN(),
5652 },
5653 INTERNAL,
5654 { },
5655 { { 0, 6 } },
5656 },
5657 {
5658 "ALU64_MUL_X: 1 * 2147483647 = 2147483647",
5659 .u.insns_int = {
5660 BPF_LD_IMM64(R0, 1),
5661 BPF_ALU32_IMM(BPF_MOV, R1, 2147483647),
5662 BPF_ALU64_REG(BPF_MUL, R0, R1),
5663 BPF_EXIT_INSN(),
5664 },
5665 INTERNAL,
5666 { },
5667 { { 0, 2147483647 } },
5668 },
5669 {
5670 "ALU64_MUL_X: 64x64 multiply, low word",
5671 .u.insns_int = {
5672 BPF_LD_IMM64(R0, 0x0fedcba987654321LL),
5673 BPF_LD_IMM64(R1, 0x123456789abcdef0LL),
5674 BPF_ALU64_REG(BPF_MUL, R0, R1),
5675 BPF_EXIT_INSN(),
5676 },
5677 INTERNAL,
5678 { },
5679 { { 0, 0xe5618cf0 } }
5680 },
5681 {
5682 "ALU64_MUL_X: 64x64 multiply, high word",
5683 .u.insns_int = {
5684 BPF_LD_IMM64(R0, 0x0fedcba987654321LL),
5685 BPF_LD_IMM64(R1, 0x123456789abcdef0LL),
5686 BPF_ALU64_REG(BPF_MUL, R0, R1),
5687 BPF_ALU64_IMM(BPF_RSH, R0, 32),
5688 BPF_EXIT_INSN(),
5689 },
5690 INTERNAL,
5691 { },
5692 { { 0, 0x2236d88f } }
5693 },
5694 /* BPF_ALU | BPF_MUL | BPF_K */
5695 {
5696 "ALU_MUL_K: 2 * 3 = 6",
5697 .u.insns_int = {
5698 BPF_LD_IMM64(R0, 2),
5699 BPF_ALU32_IMM(BPF_MUL, R0, 3),
5700 BPF_EXIT_INSN(),
5701 },
5702 INTERNAL,
5703 { },
5704 { { 0, 6 } },
5705 },
5706 {
5707 "ALU_MUL_K: 3 * 1 = 3",
5708 .u.insns_int = {
5709 BPF_LD_IMM64(R0, 3),
5710 BPF_ALU32_IMM(BPF_MUL, R0, 1),
5711 BPF_EXIT_INSN(),
5712 },
5713 INTERNAL,
5714 { },
5715 { { 0, 3 } },
5716 },
5717 {
5718 "ALU_MUL_K: 2 * 0x7FFFFFF8 = 0xFFFFFFF0",
5719 .u.insns_int = {
5720 BPF_LD_IMM64(R0, 2),
5721 BPF_ALU32_IMM(BPF_MUL, R0, 0x7FFFFFF8),
5722 BPF_EXIT_INSN(),
5723 },
5724 INTERNAL,
5725 { },
5726 { { 0, 0xFFFFFFF0 } },
5727 },
5728 {
5729 "ALU_MUL_K: 1 * (-1) = 0x00000000ffffffff",
5730 .u.insns_int = {
5731 BPF_LD_IMM64(R2, 0x1),
5732 BPF_LD_IMM64(R3, 0x00000000ffffffff),
5733 BPF_ALU32_IMM(BPF_MUL, R2, 0xffffffff),
5734 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5735 BPF_MOV32_IMM(R0, 2),
5736 BPF_EXIT_INSN(),
5737 BPF_MOV32_IMM(R0, 1),
5738 BPF_EXIT_INSN(),
5739 },
5740 INTERNAL,
5741 { },
5742 { { 0, 0x1 } },
5743 },
5744 {
5745 "ALU64_MUL_K: 2 * 3 = 6",
5746 .u.insns_int = {
5747 BPF_LD_IMM64(R0, 2),
5748 BPF_ALU64_IMM(BPF_MUL, R0, 3),
5749 BPF_EXIT_INSN(),
5750 },
5751 INTERNAL,
5752 { },
5753 { { 0, 6 } },
5754 },
5755 {
5756 "ALU64_MUL_K: 3 * 1 = 3",
5757 .u.insns_int = {
5758 BPF_LD_IMM64(R0, 3),
5759 BPF_ALU64_IMM(BPF_MUL, R0, 1),
5760 BPF_EXIT_INSN(),
5761 },
5762 INTERNAL,
5763 { },
5764 { { 0, 3 } },
5765 },
5766 {
5767 "ALU64_MUL_K: 1 * 2147483647 = 2147483647",
5768 .u.insns_int = {
5769 BPF_LD_IMM64(R0, 1),
5770 BPF_ALU64_IMM(BPF_MUL, R0, 2147483647),
5771 BPF_EXIT_INSN(),
5772 },
5773 INTERNAL,
5774 { },
5775 { { 0, 2147483647 } },
5776 },
5777 {
5778 "ALU64_MUL_K: 1 * -2147483647 = -2147483647",
5779 .u.insns_int = {
5780 BPF_LD_IMM64(R0, 1),
5781 BPF_ALU64_IMM(BPF_MUL, R0, -2147483647),
5782 BPF_EXIT_INSN(),
5783 },
5784 INTERNAL,
5785 { },
5786 { { 0, -2147483647 } },
5787 },
5788 {
5789 "ALU64_MUL_K: 1 * (-1) = 0xffffffffffffffff",
5790 .u.insns_int = {
5791 BPF_LD_IMM64(R2, 0x1),
5792 BPF_LD_IMM64(R3, 0xffffffffffffffffLL),
5793 BPF_ALU64_IMM(BPF_MUL, R2, 0xffffffff),
5794 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5795 BPF_MOV32_IMM(R0, 2),
5796 BPF_EXIT_INSN(),
5797 BPF_MOV32_IMM(R0, 1),
5798 BPF_EXIT_INSN(),
5799 },
5800 INTERNAL,
5801 { },
5802 { { 0, 0x1 } },
5803 },
5804 {
5805 "ALU64_MUL_K: 64x32 multiply, low word",
5806 .u.insns_int = {
5807 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
5808 BPF_ALU64_IMM(BPF_MUL, R0, 0x12345678),
5809 BPF_EXIT_INSN(),
5810 },
5811 INTERNAL,
5812 { },
5813 { { 0, 0xe242d208 } }
5814 },
5815 {
5816 "ALU64_MUL_K: 64x32 multiply, high word",
5817 .u.insns_int = {
5818 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
5819 BPF_ALU64_IMM(BPF_MUL, R0, 0x12345678),
5820 BPF_ALU64_IMM(BPF_RSH, R0, 32),
5821 BPF_EXIT_INSN(),
5822 },
5823 INTERNAL,
5824 { },
5825 { { 0, 0xc28f5c28 } }
5826 },
5827 /* BPF_ALU | BPF_DIV | BPF_X */
5828 {
5829 "ALU_DIV_X: 6 / 2 = 3",
5830 .u.insns_int = {
5831 BPF_LD_IMM64(R0, 6),
5832 BPF_ALU32_IMM(BPF_MOV, R1, 2),
5833 BPF_ALU32_REG(BPF_DIV, R0, R1),
5834 BPF_EXIT_INSN(),
5835 },
5836 INTERNAL,
5837 { },
5838 { { 0, 3 } },
5839 },
5840 {
5841 "ALU_DIV_X: 4294967295 / 4294967295 = 1",
5842 .u.insns_int = {
5843 BPF_LD_IMM64(R0, 4294967295U),
5844 BPF_ALU32_IMM(BPF_MOV, R1, 4294967295U),
5845 BPF_ALU32_REG(BPF_DIV, R0, R1),
5846 BPF_EXIT_INSN(),
5847 },
5848 INTERNAL,
5849 { },
5850 { { 0, 1 } },
5851 },
5852 {
5853 "ALU64_DIV_X: 6 / 2 = 3",
5854 .u.insns_int = {
5855 BPF_LD_IMM64(R0, 6),
5856 BPF_ALU32_IMM(BPF_MOV, R1, 2),
5857 BPF_ALU64_REG(BPF_DIV, R0, R1),
5858 BPF_EXIT_INSN(),
5859 },
5860 INTERNAL,
5861 { },
5862 { { 0, 3 } },
5863 },
5864 {
5865 "ALU64_DIV_X: 2147483647 / 2147483647 = 1",
5866 .u.insns_int = {
5867 BPF_LD_IMM64(R0, 2147483647),
5868 BPF_ALU32_IMM(BPF_MOV, R1, 2147483647),
5869 BPF_ALU64_REG(BPF_DIV, R0, R1),
5870 BPF_EXIT_INSN(),
5871 },
5872 INTERNAL,
5873 { },
5874 { { 0, 1 } },
5875 },
5876 {
5877 "ALU64_DIV_X: 0xffffffffffffffff / (-1) = 0x0000000000000001",
5878 .u.insns_int = {
5879 BPF_LD_IMM64(R2, 0xffffffffffffffffLL),
5880 BPF_LD_IMM64(R4, 0xffffffffffffffffLL),
5881 BPF_LD_IMM64(R3, 0x0000000000000001LL),
5882 BPF_ALU64_REG(BPF_DIV, R2, R4),
5883 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5884 BPF_MOV32_IMM(R0, 2),
5885 BPF_EXIT_INSN(),
5886 BPF_MOV32_IMM(R0, 1),
5887 BPF_EXIT_INSN(),
5888 },
5889 INTERNAL,
5890 { },
5891 { { 0, 0x1 } },
5892 },
5893 /* BPF_ALU | BPF_DIV | BPF_K */
5894 {
5895 "ALU_DIV_K: 6 / 2 = 3",
5896 .u.insns_int = {
5897 BPF_LD_IMM64(R0, 6),
5898 BPF_ALU32_IMM(BPF_DIV, R0, 2),
5899 BPF_EXIT_INSN(),
5900 },
5901 INTERNAL,
5902 { },
5903 { { 0, 3 } },
5904 },
5905 {
5906 "ALU_DIV_K: 3 / 1 = 3",
5907 .u.insns_int = {
5908 BPF_LD_IMM64(R0, 3),
5909 BPF_ALU32_IMM(BPF_DIV, R0, 1),
5910 BPF_EXIT_INSN(),
5911 },
5912 INTERNAL,
5913 { },
5914 { { 0, 3 } },
5915 },
5916 {
5917 "ALU_DIV_K: 4294967295 / 4294967295 = 1",
5918 .u.insns_int = {
5919 BPF_LD_IMM64(R0, 4294967295U),
5920 BPF_ALU32_IMM(BPF_DIV, R0, 4294967295U),
5921 BPF_EXIT_INSN(),
5922 },
5923 INTERNAL,
5924 { },
5925 { { 0, 1 } },
5926 },
5927 {
5928 "ALU_DIV_K: 0xffffffffffffffff / (-1) = 0x1",
5929 .u.insns_int = {
5930 BPF_LD_IMM64(R2, 0xffffffffffffffffLL),
5931 BPF_LD_IMM64(R3, 0x1UL),
5932 BPF_ALU32_IMM(BPF_DIV, R2, 0xffffffff),
5933 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5934 BPF_MOV32_IMM(R0, 2),
5935 BPF_EXIT_INSN(),
5936 BPF_MOV32_IMM(R0, 1),
5937 BPF_EXIT_INSN(),
5938 },
5939 INTERNAL,
5940 { },
5941 { { 0, 0x1 } },
5942 },
5943 {
5944 "ALU64_DIV_K: 6 / 2 = 3",
5945 .u.insns_int = {
5946 BPF_LD_IMM64(R0, 6),
5947 BPF_ALU64_IMM(BPF_DIV, R0, 2),
5948 BPF_EXIT_INSN(),
5949 },
5950 INTERNAL,
5951 { },
5952 { { 0, 3 } },
5953 },
5954 {
5955 "ALU64_DIV_K: 3 / 1 = 3",
5956 .u.insns_int = {
5957 BPF_LD_IMM64(R0, 3),
5958 BPF_ALU64_IMM(BPF_DIV, R0, 1),
5959 BPF_EXIT_INSN(),
5960 },
5961 INTERNAL,
5962 { },
5963 { { 0, 3 } },
5964 },
5965 {
5966 "ALU64_DIV_K: 2147483647 / 2147483647 = 1",
5967 .u.insns_int = {
5968 BPF_LD_IMM64(R0, 2147483647),
5969 BPF_ALU64_IMM(BPF_DIV, R0, 2147483647),
5970 BPF_EXIT_INSN(),
5971 },
5972 INTERNAL,
5973 { },
5974 { { 0, 1 } },
5975 },
5976 {
5977 "ALU64_DIV_K: 0xffffffffffffffff / (-1) = 0x0000000000000001",
5978 .u.insns_int = {
5979 BPF_LD_IMM64(R2, 0xffffffffffffffffLL),
5980 BPF_LD_IMM64(R3, 0x0000000000000001LL),
5981 BPF_ALU64_IMM(BPF_DIV, R2, 0xffffffff),
5982 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
5983 BPF_MOV32_IMM(R0, 2),
5984 BPF_EXIT_INSN(),
5985 BPF_MOV32_IMM(R0, 1),
5986 BPF_EXIT_INSN(),
5987 },
5988 INTERNAL,
5989 { },
5990 { { 0, 0x1 } },
5991 },
5992 /* BPF_ALU | BPF_MOD | BPF_X */
5993 {
5994 "ALU_MOD_X: 3 % 2 = 1",
5995 .u.insns_int = {
5996 BPF_LD_IMM64(R0, 3),
5997 BPF_ALU32_IMM(BPF_MOV, R1, 2),
5998 BPF_ALU32_REG(BPF_MOD, R0, R1),
5999 BPF_EXIT_INSN(),
6000 },
6001 INTERNAL,
6002 { },
6003 { { 0, 1 } },
6004 },
6005 {
6006 "ALU_MOD_X: 4294967295 % 4294967293 = 2",
6007 .u.insns_int = {
6008 BPF_LD_IMM64(R0, 4294967295U),
6009 BPF_ALU32_IMM(BPF_MOV, R1, 4294967293U),
6010 BPF_ALU32_REG(BPF_MOD, R0, R1),
6011 BPF_EXIT_INSN(),
6012 },
6013 INTERNAL,
6014 { },
6015 { { 0, 2 } },
6016 },
6017 {
6018 "ALU64_MOD_X: 3 % 2 = 1",
6019 .u.insns_int = {
6020 BPF_LD_IMM64(R0, 3),
6021 BPF_ALU32_IMM(BPF_MOV, R1, 2),
6022 BPF_ALU64_REG(BPF_MOD, R0, R1),
6023 BPF_EXIT_INSN(),
6024 },
6025 INTERNAL,
6026 { },
6027 { { 0, 1 } },
6028 },
6029 {
6030 "ALU64_MOD_X: 2147483647 % 2147483645 = 2",
6031 .u.insns_int = {
6032 BPF_LD_IMM64(R0, 2147483647),
6033 BPF_ALU32_IMM(BPF_MOV, R1, 2147483645),
6034 BPF_ALU64_REG(BPF_MOD, R0, R1),
6035 BPF_EXIT_INSN(),
6036 },
6037 INTERNAL,
6038 { },
6039 { { 0, 2 } },
6040 },
6041 /* BPF_ALU | BPF_MOD | BPF_K */
6042 {
6043 "ALU_MOD_K: 3 % 2 = 1",
6044 .u.insns_int = {
6045 BPF_LD_IMM64(R0, 3),
6046 BPF_ALU32_IMM(BPF_MOD, R0, 2),
6047 BPF_EXIT_INSN(),
6048 },
6049 INTERNAL,
6050 { },
6051 { { 0, 1 } },
6052 },
6053 {
6054 "ALU_MOD_K: 3 % 1 = 0",
6055 .u.insns_int = {
6056 BPF_LD_IMM64(R0, 3),
6057 BPF_ALU32_IMM(BPF_MOD, R0, 1),
6058 BPF_EXIT_INSN(),
6059 },
6060 INTERNAL,
6061 { },
6062 { { 0, 0 } },
6063 },
6064 {
6065 "ALU_MOD_K: 4294967295 % 4294967293 = 2",
6066 .u.insns_int = {
6067 BPF_LD_IMM64(R0, 4294967295U),
6068 BPF_ALU32_IMM(BPF_MOD, R0, 4294967293U),
6069 BPF_EXIT_INSN(),
6070 },
6071 INTERNAL,
6072 { },
6073 { { 0, 2 } },
6074 },
6075 {
6076 "ALU64_MOD_K: 3 % 2 = 1",
6077 .u.insns_int = {
6078 BPF_LD_IMM64(R0, 3),
6079 BPF_ALU64_IMM(BPF_MOD, R0, 2),
6080 BPF_EXIT_INSN(),
6081 },
6082 INTERNAL,
6083 { },
6084 { { 0, 1 } },
6085 },
6086 {
6087 "ALU64_MOD_K: 3 % 1 = 0",
6088 .u.insns_int = {
6089 BPF_LD_IMM64(R0, 3),
6090 BPF_ALU64_IMM(BPF_MOD, R0, 1),
6091 BPF_EXIT_INSN(),
6092 },
6093 INTERNAL,
6094 { },
6095 { { 0, 0 } },
6096 },
6097 {
6098 "ALU64_MOD_K: 2147483647 % 2147483645 = 2",
6099 .u.insns_int = {
6100 BPF_LD_IMM64(R0, 2147483647),
6101 BPF_ALU64_IMM(BPF_MOD, R0, 2147483645),
6102 BPF_EXIT_INSN(),
6103 },
6104 INTERNAL,
6105 { },
6106 { { 0, 2 } },
6107 },
6108 /* BPF_ALU | BPF_AND | BPF_X */
6109 {
6110 "ALU_AND_X: 3 & 2 = 2",
6111 .u.insns_int = {
6112 BPF_LD_IMM64(R0, 3),
6113 BPF_ALU32_IMM(BPF_MOV, R1, 2),
6114 BPF_ALU32_REG(BPF_AND, R0, R1),
6115 BPF_EXIT_INSN(),
6116 },
6117 INTERNAL,
6118 { },
6119 { { 0, 2 } },
6120 },
6121 {
6122 "ALU_AND_X: 0xffffffff & 0xffffffff = 0xffffffff",
6123 .u.insns_int = {
6124 BPF_LD_IMM64(R0, 0xffffffff),
6125 BPF_ALU32_IMM(BPF_MOV, R1, 0xffffffff),
6126 BPF_ALU32_REG(BPF_AND, R0, R1),
6127 BPF_EXIT_INSN(),
6128 },
6129 INTERNAL,
6130 { },
6131 { { 0, 0xffffffff } },
6132 },
6133 {
6134 "ALU64_AND_X: 3 & 2 = 2",
6135 .u.insns_int = {
6136 BPF_LD_IMM64(R0, 3),
6137 BPF_ALU32_IMM(BPF_MOV, R1, 2),
6138 BPF_ALU64_REG(BPF_AND, R0, R1),
6139 BPF_EXIT_INSN(),
6140 },
6141 INTERNAL,
6142 { },
6143 { { 0, 2 } },
6144 },
6145 {
6146 "ALU64_AND_X: 0xffffffff & 0xffffffff = 0xffffffff",
6147 .u.insns_int = {
6148 BPF_LD_IMM64(R0, 0xffffffff),
6149 BPF_ALU32_IMM(BPF_MOV, R1, 0xffffffff),
6150 BPF_ALU64_REG(BPF_AND, R0, R1),
6151 BPF_EXIT_INSN(),
6152 },
6153 INTERNAL,
6154 { },
6155 { { 0, 0xffffffff } },
6156 },
6157 /* BPF_ALU | BPF_AND | BPF_K */
6158 {
6159 "ALU_AND_K: 3 & 2 = 2",
6160 .u.insns_int = {
6161 BPF_LD_IMM64(R0, 3),
6162 BPF_ALU32_IMM(BPF_AND, R0, 2),
6163 BPF_EXIT_INSN(),
6164 },
6165 INTERNAL,
6166 { },
6167 { { 0, 2 } },
6168 },
6169 {
6170 "ALU_AND_K: 0xffffffff & 0xffffffff = 0xffffffff",
6171 .u.insns_int = {
6172 BPF_LD_IMM64(R0, 0xffffffff),
6173 BPF_ALU32_IMM(BPF_AND, R0, 0xffffffff),
6174 BPF_EXIT_INSN(),
6175 },
6176 INTERNAL,
6177 { },
6178 { { 0, 0xffffffff } },
6179 },
6180 {
6181 "ALU_AND_K: Small immediate",
6182 .u.insns_int = {
6183 BPF_ALU32_IMM(BPF_MOV, R0, 0x01020304),
6184 BPF_ALU32_IMM(BPF_AND, R0, 15),
6185 BPF_EXIT_INSN(),
6186 },
6187 INTERNAL,
6188 { },
6189 { { 0, 4 } }
6190 },
6191 {
6192 "ALU_AND_K: Large immediate",
6193 .u.insns_int = {
6194 BPF_ALU32_IMM(BPF_MOV, R0, 0xf1f2f3f4),
6195 BPF_ALU32_IMM(BPF_AND, R0, 0xafbfcfdf),
6196 BPF_EXIT_INSN(),
6197 },
6198 INTERNAL,
6199 { },
6200 { { 0, 0xa1b2c3d4 } }
6201 },
6202 {
6203 "ALU_AND_K: Zero extension",
6204 .u.insns_int = {
6205 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6206 BPF_LD_IMM64(R1, 0x0000000080a0c0e0LL),
6207 BPF_ALU32_IMM(BPF_AND, R0, 0xf0f0f0f0),
6208 BPF_JMP_REG(BPF_JEQ, R0, R1, 2),
6209 BPF_MOV32_IMM(R0, 2),
6210 BPF_EXIT_INSN(),
6211 BPF_MOV32_IMM(R0, 1),
6212 BPF_EXIT_INSN(),
6213 },
6214 INTERNAL,
6215 { },
6216 { { 0, 1 } }
6217 },
6218 {
6219 "ALU64_AND_K: 3 & 2 = 2",
6220 .u.insns_int = {
6221 BPF_LD_IMM64(R0, 3),
6222 BPF_ALU64_IMM(BPF_AND, R0, 2),
6223 BPF_EXIT_INSN(),
6224 },
6225 INTERNAL,
6226 { },
6227 { { 0, 2 } },
6228 },
6229 {
6230 "ALU64_AND_K: 0xffffffff & 0xffffffff = 0xffffffff",
6231 .u.insns_int = {
6232 BPF_LD_IMM64(R0, 0xffffffff),
6233 BPF_ALU64_IMM(BPF_AND, R0, 0xffffffff),
6234 BPF_EXIT_INSN(),
6235 },
6236 INTERNAL,
6237 { },
6238 { { 0, 0xffffffff } },
6239 },
6240 {
6241 "ALU64_AND_K: 0x0000ffffffff0000 & 0x0 = 0x0000000000000000",
6242 .u.insns_int = {
6243 BPF_LD_IMM64(R2, 0x0000ffffffff0000LL),
6244 BPF_LD_IMM64(R3, 0x0000000000000000LL),
6245 BPF_ALU64_IMM(BPF_AND, R2, 0x0),
6246 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
6247 BPF_MOV32_IMM(R0, 2),
6248 BPF_EXIT_INSN(),
6249 BPF_MOV32_IMM(R0, 1),
6250 BPF_EXIT_INSN(),
6251 },
6252 INTERNAL,
6253 { },
6254 { { 0, 0x1 } },
6255 },
6256 {
6257 "ALU64_AND_K: 0x0000ffffffff0000 & -1 = 0x0000ffffffff0000",
6258 .u.insns_int = {
6259 BPF_LD_IMM64(R2, 0x0000ffffffff0000LL),
6260 BPF_LD_IMM64(R3, 0x0000ffffffff0000LL),
6261 BPF_ALU64_IMM(BPF_AND, R2, 0xffffffff),
6262 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
6263 BPF_MOV32_IMM(R0, 2),
6264 BPF_EXIT_INSN(),
6265 BPF_MOV32_IMM(R0, 1),
6266 BPF_EXIT_INSN(),
6267 },
6268 INTERNAL,
6269 { },
6270 { { 0, 0x1 } },
6271 },
6272 {
6273 "ALU64_AND_K: 0xffffffffffffffff & -1 = 0xffffffffffffffff",
6274 .u.insns_int = {
6275 BPF_LD_IMM64(R2, 0xffffffffffffffffLL),
6276 BPF_LD_IMM64(R3, 0xffffffffffffffffLL),
6277 BPF_ALU64_IMM(BPF_AND, R2, 0xffffffff),
6278 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
6279 BPF_MOV32_IMM(R0, 2),
6280 BPF_EXIT_INSN(),
6281 BPF_MOV32_IMM(R0, 1),
6282 BPF_EXIT_INSN(),
6283 },
6284 INTERNAL,
6285 { },
6286 { { 0, 0x1 } },
6287 },
6288 {
6289 "ALU64_AND_K: Sign extension 1",
6290 .u.insns_int = {
6291 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6292 BPF_LD_IMM64(R1, 0x00000000090b0d0fLL),
6293 BPF_ALU64_IMM(BPF_AND, R0, 0x0f0f0f0f),
6294 BPF_JMP_REG(BPF_JEQ, R0, R1, 2),
6295 BPF_MOV32_IMM(R0, 2),
6296 BPF_EXIT_INSN(),
6297 BPF_MOV32_IMM(R0, 1),
6298 BPF_EXIT_INSN(),
6299 },
6300 INTERNAL,
6301 { },
6302 { { 0, 1 } }
6303 },
6304 {
6305 "ALU64_AND_K: Sign extension 2",
6306 .u.insns_int = {
6307 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6308 BPF_LD_IMM64(R1, 0x0123456780a0c0e0LL),
6309 BPF_ALU64_IMM(BPF_AND, R0, 0xf0f0f0f0),
6310 BPF_JMP_REG(BPF_JEQ, R0, R1, 2),
6311 BPF_MOV32_IMM(R0, 2),
6312 BPF_EXIT_INSN(),
6313 BPF_MOV32_IMM(R0, 1),
6314 BPF_EXIT_INSN(),
6315 },
6316 INTERNAL,
6317 { },
6318 { { 0, 1 } }
6319 },
6320 /* BPF_ALU | BPF_OR | BPF_X */
6321 {
6322 "ALU_OR_X: 1 | 2 = 3",
6323 .u.insns_int = {
6324 BPF_LD_IMM64(R0, 1),
6325 BPF_ALU32_IMM(BPF_MOV, R1, 2),
6326 BPF_ALU32_REG(BPF_OR, R0, R1),
6327 BPF_EXIT_INSN(),
6328 },
6329 INTERNAL,
6330 { },
6331 { { 0, 3 } },
6332 },
6333 {
6334 "ALU_OR_X: 0x0 | 0xffffffff = 0xffffffff",
6335 .u.insns_int = {
6336 BPF_LD_IMM64(R0, 0),
6337 BPF_ALU32_IMM(BPF_MOV, R1, 0xffffffff),
6338 BPF_ALU32_REG(BPF_OR, R0, R1),
6339 BPF_EXIT_INSN(),
6340 },
6341 INTERNAL,
6342 { },
6343 { { 0, 0xffffffff } },
6344 },
6345 {
6346 "ALU64_OR_X: 1 | 2 = 3",
6347 .u.insns_int = {
6348 BPF_LD_IMM64(R0, 1),
6349 BPF_ALU32_IMM(BPF_MOV, R1, 2),
6350 BPF_ALU64_REG(BPF_OR, R0, R1),
6351 BPF_EXIT_INSN(),
6352 },
6353 INTERNAL,
6354 { },
6355 { { 0, 3 } },
6356 },
6357 {
6358 "ALU64_OR_X: 0 | 0xffffffff = 0xffffffff",
6359 .u.insns_int = {
6360 BPF_LD_IMM64(R0, 0),
6361 BPF_ALU32_IMM(BPF_MOV, R1, 0xffffffff),
6362 BPF_ALU64_REG(BPF_OR, R0, R1),
6363 BPF_EXIT_INSN(),
6364 },
6365 INTERNAL,
6366 { },
6367 { { 0, 0xffffffff } },
6368 },
6369 /* BPF_ALU | BPF_OR | BPF_K */
6370 {
6371 "ALU_OR_K: 1 | 2 = 3",
6372 .u.insns_int = {
6373 BPF_LD_IMM64(R0, 1),
6374 BPF_ALU32_IMM(BPF_OR, R0, 2),
6375 BPF_EXIT_INSN(),
6376 },
6377 INTERNAL,
6378 { },
6379 { { 0, 3 } },
6380 },
6381 {
6382 "ALU_OR_K: 0 & 0xffffffff = 0xffffffff",
6383 .u.insns_int = {
6384 BPF_LD_IMM64(R0, 0),
6385 BPF_ALU32_IMM(BPF_OR, R0, 0xffffffff),
6386 BPF_EXIT_INSN(),
6387 },
6388 INTERNAL,
6389 { },
6390 { { 0, 0xffffffff } },
6391 },
6392 {
6393 "ALU_OR_K: Small immediate",
6394 .u.insns_int = {
6395 BPF_ALU32_IMM(BPF_MOV, R0, 0x01020304),
6396 BPF_ALU32_IMM(BPF_OR, R0, 1),
6397 BPF_EXIT_INSN(),
6398 },
6399 INTERNAL,
6400 { },
6401 { { 0, 0x01020305 } }
6402 },
6403 {
6404 "ALU_OR_K: Large immediate",
6405 .u.insns_int = {
6406 BPF_ALU32_IMM(BPF_MOV, R0, 0x01020304),
6407 BPF_ALU32_IMM(BPF_OR, R0, 0xa0b0c0d0),
6408 BPF_EXIT_INSN(),
6409 },
6410 INTERNAL,
6411 { },
6412 { { 0, 0xa1b2c3d4 } }
6413 },
6414 {
6415 "ALU_OR_K: Zero extension",
6416 .u.insns_int = {
6417 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6418 BPF_LD_IMM64(R1, 0x00000000f9fbfdffLL),
6419 BPF_ALU32_IMM(BPF_OR, R0, 0xf0f0f0f0),
6420 BPF_JMP_REG(BPF_JEQ, R0, R1, 2),
6421 BPF_MOV32_IMM(R0, 2),
6422 BPF_EXIT_INSN(),
6423 BPF_MOV32_IMM(R0, 1),
6424 BPF_EXIT_INSN(),
6425 },
6426 INTERNAL,
6427 { },
6428 { { 0, 1 } }
6429 },
6430 {
6431 "ALU64_OR_K: 1 | 2 = 3",
6432 .u.insns_int = {
6433 BPF_LD_IMM64(R0, 1),
6434 BPF_ALU64_IMM(BPF_OR, R0, 2),
6435 BPF_EXIT_INSN(),
6436 },
6437 INTERNAL,
6438 { },
6439 { { 0, 3 } },
6440 },
6441 {
6442 "ALU64_OR_K: 0 & 0xffffffff = 0xffffffff",
6443 .u.insns_int = {
6444 BPF_LD_IMM64(R0, 0),
6445 BPF_ALU64_IMM(BPF_OR, R0, 0xffffffff),
6446 BPF_EXIT_INSN(),
6447 },
6448 INTERNAL,
6449 { },
6450 { { 0, 0xffffffff } },
6451 },
6452 {
6453 "ALU64_OR_K: 0x0000ffffffff0000 | 0x0 = 0x0000ffffffff0000",
6454 .u.insns_int = {
6455 BPF_LD_IMM64(R2, 0x0000ffffffff0000LL),
6456 BPF_LD_IMM64(R3, 0x0000ffffffff0000LL),
6457 BPF_ALU64_IMM(BPF_OR, R2, 0x0),
6458 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
6459 BPF_MOV32_IMM(R0, 2),
6460 BPF_EXIT_INSN(),
6461 BPF_MOV32_IMM(R0, 1),
6462 BPF_EXIT_INSN(),
6463 },
6464 INTERNAL,
6465 { },
6466 { { 0, 0x1 } },
6467 },
6468 {
6469 "ALU64_OR_K: 0x0000ffffffff0000 | -1 = 0xffffffffffffffff",
6470 .u.insns_int = {
6471 BPF_LD_IMM64(R2, 0x0000ffffffff0000LL),
6472 BPF_LD_IMM64(R3, 0xffffffffffffffffLL),
6473 BPF_ALU64_IMM(BPF_OR, R2, 0xffffffff),
6474 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
6475 BPF_MOV32_IMM(R0, 2),
6476 BPF_EXIT_INSN(),
6477 BPF_MOV32_IMM(R0, 1),
6478 BPF_EXIT_INSN(),
6479 },
6480 INTERNAL,
6481 { },
6482 { { 0, 0x1 } },
6483 },
6484 {
6485 "ALU64_OR_K: 0x000000000000000 | -1 = 0xffffffffffffffff",
6486 .u.insns_int = {
6487 BPF_LD_IMM64(R2, 0x0000000000000000LL),
6488 BPF_LD_IMM64(R3, 0xffffffffffffffffLL),
6489 BPF_ALU64_IMM(BPF_OR, R2, 0xffffffff),
6490 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
6491 BPF_MOV32_IMM(R0, 2),
6492 BPF_EXIT_INSN(),
6493 BPF_MOV32_IMM(R0, 1),
6494 BPF_EXIT_INSN(),
6495 },
6496 INTERNAL,
6497 { },
6498 { { 0, 0x1 } },
6499 },
6500 {
6501 "ALU64_OR_K: Sign extension 1",
6502 .u.insns_int = {
6503 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6504 BPF_LD_IMM64(R1, 0x012345678fafcfefLL),
6505 BPF_ALU64_IMM(BPF_OR, R0, 0x0f0f0f0f),
6506 BPF_JMP_REG(BPF_JEQ, R0, R1, 2),
6507 BPF_MOV32_IMM(R0, 2),
6508 BPF_EXIT_INSN(),
6509 BPF_MOV32_IMM(R0, 1),
6510 BPF_EXIT_INSN(),
6511 },
6512 INTERNAL,
6513 { },
6514 { { 0, 1 } }
6515 },
6516 {
6517 "ALU64_OR_K: Sign extension 2",
6518 .u.insns_int = {
6519 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6520 BPF_LD_IMM64(R1, 0xfffffffff9fbfdffLL),
6521 BPF_ALU64_IMM(BPF_OR, R0, 0xf0f0f0f0),
6522 BPF_JMP_REG(BPF_JEQ, R0, R1, 2),
6523 BPF_MOV32_IMM(R0, 2),
6524 BPF_EXIT_INSN(),
6525 BPF_MOV32_IMM(R0, 1),
6526 BPF_EXIT_INSN(),
6527 },
6528 INTERNAL,
6529 { },
6530 { { 0, 1 } }
6531 },
6532 /* BPF_ALU | BPF_XOR | BPF_X */
6533 {
6534 "ALU_XOR_X: 5 ^ 6 = 3",
6535 .u.insns_int = {
6536 BPF_LD_IMM64(R0, 5),
6537 BPF_ALU32_IMM(BPF_MOV, R1, 6),
6538 BPF_ALU32_REG(BPF_XOR, R0, R1),
6539 BPF_EXIT_INSN(),
6540 },
6541 INTERNAL,
6542 { },
6543 { { 0, 3 } },
6544 },
6545 {
6546 "ALU_XOR_X: 0x1 ^ 0xffffffff = 0xfffffffe",
6547 .u.insns_int = {
6548 BPF_LD_IMM64(R0, 1),
6549 BPF_ALU32_IMM(BPF_MOV, R1, 0xffffffff),
6550 BPF_ALU32_REG(BPF_XOR, R0, R1),
6551 BPF_EXIT_INSN(),
6552 },
6553 INTERNAL,
6554 { },
6555 { { 0, 0xfffffffe } },
6556 },
6557 {
6558 "ALU64_XOR_X: 5 ^ 6 = 3",
6559 .u.insns_int = {
6560 BPF_LD_IMM64(R0, 5),
6561 BPF_ALU32_IMM(BPF_MOV, R1, 6),
6562 BPF_ALU64_REG(BPF_XOR, R0, R1),
6563 BPF_EXIT_INSN(),
6564 },
6565 INTERNAL,
6566 { },
6567 { { 0, 3 } },
6568 },
6569 {
6570 "ALU64_XOR_X: 1 ^ 0xffffffff = 0xfffffffe",
6571 .u.insns_int = {
6572 BPF_LD_IMM64(R0, 1),
6573 BPF_ALU32_IMM(BPF_MOV, R1, 0xffffffff),
6574 BPF_ALU64_REG(BPF_XOR, R0, R1),
6575 BPF_EXIT_INSN(),
6576 },
6577 INTERNAL,
6578 { },
6579 { { 0, 0xfffffffe } },
6580 },
6581 /* BPF_ALU | BPF_XOR | BPF_K */
6582 {
6583 "ALU_XOR_K: 5 ^ 6 = 3",
6584 .u.insns_int = {
6585 BPF_LD_IMM64(R0, 5),
6586 BPF_ALU32_IMM(BPF_XOR, R0, 6),
6587 BPF_EXIT_INSN(),
6588 },
6589 INTERNAL,
6590 { },
6591 { { 0, 3 } },
6592 },
6593 {
6594 "ALU_XOR_K: 1 ^ 0xffffffff = 0xfffffffe",
6595 .u.insns_int = {
6596 BPF_LD_IMM64(R0, 1),
6597 BPF_ALU32_IMM(BPF_XOR, R0, 0xffffffff),
6598 BPF_EXIT_INSN(),
6599 },
6600 INTERNAL,
6601 { },
6602 { { 0, 0xfffffffe } },
6603 },
6604 {
6605 "ALU_XOR_K: Small immediate",
6606 .u.insns_int = {
6607 BPF_ALU32_IMM(BPF_MOV, R0, 0x01020304),
6608 BPF_ALU32_IMM(BPF_XOR, R0, 15),
6609 BPF_EXIT_INSN(),
6610 },
6611 INTERNAL,
6612 { },
6613 { { 0, 0x0102030b } }
6614 },
6615 {
6616 "ALU_XOR_K: Large immediate",
6617 .u.insns_int = {
6618 BPF_ALU32_IMM(BPF_MOV, R0, 0xf1f2f3f4),
6619 BPF_ALU32_IMM(BPF_XOR, R0, 0xafbfcfdf),
6620 BPF_EXIT_INSN(),
6621 },
6622 INTERNAL,
6623 { },
6624 { { 0, 0x5e4d3c2b } }
6625 },
6626 {
6627 "ALU_XOR_K: Zero extension",
6628 .u.insns_int = {
6629 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6630 BPF_LD_IMM64(R1, 0x00000000795b3d1fLL),
6631 BPF_ALU32_IMM(BPF_XOR, R0, 0xf0f0f0f0),
6632 BPF_JMP_REG(BPF_JEQ, R0, R1, 2),
6633 BPF_MOV32_IMM(R0, 2),
6634 BPF_EXIT_INSN(),
6635 BPF_MOV32_IMM(R0, 1),
6636 BPF_EXIT_INSN(),
6637 },
6638 INTERNAL,
6639 { },
6640 { { 0, 1 } }
6641 },
6642 {
6643 "ALU64_XOR_K: 5 ^ 6 = 3",
6644 .u.insns_int = {
6645 BPF_LD_IMM64(R0, 5),
6646 BPF_ALU64_IMM(BPF_XOR, R0, 6),
6647 BPF_EXIT_INSN(),
6648 },
6649 INTERNAL,
6650 { },
6651 { { 0, 3 } },
6652 },
6653 {
6654 "ALU64_XOR_K: 1 ^ 0xffffffff = 0xfffffffe",
6655 .u.insns_int = {
6656 BPF_LD_IMM64(R0, 1),
6657 BPF_ALU64_IMM(BPF_XOR, R0, 0xffffffff),
6658 BPF_EXIT_INSN(),
6659 },
6660 INTERNAL,
6661 { },
6662 { { 0, 0xfffffffe } },
6663 },
6664 {
6665 "ALU64_XOR_K: 0x0000ffffffff0000 ^ 0x0 = 0x0000ffffffff0000",
6666 .u.insns_int = {
6667 BPF_LD_IMM64(R2, 0x0000ffffffff0000LL),
6668 BPF_LD_IMM64(R3, 0x0000ffffffff0000LL),
6669 BPF_ALU64_IMM(BPF_XOR, R2, 0x0),
6670 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
6671 BPF_MOV32_IMM(R0, 2),
6672 BPF_EXIT_INSN(),
6673 BPF_MOV32_IMM(R0, 1),
6674 BPF_EXIT_INSN(),
6675 },
6676 INTERNAL,
6677 { },
6678 { { 0, 0x1 } },
6679 },
6680 {
6681 "ALU64_XOR_K: 0x0000ffffffff0000 ^ -1 = 0xffff00000000ffff",
6682 .u.insns_int = {
6683 BPF_LD_IMM64(R2, 0x0000ffffffff0000LL),
6684 BPF_LD_IMM64(R3, 0xffff00000000ffffLL),
6685 BPF_ALU64_IMM(BPF_XOR, R2, 0xffffffff),
6686 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
6687 BPF_MOV32_IMM(R0, 2),
6688 BPF_EXIT_INSN(),
6689 BPF_MOV32_IMM(R0, 1),
6690 BPF_EXIT_INSN(),
6691 },
6692 INTERNAL,
6693 { },
6694 { { 0, 0x1 } },
6695 },
6696 {
6697 "ALU64_XOR_K: 0x000000000000000 ^ -1 = 0xffffffffffffffff",
6698 .u.insns_int = {
6699 BPF_LD_IMM64(R2, 0x0000000000000000LL),
6700 BPF_LD_IMM64(R3, 0xffffffffffffffffLL),
6701 BPF_ALU64_IMM(BPF_XOR, R2, 0xffffffff),
6702 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
6703 BPF_MOV32_IMM(R0, 2),
6704 BPF_EXIT_INSN(),
6705 BPF_MOV32_IMM(R0, 1),
6706 BPF_EXIT_INSN(),
6707 },
6708 INTERNAL,
6709 { },
6710 { { 0, 0x1 } },
6711 },
6712 {
6713 "ALU64_XOR_K: Sign extension 1",
6714 .u.insns_int = {
6715 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6716 BPF_LD_IMM64(R1, 0x0123456786a4c2e0LL),
6717 BPF_ALU64_IMM(BPF_XOR, R0, 0x0f0f0f0f),
6718 BPF_JMP_REG(BPF_JEQ, R0, R1, 2),
6719 BPF_MOV32_IMM(R0, 2),
6720 BPF_EXIT_INSN(),
6721 BPF_MOV32_IMM(R0, 1),
6722 BPF_EXIT_INSN(),
6723 },
6724 INTERNAL,
6725 { },
6726 { { 0, 1 } }
6727 },
6728 {
6729 "ALU64_XOR_K: Sign extension 2",
6730 .u.insns_int = {
6731 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6732 BPF_LD_IMM64(R1, 0xfedcba98795b3d1fLL),
6733 BPF_ALU64_IMM(BPF_XOR, R0, 0xf0f0f0f0),
6734 BPF_JMP_REG(BPF_JEQ, R0, R1, 2),
6735 BPF_MOV32_IMM(R0, 2),
6736 BPF_EXIT_INSN(),
6737 BPF_MOV32_IMM(R0, 1),
6738 BPF_EXIT_INSN(),
6739 },
6740 INTERNAL,
6741 { },
6742 { { 0, 1 } }
6743 },
6744 /* BPF_ALU | BPF_LSH | BPF_X */
6745 {
6746 "ALU_LSH_X: 1 << 1 = 2",
6747 .u.insns_int = {
6748 BPF_LD_IMM64(R0, 1),
6749 BPF_ALU32_IMM(BPF_MOV, R1, 1),
6750 BPF_ALU32_REG(BPF_LSH, R0, R1),
6751 BPF_EXIT_INSN(),
6752 },
6753 INTERNAL,
6754 { },
6755 { { 0, 2 } },
6756 },
6757 {
6758 "ALU_LSH_X: 1 << 31 = 0x80000000",
6759 .u.insns_int = {
6760 BPF_LD_IMM64(R0, 1),
6761 BPF_ALU32_IMM(BPF_MOV, R1, 31),
6762 BPF_ALU32_REG(BPF_LSH, R0, R1),
6763 BPF_EXIT_INSN(),
6764 },
6765 INTERNAL,
6766 { },
6767 { { 0, 0x80000000 } },
6768 },
6769 {
6770 "ALU_LSH_X: 0x12345678 << 12 = 0x45678000",
6771 .u.insns_int = {
6772 BPF_ALU32_IMM(BPF_MOV, R0, 0x12345678),
6773 BPF_ALU32_IMM(BPF_MOV, R1, 12),
6774 BPF_ALU32_REG(BPF_LSH, R0, R1),
6775 BPF_EXIT_INSN(),
6776 },
6777 INTERNAL,
6778 { },
6779 { { 0, 0x45678000 } }
6780 },
6781 {
6782 "ALU64_LSH_X: 1 << 1 = 2",
6783 .u.insns_int = {
6784 BPF_LD_IMM64(R0, 1),
6785 BPF_ALU32_IMM(BPF_MOV, R1, 1),
6786 BPF_ALU64_REG(BPF_LSH, R0, R1),
6787 BPF_EXIT_INSN(),
6788 },
6789 INTERNAL,
6790 { },
6791 { { 0, 2 } },
6792 },
6793 {
6794 "ALU64_LSH_X: 1 << 31 = 0x80000000",
6795 .u.insns_int = {
6796 BPF_LD_IMM64(R0, 1),
6797 BPF_ALU32_IMM(BPF_MOV, R1, 31),
6798 BPF_ALU64_REG(BPF_LSH, R0, R1),
6799 BPF_EXIT_INSN(),
6800 },
6801 INTERNAL,
6802 { },
6803 { { 0, 0x80000000 } },
6804 },
6805 {
6806 "ALU64_LSH_X: Shift < 32, low word",
6807 .u.insns_int = {
6808 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6809 BPF_ALU32_IMM(BPF_MOV, R1, 12),
6810 BPF_ALU64_REG(BPF_LSH, R0, R1),
6811 BPF_EXIT_INSN(),
6812 },
6813 INTERNAL,
6814 { },
6815 { { 0, 0xbcdef000 } }
6816 },
6817 {
6818 "ALU64_LSH_X: Shift < 32, high word",
6819 .u.insns_int = {
6820 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6821 BPF_ALU32_IMM(BPF_MOV, R1, 12),
6822 BPF_ALU64_REG(BPF_LSH, R0, R1),
6823 BPF_ALU64_IMM(BPF_RSH, R0, 32),
6824 BPF_EXIT_INSN(),
6825 },
6826 INTERNAL,
6827 { },
6828 { { 0, 0x3456789a } }
6829 },
6830 {
6831 "ALU64_LSH_X: Shift > 32, low word",
6832 .u.insns_int = {
6833 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6834 BPF_ALU32_IMM(BPF_MOV, R1, 36),
6835 BPF_ALU64_REG(BPF_LSH, R0, R1),
6836 BPF_EXIT_INSN(),
6837 },
6838 INTERNAL,
6839 { },
6840 { { 0, 0 } }
6841 },
6842 {
6843 "ALU64_LSH_X: Shift > 32, high word",
6844 .u.insns_int = {
6845 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6846 BPF_ALU32_IMM(BPF_MOV, R1, 36),
6847 BPF_ALU64_REG(BPF_LSH, R0, R1),
6848 BPF_ALU64_IMM(BPF_RSH, R0, 32),
6849 BPF_EXIT_INSN(),
6850 },
6851 INTERNAL,
6852 { },
6853 { { 0, 0x9abcdef0 } }
6854 },
6855 {
6856 "ALU64_LSH_X: Shift == 32, low word",
6857 .u.insns_int = {
6858 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6859 BPF_ALU32_IMM(BPF_MOV, R1, 32),
6860 BPF_ALU64_REG(BPF_LSH, R0, R1),
6861 BPF_EXIT_INSN(),
6862 },
6863 INTERNAL,
6864 { },
6865 { { 0, 0 } }
6866 },
6867 {
6868 "ALU64_LSH_X: Shift == 32, high word",
6869 .u.insns_int = {
6870 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6871 BPF_ALU32_IMM(BPF_MOV, R1, 32),
6872 BPF_ALU64_REG(BPF_LSH, R0, R1),
6873 BPF_ALU64_IMM(BPF_RSH, R0, 32),
6874 BPF_EXIT_INSN(),
6875 },
6876 INTERNAL,
6877 { },
6878 { { 0, 0x89abcdef } }
6879 },
6880 {
6881 "ALU64_LSH_X: Zero shift, low word",
6882 .u.insns_int = {
6883 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6884 BPF_ALU32_IMM(BPF_MOV, R1, 0),
6885 BPF_ALU64_REG(BPF_LSH, R0, R1),
6886 BPF_EXIT_INSN(),
6887 },
6888 INTERNAL,
6889 { },
6890 { { 0, 0x89abcdef } }
6891 },
6892 {
6893 "ALU64_LSH_X: Zero shift, high word",
6894 .u.insns_int = {
6895 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6896 BPF_ALU32_IMM(BPF_MOV, R1, 0),
6897 BPF_ALU64_REG(BPF_LSH, R0, R1),
6898 BPF_ALU64_IMM(BPF_RSH, R0, 32),
6899 BPF_EXIT_INSN(),
6900 },
6901 INTERNAL,
6902 { },
6903 { { 0, 0x01234567 } }
6904 },
6905 /* BPF_ALU | BPF_LSH | BPF_K */
6906 {
6907 "ALU_LSH_K: 1 << 1 = 2",
6908 .u.insns_int = {
6909 BPF_LD_IMM64(R0, 1),
6910 BPF_ALU32_IMM(BPF_LSH, R0, 1),
6911 BPF_EXIT_INSN(),
6912 },
6913 INTERNAL,
6914 { },
6915 { { 0, 2 } },
6916 },
6917 {
6918 "ALU_LSH_K: 1 << 31 = 0x80000000",
6919 .u.insns_int = {
6920 BPF_LD_IMM64(R0, 1),
6921 BPF_ALU32_IMM(BPF_LSH, R0, 31),
6922 BPF_EXIT_INSN(),
6923 },
6924 INTERNAL,
6925 { },
6926 { { 0, 0x80000000 } },
6927 },
6928 {
6929 "ALU_LSH_K: 0x12345678 << 12 = 0x45678000",
6930 .u.insns_int = {
6931 BPF_ALU32_IMM(BPF_MOV, R0, 0x12345678),
6932 BPF_ALU32_IMM(BPF_LSH, R0, 12),
6933 BPF_EXIT_INSN(),
6934 },
6935 INTERNAL,
6936 { },
6937 { { 0, 0x45678000 } }
6938 },
6939 {
6940 "ALU_LSH_K: 0x12345678 << 0 = 0x12345678",
6941 .u.insns_int = {
6942 BPF_ALU32_IMM(BPF_MOV, R0, 0x12345678),
6943 BPF_ALU32_IMM(BPF_LSH, R0, 0),
6944 BPF_EXIT_INSN(),
6945 },
6946 INTERNAL,
6947 { },
6948 { { 0, 0x12345678 } }
6949 },
6950 {
6951 "ALU64_LSH_K: 1 << 1 = 2",
6952 .u.insns_int = {
6953 BPF_LD_IMM64(R0, 1),
6954 BPF_ALU64_IMM(BPF_LSH, R0, 1),
6955 BPF_EXIT_INSN(),
6956 },
6957 INTERNAL,
6958 { },
6959 { { 0, 2 } },
6960 },
6961 {
6962 "ALU64_LSH_K: 1 << 31 = 0x80000000",
6963 .u.insns_int = {
6964 BPF_LD_IMM64(R0, 1),
6965 BPF_ALU64_IMM(BPF_LSH, R0, 31),
6966 BPF_EXIT_INSN(),
6967 },
6968 INTERNAL,
6969 { },
6970 { { 0, 0x80000000 } },
6971 },
6972 {
6973 "ALU64_LSH_K: Shift < 32, low word",
6974 .u.insns_int = {
6975 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6976 BPF_ALU64_IMM(BPF_LSH, R0, 12),
6977 BPF_EXIT_INSN(),
6978 },
6979 INTERNAL,
6980 { },
6981 { { 0, 0xbcdef000 } }
6982 },
6983 {
6984 "ALU64_LSH_K: Shift < 32, high word",
6985 .u.insns_int = {
6986 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6987 BPF_ALU64_IMM(BPF_LSH, R0, 12),
6988 BPF_ALU64_IMM(BPF_RSH, R0, 32),
6989 BPF_EXIT_INSN(),
6990 },
6991 INTERNAL,
6992 { },
6993 { { 0, 0x3456789a } }
6994 },
6995 {
6996 "ALU64_LSH_K: Shift > 32, low word",
6997 .u.insns_int = {
6998 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
6999 BPF_ALU64_IMM(BPF_LSH, R0, 36),
7000 BPF_EXIT_INSN(),
7001 },
7002 INTERNAL,
7003 { },
7004 { { 0, 0 } }
7005 },
7006 {
7007 "ALU64_LSH_K: Shift > 32, high word",
7008 .u.insns_int = {
7009 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7010 BPF_ALU64_IMM(BPF_LSH, R0, 36),
7011 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7012 BPF_EXIT_INSN(),
7013 },
7014 INTERNAL,
7015 { },
7016 { { 0, 0x9abcdef0 } }
7017 },
7018 {
7019 "ALU64_LSH_K: Shift == 32, low word",
7020 .u.insns_int = {
7021 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7022 BPF_ALU64_IMM(BPF_LSH, R0, 32),
7023 BPF_EXIT_INSN(),
7024 },
7025 INTERNAL,
7026 { },
7027 { { 0, 0 } }
7028 },
7029 {
7030 "ALU64_LSH_K: Shift == 32, high word",
7031 .u.insns_int = {
7032 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7033 BPF_ALU64_IMM(BPF_LSH, R0, 32),
7034 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7035 BPF_EXIT_INSN(),
7036 },
7037 INTERNAL,
7038 { },
7039 { { 0, 0x89abcdef } }
7040 },
7041 {
7042 "ALU64_LSH_K: Zero shift",
7043 .u.insns_int = {
7044 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7045 BPF_ALU64_IMM(BPF_LSH, R0, 0),
7046 BPF_EXIT_INSN(),
7047 },
7048 INTERNAL,
7049 { },
7050 { { 0, 0x89abcdef } }
7051 },
7052 /* BPF_ALU | BPF_RSH | BPF_X */
7053 {
7054 "ALU_RSH_X: 2 >> 1 = 1",
7055 .u.insns_int = {
7056 BPF_LD_IMM64(R0, 2),
7057 BPF_ALU32_IMM(BPF_MOV, R1, 1),
7058 BPF_ALU32_REG(BPF_RSH, R0, R1),
7059 BPF_EXIT_INSN(),
7060 },
7061 INTERNAL,
7062 { },
7063 { { 0, 1 } },
7064 },
7065 {
7066 "ALU_RSH_X: 0x80000000 >> 31 = 1",
7067 .u.insns_int = {
7068 BPF_LD_IMM64(R0, 0x80000000),
7069 BPF_ALU32_IMM(BPF_MOV, R1, 31),
7070 BPF_ALU32_REG(BPF_RSH, R0, R1),
7071 BPF_EXIT_INSN(),
7072 },
7073 INTERNAL,
7074 { },
7075 { { 0, 1 } },
7076 },
7077 {
7078 "ALU_RSH_X: 0x12345678 >> 20 = 0x123",
7079 .u.insns_int = {
7080 BPF_ALU32_IMM(BPF_MOV, R0, 0x12345678),
7081 BPF_ALU32_IMM(BPF_MOV, R1, 20),
7082 BPF_ALU32_REG(BPF_RSH, R0, R1),
7083 BPF_EXIT_INSN(),
7084 },
7085 INTERNAL,
7086 { },
7087 { { 0, 0x123 } }
7088 },
7089 {
7090 "ALU64_RSH_X: 2 >> 1 = 1",
7091 .u.insns_int = {
7092 BPF_LD_IMM64(R0, 2),
7093 BPF_ALU32_IMM(BPF_MOV, R1, 1),
7094 BPF_ALU64_REG(BPF_RSH, R0, R1),
7095 BPF_EXIT_INSN(),
7096 },
7097 INTERNAL,
7098 { },
7099 { { 0, 1 } },
7100 },
7101 {
7102 "ALU64_RSH_X: 0x80000000 >> 31 = 1",
7103 .u.insns_int = {
7104 BPF_LD_IMM64(R0, 0x80000000),
7105 BPF_ALU32_IMM(BPF_MOV, R1, 31),
7106 BPF_ALU64_REG(BPF_RSH, R0, R1),
7107 BPF_EXIT_INSN(),
7108 },
7109 INTERNAL,
7110 { },
7111 { { 0, 1 } },
7112 },
7113 {
7114 "ALU64_RSH_X: Shift < 32, low word",
7115 .u.insns_int = {
7116 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7117 BPF_ALU32_IMM(BPF_MOV, R1, 12),
7118 BPF_ALU64_REG(BPF_RSH, R0, R1),
7119 BPF_EXIT_INSN(),
7120 },
7121 INTERNAL,
7122 { },
7123 { { 0, 0x56789abc } }
7124 },
7125 {
7126 "ALU64_RSH_X: Shift < 32, high word",
7127 .u.insns_int = {
7128 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7129 BPF_ALU32_IMM(BPF_MOV, R1, 12),
7130 BPF_ALU64_REG(BPF_RSH, R0, R1),
7131 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7132 BPF_EXIT_INSN(),
7133 },
7134 INTERNAL,
7135 { },
7136 { { 0, 0x00081234 } }
7137 },
7138 {
7139 "ALU64_RSH_X: Shift > 32, low word",
7140 .u.insns_int = {
7141 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7142 BPF_ALU32_IMM(BPF_MOV, R1, 36),
7143 BPF_ALU64_REG(BPF_RSH, R0, R1),
7144 BPF_EXIT_INSN(),
7145 },
7146 INTERNAL,
7147 { },
7148 { { 0, 0x08123456 } }
7149 },
7150 {
7151 "ALU64_RSH_X: Shift > 32, high word",
7152 .u.insns_int = {
7153 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7154 BPF_ALU32_IMM(BPF_MOV, R1, 36),
7155 BPF_ALU64_REG(BPF_RSH, R0, R1),
7156 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7157 BPF_EXIT_INSN(),
7158 },
7159 INTERNAL,
7160 { },
7161 { { 0, 0 } }
7162 },
7163 {
7164 "ALU64_RSH_X: Shift == 32, low word",
7165 .u.insns_int = {
7166 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7167 BPF_ALU32_IMM(BPF_MOV, R1, 32),
7168 BPF_ALU64_REG(BPF_RSH, R0, R1),
7169 BPF_EXIT_INSN(),
7170 },
7171 INTERNAL,
7172 { },
7173 { { 0, 0x81234567 } }
7174 },
7175 {
7176 "ALU64_RSH_X: Shift == 32, high word",
7177 .u.insns_int = {
7178 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7179 BPF_ALU32_IMM(BPF_MOV, R1, 32),
7180 BPF_ALU64_REG(BPF_RSH, R0, R1),
7181 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7182 BPF_EXIT_INSN(),
7183 },
7184 INTERNAL,
7185 { },
7186 { { 0, 0 } }
7187 },
7188 {
7189 "ALU64_RSH_X: Zero shift, low word",
7190 .u.insns_int = {
7191 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7192 BPF_ALU32_IMM(BPF_MOV, R1, 0),
7193 BPF_ALU64_REG(BPF_RSH, R0, R1),
7194 BPF_EXIT_INSN(),
7195 },
7196 INTERNAL,
7197 { },
7198 { { 0, 0x89abcdef } }
7199 },
7200 {
7201 "ALU64_RSH_X: Zero shift, high word",
7202 .u.insns_int = {
7203 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7204 BPF_ALU32_IMM(BPF_MOV, R1, 0),
7205 BPF_ALU64_REG(BPF_RSH, R0, R1),
7206 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7207 BPF_EXIT_INSN(),
7208 },
7209 INTERNAL,
7210 { },
7211 { { 0, 0x81234567 } }
7212 },
7213 /* BPF_ALU | BPF_RSH | BPF_K */
7214 {
7215 "ALU_RSH_K: 2 >> 1 = 1",
7216 .u.insns_int = {
7217 BPF_LD_IMM64(R0, 2),
7218 BPF_ALU32_IMM(BPF_RSH, R0, 1),
7219 BPF_EXIT_INSN(),
7220 },
7221 INTERNAL,
7222 { },
7223 { { 0, 1 } },
7224 },
7225 {
7226 "ALU_RSH_K: 0x80000000 >> 31 = 1",
7227 .u.insns_int = {
7228 BPF_LD_IMM64(R0, 0x80000000),
7229 BPF_ALU32_IMM(BPF_RSH, R0, 31),
7230 BPF_EXIT_INSN(),
7231 },
7232 INTERNAL,
7233 { },
7234 { { 0, 1 } },
7235 },
7236 {
7237 "ALU_RSH_K: 0x12345678 >> 20 = 0x123",
7238 .u.insns_int = {
7239 BPF_ALU32_IMM(BPF_MOV, R0, 0x12345678),
7240 BPF_ALU32_IMM(BPF_RSH, R0, 20),
7241 BPF_EXIT_INSN(),
7242 },
7243 INTERNAL,
7244 { },
7245 { { 0, 0x123 } }
7246 },
7247 {
7248 "ALU_RSH_K: 0x12345678 >> 0 = 0x12345678",
7249 .u.insns_int = {
7250 BPF_ALU32_IMM(BPF_MOV, R0, 0x12345678),
7251 BPF_ALU32_IMM(BPF_RSH, R0, 0),
7252 BPF_EXIT_INSN(),
7253 },
7254 INTERNAL,
7255 { },
7256 { { 0, 0x12345678 } }
7257 },
7258 {
7259 "ALU64_RSH_K: 2 >> 1 = 1",
7260 .u.insns_int = {
7261 BPF_LD_IMM64(R0, 2),
7262 BPF_ALU64_IMM(BPF_RSH, R0, 1),
7263 BPF_EXIT_INSN(),
7264 },
7265 INTERNAL,
7266 { },
7267 { { 0, 1 } },
7268 },
7269 {
7270 "ALU64_RSH_K: 0x80000000 >> 31 = 1",
7271 .u.insns_int = {
7272 BPF_LD_IMM64(R0, 0x80000000),
7273 BPF_ALU64_IMM(BPF_RSH, R0, 31),
7274 BPF_EXIT_INSN(),
7275 },
7276 INTERNAL,
7277 { },
7278 { { 0, 1 } },
7279 },
7280 {
7281 "ALU64_RSH_K: Shift < 32, low word",
7282 .u.insns_int = {
7283 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7284 BPF_ALU64_IMM(BPF_RSH, R0, 12),
7285 BPF_EXIT_INSN(),
7286 },
7287 INTERNAL,
7288 { },
7289 { { 0, 0x56789abc } }
7290 },
7291 {
7292 "ALU64_RSH_K: Shift < 32, high word",
7293 .u.insns_int = {
7294 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7295 BPF_ALU64_IMM(BPF_RSH, R0, 12),
7296 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7297 BPF_EXIT_INSN(),
7298 },
7299 INTERNAL,
7300 { },
7301 { { 0, 0x00081234 } }
7302 },
7303 {
7304 "ALU64_RSH_K: Shift > 32, low word",
7305 .u.insns_int = {
7306 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7307 BPF_ALU64_IMM(BPF_RSH, R0, 36),
7308 BPF_EXIT_INSN(),
7309 },
7310 INTERNAL,
7311 { },
7312 { { 0, 0x08123456 } }
7313 },
7314 {
7315 "ALU64_RSH_K: Shift > 32, high word",
7316 .u.insns_int = {
7317 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7318 BPF_ALU64_IMM(BPF_RSH, R0, 36),
7319 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7320 BPF_EXIT_INSN(),
7321 },
7322 INTERNAL,
7323 { },
7324 { { 0, 0 } }
7325 },
7326 {
7327 "ALU64_RSH_K: Shift == 32, low word",
7328 .u.insns_int = {
7329 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7330 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7331 BPF_EXIT_INSN(),
7332 },
7333 INTERNAL,
7334 { },
7335 { { 0, 0x81234567 } }
7336 },
7337 {
7338 "ALU64_RSH_K: Shift == 32, high word",
7339 .u.insns_int = {
7340 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7341 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7342 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7343 BPF_EXIT_INSN(),
7344 },
7345 INTERNAL,
7346 { },
7347 { { 0, 0 } }
7348 },
7349 {
7350 "ALU64_RSH_K: Zero shift",
7351 .u.insns_int = {
7352 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7353 BPF_ALU64_IMM(BPF_RSH, R0, 0),
7354 BPF_EXIT_INSN(),
7355 },
7356 INTERNAL,
7357 { },
7358 { { 0, 0x89abcdef } }
7359 },
7360 /* BPF_ALU | BPF_ARSH | BPF_X */
7361 {
7362 "ALU32_ARSH_X: -1234 >> 7 = -10",
7363 .u.insns_int = {
7364 BPF_ALU32_IMM(BPF_MOV, R0, -1234),
7365 BPF_ALU32_IMM(BPF_MOV, R1, 7),
7366 BPF_ALU32_REG(BPF_ARSH, R0, R1),
7367 BPF_EXIT_INSN(),
7368 },
7369 INTERNAL,
7370 { },
7371 { { 0, -10 } }
7372 },
7373 {
7374 "ALU64_ARSH_X: 0xff00ff0000000000 >> 40 = 0xffffffffffff00ff",
7375 .u.insns_int = {
7376 BPF_LD_IMM64(R0, 0xff00ff0000000000LL),
7377 BPF_ALU32_IMM(BPF_MOV, R1, 40),
7378 BPF_ALU64_REG(BPF_ARSH, R0, R1),
7379 BPF_EXIT_INSN(),
7380 },
7381 INTERNAL,
7382 { },
7383 { { 0, 0xffff00ff } },
7384 },
7385 {
7386 "ALU64_ARSH_X: Shift < 32, low word",
7387 .u.insns_int = {
7388 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7389 BPF_ALU32_IMM(BPF_MOV, R1, 12),
7390 BPF_ALU64_REG(BPF_ARSH, R0, R1),
7391 BPF_EXIT_INSN(),
7392 },
7393 INTERNAL,
7394 { },
7395 { { 0, 0x56789abc } }
7396 },
7397 {
7398 "ALU64_ARSH_X: Shift < 32, high word",
7399 .u.insns_int = {
7400 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7401 BPF_ALU32_IMM(BPF_MOV, R1, 12),
7402 BPF_ALU64_REG(BPF_ARSH, R0, R1),
7403 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7404 BPF_EXIT_INSN(),
7405 },
7406 INTERNAL,
7407 { },
7408 { { 0, 0xfff81234 } }
7409 },
7410 {
7411 "ALU64_ARSH_X: Shift > 32, low word",
7412 .u.insns_int = {
7413 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7414 BPF_ALU32_IMM(BPF_MOV, R1, 36),
7415 BPF_ALU64_REG(BPF_ARSH, R0, R1),
7416 BPF_EXIT_INSN(),
7417 },
7418 INTERNAL,
7419 { },
7420 { { 0, 0xf8123456 } }
7421 },
7422 {
7423 "ALU64_ARSH_X: Shift > 32, high word",
7424 .u.insns_int = {
7425 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7426 BPF_ALU32_IMM(BPF_MOV, R1, 36),
7427 BPF_ALU64_REG(BPF_ARSH, R0, R1),
7428 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7429 BPF_EXIT_INSN(),
7430 },
7431 INTERNAL,
7432 { },
7433 { { 0, -1 } }
7434 },
7435 {
7436 "ALU64_ARSH_X: Shift == 32, low word",
7437 .u.insns_int = {
7438 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7439 BPF_ALU32_IMM(BPF_MOV, R1, 32),
7440 BPF_ALU64_REG(BPF_ARSH, R0, R1),
7441 BPF_EXIT_INSN(),
7442 },
7443 INTERNAL,
7444 { },
7445 { { 0, 0x81234567 } }
7446 },
7447 {
7448 "ALU64_ARSH_X: Shift == 32, high word",
7449 .u.insns_int = {
7450 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7451 BPF_ALU32_IMM(BPF_MOV, R1, 32),
7452 BPF_ALU64_REG(BPF_ARSH, R0, R1),
7453 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7454 BPF_EXIT_INSN(),
7455 },
7456 INTERNAL,
7457 { },
7458 { { 0, -1 } }
7459 },
7460 {
7461 "ALU64_ARSH_X: Zero shift, low word",
7462 .u.insns_int = {
7463 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7464 BPF_ALU32_IMM(BPF_MOV, R1, 0),
7465 BPF_ALU64_REG(BPF_ARSH, R0, R1),
7466 BPF_EXIT_INSN(),
7467 },
7468 INTERNAL,
7469 { },
7470 { { 0, 0x89abcdef } }
7471 },
7472 {
7473 "ALU64_ARSH_X: Zero shift, high word",
7474 .u.insns_int = {
7475 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7476 BPF_ALU32_IMM(BPF_MOV, R1, 0),
7477 BPF_ALU64_REG(BPF_ARSH, R0, R1),
7478 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7479 BPF_EXIT_INSN(),
7480 },
7481 INTERNAL,
7482 { },
7483 { { 0, 0x81234567 } }
7484 },
7485 /* BPF_ALU | BPF_ARSH | BPF_K */
7486 {
7487 "ALU32_ARSH_K: -1234 >> 7 = -10",
7488 .u.insns_int = {
7489 BPF_ALU32_IMM(BPF_MOV, R0, -1234),
7490 BPF_ALU32_IMM(BPF_ARSH, R0, 7),
7491 BPF_EXIT_INSN(),
7492 },
7493 INTERNAL,
7494 { },
7495 { { 0, -10 } }
7496 },
7497 {
7498 "ALU32_ARSH_K: -1234 >> 0 = -1234",
7499 .u.insns_int = {
7500 BPF_ALU32_IMM(BPF_MOV, R0, -1234),
7501 BPF_ALU32_IMM(BPF_ARSH, R0, 0),
7502 BPF_EXIT_INSN(),
7503 },
7504 INTERNAL,
7505 { },
7506 { { 0, -1234 } }
7507 },
7508 {
7509 "ALU64_ARSH_K: 0xff00ff0000000000 >> 40 = 0xffffffffffff00ff",
7510 .u.insns_int = {
7511 BPF_LD_IMM64(R0, 0xff00ff0000000000LL),
7512 BPF_ALU64_IMM(BPF_ARSH, R0, 40),
7513 BPF_EXIT_INSN(),
7514 },
7515 INTERNAL,
7516 { },
7517 { { 0, 0xffff00ff } },
7518 },
7519 {
7520 "ALU64_ARSH_K: Shift < 32, low word",
7521 .u.insns_int = {
7522 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7523 BPF_ALU64_IMM(BPF_RSH, R0, 12),
7524 BPF_EXIT_INSN(),
7525 },
7526 INTERNAL,
7527 { },
7528 { { 0, 0x56789abc } }
7529 },
7530 {
7531 "ALU64_ARSH_K: Shift < 32, high word",
7532 .u.insns_int = {
7533 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7534 BPF_ALU64_IMM(BPF_ARSH, R0, 12),
7535 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7536 BPF_EXIT_INSN(),
7537 },
7538 INTERNAL,
7539 { },
7540 { { 0, 0xfff81234 } }
7541 },
7542 {
7543 "ALU64_ARSH_K: Shift > 32, low word",
7544 .u.insns_int = {
7545 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7546 BPF_ALU64_IMM(BPF_ARSH, R0, 36),
7547 BPF_EXIT_INSN(),
7548 },
7549 INTERNAL,
7550 { },
7551 { { 0, 0xf8123456 } }
7552 },
7553 {
7554 "ALU64_ARSH_K: Shift > 32, high word",
7555 .u.insns_int = {
7556 BPF_LD_IMM64(R0, 0xf123456789abcdefLL),
7557 BPF_ALU64_IMM(BPF_ARSH, R0, 36),
7558 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7559 BPF_EXIT_INSN(),
7560 },
7561 INTERNAL,
7562 { },
7563 { { 0, -1 } }
7564 },
7565 {
7566 "ALU64_ARSH_K: Shift == 32, low word",
7567 .u.insns_int = {
7568 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7569 BPF_ALU64_IMM(BPF_ARSH, R0, 32),
7570 BPF_EXIT_INSN(),
7571 },
7572 INTERNAL,
7573 { },
7574 { { 0, 0x81234567 } }
7575 },
7576 {
7577 "ALU64_ARSH_K: Shift == 32, high word",
7578 .u.insns_int = {
7579 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7580 BPF_ALU64_IMM(BPF_ARSH, R0, 32),
7581 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7582 BPF_EXIT_INSN(),
7583 },
7584 INTERNAL,
7585 { },
7586 { { 0, -1 } }
7587 },
7588 {
7589 "ALU64_ARSH_K: Zero shift",
7590 .u.insns_int = {
7591 BPF_LD_IMM64(R0, 0x8123456789abcdefLL),
7592 BPF_ALU64_IMM(BPF_ARSH, R0, 0),
7593 BPF_EXIT_INSN(),
7594 },
7595 INTERNAL,
7596 { },
7597 { { 0, 0x89abcdef } }
7598 },
7599 /* BPF_ALU | BPF_NEG */
7600 {
7601 "ALU_NEG: -(3) = -3",
7602 .u.insns_int = {
7603 BPF_ALU32_IMM(BPF_MOV, R0, 3),
7604 BPF_ALU32_IMM(BPF_NEG, R0, 0),
7605 BPF_EXIT_INSN(),
7606 },
7607 INTERNAL,
7608 { },
7609 { { 0, -3 } },
7610 },
7611 {
7612 "ALU_NEG: -(-3) = 3",
7613 .u.insns_int = {
7614 BPF_ALU32_IMM(BPF_MOV, R0, -3),
7615 BPF_ALU32_IMM(BPF_NEG, R0, 0),
7616 BPF_EXIT_INSN(),
7617 },
7618 INTERNAL,
7619 { },
7620 { { 0, 3 } },
7621 },
7622 {
7623 "ALU64_NEG: -(3) = -3",
7624 .u.insns_int = {
7625 BPF_LD_IMM64(R0, 3),
7626 BPF_ALU64_IMM(BPF_NEG, R0, 0),
7627 BPF_EXIT_INSN(),
7628 },
7629 INTERNAL,
7630 { },
7631 { { 0, -3 } },
7632 },
7633 {
7634 "ALU64_NEG: -(-3) = 3",
7635 .u.insns_int = {
7636 BPF_LD_IMM64(R0, -3),
7637 BPF_ALU64_IMM(BPF_NEG, R0, 0),
7638 BPF_EXIT_INSN(),
7639 },
7640 INTERNAL,
7641 { },
7642 { { 0, 3 } },
7643 },
7644 /* BPF_ALU | BPF_END | BPF_FROM_BE */
7645 {
7646 "ALU_END_FROM_BE 16: 0x0123456789abcdef -> 0xcdef",
7647 .u.insns_int = {
7648 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7649 BPF_ENDIAN(BPF_FROM_BE, R0, 16),
7650 BPF_EXIT_INSN(),
7651 },
7652 INTERNAL,
7653 { },
7654 { { 0, cpu_to_be16(0xcdef) } },
7655 },
7656 {
7657 "ALU_END_FROM_BE 32: 0x0123456789abcdef -> 0x89abcdef",
7658 .u.insns_int = {
7659 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7660 BPF_ENDIAN(BPF_FROM_BE, R0, 32),
7661 BPF_ALU64_REG(BPF_MOV, R1, R0),
7662 BPF_ALU64_IMM(BPF_RSH, R1, 32),
7663 BPF_ALU32_REG(BPF_ADD, R0, R1), /* R1 = 0 */
7664 BPF_EXIT_INSN(),
7665 },
7666 INTERNAL,
7667 { },
7668 { { 0, cpu_to_be32(0x89abcdef) } },
7669 },
7670 {
7671 "ALU_END_FROM_BE 64: 0x0123456789abcdef -> 0x89abcdef",
7672 .u.insns_int = {
7673 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7674 BPF_ENDIAN(BPF_FROM_BE, R0, 64),
7675 BPF_EXIT_INSN(),
7676 },
7677 INTERNAL,
7678 { },
7679 { { 0, (u32) cpu_to_be64(0x0123456789abcdefLL) } },
7680 },
7681 {
7682 "ALU_END_FROM_BE 64: 0x0123456789abcdef >> 32 -> 0x01234567",
7683 .u.insns_int = {
7684 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7685 BPF_ENDIAN(BPF_FROM_BE, R0, 64),
7686 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7687 BPF_EXIT_INSN(),
7688 },
7689 INTERNAL,
7690 { },
7691 { { 0, (u32) (cpu_to_be64(0x0123456789abcdefLL) >> 32) } },
7692 },
7693 /* BPF_ALU | BPF_END | BPF_FROM_BE, reversed */
7694 {
7695 "ALU_END_FROM_BE 16: 0xfedcba9876543210 -> 0x3210",
7696 .u.insns_int = {
7697 BPF_LD_IMM64(R0, 0xfedcba9876543210ULL),
7698 BPF_ENDIAN(BPF_FROM_BE, R0, 16),
7699 BPF_EXIT_INSN(),
7700 },
7701 INTERNAL,
7702 { },
7703 { { 0, cpu_to_be16(0x3210) } },
7704 },
7705 {
7706 "ALU_END_FROM_BE 32: 0xfedcba9876543210 -> 0x76543210",
7707 .u.insns_int = {
7708 BPF_LD_IMM64(R0, 0xfedcba9876543210ULL),
7709 BPF_ENDIAN(BPF_FROM_BE, R0, 32),
7710 BPF_ALU64_REG(BPF_MOV, R1, R0),
7711 BPF_ALU64_IMM(BPF_RSH, R1, 32),
7712 BPF_ALU32_REG(BPF_ADD, R0, R1), /* R1 = 0 */
7713 BPF_EXIT_INSN(),
7714 },
7715 INTERNAL,
7716 { },
7717 { { 0, cpu_to_be32(0x76543210) } },
7718 },
7719 {
7720 "ALU_END_FROM_BE 64: 0xfedcba9876543210 -> 0x76543210",
7721 .u.insns_int = {
7722 BPF_LD_IMM64(R0, 0xfedcba9876543210ULL),
7723 BPF_ENDIAN(BPF_FROM_BE, R0, 64),
7724 BPF_EXIT_INSN(),
7725 },
7726 INTERNAL,
7727 { },
7728 { { 0, (u32) cpu_to_be64(0xfedcba9876543210ULL) } },
7729 },
7730 {
7731 "ALU_END_FROM_BE 64: 0xfedcba9876543210 >> 32 -> 0xfedcba98",
7732 .u.insns_int = {
7733 BPF_LD_IMM64(R0, 0xfedcba9876543210ULL),
7734 BPF_ENDIAN(BPF_FROM_BE, R0, 64),
7735 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7736 BPF_EXIT_INSN(),
7737 },
7738 INTERNAL,
7739 { },
7740 { { 0, (u32) (cpu_to_be64(0xfedcba9876543210ULL) >> 32) } },
7741 },
7742 /* BPF_ALU | BPF_END | BPF_FROM_LE */
7743 {
7744 "ALU_END_FROM_LE 16: 0x0123456789abcdef -> 0xefcd",
7745 .u.insns_int = {
7746 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7747 BPF_ENDIAN(BPF_FROM_LE, R0, 16),
7748 BPF_EXIT_INSN(),
7749 },
7750 INTERNAL,
7751 { },
7752 { { 0, cpu_to_le16(0xcdef) } },
7753 },
7754 {
7755 "ALU_END_FROM_LE 32: 0x0123456789abcdef -> 0xefcdab89",
7756 .u.insns_int = {
7757 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7758 BPF_ENDIAN(BPF_FROM_LE, R0, 32),
7759 BPF_ALU64_REG(BPF_MOV, R1, R0),
7760 BPF_ALU64_IMM(BPF_RSH, R1, 32),
7761 BPF_ALU32_REG(BPF_ADD, R0, R1), /* R1 = 0 */
7762 BPF_EXIT_INSN(),
7763 },
7764 INTERNAL,
7765 { },
7766 { { 0, cpu_to_le32(0x89abcdef) } },
7767 },
7768 {
7769 "ALU_END_FROM_LE 64: 0x0123456789abcdef -> 0x67452301",
7770 .u.insns_int = {
7771 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7772 BPF_ENDIAN(BPF_FROM_LE, R0, 64),
7773 BPF_EXIT_INSN(),
7774 },
7775 INTERNAL,
7776 { },
7777 { { 0, (u32) cpu_to_le64(0x0123456789abcdefLL) } },
7778 },
7779 {
7780 "ALU_END_FROM_LE 64: 0x0123456789abcdef >> 32 -> 0xefcdab89",
7781 .u.insns_int = {
7782 BPF_LD_IMM64(R0, 0x0123456789abcdefLL),
7783 BPF_ENDIAN(BPF_FROM_LE, R0, 64),
7784 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7785 BPF_EXIT_INSN(),
7786 },
7787 INTERNAL,
7788 { },
7789 { { 0, (u32) (cpu_to_le64(0x0123456789abcdefLL) >> 32) } },
7790 },
7791 /* BPF_ALU | BPF_END | BPF_FROM_LE, reversed */
7792 {
7793 "ALU_END_FROM_LE 16: 0xfedcba9876543210 -> 0x1032",
7794 .u.insns_int = {
7795 BPF_LD_IMM64(R0, 0xfedcba9876543210ULL),
7796 BPF_ENDIAN(BPF_FROM_LE, R0, 16),
7797 BPF_EXIT_INSN(),
7798 },
7799 INTERNAL,
7800 { },
7801 { { 0, cpu_to_le16(0x3210) } },
7802 },
7803 {
7804 "ALU_END_FROM_LE 32: 0xfedcba9876543210 -> 0x10325476",
7805 .u.insns_int = {
7806 BPF_LD_IMM64(R0, 0xfedcba9876543210ULL),
7807 BPF_ENDIAN(BPF_FROM_LE, R0, 32),
7808 BPF_ALU64_REG(BPF_MOV, R1, R0),
7809 BPF_ALU64_IMM(BPF_RSH, R1, 32),
7810 BPF_ALU32_REG(BPF_ADD, R0, R1), /* R1 = 0 */
7811 BPF_EXIT_INSN(),
7812 },
7813 INTERNAL,
7814 { },
7815 { { 0, cpu_to_le32(0x76543210) } },
7816 },
7817 {
7818 "ALU_END_FROM_LE 64: 0xfedcba9876543210 -> 0x10325476",
7819 .u.insns_int = {
7820 BPF_LD_IMM64(R0, 0xfedcba9876543210ULL),
7821 BPF_ENDIAN(BPF_FROM_LE, R0, 64),
7822 BPF_EXIT_INSN(),
7823 },
7824 INTERNAL,
7825 { },
7826 { { 0, (u32) cpu_to_le64(0xfedcba9876543210ULL) } },
7827 },
7828 {
7829 "ALU_END_FROM_LE 64: 0xfedcba9876543210 >> 32 -> 0x98badcfe",
7830 .u.insns_int = {
7831 BPF_LD_IMM64(R0, 0xfedcba9876543210ULL),
7832 BPF_ENDIAN(BPF_FROM_LE, R0, 64),
7833 BPF_ALU64_IMM(BPF_RSH, R0, 32),
7834 BPF_EXIT_INSN(),
7835 },
7836 INTERNAL,
7837 { },
7838 { { 0, (u32) (cpu_to_le64(0xfedcba9876543210ULL) >> 32) } },
7839 },
7840 /* BPF_LDX_MEM B/H/W/DW */
7841 {
7842 "BPF_LDX_MEM | BPF_B, base",
7843 .u.insns_int = {
7844 BPF_LD_IMM64(R1, 0x0102030405060708ULL),
7845 BPF_LD_IMM64(R2, 0x0000000000000008ULL),
7846 BPF_STX_MEM(BPF_DW, R10, R1, -8),
7847 #ifdef __BIG_ENDIAN
7848 BPF_LDX_MEM(BPF_B, R0, R10, -1),
7849 #else
7850 BPF_LDX_MEM(BPF_B, R0, R10, -8),
7851 #endif
7852 BPF_JMP_REG(BPF_JNE, R0, R2, 1),
7853 BPF_ALU64_IMM(BPF_MOV, R0, 0),
7854 BPF_EXIT_INSN(),
7855 },
7856 INTERNAL,
7857 { },
7858 { { 0, 0 } },
7859 .stack_depth = 8,
7860 },
7861 {
7862 "BPF_LDX_MEM | BPF_B, MSB set",
7863 .u.insns_int = {
7864 BPF_LD_IMM64(R1, 0x8182838485868788ULL),
7865 BPF_LD_IMM64(R2, 0x0000000000000088ULL),
7866 BPF_STX_MEM(BPF_DW, R10, R1, -8),
7867 #ifdef __BIG_ENDIAN
7868 BPF_LDX_MEM(BPF_B, R0, R10, -1),
7869 #else
7870 BPF_LDX_MEM(BPF_B, R0, R10, -8),
7871 #endif
7872 BPF_JMP_REG(BPF_JNE, R0, R2, 1),
7873 BPF_ALU64_IMM(BPF_MOV, R0, 0),
7874 BPF_EXIT_INSN(),
7875 },
7876 INTERNAL,
7877 { },
7878 { { 0, 0 } },
7879 .stack_depth = 8,
7880 },
7881 {
7882 "BPF_LDX_MEM | BPF_B, negative offset",
7883 .u.insns_int = {
7884 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
7885 BPF_LD_IMM64(R3, 0x0000000000000088ULL),
7886 BPF_ALU64_IMM(BPF_ADD, R1, 512),
7887 BPF_STX_MEM(BPF_B, R1, R2, -256),
7888 BPF_LDX_MEM(BPF_B, R0, R1, -256),
7889 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
7890 BPF_ALU64_IMM(BPF_MOV, R0, 0),
7891 BPF_EXIT_INSN(),
7892 },
7893 INTERNAL | FLAG_LARGE_MEM,
7894 { },
7895 { { 512, 0 } },
7896 .stack_depth = 0,
7897 },
7898 {
7899 "BPF_LDX_MEM | BPF_B, small positive offset",
7900 .u.insns_int = {
7901 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
7902 BPF_LD_IMM64(R3, 0x0000000000000088ULL),
7903 BPF_STX_MEM(BPF_B, R1, R2, 256),
7904 BPF_LDX_MEM(BPF_B, R0, R1, 256),
7905 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
7906 BPF_ALU64_IMM(BPF_MOV, R0, 0),
7907 BPF_EXIT_INSN(),
7908 },
7909 INTERNAL | FLAG_LARGE_MEM,
7910 { },
7911 { { 512, 0 } },
7912 .stack_depth = 0,
7913 },
7914 {
7915 "BPF_LDX_MEM | BPF_B, large positive offset",
7916 .u.insns_int = {
7917 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
7918 BPF_LD_IMM64(R3, 0x0000000000000088ULL),
7919 BPF_STX_MEM(BPF_B, R1, R2, 4096),
7920 BPF_LDX_MEM(BPF_B, R0, R1, 4096),
7921 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
7922 BPF_ALU64_IMM(BPF_MOV, R0, 0),
7923 BPF_EXIT_INSN(),
7924 },
7925 INTERNAL | FLAG_LARGE_MEM,
7926 { },
7927 { { 4096 + 16, 0 } },
7928 .stack_depth = 0,
7929 },
7930 {
7931 "BPF_LDX_MEM | BPF_H, base",
7932 .u.insns_int = {
7933 BPF_LD_IMM64(R1, 0x0102030405060708ULL),
7934 BPF_LD_IMM64(R2, 0x0000000000000708ULL),
7935 BPF_STX_MEM(BPF_DW, R10, R1, -8),
7936 #ifdef __BIG_ENDIAN
7937 BPF_LDX_MEM(BPF_H, R0, R10, -2),
7938 #else
7939 BPF_LDX_MEM(BPF_H, R0, R10, -8),
7940 #endif
7941 BPF_JMP_REG(BPF_JNE, R0, R2, 1),
7942 BPF_ALU64_IMM(BPF_MOV, R0, 0),
7943 BPF_EXIT_INSN(),
7944 },
7945 INTERNAL,
7946 { },
7947 { { 0, 0 } },
7948 .stack_depth = 8,
7949 },
7950 {
7951 "BPF_LDX_MEM | BPF_H, MSB set",
7952 .u.insns_int = {
7953 BPF_LD_IMM64(R1, 0x8182838485868788ULL),
7954 BPF_LD_IMM64(R2, 0x0000000000008788ULL),
7955 BPF_STX_MEM(BPF_DW, R10, R1, -8),
7956 #ifdef __BIG_ENDIAN
7957 BPF_LDX_MEM(BPF_H, R0, R10, -2),
7958 #else
7959 BPF_LDX_MEM(BPF_H, R0, R10, -8),
7960 #endif
7961 BPF_JMP_REG(BPF_JNE, R0, R2, 1),
7962 BPF_ALU64_IMM(BPF_MOV, R0, 0),
7963 BPF_EXIT_INSN(),
7964 },
7965 INTERNAL,
7966 { },
7967 { { 0, 0 } },
7968 .stack_depth = 8,
7969 },
7970 {
7971 "BPF_LDX_MEM | BPF_H, negative offset",
7972 .u.insns_int = {
7973 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
7974 BPF_LD_IMM64(R3, 0x0000000000008788ULL),
7975 BPF_ALU64_IMM(BPF_ADD, R1, 512),
7976 BPF_STX_MEM(BPF_H, R1, R2, -256),
7977 BPF_LDX_MEM(BPF_H, R0, R1, -256),
7978 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
7979 BPF_ALU64_IMM(BPF_MOV, R0, 0),
7980 BPF_EXIT_INSN(),
7981 },
7982 INTERNAL | FLAG_LARGE_MEM,
7983 { },
7984 { { 512, 0 } },
7985 .stack_depth = 0,
7986 },
7987 {
7988 "BPF_LDX_MEM | BPF_H, small positive offset",
7989 .u.insns_int = {
7990 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
7991 BPF_LD_IMM64(R3, 0x0000000000008788ULL),
7992 BPF_STX_MEM(BPF_H, R1, R2, 256),
7993 BPF_LDX_MEM(BPF_H, R0, R1, 256),
7994 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
7995 BPF_ALU64_IMM(BPF_MOV, R0, 0),
7996 BPF_EXIT_INSN(),
7997 },
7998 INTERNAL | FLAG_LARGE_MEM,
7999 { },
8000 { { 512, 0 } },
8001 .stack_depth = 0,
8002 },
8003 {
8004 "BPF_LDX_MEM | BPF_H, large positive offset",
8005 .u.insns_int = {
8006 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8007 BPF_LD_IMM64(R3, 0x0000000000008788ULL),
8008 BPF_STX_MEM(BPF_H, R1, R2, 8192),
8009 BPF_LDX_MEM(BPF_H, R0, R1, 8192),
8010 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
8011 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8012 BPF_EXIT_INSN(),
8013 },
8014 INTERNAL | FLAG_LARGE_MEM,
8015 { },
8016 { { 8192 + 16, 0 } },
8017 .stack_depth = 0,
8018 },
8019 {
8020 "BPF_LDX_MEM | BPF_H, misaligned offset",
8021 .u.insns_int = {
8022 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8023 BPF_LD_IMM64(R3, 0x0000000000008788ULL),
8024 BPF_STX_MEM(BPF_H, R1, R2, 13),
8025 BPF_LDX_MEM(BPF_H, R0, R1, 13),
8026 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
8027 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8028 BPF_EXIT_INSN(),
8029 },
8030 INTERNAL | FLAG_LARGE_MEM,
8031 { },
8032 { { 32, 0 } },
8033 .stack_depth = 0,
8034 },
8035 {
8036 "BPF_LDX_MEM | BPF_W, base",
8037 .u.insns_int = {
8038 BPF_LD_IMM64(R1, 0x0102030405060708ULL),
8039 BPF_LD_IMM64(R2, 0x0000000005060708ULL),
8040 BPF_STX_MEM(BPF_DW, R10, R1, -8),
8041 #ifdef __BIG_ENDIAN
8042 BPF_LDX_MEM(BPF_W, R0, R10, -4),
8043 #else
8044 BPF_LDX_MEM(BPF_W, R0, R10, -8),
8045 #endif
8046 BPF_JMP_REG(BPF_JNE, R0, R2, 1),
8047 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8048 BPF_EXIT_INSN(),
8049 },
8050 INTERNAL,
8051 { },
8052 { { 0, 0 } },
8053 .stack_depth = 8,
8054 },
8055 {
8056 "BPF_LDX_MEM | BPF_W, MSB set",
8057 .u.insns_int = {
8058 BPF_LD_IMM64(R1, 0x8182838485868788ULL),
8059 BPF_LD_IMM64(R2, 0x0000000085868788ULL),
8060 BPF_STX_MEM(BPF_DW, R10, R1, -8),
8061 #ifdef __BIG_ENDIAN
8062 BPF_LDX_MEM(BPF_W, R0, R10, -4),
8063 #else
8064 BPF_LDX_MEM(BPF_W, R0, R10, -8),
8065 #endif
8066 BPF_JMP_REG(BPF_JNE, R0, R2, 1),
8067 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8068 BPF_EXIT_INSN(),
8069 },
8070 INTERNAL,
8071 { },
8072 { { 0, 0 } },
8073 .stack_depth = 8,
8074 },
8075 {
8076 "BPF_LDX_MEM | BPF_W, negative offset",
8077 .u.insns_int = {
8078 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8079 BPF_LD_IMM64(R3, 0x0000000085868788ULL),
8080 BPF_ALU64_IMM(BPF_ADD, R1, 512),
8081 BPF_STX_MEM(BPF_W, R1, R2, -256),
8082 BPF_LDX_MEM(BPF_W, R0, R1, -256),
8083 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
8084 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8085 BPF_EXIT_INSN(),
8086 },
8087 INTERNAL | FLAG_LARGE_MEM,
8088 { },
8089 { { 512, 0 } },
8090 .stack_depth = 0,
8091 },
8092 {
8093 "BPF_LDX_MEM | BPF_W, small positive offset",
8094 .u.insns_int = {
8095 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8096 BPF_LD_IMM64(R3, 0x0000000085868788ULL),
8097 BPF_STX_MEM(BPF_W, R1, R2, 256),
8098 BPF_LDX_MEM(BPF_W, R0, R1, 256),
8099 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
8100 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8101 BPF_EXIT_INSN(),
8102 },
8103 INTERNAL | FLAG_LARGE_MEM,
8104 { },
8105 { { 512, 0 } },
8106 .stack_depth = 0,
8107 },
8108 {
8109 "BPF_LDX_MEM | BPF_W, large positive offset",
8110 .u.insns_int = {
8111 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8112 BPF_LD_IMM64(R3, 0x0000000085868788ULL),
8113 BPF_STX_MEM(BPF_W, R1, R2, 16384),
8114 BPF_LDX_MEM(BPF_W, R0, R1, 16384),
8115 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
8116 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8117 BPF_EXIT_INSN(),
8118 },
8119 INTERNAL | FLAG_LARGE_MEM,
8120 { },
8121 { { 16384 + 16, 0 } },
8122 .stack_depth = 0,
8123 },
8124 {
8125 "BPF_LDX_MEM | BPF_W, misaligned positive offset",
8126 .u.insns_int = {
8127 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8128 BPF_LD_IMM64(R3, 0x0000000085868788ULL),
8129 BPF_STX_MEM(BPF_W, R1, R2, 13),
8130 BPF_LDX_MEM(BPF_W, R0, R1, 13),
8131 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
8132 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8133 BPF_EXIT_INSN(),
8134 },
8135 INTERNAL | FLAG_LARGE_MEM,
8136 { },
8137 { { 32, 0 } },
8138 .stack_depth = 0,
8139 },
8140 {
8141 "BPF_LDX_MEM | BPF_DW, base",
8142 .u.insns_int = {
8143 BPF_LD_IMM64(R1, 0x0102030405060708ULL),
8144 BPF_STX_MEM(BPF_DW, R10, R1, -8),
8145 BPF_LDX_MEM(BPF_DW, R0, R10, -8),
8146 BPF_JMP_REG(BPF_JNE, R0, R1, 1),
8147 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8148 BPF_EXIT_INSN(),
8149 },
8150 INTERNAL,
8151 { },
8152 { { 0, 0 } },
8153 .stack_depth = 8,
8154 },
8155 {
8156 "BPF_LDX_MEM | BPF_DW, MSB set",
8157 .u.insns_int = {
8158 BPF_LD_IMM64(R1, 0x8182838485868788ULL),
8159 BPF_STX_MEM(BPF_DW, R10, R1, -8),
8160 BPF_LDX_MEM(BPF_DW, R0, R10, -8),
8161 BPF_JMP_REG(BPF_JNE, R0, R1, 1),
8162 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8163 BPF_EXIT_INSN(),
8164 },
8165 INTERNAL,
8166 { },
8167 { { 0, 0 } },
8168 .stack_depth = 8,
8169 },
8170 {
8171 "BPF_LDX_MEM | BPF_DW, negative offset",
8172 .u.insns_int = {
8173 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8174 BPF_ALU64_IMM(BPF_ADD, R1, 512),
8175 BPF_STX_MEM(BPF_DW, R1, R2, -256),
8176 BPF_LDX_MEM(BPF_DW, R0, R1, -256),
8177 BPF_JMP_REG(BPF_JNE, R0, R2, 1),
8178 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8179 BPF_EXIT_INSN(),
8180 },
8181 INTERNAL | FLAG_LARGE_MEM,
8182 { },
8183 { { 512, 0 } },
8184 .stack_depth = 0,
8185 },
8186 {
8187 "BPF_LDX_MEM | BPF_DW, small positive offset",
8188 .u.insns_int = {
8189 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8190 BPF_STX_MEM(BPF_DW, R1, R2, 256),
8191 BPF_LDX_MEM(BPF_DW, R0, R1, 256),
8192 BPF_JMP_REG(BPF_JNE, R0, R2, 1),
8193 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8194 BPF_EXIT_INSN(),
8195 },
8196 INTERNAL | FLAG_LARGE_MEM,
8197 { },
8198 { { 512, 0 } },
8199 .stack_depth = 8,
8200 },
8201 {
8202 "BPF_LDX_MEM | BPF_DW, large positive offset",
8203 .u.insns_int = {
8204 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8205 BPF_STX_MEM(BPF_DW, R1, R2, 32768),
> 8206 BPF_LDX_MEM(BPF_DW, R0, R1, 32768),
8207 BPF_JMP_REG(BPF_JNE, R0, R2, 1),
8208 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8209 BPF_EXIT_INSN(),
8210 },
8211 INTERNAL | FLAG_LARGE_MEM,
8212 { },
8213 { { 32768 + 16, 0 } },
8214 .stack_depth = 0,
8215 },
8216 {
8217 "BPF_LDX_MEM | BPF_DW, misaligned positive offset",
8218 .u.insns_int = {
8219 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8220 BPF_STX_MEM(BPF_DW, R1, R2, 13),
8221 BPF_LDX_MEM(BPF_DW, R0, R1, 13),
8222 BPF_JMP_REG(BPF_JNE, R0, R2, 1),
8223 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8224 BPF_EXIT_INSN(),
8225 },
8226 INTERNAL | FLAG_LARGE_MEM,
8227 { },
8228 { { 32, 0 } },
8229 .stack_depth = 0,
8230 },
8231 /* BPF_STX_MEM B/H/W/DW */
8232 {
8233 "BPF_STX_MEM | BPF_B",
8234 .u.insns_int = {
8235 BPF_LD_IMM64(R1, 0x8090a0b0c0d0e0f0ULL),
8236 BPF_LD_IMM64(R2, 0x0102030405060708ULL),
8237 BPF_LD_IMM64(R3, 0x8090a0b0c0d0e008ULL),
8238 BPF_STX_MEM(BPF_DW, R10, R1, -8),
8239 #ifdef __BIG_ENDIAN
8240 BPF_STX_MEM(BPF_B, R10, R2, -1),
8241 #else
8242 BPF_STX_MEM(BPF_B, R10, R2, -8),
8243 #endif
8244 BPF_LDX_MEM(BPF_DW, R0, R10, -8),
8245 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
8246 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8247 BPF_EXIT_INSN(),
8248 },
8249 INTERNAL,
8250 { },
8251 { { 0, 0 } },
8252 .stack_depth = 8,
8253 },
8254 {
8255 "BPF_STX_MEM | BPF_B, MSB set",
8256 .u.insns_int = {
8257 BPF_LD_IMM64(R1, 0x8090a0b0c0d0e0f0ULL),
8258 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8259 BPF_LD_IMM64(R3, 0x8090a0b0c0d0e088ULL),
8260 BPF_STX_MEM(BPF_DW, R10, R1, -8),
8261 #ifdef __BIG_ENDIAN
8262 BPF_STX_MEM(BPF_B, R10, R2, -1),
8263 #else
8264 BPF_STX_MEM(BPF_B, R10, R2, -8),
8265 #endif
8266 BPF_LDX_MEM(BPF_DW, R0, R10, -8),
8267 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
8268 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8269 BPF_EXIT_INSN(),
8270 },
8271 INTERNAL,
8272 { },
8273 { { 0, 0 } },
8274 .stack_depth = 8,
8275 },
8276 {
8277 "BPF_STX_MEM | BPF_H",
8278 .u.insns_int = {
8279 BPF_LD_IMM64(R1, 0x8090a0b0c0d0e0f0ULL),
8280 BPF_LD_IMM64(R2, 0x0102030405060708ULL),
8281 BPF_LD_IMM64(R3, 0x8090a0b0c0d00708ULL),
8282 BPF_STX_MEM(BPF_DW, R10, R1, -8),
8283 #ifdef __BIG_ENDIAN
8284 BPF_STX_MEM(BPF_H, R10, R2, -2),
8285 #else
8286 BPF_STX_MEM(BPF_H, R10, R2, -8),
8287 #endif
8288 BPF_LDX_MEM(BPF_DW, R0, R10, -8),
8289 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
8290 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8291 BPF_EXIT_INSN(),
8292 },
8293 INTERNAL,
8294 { },
8295 { { 0, 0 } },
8296 .stack_depth = 8,
8297 },
8298 {
8299 "BPF_STX_MEM | BPF_H, MSB set",
8300 .u.insns_int = {
8301 BPF_LD_IMM64(R1, 0x8090a0b0c0d0e0f0ULL),
8302 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8303 BPF_LD_IMM64(R3, 0x8090a0b0c0d08788ULL),
8304 BPF_STX_MEM(BPF_DW, R10, R1, -8),
8305 #ifdef __BIG_ENDIAN
8306 BPF_STX_MEM(BPF_H, R10, R2, -2),
8307 #else
8308 BPF_STX_MEM(BPF_H, R10, R2, -8),
8309 #endif
8310 BPF_LDX_MEM(BPF_DW, R0, R10, -8),
8311 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
8312 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8313 BPF_EXIT_INSN(),
8314 },
8315 INTERNAL,
8316 { },
8317 { { 0, 0 } },
8318 .stack_depth = 8,
8319 },
8320 {
8321 "BPF_STX_MEM | BPF_W",
8322 .u.insns_int = {
8323 BPF_LD_IMM64(R1, 0x8090a0b0c0d0e0f0ULL),
8324 BPF_LD_IMM64(R2, 0x0102030405060708ULL),
8325 BPF_LD_IMM64(R3, 0x8090a0b005060708ULL),
8326 BPF_STX_MEM(BPF_DW, R10, R1, -8),
8327 #ifdef __BIG_ENDIAN
8328 BPF_STX_MEM(BPF_W, R10, R2, -4),
8329 #else
8330 BPF_STX_MEM(BPF_W, R10, R2, -8),
8331 #endif
8332 BPF_LDX_MEM(BPF_DW, R0, R10, -8),
8333 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
8334 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8335 BPF_EXIT_INSN(),
8336 },
8337 INTERNAL,
8338 { },
8339 { { 0, 0 } },
8340 .stack_depth = 8,
8341 },
8342 {
8343 "BPF_STX_MEM | BPF_W, MSB set",
8344 .u.insns_int = {
8345 BPF_LD_IMM64(R1, 0x8090a0b0c0d0e0f0ULL),
8346 BPF_LD_IMM64(R2, 0x8182838485868788ULL),
8347 BPF_LD_IMM64(R3, 0x8090a0b085868788ULL),
8348 BPF_STX_MEM(BPF_DW, R10, R1, -8),
8349 #ifdef __BIG_ENDIAN
8350 BPF_STX_MEM(BPF_W, R10, R2, -4),
8351 #else
8352 BPF_STX_MEM(BPF_W, R10, R2, -8),
8353 #endif
8354 BPF_LDX_MEM(BPF_DW, R0, R10, -8),
8355 BPF_JMP_REG(BPF_JNE, R0, R3, 1),
8356 BPF_ALU64_IMM(BPF_MOV, R0, 0),
8357 BPF_EXIT_INSN(),
8358 },
8359 INTERNAL,
8360 { },
8361 { { 0, 0 } },
8362 .stack_depth = 8,
8363 },
8364 /* BPF_ST(X) | BPF_MEM | BPF_B/H/W/DW */
8365 {
8366 "ST_MEM_B: Store/Load byte: max negative",
8367 .u.insns_int = {
8368 BPF_ALU32_IMM(BPF_MOV, R0, 1),
8369 BPF_ST_MEM(BPF_B, R10, -40, 0xff),
8370 BPF_LDX_MEM(BPF_B, R0, R10, -40),
8371 BPF_EXIT_INSN(),
8372 },
8373 INTERNAL,
8374 { },
8375 { { 0, 0xff } },
8376 .stack_depth = 40,
8377 },
8378 {
8379 "ST_MEM_B: Store/Load byte: max positive",
8380 .u.insns_int = {
8381 BPF_ALU32_IMM(BPF_MOV, R0, 1),
8382 BPF_ST_MEM(BPF_H, R10, -40, 0x7f),
8383 BPF_LDX_MEM(BPF_H, R0, R10, -40),
8384 BPF_EXIT_INSN(),
8385 },
8386 INTERNAL,
8387 { },
8388 { { 0, 0x7f } },
8389 .stack_depth = 40,
8390 },
8391 {
8392 "STX_MEM_B: Store/Load byte: max negative",
8393 .u.insns_int = {
8394 BPF_LD_IMM64(R0, 0),
8395 BPF_LD_IMM64(R1, 0xffLL),
8396 BPF_STX_MEM(BPF_B, R10, R1, -40),
8397 BPF_LDX_MEM(BPF_B, R0, R10, -40),
8398 BPF_EXIT_INSN(),
8399 },
8400 INTERNAL,
8401 { },
8402 { { 0, 0xff } },
8403 .stack_depth = 40,
8404 },
8405 {
8406 "ST_MEM_H: Store/Load half word: max negative",
8407 .u.insns_int = {
8408 BPF_ALU32_IMM(BPF_MOV, R0, 1),
8409 BPF_ST_MEM(BPF_H, R10, -40, 0xffff),
8410 BPF_LDX_MEM(BPF_H, R0, R10, -40),
8411 BPF_EXIT_INSN(),
8412 },
8413 INTERNAL,
8414 { },
8415 { { 0, 0xffff } },
8416 .stack_depth = 40,
8417 },
8418 {
8419 "ST_MEM_H: Store/Load half word: max positive",
8420 .u.insns_int = {
8421 BPF_ALU32_IMM(BPF_MOV, R0, 1),
8422 BPF_ST_MEM(BPF_H, R10, -40, 0x7fff),
8423 BPF_LDX_MEM(BPF_H, R0, R10, -40),
8424 BPF_EXIT_INSN(),
8425 },
8426 INTERNAL,
8427 { },
8428 { { 0, 0x7fff } },
8429 .stack_depth = 40,
8430 },
8431 {
8432 "STX_MEM_H: Store/Load half word: max negative",
8433 .u.insns_int = {
8434 BPF_LD_IMM64(R0, 0),
8435 BPF_LD_IMM64(R1, 0xffffLL),
8436 BPF_STX_MEM(BPF_H, R10, R1, -40),
8437 BPF_LDX_MEM(BPF_H, R0, R10, -40),
8438 BPF_EXIT_INSN(),
8439 },
8440 INTERNAL,
8441 { },
8442 { { 0, 0xffff } },
8443 .stack_depth = 40,
8444 },
8445 {
8446 "ST_MEM_W: Store/Load word: max negative",
8447 .u.insns_int = {
8448 BPF_ALU32_IMM(BPF_MOV, R0, 1),
8449 BPF_ST_MEM(BPF_W, R10, -40, 0xffffffff),
8450 BPF_LDX_MEM(BPF_W, R0, R10, -40),
8451 BPF_EXIT_INSN(),
8452 },
8453 INTERNAL,
8454 { },
8455 { { 0, 0xffffffff } },
8456 .stack_depth = 40,
8457 },
8458 {
8459 "ST_MEM_W: Store/Load word: max positive",
8460 .u.insns_int = {
8461 BPF_ALU32_IMM(BPF_MOV, R0, 1),
8462 BPF_ST_MEM(BPF_W, R10, -40, 0x7fffffff),
8463 BPF_LDX_MEM(BPF_W, R0, R10, -40),
8464 BPF_EXIT_INSN(),
8465 },
8466 INTERNAL,
8467 { },
8468 { { 0, 0x7fffffff } },
8469 .stack_depth = 40,
8470 },
8471 {
8472 "STX_MEM_W: Store/Load word: max negative",
8473 .u.insns_int = {
8474 BPF_LD_IMM64(R0, 0),
8475 BPF_LD_IMM64(R1, 0xffffffffLL),
8476 BPF_STX_MEM(BPF_W, R10, R1, -40),
8477 BPF_LDX_MEM(BPF_W, R0, R10, -40),
8478 BPF_EXIT_INSN(),
8479 },
8480 INTERNAL,
8481 { },
8482 { { 0, 0xffffffff } },
8483 .stack_depth = 40,
8484 },
8485 {
8486 "ST_MEM_DW: Store/Load double word: max negative",
8487 .u.insns_int = {
8488 BPF_ALU32_IMM(BPF_MOV, R0, 1),
8489 BPF_ST_MEM(BPF_DW, R10, -40, 0xffffffff),
8490 BPF_LDX_MEM(BPF_DW, R0, R10, -40),
8491 BPF_EXIT_INSN(),
8492 },
8493 INTERNAL,
8494 { },
8495 { { 0, 0xffffffff } },
8496 .stack_depth = 40,
8497 },
8498 {
8499 "ST_MEM_DW: Store/Load double word: max negative 2",
8500 .u.insns_int = {
8501 BPF_LD_IMM64(R2, 0xffff00000000ffffLL),
8502 BPF_LD_IMM64(R3, 0xffffffffffffffffLL),
8503 BPF_ST_MEM(BPF_DW, R10, -40, 0xffffffff),
8504 BPF_LDX_MEM(BPF_DW, R2, R10, -40),
8505 BPF_JMP_REG(BPF_JEQ, R2, R3, 2),
8506 BPF_MOV32_IMM(R0, 2),
8507 BPF_EXIT_INSN(),
8508 BPF_MOV32_IMM(R0, 1),
8509 BPF_EXIT_INSN(),
8510 },
8511 INTERNAL,
8512 { },
8513 { { 0, 0x1 } },
8514 .stack_depth = 40,
8515 },
8516 {
8517 "ST_MEM_DW: Store/Load double word: max positive",
8518 .u.insns_int = {
8519 BPF_ALU32_IMM(BPF_MOV, R0, 1),
8520 BPF_ST_MEM(BPF_DW, R10, -40, 0x7fffffff),
8521 BPF_LDX_MEM(BPF_DW, R0, R10, -40),
8522 BPF_EXIT_INSN(),
8523 },
8524 INTERNAL,
8525 { },
8526 { { 0, 0x7fffffff } },
8527 .stack_depth = 40,
8528 },
8529 {
8530 "STX_MEM_DW: Store/Load double word: max negative",
8531 .u.insns_int = {
8532 BPF_LD_IMM64(R0, 0),
8533 BPF_LD_IMM64(R1, 0xffffffffffffffffLL),
8534 BPF_STX_MEM(BPF_DW, R10, R1, -40),
8535 BPF_LDX_MEM(BPF_DW, R0, R10, -40),
8536 BPF_EXIT_INSN(),
8537 },
8538 INTERNAL,
8539 { },
8540 { { 0, 0xffffffff } },
8541 .stack_depth = 40,
8542 },
8543 {
8544 "STX_MEM_DW: Store double word: first word in memory",
8545 .u.insns_int = {
8546 BPF_LD_IMM64(R0, 0),
8547 BPF_LD_IMM64(R1, 0x0123456789abcdefLL),
8548 BPF_STX_MEM(BPF_DW, R10, R1, -40),
8549 BPF_LDX_MEM(BPF_W, R0, R10, -40),
8550 BPF_EXIT_INSN(),
8551 },
8552 INTERNAL,
8553 { },
8554 #ifdef __BIG_ENDIAN
8555 { { 0, 0x01234567 } },
8556 #else
8557 { { 0, 0x89abcdef } },
8558 #endif
8559 .stack_depth = 40,
8560 },
8561 {
8562 "STX_MEM_DW: Store double word: second word in memory",
8563 .u.insns_int = {
8564 BPF_LD_IMM64(R0, 0),
8565 BPF_LD_IMM64(R1, 0x0123456789abcdefLL),
8566 BPF_STX_MEM(BPF_DW, R10, R1, -40),
8567 BPF_LDX_MEM(BPF_W, R0, R10, -36),
8568 BPF_EXIT_INSN(),
8569 },
8570 INTERNAL,
8571 { },
8572 #ifdef __BIG_ENDIAN
8573 { { 0, 0x89abcdef } },
8574 #else
8575 { { 0, 0x01234567 } },
8576 #endif
8577 .stack_depth = 40,
8578 },
8579 /* BPF_STX | BPF_ATOMIC | BPF_W/DW */
8580 {
8581 "STX_XADD_W: X + 1 + 1 + 1 + ...",
8582 { },
8583 INTERNAL,
8584 { },
8585 { { 0, 4134 } },
8586 .fill_helper = bpf_fill_stxw,
8587 },
8588 {
8589 "STX_XADD_DW: X + 1 + 1 + 1 + ...",
8590 { },
8591 INTERNAL,
8592 { },
8593 { { 0, 4134 } },
8594 .fill_helper = bpf_fill_stxdw,
8595 },
8596 /*
8597 * Exhaustive tests of atomic operation variants.
8598 * Individual tests are expanded from template macros for all
8599 * combinations of ALU operation, word size and fetching.
8600 */
8601 #define BPF_ATOMIC_POISON(width) ((width) == BPF_W ? (0xbaadf00dULL << 32) : 0)
8602
---
0-DAY CI Kernel Test Service
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2022-03-17 0:48 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-16 16:26 [PATCH bpf-next v3 0/4] bpf, arm64: Optimize BPF store/load using Xu Kuohai
2022-03-16 16:26 ` Xu Kuohai
2022-03-16 16:26 ` [PATCH -next v3 1/4] arm64: insn: add ldr/str with immediate offset Xu Kuohai
2022-03-16 16:26 ` Xu Kuohai
2022-03-16 16:26 ` [PATCH -next v3 2/4] bpf, arm64: Optimize BPF store/load using str/ldr " Xu Kuohai
2022-03-16 16:26 ` Xu Kuohai
2022-03-16 16:26 ` [PATCH -next v3 3/4] bpf/tests: Add tests for BPF_LDX/BPF_STX with different offsets Xu Kuohai
2022-03-16 16:26 ` Xu Kuohai
2022-03-17 0:47 ` kernel test robot
2022-03-16 16:26 ` [PATCH -next v3 4/4] bpf, arm64: adjust the offset of str/ldr(immediate) to positive number Xu Kuohai
2022-03-16 16:26 ` Xu Kuohai
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.