From: "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>
To: Michael Ellerman <mpe@ellerman.id.au>,
Daniel Borkmann <daniel@iogearbox.net>,
Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Jiri Olsa <jolsa@redhat.com>,
ykaliuta@redhat.com,
Christophe Leroy <christophe.leroy@csgroup.eu>,
song@kernel.org, johan.almbladh@anyfinetworks.com,
Hari Bathini <hbathini@linux.ibm.com>, <bpf@vger.kernel.org>,
<linuxppc-dev@lists.ozlabs.org>
Subject: [PATCH 03/13] powerpc/bpf: Update ldimm64 instructions during extra pass
Date: Thu, 6 Jan 2022 17:15:07 +0530 [thread overview]
Message-ID: <7cc162af77ba918eb3ecd26ec9e7824bc44b1fae.1641468127.git.naveen.n.rao@linux.vnet.ibm.com> (raw)
In-Reply-To: <cover.1641468127.git.naveen.n.rao@linux.vnet.ibm.com>
These instructions are updated after the initial JIT, so redo codegen
during the extra pass. Rename bpf_jit_fixup_subprog_calls() to clarify
that this is more than just subprog calls.
Fixes: 69c087ba6225b5 ("bpf: Add bpf_for_each_map_elem() helper")
Cc: stable@vger.kernel.org # v5.15
Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
---
arch/powerpc/net/bpf_jit_comp.c | 29 +++++++++++++++++++++++------
arch/powerpc/net/bpf_jit_comp32.c | 6 ++++++
arch/powerpc/net/bpf_jit_comp64.c | 7 ++++++-
3 files changed, 35 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index d6ffdd0f2309d0..56dd1f4e3e4447 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -23,15 +23,15 @@ static void bpf_jit_fill_ill_insns(void *area, unsigned int size)
memset32(area, BREAKPOINT_INSTRUCTION, size / 4);
}
-/* Fix the branch target addresses for subprog calls */
-static int bpf_jit_fixup_subprog_calls(struct bpf_prog *fp, u32 *image,
- struct codegen_context *ctx, u32 *addrs)
+/* Fix updated addresses (for subprog calls, ldimm64, et al) during extra pass */
+static int bpf_jit_fixup_addresses(struct bpf_prog *fp, u32 *image,
+ struct codegen_context *ctx, u32 *addrs)
{
const struct bpf_insn *insn = fp->insnsi;
bool func_addr_fixed;
u64 func_addr;
u32 tmp_idx;
- int i, ret;
+ int i, j, ret;
for (i = 0; i < fp->len; i++) {
/*
@@ -66,6 +66,23 @@ static int bpf_jit_fixup_subprog_calls(struct bpf_prog *fp, u32 *image,
* of the JITed sequence remains unchanged.
*/
ctx->idx = tmp_idx;
+ } else if (insn[i].code == (BPF_LD | BPF_IMM | BPF_DW)) {
+ tmp_idx = ctx->idx;
+ ctx->idx = addrs[i] / 4;
+#ifdef CONFIG_PPC32
+ PPC_LI32(ctx->b2p[insn[i].dst_reg] - 1, (u32)insn[i + 1].imm);
+ PPC_LI32(ctx->b2p[insn[i].dst_reg], (u32)insn[i].imm);
+ for (j = ctx->idx - addrs[i] / 4; j < 4; j++)
+ EMIT(PPC_RAW_NOP());
+#else
+ func_addr = ((u64)(u32)insn[i].imm) | (((u64)(u32)insn[i + 1].imm) << 32);
+ PPC_LI64(b2p[insn[i].dst_reg], func_addr);
+ /* overwrite rest with nops */
+ for (j = ctx->idx - addrs[i] / 4; j < 5; j++)
+ EMIT(PPC_RAW_NOP());
+#endif
+ ctx->idx = tmp_idx;
+ i++;
}
}
@@ -200,13 +217,13 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp)
/*
* Do not touch the prologue and epilogue as they will remain
* unchanged. Only fix the branch target address for subprog
- * calls in the body.
+ * calls in the body, and ldimm64 instructions.
*
* This does not change the offsets and lengths of the subprog
* call instruction sequences and hence, the size of the JITed
* image as well.
*/
- bpf_jit_fixup_subprog_calls(fp, code_base, &cgctx, addrs);
+ bpf_jit_fixup_addresses(fp, code_base, &cgctx, addrs);
/* There is no need to perform the usual passes. */
goto skip_codegen_passes;
diff --git a/arch/powerpc/net/bpf_jit_comp32.c b/arch/powerpc/net/bpf_jit_comp32.c
index 997a47fa615b30..2258d3886d02ec 100644
--- a/arch/powerpc/net/bpf_jit_comp32.c
+++ b/arch/powerpc/net/bpf_jit_comp32.c
@@ -293,6 +293,8 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, struct codegen_context *
bool func_addr_fixed;
u64 func_addr;
u32 true_cond;
+ u32 tmp_idx;
+ int j;
/*
* addrs[] maps a BPF bytecode address into a real offset from
@@ -908,8 +910,12 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, struct codegen_context *
* 16 byte instruction that uses two 'struct bpf_insn'
*/
case BPF_LD | BPF_IMM | BPF_DW: /* dst = (u64) imm */
+ tmp_idx = ctx->idx;
PPC_LI32(dst_reg_h, (u32)insn[i + 1].imm);
PPC_LI32(dst_reg, (u32)insn[i].imm);
+ /* padding to allow full 4 instructions for later patching */
+ for (j = ctx->idx - tmp_idx; j < 4; j++)
+ EMIT(PPC_RAW_NOP());
/* Adjust for two bpf instructions */
addrs[++i] = ctx->idx * 4;
break;
diff --git a/arch/powerpc/net/bpf_jit_comp64.c b/arch/powerpc/net/bpf_jit_comp64.c
index 472d4a551945dd..3d018ecc475b2b 100644
--- a/arch/powerpc/net/bpf_jit_comp64.c
+++ b/arch/powerpc/net/bpf_jit_comp64.c
@@ -319,6 +319,7 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, struct codegen_context *
u64 imm64;
u32 true_cond;
u32 tmp_idx;
+ int j;
/*
* addrs[] maps a BPF bytecode address into a real offset from
@@ -848,9 +849,13 @@ int bpf_jit_build_body(struct bpf_prog *fp, u32 *image, struct codegen_context *
case BPF_LD | BPF_IMM | BPF_DW: /* dst = (u64) imm */
imm64 = ((u64)(u32) insn[i].imm) |
(((u64)(u32) insn[i+1].imm) << 32);
+ tmp_idx = ctx->idx;
+ PPC_LI64(dst_reg, imm64);
+ /* padding to allow full 5 instructions for later patching */
+ for (j = ctx->idx - tmp_idx; j < 5; j++)
+ EMIT(PPC_RAW_NOP());
/* Adjust for two bpf instructions */
addrs[++i] = ctx->idx * 4;
- PPC_LI64(dst_reg, imm64);
break;
/*
--
2.34.1
next prev parent reply other threads:[~2022-01-06 11:46 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-06 11:45 [PATCH 00/13] powerpc/bpf: Some fixes and updates Naveen N. Rao
2022-01-06 11:45 ` [PATCH 01/13] bpf: Guard against accessing NULL pt_regs in bpf_get_task_stack() Naveen N. Rao
2022-01-07 10:21 ` Daniel Borkmann
2022-01-10 8:57 ` Christophe Leroy
2022-01-10 10:36 ` Naveen N. Rao
2022-01-06 11:45 ` [PATCH 02/13] powerpc32/bpf: Fix codegen for bpf-to-bpf calls Naveen N. Rao
2022-01-10 9:06 ` Christophe Leroy
2022-01-10 10:52 ` Naveen N. Rao
2022-01-06 11:45 ` Naveen N. Rao [this message]
2022-01-08 14:45 ` [PATCH 03/13] powerpc/bpf: Update ldimm64 instructions during extra pass Jiri Olsa
2022-01-10 9:27 ` Christophe Leroy
2022-01-10 10:56 ` Naveen N. Rao
2022-01-06 11:45 ` [PATCH 04/13] tools/bpf: Rename 'struct event' to avoid naming conflict Naveen N. Rao
2022-01-07 10:21 ` Daniel Borkmann
2022-01-06 11:45 ` [PATCH 05/13] powerpc/bpf: Skip branch range validation during first pass Naveen N. Rao
2022-01-06 11:45 ` [PATCH 06/13] powerpc/bpf: Emit a single branch instruction for known short branch ranges Naveen N. Rao
2022-01-06 11:45 ` [PATCH 07/13] powerpc/bpf: Handle large branch ranges with BPF_EXIT Naveen N. Rao
2022-01-06 11:45 ` [PATCH 08/13] powerpc64/bpf: Limit 'ldbrx' to processors compliant with ISA v2.06 Naveen N. Rao
2022-01-06 11:45 ` [PATCH 09/13] powerpc64/bpf: Do not save/restore LR on each call to bpf_stf_barrier() Naveen N. Rao
2022-01-06 11:45 ` [PATCH 10/13] powerpc64/bpf: Use r12 for constant blinding Naveen N. Rao
2022-01-06 11:45 ` [PATCH 11/13] powerpc64/bpf elfv2: Setup kernel TOC in r2 on entry Naveen N. Rao
2022-01-10 9:20 ` Christophe Leroy
2022-01-11 10:31 ` Naveen N. Rao
2022-01-11 14:35 ` Christophe Leroy
2022-01-11 14:43 ` Christophe Leroy
2022-01-14 11:17 ` Naveen N. Rao
2022-01-06 11:45 ` [PATCH 12/13] powerpc64/bpf elfv1: Do not load TOC before calling functions Naveen N. Rao
2022-01-06 11:45 ` [PATCH 13/13] powerpc64/bpf: Optimize instruction sequence used for function calls Naveen N. Rao
2022-01-06 21:46 ` [PATCH 00/13] powerpc/bpf: Some fixes and updates Daniel Borkmann
2022-01-07 7:36 ` Naveen N. Rao
2022-01-07 10:20 ` Daniel Borkmann
2022-01-10 3:47 ` Michael Ellerman
2022-01-16 10:41 ` Michael Ellerman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7cc162af77ba918eb3ecd26ec9e7824bc44b1fae.1641468127.git.naveen.n.rao@linux.vnet.ibm.com \
--to=naveen.n.rao@linux.vnet.ibm.com \
--cc=alexei.starovoitov@gmail.com \
--cc=bpf@vger.kernel.org \
--cc=christophe.leroy@csgroup.eu \
--cc=daniel@iogearbox.net \
--cc=hbathini@linux.ibm.com \
--cc=johan.almbladh@anyfinetworks.com \
--cc=jolsa@redhat.com \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mpe@ellerman.id.au \
--cc=song@kernel.org \
--cc=ykaliuta@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).