From: Yonghong Song <yhs@fb.com>
To: Brendan Jackman <jackmanb@google.com>, <bpf@vger.kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
KP Singh <kpsingh@chromium.org>,
Florent Revest <revest@chromium.org>
Subject: Re: [PATCH 3/7] bpf: Rename BPF_XADD and prepare to encode other atomics in .imm
Date: Mon, 23 Nov 2020 15:54:38 -0800 [thread overview]
Message-ID: <e7d336ab-524f-9d60-e9ec-8c8426cae0d7@fb.com> (raw)
In-Reply-To: <20201123173202.1335708-4-jackmanb@google.com>
On 11/23/20 9:31 AM, Brendan Jackman wrote:
> A subsequent patch will add additional atomic operations. These new
> operations will use the same opcode field as the existing XADD, with
> the immediate discriminating different operations.
>
> In preparation, rename the instruction mode BPF_ATOMIC and start
> calling the zero immediate BPF_ADD.
>
> This is possible (doesn't break existing valid BPF progs) because the
> immediate field is currently reserved MBZ and BPF_ADD is zero.
>
> All uses are removed from the tree but the BPF_XADD definition is
> kept around to avoid breaking builds for people including kernel
> headers.
>
> Signed-off-by: Brendan Jackman <jackmanb@google.com>
> ---
> Documentation/networking/filter.rst | 27 +++++++++-------
> arch/arm/net/bpf_jit_32.c | 7 ++---
> arch/arm64/net/bpf_jit_comp.c | 16 +++++++---
> arch/mips/net/ebpf_jit.c | 11 +++++--
> arch/powerpc/net/bpf_jit_comp64.c | 25 ++++++++++++---
> arch/riscv/net/bpf_jit_comp32.c | 20 +++++++++---
> arch/riscv/net/bpf_jit_comp64.c | 16 +++++++---
> arch/s390/net/bpf_jit_comp.c | 26 +++++++++-------
> arch/sparc/net/bpf_jit_comp_64.c | 14 +++++++--
> arch/x86/net/bpf_jit_comp.c | 30 +++++++++++-------
> arch/x86/net/bpf_jit_comp32.c | 6 ++--
> drivers/net/ethernet/netronome/nfp/bpf/jit.c | 14 ++++++---
> drivers/net/ethernet/netronome/nfp/bpf/main.h | 4 +--
> .../net/ethernet/netronome/nfp/bpf/verifier.c | 13 +++++---
> include/linux/filter.h | 8 +++--
> include/uapi/linux/bpf.h | 3 +-
> kernel/bpf/core.c | 31 +++++++++++++------
> kernel/bpf/disasm.c | 6 ++--
> kernel/bpf/verifier.c | 24 ++++++++------
> lib/test_bpf.c | 2 +-
> samples/bpf/bpf_insn.h | 4 +--
> samples/bpf/sock_example.c | 3 +-
> samples/bpf/test_cgrp2_attach.c | 6 ++--
> tools/include/linux/filter.h | 7 +++--
> tools/include/uapi/linux/bpf.h | 3 +-
> .../bpf/prog_tests/cgroup_attach_multi.c | 6 ++--
> tools/testing/selftests/bpf/verifier/ctx.c | 6 ++--
> .../testing/selftests/bpf/verifier/leak_ptr.c | 4 +--
> tools/testing/selftests/bpf/verifier/unpriv.c | 3 +-
> tools/testing/selftests/bpf/verifier/xadd.c | 2 +-
> 30 files changed, 230 insertions(+), 117 deletions(-)
>
> diff --git a/Documentation/networking/filter.rst b/Documentation/networking/filter.rst
> index debb59e374de..a9847662bbab 100644
> --- a/Documentation/networking/filter.rst
> +++ b/Documentation/networking/filter.rst
> @@ -1006,13 +1006,13 @@ Size modifier is one of ...
>
> Mode modifier is one of::
>
> - BPF_IMM 0x00 /* used for 32-bit mov in classic BPF and 64-bit in eBPF */
> - BPF_ABS 0x20
> - BPF_IND 0x40
> - BPF_MEM 0x60
> - BPF_LEN 0x80 /* classic BPF only, reserved in eBPF */
> - BPF_MSH 0xa0 /* classic BPF only, reserved in eBPF */
> - BPF_XADD 0xc0 /* eBPF only, exclusive add */
> + BPF_IMM 0x00 /* used for 32-bit mov in classic BPF and 64-bit in eBPF */
> + BPF_ABS 0x20
> + BPF_IND 0x40
> + BPF_MEM 0x60
> + BPF_LEN 0x80 /* classic BPF only, reserved in eBPF */
> + BPF_MSH 0xa0 /* classic BPF only, reserved in eBPF */
> + BPF_ATOMIC 0xc0 /* eBPF only, atomic operations */
>
> eBPF has two non-generic instructions: (BPF_ABS | <size> | BPF_LD) and
> (BPF_IND | <size> | BPF_LD) which are used to access packet data.
> @@ -1044,11 +1044,16 @@ Unlike classic BPF instruction set, eBPF has generic load/store operations::
> BPF_MEM | <size> | BPF_STX: *(size *) (dst_reg + off) = src_reg
> BPF_MEM | <size> | BPF_ST: *(size *) (dst_reg + off) = imm32
> BPF_MEM | <size> | BPF_LDX: dst_reg = *(size *) (src_reg + off)
> - BPF_XADD | BPF_W | BPF_STX: lock xadd *(u32 *)(dst_reg + off16) += src_reg
> - BPF_XADD | BPF_DW | BPF_STX: lock xadd *(u64 *)(dst_reg + off16) += src_reg
>
> -Where size is one of: BPF_B or BPF_H or BPF_W or BPF_DW. Note that 1 and
> -2 byte atomic increments are not supported.
> +Where size is one of: BPF_B or BPF_H or BPF_W or BPF_DW.
> +
> +It also includes atomic operations, which use the immediate field for extra
> +encoding.
> +
> + BPF_ADD, BPF_ATOMIC | BPF_W | BPF_STX: lock xadd *(u32 *)(dst_reg + off16) += src_reg
> + BPF_ADD, BPF_ATOMIC | BPF_DW | BPF_STX: lock xadd *(u64 *)(dst_reg + off16) += src_reg
> +
> +Note that 1 and 2 byte atomic operations are not supported.
>
> eBPF has one 16-byte instruction: BPF_LD | BPF_DW | BPF_IMM which consists
> of two consecutive ``struct bpf_insn`` 8-byte blocks and interpreted as single
> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
> index 0207b6ea6e8a..897634d0a67c 100644
> --- a/arch/arm/net/bpf_jit_32.c
> +++ b/arch/arm/net/bpf_jit_32.c
> @@ -1620,10 +1620,9 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx)
> }
> emit_str_r(dst_lo, tmp2, off, ctx, BPF_SIZE(code));
> break;
> - /* STX XADD: lock *(u32 *)(dst + off) += src */
> - case BPF_STX | BPF_XADD | BPF_W:
> - /* STX XADD: lock *(u64 *)(dst + off) += src */
> - case BPF_STX | BPF_XADD | BPF_DW:
> + /* Atomic ops */
> + case BPF_STX | BPF_ATOMIC | BPF_W:
> + case BPF_STX | BPF_ATOMIC | BPF_DW:
> goto notyet;
> /* STX: *(size *)(dst + off) = src */
> case BPF_STX | BPF_MEM | BPF_W:
> diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
> index ef9f1d5e989d..f7b194878a99 100644
> --- a/arch/arm64/net/bpf_jit_comp.c
> +++ b/arch/arm64/net/bpf_jit_comp.c
> @@ -875,10 +875,18 @@ static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx,
> }
> break;
>
> - /* STX XADD: lock *(u32 *)(dst + off) += src */
> - case BPF_STX | BPF_XADD | BPF_W:
> - /* STX XADD: lock *(u64 *)(dst + off) += src */
> - case BPF_STX | BPF_XADD | BPF_DW:
> + case BPF_STX | BPF_ATOMIC | BPF_W:
> + case BPF_STX | BPF_ATOMIC | BPF_DW:
> + if (insn->imm != BPF_ADD) {
Currently BPF_ADD (although it is 0) is encoded at bit 4-7 of imm.
Do you think we should encode it in 0-3 to make such a comparision
and subsequent insn->imm = BPF_ADD making more sense?
> + pr_err_once("unknown atomic op code %02x\n", insn->imm);
> + return -EINVAL;
> + }
> +
> + /* STX XADD: lock *(u32 *)(dst + off) += src
> + * and
> + * STX XADD: lock *(u64 *)(dst + off) += src
> + */
> +
> if (!off) {
> reg = dst;
> } else {
[...]
> diff --git a/drivers/net/ethernet/netronome/nfp/bpf/jit.c b/drivers/net/ethernet/netronome/nfp/bpf/jit.c
> index 0a721f6e8676..0767d7b579e9 100644
> --- a/drivers/net/ethernet/netronome/nfp/bpf/jit.c
> +++ b/drivers/net/ethernet/netronome/nfp/bpf/jit.c
> @@ -3109,13 +3109,19 @@ mem_xadd(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta, bool is64)
> return 0;
> }
>
> -static int mem_xadd4(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta)
> +static int mem_atm4(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta)
> {
> + if (meta->insn.off != BPF_ADD)
> + return -EOPNOTSUPP;
meta->insn.imm?
> +
> return mem_xadd(nfp_prog, meta, false);
> }
>
> -static int mem_xadd8(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta)
> +static int mem_atm(struct nfp_prog *nfp_prog, struct nfp_insn_meta *meta)
> {
> + if (meta->insn.off != BPF_ADD)
meta->insn.imm?
> + return -EOPNOTSUPP;
> +
> return mem_xadd(nfp_prog, meta, true);
> }
>
> @@ -3475,8 +3481,8 @@ static const instr_cb_t instr_cb[256] = {
> [BPF_STX | BPF_MEM | BPF_H] = mem_stx2,
> [BPF_STX | BPF_MEM | BPF_W] = mem_stx4,
> [BPF_STX | BPF_MEM | BPF_DW] = mem_stx8,
> - [BPF_STX | BPF_XADD | BPF_W] = mem_xadd4,
> - [BPF_STX | BPF_XADD | BPF_DW] = mem_xadd8,
> + [BPF_STX | BPF_ATOMIC | BPF_W] = mem_atm4,
> + [BPF_STX | BPF_ATOMIC | BPF_DW] = mem_atm8,
> [BPF_ST | BPF_MEM | BPF_B] = mem_st1,
> [BPF_ST | BPF_MEM | BPF_H] = mem_st2,
> [BPF_ST | BPF_MEM | BPF_W] = mem_st4,
[...]
next prev parent reply other threads:[~2020-11-23 23:55 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-23 17:31 [PATCH 0/7] Atomics for eBPF Brendan Jackman
2020-11-23 17:31 ` [PATCH 1/7] bpf: Factor out emission of ModR/M for *(reg + off) Brendan Jackman
2020-11-23 17:31 ` [PATCH 2/7] bpf: x86: Factor out emission of REX byte Brendan Jackman
2020-11-23 17:31 ` [PATCH 3/7] bpf: Rename BPF_XADD and prepare to encode other atomics in .imm Brendan Jackman
2020-11-23 23:54 ` Yonghong Song [this message]
2020-11-24 11:02 ` Brendan Jackman
2020-11-24 16:04 ` Yonghong Song
2020-11-24 3:28 ` kernel test robot
2020-11-24 6:50 ` Alexei Starovoitov
2020-11-24 11:21 ` Brendan Jackman
2020-11-24 22:43 ` Alexei Starovoitov
2020-11-23 17:31 ` [PATCH 4/7] bpf: Move BPF_STX reserved field check into BPF_STX verifier code Brendan Jackman
2020-11-23 17:32 ` [PATCH 5/7] bpf: Add BPF_FETCH field / create atomic_fetch_add instruction Brendan Jackman
2020-11-23 21:12 ` kernel test robot
2020-11-23 21:51 ` kernel test robot
2020-11-24 6:52 ` Alexei Starovoitov
2020-11-24 10:48 ` Brendan Jackman
2020-11-23 17:32 ` [PATCH 6/7] bpf: Add instructions for atomic_cmpxchg and friends Brendan Jackman
2020-11-23 19:29 ` Brendan Jackman
2020-11-24 6:40 ` Alexei Starovoitov
2020-11-24 10:55 ` Brendan Jackman
2020-11-24 22:51 ` Alexei Starovoitov
2020-11-23 17:32 ` [PATCH 7/7] bpf: Add tests for new BPF atomic operations Brendan Jackman
2020-11-24 0:26 ` Yonghong Song
2020-11-24 13:10 ` Brendan Jackman
2020-11-23 17:36 ` [PATCH 0/7] Atomics for eBPF Brendan Jackman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e7d336ab-524f-9d60-e9ec-8c8426cae0d7@fb.com \
--to=yhs@fb.com \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=jackmanb@google.com \
--cc=kpsingh@chromium.org \
--cc=revest@chromium.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).