From: Joanne Koong <joannelkoong@gmail.com>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>,
Martin KaFai Lau <martin.lau@linux.dev>,
Daniel Borkmann <daniel@iogearbox.net>,
Andrii Nakryiko <andrii@kernel.org>,
Martin KaFai Lau <martin.lau@kernel.org>,
Alexei Starovoitov <ast@kernel.org>,
Network Development <netdev@vger.kernel.org>,
Kumar Kartikeya Dwivedi <memxor@gmail.com>,
Kernel Team <kernel-team@fb.com>, bpf <bpf@vger.kernel.org>
Subject: Re: [PATCH v9 bpf-next 3/5] bpf: Add skb dynptrs
Date: Tue, 31 Jan 2023 12:47:52 -0800 [thread overview]
Message-ID: <CAJnrk1a2SY5NqhibczOhd+jqL3W9U1rbTeiQpYw-oUS8_Cr1_g@mail.gmail.com> (raw)
In-Reply-To: <CAADnVQJ9Pb10boAR=ZVaXOJwjHPkFXKn9n9RWrzXgK3GaQ1N0g@mail.gmail.com>
On Tue, Jan 31, 2023 at 11:59 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Tue, Jan 31, 2023 at 10:30 AM Joanne Koong <joannelkoong@gmail.com> wrote:
> >
> > On Mon, Jan 30, 2023 at 5:04 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Mon, Jan 30, 2023 at 2:31 PM Alexei Starovoitov
> > > <alexei.starovoitov@gmail.com> wrote:
> > > >
> > > > On Mon, Jan 30, 2023 at 02:04:08PM -0800, Martin KaFai Lau wrote:
> > > > > On 1/27/23 11:17 AM, Joanne Koong wrote:
> > > > > > @@ -8243,6 +8316,28 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
> > > > > > mark_reg_known_zero(env, regs, BPF_REG_0);
> > > > > > regs[BPF_REG_0].type = PTR_TO_MEM | ret_flag;
> > > > > > regs[BPF_REG_0].mem_size = meta.mem_size;
> > > > > > + if (func_id == BPF_FUNC_dynptr_data &&
> > > > > > + dynptr_type == BPF_DYNPTR_TYPE_SKB) {
> > > > > > + bool seen_direct_write = env->seen_direct_write;
> > > > > > +
> > > > > > + regs[BPF_REG_0].type |= DYNPTR_TYPE_SKB;
> > > > > > + if (!may_access_direct_pkt_data(env, NULL, BPF_WRITE))
> > > > > > + regs[BPF_REG_0].type |= MEM_RDONLY;
> > > > > > + else
> > > > > > + /*
> > > > > > + * Calling may_access_direct_pkt_data() will set
> > > > > > + * env->seen_direct_write to true if the skb is
> > > > > > + * writable. As an optimization, we can ignore
> > > > > > + * setting env->seen_direct_write.
> > > > > > + *
> > > > > > + * env->seen_direct_write is used by skb
> > > > > > + * programs to determine whether the skb's page
> > > > > > + * buffers should be cloned. Since data slice
> > > > > > + * writes would only be to the head, we can skip
> > > > > > + * this.
> > > > > > + */
> > > > > > + env->seen_direct_write = seen_direct_write;
> > > > > > + }
> > > > >
> > > > > [ ... ]
> > > > >
> > > > > > @@ -9263,17 +9361,26 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_
> > > > > > return ret;
> > > > > > break;
> > > > > > case KF_ARG_PTR_TO_DYNPTR:
> > > > > > + {
> > > > > > + enum bpf_arg_type dynptr_arg_type = ARG_PTR_TO_DYNPTR;
> > > > > > +
> > > > > > if (reg->type != PTR_TO_STACK &&
> > > > > > reg->type != CONST_PTR_TO_DYNPTR) {
> > > > > > verbose(env, "arg#%d expected pointer to stack or dynptr_ptr\n", i);
> > > > > > return -EINVAL;
> > > > > > }
> > > > > > - ret = process_dynptr_func(env, regno, insn_idx,
> > > > > > - ARG_PTR_TO_DYNPTR | MEM_RDONLY);
> > > > > > + if (meta->func_id == special_kfunc_list[KF_bpf_dynptr_from_skb])
> > > > > > + dynptr_arg_type |= MEM_UNINIT | DYNPTR_TYPE_SKB;
> > > > > > + else
> > > > > > + dynptr_arg_type |= MEM_RDONLY;
> > > > > > +
> > > > > > + ret = process_dynptr_func(env, regno, insn_idx, dynptr_arg_type,
> > > > > > + meta->func_id);
> > > > > > if (ret < 0)
> > > > > > return ret;
> > > > > > break;
> > > > > > + }
> > > > > > case KF_ARG_PTR_TO_LIST_HEAD:
> > > > > > if (reg->type != PTR_TO_MAP_VALUE &&
> > > > > > reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) {
> > > > > > @@ -15857,6 +15964,14 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
> > > > > > desc->func_id == special_kfunc_list[KF_bpf_rdonly_cast]) {
> > > > > > insn_buf[0] = BPF_MOV64_REG(BPF_REG_0, BPF_REG_1);
> > > > > > *cnt = 1;
> > > > > > + } else if (desc->func_id == special_kfunc_list[KF_bpf_dynptr_from_skb]) {
> > > > > > + bool is_rdonly = !may_access_direct_pkt_data(env, NULL, BPF_WRITE);
> > > > >
> > > > > Does it need to restore the env->seen_direct_write here also?
> > > > >
> > > > > It seems this 'seen_direct_write' saving/restoring is needed now because
> > > > > 'may_access_direct_pkt_data(BPF_WRITE)' is not only called when it is
> > > > > actually writing the packet. Some refactoring can help to avoid issue like
> > > > > this.
> > > > >
> > > > > While at 'seen_direct_write', Alexei has also pointed out that the verifier
> > > > > needs to track whether the (packet) 'slice' returned by bpf_dynptr_data()
> > > > > has been written. It should be tracked in 'seen_direct_write'. Take a look
> > > > > at how reg_is_pkt_pointer() and may_access_direct_pkt_data() are done in
> > > > > check_mem_access(). iirc, this reg_is_pkt_pointer() part got loss somewhere
> > > > > in v5 (or v4?) when bpf_dynptr_data() was changed to return register typed
> > > > > PTR_TO_MEM instead of PTR_TO_PACKET.
> > > >
> > > > btw tc progs are using gen_prologue() approach because data/data_end are not kfuncs
> > > > (nothing is being called by the bpf prog).
> > > > In this case we don't need to repeat this approach. If so we don't need to
> > > > set seen_direct_write.
> > > > Instead bpf_dynptr_data() can call bpf_skb_pull_data() directly.
> > > > And technically we don't need to limit it to skb head. It can handle any off/len.
> > > > It will work for skb, but there is no equivalent for xdp_pull_data().
> > > > I don't think we can implement xdp_pull_data in all drivers.
> > > > That's massive amount of work, but we need to be consistent if we want
> > > > dynptr to wrap both skb and xdp.
> > > > We can say dynptr_data is for head only, but we've seen bugs where people
> > > > had to switch from data/data_end to load_bytes.
> > > >
> > > > Also bpf_skb_pull_data is quite heavy. For progs that only want to parse
> > > > the packet calling that in bpf_dynptr_data is a heavy hammer.
> > > >
> > > > It feels that we need to go back to skb_header_pointer-like discussion.
> > > > Something like:
> > > > bpf_dynptr_slice(const struct bpf_dynptr *ptr, u32 offset, u32 len, void *buffer)
> > > > Whether buffer is a part of dynptr or program provided is tbd.
> > >
> > > making it hidden within dynptr would make this approach unreliable
> > > (memory allocations, which can fail, etc). But if we ask users to pass
> > > it directly, then it should be relatively easy to use in practice with
> > > some pre-allocated per-CPU buffer:
> > >
> > >
> > > struct {
> > > __int(type, BPF_MAP_TYPE_PERCPU_ARRAY);
> > > __int(max_entries, 1);
> > > __type(key, int);
> > > __type(value, char[4096]);
> > > } scratch SEC(".maps");
> > >
> > >
> > > ...
> > >
> > >
> > > struct dyn_ptr *dp = bpf_dynptr_from_skb(...).
> > > void *p, *buf;
> > > int zero = 0;
> > >
> > > buf = bpf_map_lookup_elem(&scratch, &zero);
> > > if (!buf) return 0; /* can't happen */
> > >
> > > p = bpf_dynptr_slice(dp, off, 16, buf);
> > > if (p == NULL) {
> > > /* out of range */
> > > } else {
> > > /* work with p directly */
> > > }
> > >
> > > /* if we wrote something to p and it was copied to buffer, write it back */
> > > if (p == buf) {
> > > bpf_dynptr_write(dp, buf, 16);
> > > }
> > >
> > >
> > > We'll just need to teach verifier to make sure that buf is at least 16
> > > byte long.
> >
> > I'm confused what the benefit of passing in the buffer is. If it's to
> > avoid the uncloning, this will still need to happen if the user writes
> > back the data to the skb (which will be the majority of cases). If
> > it's to avoid uncloning if the user only reads the data of a writable
> > prog, then we could add logic in the verifier so that we don't pull
> > the data in this case; the uncloning might still happen regardless if
> > another part of the program does a direct write. If the benefit is to
> > avoid needing to pull the data, then can't the user just use
> > bpf_dynptr_read, which takes in a buffer?
>
> There is no unclone and there is no pull in xdp.
> The main idea of this semantics of bpf_dynptr_slice is to make it
> work the same way on skb and xdp for _read_ case.
> Writes are going to be different between skb and xdp anyway.
> In some rare cases the writes can be the same for skb and xdp
> with this bpf_dynptr_slice + bpf_dynptr_write logic,
> but that's a minor feature addition of the api.
bpf_dynptr_read works the same way on skb and xdp. bpf_dynptr_read
takes in a buffer as well, so what is the added benefit of
bpf_dynptr_slice?
>
> I'd say in skb cases the progs do reads and either drop
> or forward the skb.
> Writes to skb are done from time to time too, because
> they're a pain to do correctly.
> nat is the main use case for skb rewrites.
> In xdp cases the progs do parse, drop, rewrite, xmit more or less equally.
next prev parent reply other threads:[~2023-01-31 20:48 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-27 19:16 [PATCH v9 bpf-next 0/5] Add skb + xdp dynptrs Joanne Koong
2023-01-27 19:16 ` [PATCH v9 bpf-next 1/5] bpf: Allow "sk_buff" and "xdp_buff" as valid kfunc arg types Joanne Koong
2023-01-27 19:17 ` [PATCH v9 bpf-next 2/5] bpf: Allow initializing dynptrs in kfuncs Joanne Koong
2023-01-27 19:17 ` [PATCH v9 bpf-next 3/5] bpf: Add skb dynptrs Joanne Koong
2023-01-29 23:39 ` Alexei Starovoitov
2023-01-31 0:44 ` Joanne Koong
2023-01-31 5:36 ` Alexei Starovoitov
2023-01-31 17:54 ` Joanne Koong
2023-01-31 19:50 ` Alexei Starovoitov
2023-01-31 21:29 ` Joanne Koong
2023-02-08 21:46 ` Joanne Koong
2023-02-08 23:22 ` Alexei Starovoitov
2023-01-30 22:04 ` Martin KaFai Lau
2023-01-30 22:31 ` Alexei Starovoitov
2023-01-30 23:11 ` Martin KaFai Lau
2023-01-31 1:04 ` Andrii Nakryiko
2023-01-31 1:49 ` Martin KaFai Lau
2023-01-31 4:43 ` Andrii Nakryiko
2023-01-31 5:30 ` Alexei Starovoitov
2023-01-31 22:07 ` Martin KaFai Lau
2023-01-31 23:17 ` Joanne Koong
2023-02-01 0:46 ` Alexei Starovoitov
2023-02-01 0:11 ` Andrii Nakryiko
2023-02-01 0:40 ` Alexei Starovoitov
2023-02-02 1:21 ` Andrii Nakryiko
2023-02-02 11:43 ` Alexei Starovoitov
2023-02-03 21:37 ` Andrii Nakryiko
2023-02-08 2:25 ` Alexei Starovoitov
2023-02-08 20:13 ` Joanne Koong
2023-02-09 0:39 ` Andrii Nakryiko
2023-01-31 18:30 ` Joanne Koong
2023-01-31 19:58 ` Alexei Starovoitov
2023-01-31 20:47 ` Joanne Koong [this message]
2023-01-31 21:10 ` Alexei Starovoitov
2023-01-31 21:33 ` Joanne Koong
2023-01-31 18:18 ` Joanne Koong
2023-01-31 0:48 ` Andrii Nakryiko
2023-01-31 0:55 ` Joanne Koong
2023-01-31 1:06 ` Andrii Nakryiko
2023-01-31 1:13 ` Joanne Koong
2023-01-31 1:19 ` Andrii Nakryiko
2023-01-27 19:17 ` [PATCH v9 bpf-next 4/5] bpf: Add xdp dynptrs Joanne Koong
2023-01-27 19:17 ` [PATCH v9 bpf-next 5/5] selftests/bpf: tests for using dynptrs to parse skb and xdp buffers Joanne Koong
2023-01-31 0:49 ` Andrii Nakryiko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAJnrk1a2SY5NqhibczOhd+jqL3W9U1rbTeiQpYw-oUS8_Cr1_g@mail.gmail.com \
--to=joannelkoong@gmail.com \
--cc=alexei.starovoitov@gmail.com \
--cc=andrii.nakryiko@gmail.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=kernel-team@fb.com \
--cc=martin.lau@kernel.org \
--cc=martin.lau@linux.dev \
--cc=memxor@gmail.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).