From: Martin KaFai Lau <kafai@fb.com>
To: Stanislav Fomichev <sdf@google.com>,
Pavel Begunkov <asml.silence@gmail.com>
Cc: <netdev@vger.kernel.org>, <bpf@vger.kernel.org>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Andrii Nakryiko <andrii@kernel.org>,
Song Liu <songliubraving@fb.com>, <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v3] cgroup/bpf: fast path skb BPF filtering
Date: Thu, 16 Dec 2021 10:14:49 -0800 [thread overview]
Message-ID: <20211216181449.p2izqxgzmfpknbsw@kafai-mbp.dhcp.thefacebook.com> (raw)
In-Reply-To: <7ca623df-73ed-9191-bec7-a4728f2f95e6@gmail.com>
On Thu, Dec 16, 2021 at 01:21:26PM +0000, Pavel Begunkov wrote:
> On 12/15/21 22:07, Stanislav Fomichev wrote:
> > On Wed, Dec 15, 2021 at 11:55 AM Pavel Begunkov <asml.silence@gmail.com> wrote:
> > >
> > > On 12/15/21 19:15, Stanislav Fomichev wrote:
> > > > On Wed, Dec 15, 2021 at 10:54 AM Pavel Begunkov <asml.silence@gmail.com> wrote:
> > > > >
> > > > > On 12/15/21 18:24, sdf@google.com wrote:
> [...]
> > > > > > I can probably do more experiments on my side once your patch is
> > > > > > accepted. I'm mostly concerned with getsockopt(TCP_ZEROCOPY_RECEIVE).
> > > > > > If you claim there is visible overhead for a direct call then there
> > > > > > should be visible benefit to using CGROUP_BPF_TYPE_ENABLED there as
> > > > > > well.
> > > > >
> > > > > Interesting, sounds getsockopt might be performance sensitive to
> > > > > someone.
> > > > >
> > > > > FWIW, I forgot to mention that for testing tx I'm using io_uring
> > > > > (for both zc and not) with good submission batching.
> > > >
> > > > Yeah, last time I saw 2-3% as well, but it was due to kmalloc, see
> > > > more details in 9cacf81f8161, it was pretty visible under perf.
> > > > That's why I'm a bit skeptical of your claims of direct calls being
> > > > somehow visible in these 2-3% (even skb pulls/pushes are not 2-3%?).
> > >
> > > migrate_disable/enable together were taking somewhat in-between
> > > 1% and 1.5% in profiling, don't remember the exact number. The rest
> > > should be from rcu_read_lock/unlock() in BPF_PROG_RUN_ARRAY_CG_FLAGS()
> > > and other extra bits on the way.
> >
> > You probably have a preemptiple kernel and preemptible rcu which most
> > likely explains why you see the overhead and I won't (non-preemptible
> > kernel in our env, rcu_read_lock is essentially a nop, just a compiler
> > barrier).
>
> Right. For reference tried out non-preemptible, perf shows the function
> taking 0.8% with a NIC and 1.2% with a dummy netdev.
>
>
> > > I'm skeptical I'll be able to measure inlining one function,
> > > variability between boots/runs is usually greater and would hide it.
> >
> > Right, that's why I suggested to mirror what we do in set/getsockopt
> > instead of the new extra CGROUP_BPF_TYPE_ENABLED. But I'll leave it up
> > to you, Martin and the rest.
I also suggested to try to stay with one way for fullsock context in v2
but it is for code readability reason.
How about calling CGROUP_BPF_TYPE_ENABLED() just next to cgroup_bpf_enabled()
in BPF_CGROUP_RUN_PROG_*SOCKOPT_*() instead ?
It is because both cgroup_bpf_enabled() and CGROUP_BPF_TYPE_ENABLED()
want to check if there is bpf to run before proceeding everything else
and then I don't need to jump to the non-inline function itself to see
if there is other prog array empty check.
Stan, do you have concern on an extra inlined sock_cgroup_ptr()
when there is bpf prog to run for set/getsockopt()? I think
it should be mostly noise from looking at
__cgroup_bpf_run_filter_*sockopt()?
next prev parent reply other threads:[~2021-12-16 18:15 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-15 14:49 [PATCH v3] cgroup/bpf: fast path skb BPF filtering Pavel Begunkov
2021-12-15 16:40 ` Jakub Kicinski
2021-12-15 17:38 ` Pavel Begunkov
2021-12-15 16:51 ` sdf
2021-12-15 17:18 ` Pavel Begunkov
2021-12-15 17:33 ` sdf
2021-12-15 17:53 ` Pavel Begunkov
2021-12-15 18:24 ` sdf
2021-12-15 18:54 ` Pavel Begunkov
2021-12-15 19:15 ` Stanislav Fomichev
2021-12-15 19:55 ` Pavel Begunkov
2021-12-15 22:07 ` Stanislav Fomichev
2021-12-16 13:21 ` Pavel Begunkov
2021-12-16 18:14 ` Martin KaFai Lau [this message]
2021-12-16 18:24 ` Stanislav Fomichev
2022-01-24 15:46 ` Pavel Begunkov
2022-01-24 18:25 ` Stanislav Fomichev
2022-01-25 18:54 ` Pavel Begunkov
2022-01-25 21:27 ` Stanislav Fomichev
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211216181449.p2izqxgzmfpknbsw@kafai-mbp.dhcp.thefacebook.com \
--to=kafai@fb.com \
--cc=andrii@kernel.org \
--cc=asml.silence@gmail.com \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=sdf@google.com \
--cc=songliubraving@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).