From: "Samudrala, Sridhar" <sridhar.samudrala@intel.com> To: Alexei Starovoitov <alexei.starovoitov@gmail.com> Cc: "Karlsson, Magnus" <magnus.karlsson@intel.com>, "Björn Töpel" <bjorn.topel@intel.com>, Netdev <netdev@vger.kernel.org>, "bpf@vger.kernel.org" <bpf@vger.kernel.org>, intel-wired-lan <intel-wired-lan@lists.osuosl.org>, "Fijalkowski, Maciej" <maciej.fijalkowski@intel.com>, "Herbert, Tom" <tom.herbert@intel.com> Subject: Re: FW: [PATCH bpf-next 2/4] xsk: allow AF_XDP sockets to receive packets directly from a queue Date: Wed, 9 Oct 2019 12:12:34 -0700 [thread overview] Message-ID: <2032d58c-916f-d26a-db14-bd5ba6ad92b9@intel.com> (raw) In-Reply-To: <CAADnVQ+qq6RLMjh5bB1ugXP5p7vYM2F1fLGFQ2pL=2vhCLiBdA@mail.gmail.com> On 10/9/2019 10:17 AM, Alexei Starovoitov wrote: > On Wed, Oct 9, 2019 at 9:53 AM Samudrala, Sridhar > <sridhar.samudrala@intel.com> wrote: >> >> >>>> + >>>> +u32 bpf_direct_xsk(const struct bpf_prog *prog, struct xdp_buff *xdp) >>>> +{ >>>> + struct xdp_sock *xsk; >>>> + >>>> + xsk = xdp_get_xsk_from_qid(xdp->rxq->dev, xdp->rxq->queue_index); >>>> + if (xsk) { >>>> + struct bpf_redirect_info *ri = >>>> + this_cpu_ptr(&bpf_redirect_info); >>>> + >>>> + ri->xsk = xsk; >>>> + return XDP_REDIRECT; >>>> + } >>>> + >>>> + return XDP_PASS; >>>> +} >>>> +EXPORT_SYMBOL(bpf_direct_xsk); >>> >>> So you're saying there is a: >>> """ >>> xdpsock rxdrop 1 core (both app and queue's irq pinned to the same core) >>> default : taskset -c 1 ./xdpsock -i enp66s0f0 -r -q 1 >>> direct-xsk :taskset -c 1 ./xdpsock -i enp66s0f0 -r -q 1 6.1x improvement in drop rate """ >>> >>> 6.1x gain running above C code vs exactly equivalent BPF code? >>> How is that possible? >> >> It seems to be due to the overhead of __bpf_prog_run on older processors >> (Ivybridge). The overhead is smaller on newer processors, but even on >> skylake i see around 1.5x improvement. >> >> perf report with default xdpsock >> ================================ >> Samples: 2K of event 'cycles:ppp', Event count (approx.): 8437658090 >> Overhead Command Shared Object Symbol >> 34.57% xdpsock xdpsock [.] main >> 17.19% ksoftirqd/1 [kernel.vmlinux] [k] ___bpf_prog_run >> 13.12% xdpsock [kernel.vmlinux] [k] ___bpf_prog_run > > That must be a bad joke. > The whole patch set is based on comparing native code to interpreter?! > It's pretty awesome that interpreter is only 1.5x slower than native x86. > Just turn the JIT on. Thanks Alexei for pointing out that i didn't have JIT on. When i turn it on, the performance improvement is a more modest 1.5x with rxdrop and 1.2x with l2fwd. > > Obvious Nack to the patch set. > Will update the patchset with the right performance data and address feedback from Bjorn. Hope you are not totally against direct XDP approach as it does provide value when an AF_XDP socket is bound to a queue and a HW filter can direct packets targeted for that queue.
WARNING: multiple messages have this Message-ID (diff)
From: Samudrala, Sridhar <sridhar.samudrala@intel.com> To: intel-wired-lan@osuosl.org Subject: [Intel-wired-lan] FW: [PATCH bpf-next 2/4] xsk: allow AF_XDP sockets to receive packets directly from a queue Date: Wed, 9 Oct 2019 12:12:34 -0700 [thread overview] Message-ID: <2032d58c-916f-d26a-db14-bd5ba6ad92b9@intel.com> (raw) In-Reply-To: <CAADnVQ+qq6RLMjh5bB1ugXP5p7vYM2F1fLGFQ2pL=2vhCLiBdA@mail.gmail.com> On 10/9/2019 10:17 AM, Alexei Starovoitov wrote: > On Wed, Oct 9, 2019 at 9:53 AM Samudrala, Sridhar > <sridhar.samudrala@intel.com> wrote: >> >> >>>> + >>>> +u32 bpf_direct_xsk(const struct bpf_prog *prog, struct xdp_buff *xdp) >>>> +{ >>>> + struct xdp_sock *xsk; >>>> + >>>> + xsk = xdp_get_xsk_from_qid(xdp->rxq->dev, xdp->rxq->queue_index); >>>> + if (xsk) { >>>> + struct bpf_redirect_info *ri = >>>> + this_cpu_ptr(&bpf_redirect_info); >>>> + >>>> + ri->xsk = xsk; >>>> + return XDP_REDIRECT; >>>> + } >>>> + >>>> + return XDP_PASS; >>>> +} >>>> +EXPORT_SYMBOL(bpf_direct_xsk); >>> >>> So you're saying there is a: >>> """ >>> xdpsock rxdrop 1 core (both app and queue's irq pinned to the same core) >>> default : taskset -c 1 ./xdpsock -i enp66s0f0 -r -q 1 >>> direct-xsk :taskset -c 1 ./xdpsock -i enp66s0f0 -r -q 1 6.1x improvement in drop rate """ >>> >>> 6.1x gain running above C code vs exactly equivalent BPF code? >>> How is that possible? >> >> It seems to be due to the overhead of __bpf_prog_run on older processors >> (Ivybridge). The overhead is smaller on newer processors, but even on >> skylake i see around 1.5x improvement. >> >> perf report with default xdpsock >> ================================ >> Samples: 2K of event 'cycles:ppp', Event count (approx.): 8437658090 >> Overhead Command Shared Object Symbol >> 34.57% xdpsock xdpsock [.] main >> 17.19% ksoftirqd/1 [kernel.vmlinux] [k] ___bpf_prog_run >> 13.12% xdpsock [kernel.vmlinux] [k] ___bpf_prog_run > > That must be a bad joke. > The whole patch set is based on comparing native code to interpreter?! > It's pretty awesome that interpreter is only 1.5x slower than native x86. > Just turn the JIT on. Thanks Alexei for pointing out that i didn't have JIT on. When i turn it on, the performance improvement is a more modest 1.5x with rxdrop and 1.2x with l2fwd. > > Obvious Nack to the patch set. > Will update the patchset with the right performance data and address feedback from Bjorn. Hope you are not totally against direct XDP approach as it does provide value when an AF_XDP socket is bound to a queue and a HW filter can direct packets targeted for that queue.
next prev parent reply other threads:[~2019-10-09 19:12 UTC|newest] Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-10-08 6:16 [PATCH bpf-next 0/4] Enable direct receive on AF_XDP sockets Sridhar Samudrala 2019-10-08 6:16 ` [Intel-wired-lan] " Sridhar Samudrala 2019-10-08 6:16 ` [PATCH bpf-next 1/4] bpf: introduce bpf_get_prog_id and bpf_set_prog_id helper functions Sridhar Samudrala 2019-10-08 6:16 ` [Intel-wired-lan] " Sridhar Samudrala 2019-10-08 6:16 ` [PATCH bpf-next 2/4] xsk: allow AF_XDP sockets to receive packets directly from a queue Sridhar Samudrala 2019-10-08 6:16 ` [Intel-wired-lan] " Sridhar Samudrala 2019-10-08 6:58 ` Toke Høiland-Jørgensen 2019-10-08 6:58 ` [Intel-wired-lan] " Toke =?unknown-8bit?q?H=C3=B8iland-J=C3=B8rgensen?= 2019-10-08 8:47 ` Björn Töpel 2019-10-08 8:47 ` [Intel-wired-lan] " =?unknown-8bit?q?Bj=C3=B6rn_T=C3=B6pel?= 2019-10-08 8:48 ` Björn Töpel 2019-10-08 8:48 ` [Intel-wired-lan] " =?unknown-8bit?q?Bj=C3=B6rn_T=C3=B6pel?= 2019-10-08 9:04 ` Toke Høiland-Jørgensen 2019-10-08 9:04 ` [Intel-wired-lan] " Toke =?unknown-8bit?q?H=C3=B8iland-J=C3=B8rgensen?= 2019-10-08 8:05 ` Björn Töpel 2019-10-08 8:05 ` [Intel-wired-lan] " =?unknown-8bit?q?Bj=C3=B6rn_T=C3=B6pel?= 2019-10-09 16:32 ` Samudrala, Sridhar 2019-10-09 16:32 ` [Intel-wired-lan] " Samudrala, Sridhar 2019-10-09 1:20 ` Alexei Starovoitov 2019-10-09 1:20 ` [Intel-wired-lan] " Alexei Starovoitov [not found] ` <3ED8E928C4210A4289A677D2FEB48235140134CE@fmsmsx111.amr.corp.intel.com> 2019-10-09 16:53 ` FW: " Samudrala, Sridhar 2019-10-09 16:53 ` [Intel-wired-lan] " Samudrala, Sridhar 2019-10-09 17:17 ` Alexei Starovoitov 2019-10-09 17:17 ` [Intel-wired-lan] " Alexei Starovoitov 2019-10-09 19:12 ` Samudrala, Sridhar [this message] 2019-10-09 19:12 ` Samudrala, Sridhar 2019-10-10 1:06 ` Alexei Starovoitov 2019-10-10 1:06 ` [Intel-wired-lan] " Alexei Starovoitov 2019-10-18 18:40 ` Samudrala, Sridhar 2019-10-18 18:40 ` [Intel-wired-lan] " Samudrala, Sridhar 2019-10-18 19:22 ` Toke Høiland-Jørgensen 2019-10-18 19:22 ` [Intel-wired-lan] " Toke =?unknown-8bit?q?H=C3=B8iland-J=C3=B8rgensen?= 2019-10-19 0:14 ` Alexei Starovoitov 2019-10-19 0:14 ` [Intel-wired-lan] " Alexei Starovoitov 2019-10-19 0:45 ` Samudrala, Sridhar 2019-10-19 0:45 ` [Intel-wired-lan] " Samudrala, Sridhar 2019-10-19 2:25 ` Alexei Starovoitov 2019-10-19 2:25 ` [Intel-wired-lan] " Alexei Starovoitov 2019-10-20 10:14 ` Toke Høiland-Jørgensen 2019-10-20 10:14 ` [Intel-wired-lan] " Toke =?unknown-8bit?q?H=C3=B8iland-J=C3=B8rgensen?= 2019-10-20 17:12 ` Björn Töpel 2019-10-20 17:12 ` =?unknown-8bit?q?Bj=C3=B6rn_T=C3=B6pel?= 2019-10-21 20:10 ` Samudrala, Sridhar 2019-10-21 20:10 ` Samudrala, Sridhar 2019-10-21 22:34 ` Alexei Starovoitov 2019-10-21 22:34 ` Alexei Starovoitov 2019-10-22 19:06 ` Samudrala, Sridhar 2019-10-22 19:06 ` Samudrala, Sridhar 2019-10-23 17:42 ` Alexei Starovoitov 2019-10-23 17:42 ` Alexei Starovoitov 2019-10-24 18:12 ` Samudrala, Sridhar 2019-10-24 18:12 ` Samudrala, Sridhar 2019-10-25 7:42 ` Björn Töpel 2019-10-25 7:42 ` =?unknown-8bit?q?Bj=C3=B6rn_T=C3=B6pel?= 2019-10-31 22:38 ` Samudrala, Sridhar 2019-10-31 22:38 ` Samudrala, Sridhar 2019-10-31 23:15 ` Alexei Starovoitov 2019-10-31 23:15 ` Alexei Starovoitov 2019-11-01 0:21 ` Jakub Kicinski 2019-11-01 0:21 ` Jakub Kicinski 2019-11-01 18:31 ` Samudrala, Sridhar 2019-11-01 18:31 ` Samudrala, Sridhar 2019-11-04 2:08 ` dan 2019-11-04 2:08 ` dan 2019-10-25 9:07 ` Björn Töpel 2019-10-25 9:07 ` =?unknown-8bit?q?Bj=C3=B6rn_T=C3=B6pel?= 2019-10-08 6:16 ` [PATCH bpf-next 3/4] libbpf: handle AF_XDP sockets created with XDP_DIRECT bind flag Sridhar Samudrala 2019-10-08 6:16 ` [Intel-wired-lan] " Sridhar Samudrala 2019-10-08 8:05 ` Björn Töpel 2019-10-08 8:05 ` [Intel-wired-lan] " =?unknown-8bit?q?Bj=C3=B6rn_T=C3=B6pel?= 2019-10-08 6:16 ` [PATCH bpf-next 4/4] xdpsock: add an option to create AF_XDP sockets in XDP_DIRECT mode Sridhar Samudrala 2019-10-08 6:16 ` [Intel-wired-lan] " Sridhar Samudrala 2019-10-08 8:05 ` Björn Töpel 2019-10-08 8:05 ` [Intel-wired-lan] " =?unknown-8bit?q?Bj=C3=B6rn_T=C3=B6pel?= 2019-10-08 8:05 ` [PATCH bpf-next 0/4] Enable direct receive on AF_XDP sockets Björn Töpel 2019-10-08 8:05 ` [Intel-wired-lan] " =?unknown-8bit?q?Bj=C3=B6rn_T=C3=B6pel?= 2019-10-09 16:19 ` Samudrala, Sridhar 2019-10-09 16:19 ` [Intel-wired-lan] " Samudrala, Sridhar 2019-10-09 0:49 ` Jakub Kicinski 2019-10-09 0:49 ` [Intel-wired-lan] " Jakub Kicinski 2019-10-09 6:29 ` Samudrala, Sridhar 2019-10-09 6:29 ` [Intel-wired-lan] " Samudrala, Sridhar 2019-10-09 16:53 ` Jakub Kicinski 2019-10-09 16:53 ` [Intel-wired-lan] " Jakub Kicinski
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=2032d58c-916f-d26a-db14-bd5ba6ad92b9@intel.com \ --to=sridhar.samudrala@intel.com \ --cc=alexei.starovoitov@gmail.com \ --cc=bjorn.topel@intel.com \ --cc=bpf@vger.kernel.org \ --cc=intel-wired-lan@lists.osuosl.org \ --cc=maciej.fijalkowski@intel.com \ --cc=magnus.karlsson@intel.com \ --cc=netdev@vger.kernel.org \ --cc=tom.herbert@intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.