From: "Björn Töpel" <bjorn.topel@gmail.com>
To: Jesper Dangaard Brouer <brouer@redhat.com>
Cc: Netdev <netdev@vger.kernel.org>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>, bpf <bpf@vger.kernel.org>,
David Miller <davem@davemloft.net>,
Jakub Kicinski <jakub.kicinski@netronome.com>,
Jesper Dangaard Brouer <hawk@kernel.org>,
John Fastabend <john.fastabend@gmail.com>,
"Karlsson, Magnus" <magnus.karlsson@intel.com>,
Jonathan Lemon <jonathan.lemon@gmail.com>,
Maciej Fijalkowski <maciejromanfijalkowski@gmail.com>
Subject: Re: [PATCH bpf-next 0/8] Simplify xdp_do_redirect_map()/xdp_do_flush_map() and XDP maps
Date: Wed, 18 Dec 2019 13:48:27 +0100 [thread overview]
Message-ID: <CAJ+HfNh_GyhCr+yLHG_0UU_=6bsBqbjDqoMNTxU40SvZ=MvJJw@mail.gmail.com> (raw)
In-Reply-To: <20191218134001.319349bc@carbon>
On Wed, 18 Dec 2019 at 13:40, Jesper Dangaard Brouer <brouer@redhat.com> wrote:
>
> On Wed, 18 Dec 2019 13:18:10 +0100
> Björn Töpel <bjorn.topel@gmail.com> wrote:
>
> > On Wed, 18 Dec 2019 at 13:04, Jesper Dangaard Brouer <brouer@redhat.com> wrote:
> > >
> > > On Wed, 18 Dec 2019 12:39:53 +0100
> > > Björn Töpel <bjorn.topel@gmail.com> wrote:
> > >
> > > > On Wed, 18 Dec 2019 at 12:11, Jesper Dangaard Brouer <brouer@redhat.com> wrote:
> > > > >
> > > > > On Wed, 18 Dec 2019 11:53:52 +0100
> > > > > Björn Töpel <bjorn.topel@gmail.com> wrote:
> > > > >
> > > > > > $ sudo ./xdp_redirect_cpu --dev enp134s0f0 --cpu 22 xdp_cpu_map0
> > > > > >
> > > > > > Running XDP/eBPF prog_name:xdp_cpu_map5_lb_hash_ip_pairs
> > > > > > XDP-cpumap CPU:to pps drop-pps extra-info
> > > > > > XDP-RX 20 7723038 0 0
> > > > > > XDP-RX total 7723038 0
> > > > > > cpumap_kthread total 0 0 0
> > > > > > redirect_err total 0 0
> > > > > > xdp_exception total 0 0
> > > > >
> > > > > Hmm... I'm missing some counters on the kthread side.
> > > > >
> > > >
> > > > Oh? Any ideas why? I just ran the upstream sample straight off.
> > >
> > > Looks like it happened in commit: bbaf6029c49c ("samples/bpf: Convert
> > > XDP samples to libbpf usage") (Cc Maciej).
> > >
> > > The old bpf_load.c will auto attach the tracepoints... for and libbpf
> > > you have to be explicit about it.
> > >
> > > Can I ask you to also run a test with --stress-mode for
> > > ./xdp_redirect_cpu, to flush out any potential RCU race-conditions
> > > (don't provide output, this is just a robustness test).
> > >
> >
> > Sure! Other than that, does the command line above make sense? I'm
> > blasting UDP packets to core 20, and the idea was to re-route them to
> > 22.
>
> Yes, and I love that you are using CPUMAP xdp_redirect_cpu as a test.
>
> Explaining what is doing on (so you can say if this is what you wanted
> to test):
>
I wanted to see whether one could receive (Rx + bpf_redirect_map)
more with the change. I figured out that at least bpf_redirect_map was
correctly executed, and that the numbers went up. :-P
> The "XDP-RX" number is the raw XDP redirect number, but the remote CPU,
> where the network stack is started, cannot operate at 7.7Mpps. Which the
> lacking tracepoint numbers should have shown. You still can observe
> results via nstat, e.g.:
>
> # nstat -n && sleep 1 && nstat
>
> On the remote CPU 22, the SKB will be constructed, and likely dropped
> due overloading network stack and due to not having an UDP listen port.
>
> I sometimes use:
> # iptables -t raw -I PREROUTING -p udp --dport 9 -j DROP
> To drop the UDP packets in a earlier and consistent stage.
>
> The CPUMAP have carefully been designed to avoid that a "producer" can
> be slowed down by memory operations done by the "consumer", this is
> mostly achieved via ptr_ring and careful bulking (cache-lines). As
> your driver i40e doesn't have 'page_pool', then you are not affected by
> the return channel.
>
> Funny test/details: i40e uses a refcnt recycle scheme, based off the
> size of the RX-ring, thus it is affected by a longer outstanding queue.
> The CPUMAP have an intermediate queue, that will be full in this
> overload setting. Try to increase or decrease the parameter --qsize
> (remember to place it as first argument), and see if this was the
> limiting factor for your XDP-RX number.
>
Thanks for the elaborate description!
(Maybe it's time for samples/bpf manpages? ;-))
Björn
next prev parent reply other threads:[~2019-12-18 12:48 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-18 10:53 [PATCH bpf-next 0/8] Simplify xdp_do_redirect_map()/xdp_do_flush_map() and XDP maps Björn Töpel
2019-12-18 10:53 ` [PATCH bpf-next 1/8] xdp: simplify devmap cleanup Björn Töpel
2019-12-18 11:14 ` Toke Høiland-Jørgensen
2019-12-18 10:53 ` [PATCH bpf-next 2/8] xdp: simplify cpumap cleanup Björn Töpel
2019-12-18 11:15 ` Toke Høiland-Jørgensen
2019-12-18 17:47 ` Jakub Kicinski
2019-12-18 17:48 ` Björn Töpel
2019-12-18 10:53 ` [PATCH bpf-next 3/8] xdp: fix graze->grace type-o in cpumap comments Björn Töpel
2019-12-18 11:18 ` Toke Høiland-Jørgensen
2019-12-18 10:53 ` [PATCH bpf-next 4/8] xsk: make xskmap flush_list common for all map instances Björn Töpel
2019-12-18 11:19 ` Toke Høiland-Jørgensen
2019-12-18 10:53 ` [PATCH bpf-next 5/8] xdp: make devmap " Björn Töpel
2019-12-18 11:19 ` Toke Høiland-Jørgensen
2019-12-18 10:53 ` [PATCH bpf-next 6/8] xdp: make cpumap " Björn Töpel
2019-12-18 11:19 ` Toke Høiland-Jørgensen
2019-12-18 10:53 ` [PATCH bpf-next 7/8] xdp: remove map_to_flush and map swap detection Björn Töpel
2019-12-18 11:20 ` Toke Høiland-Jørgensen
2019-12-18 13:03 ` Jesper Dangaard Brouer
2019-12-18 13:09 ` Björn Töpel
2019-12-18 10:54 ` [PATCH bpf-next 8/8] xdp: simplify __bpf_tx_xdp_map() Björn Töpel
2019-12-18 11:20 ` Toke Høiland-Jørgensen
2019-12-18 11:11 ` [PATCH bpf-next 0/8] Simplify xdp_do_redirect_map()/xdp_do_flush_map() and XDP maps Jesper Dangaard Brouer
2019-12-18 11:39 ` Björn Töpel
2019-12-18 12:03 ` Jesper Dangaard Brouer
2019-12-18 12:18 ` Björn Töpel
2019-12-18 12:32 ` Björn Töpel
2019-12-18 12:40 ` Jesper Dangaard Brouer
2019-12-18 12:48 ` Björn Töpel [this message]
2019-12-19 0:39 ` Andrii Nakryiko
2019-12-19 19:33 ` Jesper Dangaard Brouer
2019-12-19 20:08 ` Daniel Borkmann
2019-12-19 22:56 ` Andrii Nakryiko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAJ+HfNh_GyhCr+yLHG_0UU_=6bsBqbjDqoMNTxU40SvZ=MvJJw@mail.gmail.com' \
--to=bjorn.topel@gmail.com \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=brouer@redhat.com \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=hawk@kernel.org \
--cc=jakub.kicinski@netronome.com \
--cc=john.fastabend@gmail.com \
--cc=jonathan.lemon@gmail.com \
--cc=maciejromanfijalkowski@gmail.com \
--cc=magnus.karlsson@intel.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).