BPF Archive on lore.kernel.org
 help / color / Atom feed
From: "Toke Høiland-Jørgensen" <toke@redhat.com>
To: Hangbin Liu <liuhangbin@gmail.com>
Cc: bpf@vger.kernel.org, netdev@vger.kernel.org,
	Jiri Benc <jbenc@redhat.com>,
	Jesper Dangaard Brouer <brouer@redhat.com>,
	Eelco Chaudron <echaudro@redhat.com>,
	ast@kernel.org, Daniel Borkmann <daniel@iogearbox.net>,
	Lorenzo Bianconi <lorenzo.bianconi@redhat.com>
Subject: Re: [PATCHv4 bpf-next 0/2] xdp: add dev map multicast support
Date: Thu, 04 Jun 2020 14:37:23 +0200
Message-ID: <87bllzj9bw.fsf@toke.dk> (raw)
In-Reply-To: <20200604121212.GM102436@dhcp-12-153.nay.redhat.com>

Hangbin Liu <liuhangbin@gmail.com> writes:

> On Thu, Jun 04, 2020 at 11:44:24AM +0200, Toke Høiland-Jørgensen wrote:
>> Hangbin Liu <liuhangbin@gmail.com> writes:
>> > Here is the test topology, which looks like
>> >
>> >  Host A    |     Host B        |        Host C
>> >  eth0      +    eth0 - eth1    +        eth0
>> >
>> > I did pktgen sending on Host A, forwarding on Host B.
>> > Host B is a Dell PowerEdge R730 (128G memory, Intel(R) Xeon(R) CPU E5-2690 v3)
>> > eth0, eth1 is an onboard i40e 10G driver
>> >
>> > Test 1: add eth0, eth1 to br0 and test bridge forwarding
>> > Test 2: Test xdp_redirect_map(), eth0 is ingress, eth1 is egress
>> > Test 3: Test xdp_redirect_map_multi(), eth0 is ingress, eth1 is egress
>> 
>> Right, that all seems reasonable, but that machine is comparable to
>> my test machine, so you should be getting way more than 2.75 MPPS on a
>> regular redirect test. Are you bottlenecked on pktgen or something?
>
> Yes, I found the pktgen is bottleneck. I only use 1 thread.
> By using the cmd you gave to me
> ./pktgen_sample03_burst_single_flow.sh  -i eno1 -d 192.168.200.1 -m f8:bc:12:14:11:20 -t 4  -s 64
>
> Now I could get higher speed.
>
>> 
>> Could you please try running Jesper's ethtool stats poller:
>> https://github.com/netoptimizer/network-testing/blob/master/bin/ethtool_stats.pl
>
> Nice tool.
>
>> > I though you want me also test with bridge forwarding. Am I missing something?
>> 
>> Yes, but what does this mean:
>> > (I use sample/bpf/xdp1 to count the PPS, so there are two modes data):
>> 
>> or rather, why are there two numbers? :)
>
> Just as it said, to test bridge forwarding speed. I use the xdp tool
> sample/bpf/xdp1 to count the PPS. But there are two modes when attach xdp
> to eth0, general and driver mode. So there are 2 number..
>
> Now I use the ethtool_stats.pl to count forwarding speed and here is the result:
>
> With kernel 5.7(ingress i40e, egress i40e)
> XDP:
> bridge: 1.8M PPS
> xdp_redirect_map:
>   generic mode: 1.9M PPS
>   driver mode: 10.4M PPS

Ah, now we're getting somewhere! :)

> Kernel 5.7 + my patch(ingress i40e, egress i40e)
> bridge: 1.8M
> xdp_redirect_map:
>   generic mode: 1.86M PPS
>   driver mode: 10.17M PPS

Right, so this corresponds to a ~2ns overhead (10**9/10400000 -
10**9/10170000). This is not too far from being in the noise, I suppose;
is the difference consistent?

> xdp_redirect_map_multi:
>   generic mode: 1.53M PPS
>   driver mode: 7.22M PPS
>
> Kernel 5.7 + my patch(ingress i40e, egress veth)
> xdp_redirect_map:
>   generic mode: 1.38M PPS
>   driver mode: 4.15M PPS
> xdp_redirect_map_multi:
>   generic mode: 1.13M PPS
>   driver mode: 3.55M PPS
>
> Kernel 5.7 + my patch(ingress i40e, egress i40e + veth)
> xdp_redirect_map_multi:
>   generic mode: 1.13M PPS
>   driver mode: 3.47M PPS
>
> I added a group that with i40e ingress and veth egress, which shows
> a significant drop on the speed. It looks like veth driver is a bottleneck,
> but I don't have more i40e NICs on the test bed...

I suspect this may be because veth ends up creating an SKB for each
packet after receiving the frame on the peer device (even though it's
immediately dropped). Could you please try adding an XDP program that
drops the packets on the veth peer of your target, and see if that
helps?

-Toke


  reply index

Thread overview: 94+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-15  8:54 [RFC PATCH " Hangbin Liu
2020-04-15  8:54 ` [RFC PATCH bpf-next 1/2] " Hangbin Liu
2020-04-20  9:52   ` Hangbin Liu
2020-04-15  8:54 ` [RFC PATCH bpf-next 2/2] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-04-24  8:56 ` [RFC PATCHv2 bpf-next 0/2] xdp: add dev map multicast support Hangbin Liu
2020-04-24  8:56   ` [RFC PATCHv2 bpf-next 1/2] xdp: add a new helper for " Hangbin Liu
2020-04-24 14:19     ` Lorenzo Bianconi
2020-04-28 11:09       ` Eelco Chaudron
2020-05-06  9:35       ` Hangbin Liu
2020-04-24 14:34     ` Toke Høiland-Jørgensen
2020-05-06  9:14       ` Hangbin Liu
2020-05-06 10:00         ` Toke Høiland-Jørgensen
2020-05-08  8:53           ` Hangbin Liu
2020-05-08 14:58             ` Toke Høiland-Jørgensen
2020-05-18  8:45       ` Hangbin Liu
2020-05-19 10:15         ` Jesper Dangaard Brouer
2020-05-20  1:24           ` Hangbin Liu
2020-04-24  8:56   ` [RFC PATCHv2 bpf-next 2/2] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-04-24 14:21     ` Lorenzo Bianconi
2020-05-23  6:05 ` [PATCHv3 bpf-next 0/2] xdp: add dev map multicast support Hangbin Liu
2020-05-23  6:05   ` [PATCHv3 bpf-next 1/2] xdp: add a new helper for " Hangbin Liu
2020-05-26  7:34     ` kbuild test robot
2020-05-23  6:05   ` [PATCHv3 bpf-next 2/2] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-05-26 14:05 ` [PATCHv4 bpf-next 0/2] xdp: add dev map multicast support Hangbin Liu
2020-05-26 14:05   ` [PATCHv4 bpf-next 1/2] xdp: add a new helper for " Hangbin Liu
2020-05-27 10:29     ` Toke Høiland-Jørgensen
2020-06-10 10:18     ` Jesper Dangaard Brouer
2020-06-12  8:54       ` Hangbin Liu
2020-06-16  8:55         ` Jesper Dangaard Brouer
2020-06-16 10:11           ` Hangbin Liu
2020-06-16 14:38             ` Jesper Dangaard Brouer
2020-06-10 10:21     ` Jesper Dangaard Brouer
2020-06-10 10:29       ` Toke Høiland-Jørgensen
2020-06-16  9:04         ` Jesper Dangaard Brouer
2020-05-26 14:05   ` [PATCHv4 bpf-next 2/2] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-05-27 10:21   ` [PATCHv4 bpf-next 0/2] xdp: add dev map multicast support Toke Høiland-Jørgensen
2020-05-27 10:32     ` Eelco Chaudron
2020-05-27 12:38     ` Hangbin Liu
2020-05-27 15:04       ` Toke Høiland-Jørgensen
2020-06-16  9:09         ` Jesper Dangaard Brouer
2020-06-16  9:47           ` Hangbin Liu
2020-06-03  2:40     ` Hangbin Liu
2020-06-03 11:05       ` Toke Høiland-Jørgensen
2020-06-04  4:09         ` Hangbin Liu
2020-06-04  9:44           ` Toke Høiland-Jørgensen
2020-06-04 12:12             ` Hangbin Liu
2020-06-04 12:37               ` Toke Høiland-Jørgensen [this message]
2020-06-04 14:41                 ` Hangbin Liu
2020-06-04 16:02                   ` Toke Høiland-Jørgensen
2020-06-05  6:26                     ` Hangbin Liu
2020-06-08 15:32                       ` Toke Høiland-Jørgensen
2020-06-09  3:03                         ` Hangbin Liu
2020-06-09 20:31                           ` Toke Høiland-Jørgensen
2020-06-10  2:35                             ` Hangbin Liu
2020-06-10 10:03                               ` Jesper Dangaard Brouer
2020-07-01  4:19   ` [PATCHv5 bpf-next 0/3] xdp: add a new helper for " Hangbin Liu
2020-07-01  4:19     ` [PATCHv5 bpf-next 1/3] " Hangbin Liu
2020-07-01  5:09       ` Andrii Nakryiko
2020-07-01  6:51         ` Hangbin Liu
2020-07-01 18:33       ` kernel test robot
2020-07-01  4:19     ` [PATCHv5 bpf-next 2/3] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-07-01  4:19     ` [PATCHv5 bpf-next 3/3] selftests/bpf: add xdp_redirect_multi test Hangbin Liu
2020-07-09  1:30     ` [PATCHv6 bpf-next 0/3] xdp: add a new helper for dev map multicast support Hangbin Liu
2020-07-09  1:30       ` [PATCHv6 bpf-next 1/3] " Hangbin Liu
2020-07-09 16:33         ` David Ahern
2020-07-10  6:55           ` Hangbin Liu
2020-07-10 13:46             ` David Ahern
2020-07-11  0:26               ` Hangbin Liu
2020-07-11 16:09                 ` David Ahern
2020-07-09  1:30       ` [PATCHv6 bpf-next 2/3] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-07-09 22:40         ` Daniel Borkmann
2020-07-10  6:41           ` Hangbin Liu
2020-07-10 14:32             ` Daniel Borkmann
2020-07-11  0:21               ` Hangbin Liu
2020-07-09  1:30       ` [PATCHv6 bpf-next 3/3] selftests/bpf: add xdp_redirect_multi test Hangbin Liu
2020-07-09 22:37       ` [PATCHv6 bpf-next 0/3] xdp: add a new helper for dev map multicast support Daniel Borkmann
2020-07-10  7:36         ` Hangbin Liu
2020-07-10 15:02           ` Daniel Borkmann
2020-07-10 16:52             ` David Ahern
2020-07-14  6:32       ` [PATCHv7 " Hangbin Liu
2020-07-14  6:32         ` [PATCHv7 bpf-next 1/3] " Hangbin Liu
2020-07-14 21:52           ` Toke Høiland-Jørgensen
2020-07-15 12:25             ` Hangbin Liu
2020-07-14  6:32         ` [PATCHv7 bpf-next 2/3] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-07-14  6:32         ` [PATCHv7 bpf-next 3/3] selftests/bpf: add xdp_redirect_multi test Hangbin Liu
2020-07-14 12:29         ` [PATCHv7 bpf-next 0/3] xdp: add a new helper for dev map multicast support Toke Høiland-Jørgensen
2020-07-14 17:12           ` David Ahern
2020-07-14 21:53             ` Toke Høiland-Jørgensen
2020-07-15 12:31               ` Hangbin Liu
2020-07-15  3:45             ` Hangbin Liu
2020-07-15 13:08         ` [PATCHv8 " Hangbin Liu
2020-07-15 13:08           ` [PATCHv8 bpf-next 1/3] " Hangbin Liu
2020-07-15 13:08           ` [PATCHv8 bpf-next 2/3] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-07-15 13:08           ` [PATCHv8 bpf-next 3/3] selftests/bpf: add xdp_redirect_multi test Hangbin Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87bllzj9bw.fsf@toke.dk \
    --to=toke@redhat.com \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=brouer@redhat.com \
    --cc=daniel@iogearbox.net \
    --cc=echaudro@redhat.com \
    --cc=jbenc@redhat.com \
    --cc=liuhangbin@gmail.com \
    --cc=lorenzo.bianconi@redhat.com \
    --cc=netdev@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

BPF Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/bpf/0 bpf/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 bpf bpf/ https://lore.kernel.org/bpf \
		bpf@vger.kernel.org
	public-inbox-index bpf

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.bpf


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git