BPF Archive on lore.kernel.org
 help / color / Atom feed
From: Jesper Dangaard Brouer <brouer@redhat.com>
To: David Ahern <dsahern@gmail.com>
Cc: "Toke Høiland-Jørgensen" <toke@redhat.com>,
	"Alexei Starovoitov" <alexei.starovoitov@gmail.com>,
	"Hangbin Liu" <liuhangbin@gmail.com>, bpf <bpf@vger.kernel.org>,
	"Network Development" <netdev@vger.kernel.org>,
	"Jiri Benc" <jbenc@redhat.com>,
	"Eelco Chaudron" <echaudro@redhat.com>,
	"Alexei Starovoitov" <ast@kernel.org>,
	"Daniel Borkmann" <daniel@iogearbox.net>,
	"Lorenzo Bianconi" <lorenzo.bianconi@redhat.com>,
	"Andrii Nakryiko" <andrii.nakryiko@gmail.com>,
	brouer@redhat.com
Subject: Re: [PATCHv11 bpf-next 2/5] xdp: add a new helper for dev map multicast support
Date: Fri, 11 Sep 2020 09:58:20 +0200
Message-ID: <20200911095820.304d9877@carbon> (raw)
In-Reply-To: <47566856-75e2-8f2b-4347-f03a7cb5493b@gmail.com>

On Thu, 10 Sep 2020 12:35:33 -0600
David Ahern <dsahern@gmail.com> wrote:

> On 9/10/20 11:50 AM, Jesper Dangaard Brouer wrote:
> > Maybe we should change the devmap-prog approach, and run this on the
> > xdp_frame's (in bq_xmit_all() to be precise) .  Hangbin's patchset
> > clearly shows that we need this "layer" between running the xdp_prog and
> > the devmap-prog.   
> 
> I would prefer to leave it in dev_map_enqueue.
> 
> The main premise at the moment is that the program attached to the
> DEVMAP entry is an ACL specific to that dev. If the program is going to
> drop the packet, then no sense queueing it.
> 
> I also expect a follow on feature will be useful to allow the DEVMAP
> program to do another REDIRECT (e.g., potentially after modifying). It
> is not handled at the moment as it needs thought - e.g., limiting the
> number of iterative redirects. If such a feature does happen, then no
> sense queueing it to the current device.

It makes a lot of sense to do queuing before redirecting again.  The
(hidden) bulking we do at XDP redirect is the primary reason for the
performance boost. We all remember performance difference between
non-map version of redirect (which Toke fixed via always having the
bulking available in net_device->xdp_bulkq).

In a simple micro-benchmark I bet it will look better running the
devmap-prog right after the xdp_prog (which is what we have today). But
I claim this is the wrong approach, as soon as (1) traffic is more
intermixed, and (2) devmap-prog gets bigger and becomes more specific
to the egress-device (e.g. BPF update constants per egress-device).
When this happens performance suffers, as I-cache and data-access to
each egress-device gets pushed out of cache. (Hint VPP/fd.io approach)

Queuing xdp_frames up for your devmap-prog makes sense, as these share
common properties.  With intermix traffic the first xdp_prog will sort
packets into egress-devices, and then the devmap-prog can operate on
these.  The best illustration[1] of this sorting I saw in a Netflix
blogpost[2] about FreeBSD, section "RSS Assisted LRO" (not directly
related, but illustration was good).


[1] https://miro.medium.com/max/700/1%2alTGL1_D6hTMEMa7EDV8yZA.png
[2] https://netflixtechblog.com/serving-100-gbps-from-an-open-connect-appliance-cdb51dda3b99
-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer


  reply index

Thread overview: 127+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-15  8:54 [RFC PATCH bpf-next 0/2] xdp: add " Hangbin Liu
2020-04-15  8:54 ` [RFC PATCH bpf-next 1/2] " Hangbin Liu
2020-04-20  9:52   ` Hangbin Liu
2020-04-15  8:54 ` [RFC PATCH bpf-next 2/2] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-04-24  8:56 ` [RFC PATCHv2 bpf-next 0/2] xdp: add dev map multicast support Hangbin Liu
2020-04-24  8:56   ` [RFC PATCHv2 bpf-next 1/2] xdp: add a new helper for " Hangbin Liu
2020-04-24 14:19     ` Lorenzo Bianconi
2020-04-28 11:09       ` Eelco Chaudron
2020-05-06  9:35       ` Hangbin Liu
2020-04-24 14:34     ` Toke Høiland-Jørgensen
2020-05-06  9:14       ` Hangbin Liu
2020-05-06 10:00         ` Toke Høiland-Jørgensen
2020-05-08  8:53           ` Hangbin Liu
2020-05-08 14:58             ` Toke Høiland-Jørgensen
2020-05-18  8:45       ` Hangbin Liu
2020-05-19 10:15         ` Jesper Dangaard Brouer
2020-05-20  1:24           ` Hangbin Liu
2020-04-24  8:56   ` [RFC PATCHv2 bpf-next 2/2] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-04-24 14:21     ` Lorenzo Bianconi
2020-05-23  6:05 ` [PATCHv3 bpf-next 0/2] xdp: add dev map multicast support Hangbin Liu
2020-05-23  6:05   ` [PATCHv3 bpf-next 1/2] xdp: add a new helper for " Hangbin Liu
2020-05-26  7:34     ` kbuild test robot
2020-05-23  6:05   ` [PATCHv3 bpf-next 2/2] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-05-26 14:05 ` [PATCHv4 bpf-next 0/2] xdp: add dev map multicast support Hangbin Liu
2020-05-26 14:05   ` [PATCHv4 bpf-next 1/2] xdp: add a new helper for " Hangbin Liu
2020-05-27 10:29     ` Toke Høiland-Jørgensen
2020-06-10 10:18     ` Jesper Dangaard Brouer
2020-06-12  8:54       ` Hangbin Liu
2020-06-16  8:55         ` Jesper Dangaard Brouer
2020-06-16 10:11           ` Hangbin Liu
2020-06-16 14:38             ` Jesper Dangaard Brouer
2020-06-10 10:21     ` Jesper Dangaard Brouer
2020-06-10 10:29       ` Toke Høiland-Jørgensen
2020-06-16  9:04         ` Jesper Dangaard Brouer
2020-05-26 14:05   ` [PATCHv4 bpf-next 2/2] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-05-27 10:21   ` [PATCHv4 bpf-next 0/2] xdp: add dev map multicast support Toke Høiland-Jørgensen
2020-05-27 10:32     ` Eelco Chaudron
2020-05-27 12:38     ` Hangbin Liu
2020-05-27 15:04       ` Toke Høiland-Jørgensen
2020-06-16  9:09         ` Jesper Dangaard Brouer
2020-06-16  9:47           ` Hangbin Liu
2020-06-03  2:40     ` Hangbin Liu
2020-06-03 11:05       ` Toke Høiland-Jørgensen
2020-06-04  4:09         ` Hangbin Liu
2020-06-04  9:44           ` Toke Høiland-Jørgensen
2020-06-04 12:12             ` Hangbin Liu
2020-06-04 12:37               ` Toke Høiland-Jørgensen
2020-06-04 14:41                 ` Hangbin Liu
2020-06-04 16:02                   ` Toke Høiland-Jørgensen
2020-06-05  6:26                     ` Hangbin Liu
2020-06-08 15:32                       ` Toke Høiland-Jørgensen
2020-06-09  3:03                         ` Hangbin Liu
2020-06-09 20:31                           ` Toke Høiland-Jørgensen
2020-06-10  2:35                             ` Hangbin Liu
2020-06-10 10:03                               ` Jesper Dangaard Brouer
2020-07-01  4:19   ` [PATCHv5 bpf-next 0/3] xdp: add a new helper for " Hangbin Liu
2020-07-01  4:19     ` [PATCHv5 bpf-next 1/3] " Hangbin Liu
2020-07-01  5:09       ` Andrii Nakryiko
2020-07-01  6:51         ` Hangbin Liu
2020-07-01 18:33       ` kernel test robot
2020-07-01  4:19     ` [PATCHv5 bpf-next 2/3] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-07-01  4:19     ` [PATCHv5 bpf-next 3/3] selftests/bpf: add xdp_redirect_multi test Hangbin Liu
2020-07-09  1:30     ` [PATCHv6 bpf-next 0/3] xdp: add a new helper for dev map multicast support Hangbin Liu
2020-07-09  1:30       ` [PATCHv6 bpf-next 1/3] " Hangbin Liu
2020-07-09 16:33         ` David Ahern
2020-07-10  6:55           ` Hangbin Liu
2020-07-10 13:46             ` David Ahern
2020-07-11  0:26               ` Hangbin Liu
2020-07-11 16:09                 ` David Ahern
2020-07-09  1:30       ` [PATCHv6 bpf-next 2/3] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-07-09 22:40         ` Daniel Borkmann
2020-07-10  6:41           ` Hangbin Liu
2020-07-10 14:32             ` Daniel Borkmann
2020-07-11  0:21               ` Hangbin Liu
2020-07-09  1:30       ` [PATCHv6 bpf-next 3/3] selftests/bpf: add xdp_redirect_multi test Hangbin Liu
2020-07-09 22:37       ` [PATCHv6 bpf-next 0/3] xdp: add a new helper for dev map multicast support Daniel Borkmann
2020-07-10  7:36         ` Hangbin Liu
2020-07-10 15:02           ` Daniel Borkmann
2020-07-10 16:52             ` David Ahern
2020-07-14  6:32       ` [PATCHv7 " Hangbin Liu
2020-07-14  6:32         ` [PATCHv7 bpf-next 1/3] " Hangbin Liu
2020-07-14 21:52           ` Toke Høiland-Jørgensen
2020-07-15 12:25             ` Hangbin Liu
2020-07-14  6:32         ` [PATCHv7 bpf-next 2/3] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-07-14  6:32         ` [PATCHv7 bpf-next 3/3] selftests/bpf: add xdp_redirect_multi test Hangbin Liu
2020-07-14 12:29         ` [PATCHv7 bpf-next 0/3] xdp: add a new helper for dev map multicast support Toke Høiland-Jørgensen
2020-07-14 17:12           ` David Ahern
2020-07-14 21:53             ` Toke Høiland-Jørgensen
2020-07-15 12:31               ` Hangbin Liu
2020-07-15  3:45             ` Hangbin Liu
2020-07-15 13:08         ` [PATCHv8 " Hangbin Liu
2020-07-15 13:08           ` [PATCHv8 bpf-next 1/3] " Hangbin Liu
2020-07-15 13:08           ` [PATCHv8 bpf-next 2/3] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-07-15 13:08           ` [PATCHv8 bpf-next 3/3] selftests/bpf: add xdp_redirect_multi test Hangbin Liu
2020-08-26 13:19           ` [PATCHv9 bpf-next 0/5] xdp: add a new helper for dev map multicast support Hangbin Liu
2020-08-26 13:19             ` [PATCHv9 bpf-next 1/5] bpf: add a new bpf argument type ARG_CONST_MAP_PTR_OR_NULL Hangbin Liu
2020-08-28 21:56               ` Daniel Borkmann
2020-09-03  9:15                 ` Hangbin Liu
2020-08-26 13:19             ` [PATCHv9 bpf-next 2/5] xdp: add a new helper for dev map multicast support Hangbin Liu
2020-08-28 22:05               ` Daniel Borkmann
2020-08-26 13:20             ` [PATCHv9 bpf-next 3/5] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-08-26 13:20             ` [PATCHv9 bpf-next 4/5] selftests/bpf: add xdp_redirect_multi test Hangbin Liu
2020-08-26 13:20             ` [PATCHv9 bpf-next 5/5] selftests/bpf: Add verifier tests for bpf arg ARG_CONST_MAP_PTR_OR_NULL Hangbin Liu
2020-09-03 10:26             ` [PATCHv10 bpf-next 0/5] xdp: add a new helper for dev map multicast support Hangbin Liu
2020-09-03 10:26               ` [PATCHv10 bpf-next 1/5] bpf: add a new bpf argument type ARG_CONST_MAP_PTR_OR_NULL Hangbin Liu
2020-09-04 14:26                 ` Daniel Borkmann
2020-09-03 10:26               ` [PATCHv10 bpf-next 2/5] xdp: add a new helper for dev map multicast support Hangbin Liu
2020-09-04 14:22                 ` Daniel Borkmann
2020-09-03 10:26               ` [PATCHv10 bpf-next 3/5] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-09-03 10:27               ` [PATCHv10 bpf-next 4/5] selftests/bpf: add xdp_redirect_multi test Hangbin Liu
2020-09-03 10:27               ` [PATCHv10 bpf-next 5/5] selftests/bpf: Add verifier tests for bpf arg ARG_CONST_MAP_PTR_OR_NULL Hangbin Liu
2020-09-07  8:27               ` [PATCHv11 bpf-next 0/5] xdp: add a new helper for dev map multicast support Hangbin Liu
2020-09-07  8:27                 ` [PATCHv11 bpf-next 1/5] bpf: add a new bpf argument type ARG_CONST_MAP_PTR_OR_NULL Hangbin Liu
2020-09-07  8:27                 ` [PATCHv11 bpf-next 2/5] xdp: add a new helper for dev map multicast support Hangbin Liu
2020-09-09 21:52                   ` Alexei Starovoitov
2020-09-10  2:35                     ` Hangbin Liu
2020-09-10  3:30                       ` David Ahern
2020-09-10  5:35                         ` Alexei Starovoitov
2020-09-10  9:44                           ` Toke Høiland-Jørgensen
2020-09-10 15:39                             ` Alexei Starovoitov
2020-09-10 17:50                             ` Jesper Dangaard Brouer
2020-09-10 18:35                               ` David Ahern
2020-09-11  7:58                                 ` Jesper Dangaard Brouer [this message]
2020-09-15 16:12                                   ` David Ahern
2020-09-07  8:27                 ` [PATCHv11 bpf-next 3/5] sample/bpf: add xdp_redirect_map_multicast test Hangbin Liu
2020-09-07  8:27                 ` [PATCHv11 bpf-next 4/5] selftests/bpf: add xdp_redirect_multi test Hangbin Liu
2020-09-07  8:27                 ` [PATCHv11 bpf-next 5/5] selftests/bpf: Add verifier tests for bpf arg ARG_CONST_MAP_PTR_OR_NULL Hangbin Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200911095820.304d9877@carbon \
    --to=brouer@redhat.com \
    --cc=alexei.starovoitov@gmail.com \
    --cc=andrii.nakryiko@gmail.com \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=dsahern@gmail.com \
    --cc=echaudro@redhat.com \
    --cc=jbenc@redhat.com \
    --cc=liuhangbin@gmail.com \
    --cc=lorenzo.bianconi@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=toke@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

BPF Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/bpf/0 bpf/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 bpf bpf/ https://lore.kernel.org/bpf \
		bpf@vger.kernel.org
	public-inbox-index bpf

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.bpf


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git