Re: [RFC PATCH v2 bpf-next 00/15] xdp_flow: Flow offload to XDP

From: Jesper Dangaard Brouer <jbrouer@redhat.com>
To: "Toke Høiland-Jørgensen" <toke@redhat.com>
Cc: David Ahern <dsahern@gmail.com>,
	Toshiaki Makita <toshiaki.makita1@gmail.com>,
	John Fastabend <john.fastabend@gmail.com>,
	Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
	Yonghong Song <yhs@fb.com>,
	"David S. Miller" <davem@davemloft.net>,
	Jakub Kicinski <jakub.kicinski@netronome.com>,
	Jesper Dangaard Brouer <hawk@kernel.org>,
	Jamal Hadi Salim <jhs@mojatatu.com>,
	Cong Wang <xiyou.wangcong@gmail.com>,
	Jiri Pirko <jiri@resnulli.us>,
	Pablo Neira Ayuso <pablo@netfilter.org>,
	Jozsef Kadlecsik <kadlec@netfilter.org>,
	Florian Westphal <fw@strlen.de>,
	Pravin B Shelar <pshelar@ovn.org>,
	netdev@vger.kernel.org, bpf@vger.kernel.org,
	William Tu <u9012063@gmail.com>,
	Stanislav Fomichev <sdf@fomichev.me>
Subject: Re: [RFC PATCH v2 bpf-next 00/15] xdp_flow: Flow offload to XDP
Date: Mon, 28 Oct 2019 11:08:28 +0100	[thread overview]
Message-ID: <20191028110828.512eb99c@carbon> (raw)
In-Reply-To: <87o8y1s1vn.fsf@toke.dk>

On Mon, 28 Oct 2019 09:36:12 +0100
Toke Høiland-Jørgensen <toke@redhat.com> wrote:

> David Ahern <dsahern@gmail.com> writes:
> 
> > On 10/27/19 9:21 AM, Toke Høiland-Jørgensen wrote:  
> >> Rather, what we should be doing is exposing the functionality through
> >> helpers so XDP can hook into the data structures already present in the
> >> kernel and make decisions based on what is contained there. We already
> >> have that for routing; L2 bridging, and some kind of connection
> >> tracking, are obvious contenders for similar additions.  
> >
> > The way OVS is coded and expected to flow (ovs_vport_receive ->
> > ovs_dp_process_packet -> ovs_execute_actions -> do_execute_actions) I do
> > not see any way to refactor it to expose a hook to XDP. But, if the use
> > case is not doing anything big with OVS (e.g., just ACLs and forwarding)
> > that is easy to replicate in XDP - but then that means duplicate data
> > and code.  
> 
> Yeah, I didn't mean that part for OVS, that was a general comment for
> reusing kernel functionality.
> 
> > Linux bridge on the other hand seems fairly straightforward to
> > refactor. One helper is needed to convert ingress <port,mac,vlan> to
> > an L2 device (and needs to consider stacked devices) and then a second
> > one to access the fdb for that device.  
> 
> Why not just a single lookup like what you did for routing? Not too
> familiar with the routing code...

I'm also very interested in hearing more about how we can create an XDP
bridge lookup BPF-helper...

> > Either way, bypassing the bridge has mixed results: latency improves
> > but throughput takes a hit (no GRO).  
> 
> Well, for some traffic mixes XDP should be able to keep up without GRO.
> And longer term, we probably want to support GRO with XDP anyway

Do you have any numbers to back up your expected throughput decrease,
due to lack of GRO?  Or is it a theory?

GRO mainly gains performance due to the bulking effect.  XDP redirect
also have bulking.  For bridging, I would claim that XDP redirect
bulking works better, because it does bulking based on egress
net_device. (Even for intermixed packets per NAPI budget).  You might
worry that XDP will do a bridge-lookup per frame, but as the likely fit
in the CPU I-cache, then this will have very little effect.

> (I believe Jesper has plans for supporting bigger XDP frames)...

Yes [1], but it's orthogonal and mostly that to support HW features,
like TSO, jumbo-frames, packet header split.

 [1] https://github.com/xdp-project/xdp-project/blob/master/areas/core/xdp-multi-buffer01-design.org
-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer