Re: [RFC PATCH v2 bpf-next 00/15] xdp_flow: Flow offload to XDP

From: William Tu <u9012063@gmail.com>
To: Toshiaki Makita <toshiaki.makita1@gmail.com>
Cc: "David Miller" <davem@davemloft.net>,
	"Toke Høiland-Jørgensen" <toke@redhat.com>,
	"John Fastabend" <john.fastabend@gmail.com>,
	"Alexei Starovoitov" <ast@kernel.org>,
	"Daniel Borkmann" <daniel@iogearbox.net>,
	"Martin KaFai Lau" <kafai@fb.com>,
	"Song Liu" <songliubraving@fb.com>, "Yonghong Song" <yhs@fb.com>,
	"Jakub Kicinski" <jakub.kicinski@netronome.com>,
	"Jesper Dangaard Brouer" <hawk@kernel.org>,
	"Jamal Hadi Salim" <jhs@mojatatu.com>,
	"Cong Wang" <xiyou.wangcong@gmail.com>,
	"Jiri Pirko" <jiri@resnulli.us>,
	"Pablo Neira Ayuso" <pablo@netfilter.org>,
	"Jozsef Kadlecsik" <kadlec@netfilter.org>,
	"Florian Westphal" <fw@strlen.de>,
	"pravin shelar" <pshelar@ovn.org>,
	"Linux Kernel Network Developers" <netdev@vger.kernel.org>,
	bpf <bpf@vger.kernel.org>, "Stanislav Fomichev" <sdf@fomichev.me>
Subject: Re: [RFC PATCH v2 bpf-next 00/15] xdp_flow: Flow offload to XDP
Date: Thu, 14 Nov 2019 09:09:42 -0800	[thread overview]
Message-ID: <CALDO+SbqcoiwJn3jskpPTjdJyK5932r0cEzs=1R6p=CWgERLuw@mail.gmail.com> (raw)
In-Reply-To: <53538a02-e6d3-5443-8251-bef381c691a0@gmail.com>

On Thu, Nov 14, 2019 at 2:06 AM Toshiaki Makita
<toshiaki.makita1@gmail.com> wrote:
>
> On 2019/11/13 2:50, William Tu wrote:
> > On Wed, Oct 30, 2019 at 5:32 PM Toshiaki Makita
> > <toshiaki.makita1@gmail.com> wrote:
> >>
> >> On 2019/10/28 4:17, David Miller wrote:
> >>> From: Toke Høiland-Jørgensen <toke@redhat.com>
> >>> Date: Sun, 27 Oct 2019 16:24:24 +0100
> >>>
> >>>> The results in the paper also shows somewhat disappointing performance
> >>>> for the eBPF implementation, but that is not too surprising given that
> >>>> it's implemented as a TC eBPF hook, not an XDP program. I seem to recall
> >>>> that this was also one of the things puzzling to me back when this was
> >>>> presented...
> >>>
> >>> Also, no attempt was made to dyanamically optimize the data structures
> >>> and code generated in response to features actually used.
> >>>
> >>> That's the big error.
> >>>
> >>> The full OVS key is huge, OVS is really quite a monster.
> >>>
> >>> But people don't use the entire key, nor do they use the totality of
> >>> the data paths.
> >>>
> >>> So just doing a 1-to-1 translation of the OVS datapath into BPF makes
> >>> absolutely no sense whatsoever and it is guaranteed to have worse
> >>> performance.
> >
> > 1-to-1 translation has nothing to do with performance.
>
> I think at least key size matters.
> One big part of hot spots in xdp_flow bpf program is hash table lookup.
> Especially hash calculation by jhash and key comparison are heavy.
> The computational cost heavily depends on key size.
>
> If umh can determine some keys won't be used in some way (not sure if it's
> practical though), umh can load an XDP program which uses less sized
> key. Also it can remove unnecessary key parser routines.
> If it's possible, the performance will increase.
>
Yes, that's a good point.
In other meeting people also gave me this suggestions.

Basically it's "on-demand flow key parsing using eBPF"
The key parsing consists of multiple eBPF programs, and
based on the existing rules, load the program and parse minimum
necessary fields required by existing rules. This will definitely
have better performance.

I didn't try it at all because most of our use cases use overlay
tunnel and connection tracking.  There is little chance of rules
using only L2 or L3 fields. Another way is to do flow key compression
something like miniflow in OVS userspace datapath.

Regards,
William