Re: [PATCH net-next v2] samples/bpf: fixup some tools to be able to support xdp multibuffer

From: Tariq Toukan <ttoukan.linux@gmail.com>
To: "Toke Høiland-Jørgensen" <toke@redhat.com>,
	"Lorenzo Bianconi" <lorenzo@kernel.org>,
	"Jakub Kicinski" <kuba@kernel.org>
Cc: Andy Gospodarek <andrew.gospodarek@broadcom.com>,
	ast@kernel.org, daniel@iogearbox.net, davem@davemloft.net,
	hawk@kernel.org, john.fastabend@gmail.com, andrii@kernel.org,
	kafai@fb.com, songliubraving@fb.com, yhs@fb.com,
	kpsingh@kernel.org, lorenzo.bianconi@redhat.com,
	netdev@vger.kernel.org, bpf@vger.kernel.org,
	Jesper Dangaard Brouer <brouer@redhat.com>,
	Ilias Apalodimas <ilias.apalodimas@linaro.org>,
	Andy Gospodarek <gospo@broadcom.com>,
	gal@nvidia.com, Saeed Mahameed <saeedm@nvidia.com>,
	tariqt@nvidia.com
Subject: Re: [PATCH net-next v2] samples/bpf: fixup some tools to be able to support xdp multibuffer
Date: Thu, 5 Jan 2023 09:20:18 +0200	[thread overview]
Message-ID: <a12de9d9-c022-3b57-0a15-e22cdae210fa@gmail.com> (raw)
In-Reply-To: <87bkne32ly.fsf@toke.dk>

On 04/01/2023 14:28, Toke Høiland-Jørgensen wrote:
> Lorenzo Bianconi <lorenzo@kernel.org> writes:
> 
>>> On Tue, 03 Jan 2023 16:19:49 +0100 Toke Høiland-Jørgensen wrote:
>>>> Hmm, good question! I don't think we've ever explicitly documented any
>>>> assumptions one way or the other. My own mental model has certainly
>>>> always assumed the first frag would continue to be the same size as in
>>>> non-multi-buf packets.
>>>
>>> Interesting! :) My mental model was closer to GRO by frags
>>> so the linear part would have no data, just headers.
>>
>> That is assumption as well.
> 
> Right, okay, so how many headers? Only Ethernet, or all the way up to
> L4 (TCP/UDP)?
> 
> I do seem to recall a discussion around the header/data split for TCP
> specifically, but I think I mentally put that down as "something people
> may way to do at some point in the future", which is why it hasn't made
> it into my own mental model (yet?) :)
> 
> -Toke
> 

I don't think that all the different GRO layers assume having their 
headers/data in the linear part. IMO they will just perform better if 
these parts are already there. Otherwise, the GRO flow manages, and 
pulls the needed amount into the linear part.
As examples, see calls to gro_pull_from_frag0 in net/core/gro.c, and the 
call to pskb_may_pull() from skb_gro_header_slow().

This resembles the bpf_xdp_load_bytes() API used here in the xdp prog.

The context of my questions is that I'm looking for the right memory 
scheme for adding xdp-mb support to mlx5e striding RQ.
In striding RQ, the RX buffer consists of "strides" of a fixed size set 
by the driver. An incoming packet is written to the buffer starting from 
the beginning of the next available stride, consuming as much strides as 
needed.

Due to the need for headroom and tailroom, there's no easy way of 
building the xdp_buf in place (around the packet), so it should go to a 
side buffer.

By using 0-length linear part in a side buffer, I can address two 
challenging issues: (1) save the in-driver headers memcpy (copy might 
still exist in the xdp program though), and (2) conform to the 
"fragments of the same size" requirement/assumption in xdp-mb. 
Otherwise, if we pull from frag[0] into the linear part, frag[0] becomes 
smaller than the next fragments.

Regards,
Tariq