Observed increased rate of non-linear UDP skb on 5.10 sfc driver

Observed increased rate of non-linear UDP skb on 5.10 sfc driver

[Mark Pashmfouroush]

Hi netdev,

I'm currently conducting an experiment to observe the frequency of
fragmented/non-linear UDP skbs being allocated on the receive path by the sfc
driver. By non-linear I mean that parts of the packet are not contained within
the linear portion of the skb.

I've collected some data on kernel 5.4 with the out-of-tree sfc driver, and now
the mainline sfc driver on kernel 5.10. On kernel 5.4 with the out-of-tree sfc
driver I saw that roughly 10-15% of received UDP skbs were non-linear. On the
in-tree driver I see a significant increase to around 30%. My question is, what
can cause a fragmented/non-linear UDP skb to be allocated by the sfc driver, and
why has this frequency increased since transitioning to the in-tree driver? Is
there a strict size before a UDP packet has an skb with data in the non-linear
buffer? I'm observing this using a tc eBPF program.

While it doesn't seem to cause any issues I'm still curious why there is such a
big difference.

Observed on a Solaflare SFC9120 on multiple different firmware versions with no
difference, currently running version 6.2.7.1001.

Thanks,
Mark

Re: Observed increased rate of non-linear UDP skb on 5.10 sfc driver

[Edward Cree]

On 21/01/2021 13:04, Mark Pashmfouroush wrote:
> My question is, what can cause a fragmented/non-linear UDP skb to be> allocated by the sfc driver, and why has this frequency increased since
> transitioning to the in-tree driver?
By default sfc doesn't RX things as linear; we get one or more RX buffers
 from the hardware and then we attach them as frags to an skb; see
 efx_rx_mk_skb().  We then pull 'hdr_len' bytes from the first frag into
 the linear area; if the total packet length happens to be <= hdr_len then
 the resulting skb will be linear.
hdr_len comes from the caller, efx_rx_deliver().  In-tree we pass
 EFX_SKB_HEADERS, #defined as 128; out-of-tree it's a module parameter
 'rx_copybreak' (internally rx_cb_size) which defaults to 192.  That's
 probably what's causing your difference: some of your packets are between
 128 and 192 bytes in length.
You could verify this theory by seeing what happens if you set rx_copybreak
 to 128 on the out-of-tree driver; you should get the same behaviour as
 with in-tree.

-ed