Re: [PATCH net] net: avoid 32 x truesize under-estimation for tiny skbs

[Paolo Abeni <pabeni@redhat.com>]

On Wed, 2022-09-07 at 13:40 -0700, Eric Dumazet wrote:
> On 9/7/22 13:19, Paolo Abeni wrote:
> > Hello,
> > 
> > reviving an old thread...
> > On Wed, 2021-01-13 at 08:18 -0800, Eric Dumazet wrote:
> > > While using page fragments instead of a kmalloc backed skb->head might give
> > > a small performance improvement in some cases, there is a huge risk of
> > > under estimating memory usage.
> > [...]
> > 
> > > Note that we might in the future use the sk_buff napi cache,
> > > instead of going through a more expensive __alloc_skb()
> > > 
> > > Another idea would be to use separate page sizes depending
> > > on the allocated length (to never have more than 4 frags per page)
> > I'm investigating a couple of performance regressions pointing to this
> > change and I'd like to have a try to the 2nd suggestion above.
> > 
> > If I read correctly, it means:
> > - extend the page_frag_cache alloc API to allow forcing max order==0
> > - add a 2nd page_frag_cache into napi_alloc_cache (say page_order0 or
> > page_small)
> > - in __napi_alloc_skb(), when len <= SKB_WITH_OVERHEAD(1024), use the
> > page_small cache with order 0 allocation.
> > (all the above constrained to host with 4K pages)
> > 
> > I'm not quite sure about the "never have more than 4 frags per page"
> > part.
> > 
> > What outlined above will allow for 10 min size frags in page_order0, as
> > (SKB_DATA_ALIGN(0) + SKB_DATA_ALIGN(struct skb_shared_info) == 384. I'm
> > not sure that anything will allocate such small frags.
> > With a more reasonable GRO_MAX_HEAD, there will be 6 frags per page.
> 
> Well, some arches have PAGE_SIZE=65536 :/

Yes, the idea is to implement all the above only for arches with
PAGE_SIZE==4K. Would that be reasonable? 

Thanks!

Paolo