Re: xennet: skb rides the rocket: 20 slots

[Ian Campbell <Ian.Campbell@citrix.com>]

On Thu, 2013-01-10 at 11:22 +0000, ANNIE LI wrote:
> I am thinking to do re-fragment in netfront for these skbs like following,
> 
> Create a new skb, copy linear data and frag data from original skb into 
> this one, and make every frags data size is PAGE_SIZE except for the 
> last fragment. It is possible that the last fragment length is less than 
> PAGE_SIZE, then free the original skb. The skb packet is large, and 
> there will be lots of copys.

You don't need (or I suspect want) to copy, you can directly add pages
from the source skb's frags to the destination skb's frags, with
appropriate refcount frobbing. You can also share a page between two (or
more) skbs in the case where the boundary between two skbs happens to be
in the middle of a page.

But more importantly than all that you need to do more than just
refragment, you actually need to resegment i.e. you need to duplicate
the headers (Ethernet, IP, TCP) at the front of each new skb and adjust
the (psuedo-)checksums as appropriate (which will depend on whether the
SKB is GSO, or just has checksum offload or nothing).

So you need to go from
  <Ether><IP><TCP><...data... ...data...>
to
  <Ether><IP><TCP><...data...> <Ether><IP><TCP><...data...>

Where the headers are adjusted to cope with this. Some of data might be
in frags but equally some might be in skb->data.

I'm guessing that Linux already has code which can do this for you,
since it has a software fallback for GSO.

Ian.

> 
> struct skbuff *xennet_refrag_skb(skb)
> {
>     create newskb
>     copying data and doing fragmentation
>     return newskb
> }
> 
> .......
> 
> if (unlikely(slots>  MAX_SKB_FRAGS + 1)) {
>          net_alert_ratelimited(
>                "xennet: skb rides the rocket: %d slots\n", slots);
>          skb = xennet_refrag_skb(skb);
> }
> .....
> 
> Thanks
> Annie
> >
> >> "netchannel vs MAX_SKB_FRAGS". Maybe these two mechanism are all
> >> necessary?
> > Lets see first if this is indeed the problem. Perhaps a simple debug
> > patch that just does:
> >
> > 	s/MAX_SKB_FRAGS/DEBUG_MAX_FRAGS/
> > 	#define DEBUG_MAX_FRAGS 21
> >
> > in both netback and netfront to set the maximum number of frags we can
> > handle to 21? If that works with Sander test - then yes, it looks like
> > we really need to get this 'feature-max-skb-frags' done.
> >