From mboxrd@z Thu Jan  1 00:00:00 1970
From: ANNIE LI <annie.li@oracle.com>
Subject: Re: xennet: skb rides the rocket: 20 slots
Date: Fri, 11 Jan 2013 15:34:17 +0800
Message-ID: <50EFC079.1000601@oracle.com>
References: <72958707.20130104172854@eikelenboom.it>
	<1357556115.7989.13.camel@zakaz.uk.xensource.com>
	<50EB8091.90705@oracle.com>
	<323202711.20130108215503@eikelenboom.it>
	<50ED1800.1080208@oracle.com>
	<20130109150850.GI18395@phenom.dumpdata.com>
	<50EEA46E.7000604@oracle.com>
	<1357820816.9456.15.camel@zakaz.uk.xensource.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xen.org>
In-Reply-To: <1357820816.9456.15.camel@zakaz.uk.xensource.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Ian Campbell <Ian.Campbell@citrix.com>
Cc: Sander Eikelenboom <linux@eikelenboom.it>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, xen-devel <xen-devel@lists.xen.org>
List-Id: xen-devel@lists.xenproject.org



On 2013-1-10 20:26, Ian Campbell wrote:
> On Thu, 2013-01-10 at 11:22 +0000, ANNIE LI wrote:
>> I am thinking to do re-fragment in netfront for these skbs like following,
>>
>> Create a new skb, copy linear data and frag data from original skb into
>> this one, and make every frags data size is PAGE_SIZE except for the
>> last fragment. It is possible that the last fragment length is less than
>> PAGE_SIZE, then free the original skb. The skb packet is large, and
>> there will be lots of copys.
> You don't need (or I suspect want) to copy, you can directly add pages
> from the source skb's frags to the destination skb's frags, with
> appropriate refcount frobbing. You can also share a page between two (or
> more) skbs in the case where the boundary between two skbs happens to be
> in the middle of a page.
>
> But more importantly than all that you need to do more than just
> refragment, you actually need to resegment i.e. you need to duplicate
> the headers (Ethernet, IP, TCP) at the front of each new skb and adjust
> the (psuedo-)checksums as appropriate (which will depend on whether the
> SKB is GSO, or just has checksum offload or nothing).
>
> So you need to go from
>    <Ether><IP><TCP><...data... ...data...>
> to
>    <Ether><IP><TCP><...data...>  <Ether><IP><TCP><...data...>
>
> Where the headers are adjusted to cope with this. Some of data might be
> in frags but equally some might be in skb->data.
>
> I'm guessing that Linux already has code which can do this for you,
> since it has a software fallback for GSO.

There is "skb_gso_segment" existing to perform segmentation on skb and 
return a list of segments. From the code, it seems this function is 
specific to GSO packets, maybe it is not available to other non-gso 
large packets which requires slots is larger than MAX_SKB_FRAGS + 1". So 
for non-gso offload packets, I would go forward to write a function to 
do resegment.

Thanks
Annie
>
> Ian.
>
>> struct skbuff *xennet_refrag_skb(skb)
>> {
>>      create newskb
>>      copying data and doing fragmentation
>>      return newskb
>> }
>>
>> .......
>>
>> if (unlikely(slots>   MAX_SKB_FRAGS + 1)) {
>>           net_alert_ratelimited(
>>                 "xennet: skb rides the rocket: %d slots\n", slots);
>>           skb = xennet_refrag_skb(skb);
>> }
>> .....
>>
>> Thanks
>> Annie
>>>> "netchannel vs MAX_SKB_FRAGS". Maybe these two mechanism are all
>>>> necessary?
>>> Lets see first if this is indeed the problem. Perhaps a simple debug
>>> patch that just does:
>>>
>>> 	s/MAX_SKB_FRAGS/DEBUG_MAX_FRAGS/
>>> 	#define DEBUG_MAX_FRAGS 21
>>>
>>> in both netback and netfront to set the maximum number of frags we can
>>> handle to 21? If that works with Sander test - then yes, it looks like
>>> we really need to get this 'feature-max-skb-frags' done.
>>>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel