NAPI on USB network drivers

NAPI on USB network drivers

[Oliver Neukum]

Hi,

looking at r8152 I noticed that it uses NAPI. I never considered
this for the generic USB networking code as you cannot disable
interrupts for USB. Is it still worth it? What are the benefits?

	Regards
		Oliver

RE: NAPI on USB network drivers

[Hayes Wang]

Oliver Neukum [mailto:oneukum@suse.com]
> Sent: Wednesday, January 25, 2017 5:35 PM
[...]
> looking at r8152 I noticed that it uses NAPI. I never considered
> this for the generic USB networking code as you cannot disable
> interrupts for USB. Is it still worth it? What are the benefits?

You could use napi_gro_receive() and it influences the performance.

Best Regards,
Hayes

Re: NAPI on USB network drivers

[Oliver Hartkopp]

On 01/25/2017 10:39 AM, Hayes Wang wrote:
> Oliver Neukum [mailto:oneukum-IBi9RG/b67k@public.gmane.org]
>> Sent: Wednesday, January 25, 2017 5:35 PM
> [...]
>> looking at r8152 I noticed that it uses NAPI. I never considered
>> this for the generic USB networking code as you cannot disable
>> interrupts for USB. Is it still worth it? What are the benefits?
>
> You could use napi_gro_receive() and it influences the performance.

Another positive effect with NAPI is that you won't face out-of-order 
ethernet frames as you get with non-NAPI drivers, e.g. ax88179_178a

http://marc.info/?l=linux-can&m=148049063812807&w=2

We have the issue with CAN drivers where all USB drivers and >90% of the 
I/O mapped drivers do not use NAPI.

I wonder whether it makes sense to add NAPI to a driver which only has 
ONE RX buffer ... but when searching for a solution for o-o-o frames I 
was always pointed to NAPI.

Regards,
Oliver

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: NAPI on USB network drivers

[Alexander Duyck]

On Wed, Jan 25, 2017 at 5:33 AM, Oliver Hartkopp <socketcan@hartkopp.net> wrote:
> On 01/25/2017 10:39 AM, Hayes Wang wrote:
>>
>> Oliver Neukum [mailto:oneukum@suse.com]
>>>
>>> Sent: Wednesday, January 25, 2017 5:35 PM
>>
>> [...]
>>>
>>> looking at r8152 I noticed that it uses NAPI. I never considered
>>> this for the generic USB networking code as you cannot disable
>>> interrupts for USB. Is it still worth it? What are the benefits?
>>
>>
>> You could use napi_gro_receive() and it influences the performance.
>
>
> Another positive effect with NAPI is that you won't face out-of-order
> ethernet frames as you get with non-NAPI drivers, e.g. ax88179_178a
>
> http://marc.info/?l=linux-can&m=148049063812807&w=2
>
> We have the issue with CAN drivers where all USB drivers and >90% of the I/O
> mapped drivers do not use NAPI.
>
> I wonder whether it makes sense to add NAPI to a driver which only has ONE
> RX buffer ... but when searching for a solution for o-o-o frames I was
> always pointed to NAPI.
>
> Regards,
> Oliver
>

You could probably get around the o-o-o problem by enabling RPS for
the interface.  I have found that it works for me to do that in order
to resolve o-o-o frames generated by VMs on virtual interfaces that
can't use NAPI.

- Alex

Re: NAPI on USB network drivers

[Oliver Hartkopp]

On 01/25/2017 09:57 PM, Alexander Duyck wrote:
> On Wed, Jan 25, 2017 at 5:33 AM, Oliver Hartkopp <socketcan@hartkopp.net> wrote:
>
> You could probably get around the o-o-o problem by enabling RPS for
> the interface.  I have found that it works for me to do that in order
> to resolve o-o-o frames generated by VMs on virtual interfaces that
> can't use NAPI.

Sounds promising!

In fact I tried to fix this o-o-o stuff here:

http://marc.info/?l=linux-can&m=143637774606287&w=2

Do you have an example how to do it right?

Best regards,
Oliver

Re: NAPI on USB network drivers

[Eric Dumazet]

On Wed, 2017-01-25 at 09:39 +0000, Hayes Wang wrote:
> Oliver Neukum [mailto:oneukum@suse.com]
> > Sent: Wednesday, January 25, 2017 5:35 PM
> [...]
> > looking at r8152 I noticed that it uses NAPI. I never considered
> > this for the generic USB networking code as you cannot disable
> > interrupts for USB. Is it still worth it? What are the benefits?
> 
> You could use napi_gro_receive() and it influences the performance.

You also could use napi_complete_done() instead of napi_complete(), as
it allows users to tune the performance vs latency for GRO.

Looking at this driver, I do not see any limitation on the number of
skbs that can be pushed into tp->rx_queue.

I wonder if this queue can end up consuming all memory of a host under
stress.

diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c
index e1466b4d2b6c727148a884672bbd9593bf04b3ac..221df4a931b5c1073f1922d0fa0bbff158c73b7d 100644
--- a/drivers/net/usb/r8152.c
+++ b/drivers/net/usb/r8152.c
@@ -1840,7 +1840,10 @@ static int rx_bottom(struct r8152 *tp, int budget)
 				stats->rx_packets++;
 				stats->rx_bytes += pkt_len;
 			} else {
-				__skb_queue_tail(&tp->rx_queue, skb);
+				if (unlikely(skb_queue_len(&tp->rx_queue) >= 1000))
+					kfree_skb(skb);
+				else
+					__skb_queue_tail(&tp->rx_queue, skb);
 			}
 
 find_next_rx:

RE: NAPI on USB network drivers

[Hayes Wang]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1287 bytes --]

Eric Dumazet [mailto:eric.dumazet@gmail.com]
> Sent: Wednesday, January 25, 2017 10:13 PM
[...]
> You also could use napi_complete_done() instead of napi_complete(), as
> it allows users to tune the performance vs latency for GRO.
> 
> Looking at this driver, I do not see any limitation on the number of
> skbs that can be pushed into tp->rx_queue.
> 
> I wonder if this queue can end up consuming all memory of a host under
> stress.
> 
> diff --git a/drivers/net/usb/r8152.c b/drivers/net/usb/r8152.c
> index
> e1466b4d2b6c727148a884672bbd9593bf04b3ac..221df4a931b5c1073f1922d0fa0b
> bff158c73b7d 100644
> --- a/drivers/net/usb/r8152.c
> +++ b/drivers/net/usb/r8152.c
> @@ -1840,7 +1840,10 @@ static int rx_bottom(struct r8152 *tp, int budget)
>  				stats->rx_packets++;
>  				stats->rx_bytes += pkt_len;
>  			} else {
> -				__skb_queue_tail(&tp->rx_queue, skb);
> +				if (unlikely(skb_queue_len(&tp->rx_queue) >= 1000))
> +					kfree_skb(skb);
> +				else
> +					__skb_queue_tail(&tp->rx_queue, skb);
>  			}
> 
>  find_next_rx:

Thanks for your suggestion. I would submit it later.

Best Regards,
Hayes



N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±ºÆâžØ^n‡r¡ö¦zË\x1aëh™¨è­Ú&¢îý»\x05ËÛÔØï¦v¬Îf\x1dp)¹¹br	šê+€Ê+zf£¢·hšˆ§~†­†Ûiÿûàz¹\x1e®w¥¢¸?™¨è­Ú&¢)ߢ^[f

Re: NAPI on USB network drivers

[David Miller]

From: Oliver Neukum <oneukum@suse.com>
Date: Wed, 25 Jan 2017 10:34:41 +0100

> looking at r8152 I noticed that it uses NAPI. I never considered
> this for the generic USB networking code as you cannot disable
> interrupts for USB. Is it still worth it? What are the benefits?

I think it's not a good approach for getting GRO.

Exactly because of the issue you raise, in that USB drivers
cannot stop the URBs from coming in while running their poll
method.