napi layer and packet throttling

napi layer and packet throttling

[Holger Brunck]

Hi all,
I have a general question to the napi layer. I have a board with a MPC8360
processor which uses the ucc_geth.c driver for it's ethernet device. The kernel
version is 3.9. We see in lab tests that the board gets unresponsive if we send
a high packet rate to this ethernet device via an packet generator. In the end
we would like to limit the amount of incoming packets at a very early stage, to
prevent the system of a to high softirq load.

The NAPI layer description I read on:
http://www.linuxfoundation.org/collaborate/workgroups/networking/napi
pointed two main functions out for the napi layer.

a) Interrupt mitigation
Which works perfectly on the ucc_geth.c driver under high load the hard irqs for
the RX packets are disabled and we switch to a polling mode. If the load is more
relaxed we switch back.

b) Packet-Throttling
Here the description says "NAPI-compliant drivers can often cause packets to be
dropped in the network adaptor itself, before the kernel sees them at all."

This is exactly what I need for my usecase. But I don't see any hints how this
can be implemented with the napi layer. So what I need would be something like a
possibility to accept on the eth0 device in timeframe x only packetcount y. If
this packetcount is reached disable hard irq and napi_poll until the timeframe
is expired.

So are there any pointers to other ethernet drivers doing something like this?
Do I misunderstand the packet throttling purpose of the NAPI layer completely?

Any help is appreciated.

Regards
Holger

Re: napi layer and packet throttling

[Ben Hutchings]

On Thu, 2013-05-23 at 13:52 +0200, Holger Brunck wrote:
> Hi all,
> I have a general question to the napi layer. I have a board with a MPC8360
> processor which uses the ucc_geth.c driver for it's ethernet device. The kernel
> version is 3.9. We see in lab tests that the board gets unresponsive if we send
> a high packet rate to this ethernet device via an packet generator. In the end
> we would like to limit the amount of incoming packets at a very early stage, to
> prevent the system of a to high softirq load.
> 
> The NAPI layer description I read on:
> http://www.linuxfoundation.org/collaborate/workgroups/networking/napi
> pointed two main functions out for the napi layer.
> 
> a) Interrupt mitigation
> Which works perfectly on the ucc_geth.c driver under high load the hard irqs for
> the RX packets are disabled and we switch to a polling mode. If the load is more
> relaxed we switch back.
> 
> b) Packet-Throttling
> Here the description says "NAPI-compliant drivers can often cause packets to be
> dropped in the network adaptor itself, before the kernel sees them at all."
> 
> This is exactly what I need for my usecase. But I don't see any hints how this
> can be implemented with the napi layer.
[...]

If the RX ring is not cleaned and refilled quickly enough, the network
controller will naturally start to drop packets.  It's not something you
should do explicitly in the driver.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: napi layer and packet throttling

[Holger Brunck]

Hi Ben,

On 05/24/2013 12:27 AM, Ben Hutchings wrote:
> On Thu, 2013-05-23 at 13:52 +0200, Holger Brunck wrote:
>> b) Packet-Throttling
>> Here the description says "NAPI-compliant drivers can often cause packets to be
>> dropped in the network adaptor itself, before the kernel sees them at all."
>>
>> This is exactly what I need for my usecase. But I don't see any hints how this
>> can be implemented with the napi layer.
> [...]
> 
> If the RX ring is not cleaned and refilled quickly enough, the network
> controller will naturally start to drop packets.  It's not something you
> should do explicitly in the driver.
> 

yes. But what if the remaining amount of packets which are getting through the
napi_poll function into the linux system are still to many and generate
therefore a to high softirq load on the system which leads to the problems I
see. Ok I could use a smaller amount of RX ring buffers, but then the system
would get more intolerant for RX bursts what I don't want. I would like to
protect the system if someone sends continuously a high packet rate to the
interface, similar to DoS attacks.

Regards
Holger

Re: napi layer and packet throttling

[devendra.aaru]

On Fri, May 24, 2013 at 12:58 PM, Holger Brunck
<holger.brunck@keymile.com> wrote:
> Hi Ben,
>
> On 05/24/2013 12:27 AM, Ben Hutchings wrote:
>> On Thu, 2013-05-23 at 13:52 +0200, Holger Brunck wrote:
>>> b) Packet-Throttling
>>> Here the description says "NAPI-compliant drivers can often cause packets to be
>>> dropped in the network adaptor itself, before the kernel sees them at all."
>>>
>>> This is exactly what I need for my usecase. But I don't see any hints how this
>>> can be implemented with the napi layer.
>> [...]
>>
>> If the RX ring is not cleaned and refilled quickly enough, the network
>> controller will naturally start to drop packets.  It's not something you
>> should do explicitly in the driver.
>>
>
> yes. But what if the remaining amount of packets which are getting through the
> napi_poll function into the linux system are still to many and generate
> therefore a to high softirq load on the system which leads to the problems I
> see. Ok I could use a smaller amount of RX ring buffers, but then the system
> would get more intolerant for RX bursts what I don't want. I would like to
> protect the system if someone sends continuously a high packet rate to the
> interface, similar to DoS attacks.


what if i measure no. of packets over a period of time and find the
rate of arrival (packets/msec) and if that rate is greater than the
choosen rate, allow only packets that are in the window that you can
accept, This window can be time window or the number of packets (like
1 packet in 10 usec or 1 packet out of 100 packets). Like wise this
measurement can be repeated after every full period.


>
> Regards
> Holger
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: napi layer and packet throttling

[Holger Brunck]

On 05/24/2013 11:20 AM, devendra.aaru wrote:
> On Fri, May 24, 2013 at 12:58 PM, Holger Brunck
>>
>> On 05/24/2013 12:27 AM, Ben Hutchings wrote:
>>> On Thu, 2013-05-23 at 13:52 +0200, Holger Brunck wrote:
>>>> b) Packet-Throttling
>>>> Here the description says "NAPI-compliant drivers can often cause packets to be
>>>> dropped in the network adaptor itself, before the kernel sees them at all."
>>>>
>>>> This is exactly what I need for my usecase. But I don't see any hints how this
>>>> can be implemented with the napi layer.
>>> [...]
>>>
>>> If the RX ring is not cleaned and refilled quickly enough, the network
>>> controller will naturally start to drop packets.  It's not something you
>>> should do explicitly in the driver.
>>>
>>
>> yes. But what if the remaining amount of packets which are getting through the
>> napi_poll function into the linux system are still to many and generate
>> therefore a to high softirq load on the system which leads to the problems I
>> see. Ok I could use a smaller amount of RX ring buffers, but then the system
>> would get more intolerant for RX bursts what I don't want. I would like to
>> protect the system if someone sends continuously a high packet rate to the
>> interface, similar to DoS attacks.
> 
> 
> what if i measure no. of packets over a period of time and find the
> rate of arrival (packets/msec) and if that rate is greater than the
> choosen rate, allow only packets that are in the window that you can
> accept, This window can be time window or the number of packets (like
> 1 packet in 10 usec or 1 packet out of 100 packets). Like wise this
> measurement can be repeated after every full period.
> 

thats similar to what I have implemented in ucc_geth.c for my workspace and yes
whith such a mechanism my problems seems to be fixed. But I still ask myself if
there is a common approach for such a problem already implemented in similar
drivers...

Regards
Holger

Re: napi layer and packet throttling

[Ben Hutchings]

On Fri, 2013-05-24 at 09:28 +0200, Holger Brunck wrote:
> Hi Ben,
> 
> On 05/24/2013 12:27 AM, Ben Hutchings wrote:
> > On Thu, 2013-05-23 at 13:52 +0200, Holger Brunck wrote:
> >> b) Packet-Throttling
> >> Here the description says "NAPI-compliant drivers can often cause packets to be
> >> dropped in the network adaptor itself, before the kernel sees them at all."
> >>
> >> This is exactly what I need for my usecase. But I don't see any hints how this
> >> can be implemented with the napi layer.
> > [...]
> > 
> > If the RX ring is not cleaned and refilled quickly enough, the network
> > controller will naturally start to drop packets.  It's not something you
> > should do explicitly in the driver.
> > 
> 
> yes. But what if the remaining amount of packets which are getting through the
> napi_poll function into the linux system are still to many and generate
> therefore a to high softirq load on the system which leads to the problems I
> see.

Soft IRQs are then handled in a ksoftirqd thread, which is scheduled
like any other task (though it has high priority).

> Ok I could use a smaller amount of RX ring buffers, but then the system
> would get more intolerant for RX bursts what I don't want. I would like to
> protect the system if someone sends continuously a high packet rate to the
> interface, similar to DoS attacks.

This is not the job of the driver (beyond using NAPI in the first
place).

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.