Re: [PATCH] Software receive packet steering

[Tom Herbert <therbert@google.com>]

>>> -extern int          netif_receive_skb(struct sk_buff *skb);
>>> +extern int            __netif_receive_skb(struct sk_buff *skb);
>>> +
>>> +static inline int netif_receive_skb(struct sk_buff *skb)
>>> +{
>>> +#ifdef CONFIG_NET_SOFTRPS
>>> +    return netif_rx(skb);
>>> +#else
>>> +    return __netif_receive_skb(skb);
>>> +#endif
>>> +}
>>
>> Ugh, this forces all devices receiving back into a single backlog
>> queue.
>
> Yes, it basically turns off NAPI.
>

NAPI is still useful, but it does take a higher packet load before
polling kicks in.  I believe this is similarly true for HW multi
queue, and could actually be worse depending on the number of queues
traffic is being split across (in my bnx2x experiment 16 core AMD with
16 queues, I was seeing around 300K interrupts per second, no benefit
from NAPI).

The bimodal behavior between polling and non-polling states does give
us fits.  I looked at the parked mode idea, but the latency hit seems
too high.  We've considered holding the interface in polling state for
longer periods of time, maybe this could trade off CPU cycles (on the
core taking interrupts) for lower latency and higher throughput.