Intel 82598, RSS and PPPoE

Intel 82598, RSS and PPPoE

[Alexander Lapidus]

Hello, everybody!
 Help is needed in such case. We have a linux-box, running under CentOS 5.4 
with a latest kernel. Intel 10 Gigabit AF DA Dual Port Server Adapter (based 
on Intel 82598EB) is used for routing purposes. Linux-box is dual CPU Xeon 
E5530 server.  Interface eth0 is an uplink to the network core with pure IPv4 
trafic on it. Interface eth1 is an downlink to customers. We have 3000 802.1q 
VLANs on it. PPPoE server daemon is running on all VLAN interfaces. So we have 
ethernet frames with 802.1q VLAN tag and PPPoE header incapsulated. Our 
problem is with RX packet distribution between rx-queues on this  interface. 
All those packets are coming to RX-queue-0. As i understood from Intel 
specification on 82598 chipset, problem is with RSS, which can't decode PPPoE 
header, so packet is going only to queue 0. My question is how to force such 
packets to spread between different queues. May be it's possible to use VMDq 
in this configuration, or any other solution?

P.S. We tried this configuration with in-kernel driver and with latest driver 
from Intel (ixgbe-2.0.44.14). Situation is the same.


-- 
Thanks a lot,
Alexander Lapidus
AL546-RIPE
WildPark ISP,
Nikolaev, UA

Re: Intel 82598, RSS and PPPoE

[Eric Dumazet]

Le jeudi 28 janvier 2010 à 13:34 +0200, Alexander Lapidus a écrit :
> Hello, everybody!
>  Help is needed in such case. We have a linux-box, running under CentOS 5.4 
> with a latest kernel. Intel 10 Gigabit AF DA Dual Port Server Adapter (based 
> on Intel 82598EB) is used for routing purposes. Linux-box is dual CPU Xeon 
> E5530 server.  Interface eth0 is an uplink to the network core with pure IPv4 
> trafic on it. Interface eth1 is an downlink to customers. We have 3000 802.1q 
> VLANs on it. PPPoE server daemon is running on all VLAN interfaces. So we have 
> ethernet frames with 802.1q VLAN tag and PPPoE header incapsulated. Our 
> problem is with RX packet distribution between rx-queues on this  interface. 
> All those packets are coming to RX-queue-0. As i understood from Intel 
> specification on 82598 chipset, problem is with RSS, which can't decode PPPoE 
> header, so packet is going only to queue 0. My question is how to force such 
> packets to spread between different queues. May be it's possible to use VMDq 
> in this configuration, or any other solution?
> 
> P.S. We tried this configuration with in-kernel driver and with latest driver 
> from Intel (ixgbe-2.0.44.14). Situation is the same.
> 
> 

What exact linux kernel version are you using ?

If hardware is not able to spread packets into multiple receive queues
because of VLAN or PPOE encapsulation then you might try RPS (not yet
included in kernel) and add appropriate
PPOE decoding in low level frame handling.

RPS v5 pointer : http://www.spinics.net/lists/netdev/msg118378.html

Re: Intel 82598, RSS and PPPoE

[Brandeburg, Jesse]

On Thu, 28 Jan 2010, Eric Dumazet wrote:

> Le jeudi 28 janvier 2010 à 13:34 +0200, Alexander Lapidus a écrit :
> > Hello, everybody!
> >  Help is needed in such case. We have a linux-box, running under CentOS 5.4 
> > with a latest kernel. Intel 10 Gigabit AF DA Dual Port Server Adapter (based 
> > on Intel 82598EB) is used for routing purposes. Linux-box is dual CPU Xeon 
> > E5530 server.  Interface eth0 is an uplink to the network core with pure IPv4 
> > trafic on it. Interface eth1 is an downlink to customers. We have 3000 802.1q 
> > VLANs on it. PPPoE server daemon is running on all VLAN interfaces. So we have 
> > ethernet frames with 802.1q VLAN tag and PPPoE header incapsulated. Our 
> > problem is with RX packet distribution between rx-queues on this  interface. 
> > All those packets are coming to RX-queue-0. As i understood from Intel 
> > specification on 82598 chipset, problem is with RSS, which can't decode PPPoE 
> > header, so packet is going only to queue 0. My question is how to force such 
> > packets to spread between different queues. May be it's possible to use VMDq 
> > in this configuration, or any other solution?
> > 
> > P.S. We tried this configuration with in-kernel driver and with latest driver 
> > from Intel (ixgbe-2.0.44.14). Situation is the same.
> > 
> > 
> 
> What exact linux kernel version are you using ?
> 
> If hardware is not able to spread packets into multiple receive queues
> because of VLAN or PPOE encapsulation then you might try RPS (not yet
> included in kernel) and add appropriate
> PPOE decoding in low level frame handling.
> 
> RPS v5 pointer : http://www.spinics.net/lists/netdev/msg118378.html

In addition or instead of RPS, using the 82598 software developers manual 
you could pretty quickly code up a queue spreading model using the vlan 
tags and VMDq mode.  I believe you can just turn on VMDq mode and get the 
vlan filtering behavior enabled with a single register write.

It appears the VFTA table has some entries in it that can control the 
queue picked for each VLAN id hash match in the table.  I think even 
filling this with an incrementing value would allow vlan packets with 
non-IP traffic to be automatically spread out to all available queues (as 
long as there was enough randomness in the hash result for the input vlan 
ids.

Hope this helps, you might want to consider sending queries like this in 
the future to e1000-devel@lists.sourceforge.net

Jesse