sky2, vlan and nat/masquerading

sky2, vlan and nat/masquerading

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 1126 bytes --]

Hello everybody,

I have a Samsung NF310, running kernel 2.6.37.3 with a patch to make my
ethernet controller work for vlans. It was discussed with the subject "sky2:
convert to new VLAN model (v0.2)" and made it to to kernel tree with commit
86aa77854f47ab6f5f9c687507af1f57d2b89004.
However it does not work properly, here are the details:

* Switch with one trunk port and several port in corresponding vlan ports
* Host connected to one of the vlan ports
* Samsung Netbook (see above) connected to the trunk port.

I get an IP address 192.168.x.x/24 via DHCP on interface connected to vlan 1.
The interface connected to vlan 2 has 172.16.0.1/24 and serves addresses via
DHCP. The system is set up to masquerade from 172.16.0.1/24.

I can access my netbook from the host in vlan 2, however I can not access
anything behind. The packets contain a broken vlan tag and the host does not
recognize them.
I've attached a tcpdump log. Please take a look at the icmp echo request and
reply packets, especially the last one.

I've tested with an usb-ethernet-adapter and everything works fine, so my
setup is ok.
-- 
Regards,
Chris

[-- Attachment #2: tcpdump.log --]
[-- Type: application/octet-stream, Size: 688 bytes --]

Re: sky2, vlan and nat/masquerading

[Jesse Gross]

On Wed, Mar 9, 2011 at 9:15 AM, Christian Hesse <mail@eworm.de> wrote:
> Hello everybody,
>
> I have a Samsung NF310, running kernel 2.6.37.3 with a patch to make my
> ethernet controller work for vlans. It was discussed with the subject "sky2:
> convert to new VLAN model (v0.2)" and made it to to kernel tree with commit
> 86aa77854f47ab6f5f9c687507af1f57d2b89004.

Does that commit actually change the behavior that you are seeing?  It
shouldn't be necessary for correct functionality.  Do you know if this
worked at some point in the past?

> However it does not work properly, here are the details:
>
> * Switch with one trunk port and several port in corresponding vlan ports
> * Host connected to one of the vlan ports
> * Samsung Netbook (see above) connected to the trunk port.
>
> I get an IP address 192.168.x.x/24 via DHCP on interface connected to vlan 1.
> The interface connected to vlan 2 has 172.16.0.1/24 and serves addresses via
> DHCP. The system is set up to masquerade from 172.16.0.1/24.
>
> I can access my netbook from the host in vlan 2, however I can not access
> anything behind. The packets contain a broken vlan tag and the host does not
> recognize them.

When you say "the host does not recognize them", what host do you
mean?  This is a different host on vlan 1?

> I've attached a tcpdump log. Please take a look at the icmp echo request and
> reply packets, especially the last one.

What do you mean by broken?  I only see one tag in the trace, which is
on the packet originating from 192.168.100.3 and it has a vid of 0.

Where was this trace captured?

Re: sky2, vlan and nat/masquerading

[Christian Hesse]

On Fri, 11 Mar 2011 16:39:02 -0800 Jesse Gross <jesse@nicira.com> wrote:
> On Wed, Mar 9, 2011 at 9:15 AM, Christian Hesse <mail@eworm.de> wrote:
> > Hello everybody,
> >
> > I have a Samsung NF310, running kernel 2.6.37.3 with a patch to make my
> > ethernet controller work for vlans. It was discussed with the subject
> > "sky2: convert to new VLAN model (v0.2)" and made it to to kernel tree
> > with commit 86aa77854f47ab6f5f9c687507af1f57d2b89004.
> 
> Does that commit actually change the behavior that you are seeing?  It
> shouldn't be necessary for correct functionality.

Plain 2.6.37 did not work at all. Received packets with vlan tag did not make
it to the vlan interface but ended on the native ethernet interface iirc.

> Do you know if this
> worked at some point in the past?

This was the first time I used nat with two vlan interfaces.

> > However it does not work properly, here are the details:
> >
> > * Switch with one trunk port and several port in corresponding vlan ports
> > * Host connected to one of the vlan ports
> > * Samsung Netbook (see above) connected to the trunk port.
> >
> > I get an IP address 192.168.x.x/24 via DHCP on interface connected to
> > vlan 1. The interface connected to vlan 2 has 172.16.0.1/24 and serves
> > addresses via DHCP. The system is set up to masquerade from 172.16.0.1/24.
> >
> > I can access my netbook from the host in vlan 2, however I can not access
> > anything behind. The packets contain a broken vlan tag and the host does
> > not recognize them.
> 
> When you say "the host does not recognize them", what host do you
> mean?  This is a different host on vlan 1?

No, this is the host in vlan2.
The packet contains the vlan tag with vid 0. It should not. The host discards
it as it does not have a vlan interface with vid 0.

> > I've attached a tcpdump log. Please take a look at the icmp echo request
> > and reply packets, especially the last one.
> 
> What do you mean by broken?  I only see one tag in the trace, which is
> on the packet originating from 192.168.100.3 and it has a vid of 0.

The tag itself is valid. But it should not be there as it comes from
a native ethernet port.

> Where was this trace captured?

This trace was captured on the host in vlan2.

Ok, let me explain step by step:

* Host sends icmp echo request (172.16.0.21 -> 192.168.100.3) to router
  172.16.0.1, the packet is untagged.
* Switch receives the packet on native interface with vid 2, tags it and sends
  it to the trunk)
* Netbook receives the packet from trunk, untags it an queues it to vlan
  interface 2.
* Netbook nats the packet (192.168.x.140 > 192.168.100.3), tags it with vlan
  2 and sends it to the trunk.
* Switch receives the packet from trunk, untags it and sends it to native
  interface with vlan 1.
* The packet and its answer (192.168.100.3 -> 192.168.x.140) make their way
  through the network.
* Switch receives the icmp echo reply on native interface with vlan 1, tags
  it and sends it to the trunk
* Netbook receives the packet from trunk, untags it an queues it to vlan
  interface 1.
* Netbooks restores the original addresses from nat (192.168.100.3 ->
  172.16.0.21), _tags_it_with_vlan_0_, tags it with vlan 2 and sends it to the
  trunk.
* Switch receives packet from trunk, untags it (the tag with vid 2) and sends
  it to native interface with vid 2.
* Host receives the packet and discards it as it still contains a vlan tag
  with vid 0.
-- 
Schoene Gruesse
Chris

Re: sky2, vlan and nat/masquerading

[Jesse Gross]

On Mon, Mar 14, 2011 at 3:11 AM, Christian Hesse <mail@eworm.de> wrote:
> On Fri, 11 Mar 2011 16:39:02 -0800 Jesse Gross <jesse@nicira.com> wrote:
>> On Wed, Mar 9, 2011 at 9:15 AM, Christian Hesse <mail@eworm.de> wrote:
>> > Hello everybody,
>> >
>> > I have a Samsung NF310, running kernel 2.6.37.3 with a patch to make my
>> > ethernet controller work for vlans. It was discussed with the subject
>> > "sky2: convert to new VLAN model (v0.2)" and made it to to kernel tree
>> > with commit 86aa77854f47ab6f5f9c687507af1f57d2b89004.
>>
>> Does that commit actually change the behavior that you are seeing?  It
>> shouldn't be necessary for correct functionality.
>
> Plain 2.6.37 did not work at all. Received packets with vlan tag did not make
> it to the vlan interface but ended on the native ethernet interface iirc.

Hmm, that is very odd.  I don't see anything in the patch that would
change behavior in that regard.

>
>> Do you know if this
>> worked at some point in the past?
>
> This was the first time I used nat with two vlan interfaces.
>
>> > However it does not work properly, here are the details:
>> >
>> > * Switch with one trunk port and several port in corresponding vlan ports
>> > * Host connected to one of the vlan ports
>> > * Samsung Netbook (see above) connected to the trunk port.
>> >
>> > I get an IP address 192.168.x.x/24 via DHCP on interface connected to
>> > vlan 1. The interface connected to vlan 2 has 172.16.0.1/24 and serves
>> > addresses via DHCP. The system is set up to masquerade from 172.16.0.1/24.
>> >
>> > I can access my netbook from the host in vlan 2, however I can not access
>> > anything behind. The packets contain a broken vlan tag and the host does
>> > not recognize them.
>>
>> When you say "the host does not recognize them", what host do you
>> mean?  This is a different host on vlan 1?
>
> No, this is the host in vlan2.
> The packet contains the vlan tag with vid 0. It should not. The host discards
> it as it does not have a vlan interface with vid 0.
>
>> > I've attached a tcpdump log. Please take a look at the icmp echo request
>> > and reply packets, especially the last one.
>>
>> What do you mean by broken?  I only see one tag in the trace, which is
>> on the packet originating from 192.168.100.3 and it has a vid of 0.
>
> The tag itself is valid. But it should not be there as it comes from
> a native ethernet port.
>
>> Where was this trace captured?
>
> This trace was captured on the host in vlan2.
>
> Ok, let me explain step by step:

Thank you, this helps a lot in understanding your setup.

>
> * Host sends icmp echo request (172.16.0.21 -> 192.168.100.3) to router
>  172.16.0.1, the packet is untagged.
> * Switch receives the packet on native interface with vid 2, tags it and sends
>  it to the trunk)
> * Netbook receives the packet from trunk, untags it an queues it to vlan
>  interface 2.
> * Netbook nats the packet (192.168.x.140 > 192.168.100.3), tags it with vlan
>  2 and sends it to the trunk.

For clarity, I'm assuming that this is supposed to be vlan 1?

> * Switch receives the packet from trunk, untags it and sends it to native
>  interface with vlan 1.
> * The packet and its answer (192.168.100.3 -> 192.168.x.140) make their way
>  through the network.
> * Switch receives the icmp echo reply on native interface with vlan 1, tags
>  it and sends it to the trunk
> * Netbook receives the packet from trunk, untags it an queues it to vlan
>  interface 1.
> * Netbooks restores the original addresses from nat (192.168.100.3 ->
>  172.16.0.21), _tags_it_with_vlan_0_, tags it with vlan 2 and sends it to the
>  trunk

Can you capture a packet trace on the netbook's Ethernet interface to
see what it thinks it is sending?

Re: sky2, vlan and nat/masquerading

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 1805 bytes --]

On Mon, 14 Mar 2011 18:55:17 -0700 Jesse Gross <jesse@nicira.com> wrote:
> On Mon, Mar 14, 2011 at 3:11 AM, Christian Hesse <mail@eworm.de> wrote:
> > Ok, let me explain step by step:
> 
> Thank you, this helps a lot in understanding your setup.
> 
> >
> > * Host sends icmp echo request (172.16.0.21 -> 192.168.100.3) to router
> >  172.16.0.1, the packet is untagged.
> > * Switch receives the packet on native interface with vid 2, tags it and
> > sends it to the trunk)
> > * Netbook receives the packet from trunk, untags it an queues it to vlan
> >  interface 2.
> > * Netbook nats the packet (192.168.x.140 > 192.168.100.3), tags it with
> > vlan 2 and sends it to the trunk.
> 
> For clarity, I'm assuming that this is supposed to be vlan 1?

Sorry, little typo. Yes, you are right.

> > * Switch receives the packet from trunk, untags it and sends it to native
> >  interface with vlan 1.
> > * The packet and its answer (192.168.100.3 -> 192.168.x.140) make their
> > way through the network.
> > * Switch receives the icmp echo reply on native interface with vlan 1,
> > tags it and sends it to the trunk
> > * Netbook receives the packet from trunk, untags it an queues it to vlan
> >  interface 1.
> > * Netbooks restores the original addresses from nat (192.168.100.3 ->
> >  172.16.0.21), _tags_it_with_vlan_0_, tags it with vlan 2 and sends it to
> > the trunk
> 
> Can you capture a packet trace on the netbook's Ethernet interface to
> see what it thinks it is sending?

Ok, I have two traces for you: from the vlan interface and from the native
interface. First ping to 172.16.0.65 is ok, second one to 192.168.100.3 fails.

Please don't be confused, vlan 1 is vlan 3 this time and addresses
changed a little bit. ;)
-- 
Schoene Gruesse
Chris

[-- Attachment #2: tcpdump-eth.2.log --]
[-- Type: application/octet-stream, Size: 480 bytes --]

[-- Attachment #3: tcpdump-eth.log --]
[-- Type: application/octet-stream, Size: 740 bytes --]

Re: sky2, vlan and nat/masquerading

[Christian Hesse]

On Tue, 15 Mar 2011 08:53:26 +0100 Christian Hesse <mail@eworm.de> wrote:
> Ok, I have two traces for you: from the vlan interface and from the native
> interface. First ping to 172.16.0.65 is ok, second one to 192.168.100.3
> fails.

BTW, this is with plain 2.6.38-rc8 now.
-- 
Schoene Gruesse
Chris

Re: sky2, vlan and nat/masquerading

[Jesse Gross]

On Tue, Mar 15, 2011 at 12:53 AM, Christian Hesse <mail@eworm.de> wrote:
> On Mon, 14 Mar 2011 18:55:17 -0700 Jesse Gross <jesse@nicira.com> wrote:
>> On Mon, Mar 14, 2011 at 3:11 AM, Christian Hesse <mail@eworm.de> wrote:
>> > Ok, let me explain step by step:
>>
>> Thank you, this helps a lot in understanding your setup.
>>
>> >
>> > * Host sends icmp echo request (172.16.0.21 -> 192.168.100.3) to router
>> >  172.16.0.1, the packet is untagged.
>> > * Switch receives the packet on native interface with vid 2, tags it and
>> > sends it to the trunk)
>> > * Netbook receives the packet from trunk, untags it an queues it to vlan
>> >  interface 2.
>> > * Netbook nats the packet (192.168.x.140 > 192.168.100.3), tags it with
>> > vlan 2 and sends it to the trunk.
>>
>> For clarity, I'm assuming that this is supposed to be vlan 1?
>
> Sorry, little typo. Yes, you are right.
>
>> > * Switch receives the packet from trunk, untags it and sends it to native
>> >  interface with vlan 1.
>> > * The packet and its answer (192.168.100.3 -> 192.168.x.140) make their
>> > way through the network.
>> > * Switch receives the icmp echo reply on native interface with vlan 1,
>> > tags it and sends it to the trunk
>> > * Netbook receives the packet from trunk, untags it an queues it to vlan
>> >  interface 1.
>> > * Netbooks restores the original addresses from nat (192.168.100.3 ->
>> >  172.16.0.21), _tags_it_with_vlan_0_, tags it with vlan 2 and sends it to
>> > the trunk
>>
>> Can you capture a packet trace on the netbook's Ethernet interface to
>> see what it thinks it is sending?
>
> Ok, I have two traces for you: from the vlan interface and from the native
> interface. First ping to 172.16.0.65 is ok, second one to 192.168.100.3 fails.
>
> Please don't be confused, vlan 1 is vlan 3 this time and addresses
> changed a little bit. ;)

Hmm, it's pretty interesting that the extra vlan tag magically
appears.  I'll have to reproduce it to investigate further, as the
source isn't readily obvious to me.  You said that if you swap out a
different NIC but keep everything else the same the problem goes away?
 That also is strange because the packet capture should take place
before the driver.

Can you try using ethtool to turn off txvlan and see if that makes a difference?

Re: sky2, vlan and nat/masquerading

[Christian Hesse]

On Thu, 17 Mar 2011 12:23:48 -0700 Jesse Gross <jesse@nicira.com> wrote:
> Hmm, it's pretty interesting that the extra vlan tag magically
> appears.  I'll have to reproduce it to investigate further, as the
> source isn't readily obvious to me.  You said that if you swap out a
> different NIC but keep everything else the same the problem goes away?
>  That also is strange because the packet capture should take place
> before the driver.

Correctly. I tested with asix and mcs7830.

> Can you try using ethtool to turn off txvlan and see if that makes a
> difference?

Sure. I will give it a try tomorrow.
-- 
Schoene Gruesse
Chris

Re: sky2, vlan and nat/masquerading

[Christian Hesse]

On Thu, 17 Mar 2011 22:40:44 +0100 Christian Hesse <mail@eworm.de> wrote:
> On Thu, 17 Mar 2011 12:23:48 -0700 Jesse Gross <jesse@nicira.com> wrote:
> > Can you try using ethtool to turn off txvlan and see if that makes a
> > difference?
> 
> Sure. I will give it a try tomorrow.

No, that does not make a difference.
-- 
Schoene Gruesse
Chris

Re: sky2, vlan and nat/masquerading

[Jesse Gross]

On Thu, Mar 17, 2011 at 11:46 PM, Christian Hesse <mail@eworm.de> wrote:
> On Thu, 17 Mar 2011 22:40:44 +0100 Christian Hesse <mail@eworm.de> wrote:
>> On Thu, 17 Mar 2011 12:23:48 -0700 Jesse Gross <jesse@nicira.com> wrote:
>> > Can you try using ethtool to turn off txvlan and see if that makes a
>> > difference?
>>
>> Sure. I will give it a try tomorrow.
>
> No, that does not make a difference.

Hmm, this problem seems very strange to me.  I'm not sure that there
is much more I can do without reproducing the problem and it sounds
like I won't be able to do that on my hardware.

If you're able to, the best thing to do would be insert some debugging
statements along the transmit path to find out exactly where the
additional vlan tag is being inserted.  Although it only happens with
a single driver, it must be occurring in software because the tag
shows up in tcpdump on the transmit side.