[Bridge] vlan and bond integration in bridge

[Bridge] vlan and bond integration in bridge

[Lv Zheng]

Hi, all

I'm new to this list.

I'm trying to build a 5 ports ethernet switch using Linux.
There's hardware VLAN aware switch engine in the appliance
forwarding packets between these ports, link aggregation
is also supported in the switch chip.

Following functionalities should is required for the appliance:

802.1D: require userspace RSTP
802.1Q: MSTP/tag based VLAN is not required currently,
        but port based VLAN is required
802.3ad: auto negotiated link aggregation

Since VLAN device in Linux cannot support VID duplication
across multiple lowers, that is, following commands will fail:

vconfig set_name_type DEV_PLUS_VID_NO_PAD
vconfig add eth0 1
vconfig add eth1 1 <--- failure

I have to create netdevices as follows:
+--------------------+-------------+
| vlan1              | vlan2       |
+--------------------+-------------+
|          br0       :             |
+-------------+------+------+------+
| bond0       | eth2 | eth3 | eth4 |
+------+------+      |      |      |
| eth0 | eth1 |      |      |      |
+------+------+------+------+------+
vlans are built on top of br, eth0/1/2 should be assigned
to vlan1, eth3/4 should be assigned to vlan2, eth0/1 may
be manually aggregated.

I have encountered following problems:

1. Are there any hw-accel mechanism that allows
   replacement of the software FDB in br with switch
   chip's MAC table.
   As you may know, without hw-accel, problems will occur
   to learn frames.
   When MAC based authentication (802.1X, etc.) support is
   required, every learn frames received by switch chip
   should also be copied to CPU.  If br handled every
   learn frame, br module's FDB might grow too large to
   maintain in case that the switch chip is running in a
   limited memory system.
2. There's VLAN packet hw-accel in VLAN module, which is
   supported by bond. But without VLAN hw-accel support
   in br, the hw-accel will be broken in the br layer.
3. br do not support auto negotiated link aggregation.
   IMO, br should be bond aware, LACP may be handled by
   br through a hook between br and bond.  br then can
   dynamically combine multiple physical eth devices to
   single logical bond device.

Are there any existing implementations that can solve
these issues, or any other solutions to avoid such
problems?

Best regards/Lv Zheng

Re: [Bridge] vlan and bond integration in bridge

[Ben Greear]

Lv Zheng wrote:

> Since VLAN device in Linux cannot support VID duplication
> across multiple lowers, that is, following commands will fail:
> 
> vconfig set_name_type DEV_PLUS_VID_NO_PAD
> vconfig add eth0 1
> vconfig add eth1 1 <--- failure

This should work fine..what are the errors?

Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: [Bridge] vlan and bond integration in bridge

[richardvoigt]

On Tue, Jan 6, 2009 at 5:19 PM, Ben Greear <greearb@candelatech.com> wrote:
> Lv Zheng wrote:
>
>> Since VLAN device in Linux cannot support VID duplication
>> across multiple lowers, that is, following commands will fail:
>>
>> vconfig set_name_type DEV_PLUS_VID_NO_PAD
>> vconfig add eth0 1
>> vconfig add eth1 1 <--- failure
>
> This should work fine..what are the errors?

And what kernel version is being discussed?  Many devices matching
"hardware VLAN aware switch engine in the appliance" are still on
2.4.x

>
> Ben
>
> --
> Ben Greear <greearb@candelatech.com>
> Candela Technologies Inc  http://www.candelatech.com
>
> _______________________________________________
> Bridge mailing list
> Bridge@lists.linux-foundation.org
> https://lists.linux-foundation.org/mailman/listinfo/bridge
>

Re: [Bridge] vlan and bond integration in bridge

[Lv Zheng]

[-- Attachment #1: Type: text/plain, Size: 2234 bytes --]

>> vconfig set_name_type DEV_PLUS_VID_NO_PAD

I made a mistake here. As we want single
management interface per VLAN (single L3
management address per VLAN, we will run
single L3 authentication service instance
-- such as DHCP -- per VLAN), we tried
VLAN_PLUS_VID_NO_PAD.

Using name_type as DEV_PLUS_VID or
DEV_PLUS_VID_NO_PAD will end up without
failures, but we will get tow management
interface for single VLAN: eth0.1/eth1.1

Failure screen shot is as follows:

root@zlv:/root# vconfig set_name_type VLAN_PLUS_VID_NO_PAD
Set name-type for VLAN subsystem. Should be visible
in /proc/net/vlan/config
root@zlv:/root# vconfig add eth0 1
Added VLAN with VID == 1 to IF -:eth0:-
WARNING:  VLAN 1 does not work with many switches,
consider another number if you have problems.
root@zlv:/root# vconfig add eth1 1
ERROR: trying to add VLAN #1 to IF -:eth1:-  error: Invalid argument
root@zlv:/root# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:E0:4C:17:05:4F
          inet addr:192.168.100.100  Bcast:192.168.100.255
Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:495 errors:0 dropped:0 overruns:0 frame:0
          TX packets:48 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:51480 (50.2 KiB)  TX bytes:4333 (4.2 KiB)
          Interrupt:169 Base address:0xd800

eth1      Link encap:Ethernet  HWaddr 00:14:2A:DE:A9:FC
          inet addr:10.241.110.78  Bcast:10.241.110.255
Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8204 errors:0 dropped:0 overruns:0 frame:0
          TX packets:501 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1244653 (1.1 MiB)  TX bytes:40928 (39.9 KiB)
          Interrupt:177 Base address:0xd400

vlan1     Link encap:Ethernet  HWaddr 00:E0:4C:17:05:4F
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

Best regards/Zetalog

[-- Attachment #2: Type: text/html, Size: 5452 bytes --]

Re: [Bridge] vlan and bond integration in bridge

[Lv Zheng]

> And what kernel version is being discussed?  Many devices matching
> "hardware VLAN aware switch engine in the appliance" are still on
> 2.4.x

We are building our application on stock kernel 2.6.23.x.

Best regards/Lv Zheng

Re: [Bridge] vlan and bond integration in bridge

[Ben Greear]

Lv Zheng wrote:
> >> vconfig set_name_type DEV_PLUS_VID_NO_PAD
>  
> I made a mistake here. As we want single
> management interface per VLAN (single L3
> management address per VLAN, we will run
> single L3 authentication service instance
> -- such as DHCP -- per VLAN), we tried
> VLAN_PLUS_VID_NO_PAD.

Maybe leave the naming as DEV_PLUS_VID and then add all of those VLAN 
interfaces
to a bridge and put your mgt IP on that bridge?

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com> 
Candela Technologies Inc  http://www.candelatech.com

Re: [Bridge] vlan and bond integration in bridge

[Lv Zheng]

> Maybe leave the naming as DEV_PLUS_VID and then add all of those VLAN 
> interfaces
> to a bridge and put your mgt IP on that bridge?

Do you mean:

+---------------------------+-----------------+
|          br0              |      br1        |
+------------------+--------------------------+
| bond0.1          | eth2.1 | eth3.2 | eth4.2 |
+------------------+--------+--------+--------+
| bond0            | eth2   | eth3   | eth4   |
+---------+--------+        |        |        |
| eth0    | eth1   |        |        |        |
+---------+--------+--------+--------+--------+

It looks good for port based VLAN. Thanks.

For tag based VLAN, the topology will looks like:

+---------------------------+--------------------------+
|          br0              |          br1             |
+------------------+--------+--------+-----------------+
| bond0.1          | eth2.1 | eth2.2 | eth3.2 | eth4.2 |
+------------------+--------+--------+--------+--------+
| bond0            | eth2            | eth3   | eth4   |
+---------+--------+                 |        |        |
| eth0    | eth1   |                 |        |        |
+---------+--------+-----------------+--------+--------+

port eth2 will get tow STP instances running on top of it,
this might confuse it's MAC partner if MSTP is not supported.
How could Linux solve this?

Best regards/Lv Zheng

Re: [Bridge] vlan and bond integration in bridge

[Boris Dores]

On Thu, Jan 08, 2009 at 09:47:58AM (GMT+0800), Lv Zheng wrote:
> +---------------------------+-----------------+
> |          br0              |      br1        |
> +------------------+--------------------------+
> | bond0.1          | eth2.1 | eth3.2 | eth4.2 |
> +------------------+--------+--------+--------+
> | bond0            | eth2   | eth3   | eth4   |
> +---------+--------+        |        |        |
> | eth0    | eth1   |        |        |        |
> +---------+--------+--------+--------+--------+
> 
> It looks good for port based VLAN. Thanks.

  For port based VLAN, I think it should rather simply be :

+---------------------------+-----------------+
|          br0              |      br1        |
+------------------+--------+--------+--------+
| bond0            | eth2   | eth3   | eth4   |
+---------+--------+        |        |        |
| eth0    | eth1   |        |        |        |
+---------+--------+--------+--------+--------+

  without the VLAN layer, otherwise packets will be tagged.

-- 
Boris Dorès

Re: [Bridge] vlan and bond integration in bridge

[Lv Zheng]

> +---------------------------+-----------------+
> |          br0              |      br1        |
> +------------------+--------+--------+--------+
> | bond0            | eth2   | eth3   | eth4   |
> +---------+--------+        |        |        |
> | eth0    | eth1   |        |        |        |
> +---------+--------+--------+--------+--------+
> 
>  without the VLAN layer, otherwise packets will be tagged.

Do you mean BPDU will also be tagged and no longer a BPDU
on the wire?

Re: [Bridge] vlan and bond integration in bridge

[Boris Dores]

On Thu, Jan 08, 2009 at 06:13:07PM (GMT+0800), Lv Zheng wrote:
> >+---------------------------+-----------------+
> >|          br0              |      br1        |
> >+------------------+--------+--------+--------+
> >| bond0            | eth2   | eth3   | eth4   |
> >+---------+--------+        |        |        |
> >| eth0    | eth1   |        |        |        |
> >+---------+--------+--------+--------+--------+
> >
> > without the VLAN layer, otherwise packets will be tagged.
> 
> Do you mean BPDU will also be tagged and no longer a BPDU
> on the wire?

  Well, I'm not sure about BPDU packets, although I don't see why they
would be treated differently (but someone else on this list must have a
definite answer).

  I was talking about regular data packets. Usually, when refering to
"port based" VLAN, we mean that the host (or the hosts) connected to
that port has no knowledge whatsoever of what a VLAN is.

  If (and only if) this is what you want, then br0 should be directly
connected to the physical port (bond or eth), and not on top of a vlan
layer which will filter out untagged incoming packets, and tag outgoing
packets.

-- 
Boris Dorès

Re: [Bridge] vlan and bond integration in bridge

[Lv Zheng]

>  I was talking about regular data packets. Usually, when refering to
> "port based" VLAN, we mean that the host (or the hosts) connected to
> that port has no knowledge whatsoever of what a VLAN is.
> 
>  If (and only if) this is what you want, then br0 should be directly
> connected to the physical port (bond or eth), and not on top of a vlan
> layer which will filter out untagged incoming packets, and tag outgoing
> packets.

Greetings.

802.1Q defines that each bridge port

shall support following parameters:
1. acceptable frame types, one of the following:
  A. admit only VLAN tagged frames
  B. admit only untagged and priority-tagged frames
  C. admit all frames (default)
2. a PVID for port based VLAN classification
may support following parameters:
3. a VID set for port and protocol based classification

For port based VLAN, a netdevice might still be
VLAN-aware and at least 1 & 2 should be configurable
for the netdevice (known as EISS support).

I noticed following differences are exist between
802.1Q compatible switch chip's MAC port devices
and a normal NIC devices:

Switch port do support 802.1Q parameters will
1. drop untagged frames if it is configured to admit
  only VLAN tagged frames and drop frames not
  admitted
2. drop any tagged frames whose tag is not in the
  VID set if it is configured to admit only VLAN
  tagged frames and drop frames not admitted
3. handle any untagged frames as PVID tagged frames
  if it is configured to admit untagged and
  priority-tagged frames and handle such frames as
  if they are coming from the default vlan
4. handle any tagged frames whose tag is not in the
  VID set as PVID tagged frames if it is configured
  to admit untagged and priority-tagged frames and
  handle such frames as if they are coming from the
  default vlan

While Linux does not support such features.

Should following attributes be added to netdevice
to support port based VLAN?
1. pvid
2. admit_frame_types
  VLAN ioctl command may be added for all
  netdevice (like ADD_VLAN_CMD) to
  manipulate above 2 vlan specific parameters:
  SET_PVID_CMD
  SET_FRAME_TYPES_CMD
  SET_FRAME_ACTIONS_CMD
  Addtional hooks might also be added to
  netdevice:
  vlan_set_pvid
  vlan_set_admit_frames (types & actions)
  if frames is not admitted, actions might be configured
  to allow:
  2.1 DROP: drop the frame
  2.2 DEF_VLAN: handle the frame as if it is pvid tagged
3. vlan_nr_groups:
  set by NIC driver, its default value might be a Kconfig
  option such as CONFIG_VLAN_8021Q_GROUPS
4. vlan_gid_map
  GID enabling bits, might be BITMAP(vlan_nr_groups)
5. vlan_groups
  VID <-> GID mappings, might be u16[vlan_nr_groups]
  register_vlan_device / unregister_vlan_device
  might maintain this map and gid array

If hardware do support 802.1Q parameters, it might
report vlan_nr_groups on registration.
Hardware vlan_gid_map and vlan_groups might be
synchronized by supporting
vlan_rx_add_vid / vlan_rx_kill_vid.
Hardware PVID and admit_frame_types might be
synchronized by supporting
vlan_set_pvid / vlan_set_admit_frames.

If PVID in netdevice was implemented, there might be
tow options we could choose:
1. ensure there is always a ethx.pvid device exist
  might be suitable for tag based switch applications
2. ensure there is always not a ethx.pvid device exist
  might be suitable for port based switch applications

vlan_skb_recv might be modified to support EISS features
because this function will handle all VLAN-tagged frames

netif_recv_skb might be modified to support EISS features
because this function will handle all VLAN-untagged frames

dev_hard_start_xmit might be modified to support EISS
feauturs to do tagging on outgoing packets before hard_xmit
if pvid is set and no ethx.pvid device exists.

If I were wrong, pls let me know. Thanks in advance.

Best regards/Lv Zheng

Re: [Bridge] vlan and bond integration in bridge

[Simon Barber]

I did release a patch to do most of this a while ago. It was for the  
2.4 kernel series, but could be updated for 2.6 without too much  
trouble I'm sure. Search the list archives about a couple of years ago.

Simon

Sent from my iPhone

On Jan 12, 2009, at 4:41 AM, "Lv Zheng" <lv.zheng@soliton.com.cn> wrote:

>> I was talking about regular data packets. Usually, when refering to
>> "port based" VLAN, we mean that the host (or the hosts) connected to
>> that port has no knowledge whatsoever of what a VLAN is.
>>
>> If (and only if) this is what you want, then br0 should be directly
>> connected to the physical port (bond or eth), and not on top of a  
>> vlan
>> layer which will filter out untagged incoming packets, and tag  
>> outgoing
>> packets.
>
> Greetings.
>
> 802.1Q defines that each bridge port
>
> shall support following parameters:
> 1. acceptable frame types, one of the following:
>  A. admit only VLAN tagged frames
>  B. admit only untagged and priority-tagged frames
>  C. admit all frames (default)
> 2. a PVID for port based VLAN classification
> may support following parameters:
> 3. a VID set for port and protocol based classification
>
> For port based VLAN, a netdevice might still be
> VLAN-aware and at least 1 & 2 should be configurable
> for the netdevice (known as EISS support).
>
> I noticed following differences are exist between
> 802.1Q compatible switch chip's MAC port devices
> and a normal NIC devices:
>
> Switch port do support 802.1Q parameters will
> 1. drop untagged frames if it is configured to admit
>  only VLAN tagged frames and drop frames not
>  admitted
> 2. drop any tagged frames whose tag is not in the
>  VID set if it is configured to admit only VLAN
>  tagged frames and drop frames not admitted
> 3. handle any untagged frames as PVID tagged frames
>  if it is configured to admit untagged and
>  priority-tagged frames and handle such frames as
>  if they are coming from the default vlan
> 4. handle any tagged frames whose tag is not in the
>  VID set as PVID tagged frames if it is configured
>  to admit untagged and priority-tagged frames and
>  handle such frames as if they are coming from the
>  default vlan
>
> While Linux does not support such features.
>
> Should following attributes be added to netdevice
> to support port based VLAN?
> 1. pvid
> 2. admit_frame_types
>  VLAN ioctl command may be added for all
>  netdevice (like ADD_VLAN_CMD) to
>  manipulate above 2 vlan specific parameters:
>  SET_PVID_CMD
>  SET_FRAME_TYPES_CMD
>  SET_FRAME_ACTIONS_CMD
>  Addtional hooks might also be added to
>  netdevice:
>  vlan_set_pvid
>  vlan_set_admit_frames (types & actions)
>  if frames is not admitted, actions might be configured
>  to allow:
>  2.1 DROP: drop the frame
>  2.2 DEF_VLAN: handle the frame as if it is pvid tagged
> 3. vlan_nr_groups:
>  set by NIC driver, its default value might be a Kconfig
>  option such as CONFIG_VLAN_8021Q_GROUPS
> 4. vlan_gid_map
>  GID enabling bits, might be BITMAP(vlan_nr_groups)
> 5. vlan_groups
>  VID <-> GID mappings, might be u16[vlan_nr_groups]
>  register_vlan_device / unregister_vlan_device
>  might maintain this map and gid array
>
> If hardware do support 802.1Q parameters, it might
> report vlan_nr_groups on registration.
> Hardware vlan_gid_map and vlan_groups might be
> synchronized by supporting
> vlan_rx_add_vid / vlan_rx_kill_vid.
> Hardware PVID and admit_frame_types might be
> synchronized by supporting
> vlan_set_pvid / vlan_set_admit_frames.
>
> If PVID in netdevice was implemented, there might be
> tow options we could choose:
> 1. ensure there is always a ethx.pvid device exist
>  might be suitable for tag based switch applications
> 2. ensure there is always not a ethx.pvid device exist
>  might be suitable for port based switch applications
>
> vlan_skb_recv might be modified to support EISS features
> because this function will handle all VLAN-tagged frames
>
> netif_recv_skb might be modified to support EISS features
> because this function will handle all VLAN-untagged frames
>
> dev_hard_start_xmit might be modified to support EISS
> feauturs to do tagging on outgoing packets before hard_xmit
> if pvid is set and no ethx.pvid device exists.
>
> If I were wrong, pls let me know. Thanks in advance.
>
> Best regards/Lv Zheng
> _______________________________________________
> Bridge mailing list
> Bridge@lists.linux-foundation.org
> https://lists.linux-foundation.org/mailman/listinfo/bridge

Re: [Bridge] vlan and bond integration in bridge

[Benny Amorsen]

"Lv Zheng" <lv.zheng@soliton.com.cn> writes:

> Switch port do support 802.1Q parameters will
> 1. drop untagged frames if it is configured to admit
>   only VLAN tagged frames and drop frames not
>   admitted
> 2. drop any tagged frames whose tag is not in the
>   VID set if it is configured to admit only VLAN
>   tagged frames and drop frames not admitted
> 3. handle any untagged frames as PVID tagged frames
>   if it is configured to admit untagged and
>   priority-tagged frames and handle such frames as
>   if they are coming from the default vlan
> 4. handle any tagged frames whose tag is not in the
>   VID set as PVID tagged frames if it is configured
>   to admit untagged and priority-tagged frames and
>   handle such frames as if they are coming from the
>   default vlan
>
> While Linux does not support such features.

I would prefer if Linux continued to not support those features. As it
is, you can get the untagged frames at eth0, the 100 tagged ones as
eth0.100, the 100+101 tagges ones as eth0.100.101 and so on. Linux
doesn't suffer from global VLAN significance and other switch
silliness, and I really hope it stays that way.

The only challenge is spanning tree, because Cisco figured out too
late that spanning tree packets should be per-VLAN. The sane solution
is to simply tag BPDU packets like all other packets. Linux does that,
but the rest of the world is stuck with MST or PVST.


/Benny

Re: [Bridge] vlan and bond integration in bridge

[Lv Zheng]

> but the rest of the world is stuck with MST or PVST.

Thanks for your suggestion, we will try Linux solution in
our appliance.

Best regards/Lv Zheng