[BUG] vlan-aware bridge breaks vlan on another port on same gmac

[BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Frank Wunderlich]

Hi,

noticed a bug while testing systemd, but it is reproducable with iproute2

tested on bananapi-r2 with kernel 5.15.80 and bananapi-r3 with kernel 6.2-rc1,
both use mt7530 dsa driver but different configs (mt7530 vs. mt7531).
have no other devices to test.

first create vlan on wan-port (wan and lan0 are dsa-user-ports on same gmac)

netif=wan
ip link set $netif up
ip link add link $netif name vlan110 type vlan id 110
ip link set vlan110 up
ip addr add 192.168.110.1/24 dev vlan110

vlan works now, other side pingable, vlan-tagged packets visible in tcpdump on both sides

now create the vlan-sware bridge (without vlan_filtering it works in my test)

BRIDGE=lanbr0
ip link add name ${BRIDGE} type bridge vlan_filtering 1 vlan_default_pvid 1
ip link set ${BRIDGE} up
ip link set lan0 master ${BRIDGE}
ip link set lan0 up

takes some time before it is applied and ping got lost

packets are received by other end but without vlan-tag

regards Frank

Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Vladimir Oltean]

Hi Frank,

On Fri, Jan 20, 2023 at 05:45:35PM +0100, Frank Wunderlich wrote:
> Hi,
> 
> noticed a bug while testing systemd, but it is reproducable with iproute2
> 
> tested on bananapi-r2 with kernel 5.15.80 and bananapi-r3 with kernel 6.2-rc1,
> both use mt7530 dsa driver but different configs (mt7530 vs. mt7531).
> have no other devices to test.
> 
> first create vlan on wan-port (wan and lan0 are dsa-user-ports on same gmac)
> 
> netif=wan
> ip link set $netif up
> ip link add link $netif name vlan110 type vlan id 110
> ip link set vlan110 up
> ip addr add 192.168.110.1/24 dev vlan110
> 
> vlan works now, other side pingable, vlan-tagged packets visible in tcpdump on both sides

VLAN 110 is a software VLAN, it is never committed to hardware in the
switch.

> now create the vlan-sware bridge (without vlan_filtering it works in my test)
> 
> BRIDGE=lanbr0
> ip link add name ${BRIDGE} type bridge vlan_filtering 1 vlan_default_pvid 1
> ip link set ${BRIDGE} up
> ip link set lan0 master ${BRIDGE}
> ip link set lan0 up
> 
> takes some time before it is applied and ping got lost
> 
> packets are received by other end but without vlan-tag

What happens in mt7530_port_vlan_filtering() is that the user port (lan0)
*and* the CPU port become VLAN aware. I guess it is the change on the
CPU port that affects the traffic to "wan". But I don't see yet why this
affects the traffic in the way you mention (the CPU port strips the tag
instead of dropping packets with VLAN 110).

I have 2 random things to suggest you try.

First is this

From 2991f704e6f341bd81296e91fbb4381f528f8c7f Mon Sep 17 00:00:00 2001
From: Vladimir Oltean <vladimir.oltean@nxp.com>
Date: Fri, 20 Jan 2023 19:17:16 +0200
Subject: [PATCH] mt7530 don't make the CPU port a VLAN user port

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 drivers/net/dsa/mt7530.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/dsa/mt7530.c b/drivers/net/dsa/mt7530.c
index 616b21c90d05..7265c120c767 100644
--- a/drivers/net/dsa/mt7530.c
+++ b/drivers/net/dsa/mt7530.c
@@ -1524,7 +1524,7 @@ mt7530_port_vlan_filtering(struct dsa_switch *ds, int port, bool vlan_filtering,
 		 * for becoming a VLAN-aware port.
 		 */
 		mt7530_port_set_vlan_aware(ds, port);
-		mt7530_port_set_vlan_aware(ds, cpu_dp->index);
+//		mt7530_port_set_vlan_aware(ds, cpu_dp->index);
 	} else {
 		mt7530_port_set_vlan_unaware(ds, port);
 	}

If this works, I expect it will break VLAN tagged traffic over lan0 now :)
So I would then like you to remove the first patch and try the next one

From 1b6842c8fc57f6fda28db576170173f5c146e470 Mon Sep 17 00:00:00 2001
From: Vladimir Oltean <vladimir.oltean@nxp.com>
Date: Fri, 20 Jan 2023 19:17:51 +0200
Subject: [PATCH 2/2] tag_mtk only combine VLAN tag with MTK tag is user port
 is VLAN aware

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 net/dsa/tag_mtk.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/net/dsa/tag_mtk.c b/net/dsa/tag_mtk.c
index 40af80452747..ab027c233bee 100644
--- a/net/dsa/tag_mtk.c
+++ b/net/dsa/tag_mtk.c
@@ -35,14 +35,13 @@ static struct sk_buff *mtk_tag_xmit(struct sk_buff *skb,
 	 * the both special and VLAN tag at the same time and then look up VLAN
 	 * table with VID.
 	 */
-	switch (skb->protocol) {
-	case htons(ETH_P_8021Q):
+	if (dsa_port_is_vlan_filtering(dp) &&
+	    skb->protocol == htons(ETH_P_8021Q)) {
 		xmit_tpid = MTK_HDR_XMIT_TAGGED_TPID_8100;
-		break;
-	case htons(ETH_P_8021AD):
+	} else if (dsa_port_is_vlan_filtering(dp) &&
+		   skb->protocol == htons(ETH_P_8021AD)) {
 		xmit_tpid = MTK_HDR_XMIT_TAGGED_TPID_88A8;
-		break;
-	default:
+	} else {
 		xmit_tpid = MTK_HDR_XMIT_UNTAGGED;
 		skb_push(skb, MTK_HDR_LEN);
 		dsa_alloc_etype_header(skb, MTK_HDR_LEN);

Re: [PATCH] mt7530 don't make the CPU port a VLAN user port

[kernel test robot]

Hi Vladimir,

I love your patch! Perhaps something to improve:

[auto build test WARNING on linus/master]
[also build test WARNING on v6.2-rc4 next-20230120]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Vladimir-Oltean/mt7530-don-t-make-the-CPU-port-a-VLAN-user-port/20230121-012326
patch link:    https://lore.kernel.org/r/20230120172132.rfo3kf4fmkxtw4cl%40skbuf
patch subject: [PATCH] mt7530 don't make the CPU port a VLAN user port
config: i386-randconfig-a005 (https://download.01.org/0day-ci/archive/20230121/202301210746.xjCzhDc3-lkp@intel.com/config)
compiler: gcc-11 (Debian 11.3.0-8) 11.3.0
reproduce (this is a W=1 build):
        # https://github.com/intel-lab-lkp/linux/commit/563aaac5703ee8530858faf04f93f30785d74cf6
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Vladimir-Oltean/mt7530-don-t-make-the-CPU-port-a-VLAN-user-port/20230121-012326
        git checkout 563aaac5703ee8530858faf04f93f30785d74cf6
        # save the config file
        mkdir build_dir && cp config build_dir/.config
        make W=1 O=build_dir ARCH=i386 olddefconfig
        make W=1 O=build_dir ARCH=i386 SHELL=/bin/bash drivers/net/dsa/

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

   drivers/net/dsa/mt7530.c: In function 'mt7530_port_vlan_filtering':
>> drivers/net/dsa/mt7530.c:1525:26: warning: unused variable 'cpu_dp' [-Wunused-variable]
    1525 |         struct dsa_port *cpu_dp = dp->cpu_dp;
         |                          ^~~~~~


vim +/cpu_dp +1525 drivers/net/dsa/mt7530.c

83163f7dca5684 Sean Wang        2017-12-15  1519  
83163f7dca5684 Sean Wang        2017-12-15  1520  static int
89153ed6ebc148 Vladimir Oltean  2021-02-13  1521  mt7530_port_vlan_filtering(struct dsa_switch *ds, int port, bool vlan_filtering,
89153ed6ebc148 Vladimir Oltean  2021-02-13  1522  			   struct netlink_ext_ack *extack)
83163f7dca5684 Sean Wang        2017-12-15  1523  {
1f9a6abecf538c Frank Wunderlich 2022-06-10  1524  	struct dsa_port *dp = dsa_to_port(ds, port);
1f9a6abecf538c Frank Wunderlich 2022-06-10 @1525  	struct dsa_port *cpu_dp = dp->cpu_dp;
1f9a6abecf538c Frank Wunderlich 2022-06-10  1526  
83163f7dca5684 Sean Wang        2017-12-15  1527  	if (vlan_filtering) {
83163f7dca5684 Sean Wang        2017-12-15  1528  		/* The port is being kept as VLAN-unaware port when bridge is
83163f7dca5684 Sean Wang        2017-12-15  1529  		 * set up with vlan_filtering not being set, Otherwise, the
83163f7dca5684 Sean Wang        2017-12-15  1530  		 * port and the corresponding CPU port is required the setup
83163f7dca5684 Sean Wang        2017-12-15  1531  		 * for becoming a VLAN-aware port.
83163f7dca5684 Sean Wang        2017-12-15  1532  		 */
83163f7dca5684 Sean Wang        2017-12-15  1533  		mt7530_port_set_vlan_aware(ds, port);
563aaac5703ee8 Vladimir Oltean  2023-01-20  1534  //		mt7530_port_set_vlan_aware(ds, cpu_dp->index);
e3ee07d14fac20 Vladimir Oltean  2019-04-28  1535  	} else {
e3ee07d14fac20 Vladimir Oltean  2019-04-28  1536  		mt7530_port_set_vlan_unaware(ds, port);
83163f7dca5684 Sean Wang        2017-12-15  1537  	}
83163f7dca5684 Sean Wang        2017-12-15  1538  
83163f7dca5684 Sean Wang        2017-12-15  1539  	return 0;
83163f7dca5684 Sean Wang        2017-12-15  1540  }
83163f7dca5684 Sean Wang        2017-12-15  1541  

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests

Aw: Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Frank Wunderlich]

Hi

first thanks for the really fast and detailed answer.

> Gesendet: Freitag, 20. Januar 2023 um 18:21 Uhr
> Von: "Vladimir Oltean" <olteanv@gmail.com>
> An: "Frank Wunderlich" <frank-w@public-files.de>
> Cc: "Andrew Lunn" <andrew@lunn.ch>, "Florian Fainelli" <f.fainelli@gmail.com>, "David S. Miller" <davem@davemloft.net>, "Eric Dumazet" <edumazet@google.com>, "Jakub Kicinski" <kuba@kernel.org>, "Paolo Abeni" <pabeni@redhat.com>, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, "Landen Chao" <Landen.Chao@mediatek.com>, "Sean Wang" <sean.wang@mediatek.com>, "DENG Qingfang" <dqfext@gmail.com>, "Matthias Brugger" <matthias.bgg@gmail.com>, "Daniel Golle" <daniel@makrotopia.org>
> Betreff: Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac
>
> Hi Frank,
>
> On Fri, Jan 20, 2023 at 05:45:35PM +0100, Frank Wunderlich wrote:
> > Hi,
> >
> > noticed a bug while testing systemd, but it is reproducable with iproute2
> >
> > tested on bananapi-r2 with kernel 5.15.80 and bananapi-r3 with kernel 6.2-rc1,
> > both use mt7530 dsa driver but different configs (mt7530 vs. mt7531).
> > have no other devices to test.
> >
> > first create vlan on wan-port (wan and lan0 are dsa-user-ports on same gmac)
> >
> > netif=wan
> > ip link set $netif up
> > ip link add link $netif name vlan110 type vlan id 110
> > ip link set vlan110 up
> > ip addr add 192.168.110.1/24 dev vlan110
> >
> > vlan works now, other side pingable, vlan-tagged packets visible in tcpdump on both sides
>
> VLAN 110 is a software VLAN, it is never committed to hardware in the
> switch.
>
> > now create the vlan-sware bridge (without vlan_filtering it works in my test)
> >
> > BRIDGE=lanbr0
> > ip link add name ${BRIDGE} type bridge vlan_filtering 1 vlan_default_pvid 1
> > ip link set ${BRIDGE} up
> > ip link set lan0 master ${BRIDGE}
> > ip link set lan0 up
> >
> > takes some time before it is applied and ping got lost
> >
> > packets are received by other end but without vlan-tag
>
> What happens in mt7530_port_vlan_filtering() is that the user port (lan0)
> *and* the CPU port become VLAN aware. I guess it is the change on the
> CPU port that affects the traffic to "wan".

interesting, and funny to see that vlan_aware on gmac is added, but not removed in else branch :p

> But I don't see yet why this
> affects the traffic in the way you mention (the CPU port strips the tag
> instead of dropping packets with VLAN 110).

i find it strange that packets will find the way to my laptop (maybe because mac was cached from tests without the bridge and mac is same for mac and vlan there). Have not verified packets are ok, only grepped output of tcpdump for the vlan-ip as tcpdump with vlan-option does not show anything with bridge.

have posted the dumps in systemd-issue as i thought it is a systemd problem.

https://github.com/systemd/systemd/issues/25970

> I have 2 random things to suggest you try.
>
> First is this
>
> From 2991f704e6f341bd81296e91fbb4381f528f8c7f Mon Sep 17 00:00:00 2001
> From: Vladimir Oltean <vladimir.oltean@nxp.com>
> Date: Fri, 20 Jan 2023 19:17:16 +0200
> Subject: [PATCH] mt7530 don't make the CPU port a VLAN user port
>
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
> ---
>  drivers/net/dsa/mt7530.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/dsa/mt7530.c b/drivers/net/dsa/mt7530.c
> index 616b21c90d05..7265c120c767 100644
> --- a/drivers/net/dsa/mt7530.c
> +++ b/drivers/net/dsa/mt7530.c
> @@ -1524,7 +1524,7 @@ mt7530_port_vlan_filtering(struct dsa_switch *ds, int port, bool vlan_filtering,
>  		 * for becoming a VLAN-aware port.
>  		 */
>  		mt7530_port_set_vlan_aware(ds, port);
> -		mt7530_port_set_vlan_aware(ds, cpu_dp->index);
> +//		mt7530_port_set_vlan_aware(ds, cpu_dp->index);
>  	} else {
>  		mt7530_port_set_vlan_unaware(ds, port);
>  	}
>
> If this works, I expect it will break VLAN tagged traffic over lan0 now :)
> So I would then like you to remove the first patch and try the next one

tried first patch, and wan stays working, now i try to figure out how i can access the vlan in the bridge to set ip-address..

ip link del vlan110 #delete vlan-interface from wan to have clean routing
bridge vlan add vid 110 dev lan0
bridge vlan add vid 110 dev lanbr0 self

how can i now set ip-address to the vlan110 (imho need to extract the vlan as separate netdev) for testing that lan0 still works?

regards Frank

Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Vladimir Oltean]

On Sat, Jan 21, 2023 at 01:11:43PM +0100, Frank Wunderlich wrote:
> > What happens in mt7530_port_vlan_filtering() is that the user port (lan0)
> > *and* the CPU port become VLAN aware. I guess it is the change on the
> > CPU port that affects the traffic to "wan".
> 
> interesting, and funny to see that vlan_aware on gmac is added, but not removed in else branch :p

it is... see "if (all_user_ports_removed)" in mt7530_port_set_vlan_unaware().

> > If this works, I expect it will break VLAN tagged traffic over lan0 now :)
> > So I would then like you to remove the first patch and try the next one
> 
> tried first patch, and wan stays working, now i try to figure out how i can access the vlan in the bridge to set ip-address..
> 
> ip link del vlan110 #delete vlan-interface from wan to have clean routing
> bridge vlan add vid 110 dev lan0
> bridge vlan add vid 110 dev lanbr0 self
> 
> how can i now set ip-address to the vlan110 (imho need to extract the vlan as separate netdev) for testing that lan0 still works?

ip link add link lanbr0 name lanbr0.110 type vlan id 110

Can you try the second patch instead of the first one? Without digging
deeply into mt7530 hardware docs, that's the best chance of making
things work without changing how the hardware operates.

Aw: Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Frank Wunderlich]

> Gesendet: Samstag, 21. Januar 2023 um 13:22 Uhr
> Von: "Vladimir Oltean" <olteanv@gmail.com>
>
> On Sat, Jan 21, 2023 at 01:11:43PM +0100, Frank Wunderlich wrote:
> > > What happens in mt7530_port_vlan_filtering() is that the user port (lan0)
> > > *and* the CPU port become VLAN aware. I guess it is the change on the
> > > CPU port that affects the traffic to "wan".
> >
> > interesting, and funny to see that vlan_aware on gmac is added, but not removed in else branch :p
>
> it is... see "if (all_user_ports_removed)" in mt7530_port_set_vlan_unaware().
>
> > > If this works, I expect it will break VLAN tagged traffic over lan0 now :)
> > > So I would then like you to remove the first patch and try the next one
> >
> > tried first patch, and wan stays working, now i try to figure out how i can access the vlan in the bridge to set ip-address..
> >
> > ip link del vlan110 #delete vlan-interface from wan to have clean routing
> > bridge vlan add vid 110 dev lan0
> > bridge vlan add vid 110 dev lanbr0 self
> >
> > how can i now set ip-address to the vlan110 (imho need to extract the vlan as separate netdev) for testing that lan0 still works?
>
> ip link add link lanbr0 name lanbr0.110 type vlan id 110

thanks found it already ;)

found out how to get access to the vlan from the bridge and it is still working

> If this works, I expect it will break VLAN tagged traffic over lan0 now :)
> So I would then like you to remove the first patch and try the next one

> tried first patch, and wan stays working, now i try to figure out how i can access the vlan in the bridge to set ip-address..

> ip link del vlan110 #delete vlan-interface from wan to have clean routing
> bridge vlan add vid 110 dev lan0
> bridge vlan add vid 110 dev lanbr0 self

> how can i now set ip-address to the vlan110 (imho need to extract the vlan as separate netdev) for testing that lan0 still works?

root@bpi-r3:~# ip link add link lanbr0 name lanbr0.100 type vlan id 110
root@bpi-r3:~# ip a a 192.168.110.5/24 dev lanbr0.100
root@bpi-r3:~# ip link set lanbr0.100 up

did "ping" and look at the remote side

root@frank-G5:~# tcpdump -i enp3s0 -nn -e vlan
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on enp3s0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
13:20:39.908845 96:3f:c5:84:65:f0 > e4:b9:7a:f7:c4:8b, ethertype 802.1Q (0x8100), length 102: vlan 110, p 0, ethertype IPv4 (0x0800), 192.168.110.5 > 192.168.110.3: ICMP echo request, id 48077, seq 1, length 64
13:20:39.908878 e4:b9:7a:f7:c4:8b > 96:3f:c5:84:65:f0, ethertype 802.1Q (0x8100), length 102: vlan 110, p 0, ethertype IPv4 (0x0800), 192.168.110.3 > 192.168.110.5: ICMP echo reply, id 48077, seq 1, length 64

so first patch fixes the behaviour on bpi-r3 (mt7531)...but maybe mt7530 need the tagging on cpu-port

> Can you try the second patch instead of the first one? Without digging
> deeply into mt7530 hardware docs, that's the best chance of making
> things work without changing how the hardware operates.

second patch works for wan, but vlan on bridge is broken, no packets receiving my laptop (also no untagged ones).

regards Frank

Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Vladimir Oltean]

On Sat, Jan 21, 2023 at 01:32:42PM +0100, Frank Wunderlich wrote:
> so first patch fixes the behaviour on bpi-r3 (mt7531)...but maybe mt7530 need the tagging on cpu-port
> 
> > Can you try the second patch instead of the first one? Without digging
> > deeply into mt7530 hardware docs, that's the best chance of making
> > things work without changing how the hardware operates.
> 
> second patch works for wan, but vlan on bridge is broken, no packets receiving my laptop (also no untagged ones).

It's hard for me to understand how applying only patch "tag_mtk only
combine VLAN tag with MTK tag is user port is VLAN aware" can produce
the results you describe... For packets sent to port lan0, nothing
should have been changed by that patch, because dsa_port_is_vlan_filtering(dp)
should return true.

If you can confirm there isn't any mistake in the testing procedure,
I'll take a look later today at the hardware documentation and try to
figure out why the CPU port is configured the way it is.

Aw: Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Frank Wunderlich]

Hi

> Gesendet: Samstag, 21. Januar 2023 um 14:35 Uhr
> Von: "Vladimir Oltean" <olteanv@gmail.com>
> An: "Frank Wunderlich" <frank-w@public-files.de>
> Cc: "Andrew Lunn" <andrew@lunn.ch>, "Florian Fainelli" <f.fainelli@gmail.com>, "David S. Miller" <davem@davemloft.net>, "Eric Dumazet" <edumazet@google.com>, "Jakub Kicinski" <kuba@kernel.org>, "Paolo Abeni" <pabeni@redhat.com>, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, "Landen Chao" <Landen.Chao@mediatek.com>, "Sean Wang" <sean.wang@mediatek.com>, "DENG Qingfang" <dqfext@gmail.com>, "Matthias Brugger" <matthias.bgg@gmail.com>, "Daniel Golle" <daniel@makrotopia.org>
> Betreff: Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac
>
> On Sat, Jan 21, 2023 at 01:32:42PM +0100, Frank Wunderlich wrote:
> > so first patch fixes the behaviour on bpi-r3 (mt7531)...but maybe mt7530 need the tagging on cpu-port
> >
> > > Can you try the second patch instead of the first one? Without digging
> > > deeply into mt7530 hardware docs, that's the best chance of making
> > > things work without changing how the hardware operates.
> >
> > second patch works for wan, but vlan on bridge is broken, no packets receiving my laptop (also no untagged ones).
>
> It's hard for me to understand how applying only patch "tag_mtk only
> combine VLAN tag with MTK tag is user port is VLAN aware" can produce
> the results you describe... For packets sent to port lan0, nothing
> should have been changed by that patch, because dsa_port_is_vlan_filtering(dp)
> should return true.
>
> If you can confirm there isn't any mistake in the testing procedure,
> I'll take a look later today at the hardware documentation and try to
> figure out why the CPU port is configured the way it is.

ok, booted again the kernel with first patch ("mt7530 don't make the CPU port a VLAN user port")
and yes lan0-vlan is broken...
seems i need to reboot after each lan/wan test to at least clean arp-cache.

but patch2 ("tag_mtk only combine VLAN tag with MTK tag is user port is VLAN aware") still not
works on lanbridge vlan (no packet received on target).

regards Frank

Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Vladimir Oltean]

Hi Frank,

On Sat, Jan 21, 2023 at 03:12:50PM +0100, Frank Wunderlich wrote:
> Hi
> 
> > Gesendet: Samstag, 21. Januar 2023 um 14:35 Uhr
> > Von: "Vladimir Oltean" <olteanv@gmail.com>
> > An: "Frank Wunderlich" <frank-w@public-files.de>
> > Cc: "Andrew Lunn" <andrew@lunn.ch>, "Florian Fainelli" <f.fainelli@gmail.com>, "David S. Miller" <davem@davemloft.net>, "Eric Dumazet" <edumazet@google.com>, "Jakub Kicinski" <kuba@kernel.org>, "Paolo Abeni" <pabeni@redhat.com>, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, "Landen Chao" <Landen.Chao@mediatek.com>, "Sean Wang" <sean.wang@mediatek.com>, "DENG Qingfang" <dqfext@gmail.com>, "Matthias Brugger" <matthias.bgg@gmail.com>, "Daniel Golle" <daniel@makrotopia.org>
> > Betreff: Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac
> >
> > On Sat, Jan 21, 2023 at 01:32:42PM +0100, Frank Wunderlich wrote:
> > > so first patch fixes the behaviour on bpi-r3 (mt7531)...but maybe mt7530 need the tagging on cpu-port
> > >
> > > > Can you try the second patch instead of the first one? Without digging
> > > > deeply into mt7530 hardware docs, that's the best chance of making
> > > > things work without changing how the hardware operates.
> > >
> > > second patch works for wan, but vlan on bridge is broken, no packets receiving my laptop (also no untagged ones).
> >
> > It's hard for me to understand how applying only patch "tag_mtk only
> > combine VLAN tag with MTK tag is user port is VLAN aware" can produce
> > the results you describe... For packets sent to port lan0, nothing
> > should have been changed by that patch, because dsa_port_is_vlan_filtering(dp)
> > should return true.
> >
> > If you can confirm there isn't any mistake in the testing procedure,
> > I'll take a look later today at the hardware documentation and try to
> > figure out why the CPU port is configured the way it is.
> 
> ok, booted again the kernel with first patch ("mt7530 don't make the CPU port a VLAN user port")
> and yes lan0-vlan is broken...
> seems i need to reboot after each lan/wan test to at least clean arp-cache.
> 
> but patch2 ("tag_mtk only combine VLAN tag with MTK tag is user port is VLAN aware") still not
> works on lanbridge vlan (no packet received on target).
> 
> regards Frank

Sorry for the delay and thanks again for testing.

I simply didn't have time to sit down with the hardware documentation
and (re)understand the concepts governing this switch.

I now have the patch below which should have everything working. Would
you mind testing it?

From 9110460832d99c3b3e86ffcda472a27a52cdf259 Mon Sep 17 00:00:00 2001
From: Vladimir Oltean <vladimir.oltean@nxp.com>
Date: Mon, 30 Jan 2023 14:31:17 +0200
Subject: [PATCH] net: dsa: mt7530: don't change PVC_EG_TAG when CPU port
 becomes VLAN-aware

Frank reports that in a mt7530 setup where some ports are standalone and
some are in a VLAN-aware bridge, 8021q uppers of the standalone ports
lose their VLAN tag on xmit, as seen by the link partner.

This seems to occur because once the other ports join the VLAN-aware
bridge, mt7530_port_vlan_filtering() also calls
mt7530_port_set_vlan_aware(ds, cpu_dp->index), and this affects the way
that the switch processes the traffic of the standalone port.

Relevant is the PVC_EG_TAG bit. The MT7530 documentation says about it:

EG_TAG: Incoming Port Egress Tag VLAN Attribution
0: disabled (system default)
1: consistent (keep the original ingress tag attribute)

My interpretation is that this setting applies on the ingress port, and
"disabled" is basically the normal behavior, where the egress tag format
of the packet (tagged or untagged) is decided by the VLAN table
(MT7530_VLAN_EGRESS_UNTAG or MT7530_VLAN_EGRESS_TAG).

But there is also an option of overriding the system default behavior,
and for the egress tagging format of packets to be decided not by the
VLAN table, but simply by copying the ingress tag format (if ingress was
tagged, egress is tagged; if ingress was untagged, egress is untagged;
aka "consistent). This is useful in 2 scenarios:

- VLAN-unaware bridge ports will always encounter a miss in the VLAN
  table. They should forward a packet as-is, though. So we use
  "consistent" there. See commit e045124e9399 ("net: dsa: mt7530: fix
  tagged frames pass-through in VLAN-unaware mode").

- Traffic injected from the CPU port. The operating system is in god
  mode; if it wants a packet to exit as VLAN-tagged, it sends it as
  VLAN-tagged. Otherwise it sends it as VLAN-untagged*.

*This is true only if we don't consider the bridge TX forwarding offload
feature, which mt7530 doesn't support.

So for now, make the CPU port always stay in "consistent" mode to allow
software VLANs to be forwarded to their egress ports with the VLAN tag
intact, and not stripped.

Link: https://lore.kernel.org/netdev/trinity-e6294d28-636c-4c40-bb8b-b523521b00be-1674233135062@3c-app-gmx-bs36/
Fixes: e045124e9399 ("net: dsa: mt7530: fix tagged frames pass-through in VLAN-unaware mode")
Reported-by: Frank Wunderlich <frank-w@public-files.de>
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 drivers/net/dsa/mt7530.c | 26 +++++++++++++++++++-------
 1 file changed, 19 insertions(+), 7 deletions(-)

diff --git a/drivers/net/dsa/mt7530.c b/drivers/net/dsa/mt7530.c
index 616b21c90d05..3a15015bc409 100644
--- a/drivers/net/dsa/mt7530.c
+++ b/drivers/net/dsa/mt7530.c
@@ -1302,14 +1302,26 @@ mt7530_port_set_vlan_aware(struct dsa_switch *ds, int port)
 		if (!priv->ports[port].pvid)
 			mt7530_rmw(priv, MT7530_PVC_P(port), ACC_FRM_MASK,
 				   MT7530_VLAN_ACC_TAGGED);
-	}
 
-	/* Set the port as a user port which is to be able to recognize VID
-	 * from incoming packets before fetching entry within the VLAN table.
-	 */
-	mt7530_rmw(priv, MT7530_PVC_P(port), VLAN_ATTR_MASK | PVC_EG_TAG_MASK,
-		   VLAN_ATTR(MT7530_VLAN_USER) |
-		   PVC_EG_TAG(MT7530_VLAN_EG_DISABLED));
+		/* Set the port as a user port which is to be able to recognize
+		 * VID from incoming packets before fetching entry within the
+		 * VLAN table.
+		 */
+		mt7530_rmw(priv, MT7530_PVC_P(port),
+			   VLAN_ATTR_MASK | PVC_EG_TAG_MASK,
+			   VLAN_ATTR(MT7530_VLAN_USER) |
+			   PVC_EG_TAG(MT7530_VLAN_EG_DISABLED));
+	} else {
+		/* Also set CPU ports to the "user" VLAN port attribute, to
+		 * allow VLAN classification, but keep the EG_TAG attribute as
+		 * "consistent" (i.o.w. don't change its value) for packets
+		 * received by the switch from the CPU, so that tagged packets
+		 * are forwarded to user ports as tagged, and untagged as
+		 * untagged.
+		 */
+		mt7530_rmw(priv, MT7530_PVC_P(port), VLAN_ATTR_MASK,
+			   VLAN_ATTR(MT7530_VLAN_USER));
+	}
 }
 
 static void
-- 
2.34.1

Aw: Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Frank Wunderlich]

Hi Vladimir,


> Gesendet: Montag, 30. Januar 2023 um 13:58 Uhr
> Von: "Vladimir Oltean" <olteanv@gmail.com>
> Hi Frank,
> Sorry for the delay and thanks again for testing.
>
> I simply didn't have time to sit down with the hardware documentation
> and (re)understand the concepts governing this switch.

no problem, same here...not have every day time to dive into it :)

> I now have the patch below which should have everything working. Would
> you mind testing it?

thanks for your Patch, but unfortunately it looks like does not change behaviour (have reverted all prevously applied patches,
only have felix series in).

i can ping over software-vlan on wan-port (and see tagged packets on other side), till the point i setup the vlan-aware bridge over lan-ports. ping works some time (imho till arp-cache is cleared) and i see untagged packets leaving wan-port (seen on other end) which should be tagged (wan.110).

and before anything ask: yes, i have set different mac to wan-port (and its vlan-interfaces) and lanbr0

15: lanbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 96:3f:c5:84:65:f0 brd ff:ff:ff:ff:ff:ff
17: wan.140@wan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 02:11:02:03:01:40 brd ff:ff:ff:ff:ff:ff
    inet 192.168.140.1/24 brd 192.168.140.255 scope global wan.140
       valid_lft forever preferred_lft forever
    inet6 fe80::11:2ff:fe03:140/64 scope link
       valid_lft forever preferred_lft forever
18: wan.110@wan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 02:11:02:03:01:10 brd ff:ff:ff:ff:ff:ff
    inet 192.168.110.1/24 brd 192.168.110.255 scope global wan.110
       valid_lft forever preferred_lft forever
    inet6 fe80::11:2ff:fe03:110/64 scope link
       valid_lft forever preferred_lft forever

have not yet defined any vlans in the bridge...only set vlan_awareness...maybe i need to add the wan-vlan
to the lan bridge too to pass filtering?

i'm unsure if tcpdump on the host interface should see vlan-traffic too (but do not show the vlan itself)...
in working state i see icmp in both tcpdump modes (pinging the full time without the bridge enabled only
changed tcpdump on the other side):

# tcpdump -nni lanbr0 | grep '\.110\.'

17:13:36.071047 IP 192.168.110.1 > 192.168.110.3: ICMP echo request, id 1617, seq 47, length 64
17:13:36.071290 IP 192.168.110.3 > 192.168.110.1: ICMP echo reply, id 1617, seq 47, length 64

and

tcpdump -nni lanbr0 -e vlan | grep '\.110\.'

17:16:35.032417 02:11:02:03:01:10 > 08:02:00:00:00:10, ethertype 802.1Q (0x8100), length 102: vlan 110, p 0, ethertype IPv4, 192.168.110.1 > 192.168.110.3: ICMP echo request, id 1617, seq 219, length 64
17:16:35.032609 08:02:00:00:00:10 > 02:11:02:03:01:10, ethertype 802.1Q (0x8100), length 102: vlan 110, p 0, ethertype IPv4, 192.168.110.3 > 192.168.110.1: ICMP echo reply, id 1617, seq 219, length 64

after the vlan_aware bridge goes up i see packets in the non-vlan-mode

if needed here is my current codebase:
https://github.com/frank-w/BPI-Router-Linux/commits/6.2-rc

regards Frank

Aw: Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Frank Wunderlich]

Hi,

sorry for the delay, i'm very busy recently :(

noticed that i missed 2 commands ("bridge vlan add vid ..." below) when testing the vlan-aware bridge...now both ports are working with vlan-tagging...the one inside (lan0) the bridge (lanbr0) and the one outside (wan).

BRIDGE=lanbr0
netif=lan0
vid=500
#ip link add name ${BRIDGE} type bridge
ip link add name ${BRIDGE} type bridge vlan_filtering 1 vlan_default_pvid 1
ip link set ${BRIDGE} up
ip link set $netif master ${BRIDGE}
ip link set $netif up
bridge vlan add vid $vid dev ${BRIDGE} self
bridge vlan add vid $vid dev $netif

#extract vlan from bridge to own netdev
ip link add link ${BRIDGE} name vlan$vid type vlan id $vid
ip a a 192.168.110.5/24 dev vlan$vid
ip link set vlan$vid up

btw can i see somehow if a bridge is vlan-aware (the flag itself)..."bridge vlan" command also lists non-vlan-aware bridges with vlan-id "1 pvid egress untagged"

so vladimir your last patch works well, thx for it. you can add my tested-by when upstreaming

regards Frank


> Gesendet: Dienstag, 31. Januar 2023 um 17:23 Uhr
> Von: "Frank Wunderlich" <frank-w@public-files.de>
> An: "Vladimir Oltean" <olteanv@gmail.com>
> Cc: "Andrew Lunn" <andrew@lunn.ch>, "Florian Fainelli" <f.fainelli@gmail.com>, "David S. Miller" <davem@davemloft.net>, "Eric Dumazet" <edumazet@google.com>, "Jakub Kicinski" <kuba@kernel.org>, "Paolo Abeni" <pabeni@redhat.com>, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, "Landen Chao" <Landen.Chao@mediatek.com>, "Sean Wang" <sean.wang@mediatek.com>, "DENG Qingfang" <dqfext@gmail.com>, "Matthias Brugger" <matthias.bgg@gmail.com>, "Daniel Golle" <daniel@makrotopia.org>
> Betreff: Aw: Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac
>
> Hi Vladimir,
>
>
> > Gesendet: Montag, 30. Januar 2023 um 13:58 Uhr
> > Von: "Vladimir Oltean" <olteanv@gmail.com>
> > Hi Frank,
> > Sorry for the delay and thanks again for testing.
> >
> > I simply didn't have time to sit down with the hardware documentation
> > and (re)understand the concepts governing this switch.
>
> no problem, same here...not have every day time to dive into it :)
>
> > I now have the patch below which should have everything working. Would
> > you mind testing it?
>
> thanks for your Patch, but unfortunately it looks like does not change behaviour (have reverted all prevously applied patches,
> only have felix series in).
>
> i can ping over software-vlan on wan-port (and see tagged packets on other side), till the point i setup the vlan-aware bridge over lan-ports. ping works some time (imho till arp-cache is cleared) and i see untagged packets leaving wan-port (seen on other end) which should be tagged (wan.110).
>
> and before anything ask: yes, i have set different mac to wan-port (and its vlan-interfaces) and lanbr0
>
> 15: lanbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
>     link/ether 96:3f:c5:84:65:f0 brd ff:ff:ff:ff:ff:ff
> 17: wan.140@wan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
>     link/ether 02:11:02:03:01:40 brd ff:ff:ff:ff:ff:ff
>     inet 192.168.140.1/24 brd 192.168.140.255 scope global wan.140
>        valid_lft forever preferred_lft forever
>     inet6 fe80::11:2ff:fe03:140/64 scope link
>        valid_lft forever preferred_lft forever
> 18: wan.110@wan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
>     link/ether 02:11:02:03:01:10 brd ff:ff:ff:ff:ff:ff
>     inet 192.168.110.1/24 brd 192.168.110.255 scope global wan.110
>        valid_lft forever preferred_lft forever
>     inet6 fe80::11:2ff:fe03:110/64 scope link
>        valid_lft forever preferred_lft forever
>
> have not yet defined any vlans in the bridge...only set vlan_awareness...maybe i need to add the wan-vlan
> to the lan bridge too to pass filtering?
>
> i'm unsure if tcpdump on the host interface should see vlan-traffic too (but do not show the vlan itself)...
> in working state i see icmp in both tcpdump modes (pinging the full time without the bridge enabled only
> changed tcpdump on the other side):
>
> # tcpdump -nni lanbr0 | grep '\.110\.'
>
> 17:13:36.071047 IP 192.168.110.1 > 192.168.110.3: ICMP echo request, id 1617, seq 47, length 64
> 17:13:36.071290 IP 192.168.110.3 > 192.168.110.1: ICMP echo reply, id 1617, seq 47, length 64
>
> and
>
> tcpdump -nni lanbr0 -e vlan | grep '\.110\.'
>
> 17:16:35.032417 02:11:02:03:01:10 > 08:02:00:00:00:10, ethertype 802.1Q (0x8100), length 102: vlan 110, p 0, ethertype IPv4, 192.168.110.1 > 192.168.110.3: ICMP echo request, id 1617, seq 219, length 64
> 17:16:35.032609 08:02:00:00:00:10 > 02:11:02:03:01:10, ethertype 802.1Q (0x8100), length 102: vlan 110, p 0, ethertype IPv4, 192.168.110.3 > 192.168.110.1: ICMP echo reply, id 1617, seq 219, length 64
>
> after the vlan_aware bridge goes up i see packets in the non-vlan-mode
>
> if needed here is my current codebase:
> https://github.com/frank-w/BPI-Router-Linux/commits/6.2-rc
>
> regards Frank

Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Vladimir Oltean]

Hi Frank,

On Sun, Feb 05, 2023 at 02:48:55PM +0100, Frank Wunderlich wrote:
> Hi,
> 
> sorry for the delay, i'm very busy recently :(
> 
> noticed that i missed 2 commands ("bridge vlan add vid ..." below)
> when testing the vlan-aware bridge...now both ports are working with
> vlan-tagging...the one inside (lan0) the bridge (lanbr0) and the one
> outside (wan).
> 
> BRIDGE=lanbr0
> netif=lan0
> vid=500
> #ip link add name ${BRIDGE} type bridge
> ip link add name ${BRIDGE} type bridge vlan_filtering 1 vlan_default_pvid 1
> ip link set ${BRIDGE} up
> ip link set $netif master ${BRIDGE}
> ip link set $netif up
> bridge vlan add vid $vid dev ${BRIDGE} self
> bridge vlan add vid $vid dev $netif
> 
> #extract vlan from bridge to own netdev
> ip link add link ${BRIDGE} name vlan$vid type vlan id $vid
> ip a a 192.168.110.5/24 dev vlan$vid
> ip link set vlan$vid up
> 
> btw can i see somehow if a bridge is vlan-aware (the flag itself)...
> "bridge vlan" command also lists non-vlan-aware bridges with vlan-id
> "1 pvid egress untagged"
> 
> so vladimir your last patch works well, thx for it. you can add my tested-by when upstreaming
> 
> regards Frank

Thanks for double-checking. I was wondering what could have been wrong
with the patch and just not seeing it.

You can see if a bridge is VLAN aware with "ip -d link show lanbr0".
Add "-j" to the list of arguments, and you get json output which you can
parse with jq if you need the info in a script or other program.

The software model of the Linux bridge is that where you can add, delete
and see VLANs on a bridge even if it is VLAN unaware. Those VLANs are
simply inactive until the bridge becomes VLAN aware.

I will send the patch today with your tested tag on it.

Aw: Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Frank Wunderlich]

> Gesendet: Samstag, 21. Januar 2023 um 14:35 Uhr
> Von: "Vladimir Oltean" <olteanv@gmail.com>
> It's hard for me to understand how applying only patch "tag_mtk only
> combine VLAN tag with MTK tag is user port is VLAN aware" can produce
> the results you describe... For packets sent to port lan0, nothing
> should have been changed by that patch, because dsa_port_is_vlan_filtering(dp)
> should return true.
>
> If you can confirm there isn't any mistake in the testing procedure,
> I'll take a look later today at the hardware documentation and try to
> figure out why the CPU port is configured the way it is.

booted an older kernel without your patches, and tried the vlan-aware bridge the same way,
and it is not working there too. So not broken with your patches.

this is how i have created the bridge completely (to exclude a mistake in my setup):

BRIDGE=lanbr0
ip link add name ${BRIDGE} type bridge vlan_filtering 1 vlan_default_pvid 1
ip link set ${BRIDGE} up
ip link set lan0 master ${BRIDGE}
ip link set lan0 up

ip link add link lanbr0 name lanbr0.100 type vlan id 110
ip a a 192.168.110.5/24 dev lanbr0.100
ip link set lanbr0.100 up

btw. why is my vlan software-only and not pushed to hardware?

regards Frank

Re: [BUG] vlan-aware bridge breaks vlan on another port on same gmac

[Vladimir Oltean]

On Sat, Jan 21, 2023 at 03:35:28PM +0100, Frank Wunderlich wrote:
> btw. why is my vlan software-only and not pushed to hardware?

Short story, because committing it to hardware is a useless complication.
A standalone port should be VLAN-unaware, or i.o.w. it should not drop
based on VLAN port membership, shouldn't add or strip any VLAN header,
and should forward as if the VLAN wasn't there.

So the behavior of a standalone port is absolutely sufficient as a basis
for an 8021q upper interface to see the traffic it needs, and for the
traffic it sends to reach the outside world as it intended.