[RFC PATCH net-next 0/4] Reduce coupling between DSA and Broadcom SYSTEMPORT driver

[RFC PATCH net-next 0/4] Reduce coupling between DSA and Broadcom SYSTEMPORT driver

[Vladimir Oltean]

Upon a quick inspection, it seems that there is some code in the generic
DSA layer that is somehow specific to the Broadcom SYSTEMPORT driver.
The challenge there is that the hardware integration is very tight between
the switch and the DSA master interface. However this does not mean that
the drivers must also be as integrated as the hardware is. We can avoid
creating a DSA notifier just for the Broadcom SYSTEMPORT, and we can
move some Broadcom-specific queue mapping helpers outside of the common
include/net/dsa.h.

Vladimir Oltean (4):
  net: dsa: move the Broadcom tag information in a separate header file
  net: dsa: export dsa_slave_dev_check
  net: systemport: use standard netdevice notifier to detect DSA
    presence
  net: dsa: remove the DSA specific notifiers

 MAINTAINERS                                |  1 +
 drivers/net/ethernet/broadcom/bcmsysport.c | 77 +++++++++-------------
 drivers/net/ethernet/broadcom/bcmsysport.h |  2 +-
 include/linux/dsa/brcm.h                   | 16 +++++
 include/net/dsa.h                          | 48 +-------------
 net/dsa/dsa.c                              | 22 -------
 net/dsa/dsa_priv.h                         |  1 -
 net/dsa/slave.c                            | 18 +----
 net/dsa/tag_brcm.c                         |  1 +
 9 files changed, 55 insertions(+), 131 deletions(-)
 create mode 100644 include/linux/dsa/brcm.h

-- 
2.25.1

[RFC PATCH net-next 1/4] net: dsa: move the Broadcom tag information in a separate header file

[Vladimir Oltean]

It is a bit strange to see something as specific as Broadcom SYSTEMPORT
bits in the main DSA include file. Move these away into a separate
header, and have the tagger and the SYSTEMPORT driver include them.

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 MAINTAINERS                                |  1 +
 drivers/net/ethernet/broadcom/bcmsysport.c |  1 +
 include/linux/dsa/brcm.h                   | 16 ++++++++++++++++
 include/net/dsa.h                          |  6 ------
 net/dsa/tag_brcm.c                         |  1 +
 5 files changed, 19 insertions(+), 6 deletions(-)
 create mode 100644 include/linux/dsa/brcm.h

diff --git a/MAINTAINERS b/MAINTAINERS
index a355db292486..e2d1d852a3a8 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3389,6 +3389,7 @@ L:	openwrt-devel@lists.openwrt.org (subscribers-only)
 S:	Supported
 F:	Documentation/devicetree/bindings/net/dsa/b53.txt
 F:	drivers/net/dsa/b53/*
+F:	include/linux/dsa/brcm.h
 F:	include/linux/platform_data/b53.h
 
 BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE
diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c b/drivers/net/ethernet/broadcom/bcmsysport.c
index 0fdd19d99d99..82541352b1eb 100644
--- a/drivers/net/ethernet/broadcom/bcmsysport.c
+++ b/drivers/net/ethernet/broadcom/bcmsysport.c
@@ -12,6 +12,7 @@
 #include <linux/module.h>
 #include <linux/kernel.h>
 #include <linux/netdevice.h>
+#include <linux/dsa/brcm.h>
 #include <linux/etherdevice.h>
 #include <linux/platform_device.h>
 #include <linux/of.h>
diff --git a/include/linux/dsa/brcm.h b/include/linux/dsa/brcm.h
new file mode 100644
index 000000000000..47545a948784
--- /dev/null
+++ b/include/linux/dsa/brcm.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0-only
+ * Copyright (C) 2014 Broadcom Corporation
+ */
+
+/* Included by drivers/net/ethernet/broadcom/bcmsysport.c and
+ * net/dsa/tag_brcm.c
+ */
+#ifndef _NET_DSA_BRCM_H
+#define _NET_DSA_BRCM_H
+
+/* Broadcom tag specific helpers to insert and extract queue/port number */
+#define BRCM_TAG_SET_PORT_QUEUE(p, q)	((p) << 8 | q)
+#define BRCM_TAG_GET_PORT(v)		((v) >> 8)
+#define BRCM_TAG_GET_QUEUE(v)		((v) & 0xff)
+
+#endif
diff --git a/include/net/dsa.h b/include/net/dsa.h
index 4e60d2610f20..af9a4f9ee764 100644
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -873,12 +873,6 @@ static inline int call_dsa_notifiers(unsigned long val, struct net_device *dev,
 }
 #endif
 
-/* Broadcom tag specific helpers to insert and extract queue/port number */
-#define BRCM_TAG_SET_PORT_QUEUE(p, q)	((p) << 8 | q)
-#define BRCM_TAG_GET_PORT(v)		((v) >> 8)
-#define BRCM_TAG_GET_QUEUE(v)		((v) & 0xff)
-
-
 netdev_tx_t dsa_enqueue_skb(struct sk_buff *skb, struct net_device *dev);
 int dsa_port_get_phy_strings(struct dsa_port *dp, uint8_t *data);
 int dsa_port_get_ethtool_phy_stats(struct dsa_port *dp, uint64_t *data);
diff --git a/net/dsa/tag_brcm.c b/net/dsa/tag_brcm.c
index e934dace3922..e2577a7dcbca 100644
--- a/net/dsa/tag_brcm.c
+++ b/net/dsa/tag_brcm.c
@@ -5,6 +5,7 @@
  * Copyright (C) 2014 Broadcom Corporation
  */
 
+#include <linux/dsa/brcm.h>
 #include <linux/etherdevice.h>
 #include <linux/list.h>
 #include <linux/slab.h>
-- 
2.25.1

[RFC PATCH net-next 2/4] net: dsa: export dsa_slave_dev_check

[Vladimir Oltean]

Using the NETDEV_CHANGEUPPER notifications, drivers can be aware when
they are enslaved to e.g. a bridge by calling netif_is_bridge_master().

Export this helper from DSA to get the equivalent functionality of
determining whether the upper interface of a CHANGEUPPER notifier is a
DSA switch interface or not.

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 include/net/dsa.h  | 6 ++++++
 net/dsa/dsa_priv.h | 1 -
 net/dsa/slave.c    | 1 +
 3 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/include/net/dsa.h b/include/net/dsa.h
index af9a4f9ee764..5badfd6403c5 100644
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -855,6 +855,7 @@ int register_dsa_notifier(struct notifier_block *nb);
 int unregister_dsa_notifier(struct notifier_block *nb);
 int call_dsa_notifiers(unsigned long val, struct net_device *dev,
 		       struct dsa_notifier_info *info);
+bool dsa_slave_dev_check(const struct net_device *dev);
 #else
 static inline int register_dsa_notifier(struct notifier_block *nb)
 {
@@ -871,6 +872,11 @@ static inline int call_dsa_notifiers(unsigned long val, struct net_device *dev,
 {
 	return NOTIFY_DONE;
 }
+
+static inline bool dsa_slave_dev_check(const struct net_device *dev)
+{
+	return false;
+}
 #endif
 
 netdev_tx_t dsa_enqueue_skb(struct sk_buff *skb, struct net_device *dev);
diff --git a/net/dsa/dsa_priv.h b/net/dsa/dsa_priv.h
index 7c96aae9062c..33c082f10bb9 100644
--- a/net/dsa/dsa_priv.h
+++ b/net/dsa/dsa_priv.h
@@ -172,7 +172,6 @@ extern const struct dsa_device_ops notag_netdev_ops;
 void dsa_slave_mii_bus_init(struct dsa_switch *ds);
 int dsa_slave_create(struct dsa_port *dp);
 void dsa_slave_destroy(struct net_device *slave_dev);
-bool dsa_slave_dev_check(const struct net_device *dev);
 int dsa_slave_suspend(struct net_device *slave_dev);
 int dsa_slave_resume(struct net_device *slave_dev);
 int dsa_slave_register_notifier(void);
diff --git a/net/dsa/slave.c b/net/dsa/slave.c
index 4a0498bf6c65..c01bc7ebeb14 100644
--- a/net/dsa/slave.c
+++ b/net/dsa/slave.c
@@ -1924,6 +1924,7 @@ bool dsa_slave_dev_check(const struct net_device *dev)
 {
 	return dev->netdev_ops == &dsa_slave_netdev_ops;
 }
+EXPORT_SYMBOL_GPL(dsa_slave_dev_check);
 
 static int dsa_slave_changeupper(struct net_device *dev,
 				 struct netdev_notifier_changeupper_info *info)
-- 
2.25.1

[RFC PATCH net-next 3/4] net: systemport: use standard netdevice notifier to detect DSA presence

[Vladimir Oltean]

The SYSTEMPORT driver maps each port of the embedded Broadcom DSA switch
port to a certain queue of the master Ethernet controller. For that it
currently uses a dedicated notifier infrastructure which was added in
commit 60724d4bae14 ("net: dsa: Add support for DSA specific notifiers").

However, since commit 2f1e8ea726e9 ("net: dsa: link interfaces with the
DSA master to get rid of lockdep warnings"), DSA is actually an upper of
the Broadcom SYSTEMPORT as far as the netdevice adjacency lists are
concerned. So naturally, the plain NETDEV_CHANGEUPPER net device notifiers
are emitted. It looks like there is enough API exposed by DSA to the
outside world already to make the call_dsa_notifiers API redundant. So
let's convert its only user to plain netdev notifiers.

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 drivers/net/ethernet/broadcom/bcmsysport.c | 76 +++++++++-------------
 drivers/net/ethernet/broadcom/bcmsysport.h |  2 +-
 2 files changed, 32 insertions(+), 46 deletions(-)

diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c b/drivers/net/ethernet/broadcom/bcmsysport.c
index 82541352b1eb..c5df235975e7 100644
--- a/drivers/net/ethernet/broadcom/bcmsysport.c
+++ b/drivers/net/ethernet/broadcom/bcmsysport.c
@@ -2311,33 +2311,22 @@ static const struct net_device_ops bcm_sysport_netdev_ops = {
 	.ndo_select_queue	= bcm_sysport_select_queue,
 };
 
-static int bcm_sysport_map_queues(struct notifier_block *nb,
-				  struct dsa_notifier_register_info *info)
+static int bcm_sysport_map_queues(struct net_device *dev,
+				  struct net_device *slave_dev)
 {
+	struct dsa_port *dp = dsa_port_from_netdev(slave_dev);
+	struct bcm_sysport_priv *priv = netdev_priv(dev);
 	struct bcm_sysport_tx_ring *ring;
-	struct bcm_sysport_priv *priv;
-	struct net_device *slave_dev;
 	unsigned int num_tx_queues;
 	unsigned int q, qp, port;
-	struct net_device *dev;
-
-	priv = container_of(nb, struct bcm_sysport_priv, dsa_notifier);
-	if (priv->netdev != info->master)
-		return 0;
-
-	dev = info->master;
 
 	/* We can't be setting up queue inspection for non directly attached
 	 * switches
 	 */
-	if (info->switch_number)
+	if (dp->ds->index)
 		return 0;
 
-	if (dev->netdev_ops != &bcm_sysport_netdev_ops)
-		return 0;
-
-	port = info->port_number;
-	slave_dev = info->info.dev;
+	port = dp->index;
 
 	/* On SYSTEMPORT Lite we have twice as less queues, so we cannot do a
 	 * 1:1 mapping, we can only do a 2:1 mapping. By reducing the number of
@@ -2377,27 +2366,16 @@ static int bcm_sysport_map_queues(struct notifier_block *nb,
 	return 0;
 }
 
-static int bcm_sysport_unmap_queues(struct notifier_block *nb,
-				    struct dsa_notifier_register_info *info)
+static int bcm_sysport_unmap_queues(struct net_device *dev,
+				    struct net_device *slave_dev)
 {
+	struct dsa_port *dp = dsa_port_from_netdev(slave_dev);
+	struct bcm_sysport_priv *priv = netdev_priv(dev);
 	struct bcm_sysport_tx_ring *ring;
-	struct bcm_sysport_priv *priv;
-	struct net_device *slave_dev;
 	unsigned int num_tx_queues;
-	struct net_device *dev;
 	unsigned int q, qp, port;
 
-	priv = container_of(nb, struct bcm_sysport_priv, dsa_notifier);
-	if (priv->netdev != info->master)
-		return 0;
-
-	dev = info->master;
-
-	if (dev->netdev_ops != &bcm_sysport_netdev_ops)
-		return 0;
-
-	port = info->port_number;
-	slave_dev = info->info.dev;
+	port = dp->index;
 
 	num_tx_queues = slave_dev->real_num_tx_queues;
 
@@ -2418,17 +2396,25 @@ static int bcm_sysport_unmap_queues(struct notifier_block *nb,
 	return 0;
 }
 
-static int bcm_sysport_dsa_notifier(struct notifier_block *nb,
-				    unsigned long event, void *ptr)
+static int bcm_sysport_netdevice_event(struct notifier_block *nb,
+				       unsigned long event, void *ptr)
 {
-	int ret = NOTIFY_DONE;
+	struct net_device *dev = netdev_notifier_info_to_dev(ptr);
+	struct netdev_notifier_changeupper_info *info = ptr;
+	int ret = 0;
 
 	switch (event) {
-	case DSA_PORT_REGISTER:
-		ret = bcm_sysport_map_queues(nb, ptr);
-		break;
-	case DSA_PORT_UNREGISTER:
-		ret = bcm_sysport_unmap_queues(nb, ptr);
+	case NETDEV_CHANGEUPPER:
+		if (dev->netdev_ops != &bcm_sysport_netdev_ops)
+			return NOTIFY_DONE;
+
+		if (!dsa_slave_dev_check(info->upper_dev))
+			return NOTIFY_DONE;
+
+		if (info->linking)
+			ret = bcm_sysport_map_queues(dev, info->upper_dev);
+		else
+			ret = bcm_sysport_unmap_queues(dev, info->upper_dev);
 		break;
 	}
 
@@ -2600,9 +2586,9 @@ static int bcm_sysport_probe(struct platform_device *pdev)
 	priv->rx_max_coalesced_frames = 1;
 	u64_stats_init(&priv->syncp);
 
-	priv->dsa_notifier.notifier_call = bcm_sysport_dsa_notifier;
+	priv->netdev_notifier.notifier_call = bcm_sysport_netdevice_event;
 
-	ret = register_dsa_notifier(&priv->dsa_notifier);
+	ret = register_netdevice_notifier(&priv->netdev_notifier);
 	if (ret) {
 		dev_err(&pdev->dev, "failed to register DSA notifier\n");
 		goto err_deregister_fixed_link;
@@ -2629,7 +2615,7 @@ static int bcm_sysport_probe(struct platform_device *pdev)
 	return 0;
 
 err_deregister_notifier:
-	unregister_dsa_notifier(&priv->dsa_notifier);
+	unregister_netdevice_notifier(&priv->netdev_notifier);
 err_deregister_fixed_link:
 	if (of_phy_is_fixed_link(dn))
 		of_phy_deregister_fixed_link(dn);
@@ -2647,7 +2633,7 @@ static int bcm_sysport_remove(struct platform_device *pdev)
 	/* Not much to do, ndo_close has been called
 	 * and we use managed allocations
 	 */
-	unregister_dsa_notifier(&priv->dsa_notifier);
+	unregister_netdevice_notifier(&priv->netdev_notifier);
 	unregister_netdev(dev);
 	if (of_phy_is_fixed_link(dn))
 		of_phy_deregister_fixed_link(dn);
diff --git a/drivers/net/ethernet/broadcom/bcmsysport.h b/drivers/net/ethernet/broadcom/bcmsysport.h
index 3a5cb6f128f5..fefd3ccf0379 100644
--- a/drivers/net/ethernet/broadcom/bcmsysport.h
+++ b/drivers/net/ethernet/broadcom/bcmsysport.h
@@ -787,7 +787,7 @@ struct bcm_sysport_priv {
 	struct u64_stats_sync	syncp;
 
 	/* map information between switch port queues and local queues */
-	struct notifier_block	dsa_notifier;
+	struct notifier_block	netdev_notifier;
 	unsigned int		per_port_num_tx_queues;
 	struct bcm_sysport_tx_ring *ring_map[DSA_MAX_PORTS * 8];
 
-- 
2.25.1

[RFC PATCH net-next 4/4] net: dsa: remove the DSA specific notifiers

[Vladimir Oltean]

This effectively reverts commit 60724d4bae14 ("net: dsa: Add support for
DSA specific notifiers"). The reason is that since commit 2f1e8ea726e9
("net: dsa: link interfaces with the DSA master to get rid of lockdep
warnings"), it appears that there is a generic way to achieve the same
purpose. The only user thus far, the Broadcom SYSTEMPORT driver, was
converted to use the generic notifiers.

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 include/net/dsa.h | 42 ------------------------------------------
 net/dsa/dsa.c     | 22 ----------------------
 net/dsa/slave.c   | 17 -----------------
 3 files changed, 81 deletions(-)

diff --git a/include/net/dsa.h b/include/net/dsa.h
index 5badfd6403c5..3950e4832a33 100644
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -828,51 +828,9 @@ static inline int dsa_switch_resume(struct dsa_switch *ds)
 }
 #endif /* CONFIG_PM_SLEEP */
 
-enum dsa_notifier_type {
-	DSA_PORT_REGISTER,
-	DSA_PORT_UNREGISTER,
-};
-
-struct dsa_notifier_info {
-	struct net_device *dev;
-};
-
-struct dsa_notifier_register_info {
-	struct dsa_notifier_info info;	/* must be first */
-	struct net_device *master;
-	unsigned int port_number;
-	unsigned int switch_number;
-};
-
-static inline struct net_device *
-dsa_notifier_info_to_dev(const struct dsa_notifier_info *info)
-{
-	return info->dev;
-}
-
 #if IS_ENABLED(CONFIG_NET_DSA)
-int register_dsa_notifier(struct notifier_block *nb);
-int unregister_dsa_notifier(struct notifier_block *nb);
-int call_dsa_notifiers(unsigned long val, struct net_device *dev,
-		       struct dsa_notifier_info *info);
 bool dsa_slave_dev_check(const struct net_device *dev);
 #else
-static inline int register_dsa_notifier(struct notifier_block *nb)
-{
-	return 0;
-}
-
-static inline int unregister_dsa_notifier(struct notifier_block *nb)
-{
-	return 0;
-}
-
-static inline int call_dsa_notifiers(unsigned long val, struct net_device *dev,
-				     struct dsa_notifier_info *info)
-{
-	return NOTIFY_DONE;
-}
-
 static inline bool dsa_slave_dev_check(const struct net_device *dev)
 {
 	return false;
diff --git a/net/dsa/dsa.c b/net/dsa/dsa.c
index a1b1dc8a4d87..df75481b12ed 100644
--- a/net/dsa/dsa.c
+++ b/net/dsa/dsa.c
@@ -309,28 +309,6 @@ bool dsa_schedule_work(struct work_struct *work)
 	return queue_work(dsa_owq, work);
 }
 
-static ATOMIC_NOTIFIER_HEAD(dsa_notif_chain);
-
-int register_dsa_notifier(struct notifier_block *nb)
-{
-	return atomic_notifier_chain_register(&dsa_notif_chain, nb);
-}
-EXPORT_SYMBOL_GPL(register_dsa_notifier);
-
-int unregister_dsa_notifier(struct notifier_block *nb)
-{
-	return atomic_notifier_chain_unregister(&dsa_notif_chain, nb);
-}
-EXPORT_SYMBOL_GPL(unregister_dsa_notifier);
-
-int call_dsa_notifiers(unsigned long val, struct net_device *dev,
-		       struct dsa_notifier_info *info)
-{
-	info->dev = dev;
-	return atomic_notifier_call_chain(&dsa_notif_chain, val, info);
-}
-EXPORT_SYMBOL_GPL(call_dsa_notifiers);
-
 int dsa_devlink_param_get(struct devlink *dl, u32 id,
 			  struct devlink_param_gset_ctx *ctx)
 {
diff --git a/net/dsa/slave.c b/net/dsa/slave.c
index c01bc7ebeb14..1b511895e7a5 100644
--- a/net/dsa/slave.c
+++ b/net/dsa/slave.c
@@ -1764,20 +1764,6 @@ int dsa_slave_resume(struct net_device *slave_dev)
 	return 0;
 }
 
-static void dsa_slave_notify(struct net_device *dev, unsigned long val)
-{
-	struct net_device *master = dsa_slave_to_master(dev);
-	struct dsa_port *dp = dsa_slave_to_port(dev);
-	struct dsa_notifier_register_info rinfo = {
-		.switch_number = dp->ds->index,
-		.port_number = dp->index,
-		.master = master,
-		.info.dev = dev,
-	};
-
-	call_dsa_notifiers(val, dev, &rinfo.info);
-}
-
 int dsa_slave_create(struct dsa_port *port)
 {
 	const struct dsa_port *cpu_dp = port->cpu_dp;
@@ -1863,8 +1849,6 @@ int dsa_slave_create(struct dsa_port *port)
 		goto out_gcells;
 	}
 
-	dsa_slave_notify(slave_dev, DSA_PORT_REGISTER);
-
 	rtnl_lock();
 
 	ret = register_netdevice(slave_dev);
@@ -1913,7 +1897,6 @@ void dsa_slave_destroy(struct net_device *slave_dev)
 	phylink_disconnect_phy(dp->pl);
 	rtnl_unlock();
 
-	dsa_slave_notify(slave_dev, DSA_PORT_UNREGISTER);
 	phylink_destroy(dp->pl);
 	gro_cells_destroy(&p->gcells);
 	free_percpu(slave_dev->tstats);
-- 
2.25.1

Re: [RFC PATCH net-next 1/4] net: dsa: move the Broadcom tag information in a separate header file

[Florian Fainelli]

On 12/18/2020 2:38 PM, Vladimir Oltean wrote:
> It is a bit strange to see something as specific as Broadcom SYSTEMPORT
> bits in the main DSA include file. Move these away into a separate
> header, and have the tagger and the SYSTEMPORT driver include them.
> 
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>

Acked-by: Florian Fainelli <f.fainelli@gmail.com>
-- 
Florian

Re: [RFC PATCH net-next 2/4] net: dsa: export dsa_slave_dev_check

[Florian Fainelli]

On 12/18/2020 2:38 PM, Vladimir Oltean wrote:
> Using the NETDEV_CHANGEUPPER notifications, drivers can be aware when
> they are enslaved to e.g. a bridge by calling netif_is_bridge_master().
> 
> Export this helper from DSA to get the equivalent functionality of
> determining whether the upper interface of a CHANGEUPPER notifier is a
> DSA switch interface or not.
> 
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>

Acked-by: Florian Fainelli <f.fainelli@gmail.com>
-- 
Florian

Re: [RFC PATCH net-next 4/4] net: dsa: remove the DSA specific notifiers

[Florian Fainelli]

On 12/18/2020 2:38 PM, Vladimir Oltean wrote:
> This effectively reverts commit 60724d4bae14 ("net: dsa: Add support for
> DSA specific notifiers"). The reason is that since commit 2f1e8ea726e9
> ("net: dsa: link interfaces with the DSA master to get rid of lockdep
> warnings"), it appears that there is a generic way to achieve the same
> purpose. The only user thus far, the Broadcom SYSTEMPORT driver, was
> converted to use the generic notifiers.
> 
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>

Acked-by: Florian Fainelli <f.fainelli@gmail.com>
-- 
Florian

Re: [RFC PATCH net-next 3/4] net: systemport: use standard netdevice notifier to detect DSA presence

[Florian Fainelli]

On 12/18/2020 2:38 PM, Vladimir Oltean wrote:
> The SYSTEMPORT driver maps each port of the embedded Broadcom DSA switch
> port to a certain queue of the master Ethernet controller. For that it
> currently uses a dedicated notifier infrastructure which was added in
> commit 60724d4bae14 ("net: dsa: Add support for DSA specific notifiers").
> 
> However, since commit 2f1e8ea726e9 ("net: dsa: link interfaces with the
> DSA master to get rid of lockdep warnings"), DSA is actually an upper of
> the Broadcom SYSTEMPORT as far as the netdevice adjacency lists are
> concerned. So naturally, the plain NETDEV_CHANGEUPPER net device notifiers
> are emitted. It looks like there is enough API exposed by DSA to the
> outside world already to make the call_dsa_notifiers API redundant. So
> let's convert its only user to plain netdev notifiers.
> 
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
> ---
>  drivers/net/ethernet/broadcom/bcmsysport.c | 76 +++++++++-------------
>  drivers/net/ethernet/broadcom/bcmsysport.h |  2 +-
>  2 files changed, 32 insertions(+), 46 deletions(-)
> 
> diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c b/drivers/net/ethernet/broadcom/bcmsysport.c
> index 82541352b1eb..c5df235975e7 100644
> --- a/drivers/net/ethernet/broadcom/bcmsysport.c
> +++ b/drivers/net/ethernet/broadcom/bcmsysport.c
> @@ -2311,33 +2311,22 @@ static const struct net_device_ops bcm_sysport_netdev_ops = {
>  	.ndo_select_queue	= bcm_sysport_select_queue,
>  };
>  
> -static int bcm_sysport_map_queues(struct notifier_block *nb,
> -				  struct dsa_notifier_register_info *info)
> +static int bcm_sysport_map_queues(struct net_device *dev,
> +				  struct net_device *slave_dev)
>  {
> +	struct dsa_port *dp = dsa_port_from_netdev(slave_dev);
> +	struct bcm_sysport_priv *priv = netdev_priv(dev);
>  	struct bcm_sysport_tx_ring *ring;
> -	struct bcm_sysport_priv *priv;
> -	struct net_device *slave_dev;
>  	unsigned int num_tx_queues;
>  	unsigned int q, qp, port;
> -	struct net_device *dev;
> -
> -	priv = container_of(nb, struct bcm_sysport_priv, dsa_notifier);
> -	if (priv->netdev != info->master)
> -		return 0;

There are systems with two SYSTEMPORT network devices registered and
therfore this check was intended to avoid programmig the incorrect
network device upon notification, however now that we have proper
upper/lower linking, and given the decisions made by bcm_sf2.c, only one
out of the two SYSTEMPORT network devices will act as a DSA master,
therefore this should no longer be necessary.

I would like to test this before giving this patch a Acked-by or
Teteed-by tag, net-next is still closed and this should only take a few
hours if there not any non-maskable real life interrupts showing up.
-- 
Florian

Re: [RFC PATCH net-next 3/4] net: systemport: use standard netdevice notifier to detect DSA presence

[Florian Fainelli]

On 12/18/2020 2:38 PM, Vladimir Oltean wrote:
> The SYSTEMPORT driver maps each port of the embedded Broadcom DSA switch
> port to a certain queue of the master Ethernet controller. For that it
> currently uses a dedicated notifier infrastructure which was added in
> commit 60724d4bae14 ("net: dsa: Add support for DSA specific notifiers").
> 
> However, since commit 2f1e8ea726e9 ("net: dsa: link interfaces with the
> DSA master to get rid of lockdep warnings"), DSA is actually an upper of
> the Broadcom SYSTEMPORT as far as the netdevice adjacency lists are
> concerned. So naturally, the plain NETDEV_CHANGEUPPER net device notifiers
> are emitted. It looks like there is enough API exposed by DSA to the
> outside world already to make the call_dsa_notifiers API redundant. So
> let's convert its only user to plain netdev notifiers.
> 
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>

The CHANGEUPPER has a slightly different semantic than the current DSA
notifier, and so events that would look like this during
bcm_sysport_init_tx_ring() (good):

[    6.781064] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=0,port=0
[    6.789214] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=1,port=0
[    6.797337] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=2,port=0
[    6.805464] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=3,port=0
[    6.813583] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=0,port=1
[    6.821701] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=1,port=1
[    6.829819] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=2,port=1
[    6.837944] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=3,port=1
[    6.846063] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=0,port=2
[    6.854183] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=1,port=2
[    6.862303] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=2,port=2
[    6.870425] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=3,port=2
[    6.878544] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=0,port=5
[    6.886663] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=1,port=5
[    6.894783] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=2,port=5
[    6.902906] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=3,port=5

now we are getting (bad):

[    6.678157] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=0,port=0
[    6.686302] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=1,port=0
[    6.694434] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=2,port=0
[    6.702554] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=3,port=0
[    6.710679] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=0,port=0
[    6.718797] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=1,port=0
[    6.726914] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=2,port=0
[    6.735033] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=3,port=0
[    6.743156] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=0,port=1
[    6.751275] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=1,port=1
[    6.759395] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=2,port=1
[    6.767514] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=3,port=1
[    6.775636] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=0,port=1
[    6.783754] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=1,port=1
[    6.791874] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=2,port=1
[    6.799992] brcm-systemport 9300000.ethernet eth0: TDMA cfg,
size=256, switch q=3,port=1

Looking further in bcm_sysport_map_queues() we are getting the following:

    6.223042] brcm-systemport 9300000.ethernet eth0: mapping q=0, p=0
[    6.229369] brcm-systemport 9300000.ethernet eth0: mapping q=1, p=0
[    6.235659] brcm-systemport 9300000.ethernet eth0: mapping q=2, p=0
[    6.241945] brcm-systemport 9300000.ethernet eth0: mapping q=3, p=0
[    6.248232] brcm-systemport 9300000.ethernet eth0: mapping q=4, p=0
[    6.254519] brcm-systemport 9300000.ethernet eth0: mapping q=5, p=0
[    6.260805] brcm-systemport 9300000.ethernet eth0: mapping q=6, p=0
[    6.267092] brcm-systemport 9300000.ethernet eth0: mapping q=7, p=0

which means that the call to netif_set_real_num_tx_queues() that is
executed for the SYSTEMPORT Lite is not taking effect because it is
after the register_netdevice(). Insead of using a CHANGEUPPER notifier,
we can use a REGISTER notifier event and doing that works just fine with
the same semantics as the DSA notifier being removed. This incremental
patch on top of your patch works for me (tm):

https://github.com/ffainelli/linux/commit/f5095ab5c1f31db133d62273928b224674626b75

Thanks!
-- 
Florian

Re: [RFC PATCH net-next 3/4] net: systemport: use standard netdevice notifier to detect DSA presence

[Vladimir Oltean]

On Fri, Dec 18, 2020 at 08:08:56PM -0800, Florian Fainelli wrote:
> On 12/18/2020 2:38 PM, Vladimir Oltean wrote:
> > The SYSTEMPORT driver maps each port of the embedded Broadcom DSA switch
> > port to a certain queue of the master Ethernet controller. For that it
> > currently uses a dedicated notifier infrastructure which was added in
> > commit 60724d4bae14 ("net: dsa: Add support for DSA specific notifiers").
> >
> > However, since commit 2f1e8ea726e9 ("net: dsa: link interfaces with the
> > DSA master to get rid of lockdep warnings"), DSA is actually an upper of
> > the Broadcom SYSTEMPORT as far as the netdevice adjacency lists are
> > concerned. So naturally, the plain NETDEV_CHANGEUPPER net device notifiers
> > are emitted. It looks like there is enough API exposed by DSA to the
> > outside world already to make the call_dsa_notifiers API redundant. So
> > let's convert its only user to plain netdev notifiers.
> >
> > Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
>
> The CHANGEUPPER has a slightly different semantic than the current DSA
> notifier, and so events that would look like this during
> bcm_sysport_init_tx_ring() (good):
>
> [    6.781064] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=0
> [    6.789214] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=0
> [    6.797337] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=0
> [    6.805464] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=0
> [    6.813583] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=1
> [    6.821701] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=1
> [    6.829819] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=1
> [    6.837944] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=1
> [    6.846063] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=2
> [    6.854183] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=2
> [    6.862303] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=2
> [    6.870425] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=2
> [    6.878544] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=5
> [    6.886663] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=5
> [    6.894783] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=5
> [    6.902906] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=5
>
> now we are getting (bad):
>
> [    6.678157] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=0
> [    6.686302] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=0
> [    6.694434] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=0
> [    6.702554] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=0
> [    6.710679] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=0
> [    6.718797] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=0
> [    6.726914] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=0
> [    6.735033] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=0
> [    6.743156] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=1
> [    6.751275] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=1
> [    6.759395] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=1
> [    6.767514] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=1
> [    6.775636] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=1
> [    6.783754] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=1
> [    6.791874] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=1
> [    6.799992] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=1
>
> Looking further in bcm_sysport_map_queues() we are getting the following:
>
>     6.223042] brcm-systemport 9300000.ethernet eth0: mapping q=0, p=0
> [    6.229369] brcm-systemport 9300000.ethernet eth0: mapping q=1, p=0
> [    6.235659] brcm-systemport 9300000.ethernet eth0: mapping q=2, p=0
> [    6.241945] brcm-systemport 9300000.ethernet eth0: mapping q=3, p=0
> [    6.248232] brcm-systemport 9300000.ethernet eth0: mapping q=4, p=0
> [    6.254519] brcm-systemport 9300000.ethernet eth0: mapping q=5, p=0
> [    6.260805] brcm-systemport 9300000.ethernet eth0: mapping q=6, p=0
> [    6.267092] brcm-systemport 9300000.ethernet eth0: mapping q=7, p=0
>
> which means that the call to netif_set_real_num_tx_queues() that is
> executed for the SYSTEMPORT Lite is not taking effect because it is
> after the register_netdevice(). Insead of using a CHANGEUPPER notifier,
> we can use a REGISTER notifier event and doing that works just fine with
> the same semantics as the DSA notifier being removed. This incremental
> patch on top of your patch works for me (tm):
>
> https://github.com/ffainelli/linux/commit/f5095ab5c1f31db133d62273928b224674626b75

This is odd, the netif_set_real_num_tx_queues() call should not fail or
be ignored even if the interface was registered. I had tested this already
on my enetc + felix combo on LS1028A.

static int enetc_dsa_join(struct net_device *dev,
			  struct net_device *slave_dev)
{
	int err;

	netdev_err(slave_dev, "Hello!\n");

	err = netif_set_real_num_tx_queues(slave_dev,
					   slave_dev->num_tx_queues / 2);
	if (err)
		return err;

	netdev_err(slave_dev, "New number of real TX queues: %d\n",
		   slave_dev->real_num_tx_queues);

	return 0;
}

prints:

[    7.002328] mscc_felix 0000:00:00.5 swp0 (uninitialized): PHY [0000:00:00.3:10] driver [Microsemi GE VSC8514 SyncE] (irq=POLL)
[    7.021190] mscc_felix 0000:00:00.5 swp0: Hello!
[    7.028657] mscc_felix 0000:00:00.5 swp0: New number of real TX queues: 4
[    7.035589] mscc_felix 0000:00:00.5 swp0: Hello!
[    7.040380] mscc_felix 0000:00:00.5 swp0: New number of real TX queues: 4
[    7.290236] mscc_felix 0000:00:00.5 swp1 (uninitialized): PHY [0000:00:00.3:11] driver [Microsemi GE VSC8514 SyncE] (irq=POLL)
[    7.314383] mscc_felix 0000:00:00.5 swp1: Hello!
[    7.321292] mscc_felix 0000:00:00.5 swp1: New number of real TX queues: 4
[    7.328223] mscc_felix 0000:00:00.5 swp1: Hello!
[    7.332967] mscc_felix 0000:00:00.5 swp1: New number of real TX queues: 4
[    7.574254] mscc_felix 0000:00:00.5 swp2 (uninitialized): PHY [0000:00:00.3:12] driver [Microsemi GE VSC8514 SyncE] (irq=POLL)
[    7.598431] mscc_felix 0000:00:00.5 swp2: Hello!
[    7.605215] mscc_felix 0000:00:00.5 swp2: New number of real TX queues: 4
[    7.612145] mscc_felix 0000:00:00.5 swp2: Hello!
[    7.616889] mscc_felix 0000:00:00.5 swp2: New number of real TX queues: 4
[    7.858868] mscc_felix 0000:00:00.5 swp3 (uninitialized): PHY [0000:00:00.3:13] driver [Microsemi GE VSC8514 SyncE] (irq=POLL)
[    7.884240] mscc_felix 0000:00:00.5 swp3: Hello!
[    7.891086] mscc_felix 0000:00:00.5 swp3: New number of real TX queues: 4
[    7.898018] mscc_felix 0000:00:00.5 swp3: Hello!
[    7.902763] mscc_felix 0000:00:00.5 swp3: New number of real TX queues: 4

(I am not sure why the notifier is called twice though)

You are saying that here:

	num_tx_queues = slave_dev->real_num_tx_queues;

num_tx_queues remains assigned to 8? Does this mean that netif_set_real_num_tx_queues
has returned an error code? Can you check why?

Re: [RFC PATCH net-next 3/4] net: systemport: use standard netdevice notifier to detect DSA presence

[Florian Fainelli]

On 12/19/2020 4:12 AM, Vladimir Oltean wrote:
> On Fri, Dec 18, 2020 at 08:08:56PM -0800, Florian Fainelli wrote:
>> On 12/18/2020 2:38 PM, Vladimir Oltean wrote:
>>> The SYSTEMPORT driver maps each port of the embedded Broadcom DSA switch
>>> port to a certain queue of the master Ethernet controller. For that it
>>> currently uses a dedicated notifier infrastructure which was added in
>>> commit 60724d4bae14 ("net: dsa: Add support for DSA specific notifiers").
>>>
>>> However, since commit 2f1e8ea726e9 ("net: dsa: link interfaces with the
>>> DSA master to get rid of lockdep warnings"), DSA is actually an upper of
>>> the Broadcom SYSTEMPORT as far as the netdevice adjacency lists are
>>> concerned. So naturally, the plain NETDEV_CHANGEUPPER net device notifiers
>>> are emitted. It looks like there is enough API exposed by DSA to the
>>> outside world already to make the call_dsa_notifiers API redundant. So
>>> let's convert its only user to plain netdev notifiers.
>>>
>>> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
>>
>> The CHANGEUPPER has a slightly different semantic than the current DSA
>> notifier, and so events that would look like this during
>> bcm_sysport_init_tx_ring() (good):
>>
>> [    6.781064] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=0
>> [    6.789214] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=0
>> [    6.797337] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=0
>> [    6.805464] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=0
>> [    6.813583] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=1
>> [    6.821701] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=1
>> [    6.829819] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=1
>> [    6.837944] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=1
>> [    6.846063] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=2
>> [    6.854183] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=2
>> [    6.862303] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=2
>> [    6.870425] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=2
>> [    6.878544] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=5
>> [    6.886663] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=5
>> [    6.894783] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=5
>> [    6.902906] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=5
>>
>> now we are getting (bad):
>>
>> [    6.678157] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=0
>> [    6.686302] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=0
>> [    6.694434] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=0
>> [    6.702554] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=0
>> [    6.710679] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=0
>> [    6.718797] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=0
>> [    6.726914] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=0
>> [    6.735033] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=0
>> [    6.743156] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=1
>> [    6.751275] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=1
>> [    6.759395] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=1
>> [    6.767514] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=1
>> [    6.775636] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=0,port=1
>> [    6.783754] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=1,port=1
>> [    6.791874] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=2,port=1
>> [    6.799992] brcm-systemport 9300000.ethernet eth0: TDMA cfg, size=256, switch q=3,port=1
>>
>> Looking further in bcm_sysport_map_queues() we are getting the following:
>>
>>     6.223042] brcm-systemport 9300000.ethernet eth0: mapping q=0, p=0
>> [    6.229369] brcm-systemport 9300000.ethernet eth0: mapping q=1, p=0
>> [    6.235659] brcm-systemport 9300000.ethernet eth0: mapping q=2, p=0
>> [    6.241945] brcm-systemport 9300000.ethernet eth0: mapping q=3, p=0
>> [    6.248232] brcm-systemport 9300000.ethernet eth0: mapping q=4, p=0
>> [    6.254519] brcm-systemport 9300000.ethernet eth0: mapping q=5, p=0
>> [    6.260805] brcm-systemport 9300000.ethernet eth0: mapping q=6, p=0
>> [    6.267092] brcm-systemport 9300000.ethernet eth0: mapping q=7, p=0
>>
>> which means that the call to netif_set_real_num_tx_queues() that is
>> executed for the SYSTEMPORT Lite is not taking effect because it is
>> after the register_netdevice(). Insead of using a CHANGEUPPER notifier,
>> we can use a REGISTER notifier event and doing that works just fine with
>> the same semantics as the DSA notifier being removed. This incremental
>> patch on top of your patch works for me (tm):
>>
>> https://github.com/ffainelli/linux/commit/f5095ab5c1f31db133d62273928b224674626b75
> 
> This is odd, the netif_set_real_num_tx_queues() call should not fail or
> be ignored even if the interface was registered. I had tested this already
> on my enetc + felix combo on LS1028A.

Yes that part is fine, see below.

> 
> static int enetc_dsa_join(struct net_device *dev,
> 			  struct net_device *slave_dev)
> {
> 	int err;
> 
> 	netdev_err(slave_dev, "Hello!\n");
> 
> 	err = netif_set_real_num_tx_queues(slave_dev,
> 					   slave_dev->num_tx_queues / 2);
> 	if (err)
> 		return err;
> 
> 	netdev_err(slave_dev, "New number of real TX queues: %d\n",
> 		   slave_dev->real_num_tx_queues);
> 
> 	return 0;
> }
> 
> prints:
> 
> [    7.002328] mscc_felix 0000:00:00.5 swp0 (uninitialized): PHY [0000:00:00.3:10] driver [Microsemi GE VSC8514 SyncE] (irq=POLL)
> [    7.021190] mscc_felix 0000:00:00.5 swp0: Hello!
> [    7.028657] mscc_felix 0000:00:00.5 swp0: New number of real TX queues: 4
> [    7.035589] mscc_felix 0000:00:00.5 swp0: Hello!
> [    7.040380] mscc_felix 0000:00:00.5 swp0: New number of real TX queues: 4
> [    7.290236] mscc_felix 0000:00:00.5 swp1 (uninitialized): PHY [0000:00:00.3:11] driver [Microsemi GE VSC8514 SyncE] (irq=POLL)
> [    7.314383] mscc_felix 0000:00:00.5 swp1: Hello!
> [    7.321292] mscc_felix 0000:00:00.5 swp1: New number of real TX queues: 4
> [    7.328223] mscc_felix 0000:00:00.5 swp1: Hello!
> [    7.332967] mscc_felix 0000:00:00.5 swp1: New number of real TX queues: 4
> [    7.574254] mscc_felix 0000:00:00.5 swp2 (uninitialized): PHY [0000:00:00.3:12] driver [Microsemi GE VSC8514 SyncE] (irq=POLL)
> [    7.598431] mscc_felix 0000:00:00.5 swp2: Hello!
> [    7.605215] mscc_felix 0000:00:00.5 swp2: New number of real TX queues: 4
> [    7.612145] mscc_felix 0000:00:00.5 swp2: Hello!
> [    7.616889] mscc_felix 0000:00:00.5 swp2: New number of real TX queues: 4
> [    7.858868] mscc_felix 0000:00:00.5 swp3 (uninitialized): PHY [0000:00:00.3:13] driver [Microsemi GE VSC8514 SyncE] (irq=POLL)
> [    7.884240] mscc_felix 0000:00:00.5 swp3: Hello!
> [    7.891086] mscc_felix 0000:00:00.5 swp3: New number of real TX queues: 4
> [    7.898018] mscc_felix 0000:00:00.5 swp3: Hello!
> [    7.902763] mscc_felix 0000:00:00.5 swp3: New number of real TX queues: 4
> 
> (I am not sure why the notifier is called twice though)

This is the actual issue that messes up the queue assignment for
bcmsysport because we assign queue/switch port pairs and don't really
expect to be re-doing that once ring->inspect is set to true.

> 
> You are saying that here:
> 
> 	num_tx_queues = slave_dev->real_num_tx_queues;
> 
> num_tx_queues remains assigned to 8? Does this mean that netif_set_real_num_tx_queues
> has returned an error code? Can you check why?
> 

The call to netif_set_real_num_tx_queues() succeeds and
slave_dev->real_num_tx_queues is changed to 4 accordingly. The loop that
assigns the internal queue mapping (priv->ring_map) is correctly limited
to 4, however we get two calls per switch port instead of one. I did not
have much time to debug why we get called twice but I will be looking
into this tomorrow.
-- 
Florian

Re: [RFC PATCH net-next 3/4] net: systemport: use standard netdevice notifier to detect DSA presence

[Florian Fainelli]

On 12/20/2020 8:53 PM, Florian Fainelli wrote:
> 
> The call to netif_set_real_num_tx_queues() succeeds and
> slave_dev->real_num_tx_queues is changed to 4 accordingly. The loop that
> assigns the internal queue mapping (priv->ring_map) is correctly limited
> to 4, however we get two calls per switch port instead of one. I did not
> have much time to debug why we get called twice but I will be looking
> into this tomorrow.

There was not any bug other than there are two instances of a SYSTEMPORT
device in my system and they both receive the same notification.

So we do need to qualify which of the notifier block matches the device
of interest, because if we do extract the private structure from the
device being notified, it is always going to match.

Incremental fixup here:

https://github.com/ffainelli/linux/commit/0eea16e706a73c56a36d701df483ff73211aae7f

and you can add Tested-by: Florian Fainelli <f.fainelli@gmail.com> when
you resubmit.

Thanks, this is a really nice cleanup.
-- 
Florian

Re: [RFC PATCH net-next 3/4] net: systemport: use standard netdevice notifier to detect DSA presence

[Vladimir Oltean]

On Mon, Dec 21, 2020 at 02:33:16PM -0800, Florian Fainelli wrote:
> On 12/20/2020 8:53 PM, Florian Fainelli wrote:
> > The call to netif_set_real_num_tx_queues() succeeds and
> > slave_dev->real_num_tx_queues is changed to 4 accordingly. The loop that
> > assigns the internal queue mapping (priv->ring_map) is correctly limited
> > to 4, however we get two calls per switch port instead of one. I did not
> > have much time to debug why we get called twice but I will be looking
> > into this tomorrow.
>
> There was not any bug other than there are two instances of a SYSTEMPORT
> device in my system and they both receive the same notification.
>
> So we do need to qualify which of the notifier block matches the device
> of interest, because if we do extract the private structure from the
> device being notified, it is always going to match.
>
> Incremental fixup here:
>
> https://github.com/ffainelli/linux/commit/0eea16e706a73c56a36d701df483ff73211aae7f

...duh.
And when you come to think that I had deleted that code in my patch, not
understanding what it's for... Coincidentally this is also the reason
why I got the prints twice. Sorry :(

>
> and you can add Tested-by: Florian Fainelli <f.fainelli@gmail.com> when
> you resubmit.
>
> Thanks, this is a really nice cleanup.

Thanks.

Do you think we need some getters for dp->index and dp->ds->index, to preserve
some sort of data structure encapsulation from the outside world (although it's
not as if the members of struct dsa_switch and struct dsa_port still couldn't
be accessed directly)?

But then, there's the other aspect. We would have some shiny accessors for DSA
properties, but we're resetting the net_device's number of TX queues.
So much for data encapsulation.

Re: [RFC PATCH net-next 3/4] net: systemport: use standard netdevice notifier to detect DSA presence

[Florian Fainelli]

On 12/21/2020 3:06 PM, Vladimir Oltean wrote:
> On Mon, Dec 21, 2020 at 02:33:16PM -0800, Florian Fainelli wrote:
>> On 12/20/2020 8:53 PM, Florian Fainelli wrote:
>>> The call to netif_set_real_num_tx_queues() succeeds and
>>> slave_dev->real_num_tx_queues is changed to 4 accordingly. The loop that
>>> assigns the internal queue mapping (priv->ring_map) is correctly limited
>>> to 4, however we get two calls per switch port instead of one. I did not
>>> have much time to debug why we get called twice but I will be looking
>>> into this tomorrow.
>>
>> There was not any bug other than there are two instances of a SYSTEMPORT
>> device in my system and they both receive the same notification.
>>
>> So we do need to qualify which of the notifier block matches the device
>> of interest, because if we do extract the private structure from the
>> device being notified, it is always going to match.
>>
>> Incremental fixup here:
>>
>> https://github.com/ffainelli/linux/commit/0eea16e706a73c56a36d701df483ff73211aae7f
> 
> ...duh.
> And when you come to think that I had deleted that code in my patch, not
> understanding what it's for... Coincidentally this is also the reason
> why I got the prints twice. Sorry :(

No worries, I had it "automatically" in my experiment with the
REGISTER/UNREGISTER and it only clicked this morning this was the key
thing here.

> 
>>
>> and you can add Tested-by: Florian Fainelli <f.fainelli@gmail.com> when
>> you resubmit.
>>
>> Thanks, this is a really nice cleanup.
> 
> Thanks.
> 
> Do you think we need some getters for dp->index and dp->ds->index, to preserve
> some sort of data structure encapsulation from the outside world (although it's
> not as if the members of struct dsa_switch and struct dsa_port still couldn't
> be accessed directly)?
> 
> But then, there's the other aspect. We would have some shiny accessors for DSA
> properties, but we're resetting the net_device's number of TX queues.
> So much for data encapsulation.

If we move the dsa_port structure definition to be more private, and say
within dsa_priv.h, we will have to create quite some bit of churn within
the DSA driver to make them use getters and setters. Russell did a nice
job with the encapsulation with phylink and that would really be a good
model to follow, however this was a clean slate. It seems to me for now
that this is not worth the trouble.

Despite accessing the TX queues directly, the original DSA notifier was
trying to provide all the necessary data to the recipient of the
notification without having to know too much about what a DSA device is
but the amount of code eliminated is of superior value IMHO.
-- 
Florian

Re: [RFC PATCH net-next 3/4] net: systemport: use standard netdevice notifier to detect DSA presence

[Vladimir Oltean]

On Mon, Dec 21, 2020 at 03:17:02PM -0800, Florian Fainelli wrote:
> > Do you think we need some getters for dp->index and dp->ds->index, to preserve
> > some sort of data structure encapsulation from the outside world (although it's
> > not as if the members of struct dsa_switch and struct dsa_port still couldn't
> > be accessed directly)?
> >
> > But then, there's the other aspect. We would have some shiny accessors for DSA
> > properties, but we're resetting the net_device's number of TX queues.
> > So much for data encapsulation.
>
> If we move the dsa_port structure definition to be more private, and say
> within dsa_priv.h, we will have to create quite some bit of churn within
> the DSA driver to make them use getters and setters. Russell did a nice
> job with the encapsulation with phylink and that would really be a good
> model to follow, however this was a clean slate. It seems to me for now
> that this is not worth the trouble.

We could make include/net/dsa.h a semi-private ("friend") header that
the DSA drivers could keep including, but the non-DSA world wouldn't.
Then we could create a new one in its place, with just the stuff that
the outside world needs: netdev_uses_dsa, etc.

> Despite accessing the TX queues directly, the original DSA notifier was
> trying to provide all the necessary data to the recipient of the
> notification without having to know too much about what a DSA device is

I know. Personally, accessing dp->cpu_dp->master directly is where I was
going to draw the line and say that it's better to just keep the code as
is. What motivated me to make the change in the first place was the
realization that we now have the linkage visible from the outside world.

> but the amount of code eliminated is of superior value IMHO.

It's still a compromise, really. The atomic DSA notifier was ok, but if
we could do without it, why not...