[PATCH net-next 0/5] L2 multicast forwarding for Ocelot switch

[PATCH net-next 0/5] L2 multicast forwarding for Ocelot switch

[Vladimir Oltean]

This series enables the mscc_ocelot switch to forward raw L2 (non-IP)
mdb entries as configured by the bridge driver after this patch:

https://patchwork.ozlabs.org/project/netdev/patch/20201028233831.610076-1-vladimir.oltean@nxp.com/

Vladimir Oltean (5):
  net: mscc: ocelot: classify L2 mdb entries as LOCKED
  net: mscc: ocelot: use ether_addr_copy
  net: mscc: ocelot: remove the "new" variable in ocelot_port_mdb_add
  net: mscc: ocelot: make entry_type a member of struct ocelot_multicast
  net: mscc: ocelot: support L2 multicast entries

 drivers/net/ethernet/mscc/ocelot.c | 147 ++++++++++++++++++++---------
 drivers/net/ethernet/mscc/ocelot.h |  31 ++++--
 include/soc/mscc/ocelot.h          |   1 +
 3 files changed, 125 insertions(+), 54 deletions(-)

-- 
2.25.1

[PATCH net-next 1/5] net: mscc: ocelot: classify L2 mdb entries as LOCKED

[Vladimir Oltean]

ocelot.h says:

/* MAC table entry types.
 * ENTRYTYPE_NORMAL is subject to aging.
 * ENTRYTYPE_LOCKED is not subject to aging.
 * ENTRYTYPE_MACv4 is not subject to aging. For IPv4 multicast.
 * ENTRYTYPE_MACv6 is not subject to aging. For IPv6 multicast.
 */

We don't want the permanent entries added with 'bridge mdb' to be
subject to aging.

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 drivers/net/ethernet/mscc/ocelot.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/mscc/ocelot.c b/drivers/net/ethernet/mscc/ocelot.c
index 70bf8c67d7ef..25152f1f2939 100644
--- a/drivers/net/ethernet/mscc/ocelot.c
+++ b/drivers/net/ethernet/mscc/ocelot.c
@@ -958,7 +958,7 @@ static enum macaccess_entry_type ocelot_classify_mdb(const unsigned char *addr)
 		return ENTRYTYPE_MACv4;
 	if (addr[0] == 0x33 && addr[1] == 0x33)
 		return ENTRYTYPE_MACv6;
-	return ENTRYTYPE_NORMAL;
+	return ENTRYTYPE_LOCKED;
 }
 
 static int ocelot_mdb_get_pgid(struct ocelot *ocelot,
-- 
2.25.1

Re: [PATCH net-next 1/5] net: mscc: ocelot: classify L2 mdb entries as LOCKED

[Florian Fainelli]

On 10/28/2020 7:27 PM, Vladimir Oltean wrote:
> ocelot.h says:
> 
> /* MAC table entry types.
>  * ENTRYTYPE_NORMAL is subject to aging.
>  * ENTRYTYPE_LOCKED is not subject to aging.
>  * ENTRYTYPE_MACv4 is not subject to aging. For IPv4 multicast.
>  * ENTRYTYPE_MACv6 is not subject to aging. For IPv6 multicast.
>  */
> 
> We don't want the permanent entries added with 'bridge mdb' to be
> subject to aging.
> 
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>

Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
-- 
Florian

[PATCH net-next 2/5] net: mscc: ocelot: use ether_addr_copy

[Vladimir Oltean]

Since a helper is available for copying Ethernet addresses, let's use it.

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 drivers/net/ethernet/mscc/ocelot.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/mscc/ocelot.c b/drivers/net/ethernet/mscc/ocelot.c
index 25152f1f2939..763d0277eeae 100644
--- a/drivers/net/ethernet/mscc/ocelot.c
+++ b/drivers/net/ethernet/mscc/ocelot.c
@@ -997,7 +997,7 @@ static void ocelot_encode_ports_to_mdb(unsigned char *addr,
 				       struct ocelot_multicast *mc,
 				       enum macaccess_entry_type entry_type)
 {
-	memcpy(addr, mc->addr, ETH_ALEN);
+	ether_addr_copy(addr, mc->addr);
 
 	if (entry_type == ENTRYTYPE_MACv4) {
 		addr[0] = 0;
@@ -1042,7 +1042,7 @@ int ocelot_port_mdb_add(struct ocelot *ocelot, int port,
 		if (!mc)
 			return -ENOMEM;
 
-		memcpy(mc->addr, mdb->addr, ETH_ALEN);
+		ether_addr_copy(mc->addr, mdb->addr);
 		mc->vid = vid;
 		mc->pgid = pgid;
 
-- 
2.25.1

Re: [PATCH net-next 2/5] net: mscc: ocelot: use ether_addr_copy

[Florian Fainelli]

On 10/28/2020 7:27 PM, Vladimir Oltean wrote:
> Since a helper is available for copying Ethernet addresses, let's use it.
> 
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>

Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
-- 
Florian

[PATCH net-next 3/5] net: mscc: ocelot: remove the "new" variable in ocelot_port_mdb_add

[Vladimir Oltean]

It is Not Needed, a comment will suffice.

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 drivers/net/ethernet/mscc/ocelot.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/drivers/net/ethernet/mscc/ocelot.c b/drivers/net/ethernet/mscc/ocelot.c
index 763d0277eeae..ea49d715c9d0 100644
--- a/drivers/net/ethernet/mscc/ocelot.c
+++ b/drivers/net/ethernet/mscc/ocelot.c
@@ -1017,7 +1017,6 @@ int ocelot_port_mdb_add(struct ocelot *ocelot, int port,
 	unsigned char addr[ETH_ALEN];
 	struct ocelot_multicast *mc;
 	u16 vid = mdb->vid;
-	bool new = false;
 
 	if (port == ocelot->npi)
 		port = ocelot->num_phys_ports;
@@ -1029,6 +1028,7 @@ int ocelot_port_mdb_add(struct ocelot *ocelot, int port,
 
 	mc = ocelot_multicast_get(ocelot, mdb->addr, vid);
 	if (!mc) {
+		/* New entry */
 		int pgid = ocelot_mdb_get_pgid(ocelot, entry_type);
 
 		if (pgid < 0) {
@@ -1047,10 +1047,7 @@ int ocelot_port_mdb_add(struct ocelot *ocelot, int port,
 		mc->pgid = pgid;
 
 		list_add_tail(&mc->list, &ocelot->multicast);
-		new = true;
-	}
-
-	if (!new) {
+	} else {
 		ocelot_encode_ports_to_mdb(addr, mc, entry_type);
 		ocelot_mact_forget(ocelot, addr, vid);
 	}
-- 
2.25.1

Re: [PATCH net-next 3/5] net: mscc: ocelot: remove the "new" variable in ocelot_port_mdb_add

[Florian Fainelli]

On 10/28/2020 7:27 PM, Vladimir Oltean wrote:
> It is Not Needed, a comment will suffice.
> 
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>

Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
-- 
Florian

[PATCH net-next 4/5] net: mscc: ocelot: make entry_type a member of struct ocelot_multicast

[Vladimir Oltean]

This saves a re-classification of the MDB address on deletion.

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 drivers/net/ethernet/mscc/ocelot.c | 51 +++++++++++++++---------------
 drivers/net/ethernet/mscc/ocelot.h | 17 +++++-----
 2 files changed, 34 insertions(+), 34 deletions(-)

diff --git a/drivers/net/ethernet/mscc/ocelot.c b/drivers/net/ethernet/mscc/ocelot.c
index ea49d715c9d0..713ab6ec8c8d 100644
--- a/drivers/net/ethernet/mscc/ocelot.c
+++ b/drivers/net/ethernet/mscc/ocelot.c
@@ -962,7 +962,7 @@ static enum macaccess_entry_type ocelot_classify_mdb(const unsigned char *addr)
 }
 
 static int ocelot_mdb_get_pgid(struct ocelot *ocelot,
-			       enum macaccess_entry_type entry_type)
+			       const struct ocelot_multicast *mc)
 {
 	int pgid;
 
@@ -971,8 +971,8 @@ static int ocelot_mdb_get_pgid(struct ocelot *ocelot,
 	 * destination mask table (PGID), the destination set is programmed as
 	 * part of the entry MAC address.", and the DEST_IDX is set to 0.
 	 */
-	if (entry_type == ENTRYTYPE_MACv4 ||
-	    entry_type == ENTRYTYPE_MACv6)
+	if (mc->entry_type == ENTRYTYPE_MACv4 ||
+	    mc->entry_type == ENTRYTYPE_MACv6)
 		return 0;
 
 	for_each_nonreserved_multicast_dest_pgid(ocelot, pgid) {
@@ -994,16 +994,15 @@ static int ocelot_mdb_get_pgid(struct ocelot *ocelot,
 }
 
 static void ocelot_encode_ports_to_mdb(unsigned char *addr,
-				       struct ocelot_multicast *mc,
-				       enum macaccess_entry_type entry_type)
+				       struct ocelot_multicast *mc)
 {
 	ether_addr_copy(addr, mc->addr);
 
-	if (entry_type == ENTRYTYPE_MACv4) {
+	if (mc->entry_type == ENTRYTYPE_MACv4) {
 		addr[0] = 0;
 		addr[1] = mc->ports >> 8;
 		addr[2] = mc->ports & 0xff;
-	} else if (entry_type == ENTRYTYPE_MACv6) {
+	} else if (mc->entry_type == ENTRYTYPE_MACv6) {
 		addr[0] = mc->ports >> 8;
 		addr[1] = mc->ports & 0xff;
 	}
@@ -1013,7 +1012,6 @@ int ocelot_port_mdb_add(struct ocelot *ocelot, int port,
 			const struct switchdev_obj_port_mdb *mdb)
 {
 	struct ocelot_port *ocelot_port = ocelot->ports[port];
-	enum macaccess_entry_type entry_type;
 	unsigned char addr[ETH_ALEN];
 	struct ocelot_multicast *mc;
 	u16 vid = mdb->vid;
@@ -1024,12 +1022,20 @@ int ocelot_port_mdb_add(struct ocelot *ocelot, int port,
 	if (!vid)
 		vid = ocelot_port->pvid;
 
-	entry_type = ocelot_classify_mdb(mdb->addr);
-
 	mc = ocelot_multicast_get(ocelot, mdb->addr, vid);
 	if (!mc) {
 		/* New entry */
-		int pgid = ocelot_mdb_get_pgid(ocelot, entry_type);
+		int pgid;
+
+		mc = devm_kzalloc(ocelot->dev, sizeof(*mc), GFP_KERNEL);
+		if (!mc)
+			return -ENOMEM;
+
+		mc->entry_type = ocelot_classify_mdb(mdb->addr);
+		ether_addr_copy(mc->addr, mdb->addr);
+		mc->vid = vid;
+
+		pgid = ocelot_mdb_get_pgid(ocelot, mc);
 
 		if (pgid < 0) {
 			dev_err(ocelot->dev,
@@ -1038,24 +1044,19 @@ int ocelot_port_mdb_add(struct ocelot *ocelot, int port,
 			return -ENOSPC;
 		}
 
-		mc = devm_kzalloc(ocelot->dev, sizeof(*mc), GFP_KERNEL);
-		if (!mc)
-			return -ENOMEM;
-
-		ether_addr_copy(mc->addr, mdb->addr);
-		mc->vid = vid;
 		mc->pgid = pgid;
 
 		list_add_tail(&mc->list, &ocelot->multicast);
 	} else {
-		ocelot_encode_ports_to_mdb(addr, mc, entry_type);
+		ocelot_encode_ports_to_mdb(addr, mc);
 		ocelot_mact_forget(ocelot, addr, vid);
 	}
 
 	mc->ports |= BIT(port);
-	ocelot_encode_ports_to_mdb(addr, mc, entry_type);
+	ocelot_encode_ports_to_mdb(addr, mc);
 
-	return ocelot_mact_learn(ocelot, mc->pgid, addr, vid, entry_type);
+	return ocelot_mact_learn(ocelot, mc->pgid, addr, vid,
+				 mc->entry_type);
 }
 EXPORT_SYMBOL(ocelot_port_mdb_add);
 
@@ -1063,7 +1064,6 @@ int ocelot_port_mdb_del(struct ocelot *ocelot, int port,
 			const struct switchdev_obj_port_mdb *mdb)
 {
 	struct ocelot_port *ocelot_port = ocelot->ports[port];
-	enum macaccess_entry_type entry_type;
 	unsigned char addr[ETH_ALEN];
 	struct ocelot_multicast *mc;
 	u16 vid = mdb->vid;
@@ -1078,9 +1078,7 @@ int ocelot_port_mdb_del(struct ocelot *ocelot, int port,
 	if (!mc)
 		return -ENOENT;
 
-	entry_type = ocelot_classify_mdb(mdb->addr);
-
-	ocelot_encode_ports_to_mdb(addr, mc, entry_type);
+	ocelot_encode_ports_to_mdb(addr, mc);
 	ocelot_mact_forget(ocelot, addr, vid);
 
 	mc->ports &= ~BIT(port);
@@ -1090,9 +1088,10 @@ int ocelot_port_mdb_del(struct ocelot *ocelot, int port,
 		return 0;
 	}
 
-	ocelot_encode_ports_to_mdb(addr, mc, entry_type);
+	ocelot_encode_ports_to_mdb(addr, mc);
 
-	return ocelot_mact_learn(ocelot, mc->pgid, addr, vid, entry_type);
+	return ocelot_mact_learn(ocelot, mc->pgid, addr, vid,
+				 mc->entry_type);
 }
 EXPORT_SYMBOL(ocelot_port_mdb_del);
 
diff --git a/drivers/net/ethernet/mscc/ocelot.h b/drivers/net/ethernet/mscc/ocelot.h
index abb407dff93c..7f8b34c49971 100644
--- a/drivers/net/ethernet/mscc/ocelot.h
+++ b/drivers/net/ethernet/mscc/ocelot.h
@@ -41,14 +41,6 @@ struct frame_info {
 	u32 timestamp;	/* rew_val */
 };
 
-struct ocelot_multicast {
-	struct list_head list;
-	unsigned char addr[ETH_ALEN];
-	u16 vid;
-	u16 ports;
-	int pgid;
-};
-
 struct ocelot_port_tc {
 	bool block_shared;
 	unsigned long offload_cnt;
@@ -87,6 +79,15 @@ enum macaccess_entry_type {
 	ENTRYTYPE_MACv6,
 };
 
+struct ocelot_multicast {
+	struct list_head list;
+	enum macaccess_entry_type entry_type;
+	unsigned char addr[ETH_ALEN];
+	u16 vid;
+	u16 ports;
+	int pgid;
+};
+
 int ocelot_port_fdb_do_dump(const unsigned char *addr, u16 vid,
 			    bool is_static, void *data);
 int ocelot_mact_learn(struct ocelot *ocelot, int port,
-- 
2.25.1

Re: [PATCH net-next 4/5] net: mscc: ocelot: make entry_type a member of struct ocelot_multicast

[Florian Fainelli]

On 10/28/2020 7:27 PM, Vladimir Oltean wrote:
> This saves a re-classification of the MDB address on deletion.
> 
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
> ---

[snip]

>  	mc = ocelot_multicast_get(ocelot, mdb->addr, vid);
>  	if (!mc) {
>  		/* New entry */
> -		int pgid = ocelot_mdb_get_pgid(ocelot, entry_type);
> +		int pgid;
> +
> +		mc = devm_kzalloc(ocelot->dev, sizeof(*mc), GFP_KERNEL);

If the MDB object is programmed with SWITCHDEV_OBJ_ID_HOST_MDB then you
would need this gfp_t to be GFP_ATOMIC per
net/bridge/br_mdb.c::__br_mdb_notify, if this is a regular
SWITCHDEV_OBJ_ID_MDB then GFP_KERNEL appears to be fine.

Looks like this existed before, so that might have to be fixed separately.

Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
-- 
Florian

Re: [PATCH net-next 4/5] net: mscc: ocelot: make entry_type a member of struct ocelot_multicast

[Vladimir Oltean]

On Wed, Oct 28, 2020 at 07:37:09PM -0700, Florian Fainelli wrote:
> If the MDB object is programmed with SWITCHDEV_OBJ_ID_HOST_MDB then you
> would need this gfp_t to be GFP_ATOMIC per
> net/bridge/br_mdb.c::__br_mdb_notify, if this is a regular
> SWITCHDEV_OBJ_ID_MDB then GFP_KERNEL appears to be fine.

Good point, I think I would need to do something more radical anyway, as
programming anything into the MAC table calls:
ocelot_mact_learn
-> ocelot_mact_wait_for_completion
   -> readx_poll_timeout
I'm thinking to move all of this into a workqueue.

Re: [PATCH net-next 4/5] net: mscc: ocelot: make entry_type a member of struct ocelot_multicast

[Jakub Kicinski]

On Thu, 29 Oct 2020 04:27:37 +0200 Vladimir Oltean wrote:
> +		mc = devm_kzalloc(ocelot->dev, sizeof(*mc), GFP_KERNEL);
> +		if (!mc)
> +			return -ENOMEM;
> +
> +		mc->entry_type = ocelot_classify_mdb(mdb->addr);
> +		ether_addr_copy(mc->addr, mdb->addr);
> +		mc->vid = vid;
> +
> +		pgid = ocelot_mdb_get_pgid(ocelot, mc);
>  
>  		if (pgid < 0) {
>  			dev_err(ocelot->dev,
> @@ -1038,24 +1044,19 @@ int ocelot_port_mdb_add(struct ocelot *ocelot, int port,
>  			return -ENOSPC;
>  		}

Transitionally leaking mc here on pgid < 0

Re: [PATCH net-next 4/5] net: mscc: ocelot: make entry_type a member of struct ocelot_multicast

[Vladimir Oltean]

On Fri, Oct 30, 2020 at 06:16:31PM -0700, Jakub Kicinski wrote:
> On Thu, 29 Oct 2020 04:27:37 +0200 Vladimir Oltean wrote:
> > +		mc = devm_kzalloc(ocelot->dev, sizeof(*mc), GFP_KERNEL);
> > +		if (!mc)
> > +			return -ENOMEM;
> > +
> > +		mc->entry_type = ocelot_classify_mdb(mdb->addr);
> > +		ether_addr_copy(mc->addr, mdb->addr);
> > +		mc->vid = vid;
> > +
> > +		pgid = ocelot_mdb_get_pgid(ocelot, mc);
> >  
> >  		if (pgid < 0) {
> >  			dev_err(ocelot->dev,
> > @@ -1038,24 +1044,19 @@ int ocelot_port_mdb_add(struct ocelot *ocelot, int port,
> >  			return -ENOSPC;
> >  		}
> 
> Transitionally leaking mc here on pgid < 0

Is it a real leakage if it's allocated with devm though? At some point
it's still going to be freed. Nonetheless I agree there's still a lot of
work to do. Maybe I didn't choose the best moment to concentrate on a
new feature, should have focused on cleanup more beforehand, including a
change from devm to plain allocation/free of resources.

[PATCH net-next 5/5] net: mscc: ocelot: support L2 multicast entries

[Vladimir Oltean]

There is one main difference in mscc_ocelot between IP multicast and L2
multicast. With IP multicast, destination ports are encoded into the
upper bytes of the multicast MAC address. Example: to deliver the
address 01:00:5E:11:22:33 to ports 3, 8, and 9, one would need to
program the address of 00:03:08:11:22:33 into hardware. Whereas for L2
multicast, the MAC table entry points to a Port Group ID (PGID), and
that PGID contains the port mask that the packet will be forwarded to.
As to why it is this way, no clue. My guess is that not all port
combinations can be supported simultaneously with the limited number of
PGIDs, and this was somehow an issue for IP multicast but not for L2
multicast. Anyway.

Prior to this change, the raw L2 multicast code was bogus, due to the
fact that there wasn't really any way to test it using the bridge code.
There were 2 issues:
- A multicast PGID was allocated for each MDB entry, but it wasn't in
  fact programmed to hardware. It was dummy.
- In fact we don't want to reserve a multicast PGID for every single MDB
  entry. That would be odd because we can only have ~60 PGIDs, but
  thousands of MDB entries. So instead, we want to reserve a multicast
  PGID for every single port combination for multicast traffic. And
  since we can have 2 (or more) MDB entries delivered to the same port
  group (and therefore PGID), we need to reference-count the PGIDs.

Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 drivers/net/ethernet/mscc/ocelot.c | 109 ++++++++++++++++++++++-------
 drivers/net/ethernet/mscc/ocelot.h |  16 ++++-
 include/soc/mscc/ocelot.h          |   1 +
 3 files changed, 100 insertions(+), 26 deletions(-)

diff --git a/drivers/net/ethernet/mscc/ocelot.c b/drivers/net/ethernet/mscc/ocelot.c
index 713ab6ec8c8d..323dbd30661a 100644
--- a/drivers/net/ethernet/mscc/ocelot.c
+++ b/drivers/net/ethernet/mscc/ocelot.c
@@ -961,10 +961,37 @@ static enum macaccess_entry_type ocelot_classify_mdb(const unsigned char *addr)
 	return ENTRYTYPE_LOCKED;
 }
 
-static int ocelot_mdb_get_pgid(struct ocelot *ocelot,
-			       const struct ocelot_multicast *mc)
+static struct ocelot_pgid *ocelot_pgid_alloc(struct ocelot *ocelot, int index,
+					     unsigned long ports)
 {
-	int pgid;
+	struct ocelot_pgid *pgid;
+
+	pgid = kzalloc(sizeof(*pgid), GFP_KERNEL);
+	if (!pgid)
+		return ERR_PTR(-ENOMEM);
+
+	pgid->ports = ports;
+	pgid->index = index;
+	refcount_set(&pgid->refcount, 1);
+	list_add_tail(&pgid->list, &ocelot->pgids);
+
+	return pgid;
+}
+
+static void ocelot_pgid_free(struct ocelot *ocelot, struct ocelot_pgid *pgid)
+{
+	if (!refcount_dec_and_test(&pgid->refcount))
+		return;
+
+	list_del(&pgid->list);
+	kfree(pgid);
+}
+
+static struct ocelot_pgid *ocelot_mdb_get_pgid(struct ocelot *ocelot,
+					       const struct ocelot_multicast *mc)
+{
+	struct ocelot_pgid *pgid;
+	int index;
 
 	/* According to VSC7514 datasheet 3.9.1.5 IPv4 Multicast Entries and
 	 * 3.9.1.6 IPv6 Multicast Entries, "Instead of a lookup in the
@@ -973,24 +1000,34 @@ static int ocelot_mdb_get_pgid(struct ocelot *ocelot,
 	 */
 	if (mc->entry_type == ENTRYTYPE_MACv4 ||
 	    mc->entry_type == ENTRYTYPE_MACv6)
-		return 0;
+		return ocelot_pgid_alloc(ocelot, 0, mc->ports);
+
+	list_for_each_entry(pgid, &ocelot->pgids, list) {
+		/* When searching for a nonreserved multicast PGID, ignore the
+		 * dummy PGID of zero that we have for MACv4/MACv6 entries
+		 */
+		if (pgid->index && pgid->ports == mc->ports) {
+			refcount_inc(&pgid->refcount);
+			return pgid;
+		}
+	}
 
-	for_each_nonreserved_multicast_dest_pgid(ocelot, pgid) {
-		struct ocelot_multicast *mc;
+	/* Search for a free index in the nonreserved multicast PGID area */
+	for_each_nonreserved_multicast_dest_pgid(ocelot, index) {
 		bool used = false;
 
-		list_for_each_entry(mc, &ocelot->multicast, list) {
-			if (mc->pgid == pgid) {
+		list_for_each_entry(pgid, &ocelot->pgids, list) {
+			if (pgid->index == index) {
 				used = true;
 				break;
 			}
 		}
 
 		if (!used)
-			return pgid;
+			return ocelot_pgid_alloc(ocelot, index, mc->ports);
 	}
 
-	return -1;
+	return ERR_PTR(-ENOSPC);
 }
 
 static void ocelot_encode_ports_to_mdb(unsigned char *addr,
@@ -1014,6 +1051,7 @@ int ocelot_port_mdb_add(struct ocelot *ocelot, int port,
 	struct ocelot_port *ocelot_port = ocelot->ports[port];
 	unsigned char addr[ETH_ALEN];
 	struct ocelot_multicast *mc;
+	struct ocelot_pgid *pgid;
 	u16 vid = mdb->vid;
 
 	if (port == ocelot->npi)
@@ -1025,8 +1063,6 @@ int ocelot_port_mdb_add(struct ocelot *ocelot, int port,
 	mc = ocelot_multicast_get(ocelot, mdb->addr, vid);
 	if (!mc) {
 		/* New entry */
-		int pgid;
-
 		mc = devm_kzalloc(ocelot->dev, sizeof(*mc), GFP_KERNEL);
 		if (!mc)
 			return -ENOMEM;
@@ -1035,27 +1071,36 @@ int ocelot_port_mdb_add(struct ocelot *ocelot, int port,
 		ether_addr_copy(mc->addr, mdb->addr);
 		mc->vid = vid;
 
-		pgid = ocelot_mdb_get_pgid(ocelot, mc);
-
-		if (pgid < 0) {
-			dev_err(ocelot->dev,
-				"No more PGIDs available for mdb %pM vid %d\n",
-				mdb->addr, vid);
-			return -ENOSPC;
-		}
-
-		mc->pgid = pgid;
-
 		list_add_tail(&mc->list, &ocelot->multicast);
 	} else {
+		/* Existing entry. Clean up the current port mask from
+		 * hardware now, because we'll be modifying it.
+		 */
+		ocelot_pgid_free(ocelot, mc->pgid);
 		ocelot_encode_ports_to_mdb(addr, mc);
 		ocelot_mact_forget(ocelot, addr, vid);
 	}
 
 	mc->ports |= BIT(port);
+
+	pgid = ocelot_mdb_get_pgid(ocelot, mc);
+	if (IS_ERR(pgid)) {
+		dev_err(ocelot->dev,
+			"Cannot allocate PGID for mdb %pM vid %d\n",
+			mc->addr, mc->vid);
+		devm_kfree(ocelot->dev, mc);
+		return PTR_ERR(pgid);
+	}
+	mc->pgid = pgid;
+
 	ocelot_encode_ports_to_mdb(addr, mc);
 
-	return ocelot_mact_learn(ocelot, mc->pgid, addr, vid,
+	if (mc->entry_type != ENTRYTYPE_MACv4 &&
+	    mc->entry_type != ENTRYTYPE_MACv6)
+		ocelot_write_rix(ocelot, pgid->ports, ANA_PGID_PGID,
+				 pgid->index);
+
+	return ocelot_mact_learn(ocelot, pgid->index, addr, vid,
 				 mc->entry_type);
 }
 EXPORT_SYMBOL(ocelot_port_mdb_add);
@@ -1066,6 +1111,7 @@ int ocelot_port_mdb_del(struct ocelot *ocelot, int port,
 	struct ocelot_port *ocelot_port = ocelot->ports[port];
 	unsigned char addr[ETH_ALEN];
 	struct ocelot_multicast *mc;
+	struct ocelot_pgid *pgid;
 	u16 vid = mdb->vid;
 
 	if (port == ocelot->npi)
@@ -1081,6 +1127,7 @@ int ocelot_port_mdb_del(struct ocelot *ocelot, int port,
 	ocelot_encode_ports_to_mdb(addr, mc);
 	ocelot_mact_forget(ocelot, addr, vid);
 
+	ocelot_pgid_free(ocelot, mc->pgid);
 	mc->ports &= ~BIT(port);
 	if (!mc->ports) {
 		list_del(&mc->list);
@@ -1088,9 +1135,20 @@ int ocelot_port_mdb_del(struct ocelot *ocelot, int port,
 		return 0;
 	}
 
+	/* We have a PGID with fewer ports now */
+	pgid = ocelot_mdb_get_pgid(ocelot, mc);
+	if (IS_ERR(pgid))
+		return PTR_ERR(pgid);
+	mc->pgid = pgid;
+
 	ocelot_encode_ports_to_mdb(addr, mc);
 
-	return ocelot_mact_learn(ocelot, mc->pgid, addr, vid,
+	if (mc->entry_type != ENTRYTYPE_MACv4 &&
+	    mc->entry_type != ENTRYTYPE_MACv6)
+		ocelot_write_rix(ocelot, pgid->ports, ANA_PGID_PGID,
+				 pgid->index);
+
+	return ocelot_mact_learn(ocelot, pgid->index, addr, vid,
 				 mc->entry_type);
 }
 EXPORT_SYMBOL(ocelot_port_mdb_del);
@@ -1449,6 +1507,7 @@ int ocelot_init(struct ocelot *ocelot)
 		return -ENOMEM;
 
 	INIT_LIST_HEAD(&ocelot->multicast);
+	INIT_LIST_HEAD(&ocelot->pgids);
 	ocelot_mact_init(ocelot);
 	ocelot_vlan_init(ocelot);
 	ocelot_vcap_init(ocelot);
diff --git a/drivers/net/ethernet/mscc/ocelot.h b/drivers/net/ethernet/mscc/ocelot.h
index 7f8b34c49971..291d39d49c4e 100644
--- a/drivers/net/ethernet/mscc/ocelot.h
+++ b/drivers/net/ethernet/mscc/ocelot.h
@@ -79,13 +79,27 @@ enum macaccess_entry_type {
 	ENTRYTYPE_MACv6,
 };
 
+/* A (PGID) port mask structure, encoding the 2^ocelot->num_phys_ports
+ * possibilities of egress port masks for L2 multicast traffic.
+ * For a switch with 9 user ports, there are 512 possible port masks, but the
+ * hardware only has 46 individual PGIDs that it can forward multicast traffic
+ * to. So we need a structure that maps the limited PGID indices to the port
+ * destinations requested by the user for L2 multicast.
+ */
+struct ocelot_pgid {
+	unsigned long ports;
+	int index;
+	refcount_t refcount;
+	struct list_head list;
+};
+
 struct ocelot_multicast {
 	struct list_head list;
 	enum macaccess_entry_type entry_type;
 	unsigned char addr[ETH_ALEN];
 	u16 vid;
 	u16 ports;
-	int pgid;
+	struct ocelot_pgid *pgid;
 };
 
 int ocelot_port_fdb_do_dump(const unsigned char *addr, u16 vid,
diff --git a/include/soc/mscc/ocelot.h b/include/soc/mscc/ocelot.h
index 1e9db9577441..cc126d1796be 100644
--- a/include/soc/mscc/ocelot.h
+++ b/include/soc/mscc/ocelot.h
@@ -632,6 +632,7 @@ struct ocelot {
 	u32				*lags;
 
 	struct list_head		multicast;
+	struct list_head		pgids;
 
 	struct list_head		dummy_rules;
 	struct ocelot_vcap_block	block[3];
-- 
2.25.1

Re: [PATCH net-next 5/5] net: mscc: ocelot: support L2 multicast entries

[Florian Fainelli]

On 10/28/2020 7:27 PM, Vladimir Oltean wrote:
> There is one main difference in mscc_ocelot between IP multicast and L2
> multicast. With IP multicast, destination ports are encoded into the
> upper bytes of the multicast MAC address. Example: to deliver the
> address 01:00:5E:11:22:33 to ports 3, 8, and 9, one would need to
> program the address of 00:03:08:11:22:33 into hardware. Whereas for L2
> multicast, the MAC table entry points to a Port Group ID (PGID), and
> that PGID contains the port mask that the packet will be forwarded to.
> As to why it is this way, no clue. My guess is that not all port
> combinations can be supported simultaneously with the limited number of
> PGIDs, and this was somehow an issue for IP multicast but not for L2
> multicast. Anyway.
> 
> Prior to this change, the raw L2 multicast code was bogus, due to the
> fact that there wasn't really any way to test it using the bridge code.
> There were 2 issues:
> - A multicast PGID was allocated for each MDB entry, but it wasn't in
>   fact programmed to hardware. It was dummy.
> - In fact we don't want to reserve a multicast PGID for every single MDB
>   entry. That would be odd because we can only have ~60 PGIDs, but
>   thousands of MDB entries. So instead, we want to reserve a multicast
>   PGID for every single port combination for multicast traffic. And
>   since we can have 2 (or more) MDB entries delivered to the same port
>   group (and therefore PGID), we need to reference-count the PGIDs.
> 
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>

Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>

I believe you have the same gfp_t comment applicable here as in patch #4.
-- 
Florian

Re: [PATCH net-next 0/5] L2 multicast forwarding for Ocelot switch

[Jakub Kicinski]

On Thu, 29 Oct 2020 04:27:33 +0200 Vladimir Oltean wrote:
> This series enables the mscc_ocelot switch to forward raw L2 (non-IP)
> mdb entries as configured by the bridge driver after this patch:
> 
> https://patchwork.ozlabs.org/project/netdev/patch/20201028233831.610076-1-vladimir.oltean@nxp.com/

Applied, thanks!