[PATCH v1 net-next 0/1] net: ethernet: ocelot: remove num_stats initializer requirement

[PATCH v1 net-next 0/1] net: ethernet: ocelot: remove num_stats initializer requirement

[Colin Foster]

The ocelot_stats_layout structure array is common with other chips,
specifically the VSC7512. It can only be controlled externally (SPI,
PCIe...)

During the VSC7512 / Felix driver development, it was noticed that
this array can be shared with the Ocelot driver. As with other arrays
shared between the VSC7514 and VSC7512, it makes sense to define them in
drivers/net/ethernet/mscc/vsc7514_regs.c, while declaring them in
include/soc/mscc/vsc7514_regs.h

The thing that makes the stats_layout unique is that it is not accessed
indirectly by way of a shared enumeration index, but instead is looped
over. As such, the num_stats parameter has been used as the loop bounds.

Since the array size isn't necessarily fixed-length, the size has to be
determined. This was done locally in the C file with ARRAY_SIZE, but
that isn't possible if the array is declared via:
extern const struct ocelot_stat_layout ocelot_stats_layout[];

Instead, determine the size by the elemenets of the structure itself.
This way stats can be added / removed as needed (though they rarely
should) without the requirement of dragging a size variable around.

I don't have felix / seville / vsc7514 hardware to test, so I'd
appreciate if someone could give it a test. I've verified functionality
on my in-development kernel, but wouldn't mind feedback from the
existing users who have had the misfortune of having to find my mistakes
when it was too late.

Colin Foster (1):
  net: ethernet: ocelot: remove the need for num_stats initializer

 drivers/net/dsa/ocelot/felix.c             |  1 -
 drivers/net/dsa/ocelot/felix.h             |  1 -
 drivers/net/dsa/ocelot/felix_vsc9959.c     |  2 +-
 drivers/net/dsa/ocelot/seville_vsc9953.c   |  2 +-
 drivers/net/ethernet/mscc/ocelot.c         |  5 +++++
 drivers/net/ethernet/mscc/ocelot_vsc7514.c |  2 +-
 include/soc/mscc/ocelot.h                  | 10 ++++++++++
 7 files changed, 18 insertions(+), 5 deletions(-)

-- 
2.25.1

[PATCH v1 net-next 1/1] net: ethernet: ocelot: remove the need for num_stats initializer

[Colin Foster]

There is a desire to share the oclot_stats_layout struct outside of the
current vsc7514 driver. In order to do so, the length of the array needs to
be known at compile time, and defined in the struct ocelot and struct
felix_info.

Since the array is defined in a .c file and would be declared in the header
file via:
extern struct ocelot_stat_layout[];
the size of the array will not be known at compile time to outside modules.

To fix this, remove the need for defining the number of stats at compile
time and allow this number to be determined at initialization.

Signed-off-by: Colin Foster <colin.foster@in-advantage.com>
---
 drivers/net/dsa/ocelot/felix.c             |  1 -
 drivers/net/dsa/ocelot/felix.h             |  1 -
 drivers/net/dsa/ocelot/felix_vsc9959.c     |  2 +-
 drivers/net/dsa/ocelot/seville_vsc9953.c   |  2 +-
 drivers/net/ethernet/mscc/ocelot.c         |  5 +++++
 drivers/net/ethernet/mscc/ocelot_vsc7514.c |  2 +-
 include/soc/mscc/ocelot.h                  | 10 ++++++++++
 7 files changed, 18 insertions(+), 5 deletions(-)

diff --git a/drivers/net/dsa/ocelot/felix.c b/drivers/net/dsa/ocelot/felix.c
index 9e28219b223d..33cb124ca912 100644
--- a/drivers/net/dsa/ocelot/felix.c
+++ b/drivers/net/dsa/ocelot/felix.c
@@ -1197,7 +1197,6 @@ static int felix_init_structs(struct felix *felix, int num_phys_ports)
 
 	ocelot->map		= felix->info->map;
 	ocelot->stats_layout	= felix->info->stats_layout;
-	ocelot->num_stats	= felix->info->num_stats;
 	ocelot->num_mact_rows	= felix->info->num_mact_rows;
 	ocelot->vcap		= felix->info->vcap;
 	ocelot->vcap_pol.base	= felix->info->vcap_pol_base;
diff --git a/drivers/net/dsa/ocelot/felix.h b/drivers/net/dsa/ocelot/felix.h
index f083b06fdfe9..39faf1027965 100644
--- a/drivers/net/dsa/ocelot/felix.h
+++ b/drivers/net/dsa/ocelot/felix.h
@@ -24,7 +24,6 @@ struct felix_info {
 	const u32			*port_modes;
 	int				num_mact_rows;
 	const struct ocelot_stat_layout	*stats_layout;
-	unsigned int			num_stats;
 	int				num_ports;
 	int				num_tx_queues;
 	struct vcap_props		*vcap;
diff --git a/drivers/net/dsa/ocelot/felix_vsc9959.c b/drivers/net/dsa/ocelot/felix_vsc9959.c
index 52a8566071ed..081871824eaf 100644
--- a/drivers/net/dsa/ocelot/felix_vsc9959.c
+++ b/drivers/net/dsa/ocelot/felix_vsc9959.c
@@ -638,6 +638,7 @@ static const struct ocelot_stat_layout vsc9959_stats_layout[] = {
 	{ .offset = 0x10F,	.name = "drop_green_prio_5", },
 	{ .offset = 0x110,	.name = "drop_green_prio_6", },
 	{ .offset = 0x111,	.name = "drop_green_prio_7", },
+	OCELOT_STAT_END
 };
 
 static const struct vcap_field vsc9959_vcap_es0_keys[] = {
@@ -2216,7 +2217,6 @@ static const struct felix_info felix_info_vsc9959 = {
 	.map			= vsc9959_regmap,
 	.ops			= &vsc9959_ops,
 	.stats_layout		= vsc9959_stats_layout,
-	.num_stats		= ARRAY_SIZE(vsc9959_stats_layout),
 	.vcap			= vsc9959_vcap_props,
 	.vcap_pol_base		= VSC9959_VCAP_POLICER_BASE,
 	.vcap_pol_max		= VSC9959_VCAP_POLICER_MAX,
diff --git a/drivers/net/dsa/ocelot/seville_vsc9953.c b/drivers/net/dsa/ocelot/seville_vsc9953.c
index 68ef8f111bbe..48fd43a93364 100644
--- a/drivers/net/dsa/ocelot/seville_vsc9953.c
+++ b/drivers/net/dsa/ocelot/seville_vsc9953.c
@@ -636,6 +636,7 @@ static const struct ocelot_stat_layout vsc9953_stats_layout[] = {
 	{ .offset = 0x8F,	.name = "drop_green_prio_5", },
 	{ .offset = 0x90,	.name = "drop_green_prio_6", },
 	{ .offset = 0x91,	.name = "drop_green_prio_7", },
+	OCELOT_STAT_END
 };
 
 static const struct vcap_field vsc9953_vcap_es0_keys[] = {
@@ -1086,7 +1087,6 @@ static const struct felix_info seville_info_vsc9953 = {
 	.map			= vsc9953_regmap,
 	.ops			= &vsc9953_ops,
 	.stats_layout		= vsc9953_stats_layout,
-	.num_stats		= ARRAY_SIZE(vsc9953_stats_layout),
 	.vcap			= vsc9953_vcap_props,
 	.vcap_pol_base		= VSC9953_VCAP_POLICER_BASE,
 	.vcap_pol_max		= VSC9953_VCAP_POLICER_MAX,
diff --git a/drivers/net/ethernet/mscc/ocelot.c b/drivers/net/ethernet/mscc/ocelot.c
index ca71b62a44dc..0825a92599a5 100644
--- a/drivers/net/ethernet/mscc/ocelot.c
+++ b/drivers/net/ethernet/mscc/ocelot.c
@@ -3228,6 +3228,7 @@ static void ocelot_detect_features(struct ocelot *ocelot)
 
 int ocelot_init(struct ocelot *ocelot)
 {
+	const struct ocelot_stat_layout *stat;
 	char queue_name[32];
 	int i, ret;
 	u32 port;
@@ -3240,6 +3241,10 @@ int ocelot_init(struct ocelot *ocelot)
 		}
 	}
 
+	ocelot->num_stats = 0;
+	for_each_stat(ocelot, stat)
+		ocelot->num_stats++;
+
 	ocelot->stats = devm_kcalloc(ocelot->dev,
 				     ocelot->num_phys_ports * ocelot->num_stats,
 				     sizeof(u64), GFP_KERNEL);
diff --git a/drivers/net/ethernet/mscc/ocelot_vsc7514.c b/drivers/net/ethernet/mscc/ocelot_vsc7514.c
index 4f4a495a60ad..961f803aca19 100644
--- a/drivers/net/ethernet/mscc/ocelot_vsc7514.c
+++ b/drivers/net/ethernet/mscc/ocelot_vsc7514.c
@@ -190,6 +190,7 @@ static const struct ocelot_stat_layout ocelot_stats_layout[] = {
 	{ .name = "drop_green_prio_5", .offset = 0x8F, },
 	{ .name = "drop_green_prio_6", .offset = 0x90, },
 	{ .name = "drop_green_prio_7", .offset = 0x91, },
+	OCELOT_STAT_END
 };
 
 static void ocelot_pll5_init(struct ocelot *ocelot)
@@ -227,7 +228,6 @@ static int ocelot_chip_init(struct ocelot *ocelot, const struct ocelot_ops *ops)
 
 	ocelot->map = ocelot_regmap;
 	ocelot->stats_layout = ocelot_stats_layout;
-	ocelot->num_stats = ARRAY_SIZE(ocelot_stats_layout);
 	ocelot->num_mact_rows = 1024;
 	ocelot->ops = ops;
 
diff --git a/include/soc/mscc/ocelot.h b/include/soc/mscc/ocelot.h
index 9b4e6c78d0f4..5c4f57cfa785 100644
--- a/include/soc/mscc/ocelot.h
+++ b/include/soc/mscc/ocelot.h
@@ -105,6 +105,13 @@
 #define REG_RESERVED_ADDR		0xffffffff
 #define REG_RESERVED(reg)		REG(reg, REG_RESERVED_ADDR)
 
+#define OCELOT_STAT_FLAG_END		BIT(0)
+
+#define for_each_stat(ocelot, stat)				\
+	for ((stat) = ocelot->stats_layout;			\
+	     !((stat)->flags & OCELOT_STAT_FLAG_END);		\
+	     (stat)++)
+
 enum ocelot_target {
 	ANA = 1,
 	QS,
@@ -535,9 +542,12 @@ enum ocelot_ptp_pins {
 
 struct ocelot_stat_layout {
 	u32 offset;
+	u32 flags;
 	char name[ETH_GSTRING_LEN];
 };
 
+#define OCELOT_STAT_END { .flags = OCELOT_STAT_FLAG_END }
+
 struct ocelot_stats_region {
 	struct list_head node;
 	u32 offset;
-- 
2.25.1

Re: [PATCH v1 net-next 0/1] net: ethernet: ocelot: remove num_stats initializer requirement

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to netdev/net-next.git (master)
by David S. Miller <davem@davemloft.net>:

On Fri, 29 Apr 2022 14:30:35 -0700 you wrote:
> The ocelot_stats_layout structure array is common with other chips,
> specifically the VSC7512. It can only be controlled externally (SPI,
> PCIe...)
> 
> During the VSC7512 / Felix driver development, it was noticed that
> this array can be shared with the Ocelot driver. As with other arrays
> shared between the VSC7514 and VSC7512, it makes sense to define them in
> drivers/net/ethernet/mscc/vsc7514_regs.c, while declaring them in
> include/soc/mscc/vsc7514_regs.h
> 
> [...]

Here is the summary with links:
  - [v1,net-next,1/1] net: ethernet: ocelot: remove the need for num_stats initializer
    https://git.kernel.org/netdev/net-next/c/2f187bfa6f35

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Re: [PATCH v1 net-next 1/1] net: ethernet: ocelot: remove the need for num_stats initializer

[Vladimir Oltean]

On Fri, Apr 29, 2022 at 02:30:36PM -0700, Colin Foster wrote:
> There is a desire to share the oclot_stats_layout struct outside of the
> current vsc7514 driver. In order to do so, the length of the array needs to
> be known at compile time, and defined in the struct ocelot and struct
> felix_info.
> 
> Since the array is defined in a .c file and would be declared in the header
> file via:
> extern struct ocelot_stat_layout[];
> the size of the array will not be known at compile time to outside modules.
> 
> To fix this, remove the need for defining the number of stats at compile
> time and allow this number to be determined at initialization.
> 
> Signed-off-by: Colin Foster <colin.foster@in-advantage.com>
> ---
> diff --git a/include/soc/mscc/ocelot.h b/include/soc/mscc/ocelot.h
> index 9b4e6c78d0f4..5c4f57cfa785 100644
> --- a/include/soc/mscc/ocelot.h
> +++ b/include/soc/mscc/ocelot.h
> @@ -105,6 +105,13 @@
>  #define REG_RESERVED_ADDR		0xffffffff
>  #define REG_RESERVED(reg)		REG(reg, REG_RESERVED_ADDR)
>  
> +#define OCELOT_STAT_FLAG_END		BIT(0)
> +
> +#define for_each_stat(ocelot, stat)				\
> +	for ((stat) = ocelot->stats_layout;			\
> +	     !((stat)->flags & OCELOT_STAT_FLAG_END);		\
> +	     (stat)++)
> +
>  enum ocelot_target {
>  	ANA = 1,
>  	QS,
> @@ -535,9 +542,12 @@ enum ocelot_ptp_pins {
>  
>  struct ocelot_stat_layout {
>  	u32 offset;
> +	u32 flags;

Was it really necessary to add an extra u32 to struct ocelot_stat_layout?
Couldn't you check for the end of stats by looking at stat->name[0] and
comparing against the null terminator, for an empty string?

>  	char name[ETH_GSTRING_LEN];
>  };
>  
> +#define OCELOT_STAT_END { .flags = OCELOT_STAT_FLAG_END }
> +
>  struct ocelot_stats_region {
>  	struct list_head node;
>  	u32 offset;
> -- 
> 2.25.1
>

Re: [PATCH v1 net-next 1/1] net: ethernet: ocelot: remove the need for num_stats initializer

[Colin Foster]

Hi Vladimir,

On Sat, Apr 30, 2022 at 03:15:31PM +0000, Vladimir Oltean wrote:
> On Fri, Apr 29, 2022 at 02:30:36PM -0700, Colin Foster wrote:
> > There is a desire to share the oclot_stats_layout struct outside of the
> > current vsc7514 driver. In order to do so, the length of the array needs to
> > be known at compile time, and defined in the struct ocelot and struct
> > felix_info.
> > 
> > Since the array is defined in a .c file and would be declared in the header
> > file via:
> > extern struct ocelot_stat_layout[];
> > the size of the array will not be known at compile time to outside modules.
> > 
> > To fix this, remove the need for defining the number of stats at compile
> > time and allow this number to be determined at initialization.
> > 
> > Signed-off-by: Colin Foster <colin.foster@in-advantage.com>
> > ---
> > diff --git a/include/soc/mscc/ocelot.h b/include/soc/mscc/ocelot.h
> > index 9b4e6c78d0f4..5c4f57cfa785 100644
> > --- a/include/soc/mscc/ocelot.h
> > +++ b/include/soc/mscc/ocelot.h
> > @@ -105,6 +105,13 @@
> >  #define REG_RESERVED_ADDR		0xffffffff
> >  #define REG_RESERVED(reg)		REG(reg, REG_RESERVED_ADDR)
> >  
> > +#define OCELOT_STAT_FLAG_END		BIT(0)
> > +
> > +#define for_each_stat(ocelot, stat)				\
> > +	for ((stat) = ocelot->stats_layout;			\
> > +	     !((stat)->flags & OCELOT_STAT_FLAG_END);		\
> > +	     (stat)++)
> > +
> >  enum ocelot_target {
> >  	ANA = 1,
> >  	QS,
> > @@ -535,9 +542,12 @@ enum ocelot_ptp_pins {
> >  
> >  struct ocelot_stat_layout {
> >  	u32 offset;
> > +	u32 flags;
> 
> Was it really necessary to add an extra u32 to struct ocelot_stat_layout?
> Couldn't you check for the end of stats by looking at stat->name[0] and
> comparing against the null terminator, for an empty string?

I considered this as well. I could either have explicitly added the
flags field, as I did, or implicitly looked for .name == NULL (or
name[0] == '\0' as you suggest).

I figured it might be better to make this an explicit relationship by
way of flags - but I'm happy to change OCELOT_STAT_END and for_each_stat
to rely on .name if you prefer.

> 
> >  	char name[ETH_GSTRING_LEN];
> >  };
> >  
> > +#define OCELOT_STAT_END { .flags = OCELOT_STAT_FLAG_END }
> > +
> >  struct ocelot_stats_region {
> >  	struct list_head node;
> >  	u32 offset;
> > -- 
> > 2.25.1
> >

Re: [PATCH v1 net-next 1/1] net: ethernet: ocelot: remove the need for num_stats initializer

[Vladimir Oltean]

On Sat, Apr 30, 2022 at 10:47:35AM -0700, Colin Foster wrote:
> > >  struct ocelot_stat_layout {
> > >  	u32 offset;
> > > +	u32 flags;
> > 
> > Was it really necessary to add an extra u32 to struct ocelot_stat_layout?
> > Couldn't you check for the end of stats by looking at stat->name[0] and
> > comparing against the null terminator, for an empty string?
> 
> I considered this as well. I could either have explicitly added the
> flags field, as I did, or implicitly looked for .name == NULL (or
> name[0] == '\0' as you suggest).

No, you cannot check for .name == NULL. The "name" member of struct
ocelot_stat_layout is most definitely not NULL, but has the value of the
memory address of the first char from that array. Contrast this with
"char *name", where a NULL comparison can indeed be made.

> I figured it might be better to make this an explicit relationship by
> way of flags - but I'm happy to change OCELOT_STAT_END and for_each_stat
> to rely on .name if you prefer.

I would have understood introducing a flag to mark the last element of
an array as special (as opposed to introducing a dummy extra element).
But even that calculation would have been wrong.

Before:

pahole -C ocelot_stat_layout drivers/net/ethernet/mscc/ocelot.o
struct ocelot_stat_layout {
        u32                        offset;               /*     0     4 */
        char                       name[32];             /*     4    32 */

        /* size: 36, cachelines: 1, members: 2 */
        /* last cacheline: 36 bytes */
};

After:

pahole -C ocelot_stat_layout drivers/net/ethernet/mscc/ocelot.o
struct ocelot_stat_layout {
        u32                        offset;               /*     0     4 */
        u32                        flags;                /*     4     4 */
        char                       name[32];             /*     8    32 */

        /* size: 40, cachelines: 1, members: 3 */
        /* last cacheline: 40 bytes */
};

For example, vsc9959_stats_layout has 92 elements (93 with the dummy one
you've added now). The overhead of 4 bytes per element amounts to 368
extra bytes. Whereas a single dummy element at the end would have
amounted to just 36 extra bytes.

With your approach, what we get is 372 extra bytes, so worst of both worlds.

> > >  	char name[ETH_GSTRING_LEN];
> > >  };
> > >  
> > > +#define OCELOT_STAT_END { .flags = OCELOT_STAT_FLAG_END }

Re: [PATCH v1 net-next 1/1] net: ethernet: ocelot: remove the need for num_stats initializer

[Colin Foster]

On Sat, Apr 30, 2022 at 09:33:45PM +0000, Vladimir Oltean wrote:
> On Sat, Apr 30, 2022 at 10:47:35AM -0700, Colin Foster wrote:
> > > >  struct ocelot_stat_layout {
> > > >  	u32 offset;
> > > > +	u32 flags;
> > > 
> > > Was it really necessary to add an extra u32 to struct ocelot_stat_layout?
> > > Couldn't you check for the end of stats by looking at stat->name[0] and
> > > comparing against the null terminator, for an empty string?
> > 
> > I considered this as well. I could either have explicitly added the
> > flags field, as I did, or implicitly looked for .name == NULL (or
> > name[0] == '\0' as you suggest).
> 
> No, you cannot check for .name == NULL. The "name" member of struct
> ocelot_stat_layout is most definitely not NULL, but has the value of the
> memory address of the first char from that array. Contrast this with
> "char *name", where a NULL comparison can indeed be made.

My apologies - I had the structure wrong in my head and thought it was a
const char *. Checking for NULL is clearly not an option.

> 
> > I figured it might be better to make this an explicit relationship by
> > way of flags - but I'm happy to change OCELOT_STAT_END and for_each_stat
> > to rely on .name if you prefer.
> 
> I would have understood introducing a flag to mark the last element of
> an array as special (as opposed to introducing a dummy extra element).
> But even that calculation would have been wrong.
> 
> Before:
> 
> pahole -C ocelot_stat_layout drivers/net/ethernet/mscc/ocelot.o
> struct ocelot_stat_layout {
>         u32                        offset;               /*     0     4 */
>         char                       name[32];             /*     4    32 */
> 
>         /* size: 36, cachelines: 1, members: 2 */
>         /* last cacheline: 36 bytes */
> };
> 
> After:
> 
> pahole -C ocelot_stat_layout drivers/net/ethernet/mscc/ocelot.o
> struct ocelot_stat_layout {
>         u32                        offset;               /*     0     4 */
>         u32                        flags;                /*     4     4 */
>         char                       name[32];             /*     8    32 */
> 
>         /* size: 40, cachelines: 1, members: 3 */
>         /* last cacheline: 40 bytes */
> };
> 
> For example, vsc9959_stats_layout has 92 elements (93 with the dummy one
> you've added now). The overhead of 4 bytes per element amounts to 368
> extra bytes. Whereas a single dummy element at the end would have
> amounted to just 36 extra bytes.
> 
> With your approach, what we get is 372 extra bytes, so worst of both worlds.

Understood. I'll send an update momentarily. I also didn't know about
'pahole' which looks to be a useful tool!  Thanks again for the
feedback.

> 
> > > >  	char name[ETH_GSTRING_LEN];
> > > >  };
> > > >  
> > > > +#define OCELOT_STAT_END { .flags = OCELOT_STAT_FLAG_END }