[PATCH net-next 0/5] Support for RollBall 10G copper SFP modules

[PATCH net-next 0/5] Support for RollBall 10G copper SFP modules

[Marek Behún]

Hello,

this series adds support for RollBall/Hilink SFP modules.
These are copper modules capable of up to 10G via copper.
They contain a Marvell 88X3310 PHY.

These modules by default configure the internal PHY into XFI with Rate
Matching mode on the MAC side. To support devices which have MAC capable
of only lower than 10G SerDeses, the fourth patch sets the PHYs MACTYPE
in this case (in the marvell10g driver). Russell King has patches in his
tree that solve similar thing, but they depend on more complicated and
experimental patches. So in the meantime I think this patch can be
accepted (since it should not break anything that already works).

The protocol via which communication with the PHY can be done
is different than the standard one. This series therefore adds
support for this protocol into the mdio-i2c driver:
  - Russell first suggested that the protocol should be selected
    by PHY address: currently all SFP modules use PHY address 22 (0x16)
    because the PHY is accessible via I2C on address 0x56 (=0x40 + 0x16).
  - but Andrew thinks that this could cause problems in the future,
    and that instead the protocol should be selected not via PHY address,
    but on instatination of the mdiobus. This series uses this approach.

Marek

Cc: Andrew Lunn <andrew@lunn.ch>
Cc: Russell King <rmk+kernel@armlinux.org.uk>

Marek Behún (5):
  net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules
  net: phylink: allow attaching phy for SFP modules on 802.3z mode
  net: sfp: configure/destroy I2C mdiobus on transceiver plug/unplug
  net: phy: marvell10g: change MACTYPE if underlying MAC does not
    support it
  net: sfp: add support for multigig RollBall transceivers

 drivers/net/mdio/mdio-i2c.c   | 180 +++++++++++++++++++++++++++++++++-
 drivers/net/phy/marvell10g.c  |  31 ++++++
 drivers/net/phy/phylink.c     |   2 +-
 drivers/net/phy/sfp.c         |  96 ++++++++++++++++--
 include/linux/mdio/mdio-i2c.h |   8 +-
 5 files changed, 300 insertions(+), 17 deletions(-)


base-commit: cd29296fdfca919590e4004a7e4905544f4c4a32
-- 
2.26.2

[PATCH net-next 1/5] net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules

[Marek Behún]

Some multigig SFPs from RollBall and Hilink do not expose functional
MDIO access to the internal PHY of the SFP via I2C address 0x56
(although there seems to be read-only clause 22 access on this address).

Instead these SFPs PHY can be accessed via I2C via the SFP Enhanced
Digital Diagnostic Interface - I2C address 0x51.

This extends the mdio-i2c driver to support this protocol by adding a
special parameter to mdio_i2c_alloc function via which this RollBall
protocol can be selected.

Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: Andrew Lunn <andrew@lunn.ch>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
---
 drivers/net/mdio/mdio-i2c.c   | 180 +++++++++++++++++++++++++++++++++-
 drivers/net/phy/sfp.c         |   2 +-
 include/linux/mdio/mdio-i2c.h |   8 +-
 3 files changed, 183 insertions(+), 7 deletions(-)

diff --git a/drivers/net/mdio/mdio-i2c.c b/drivers/net/mdio/mdio-i2c.c
index 09200a70b315..1f5d653a4e22 100644
--- a/drivers/net/mdio/mdio-i2c.c
+++ b/drivers/net/mdio/mdio-i2c.c
@@ -3,6 +3,7 @@
  * MDIO I2C bridge
  *
  * Copyright (C) 2015-2016 Russell King
+ * Copyright (C) 2020 Marek Behun
  *
  * Network PHYs can appear on I2C buses when they are part of SFP module.
  * This driver exposes these PHYs to the networking PHY code, allowing
@@ -28,7 +29,7 @@ static unsigned int i2c_mii_phy_addr(int phy_id)
 	return phy_id + 0x40;
 }
 
-static int i2c_mii_read(struct mii_bus *bus, int phy_id, int reg)
+static int i2c_mii_read_default(struct mii_bus *bus, int phy_id, int reg)
 {
 	struct i2c_adapter *i2c = bus->priv;
 	struct i2c_msg msgs[2];
@@ -62,7 +63,7 @@ static int i2c_mii_read(struct mii_bus *bus, int phy_id, int reg)
 	return data[0] << 8 | data[1];
 }
 
-static int i2c_mii_write(struct mii_bus *bus, int phy_id, int reg, u16 val)
+static int i2c_mii_write_default(struct mii_bus *bus, int phy_id, int reg, u16 val)
 {
 	struct i2c_adapter *i2c = bus->priv;
 	struct i2c_msg msg;
@@ -91,7 +92,167 @@ static int i2c_mii_write(struct mii_bus *bus, int phy_id, int reg, u16 val)
 	return ret < 0 ? ret : 0;
 }
 
-struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c)
+/* RollBall SFPs do not access internal PHY via I2C address 0x56, but
+ * instead via address 0x51, when SFP page is set to 0x03 and password to
+ * 0xffffffff:
+ *
+ * address  size  contents  description
+ * -------  ----  --------  -----------
+ * 0x80     1     CMD       0x01/0x02/0x04 for write/read/done
+ * 0x81     1     DEV       Clause 45 device
+ * 0x82     2     REG       Clause 45 register
+ * 0x84     2     VAL       Register value
+ */
+#define ROLLBALL_PHY_I2C_ADDR		0x51
+
+#define ROLLBALL_CMD_ADDR		0x80
+#define ROLLBALL_DATA_ADDR		0x81
+
+#define ROLLBALL_CMD_WRITE		0x01
+#define ROLLBALL_CMD_READ		0x02
+#define ROLLBALL_CMD_DONE		0x04
+
+static int i2c_rollball_mii_poll(struct mii_bus *bus, int bus_addr, u8 *buf, size_t len)
+{
+	struct i2c_adapter *i2c = bus->priv;
+	struct i2c_msg msgs[2];
+	u8 buf0[2], *res;
+	int i, ret;
+
+	buf0[0] = ROLLBALL_CMD_ADDR;
+
+	msgs[0].addr = bus_addr;
+	msgs[0].flags = 0;
+	msgs[0].len = 1;
+	msgs[0].buf = &buf0[0];
+
+	res = buf ? buf : &buf0[1];
+
+	msgs[1].addr = bus_addr;
+	msgs[1].flags = I2C_M_RD;
+	msgs[1].len = buf ? len : 1;
+	msgs[1].buf = res;
+
+	/* By experiment it takes up to 70 ms to access a register for these SFPs. Sleep 20ms
+	 * between iteratios and try 10 times.
+	 */
+	i = 10;
+	do {
+		msleep(20);
+
+		ret = i2c_transfer(i2c, msgs, ARRAY_SIZE(msgs));
+		if (ret < 0)
+			return ret;
+		else if (ret != ARRAY_SIZE(msgs))
+			return -EIO;
+
+		if (*res == ROLLBALL_CMD_DONE)
+			return 0;
+	} while (i-- > 0);
+
+	dev_dbg(&bus->dev, "poll timed out\n");
+
+	return -ETIMEDOUT;
+}
+
+static int i2c_rollball_mii_cmd(struct mii_bus *bus, int bus_addr, u8 cmd, u8 *data, size_t len)
+{
+	struct i2c_adapter *i2c = bus->priv;
+	struct i2c_msg msgs[2];
+	u8 cmdbuf[2];
+	int ret;
+
+	msgs[0].addr = bus_addr;
+	msgs[0].flags = 0;
+	msgs[0].len = len;
+	msgs[0].buf = data;
+
+	cmdbuf[0] = ROLLBALL_CMD_ADDR;
+	cmdbuf[1] = cmd;
+
+	msgs[1].addr = bus_addr;
+	msgs[1].flags = 0;
+	msgs[1].len = sizeof(cmdbuf);
+	msgs[1].buf = cmdbuf;
+
+	ret = i2c_transfer(i2c, msgs, 2);
+	if (ret < 0)
+		return ret;
+
+	return ret == ARRAY_SIZE(msgs) ? 0 : -EIO;
+}
+
+static int i2c_mii_read_rollball(struct mii_bus *bus, int phy_id, int reg)
+{
+	u8 buf[4], res[6];
+	int bus_addr, ret;
+	u16 val;
+
+	if (!(reg & MII_ADDR_C45))
+		return -EOPNOTSUPP;
+
+	bus_addr = i2c_mii_phy_addr(phy_id);
+	if (bus_addr != ROLLBALL_PHY_I2C_ADDR)
+		return 0xffff;
+
+	buf[0] = ROLLBALL_DATA_ADDR;
+	buf[1] = (reg >> 16) & 0x1f;
+	buf[2] = (reg >> 8) & 0xff;
+	buf[3] = reg & 0xff;
+
+	ret = i2c_rollball_mii_cmd(bus, bus_addr, ROLLBALL_CMD_READ, buf, sizeof(buf));
+	if (ret < 0)
+		return ret;
+
+	ret = i2c_rollball_mii_poll(bus, bus_addr, res, sizeof(res));
+	if (ret == -ETIMEDOUT)
+		return 0xffff;
+	else if (ret < 0)
+		return ret;
+
+	val = res[4];
+	val <<= 8;
+	val |= res[5];
+
+	dev_dbg(&bus->dev, "read reg %02x:%04x = %04x\n", (reg >> 16) & 0x1f, reg & 0xffff, val);
+
+	return val;
+}
+
+static int i2c_mii_write_rollball(struct mii_bus *bus, int phy_id, int reg, u16 val)
+{
+	int bus_addr, ret;
+	u8 buf[6];
+
+	if (!(reg & MII_ADDR_C45))
+		return -EOPNOTSUPP;
+
+	bus_addr = i2c_mii_phy_addr(phy_id);
+	if (bus_addr != ROLLBALL_PHY_I2C_ADDR)
+		return 0;
+
+	buf[0] = ROLLBALL_DATA_ADDR;
+	buf[1] = (reg >> 16) & 0x1f;
+	buf[2] = (reg >> 8) & 0xff;
+	buf[3] = reg & 0xff;
+	buf[4] = val >> 8;
+	buf[5] = val & 0xff;
+
+	ret = i2c_rollball_mii_cmd(bus, bus_addr, ROLLBALL_CMD_WRITE, buf, sizeof(buf));
+	if (ret < 0)
+		return ret;
+
+	ret = i2c_rollball_mii_poll(bus, bus_addr, NULL, 0);
+	if (ret < 0)
+		return ret;
+
+	dev_dbg(&bus->dev, "write reg %02x:%04x = %04x\n", (reg >> 16) & 0x1f, reg & 0xffff, val);
+
+	return 0;
+}
+
+struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c,
+			       enum mdio_i2c_type type)
 {
 	struct mii_bus *mii;
 
@@ -104,10 +265,19 @@ struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c)
 
 	snprintf(mii->id, MII_BUS_ID_SIZE, "i2c:%s", dev_name(parent));
 	mii->parent = parent;
-	mii->read = i2c_mii_read;
-	mii->write = i2c_mii_write;
 	mii->priv = i2c;
 
+	switch (type) {
+	case MDIO_I2C_ROLLBALL:
+		mii->read = i2c_mii_read_rollball;
+		mii->write = i2c_mii_write_rollball;
+		break;
+	default:
+		mii->read = i2c_mii_read_default;
+		mii->write = i2c_mii_write_default;
+		break;
+	}
+
 	return mii;
 }
 EXPORT_SYMBOL_GPL(mdio_i2c_alloc);
diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c
index 1d18c10e8f82..b1f9fc3a5584 100644
--- a/drivers/net/phy/sfp.c
+++ b/drivers/net/phy/sfp.c
@@ -409,7 +409,7 @@ static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c)
 	sfp->read = sfp_i2c_read;
 	sfp->write = sfp_i2c_write;
 
-	i2c_mii = mdio_i2c_alloc(sfp->dev, i2c);
+	i2c_mii = mdio_i2c_alloc(sfp->dev, i2c, MDIO_I2C_DEFAULT);
 	if (IS_ERR(i2c_mii))
 		return PTR_ERR(i2c_mii);
 
diff --git a/include/linux/mdio/mdio-i2c.h b/include/linux/mdio/mdio-i2c.h
index b1d27f7cd23f..b65a80938806 100644
--- a/include/linux/mdio/mdio-i2c.h
+++ b/include/linux/mdio/mdio-i2c.h
@@ -11,6 +11,12 @@ struct device;
 struct i2c_adapter;
 struct mii_bus;
 
-struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c);
+enum mdio_i2c_type {
+	MDIO_I2C_DEFAULT,
+	MDIO_I2C_ROLLBALL,
+};
+
+struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c,
+			       enum mdio_i2c_type type);
 
 #endif
-- 
2.26.2

Re: [PATCH net-next 1/5] net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules

[Russell King - ARM Linux admin]

On Wed, Oct 28, 2020 at 11:14:23PM +0100, Marek Behún wrote:
> Some multigig SFPs from RollBall and Hilink do not expose functional
> MDIO access to the internal PHY of the SFP via I2C address 0x56
> (although there seems to be read-only clause 22 access on this address).
> 
> Instead these SFPs PHY can be accessed via I2C via the SFP Enhanced
> Digital Diagnostic Interface - I2C address 0x51.
> 
> This extends the mdio-i2c driver to support this protocol by adding a
> special parameter to mdio_i2c_alloc function via which this RollBall
> protocol can be selected.
> 
> Signed-off-by: Marek Behún <kabel@kernel.org>
> Cc: Andrew Lunn <andrew@lunn.ch>
> Cc: Russell King <rmk+kernel@armlinux.org.uk>

I think this is probably a better way forward, and I suspect we're going
to see more of this stuff. I wonder, however, whether the configuration
should be done here too (selecting page 1). Also, shouldn't we ensure
that we are on page 1 before attempting any access?

It would be good to pass this through checkpatch - I notice some lines
seem to be over the 80 character limit now.

A few comments below...

> +static int i2c_rollball_mii_poll(struct mii_bus *bus, int bus_addr, u8 *buf, size_t len)
> +{
> +	struct i2c_adapter *i2c = bus->priv;
> +	struct i2c_msg msgs[2];
> +	u8 buf0[2], *res;
> +	int i, ret;
> +
> +	buf0[0] = ROLLBALL_CMD_ADDR;
> +
> +	msgs[0].addr = bus_addr;
> +	msgs[0].flags = 0;
> +	msgs[0].len = 1;
> +	msgs[0].buf = &buf0[0];
> +
> +	res = buf ? buf : &buf0[1];
> +
> +	msgs[1].addr = bus_addr;
> +	msgs[1].flags = I2C_M_RD;
> +	msgs[1].len = buf ? len : 1;
> +	msgs[1].buf = res;
> +
> +	/* By experiment it takes up to 70 ms to access a register for these SFPs. Sleep 20ms
> +	 * between iteratios and try 10 times.

"iterations"

> +static int i2c_mii_read_rollball(struct mii_bus *bus, int phy_id, int reg)
> +{
> +	u8 buf[4], res[6];
> +	int bus_addr, ret;
> +	u16 val;
> +
> +	if (!(reg & MII_ADDR_C45))
> +		return -EOPNOTSUPP;
> +
> +	bus_addr = i2c_mii_phy_addr(phy_id);
> +	if (bus_addr != ROLLBALL_PHY_I2C_ADDR)
> +		return 0xffff;
> +
> +	buf[0] = ROLLBALL_DATA_ADDR;
> +	buf[1] = (reg >> 16) & 0x1f;
> +	buf[2] = (reg >> 8) & 0xff;
> +	buf[3] = reg & 0xff;
> +
> +	ret = i2c_rollball_mii_cmd(bus, bus_addr, ROLLBALL_CMD_READ, buf, sizeof(buf));
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = i2c_rollball_mii_poll(bus, bus_addr, res, sizeof(res));
> +	if (ret == -ETIMEDOUT)
> +		return 0xffff;
> +	else if (ret < 0)
> +		return ret;
> +
> +	val = res[4];
> +	val <<= 8;
> +	val |= res[5];

Was there something wrong with:

	val = res[4] << 8 | res[5];

here?

> +struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c,
> +			       enum mdio_i2c_type type)

Maybe call this "protocol" rather than "type" ?

> +enum mdio_i2c_type {
> +	MDIO_I2C_DEFAULT,
> +	MDIO_I2C_ROLLBALL,
> +};
> +
> +struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c,
> +			       enum mdio_i2c_type type);

Same here (protocol vs type).

Otherwise, I don't see much else wrong.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

Re: [PATCH net-next 1/5] net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules

[Andrew Lunn]

> It would be good to pass this through checkpatch - I notice some lines
> seem to be over the 80 character limit now.

Hi Russell

The limit got raised to something higher. I personally prefer 80, and
if a file is using 80, new code would be inconsistent with old code if
it did not use 80. So your request is reasonable anyway.

   Andrew

Re: [PATCH net-next 1/5] net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules

[Russell King - ARM Linux admin]

On Thu, Oct 29, 2020 at 01:54:39PM +0100, Andrew Lunn wrote:
> > It would be good to pass this through checkpatch - I notice some lines
> > seem to be over the 80 character limit now.
> 
> Hi Russell
> 
> The limit got raised to something higher. I personally prefer 80, and
> if a file is using 80, new code would be inconsistent with old code if
> it did not use 80. So your request is reasonable anyway.

Hi Andrew,

I do most of my kernel hacking on the Linux console, so 80x25 is the
limit of what I see - and depending on the editor I'm using, lines
longer than 80 characters are chopped to 80 characters unless I scroll
to the right (which then makes moving up and down problematical.) So,
the files I'm responsible for are likely to stay requiring an 80
character width.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

Re: [PATCH net-next 1/5] net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules

[Jakub Kicinski]

On Thu, 29 Oct 2020 13:41:07 +0000 Russell King - ARM Linux admin wrote:
> On Thu, Oct 29, 2020 at 01:54:39PM +0100, Andrew Lunn wrote:
> > > It would be good to pass this through checkpatch - I notice some lines
> > > seem to be over the 80 character limit now.  
> > 
> > Hi Russell
> > 
> > The limit got raised to something higher. I personally prefer 80, and
> > if a file is using 80, new code would be inconsistent with old code if
> > it did not use 80. So your request is reasonable anyway.  
> 
> Hi Andrew,
> 
> I do most of my kernel hacking on the Linux console, so 80x25 is the
> limit of what I see - and depending on the editor I'm using, lines
> longer than 80 characters are chopped to 80 characters unless I scroll
> to the right (which then makes moving up and down problematical.) So,
> the files I'm responsible for are likely to stay requiring an 80
> character width.

+1

Maybe we should patch checkpatch to still enforce 80 for networking.

Re: [PATCH net-next 1/5] net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules

[Marek Behún]

On Thu, 29 Oct 2020 15:53:48 -0700
Jakub Kicinski <kuba@kernel.org> wrote:

> +1
> 
> Maybe we should patch checkpatch to still enforce 80 for networking.

If you do, do that for leds as well, Pavel will be glad.

Re: [PATCH net-next 1/5] net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules

[Marek Behún]

On Thu, 29 Oct 2020 12:41:41 +0000
Russell King - ARM Linux admin <linux@armlinux.org.uk> wrote:

> I think this is probably a better way forward, and I suspect we're going
> to see more of this stuff. I wonder, however, whether the configuration
> should be done here too (selecting page 1). Also, shouldn't we ensure
> that we are on page 1 before attempting any access?

Very well.

> It would be good to pass this through checkpatch - I notice some lines
> seem to be over the 80 character limit now.

Checkpatch does not complain at over 80 characters anymore, but over
100. But I will rewrite it.

> "iterations"

/o\ thx

> > +	val = res[4];
> > +	val <<= 8;
> > +	val |= res[5];  
> 
> Was there something wrong with:
> 
> 	val = res[4] << 8 | res[5];
> 
> here?

For some reason I preferred to use the 3-liner, but I shall rewrite it.

> 
> > +struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c,
> > +			       enum mdio_i2c_type type)  
> 
> Maybe call this "protocol" rather than "type" ?

Very well, thx

[PATCH net-next 2/5] net: phylink: allow attaching phy for SFP modules on 802.3z mode

[Marek Behún]

Some SFPs may contain an internal PHY which may in some cases want to
connect with the host interface in 1000base-x/2500base-x mode.
Do not fail if such PHY is being attached in one of these PHY interface
modes.

Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: Andrew Lunn <andrew@lunn.ch>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
---
 drivers/net/phy/phylink.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/phy/phylink.c b/drivers/net/phy/phylink.c
index 5d8c015bc9f2..52954f12ca5e 100644
--- a/drivers/net/phy/phylink.c
+++ b/drivers/net/phy/phylink.c
@@ -1018,7 +1018,7 @@ static int phylink_attach_phy(struct phylink *pl, struct phy_device *phy,
 {
 	if (WARN_ON(pl->cfg_link_an_mode == MLO_AN_FIXED ||
 		    (pl->cfg_link_an_mode == MLO_AN_INBAND &&
-		     phy_interface_mode_is_8023z(interface))))
+		     phy_interface_mode_is_8023z(interface) && !pl->sfp_bus)))
 		return -EINVAL;
 
 	if (pl->phydev)
-- 
2.26.2

Re: [PATCH net-next 2/5] net: phylink: allow attaching phy for SFP modules on 802.3z mode

[Russell King - ARM Linux admin]

On Wed, Oct 28, 2020 at 11:14:24PM +0100, Marek Behún wrote:
> Some SFPs may contain an internal PHY which may in some cases want to
> connect with the host interface in 1000base-x/2500base-x mode.
> Do not fail if such PHY is being attached in one of these PHY interface
> modes.
> 
> Signed-off-by: Marek Behún <kabel@kernel.org>
> Cc: Andrew Lunn <andrew@lunn.ch>
> Cc: Russell King <rmk+kernel@armlinux.org.uk>
> ---
>  drivers/net/phy/phylink.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/phy/phylink.c b/drivers/net/phy/phylink.c
> index 5d8c015bc9f2..52954f12ca5e 100644
> --- a/drivers/net/phy/phylink.c
> +++ b/drivers/net/phy/phylink.c
> @@ -1018,7 +1018,7 @@ static int phylink_attach_phy(struct phylink *pl, struct phy_device *phy,
>  {
>  	if (WARN_ON(pl->cfg_link_an_mode == MLO_AN_FIXED ||
>  		    (pl->cfg_link_an_mode == MLO_AN_INBAND &&
> -		     phy_interface_mode_is_8023z(interface))))
> +		     phy_interface_mode_is_8023z(interface) && !pl->sfp_bus)))
>  		return -EINVAL;
>  
>  	if (pl->phydev)

I think also changing phylink_sfp_config() too since that check is no
longer relevent - although it doesn't actually end up being effective
today. So, might as well be removed along with the above change.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

[PATCH net-next 3/5] net: sfp: configure/destroy I2C mdiobus on transceiver plug/unplug

[Marek Behún]

Instead of configuring the I2C mdiobus when SFP driver is probed,
configure/destroy the mdiobus when SFP transceiver is plugged/unplugged.

This way we can tell the mdio-i2c code which protocol to use for each
SFP transceiver.

Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: Andrew Lunn <andrew@lunn.ch>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
---
 drivers/net/phy/sfp.c | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)

diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c
index b1f9fc3a5584..a392d5fc6ab4 100644
--- a/drivers/net/phy/sfp.c
+++ b/drivers/net/phy/sfp.c
@@ -399,9 +399,6 @@ static int sfp_i2c_write(struct sfp *sfp, bool a2, u8 dev_addr, void *buf,
 
 static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c)
 {
-	struct mii_bus *i2c_mii;
-	int ret;
-
 	if (!i2c_check_functionality(i2c, I2C_FUNC_I2C))
 		return -EINVAL;
 
@@ -409,7 +406,15 @@ static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c)
 	sfp->read = sfp_i2c_read;
 	sfp->write = sfp_i2c_write;
 
-	i2c_mii = mdio_i2c_alloc(sfp->dev, i2c, MDIO_I2C_DEFAULT);
+	return 0;
+}
+
+static int sfp_i2c_mdiobus_configure(struct sfp *sfp, enum mdio_i2c_type type)
+{
+	struct mii_bus *i2c_mii;
+	int ret;
+
+	i2c_mii = mdio_i2c_alloc(sfp->dev, sfp->i2c, type);
 	if (IS_ERR(i2c_mii))
 		return PTR_ERR(i2c_mii);
 
@@ -427,6 +432,12 @@ static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c)
 	return 0;
 }
 
+static void sfp_i2c_mdiobus_destroy(struct sfp *sfp)
+{
+	mdiobus_unregister(sfp->i2c_mii);
+	sfp->i2c_mii = NULL;
+}
+
 /* Interface */
 static int sfp_read(struct sfp *sfp, bool a2, u8 addr, void *buf, size_t len)
 {
@@ -1768,6 +1779,11 @@ static int sfp_sm_mod_probe(struct sfp *sfp, bool report)
 	else
 		sfp->module_t_start_up = T_START_UP;
 
+	/* Configure mdiobus */
+	ret = sfp_i2c_mdiobus_configure(sfp, MDIO_I2C_DEFAULT);
+	if (ret < 0)
+		return ret;
+
 	return 0;
 }
 
@@ -1778,6 +1794,8 @@ static void sfp_sm_mod_remove(struct sfp *sfp)
 
 	sfp_hwmon_remove(sfp);
 
+	sfp_i2c_mdiobus_destroy(sfp);
+
 	memset(&sfp->id, 0, sizeof(sfp->id));
 	sfp->module_power_mW = 0;
 
-- 
2.26.2

Re: [PATCH net-next 3/5] net: sfp: configure/destroy I2C mdiobus on transceiver plug/unplug

[Russell King - ARM Linux admin]

On Wed, Oct 28, 2020 at 11:14:25PM +0100, Marek Behún wrote:
> Instead of configuring the I2C mdiobus when SFP driver is probed,
> configure/destroy the mdiobus when SFP transceiver is plugged/unplugged.
> 
> This way we can tell the mdio-i2c code which protocol to use for each
> SFP transceiver.
> 
> Signed-off-by: Marek Behún <kabel@kernel.org>
> Cc: Andrew Lunn <andrew@lunn.ch>
> Cc: Russell King <rmk+kernel@armlinux.org.uk>
> ---
>  drivers/net/phy/sfp.c | 26 ++++++++++++++++++++++----
>  1 file changed, 22 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c
> index b1f9fc3a5584..a392d5fc6ab4 100644
> --- a/drivers/net/phy/sfp.c
> +++ b/drivers/net/phy/sfp.c
> @@ -399,9 +399,6 @@ static int sfp_i2c_write(struct sfp *sfp, bool a2, u8 dev_addr, void *buf,
>  
>  static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c)
>  {
> -	struct mii_bus *i2c_mii;
> -	int ret;
> -
>  	if (!i2c_check_functionality(i2c, I2C_FUNC_I2C))
>  		return -EINVAL;
>  
> @@ -409,7 +406,15 @@ static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c)
>  	sfp->read = sfp_i2c_read;
>  	sfp->write = sfp_i2c_write;
>  
> -	i2c_mii = mdio_i2c_alloc(sfp->dev, i2c, MDIO_I2C_DEFAULT);
> +	return 0;
> +}
> +
> +static int sfp_i2c_mdiobus_configure(struct sfp *sfp, enum mdio_i2c_type type)
> +{
> +	struct mii_bus *i2c_mii;
> +	int ret;
> +
> +	i2c_mii = mdio_i2c_alloc(sfp->dev, sfp->i2c, type);
>  	if (IS_ERR(i2c_mii))
>  		return PTR_ERR(i2c_mii);
>  
> @@ -427,6 +432,12 @@ static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c)
>  	return 0;
>  }
>  
> +static void sfp_i2c_mdiobus_destroy(struct sfp *sfp)
> +{
> +	mdiobus_unregister(sfp->i2c_mii);
> +	sfp->i2c_mii = NULL;
> +}
> +
>  /* Interface */
>  static int sfp_read(struct sfp *sfp, bool a2, u8 addr, void *buf, size_t len)
>  {
> @@ -1768,6 +1779,11 @@ static int sfp_sm_mod_probe(struct sfp *sfp, bool report)
>  	else
>  		sfp->module_t_start_up = T_START_UP;
>  
> +	/* Configure mdiobus */
> +	ret = sfp_i2c_mdiobus_configure(sfp, MDIO_I2C_DEFAULT);
> +	if (ret < 0)
> +		return ret;
> +

	return sfp_i2c_mdiobus_configure(sfp, MDIO_I2C_DEFAULT);

would be a simpler way to write this. However, I suggest handling this
elsewhere due to the point below.

> @@ -1778,6 +1794,8 @@ static void sfp_sm_mod_remove(struct sfp *sfp)
>  
>  	sfp_hwmon_remove(sfp);
>  
> +	sfp_i2c_mdiobus_destroy(sfp);
> +

This doesn't seem like the right place to handle this. This is called
from the module state machine (sfp_sm_module()) which is run before the
main state machine (sfp_sm_main()). The PHY is unregistered in the main
state machine, which means you're destroying the MII bus before the
PHY.

I guess what saves us from crashing is the refcounting, but this
doesn't seem to me to be particularly good.

Maybe create the MII bus after the "init_done" label, and destroy it
after we've called sfp_sm_phy_detach() ?

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

[PATCH net-next 4/5] net: phy: marvell10g: change MACTYPE if underlying MAC does not support it

[Marek Behún]

RollBall SFPs contain a Marvell 88X3310 PHY, but by default the MACTYPE
is set to 10GBASE-R with Rate Matching.

Some devices (for example those based on Armada 38x) only support up to
2500base-x SerDes modes.

Change the PHY's MACTYPE to 4 (which means changing between 10gbase-r,
5gbase-r, 2500base-x ans SGMII depending on copper speed) if this is the
case (which is infered from phydev->interface).

Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: Andrew Lunn <andrew@lunn.ch>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
---
 drivers/net/phy/marvell10g.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/drivers/net/phy/marvell10g.c b/drivers/net/phy/marvell10g.c
index 1901ba277413..9e8e9aa66972 100644
--- a/drivers/net/phy/marvell10g.c
+++ b/drivers/net/phy/marvell10g.c
@@ -453,6 +453,33 @@ static bool mv3310_has_pma_ngbaset_quirk(struct phy_device *phydev)
 		MV_PHY_ALASKA_NBT_QUIRK_MASK) == MV_PHY_ALASKA_NBT_QUIRK_REV;
 }
 
+static int mv3310_select_mactype(struct phy_device *phydev)
+{
+	int mac_type, ret;
+
+	/* On some devices the MAC does not support 10G mode, but may support
+	 * lower modes, such as SGMII or 2500base-x.
+	 * By changing MACTYPE of the PHY to 4 in this case, we ensure that
+	 * the MAC will link with the PHY at least for these lower speeds.
+	 */
+	switch (phydev->interface) {
+	case PHY_INTERFACE_MODE_SGMII:
+	case PHY_INTERFACE_MODE_2500BASEX:
+		mac_type = 4;
+		break;
+	default:
+		return 0;
+	}
+
+	ret = phy_modify_mmd_changed(phydev, MDIO_MMD_VEND2, MV_V2_PORT_CTRL,
+				     MV_V2_PORT_MAC_TYPE_MASK, mac_type);
+	if (ret <= 0)
+		return ret;
+
+	return phy_modify_mmd(phydev, MDIO_MMD_VEND2, MV_V2_PORT_CTRL,
+			      MV_V2_PORT_CTRL_SWRST, MV_V2_PORT_CTRL_SWRST);
+}
+
 static int mv3310_config_init(struct phy_device *phydev)
 {
 	struct mv3310_priv *priv = dev_get_drvdata(&phydev->mdio.dev);
@@ -474,6 +501,10 @@ static int mv3310_config_init(struct phy_device *phydev)
 	if (err)
 		return err;
 
+	err = mv3310_select_mactype(phydev);
+	if (err)
+		return err;
+
 	val = phy_read_mmd(phydev, MDIO_MMD_VEND2, MV_V2_PORT_CTRL);
 	if (val < 0)
 		return val;
-- 
2.26.2

Re: [PATCH net-next 4/5] net: phy: marvell10g: change MACTYPE if underlying MAC does not support it

[Russell King - ARM Linux admin]

On Wed, Oct 28, 2020 at 11:14:26PM +0100, Marek Behún wrote:
> RollBall SFPs contain a Marvell 88X3310 PHY, but by default the MACTYPE
> is set to 10GBASE-R with Rate Matching.
> 
> Some devices (for example those based on Armada 38x) only support up to
> 2500base-x SerDes modes.
> 
> Change the PHY's MACTYPE to 4 (which means changing between 10gbase-r,
> 5gbase-r, 2500base-x ans SGMII depending on copper speed) if this is the
> case (which is infered from phydev->interface).
> 
> Signed-off-by: Marek Behún <kabel@kernel.org>
> Cc: Andrew Lunn <andrew@lunn.ch>
> Cc: Russell King <rmk+kernel@armlinux.org.uk>

This'll do as a stop-gap until we have a better way to determine which
MACTYPE mode we should be using.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

[PATCH net-next 5/5] net: sfp: add support for multigig RollBall transceivers

[Marek Behún]

This adds support for multigig copper SFP modules from RollBall/Hilink.
These modules have a specific way to access clause 45 registers of the
internal PHY.

We also need to wait at least 25 seconds after deasserting TX disable
before accessing the PHY. The code waits for 30 seconds just to be sure.

Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: Andrew Lunn <andrew@lunn.ch>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
---
 drivers/net/phy/sfp.c | 72 ++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 65 insertions(+), 7 deletions(-)

diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c
index a392d5fc6ab4..379358f194ee 100644
--- a/drivers/net/phy/sfp.c
+++ b/drivers/net/phy/sfp.c
@@ -165,6 +165,7 @@ static const enum gpiod_flags gpio_flags[] = {
  * on board (for a copper SFP) time to initialise.
  */
 #define T_WAIT			msecs_to_jiffies(50)
+#define T_WAIT_LONG_PHY		msecs_to_jiffies(30000)
 #define T_START_UP		msecs_to_jiffies(300)
 #define T_START_UP_BAD_GPON	msecs_to_jiffies(60000)
 
@@ -204,8 +205,11 @@ static const enum gpiod_flags gpio_flags[] = {
 
 /* SFP modules appear to always have their PHY configured for bus address
  * 0x56 (which with mdio-i2c, translates to a PHY address of 22).
+ * RollBall SFPs access phy via SFP Enhanced Digital Diagnostic Interface
+ * via address 0x51 (mdio-i2c will use RollBall protocol on this address).
  */
-#define SFP_PHY_ADDR	22
+#define SFP_PHY_ADDR		22
+#define SFP_PHY_ADDR_ROLLBALL	17
 
 struct sff_data {
 	unsigned int gpios;
@@ -220,6 +224,7 @@ struct sfp {
 	struct phy_device *mod_phy;
 	const struct sff_data *type;
 	u32 max_power_mW;
+	int phy_addr;
 
 	unsigned int (*get_state)(struct sfp *);
 	void (*set_state)(struct sfp *, unsigned int);
@@ -248,6 +253,7 @@ struct sfp {
 	struct sfp_eeprom_id id;
 	unsigned int module_power_mW;
 	unsigned int module_t_start_up;
+	unsigned int module_t_wait;
 
 #if IS_ENABLED(CONFIG_HWMON)
 	struct sfp_diag diag;
@@ -1442,7 +1448,7 @@ static int sfp_sm_probe_phy(struct sfp *sfp, bool is_c45)
 	struct phy_device *phy;
 	int err;
 
-	phy = get_phy_device(sfp->i2c_mii, SFP_PHY_ADDR, is_c45);
+	phy = get_phy_device(sfp->i2c_mii, sfp->phy_addr, is_c45);
 	if (phy == ERR_PTR(-ENODEV))
 		return PTR_ERR(phy);
 	if (IS_ERR(phy)) {
@@ -1675,12 +1681,40 @@ static int sfp_cotsworks_fixup_check(struct sfp *sfp, struct sfp_eeprom_id *id)
 	return 0;
 }
 
+static int sfp_rollball_init_mdio(struct sfp *sfp)
+{
+	u8 page, password[4];
+	int err;
+
+	page = 3;
+
+	err = sfp_write(sfp, true, SFP_PAGE, &page, 1);
+	if (err != 1) {
+		dev_err(sfp->dev, "Failed to set SFP page for RollBall MDIO access: %d\n", err);
+		return err;
+	}
+
+	password[0] = 0xff;
+	password[1] = 0xff;
+	password[2] = 0xff;
+	password[3] = 0xff;
+
+	err = sfp_write(sfp, true, 0x7b, password, 4);
+	if (err != 4) {
+		dev_err(sfp->dev, "Failed to write password for RollBall MDIO access: %d\n", err);
+		return err;
+	}
+
+	return 0;
+}
+
 static int sfp_sm_mod_probe(struct sfp *sfp, bool report)
 {
 	/* SFP module inserted - read I2C data */
 	struct sfp_eeprom_id id;
 	bool cotsworks_sfbg;
 	bool cotsworks;
+	bool rollball;
 	u8 check;
 	int ret;
 
@@ -1755,6 +1789,24 @@ static int sfp_sm_mod_probe(struct sfp *sfp, bool report)
 		 (int)sizeof(id.ext.vendor_sn), id.ext.vendor_sn,
 		 (int)sizeof(id.ext.datecode), id.ext.datecode);
 
+	sfp->phy_addr = SFP_PHY_ADDR;
+
+	rollball = ((!memcmp(id.base.vendor_name, "OEM             ", 16) ||
+		     !memcmp(id.base.vendor_name, "Turris          ", 16)) &&
+		    (!memcmp(id.base.vendor_pn, "SFP-10G-T       ", 16) ||
+		     !memcmp(id.base.vendor_pn, "RTSFP-10", 8)));
+	if (rollball) {
+		sfp->phy_addr = SFP_PHY_ADDR_ROLLBALL;
+		ret = sfp_rollball_init_mdio(sfp);
+		if (ret < 0)
+			return ret;
+
+		/* RollBall SFPs may have wrong (zero) extended compliacne code burned in EEPROM.
+		 * For PHY probing we need the correct one.
+		 */
+		id.base.extended_cc = SFF8024_ECC_10GBASE_T_SFI;
+	}
+
 	/* Check whether we support this module */
 	if (!sfp->type->module_supported(&id)) {
 		dev_err(sfp->dev,
@@ -1779,8 +1831,13 @@ static int sfp_sm_mod_probe(struct sfp *sfp, bool report)
 	else
 		sfp->module_t_start_up = T_START_UP;
 
+	if (rollball)
+		sfp->module_t_wait = T_WAIT_LONG_PHY;
+	else
+		sfp->module_t_wait = T_WAIT;
+
 	/* Configure mdiobus */
-	ret = sfp_i2c_mdiobus_configure(sfp, MDIO_I2C_DEFAULT);
+	ret = sfp_i2c_mdiobus_configure(sfp, rollball ? MDIO_I2C_ROLLBALL : MDIO_I2C_DEFAULT);
 	if (ret < 0)
 		return ret;
 
@@ -1979,9 +2036,10 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event)
 
 		/* We need to check the TX_FAULT state, which is not defined
 		 * while TX_DISABLE is asserted. The earliest we want to do
-		 * anything (such as probe for a PHY) is 50ms.
+		 * anything (such as probe for a PHY) is 50ms (or more on
+		 * specific modules).
 		 */
-		sfp_sm_next(sfp, SFP_S_WAIT, T_WAIT);
+		sfp_sm_next(sfp, SFP_S_WAIT, sfp->module_t_wait);
 		break;
 
 	case SFP_S_WAIT:
@@ -1995,8 +2053,8 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event)
 			 * deasserting.
 			 */
 			timeout = sfp->module_t_start_up;
-			if (timeout > T_WAIT)
-				timeout -= T_WAIT;
+			if (timeout > sfp->module_t_wait)
+				timeout -= sfp->module_t_wait;
 			else
 				timeout = 1;
 
-- 
2.26.2

Re: [PATCH net-next 5/5] net: sfp: add support for multigig RollBall transceivers

[Russell King - ARM Linux admin]

On Wed, Oct 28, 2020 at 11:14:27PM +0100, Marek Behún wrote:
> This adds support for multigig copper SFP modules from RollBall/Hilink.
> These modules have a specific way to access clause 45 registers of the
> internal PHY.
> 
> We also need to wait at least 25 seconds after deasserting TX disable
> before accessing the PHY. The code waits for 30 seconds just to be sure.

Any ideas why it takes 25 seconds for the module to initialise - the
88x3310 startup is pretty fast in itself. However, it never amazes me
how broken SFP modules can be.

Extending T_WAIT is one way around this, and luckily I already catered
for the case where T_WAIT is extended beyond module_t_start_up.

Usual comment about line lengths...

> 
> Signed-off-by: Marek Behún <kabel@kernel.org>
> Cc: Andrew Lunn <andrew@lunn.ch>
> Cc: Russell King <rmk+kernel@armlinux.org.uk>
> ---
>  drivers/net/phy/sfp.c | 72 ++++++++++++++++++++++++++++++++++++++-----
>  1 file changed, 65 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c
> index a392d5fc6ab4..379358f194ee 100644
> --- a/drivers/net/phy/sfp.c
> +++ b/drivers/net/phy/sfp.c
> @@ -165,6 +165,7 @@ static const enum gpiod_flags gpio_flags[] = {
>   * on board (for a copper SFP) time to initialise.
>   */
>  #define T_WAIT			msecs_to_jiffies(50)
> +#define T_WAIT_LONG_PHY		msecs_to_jiffies(30000)

I think call this T_WAIT_ROLLBALL.

> @@ -1675,12 +1681,40 @@ static int sfp_cotsworks_fixup_check(struct sfp *sfp, struct sfp_eeprom_id *id)
>  	return 0;
>  }
>  
> +static int sfp_rollball_init_mdio(struct sfp *sfp)
> +{
> +	u8 page, password[4];
> +	int err;
> +
> +	page = 3;
> +
> +	err = sfp_write(sfp, true, SFP_PAGE, &page, 1);
> +	if (err != 1) {
> +		dev_err(sfp->dev, "Failed to set SFP page for RollBall MDIO access: %d\n", err);
> +		return err;
> +	}
> +
> +	password[0] = 0xff;
> +	password[1] = 0xff;
> +	password[2] = 0xff;
> +	password[3] = 0xff;
> +
> +	err = sfp_write(sfp, true, 0x7b, password, 4);
> +	if (err != 4) {
> +		dev_err(sfp->dev, "Failed to write password for RollBall MDIO access: %d\n", err);
> +		return err;
> +	}
> +
> +	return 0;
> +}

I think this needs to be done in the MDIO driver - if we have userspace
or otherwise expand what we're doing, relying on page 3 remaining
selected will be very fragile.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

Re: [PATCH net-next 5/5] net: sfp: add support for multigig RollBall transceivers

[Marek Behún]

On Thu, 29 Oct 2020 13:38:00 +0000
Russell King - ARM Linux admin <linux@armlinux.org.uk> wrote:

> Any ideas why it takes 25 seconds for the module to initialise - the
> 88x3310 startup is pretty fast in itself. However, it never amazes me
> how broken SFP modules can be.

I don't know, but for the first 25 seconds the SFP returns 0xffff on
all reads. I think it is due to the internal MCU doing it's own stuff
with the PHY during that time.

Marek