[PATCH net-next v5 0/5] Support for RollBall 10G copper SFP modules

[PATCH net-next v5 0/5] Support for RollBall 10G copper SFP modules

[Marek Behún]

Hello,

this is v5 of series adding support for RollBall/Hilink SFP modules.

Checked with:
  checkpatch.pl --max-line-length=80

Changes from v4:
- added patch adding SFP_PASSWORD = 0x7b to sfp.h
- patch adding RollBall MDIO I2C protocol support
  - removed forgotten comment that was no longer true
  - we now try to restore the page if the I2C transfer failed
  - unfortunately we cannot simply move the code to do paged I2C
    transfers to sfp.c, because sfp.c depends on code inmdio-i2c.c,
    and if we added dependency the other way, I don't know what
    would happen to module loader (since both SFP and MDIO_I2C can
    be compiled as modules).
    This can be refactored later, when someone needs paged I2C
    accesses in sfp.c
- added Reviewed-by tags

Changes from v3:
- RollBall mdio-i2c driver now sets/restores SFP_PAGE for every MDIO
  access.
  I first wanted to achieve this operation (setting
  SFP_PAGE/doing MDIO/restoring SFP_PAGE) via one call do i2c_transfer,
  by constructing msgs array in such a way, but it turned out that this
  doesn't work on RollBall SFPs, because changed SFP_PAGE takes into
  account only after i2c_transfer ends.
  So instead I use i2c_lock_bus/serveral __i2c_transfers/i2c_unlock_bus.
- I have removed the patch which changes MACTYPE in the marvell10g
  driver, since Russell has in his net-queue a better solution. I still
  think that my patch would have sufficed temporarily (and would not
  cause regressions), but nobody wanted to review it. If you think that
  I should sent this series again with that patch, please let me know.

Changes from v2:
- added comment into the patch adding support for RollBall I2C MDIO
  protocol, saying that we expect the SFP_PAGE not to be changed by
  the SFP code, as requested by Russell. If, in the future, SFP code
  starts modifying SFP_PAGE, we will have to handle it in mdio-i2c
  somehow
- destruction of I2C MDIO bus in patch 3/5 now depends on whether the
  MDIO bus is not NULL, instead of whether PHY exists, as suggested by
  Russell
- changed waiting time for RollBall module to initialize from 30 seconds
  to 25 seconds. Testing shows that it is never longer than 21-22
  seconds, so waiting 25 seconds instead of 30 is IMO safe enough
- added Russell's Reviewed-by tags where relevant

Changes from v1:
- wrapped to 80 columns as per Russell's request
- initialization of RollBall MDIO I2C protocol moved from sfp.c to
  mdio-i2c.c as per Russell's request
- second patch removes the 802.3z check also from phylink_sfp_config
  as suggested by Russell
- creation/destruction of mdiobus for SFP now occurs before probing
  for PHY/after releasing PHY (as suggested by Russell)
- the last patch became a little simpler after the above was done

Marek

Marek Behún (5):
  net: sfp: add SFP_PASSWORD address
  net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules
  net: phylink: allow attaching phy for SFP modules on 802.3z mode
  net: sfp: create/destroy I2C mdiobus before PHY probe/after PHY
    release
  net: sfp: add support for multigig RollBall transceivers

 drivers/net/mdio/mdio-i2c.c   | 308 +++++++++++++++++++++++++++++++++-
 drivers/net/phy/phylink.c     |   5 +-
 drivers/net/phy/sfp.c         |  66 ++++++--
 include/linux/mdio/mdio-i2c.h |   8 +-
 include/linux/sfp.h           |   1 +
 5 files changed, 368 insertions(+), 20 deletions(-)


base-commit: 0ae5b43d6dde6003070106e97cd0d41bace2eeb2
-- 
2.26.2

[PATCH net-next v5 1/5] net: sfp: add SFP_PASSWORD address

[Marek Behún]

Add SFP_PASSWORD = 0x7b to the diagnostics enumerator. This address is
described in SFF-8436 and SFF-8636.

Signed-off-by: Marek Behún <kabel@kernel.org>
---
 include/linux/sfp.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/linux/sfp.h b/include/linux/sfp.h
index 38893e4dd0f0..81b40b2d6b1b 100644
--- a/include/linux/sfp.h
+++ b/include/linux/sfp.h
@@ -490,6 +490,7 @@ enum {
 
 	SFP_EXT_STATUS			= 0x76,
 	SFP_VSL				= 0x78,
+	SFP_PASSWORD			= 0x7b,
 	SFP_PAGE			= 0x7f,
 };
 
-- 
2.26.2

Re: [PATCH net-next v5 1/5] net: sfp: add SFP_PASSWORD address

[Russell King - ARM Linux admin]

On Thu, Jan 14, 2021 at 05:43:27AM +0100, Marek Behún wrote:
> Add SFP_PASSWORD = 0x7b to the diagnostics enumerator. This address is
> described in SFF-8436 and SFF-8636.

However, as I mentioned previously, these are not relevant for SFPs,
but are for QSFPs, and QSFPs have a totally different layout.

The data structure for this enum is described by SFF-8472, and it lists

	Vendor Specific Locations [Address A2h, Bytes 120-126]

This is what the SFP_VSL definition covers.

SFF-8436 defines the layout for "QSFP+ 10 Gbs 4X PLUGGABLE TRANSCEIVER":

                            Table 17 - Lower Memory Map (A0h)
       Address       Description         Type         Passive Copper,     Optical
                                                      Active Copper,      Module                                                      Active Optical
       0             Identifier (1       Read-Only    R                   R
                     Byte)
       1-2           Status (2 Bytes)     Read-Only    See Table 18
       3-21          Interrupt Flags      Read-Only    See Tables 19-21
                     (19 Bytes)
...
       123-126       Password Entry       Read/Write   O                  O
                     Area (optional) 4
                     Bytes
       127           Page Select Byte     Read/Write   R                  R

So, SFF-8436 is not relevant - not only is it for a different address
but the whole format is different.

SFF-8636 is "Specification for Management Interface for Cabled
Environments" and defines:

                             2-Wire Serial Address 1010000x
                                     Lower Page 00h
                            0 Identifier
                        1- 2 Status
                        3- 21 Interrupt Flags
...
                      123-126 Password Entry Area (Optional)
                      127     Page Select Byte

So again, SFF-8636 is not relevant for the same reasons as SFF-8436.

We're left with SFF-8472, which mentions no password entry area, but
leaves the area from 120-127 open as "vendor specific", so I don't
think it is appropriate to start adding vendor specific definitions as
if they were generic to sfp.h

I would instead suggest defining this inside mdio-i2c.c as

#define ROLLERBALL_PASSWORD	(SFP_VSL + 3)

rather than trying to make this appear generic.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

[PATCH net-next v5 2/5] net: phy: mdio-i2c: support I2C MDIO protocol for RollBall SFP modules

[Marek Behún]

Some multigig SFPs from RollBall and Hilink do not expose functional
MDIO access to the internal PHY of the SFP via I2C address 0x56
(although there seems to be read-only clause 22 access on this address).

Instead these SFPs PHY can be accessed via I2C via the SFP Enhanced
Digital Diagnostic Interface - I2C address 0x51. The SFP_PAGE has to be
selected to 3 and the password must be filled with 0xff bytes for this
PHY communication to work.

This extends the mdio-i2c driver to support this protocol by adding a
special parameter to mdio_i2c_alloc function via which this RollBall
protocol can be selected.

Signed-off-by: Marek Behún <kabel@kernel.org>
Cc: Andrew Lunn <andrew@lunn.ch>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
---
 drivers/net/mdio/mdio-i2c.c   | 308 +++++++++++++++++++++++++++++++++-
 drivers/net/phy/sfp.c         |   2 +-
 include/linux/mdio/mdio-i2c.h |   8 +-
 3 files changed, 311 insertions(+), 7 deletions(-)

diff --git a/drivers/net/mdio/mdio-i2c.c b/drivers/net/mdio/mdio-i2c.c
index 09200a70b315..991a449b3163 100644
--- a/drivers/net/mdio/mdio-i2c.c
+++ b/drivers/net/mdio/mdio-i2c.c
@@ -3,6 +3,7 @@
  * MDIO I2C bridge
  *
  * Copyright (C) 2015-2016 Russell King
+ * Copyright (C) 2021 Marek Behun
  *
  * Network PHYs can appear on I2C buses when they are part of SFP module.
  * This driver exposes these PHYs to the networking PHY code, allowing
@@ -12,6 +13,7 @@
 #include <linux/i2c.h>
 #include <linux/mdio/mdio-i2c.h>
 #include <linux/phy.h>
+#include <linux/sfp.h>
 
 /*
  * I2C bus addresses 0x50 and 0x51 are normally an EEPROM, which is
@@ -28,7 +30,7 @@ static unsigned int i2c_mii_phy_addr(int phy_id)
 	return phy_id + 0x40;
 }
 
-static int i2c_mii_read(struct mii_bus *bus, int phy_id, int reg)
+static int i2c_mii_read_default(struct mii_bus *bus, int phy_id, int reg)
 {
 	struct i2c_adapter *i2c = bus->priv;
 	struct i2c_msg msgs[2];
@@ -62,7 +64,8 @@ static int i2c_mii_read(struct mii_bus *bus, int phy_id, int reg)
 	return data[0] << 8 | data[1];
 }
 
-static int i2c_mii_write(struct mii_bus *bus, int phy_id, int reg, u16 val)
+static int i2c_mii_write_default(struct mii_bus *bus, int phy_id, int reg,
+				 u16 val)
 {
 	struct i2c_adapter *i2c = bus->priv;
 	struct i2c_msg msg;
@@ -91,9 +94,286 @@ static int i2c_mii_write(struct mii_bus *bus, int phy_id, int reg, u16 val)
 	return ret < 0 ? ret : 0;
 }
 
-struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c)
+/* RollBall SFPs do not access internal PHY via I2C address 0x56, but
+ * instead via address 0x51, when SFP page is set to 0x03 and password to
+ * 0xffffffff.
+ *
+ * address  size  contents  description
+ * -------  ----  --------  -----------
+ * 0x80     1     CMD       0x01/0x02/0x04 for write/read/done
+ * 0x81     1     DEV       Clause 45 device
+ * 0x82     2     REG       Clause 45 register
+ * 0x84     2     VAL       Register value
+ */
+#define ROLLBALL_PHY_I2C_ADDR		0x51
+
+#define ROLLBALL_CMD_ADDR		0x80
+#define ROLLBALL_DATA_ADDR		0x81
+
+#define ROLLBALL_CMD_WRITE		0x01
+#define ROLLBALL_CMD_READ		0x02
+#define ROLLBALL_CMD_DONE		0x04
+
+#define SFP_PAGE_ROLLBALL_MDIO		3
+
+static int __i2c_transfer_err(struct i2c_adapter *i2c, struct i2c_msg *msgs,
+			      int num)
+{
+	int ret;
+
+	ret = __i2c_transfer(i2c, msgs, num);
+	if (ret < 0)
+		return ret;
+	else if (ret != num)
+		return -EIO;
+	else
+		return 0;
+}
+
+static int __i2c_rollball_get_page(struct i2c_adapter *i2c, int bus_addr,
+				   u8 *page)
+{
+	struct i2c_msg msgs[2];
+	u8 addr = SFP_PAGE;
+
+	msgs[0].addr = bus_addr;
+	msgs[0].flags = 0;
+	msgs[0].len = 1;
+	msgs[0].buf = &addr;
+
+	msgs[1].addr = bus_addr;
+	msgs[1].flags = I2C_M_RD;
+	msgs[1].len = 1;
+	msgs[1].buf = page;
+
+	return __i2c_transfer_err(i2c, msgs, 2);
+}
+
+static int __i2c_rollball_set_page(struct i2c_adapter *i2c, int bus_addr,
+				   u8 page)
+{
+	struct i2c_msg msg;
+	u8 buf[2];
+
+	buf[0] = SFP_PAGE;
+	buf[1] = page;
+
+	msg.addr = bus_addr;
+	msg.flags = 0;
+	msg.len = 2;
+	msg.buf = buf;
+
+	return __i2c_transfer_err(i2c, &msg, 1);
+}
+
+/* In order to not interfere with other SFP code (which possibly may manipulate
+ * SFP_PAGE), for every transfer we do this:
+ *   1. lock the bus
+ *   2. save content of SFP_PAGE
+ *   3. set SFP_PAGE to 3
+ *   4. do the transfer
+ *   5. restore original SFP_PAGE
+ *   6. unlock the bus
+ * Note that one might think that steps 2 to 5 could be theoretically done all
+ * in one call to i2c_transfer (by constructing msgs array in such a way), but
+ * unfortunately tests show that this does not work :-( Changed SFP_PAGE does
+ * not take into account until i2c_transfer() is done.
+ */
+static int i2c_transfer_rollball(struct i2c_adapter *i2c,
+				 struct i2c_msg *msgs, int num)
+{
+	int ret, main_err = 0;
+	u8 saved_page;
+
+	i2c_lock_bus(i2c, I2C_LOCK_SEGMENT);
+
+	/* save original page */
+	ret = __i2c_rollball_get_page(i2c, msgs->addr, &saved_page);
+	if (ret)
+		goto unlock;
+
+	/* change to RollBall MDIO page */
+	ret = __i2c_rollball_set_page(i2c, msgs->addr, SFP_PAGE_ROLLBALL_MDIO);
+	if (ret)
+		goto unlock;
+
+	/* do the transfer; we try to restore original page if this fails */
+	ret = __i2c_transfer_err(i2c, msgs, num);
+	if (ret)
+		main_err = ret;
+
+	/* restore original page */
+	ret = __i2c_rollball_set_page(i2c, msgs->addr, saved_page);
+
+unlock:
+	i2c_unlock_bus(i2c, I2C_LOCK_SEGMENT);
+
+	return main_err ? : ret;
+}
+
+static int i2c_rollball_mii_poll(struct mii_bus *bus, int bus_addr, u8 *buf,
+				 size_t len)
+{
+	struct i2c_adapter *i2c = bus->priv;
+	struct i2c_msg msgs[2];
+	u8 cmd_addr, tmp, *res;
+	int i, ret;
+
+	cmd_addr = ROLLBALL_CMD_ADDR;
+
+	res = buf ? buf : &tmp;
+	len = buf ? len : 1;
+
+	msgs[0].addr = bus_addr;
+	msgs[0].flags = 0;
+	msgs[0].len = 1;
+	msgs[0].buf = &cmd_addr;
+
+	msgs[1].addr = bus_addr;
+	msgs[1].flags = I2C_M_RD;
+	msgs[1].len = len;
+	msgs[1].buf = res;
+
+	/* By experiment it takes up to 70 ms to access a register for these
+	 * SFPs. Sleep 20ms between iterations and try 10 times.
+	 */
+	i = 10;
+	do {
+		msleep(20);
+
+		ret = i2c_transfer_rollball(i2c, msgs, ARRAY_SIZE(msgs));
+		if (ret)
+			return ret;
+
+		if (*res == ROLLBALL_CMD_DONE)
+			return 0;
+	} while (i-- > 0);
+
+	dev_dbg(&bus->dev, "poll timed out\n");
+
+	return -ETIMEDOUT;
+}
+
+static int i2c_rollball_mii_cmd(struct mii_bus *bus, int bus_addr, u8 cmd,
+				u8 *data, size_t len)
+{
+	struct i2c_adapter *i2c = bus->priv;
+	struct i2c_msg msgs[2];
+	u8 cmdbuf[2];
+
+	cmdbuf[0] = ROLLBALL_CMD_ADDR;
+	cmdbuf[1] = cmd;
+
+	msgs[0].addr = bus_addr;
+	msgs[0].flags = 0;
+	msgs[0].len = len;
+	msgs[0].buf = data;
+
+	msgs[1].addr = bus_addr;
+	msgs[1].flags = 0;
+	msgs[1].len = sizeof(cmdbuf);
+	msgs[1].buf = cmdbuf;
+
+	return i2c_transfer_rollball(i2c, msgs, ARRAY_SIZE(msgs));
+}
+
+static int i2c_mii_read_rollball(struct mii_bus *bus, int phy_id, int reg)
+{
+	u8 buf[4], res[6];
+	int bus_addr, ret;
+	u16 val;
+
+	if (!(reg & MII_ADDR_C45))
+		return -EOPNOTSUPP;
+
+	bus_addr = i2c_mii_phy_addr(phy_id);
+	if (bus_addr != ROLLBALL_PHY_I2C_ADDR)
+		return 0xffff;
+
+	buf[0] = ROLLBALL_DATA_ADDR;
+	buf[1] = (reg >> 16) & 0x1f;
+	buf[2] = (reg >> 8) & 0xff;
+	buf[3] = reg & 0xff;
+
+	ret = i2c_rollball_mii_cmd(bus, bus_addr, ROLLBALL_CMD_READ, buf,
+				   sizeof(buf));
+	if (ret < 0)
+		return ret;
+
+	ret = i2c_rollball_mii_poll(bus, bus_addr, res, sizeof(res));
+	if (ret == -ETIMEDOUT)
+		return 0xffff;
+	else if (ret < 0)
+		return ret;
+
+	val = res[4] << 8 | res[5];
+
+	return val;
+}
+
+static int i2c_mii_write_rollball(struct mii_bus *bus, int phy_id, int reg,
+				  u16 val)
+{
+	int bus_addr, ret;
+	u8 buf[6];
+
+	if (!(reg & MII_ADDR_C45))
+		return -EOPNOTSUPP;
+
+	bus_addr = i2c_mii_phy_addr(phy_id);
+	if (bus_addr != ROLLBALL_PHY_I2C_ADDR)
+		return 0;
+
+	buf[0] = ROLLBALL_DATA_ADDR;
+	buf[1] = (reg >> 16) & 0x1f;
+	buf[2] = (reg >> 8) & 0xff;
+	buf[3] = reg & 0xff;
+	buf[4] = val >> 8;
+	buf[5] = val & 0xff;
+
+	ret = i2c_rollball_mii_cmd(bus, bus_addr, ROLLBALL_CMD_WRITE, buf,
+				   sizeof(buf));
+	if (ret < 0)
+		return ret;
+
+	ret = i2c_rollball_mii_poll(bus, bus_addr, NULL, 0);
+	if (ret < 0)
+		return ret;
+
+	return 0;
+}
+
+static int i2c_mii_init_rollball(struct i2c_adapter *i2c)
+{
+	struct i2c_msg msg;
+	u8 pw[5];
+	int ret;
+
+	pw[0] = SFP_PASSWORD;
+	pw[1] = 0xff;
+	pw[2] = 0xff;
+	pw[3] = 0xff;
+	pw[4] = 0xff;
+
+	msg.addr = ROLLBALL_PHY_I2C_ADDR;
+	msg.flags = 0;
+	msg.len = sizeof(pw);
+	msg.buf = pw;
+
+	ret = i2c_transfer(i2c, &msg, 1);
+	if (ret < 0)
+		return ret;
+	else if (ret != 1)
+		return -EIO;
+	else
+		return 0;
+}
+
+struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c,
+			       enum mdio_i2c_proto protocol)
 {
 	struct mii_bus *mii;
+	int ret;
 
 	if (!i2c_check_functionality(i2c, I2C_FUNC_I2C))
 		return ERR_PTR(-EINVAL);
@@ -104,10 +384,28 @@ struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c)
 
 	snprintf(mii->id, MII_BUS_ID_SIZE, "i2c:%s", dev_name(parent));
 	mii->parent = parent;
-	mii->read = i2c_mii_read;
-	mii->write = i2c_mii_write;
 	mii->priv = i2c;
 
+	switch (protocol) {
+	case MDIO_I2C_ROLLBALL:
+		ret = i2c_mii_init_rollball(i2c);
+		if (ret < 0) {
+			dev_err(parent,
+				"Cannot initialize RollBall MDIO I2C protocol: %d\n",
+				ret);
+			mdiobus_free(mii);
+			return ERR_PTR(ret);
+		}
+
+		mii->read = i2c_mii_read_rollball;
+		mii->write = i2c_mii_write_rollball;
+		break;
+	default:
+		mii->read = i2c_mii_read_default;
+		mii->write = i2c_mii_write_default;
+		break;
+	}
+
 	return mii;
 }
 EXPORT_SYMBOL_GPL(mdio_i2c_alloc);
diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c
index b2a5ed6915fa..fac5407c4b87 100644
--- a/drivers/net/phy/sfp.c
+++ b/drivers/net/phy/sfp.c
@@ -423,7 +423,7 @@ static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c)
 	sfp->read = sfp_i2c_read;
 	sfp->write = sfp_i2c_write;
 
-	i2c_mii = mdio_i2c_alloc(sfp->dev, i2c);
+	i2c_mii = mdio_i2c_alloc(sfp->dev, i2c, MDIO_I2C_DEFAULT);
 	if (IS_ERR(i2c_mii))
 		return PTR_ERR(i2c_mii);
 
diff --git a/include/linux/mdio/mdio-i2c.h b/include/linux/mdio/mdio-i2c.h
index b1d27f7cd23f..53eedb0dc1d3 100644
--- a/include/linux/mdio/mdio-i2c.h
+++ b/include/linux/mdio/mdio-i2c.h
@@ -11,6 +11,12 @@ struct device;
 struct i2c_adapter;
 struct mii_bus;
 
-struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c);
+enum mdio_i2c_proto {
+	MDIO_I2C_DEFAULT,
+	MDIO_I2C_ROLLBALL,
+};
+
+struct mii_bus *mdio_i2c_alloc(struct device *parent, struct i2c_adapter *i2c,
+			       enum mdio_i2c_proto protocol);
 
 #endif
-- 
2.26.2

[PATCH net-next v5 3/5] net: phylink: allow attaching phy for SFP modules on 802.3z mode

[Marek Behún]

Some SFPs may contain an internal PHY which may in some cases want to
connect with the host interface in 1000base-x/2500base-x mode.
Do not fail if such PHY is being attached in one of these PHY interface
modes.

Signed-off-by: Marek Behún <kabel@kernel.org>
Reviewed-by: Russell King <rmk+kernel@armlinux.org.uk>
Reviewed-by: Pali Rohár <pali@kernel.org>
Cc: Andrew Lunn <andrew@lunn.ch>
---
 drivers/net/phy/phylink.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/drivers/net/phy/phylink.c b/drivers/net/phy/phylink.c
index 84f6e197f965..f97d041f82f4 100644
--- a/drivers/net/phy/phylink.c
+++ b/drivers/net/phy/phylink.c
@@ -1018,7 +1018,7 @@ static int phylink_attach_phy(struct phylink *pl, struct phy_device *phy,
 {
 	if (WARN_ON(pl->cfg_link_an_mode == MLO_AN_FIXED ||
 		    (pl->cfg_link_an_mode == MLO_AN_INBAND &&
-		     phy_interface_mode_is_8023z(interface))))
+		     phy_interface_mode_is_8023z(interface) && !pl->sfp_bus)))
 		return -EINVAL;
 
 	if (pl->phydev)
@@ -2069,9 +2069,6 @@ static int phylink_sfp_config(struct phylink *pl, u8 mode,
 		    phylink_an_mode_str(mode), phy_modes(config.interface),
 		    __ETHTOOL_LINK_MODE_MASK_NBITS, support);
 
-	if (phy_interface_mode_is_8023z(iface) && pl->phydev)
-		return -EINVAL;
-
 	changed = !linkmode_equal(pl->supported, support);
 	if (changed) {
 		linkmode_copy(pl->supported, support);
-- 
2.26.2

[PATCH net-next v5 4/5] net: sfp: create/destroy I2C mdiobus before PHY probe/after PHY release

[Marek Behún]

Instead of configuring the I2C mdiobus when SFP driver is probed,
create/destroy the mdiobus before the PHY is probed for/after it is
released.

This way we can tell the mdio-i2c code which protocol to use for each
SFP transceiver.

Signed-off-by: Marek Behún <kabel@kernel.org>
Reviewed-by: Pali Rohár <pali@kernel.org>
Cc: Andrew Lunn <andrew@lunn.ch>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
---
 drivers/net/phy/sfp.c | 30 ++++++++++++++++++++++++++----
 1 file changed, 26 insertions(+), 4 deletions(-)

diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c
index fac5407c4b87..d1b655f805ab 100644
--- a/drivers/net/phy/sfp.c
+++ b/drivers/net/phy/sfp.c
@@ -218,6 +218,7 @@ struct sfp {
 	struct i2c_adapter *i2c;
 	struct mii_bus *i2c_mii;
 	struct sfp_bus *sfp_bus;
+	enum mdio_i2c_proto mdio_protocol;
 	struct phy_device *mod_phy;
 	const struct sff_data *type;
 	size_t i2c_block_size;
@@ -413,9 +414,6 @@ static int sfp_i2c_write(struct sfp *sfp, bool a2, u8 dev_addr, void *buf,
 
 static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c)
 {
-	struct mii_bus *i2c_mii;
-	int ret;
-
 	if (!i2c_check_functionality(i2c, I2C_FUNC_I2C))
 		return -EINVAL;
 
@@ -423,7 +421,15 @@ static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c)
 	sfp->read = sfp_i2c_read;
 	sfp->write = sfp_i2c_write;
 
-	i2c_mii = mdio_i2c_alloc(sfp->dev, i2c, MDIO_I2C_DEFAULT);
+	return 0;
+}
+
+static int sfp_i2c_mdiobus_create(struct sfp *sfp)
+{
+	struct mii_bus *i2c_mii;
+	int ret;
+
+	i2c_mii = mdio_i2c_alloc(sfp->dev, sfp->i2c, sfp->mdio_protocol);
 	if (IS_ERR(i2c_mii))
 		return PTR_ERR(i2c_mii);
 
@@ -441,6 +447,12 @@ static int sfp_i2c_configure(struct sfp *sfp, struct i2c_adapter *i2c)
 	return 0;
 }
 
+static void sfp_i2c_mdiobus_destroy(struct sfp *sfp)
+{
+	mdiobus_unregister(sfp->i2c_mii);
+	sfp->i2c_mii = NULL;
+}
+
 /* Interface */
 static int sfp_read(struct sfp *sfp, bool a2, u8 addr, void *buf, size_t len)
 {
@@ -1881,6 +1893,8 @@ static int sfp_sm_mod_probe(struct sfp *sfp, bool report)
 	else
 		sfp->module_t_start_up = T_START_UP;
 
+	sfp->mdio_protocol = MDIO_I2C_DEFAULT;
+
 	return 0;
 }
 
@@ -2051,6 +2065,8 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event)
 			sfp_module_stop(sfp->sfp_bus);
 		if (sfp->mod_phy)
 			sfp_sm_phy_detach(sfp);
+		if (sfp->i2c_mii)
+			sfp_i2c_mdiobus_destroy(sfp);
 		sfp_module_tx_disable(sfp);
 		sfp_soft_stop_poll(sfp);
 		sfp_sm_next(sfp, SFP_S_DOWN, 0);
@@ -2113,6 +2129,12 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event)
 				     sfp->sm_fault_retries == N_FAULT_INIT);
 		} else if (event == SFP_E_TIMEOUT || event == SFP_E_TX_CLEAR) {
 	init_done:
+			/* Create mdiobus and start trying for PHY */
+			ret = sfp_i2c_mdiobus_create(sfp);
+			if (ret < 0) {
+				sfp_sm_next(sfp, SFP_S_FAIL, 0);
+				break;
+			}
 			sfp->sm_phy_retries = R_PHY_RETRY;
 			goto phy_probe;
 		}
-- 
2.26.2

Re: [PATCH net-next v5 4/5] net: sfp: create/destroy I2C mdiobus before PHY probe/after PHY release

[Russell King - ARM Linux admin]

On Thu, Jan 14, 2021 at 05:43:30AM +0100, Marek Behún wrote:
> Instead of configuring the I2C mdiobus when SFP driver is probed,
> create/destroy the mdiobus before the PHY is probed for/after it is
> released.
> 
> This way we can tell the mdio-i2c code which protocol to use for each
> SFP transceiver.

I've been thinking a bit more about this. It looks like it will
allocate and free the MDIO bus each time any module is inserted or
removed, even a fiber module that wouldn't ever have a PHY. This adds
unnecessary noise to the kernel message log.

We only probe for a PHY if one of:

- id.base.extended_cc is SFF8024_ECC_10GBASE_T_SFI,
  SFF8024_ECC_10GBASE_T_SR, SFF8024_ECC_5GBASE_T, or
  SFF8024_ECC_2_5GBASE_T.
- id.base.e1000_base_t is set.

So, we only need the MDIO bus to be registered if one of those is true.

As you are introducing "enum mdio_i2c_proto", I'm wondering whether
that should include "MDIO_I2C_NONE", and we should only register the
bus and probe for a PHY if it is not MDIO_I2C_NONE.

Maybe we should have:

enum mdio_i2c_proto {
	MDIO_I2C_NONE,
	MDIO_I2C_MARVELL_C22,
	MDIO_I2C_C45,
	MDIO_I2C_ROLLBALL,
	...
};

with:

	sfp->mdio_protocol = MDIO_I2C_NONE;
	if (((!memcmp(id.base.vendor_name, "OEM             ", 16) ||
	      !memcmp(id.base.vendor_name, "Turris          ", 16)) &&
	     (!memcmp(id.base.vendor_pn, "SFP-10G-T       ", 16) ||
	      !memcmp(id.base.vendor_pn, "RTSFP-10", 8)))) {
		sfp->mdio_protocol = MDIO_I2C_ROLLBALL;
		sfp->module_t_wait = T_WAIT_ROLLBALL;
	} else {
		switch (id.base.extended_cc) {
		...
		}
	}

static int sfp_sm_add_mdio_bus(struct sfp *sfp)
{
	int err = 0;

	if (sfp->mdio_protocol != MDIO_I2C_NONE)
		err = sfp_i2c_mdiobus_create(sfp);

	return err;
}

called from the place you call sfp_i2c_mdiobus_create(), and
sfp_sm_probe_for_phy() becomes:

static int sfp_sm_probe_for_phy(struct sfp *sfp)
{
	int err = 0;

	switch (sfp->mdio_protocol) {
	case MDIO_I2C_NONE:
		break;

	case MDIO_I2C_MARVELL_C22:
		err = sfp_sm_probe_phy(sfp, SFP_PHY_ADDR, false);
		break;

	case MDIO_I2C_C45:
		err = sfp_sm_probe_phy(sfp, SFP_PHY_ADDR, true);
		break;

	case MDIO_I2C_ROLLBALL:
		err = sfp_sm_probe_phy(sfp, SFP_PHY_ADDR_ROLLBALL, true);
		break;
	}

	return err;
}

This avoids having to add the PHY address, as well as fudge around with
id.base.extended_cc to get the PHY probed.

Thoughts?

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

Re: [PATCH net-next v5 4/5] net: sfp: create/destroy I2C mdiobus before PHY probe/after PHY release

[Pali Rohár]

On Thursday 14 January 2021 16:07:19 Russell King - ARM Linux admin wrote:
> On Thu, Jan 14, 2021 at 05:43:30AM +0100, Marek Behún wrote:
> > Instead of configuring the I2C mdiobus when SFP driver is probed,
> > create/destroy the mdiobus before the PHY is probed for/after it is
> > released.
> > 
> > This way we can tell the mdio-i2c code which protocol to use for each
> > SFP transceiver.
> 
> I've been thinking a bit more about this. It looks like it will
> allocate and free the MDIO bus each time any module is inserted or
> removed, even a fiber module that wouldn't ever have a PHY. This adds
> unnecessary noise to the kernel message log.
> 
> We only probe for a PHY if one of:
> 
> - id.base.extended_cc is SFF8024_ECC_10GBASE_T_SFI,
>   SFF8024_ECC_10GBASE_T_SR, SFF8024_ECC_5GBASE_T, or
>   SFF8024_ECC_2_5GBASE_T.
> - id.base.e1000_base_t is set.
> 
> So, we only need the MDIO bus to be registered if one of those is true.
> 
> As you are introducing "enum mdio_i2c_proto", I'm wondering whether
> that should include "MDIO_I2C_NONE", and we should only register the
> bus and probe for a PHY if it is not MDIO_I2C_NONE.
> 
> Maybe we should have:
> 
> enum mdio_i2c_proto {
> 	MDIO_I2C_NONE,
> 	MDIO_I2C_MARVELL_C22,
> 	MDIO_I2C_C45,
> 	MDIO_I2C_ROLLBALL,
> 	...
> };
> 
> with:
> 
> 	sfp->mdio_protocol = MDIO_I2C_NONE;
> 	if (((!memcmp(id.base.vendor_name, "OEM             ", 16) ||
> 	      !memcmp(id.base.vendor_name, "Turris          ", 16)) &&
> 	     (!memcmp(id.base.vendor_pn, "SFP-10G-T       ", 16) ||
> 	      !memcmp(id.base.vendor_pn, "RTSFP-10", 8)))) {
> 		sfp->mdio_protocol = MDIO_I2C_ROLLBALL;
> 		sfp->module_t_wait = T_WAIT_ROLLBALL;
> 	} else {
> 		switch (id.base.extended_cc) {
> 		...
> 		}
> 	}
> 
> static int sfp_sm_add_mdio_bus(struct sfp *sfp)
> {
> 	int err = 0;
> 
> 	if (sfp->mdio_protocol != MDIO_I2C_NONE)
> 		err = sfp_i2c_mdiobus_create(sfp);
> 
> 	return err;
> }
> 
> called from the place you call sfp_i2c_mdiobus_create(), and
> sfp_sm_probe_for_phy() becomes:
> 
> static int sfp_sm_probe_for_phy(struct sfp *sfp)
> {
> 	int err = 0;
> 
> 	switch (sfp->mdio_protocol) {
> 	case MDIO_I2C_NONE:
> 		break;
> 
> 	case MDIO_I2C_MARVELL_C22:
> 		err = sfp_sm_probe_phy(sfp, SFP_PHY_ADDR, false);
> 		break;
> 
> 	case MDIO_I2C_C45:
> 		err = sfp_sm_probe_phy(sfp, SFP_PHY_ADDR, true);
> 		break;
> 
> 	case MDIO_I2C_ROLLBALL:
> 		err = sfp_sm_probe_phy(sfp, SFP_PHY_ADDR_ROLLBALL, true);
> 		break;
> 	}
> 
> 	return err;
> }
> 
> This avoids having to add the PHY address, as well as fudge around with
> id.base.extended_cc to get the PHY probed.
> 
> Thoughts?

Hello Russell! For me this solution looks more cleaner. As all those
MDIO access protocols are vendor dependent, kernel code should not
detect them only from the standard (non-vendor) extended_cc property.

Re: [PATCH net-next v5 4/5] net: sfp: create/destroy I2C mdiobus before PHY probe/after PHY release

[Marek Behún]

On Mon, 18 Jan 2021 10:38:32 +0100
Pali Rohár <pali@kernel.org> wrote:

> On Thursday 14 January 2021 16:07:19 Russell King - ARM Linux admin
> wrote:
> > On Thu, Jan 14, 2021 at 05:43:30AM +0100, Marek Behún wrote:  
> > > Instead of configuring the I2C mdiobus when SFP driver is probed,
> > > create/destroy the mdiobus before the PHY is probed for/after it
> > > is released.
> > > 
> > > This way we can tell the mdio-i2c code which protocol to use for
> > > each SFP transceiver.  
> > 
> > I've been thinking a bit more about this. It looks like it will
> > allocate and free the MDIO bus each time any module is inserted or
> > removed, even a fiber module that wouldn't ever have a PHY. This
> > adds unnecessary noise to the kernel message log.
> > 
> > We only probe for a PHY if one of:
> > 
> > - id.base.extended_cc is SFF8024_ECC_10GBASE_T_SFI,
> >   SFF8024_ECC_10GBASE_T_SR, SFF8024_ECC_5GBASE_T, or
> >   SFF8024_ECC_2_5GBASE_T.
> > - id.base.e1000_base_t is set.
> > 
> > So, we only need the MDIO bus to be registered if one of those is
> > true.
> > 
> > As you are introducing "enum mdio_i2c_proto", I'm wondering whether
> > that should include "MDIO_I2C_NONE", and we should only register the
> > bus and probe for a PHY if it is not MDIO_I2C_NONE.
> > 
> > Maybe we should have:
> > 
> > enum mdio_i2c_proto {
> > 	MDIO_I2C_NONE,
> > 	MDIO_I2C_MARVELL_C22,
> > 	MDIO_I2C_C45,
> > 	MDIO_I2C_ROLLBALL,
> > 	...
> > };
> > 
> > with:
> > 
> > 	sfp->mdio_protocol = MDIO_I2C_NONE;
> > 	if (((!memcmp(id.base.vendor_name, "OEM             ", 16)
> > || !memcmp(id.base.vendor_name, "Turris          ", 16)) &&
> > 	     (!memcmp(id.base.vendor_pn, "SFP-10G-T       ", 16) ||
> > 	      !memcmp(id.base.vendor_pn, "RTSFP-10", 8)))) {
> > 		sfp->mdio_protocol = MDIO_I2C_ROLLBALL;
> > 		sfp->module_t_wait = T_WAIT_ROLLBALL;
> > 	} else {
> > 		switch (id.base.extended_cc) {
> > 		...
> > 		}
> > 	}
> > 
> > static int sfp_sm_add_mdio_bus(struct sfp *sfp)
> > {
> > 	int err = 0;
> > 
> > 	if (sfp->mdio_protocol != MDIO_I2C_NONE)
> > 		err = sfp_i2c_mdiobus_create(sfp);
> > 
> > 	return err;
> > }
> > 
> > called from the place you call sfp_i2c_mdiobus_create(), and
> > sfp_sm_probe_for_phy() becomes:
> > 
> > static int sfp_sm_probe_for_phy(struct sfp *sfp)
> > {
> > 	int err = 0;
> > 
> > 	switch (sfp->mdio_protocol) {
> > 	case MDIO_I2C_NONE:
> > 		break;
> > 
> > 	case MDIO_I2C_MARVELL_C22:
> > 		err = sfp_sm_probe_phy(sfp, SFP_PHY_ADDR, false);
> > 		break;
> > 
> > 	case MDIO_I2C_C45:
> > 		err = sfp_sm_probe_phy(sfp, SFP_PHY_ADDR, true);
> > 		break;
> > 
> > 	case MDIO_I2C_ROLLBALL:
> > 		err = sfp_sm_probe_phy(sfp, SFP_PHY_ADDR_ROLLBALL,
> > true); break;
> > 	}
> > 
> > 	return err;
> > }
> > 
> > This avoids having to add the PHY address, as well as fudge around
> > with id.base.extended_cc to get the PHY probed.
> > 
> > Thoughts?  
> 
> Hello Russell! For me this solution looks more cleaner. As all those
> MDIO access protocols are vendor dependent, kernel code should not
> detect them only from the standard (non-vendor) extended_cc property.

I shall respin this series with this modified, then.

Marek

[PATCH net-next v5 5/5] net: sfp: add support for multigig RollBall transceivers

[Marek Behún]

This adds support for multigig copper SFP modules from RollBall/Hilink.
These modules have a specific way to access clause 45 registers of the
internal PHY.

We also need to wait at least 22 seconds after deasserting TX disable
before accessing the PHY. The code waits for 25 seconds just to be sure.

Signed-off-by: Marek Behún <kabel@kernel.org>
Reviewed-by: Russell King <rmk+kernel@armlinux.org.uk>
Cc: Andrew Lunn <andrew@lunn.ch>
---
 drivers/net/phy/sfp.c | 36 ++++++++++++++++++++++++++++++------
 1 file changed, 30 insertions(+), 6 deletions(-)

diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c
index d1b655f805ab..9f4270c19380 100644
--- a/drivers/net/phy/sfp.c
+++ b/drivers/net/phy/sfp.c
@@ -166,6 +166,7 @@ static const enum gpiod_flags gpio_flags[] = {
  * on board (for a copper SFP) time to initialise.
  */
 #define T_WAIT			msecs_to_jiffies(50)
+#define T_WAIT_ROLLBALL		msecs_to_jiffies(25000)
 #define T_START_UP		msecs_to_jiffies(300)
 #define T_START_UP_BAD_GPON	msecs_to_jiffies(60000)
 
@@ -205,8 +206,11 @@ static const enum gpiod_flags gpio_flags[] = {
 
 /* SFP modules appear to always have their PHY configured for bus address
  * 0x56 (which with mdio-i2c, translates to a PHY address of 22).
+ * RollBall SFPs access phy via SFP Enhanced Digital Diagnostic Interface
+ * via address 0x51 (mdio-i2c will use RollBall protocol on this address).
  */
-#define SFP_PHY_ADDR	22
+#define SFP_PHY_ADDR		22
+#define SFP_PHY_ADDR_ROLLBALL	17
 
 struct sff_data {
 	unsigned int gpios;
@@ -219,6 +223,7 @@ struct sfp {
 	struct mii_bus *i2c_mii;
 	struct sfp_bus *sfp_bus;
 	enum mdio_i2c_proto mdio_protocol;
+	int phy_addr;
 	struct phy_device *mod_phy;
 	const struct sff_data *type;
 	size_t i2c_block_size;
@@ -251,6 +256,7 @@ struct sfp {
 	struct sfp_eeprom_id id;
 	unsigned int module_power_mW;
 	unsigned int module_t_start_up;
+	unsigned int module_t_wait;
 
 #if IS_ENABLED(CONFIG_HWMON)
 	struct sfp_diag diag;
@@ -1505,7 +1511,7 @@ static int sfp_sm_probe_phy(struct sfp *sfp, bool is_c45)
 	struct phy_device *phy;
 	int err;
 
-	phy = get_phy_device(sfp->i2c_mii, SFP_PHY_ADDR, is_c45);
+	phy = get_phy_device(sfp->i2c_mii, sfp->phy_addr, is_c45);
 	if (phy == ERR_PTR(-ENODEV))
 		return PTR_ERR(phy);
 	if (IS_ERR(phy)) {
@@ -1895,6 +1901,23 @@ static int sfp_sm_mod_probe(struct sfp *sfp, bool report)
 
 	sfp->mdio_protocol = MDIO_I2C_DEFAULT;
 
+	sfp->phy_addr = SFP_PHY_ADDR;
+	sfp->module_t_wait = T_WAIT;
+
+	if (((!memcmp(id.base.vendor_name, "OEM             ", 16) ||
+	      !memcmp(id.base.vendor_name, "Turris          ", 16)) &&
+	     (!memcmp(id.base.vendor_pn, "SFP-10G-T       ", 16) ||
+	      !memcmp(id.base.vendor_pn, "RTSFP-10", 8)))) {
+		sfp->mdio_protocol = MDIO_I2C_ROLLBALL;
+		sfp->phy_addr = SFP_PHY_ADDR_ROLLBALL;
+		sfp->module_t_wait = T_WAIT_ROLLBALL;
+
+		/* RollBall SFPs may have wrong (zero) extended compliance code
+		 * burned in EEPROM. For PHY probing we need the correct one.
+		 */
+		id.base.extended_cc = SFF8024_ECC_10GBASE_T_SFI;
+	}
+
 	return 0;
 }
 
@@ -2090,9 +2113,10 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event)
 
 		/* We need to check the TX_FAULT state, which is not defined
 		 * while TX_DISABLE is asserted. The earliest we want to do
-		 * anything (such as probe for a PHY) is 50ms.
+		 * anything (such as probe for a PHY) is 50ms. (or more on
+		 * specific modules).
 		 */
-		sfp_sm_next(sfp, SFP_S_WAIT, T_WAIT);
+		sfp_sm_next(sfp, SFP_S_WAIT, sfp->module_t_wait);
 		break;
 
 	case SFP_S_WAIT:
@@ -2106,8 +2130,8 @@ static void sfp_sm_main(struct sfp *sfp, unsigned int event)
 			 * deasserting.
 			 */
 			timeout = sfp->module_t_start_up;
-			if (timeout > T_WAIT)
-				timeout -= T_WAIT;
+			if (timeout > sfp->module_t_wait)
+				timeout -= sfp->module_t_wait;
 			else
 				timeout = 1;
 
-- 
2.26.2