* mv88e6xxx: 2500base-x inband AN is broken on Amethyst? what to do? @ 2021-01-13 0:18 Marek Behún 2021-01-13 4:22 ` Marek Behun 2021-01-13 10:28 ` Russell King - ARM Linux admin 0 siblings, 2 replies; 8+ messages in thread From: Marek Behún @ 2021-01-13 0:18 UTC (permalink / raw) To: Russell King - ARM Linux admin, andrew; +Cc: netdev, olteanv, pavana.sharma Hello, it seems that inband AN is broken on Amethyst when on 2500base-x mode. Even SERDES scripts for Amethyst from Marvell has autonegotiation disabled for 2500base-x mode. For all the other supported Serdes modes autonegotiation is enabled in these scripts. The current implementation in mv88e6390_serdes_pcs_config() enables autonegotiation if phylink_autoneg_inband(mode) is true: if (phylink_autoneg_inband(mode)) bmcr = val | BMCR_ANENABLE; else bmcr = val & ~BMCR_ANENABLE; But for PHY_INTERFACE_MODE_2500BASEX this is broken on Amethyst. The 2500base-x mode seems to work only with autoneg disabled. The result is that when I connect via a passive SFP cable Amethyst serdes port with a Peridot serdes port, they will not link. If I disable autonegotiation on both sides, they will link, though. What is strange is that if I don't use Peridot, but connect the SFP directly to Serdes on Armada 3720, where the mvneta driver also enables autonegotiation for 2500base-x mode, they will link even if Amethyst does not enable 2500base-x. To summarize: Amethyst <-> Peridot AN - AN - works AN - AN + does not work Amethyst <-> Armada 3720 serdes AN - AN + works (It is possible that Marvell may find some workaround by touch some undocumented registers, to solve this. I will try to open a bug report.) Should we just print an error in the serdes_pcs_config method if inband autonegotiation is being requested? phylink's code currently allows connecting SFPs in non MLO_AN_INBAND mode only for when there is Broadcom BCM84881 PHY inside the SFP (by method phylink_phy_no_inband() in phylink.c). I wonder whether we can somehow in a sane way implement code to inform phylink from the mv88e6xxx driver that inband is not supported for the specific mode. Maybe the .mac_config/.pcs_config method could return an error indicating this? Or the mv88e6xxx driver can just print an error that the mode is not supported, and try to ask the user to disable AN? That would need implementing this in ethtool for SFP, though. What do you guys think? Marek ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mv88e6xxx: 2500base-x inband AN is broken on Amethyst? what to do? 2021-01-13 0:18 mv88e6xxx: 2500base-x inband AN is broken on Amethyst? what to do? Marek Behún @ 2021-01-13 4:22 ` Marek Behun 2021-01-13 10:28 ` Russell King - ARM Linux admin 1 sibling, 0 replies; 8+ messages in thread From: Marek Behun @ 2021-01-13 4:22 UTC (permalink / raw) To: Russell King - ARM Linux admin, andrew; +Cc: netdev, olteanv, pavana.sharma It seems this problem can manifest somehow with Peridot as well, at least when in combination with 88X3310 PHY. I will do some more experiments and try to come up with a solution. Marek ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mv88e6xxx: 2500base-x inband AN is broken on Amethyst? what to do? 2021-01-13 0:18 mv88e6xxx: 2500base-x inband AN is broken on Amethyst? what to do? Marek Behún 2021-01-13 4:22 ` Marek Behun @ 2021-01-13 10:28 ` Russell King - ARM Linux admin 2021-01-13 17:03 ` Marek Behún 2021-01-13 20:08 ` Marek Behún 1 sibling, 2 replies; 8+ messages in thread From: Russell King - ARM Linux admin @ 2021-01-13 10:28 UTC (permalink / raw) To: Marek Behún; +Cc: andrew, netdev, olteanv, pavana.sharma On Wed, Jan 13, 2021 at 01:18:23AM +0100, Marek Behún wrote: > Hello, > > it seems that inband AN is broken on Amethyst when on 2500base-x mode. > > Even SERDES scripts for Amethyst from Marvell has autonegotiation > disabled for 2500base-x mode. For all the other supported Serdes modes > autonegotiation is enabled in these scripts. > > The current implementation in mv88e6390_serdes_pcs_config() enables > autonegotiation if phylink_autoneg_inband(mode) is true: > > if (phylink_autoneg_inband(mode)) > bmcr = val | BMCR_ANENABLE; > else > bmcr = val & ~BMCR_ANENABLE; > > But for PHY_INTERFACE_MODE_2500BASEX this is broken on Amethyst. The > 2500base-x mode seems to work only with autoneg disabled. > > The result is that when I connect via a passive SFP cable Amethyst > serdes port with a Peridot serdes port, they will not link. If I > disable autonegotiation on both sides, they will link, though. > > What is strange is that if I don't use Peridot, but connect the SFP > directly to Serdes on Armada 3720, where the mvneta driver also enables > autonegotiation for 2500base-x mode, they will link even if Amethyst > does not enable 2500base-x. > > To summarize: > Amethyst <-> Peridot > AN - AN - works > AN - AN + does not work > > Amethyst <-> Armada 3720 serdes > AN - AN + works > > (It is possible that Marvell may find some workaround by touch some > undocumented registers, to solve this. I will try to open a bug > report.) > > Should we just print an error in the serdes_pcs_config method if inband > autonegotiation is being requested? > > phylink's code currently allows connecting SFPs in non MLO_AN_INBAND > mode only for when there is Broadcom BCM84881 PHY inside the SFP (by > method phylink_phy_no_inband() in phylink.c). > > I wonder whether we can somehow in a sane way implement code to inform > phylink from the mv88e6xxx driver that inband is not supported for the > specific mode. Maybe the .mac_config/.pcs_config method could return an > error indicating this? Or the mv88e6xxx driver can just print an error > that the mode is not supported, and try to ask the user to disable AN? > That would need implementing this in ethtool for SFP, though. > > What do you guys think? It's regrettable that some have decided not to have working in-band AN for 2500base-X, since it prevents the automatic use of pause modes - which is essentially the only use of in-band negotiation with 1000base-X since many Gigabit MACs do not support half duplex. I don't know whether the 88x3310 on the host side uses AN or not for 2500base-X - that detail isn't mentioned in the datasheet, and I don't have a setup that I could test that with. I had assumed that most people would implement 2500base-X as merely an upclocked 1000base-X, as mvneta does - unfortunately, there is /no/ published standard for 2500base-X, so we're left guessing what this should be from the implementations available at the time. As you have found, whether AN is supported at 2500base-X appears to be pretty random. However, phylib has no way to report whether a PHY wants to use in-band AN to the MAC. Hence the hack in the phylink code for BCM84881. We really need a better way for phylib to communicate the capabilities of the PHY to its user (things like which interface modes are supported, which interface modes may be dynamically switched between, and whether they can use in-band AN, and whether in-band AN bypass is supported). Also similar properties for MAC/PCS drivers, then phylink can work out what should be done. That still leaves an issue for SFP modules - when they're capable of supporting 2500base-X, we have no knowledge of the remote side. This is where knowing whether the MAC/PCS supports in-band AN bypass or not matters - if it doesn't then disabling in-band AN for 2500base-X may make sense, otherwise having in-and AN enabled but with bypass enabled would probably be sensible. Right now, we just don't have the information to make the correct decisions. -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last! ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mv88e6xxx: 2500base-x inband AN is broken on Amethyst? what to do? 2021-01-13 10:28 ` Russell King - ARM Linux admin @ 2021-01-13 17:03 ` Marek Behún 2021-01-13 20:08 ` Marek Behún 1 sibling, 0 replies; 8+ messages in thread From: Marek Behún @ 2021-01-13 17:03 UTC (permalink / raw) To: Russell King - ARM Linux admin; +Cc: andrew, netdev, olteanv, pavana.sharma On Wed, 13 Jan 2021 10:28:49 +0000 Russell King - ARM Linux admin <linux@armlinux.org.uk> wrote: > I don't know whether the 88x3310 on the host side uses AN or not for > 2500base-X - that detail isn't mentioned in the datasheet, and I don't > have a setup that I could test that with. It does not. I have been poking the registers on 88x3310, and also on Peridot and Amethyst SERDES PHYs when cmode on the switch is set to 2500base-x, and by default the inband AN is disabled. Enabling it on the PHY in Amethyst does not work at all. On Peridot your code in mv88e6xxx enables it, but when 88x3310 is connected to Peridot SERDES, it does not work until the AN is disabled on Peridot. Enabling the AN on 2500base-x mode on 88x3310 does not seem to work. I will do some more tests by poking the registers and report this. Marek ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mv88e6xxx: 2500base-x inband AN is broken on Amethyst? what to do? 2021-01-13 10:28 ` Russell King - ARM Linux admin 2021-01-13 17:03 ` Marek Behún @ 2021-01-13 20:08 ` Marek Behún 2021-01-13 21:21 ` Russell King - ARM Linux admin 1 sibling, 1 reply; 8+ messages in thread From: Marek Behún @ 2021-01-13 20:08 UTC (permalink / raw) To: Russell King - ARM Linux admin; +Cc: andrew, netdev, olteanv, pavana.sharma OK, so I did some tests with Peridot and 88X3310: On 88E6390 switch, when CMODE in the port register for a port capable of SerDes is set to 1000base-x, the switch self-configures the SerDes PHY to inband AN: register 4.2000 has value 0x1940 but when CMODE is set to 2500base-x, the switch self-configure the PHY without inband AN: register 4.2000 has value 0x0940 Also the 88X3310 PHY, when configured with MACTYPE=4 (that is the mode that switches host interface between 10gbase-r/5gbase-r/2500base-x/sgmii, depending on copper speed) and when copper speed is 2500, the PHY self-configures without inband AN: register 4.2000 has value 0x0140 It seems to me that on these Marvell devices, they consider 2500base-x not capable of inband AN and disable it by default. Moreover when the PHY has disabled inband AN and the Peridot switch has it enabled (by software), they won't link. I tried enabling the inband AN on the PHY, but it does not seem to work. Peridot can only communicate with the PHY in 2500base-x with inband AN disabled. This means that the commit a5a6858b793ff ("net: dsa: mv88e6xxx: extend phylink to Serdes PHYs") causes a regression, since the code started enabling inband AN on 2500base-x mode on the mv88e6390 family, and they stopped working with the PHY. Russell, could we, for now, just edit the code so that when mv88e6390_serdes_pcs_config is being configured with inband mode in 2500base-x, the inband mode won't be enabled and the function will print a warning? This could come with a Fixes tag so that it is backported to stable. Afterwards we can work on refactoring the phylink code so that either the driver can inform phylink whether 2500base-x inband AN is supported, or maybe we can determine from some documentation or whatnot whether inband AN is supported on 2500base-x at all. Marek ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mv88e6xxx: 2500base-x inband AN is broken on Amethyst? what to do? 2021-01-13 20:08 ` Marek Behún @ 2021-01-13 21:21 ` Russell King - ARM Linux admin 2021-01-13 21:37 ` Marek Behún 0 siblings, 1 reply; 8+ messages in thread From: Russell King - ARM Linux admin @ 2021-01-13 21:21 UTC (permalink / raw) To: Marek Behún; +Cc: andrew, netdev, olteanv, pavana.sharma On Wed, Jan 13, 2021 at 09:08:39PM +0100, Marek Behún wrote: > OK, so I did some tests with Peridot and 88X3310: > > On 88E6390 switch, when CMODE in the port register for a port capable of > SerDes is set to 1000base-x, the switch self-configures the SerDes PHY > to inband AN: > register 4.2000 has value 0x1940 > > but when CMODE is set to 2500base-x, the switch self-configure the PHY > without inband AN: > register 4.2000 has value 0x0940 > > Also the 88X3310 PHY, when configured with MACTYPE=4 > (that is the mode that switches host interface between > 10gbase-r/5gbase-r/2500base-x/sgmii, depending on copper speed) > and when copper speed is 2500, the PHY self-configures without inband > AN: > register 4.2000 has value 0x0140 > > It seems to me that on these Marvell devices, they consider 2500base-x > not capable of inband AN and disable it by default. > > Moreover when the PHY has disabled inband AN and the Peridot switch has > it enabled (by software), they won't link. I tried enabling the inband > AN on the PHY, but it does not seem to work. Peridot can only > communicate with the PHY in 2500base-x with inband AN disabled. > > This means that the commit > a5a6858b793ff ("net: dsa: mv88e6xxx: extend phylink to Serdes PHYs") > causes a regression, since the code started enabling inband AN on > 2500base-x mode on the mv88e6390 family, and they stopped working with > the PHY. > > Russell, could we, for now, just edit the code so that when > mv88e6390_serdes_pcs_config > is being configured with inband mode in 2500base-x, the inband mode > won't be enabled and the function will print a warning? > This could come with a Fixes tag so that it is backported to stable. I don't see any other easy option, so yes, please do that. > Afterwards we can work on refactoring the phylink code so that either > the driver can inform phylink whether 2500base-x inband AN is supported, > or maybe we can determine from some documentation or whatnot whether > inband AN is supported on 2500base-x at all. I suspect there is no definitive documentation on exactly what 2500base-x actually is. I suspect it may just be easier to turn off AN for 2500base-x everywhere, so at least all Linux systems are compatible irrespective of the hardware. Yes, it means losing pause negotiation, and people will have to manually set pause on each end. One thing that I don't know is whether the GPON SFP ONT modules that use 2500base-x will still function with AN disabled - although I have the modules, it appeared that they both needed a connection to the ONU to switch from 1000base-x to 2500base-x on the host side - and as I don't have an ONU I can test with, I have no way to check their behaviour. -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last! ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mv88e6xxx: 2500base-x inband AN is broken on Amethyst? what to do? 2021-01-13 21:21 ` Russell King - ARM Linux admin @ 2021-01-13 21:37 ` Marek Behún 2021-01-13 21:48 ` Russell King - ARM Linux admin 0 siblings, 1 reply; 8+ messages in thread From: Marek Behún @ 2021-01-13 21:37 UTC (permalink / raw) To: Russell King - ARM Linux admin; +Cc: andrew, netdev, olteanv, pavana.sharma On Wed, 13 Jan 2021 21:21:25 +0000 Russell King - ARM Linux admin <linux@armlinux.org.uk> wrote: > > Russell, could we, for now, just edit the code so that when > > mv88e6390_serdes_pcs_config > > is being configured with inband mode in 2500base-x, the inband mode > > won't be enabled and the function will print a warning? > > This could come with a Fixes tag so that it is backported to stable. > > I don't see any other easy option, so yes, please do that. Will do. > > Afterwards we can work on refactoring the phylink code so that either > > the driver can inform phylink whether 2500base-x inband AN is supported, > > or maybe we can determine from some documentation or whatnot whether > > inband AN is supported on 2500base-x at all. > > I suspect there is no definitive documentation on exactly what > 2500base-x actually is. I suspect it may just be easier to turn off AN > for 2500base-x everywhere, so at least all Linux systems are compatible > irrespective of the hardware. > > Yes, it means losing pause negotiation, and people will have to > manually set pause on each end. This would need a little refactoring of the phylink_sfp_config function, but I don't think it will be that hard. > One thing that I don't know is whether the GPON SFP ONT modules that > use 2500base-x will still function with AN disabled - although I have > the modules, it appeared that they both needed a connection to the ONU > to switch from 1000base-x to 2500base-x on the host side - and as I > don't have an ONU I can test with, I have no way to check their > behaviour. We have an ISP here in Prague who is willing to lend us some GPON ONU devices to get GPON SFP modules work on Turris. I will ask him if he has one capable of 2.5g and try to borrow it. Marek ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: mv88e6xxx: 2500base-x inband AN is broken on Amethyst? what to do? 2021-01-13 21:37 ` Marek Behún @ 2021-01-13 21:48 ` Russell King - ARM Linux admin 0 siblings, 0 replies; 8+ messages in thread From: Russell King - ARM Linux admin @ 2021-01-13 21:48 UTC (permalink / raw) To: Marek Behún; +Cc: andrew, netdev, olteanv, pavana.sharma On Wed, Jan 13, 2021 at 10:37:29PM +0100, Marek Behún wrote: > On Wed, 13 Jan 2021 21:21:25 +0000 > Russell King - ARM Linux admin <linux@armlinux.org.uk> wrote: > > One thing that I don't know is whether the GPON SFP ONT modules that > > use 2500base-x will still function with AN disabled - although I have > > the modules, it appeared that they both needed a connection to the ONU > > to switch from 1000base-x to 2500base-x on the host side - and as I > > don't have an ONU I can test with, I have no way to check their > > behaviour. > > We have an ISP here in Prague who is willing to lend us some GPON > ONU devices to get GPON SFP modules work on Turris. I will ask him if he > has one capable of 2.5g and try to borrow it. I know the Huawei MA5671A and the Nokia 3FE46541AA are - see the quirks table in sfp-bus.c ! They come from an ISP on the American continent who really really wanted these modules to link at 2.5Gbps. -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last! ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-01-14 2:16 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-01-13 0:18 mv88e6xxx: 2500base-x inband AN is broken on Amethyst? what to do? Marek Behún 2021-01-13 4:22 ` Marek Behun 2021-01-13 10:28 ` Russell King - ARM Linux admin 2021-01-13 17:03 ` Marek Behún 2021-01-13 20:08 ` Marek Behún 2021-01-13 21:21 ` Russell King - ARM Linux admin 2021-01-13 21:37 ` Marek Behún 2021-01-13 21:48 ` Russell King - ARM Linux admin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).