Broken imx6 to QCA8334 connection since PHYLIB to PHYLINK conversion

Broken imx6 to QCA8334 connection since PHYLIB to PHYLINK conversion

[Michal Vokáč]

Hi folks,

I am working on kernel upgrade on our imx6dl-yapp4 platform and just realized
that since v5.9 networking is broken. Git bisect brought me to commit
b3591c2a3661 ("net: dsa: qca8k: Switch to PHYLINK instead of PHYLIB")

Kernel v5.8 NFS boot without the offending commit:

qca8k 2188000.ethernet-1:0a: Using legacy PHYLIB callbacks. Please migrate to PHYLINK!
qca8k 2188000.ethernet-1:0a: nonfatal error -95 setting MTU on port 2
qca8k 2188000.ethernet-1:0a eth2 (uninitialized): PHY [2188000.ethernet-1:01] driver [Generic PHY] (irq=POLL)
qca8k 2188000.ethernet-1:0a: nonfatal error -95 setting MTU on port 3
qca8k 2188000.ethernet-1:0a eth1 (uninitialized): PHY [2188000.ethernet-1:02] driver [Generic PHY] (irq=POLL)
eth0: mtu greater than device maximum
fec 2188000.ethernet eth0: error -22 setting MTU to include DSA overhead
DSA: tree 0 setup
Generic PHY fixed-0:00: attached PHY driver [Generic PHY] (mii_bus:phy_addr=fixed-0:00, irq=POLL)
fec 2188000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
qca8k 2188000.ethernet-1:0a eth2: configuring for phy/ link mode
qca8k 2188000.ethernet-1:0a eth2: Link is Up - 1Gbps/Full - flow control rx/tx
IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
Sending DHCP requests ., OK

Kernel v5.9 and newer with that commit:

qca8k 2188000.ethernet-1:0a: configuring for fixed/rgmii-id link mode
qca8k 2188000.ethernet-1:0a eth2 (uninitialized): PHY [2188000.ethernet-1:01] driver [Generic PHY] (irq=POLL)
qca8k 2188000.ethernet-1:0a eth1 (uninitialized): PHY [2188000.ethernet-1:02] driver [Generic PHY] (irq=POLL)
DSA: tree 0 setup
qca8k 2188000.ethernet-1:0a: Link is Up - 1Gbps/Full - flow control off
fec 2188000.ethernet eth0: Unable to connect to phy
IP-Config: Failed to open eth0
IP-Config: Failed to open eth2
IP-Config: Device `eth2' not found

Manual attempt to bring the interface up:

# ifconfig eth0 up
fec 2188000.ethernet eth0: Unable to connect to phy
ifconfig: SIOCSIFFLAGS: No such device

I have no clue what could be wrong. Maybe this change revealed that our DT
configuration [1] is not correct? Or after this change the driver does not
handle the rgmii-id configuration properly? Or something else..

Any ideas how to further debug the problem?

Thank you in advance,
Michal

[1] https://elixir.bootlin.com/linux/v5.12-rc7/source/arch/arm/boot/dts/imx6dl-yapp4-common.dtsi#L101

Re: Broken imx6 to QCA8334 connection since PHYLIB to PHYLINK conversion

[Andrew Lunn]

> [1] https://elixir.bootlin.com/linux/v5.12-rc7/source/arch/arm/boot/dts/imx6dl-yapp4-common.dtsi#L101


&fec {
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_enet>;
	phy-mode = "rgmii-id";
	phy-reset-gpios = <&gpio1 25 GPIO_ACTIVE_LOW>;
	phy-reset-duration = <20>;
	phy-supply = <&sw2_reg>;
	phy-handle = <&ethphy0>;
	status = "okay";

	mdio {
		#address-cells = <1>;
		#size-cells = <0>;

		phy_port2: phy@1 {
			reg = <1>;
		};

		phy_port3: phy@2 {
			reg = <2>;
		};

		switch@10 {
			compatible = "qca,qca8334";
			reg = <10>;

			switch_ports: ports {
				#address-cells = <1>;
				#size-cells = <0>;

				ethphy0: port@0 {
					reg = <0>;
					label = "cpu";
					phy-mode = "rgmii-id";
					ethernet = <&fec>;

					fixed-link {
						speed = <1000>;
						full-duplex;
					};
				};

The fec phy-handle = <&ethphy0>; is pointing to the PHY of switch port
0. This seems wrong. Does the FEC have a PHY? Do you connect the FEC
and the SWITCH at the RGMII level? Or with two back to back PHYs?

If you are doing it RGMII level, the FEC also needs a fixed-link.

    Andrew

Re: Broken imx6 to QCA8334 connection since PHYLIB to PHYLINK conversion

[Michal Vokáč]

On 12. 04. 21 16:14, Andrew Lunn wrote:
>> [1] https://elixir.bootlin.com/linux/v5.12-rc7/source/arch/arm/boot/dts/imx6dl-yapp4-common.dtsi#L101
> 
> &fec {
> 	pinctrl-names = "default";
> 	pinctrl-0 = <&pinctrl_enet>;
> 	phy-mode = "rgmii-id";
> 	phy-reset-gpios = <&gpio1 25 GPIO_ACTIVE_LOW>;
> 	phy-reset-duration = <20>;
> 	phy-supply = <&sw2_reg>;
> 	phy-handle = <&ethphy0>;
> 	status = "okay";
> 
> 	mdio {
> 		#address-cells = <1>;
> 		#size-cells = <0>;
> 
> 		phy_port2: phy@1 {
> 			reg = <1>;
> 		};
> 
> 		phy_port3: phy@2 {
> 			reg = <2>;
> 		};
> 
> 		switch@10 {
> 			compatible = "qca,qca8334";
> 			reg = <10>;
> 
> 			switch_ports: ports {
> 				#address-cells = <1>;
> 				#size-cells = <0>;
> 
> 				ethphy0: port@0 {
> 					reg = <0>;
> 					label = "cpu";
> 					phy-mode = "rgmii-id";
> 					ethernet = <&fec>;
> 
> 					fixed-link {
> 						speed = <1000>;
> 						full-duplex;
> 					};
> 				};
> 
> The fec phy-handle = <&ethphy0>; is pointing to the PHY of switch port
> 0. This seems wrong.

I do not understand. Why this seems wrong?
The switch has four ports. Ports 2 and 3 have a PHY and are connected
to the transformers/RJ45 connectors. Port 0 is MII/RMII/RGMII of
the switch. Port 6 (not used) is a SerDes.

> Does the FEC have a PHY? Do you connect the FEC
> and the SWITCH at the RGMII level? Or with two back to back PHYs?
> 
> If you are doing it RGMII level, the FEC also needs a fixed-link.

The FEC does not have PHY and is connected to the switch at RGMII level.
Adding the fixed-link { speed = <1000>; full-duplex; }; subnode to FEC
does not help.

Michal

Re: Broken imx6 to QCA8334 connection since PHYLIB to PHYLINK conversion

[Andrew Lunn]

On Tue, Apr 13, 2021 at 09:09:37AM +0200, Michal Vokáč wrote:
> On 12. 04. 21 16:14, Andrew Lunn wrote:
> > > [1] https://elixir.bootlin.com/linux/v5.12-rc7/source/arch/arm/boot/dts/imx6dl-yapp4-common.dtsi#L101
> > 
> > &fec {
> > 	pinctrl-names = "default";
> > 	pinctrl-0 = <&pinctrl_enet>;
> > 	phy-mode = "rgmii-id";
> > 	phy-reset-gpios = <&gpio1 25 GPIO_ACTIVE_LOW>;
> > 	phy-reset-duration = <20>;
> > 	phy-supply = <&sw2_reg>;
> > 	phy-handle = <&ethphy0>;

 
> > The fec phy-handle = <&ethphy0>; is pointing to the PHY of switch port
> > 0. This seems wrong.
> 
> I do not understand. Why this seems wrong?

> The switch has four ports. Ports 2 and 3 have a PHY and are connected
> to the transformers/RJ45 connectors. Port 0 is MII/RMII/RGMII of
> the switch. Port 6 (not used) is a SerDes.
> 
> > Does the FEC have a PHY? Do you connect the FEC
> > and the SWITCH at the RGMII level? Or with two back to back PHYs?
> > 
> > If you are doing it RGMII level, the FEC also needs a fixed-link.
> 
> The FEC does not have PHY and is connected to the switch at RGMII level.
> Adding the fixed-link { speed = <1000>; full-duplex; }; subnode to FEC
> does not help.

If the FEC does not have a PHY, it should not have a
phy-handle. Instead, you need a fixed-link.

What is currently happening is that both the switch and the FEC are
trying to connect to the same PHY. Probably the switch does its
connection first and succeeds. When the FEC tries to connect, the PHY
is in use, so an error is returned.

By providing a fixed-link, instead of a phy-handle, a simulated PHY is
generated, which the FEC can connect to.

	  Andrew

Re: Broken imx6 to QCA8334 connection since PHYLIB to PHYLINK conversion

[Michal Vokáč]

On 13. 04. 21 14:44, Andrew Lunn wrote:
> On Tue, Apr 13, 2021 at 09:09:37AM +0200, Michal Vokáč wrote:
>> On 12. 04. 21 16:14, Andrew Lunn wrote:
>> The FEC does not have PHY and is connected to the switch at RGMII level.
>> Adding the fixed-link { speed = <1000>; full-duplex; }; subnode to FEC
>> does not help.
> 
> If the FEC does not have a PHY, it should not have a
> phy-handle. Instead, you need a fixed-link.
> 
> What is currently happening is that both the switch and the FEC are
> trying to connect to the same PHY. Probably the switch does its
> connection first and succeeds. When the FEC tries to connect, the PHY
> is in use, so an error is returned.
> 
> By providing a fixed-link, instead of a phy-handle, a simulated PHY is
> generated, which the FEC can connect to.

That was it, thanks a lot Andrew! I will send a patch ASAP.

Michal

Re: Broken imx6 to QCA8334 connection since PHYLIB to PHYLINK conversion

[Ioana Ciornei]

On Tue, Apr 13, 2021 at 09:09:37AM +0200, Michal Vokáč wrote:
> On 12. 04. 21 16:14, Andrew Lunn wrote:
> > > [1] https://elixir.bootlin.com/linux/v5.12-rc7/source/arch/arm/boot/dts/imx6dl-yapp4-common.dtsi#L101
> > 
> > &fec {
> > 	pinctrl-names = "default";
> > 	pinctrl-0 = <&pinctrl_enet>;
> > 	phy-mode = "rgmii-id";
> > 	phy-reset-gpios = <&gpio1 25 GPIO_ACTIVE_LOW>;
> > 	phy-reset-duration = <20>;
> > 	phy-supply = <&sw2_reg>;
> > 	phy-handle = <&ethphy0>;
> > 	status = "okay";
> > 
> > 	mdio {
> > 		#address-cells = <1>;
> > 		#size-cells = <0>;
> > 
> > 		phy_port2: phy@1 {
> > 			reg = <1>;
> > 		};
> > 
> > 		phy_port3: phy@2 {
> > 			reg = <2>;
> > 		};
> > 
> > 		switch@10 {
> > 			compatible = "qca,qca8334";
> > 			reg = <10>;
> > 
> > 			switch_ports: ports {
> > 				#address-cells = <1>;
> > 				#size-cells = <0>;
> > 
> > 				ethphy0: port@0 {
> > 					reg = <0>;
> > 					label = "cpu";
> > 					phy-mode = "rgmii-id";
> > 					ethernet = <&fec>;
> > 
> > 					fixed-link {
> > 						speed = <1000>;
> > 						full-duplex;
> > 					};
> > 				};
> > 
> > The fec phy-handle = <&ethphy0>; is pointing to the PHY of switch port
> > 0. This seems wrong.
> 

Actually, the phy-handle is pointing directly to the switch port 0 node.

> I do not understand. Why this seems wrong?

The phy-handle property should point to a node representing a PHY
device. If a fixed-link subnode is present, no phy-handle is needed.

> The switch has four ports. Ports 2 and 3 have a PHY and are connected
> to the transformers/RJ45 connectors. Port 0 is MII/RMII/RGMII of
> the switch. Port 6 (not used) is a SerDes.
> 
> > Does the FEC have a PHY? Do you connect the FEC
> > and the SWITCH at the RGMII level? Or with two back to back PHYs?
> > 
> > If you are doing it RGMII level, the FEC also needs a fixed-link.
> 
> The FEC does not have PHY and is connected to the switch at RGMII level.
> Adding the fixed-link { speed = <1000>; full-duplex; }; subnode to FEC
> does not help.
> 

Did you also remove the extra phy-handle when you tested?

Ioana