[PATCH net-next 0/3] Cadence MACB/GEM support for ZynqMP SGMII

[PATCH net-next 0/3] Cadence MACB/GEM support for ZynqMP SGMII

[Robert Hancock]

Changes to allow SGMII mode to work properly in the GEM driver on the
Xilinx ZynqMP platform.

Robert Hancock (3):
  macb: bindings doc: added generic PHY and reset mappings for ZynqMP
  net: macb: Added ZynqMP-specific initialization
  arm64: dts: zynqmp: Added GEM reset definitions

 .../devicetree/bindings/net/macb.txt          | 33 +++++++++++++
 arch/arm64/boot/dts/xilinx/zynqmp.dtsi        |  8 ++++
 drivers/net/ethernet/cadence/macb_main.c      | 47 ++++++++++++++++++-
 3 files changed, 87 insertions(+), 1 deletion(-)

-- 
2.31.1

[PATCH net-next 1/3] macb: bindings doc: added generic PHY and reset mappings for ZynqMP

[Robert Hancock]

Updated macb DT binding documentation to reflect the phy-names, phys,
resets, reset-names properties which are now used with ZynqMP GEM
devices, and added a ZynqMP-specific DT example.

Signed-off-by: Robert Hancock <robert.hancock@calian.com>
---
 .../devicetree/bindings/net/macb.txt          | 33 +++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/Documentation/devicetree/bindings/net/macb.txt b/Documentation/devicetree/bindings/net/macb.txt
index a1b06fd1962e..e526952145b8 100644
--- a/Documentation/devicetree/bindings/net/macb.txt
+++ b/Documentation/devicetree/bindings/net/macb.txt
@@ -29,6 +29,12 @@ Required properties:
 	Optional elements: 'rx_clk' applies to cdns,zynqmp-gem
 	Optional elements: 'tsu_clk'
 - clocks: Phandles to input clocks.
+- phy_names, phys: Required with ZynqMP SoC when in SGMII mode.
+                   phy_names should be "sgmii-phy" and phys should
+                   reference PS-GTR generic PHY device for this controller
+                   instance. See ZynqMP example below.
+- resets, reset-names: Recommended with ZynqMP, specify reset control for this
+		       controller instance with zynqmp-reset driver.
 
 Optional properties:
 - mdio: node containing PHY children. If this node is not present, then PHYs
@@ -58,3 +64,30 @@ Examples:
 			reset-gpios = <&pioE 6 1>;
 		};
 	};
+
+	gem1: ethernet@ff0c0000 {
+		compatible = "cdns,zynqmp-gem", "cdns,gem";
+		interrupt-parent = <&gic>;
+		interrupts = <0 59 4>, <0 59 4>;
+		reg = <0x0 0xff0c0000 0x0 0x1000>;
+		clocks = <&zynqmp_clk LPD_LSBUS>, <&zynqmp_clk GEM1_REF>,
+			 <&zynqmp_clk GEM1_TX>, <&zynqmp_clk GEM1_RX>,
+			 <&zynqmp_clk GEM_TSU>;
+		clock-names = "pclk", "hclk", "tx_clk", "rx_clk", "tsu_clk";
+		#address-cells = <1>;
+		#size-cells = <0>;
+		#stream-id-cells = <1>;
+		iommus = <&smmu 0x875>;
+		power-domains = <&zynqmp_firmware PD_ETH_1>;
+		resets = <&zynqmp_reset ZYNQMP_RESET_GEM1>;
+		reset-names = "gem1_rst";
+		status = "okay";
+		phy-mode = "sgmii";
+		phy-names = "sgmii-phy";
+		phys = <&psgtr 1 PHY_TYPE_SGMII 1 1>;
+		fixed-link {
+			speed = <1000>;
+			full-duplex;
+			pause;
+		};
+	};
-- 
2.31.1

Re: [PATCH net-next 1/3] macb: bindings doc: added generic PHY and reset mappings for ZynqMP

[Michal Simek]

On 1/12/22 19:11, Robert Hancock wrote:
> Updated macb DT binding documentation to reflect the phy-names, phys,
> resets, reset-names properties which are now used with ZynqMP GEM
> devices, and added a ZynqMP-specific DT example.
> 
> Signed-off-by: Robert Hancock <robert.hancock@calian.com>
> ---
>   .../devicetree/bindings/net/macb.txt          | 33 +++++++++++++++++++
>   1 file changed, 33 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/net/macb.txt b/Documentation/devicetree/bindings/net/macb.txt
> index a1b06fd1962e..e526952145b8 100644
> --- a/Documentation/devicetree/bindings/net/macb.txt
> +++ b/Documentation/devicetree/bindings/net/macb.txt
> @@ -29,6 +29,12 @@ Required properties:
>   	Optional elements: 'rx_clk' applies to cdns,zynqmp-gem
>   	Optional elements: 'tsu_clk'
>   - clocks: Phandles to input clocks.
> +- phy_names, phys: Required with ZynqMP SoC when in SGMII mode.
> +                   phy_names should be "sgmii-phy" and phys should
> +                   reference PS-GTR generic PHY device for this controller
> +                   instance. See ZynqMP example below.
> +- resets, reset-names: Recommended with ZynqMP, specify reset control for this
> +		       controller instance with zynqmp-reset driver.
>   
>   Optional properties:
>   - mdio: node containing PHY children. If this node is not present, then PHYs
> @@ -58,3 +64,30 @@ Examples:
>   			reset-gpios = <&pioE 6 1>;
>   		};
>   	};
> +
> +	gem1: ethernet@ff0c0000 {
> +		compatible = "cdns,zynqmp-gem", "cdns,gem";
> +		interrupt-parent = <&gic>;
> +		interrupts = <0 59 4>, <0 59 4>;
> +		reg = <0x0 0xff0c0000 0x0 0x1000>;
> +		clocks = <&zynqmp_clk LPD_LSBUS>, <&zynqmp_clk GEM1_REF>,
> +			 <&zynqmp_clk GEM1_TX>, <&zynqmp_clk GEM1_RX>,
> +			 <&zynqmp_clk GEM_TSU>;
> +		clock-names = "pclk", "hclk", "tx_clk", "rx_clk", "tsu_clk";
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +		#stream-id-cells = <1>;
> +		iommus = <&smmu 0x875>;
> +		power-domains = <&zynqmp_firmware PD_ETH_1>;
> +		resets = <&zynqmp_reset ZYNQMP_RESET_GEM1>;
> +		reset-names = "gem1_rst";
> +		status = "okay";
> +		phy-mode = "sgmii";
> +		phy-names = "sgmii-phy";
> +		phys = <&psgtr 1 PHY_TYPE_SGMII 1 1>;
> +		fixed-link {
> +			speed = <1000>;
> +			full-duplex;
> +			pause;
> +		};
> +	};


Geert already converted this file to yaml that's why you should target this version.

Thanks,
Michal

Re: [PATCH net-next 1/3] macb: bindings doc: added generic PHY and reset mappings for ZynqMP

[Robert Hancock]

On Thu, 2022-01-13 at 08:25 +0100, Michal Simek wrote:
> 
> On 1/12/22 19:11, Robert Hancock wrote:
> > Updated macb DT binding documentation to reflect the phy-names, phys,
> > resets, reset-names properties which are now used with ZynqMP GEM
> > devices, and added a ZynqMP-specific DT example.
> > 
> > Signed-off-by: Robert Hancock <robert.hancock@calian.com>
> > ---
> >   .../devicetree/bindings/net/macb.txt          | 33 +++++++++++++++++++
> >   1 file changed, 33 insertions(+)
> > 
> > diff --git a/Documentation/devicetree/bindings/net/macb.txt
> > b/Documentation/devicetree/bindings/net/macb.txt
> > index a1b06fd1962e..e526952145b8 100644
> > --- a/Documentation/devicetree/bindings/net/macb.txt
> > +++ b/Documentation/devicetree/bindings/net/macb.txt
> > @@ -29,6 +29,12 @@ Required properties:
> >   	Optional elements: 'rx_clk' applies to cdns,zynqmp-gem
> >   	Optional elements: 'tsu_clk'
> >   - clocks: Phandles to input clocks.
> > +- phy_names, phys: Required with ZynqMP SoC when in SGMII mode.
> > +                   phy_names should be "sgmii-phy" and phys should
> > +                   reference PS-GTR generic PHY device for this controller
> > +                   instance. See ZynqMP example below.
> > +- resets, reset-names: Recommended with ZynqMP, specify reset control for
> > this
> > +		       controller instance with zynqmp-reset driver.
> >   
> >   Optional properties:
> >   - mdio: node containing PHY children. If this node is not present, then
> > PHYs
> > @@ -58,3 +64,30 @@ Examples:
> >   			reset-gpios = <&pioE 6 1>;
> >   		};
> >   	};
> > +
> > +	gem1: ethernet@ff0c0000 {
> > +		compatible = "cdns,zynqmp-gem", "cdns,gem";
> > +		interrupt-parent = <&gic>;
> > +		interrupts = <0 59 4>, <0 59 4>;
> > +		reg = <0x0 0xff0c0000 0x0 0x1000>;
> > +		clocks = <&zynqmp_clk LPD_LSBUS>, <&zynqmp_clk GEM1_REF>,
> > +			 <&zynqmp_clk GEM1_TX>, <&zynqmp_clk GEM1_RX>,
> > +			 <&zynqmp_clk GEM_TSU>;
> > +		clock-names = "pclk", "hclk", "tx_clk", "rx_clk", "tsu_clk";
> > +		#address-cells = <1>;
> > +		#size-cells = <0>;
> > +		#stream-id-cells = <1>;
> > +		iommus = <&smmu 0x875>;
> > +		power-domains = <&zynqmp_firmware PD_ETH_1>;
> > +		resets = <&zynqmp_reset ZYNQMP_RESET_GEM1>;
> > +		reset-names = "gem1_rst";
> > +		status = "okay";
> > +		phy-mode = "sgmii";
> > +		phy-names = "sgmii-phy";
> > +		phys = <&psgtr 1 PHY_TYPE_SGMII 1 1>;
> > +		fixed-link {
> > +			speed = <1000>;
> > +			full-duplex;
> > +			pause;
> > +		};
> > +	};
> 
> Geert already converted this file to yaml that's why you should target this
> version.

Is that version in a tree somewhere that can be patched against?

> 
> Thanks,
> Michal
-- 
Robert Hancock
Senior Hardware Designer, Calian Advanced Technologies
www.calian.com

Re: [PATCH net-next 1/3] macb: bindings doc: added generic PHY and reset mappings for ZynqMP

[Geert Uytterhoeven]

Hi Robert,

On Thu, Jan 13, 2022 at 5:34 PM Robert Hancock
<robert.hancock@calian.com> wrote:
> On Thu, 2022-01-13 at 08:25 +0100, Michal Simek wrote:
> > On 1/12/22 19:11, Robert Hancock wrote:
> > > Updated macb DT binding documentation to reflect the phy-names, phys,
> > > resets, reset-names properties which are now used with ZynqMP GEM
> > > devices, and added a ZynqMP-specific DT example.
> > >
> > > Signed-off-by: Robert Hancock <robert.hancock@calian.com>

> > > --- a/Documentation/devicetree/bindings/net/macb.txt
> > > +++ b/Documentation/devicetree/bindings/net/macb.txt

> > Geert already converted this file to yaml that's why you should target this
> > version.
>
> Is that version in a tree somewhere that can be patched against?

It has just entered upstream, and will be part of v5.17-rc1:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4e5b6de1f46d0ea0

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH net-next 1/3] macb: bindings doc: added generic PHY and reset mappings for ZynqMP

[Robert Hancock]

On Thu, 2022-01-13 at 18:43 +0100, Geert Uytterhoeven wrote:
> Hi Robert,
> 
> On Thu, Jan 13, 2022 at 5:34 PM Robert Hancock
> <robert.hancock@calian.com> wrote:
> > On Thu, 2022-01-13 at 08:25 +0100, Michal Simek wrote:
> > > On 1/12/22 19:11, Robert Hancock wrote:
> > > > Updated macb DT binding documentation to reflect the phy-names, phys,
> > > > resets, reset-names properties which are now used with ZynqMP GEM
> > > > devices, and added a ZynqMP-specific DT example.
> > > > 
> > > > Signed-off-by: Robert Hancock <robert.hancock@calian.com>
> > > > --- a/Documentation/devicetree/bindings/net/macb.txt
> > > > +++ b/Documentation/devicetree/bindings/net/macb.txt
> > > Geert already converted this file to yaml that's why you should target
> > > this
> > > version.
> > 
> > Is that version in a tree somewhere that can be patched against?
> 
> It has just entered upstream, and will be part of v5.17-rc1:
> https://urldefense.com/v3/__https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4e5b6de1f46d0ea0__;!!IOGos0k!yRbmcDEaC2OgoZAK9hyg-FUIkIcYg6JKqNF7y0Tyw-fbnXvKAIRsccN9K5iGDZkhTko$ 
> 

Ah, I see, it went in through the devicetree tree so it's not in net-next yet.
Should be able to pick that change up once the merge window closes and update
the patch accordingly.

> Gr{oetje,eeting}s,
> 
>                         Geert
> 
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- 
> geert@linux-m68k.org
> 
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like
> that.
>                                 -- Linus Torvalds

[PATCH net-next 2/3] net: macb: Added ZynqMP-specific initialization

[Robert Hancock]

The GEM controllers on ZynqMP were missing some initialization steps which
are required in some cases when using SGMII mode, which uses the PS-GTR
transceivers managed by the phy-zynqmp driver.

The GEM core appears to need a hardware-level reset in order to work
properly in SGMII mode in cases where the GT reference clock was not
present at initial power-on. This can be done using a reset mapped to
the zynqmp-reset driver in the device tree.

Also, when in SGMII mode, the GEM driver needs to ensure the PHY is
initialized and powered on when it is initializing.

Signed-off-by: Robert Hancock <robert.hancock@calian.com>
---
 drivers/net/ethernet/cadence/macb_main.c | 47 +++++++++++++++++++++++-
 1 file changed, 46 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index a363da928e8b..65b0360c487a 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -34,7 +34,9 @@
 #include <linux/udp.h>
 #include <linux/tcp.h>
 #include <linux/iopoll.h>
+#include <linux/phy/phy.h>
 #include <linux/pm_runtime.h>
+#include <linux/reset.h>
 #include "macb.h"
 
 /* This structure is only used for MACB on SiFive FU540 devices */
@@ -4455,6 +4457,49 @@ static int fu540_c000_init(struct platform_device *pdev)
 	return macb_init(pdev);
 }
 
+static int zynqmp_init(struct platform_device *pdev)
+{
+	struct net_device *dev = platform_get_drvdata(pdev);
+	struct macb *bp = netdev_priv(dev);
+	int ret;
+
+	/* Fully reset GEM controller at hardware level using zynqmp-reset driver,
+	 * if mapped in device tree.
+	 */
+	ret = device_reset(&pdev->dev);
+	if (ret) {
+		dev_err_probe(&pdev->dev, ret, "failed to reset controller");
+		return ret;
+	}
+
+	if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII) {
+		/* Ensure PS-GTR PHY device used in SGMII mode is ready */
+		struct phy *sgmii_phy = devm_phy_get(&pdev->dev, "sgmii-phy");
+
+		if (IS_ERR(sgmii_phy)) {
+			ret = PTR_ERR(sgmii_phy);
+			dev_err_probe(&pdev->dev, ret,
+				      "failed to get PS-GTR PHY\n");
+			return ret;
+		}
+
+		ret = phy_init(sgmii_phy);
+		if (ret) {
+			dev_err(&pdev->dev, "failed to init PS-GTR PHY: %d\n",
+				ret);
+			return ret;
+		}
+
+		ret = phy_power_on(sgmii_phy);
+		if (ret) {
+			dev_err(&pdev->dev, "failed to power on PS-GTR PHY: %d\n",
+				ret);
+			return ret;
+		}
+	}
+	return macb_init(pdev);
+}
+
 static const struct macb_usrio_config sama7g5_usrio = {
 	.mii = 0,
 	.rmii = 1,
@@ -4550,7 +4595,7 @@ static const struct macb_config zynqmp_config = {
 			MACB_CAPS_GEM_HAS_PTP | MACB_CAPS_BD_RD_PREFETCH,
 	.dma_burst_length = 16,
 	.clk_init = macb_clk_init,
-	.init = macb_init,
+	.init = zynqmp_init,
 	.jumbo_max_len = 10240,
 	.usrio = &macb_default_usrio,
 };
-- 
2.31.1

Re: [PATCH net-next 2/3] net: macb: Added ZynqMP-specific initialization

[Michal Simek]

On 1/12/22 19:11, Robert Hancock wrote:
> The GEM controllers on ZynqMP were missing some initialization steps which
> are required in some cases when using SGMII mode, which uses the PS-GTR
> transceivers managed by the phy-zynqmp driver.
> 
> The GEM core appears to need a hardware-level reset in order to work
> properly in SGMII mode in cases where the GT reference clock was not
> present at initial power-on. This can be done using a reset mapped to
> the zynqmp-reset driver in the device tree.
> 
> Also, when in SGMII mode, the GEM driver needs to ensure the PHY is
> initialized and powered on when it is initializing.
> 
> Signed-off-by: Robert Hancock <robert.hancock@calian.com>
> ---
>   drivers/net/ethernet/cadence/macb_main.c | 47 +++++++++++++++++++++++-
>   1 file changed, 46 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
> index a363da928e8b..65b0360c487a 100644
> --- a/drivers/net/ethernet/cadence/macb_main.c
> +++ b/drivers/net/ethernet/cadence/macb_main.c
> @@ -34,7 +34,9 @@
>   #include <linux/udp.h>
>   #include <linux/tcp.h>
>   #include <linux/iopoll.h>
> +#include <linux/phy/phy.h>
>   #include <linux/pm_runtime.h>
> +#include <linux/reset.h>
>   #include "macb.h"
>   
>   /* This structure is only used for MACB on SiFive FU540 devices */
> @@ -4455,6 +4457,49 @@ static int fu540_c000_init(struct platform_device *pdev)
>   	return macb_init(pdev);
>   }
>   
> +static int zynqmp_init(struct platform_device *pdev)
> +{
> +	struct net_device *dev = platform_get_drvdata(pdev);
> +	struct macb *bp = netdev_priv(dev);
> +	int ret;
> +
> +	/* Fully reset GEM controller at hardware level using zynqmp-reset driver,
> +	 * if mapped in device tree.
> +	 */
> +	ret = device_reset(&pdev->dev);
> +	if (ret) {
> +		dev_err_probe(&pdev->dev, ret, "failed to reset controller");
> +		return ret;
> +	}
> +
> +	if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII) {
> +		/* Ensure PS-GTR PHY device used in SGMII mode is ready */
> +		struct phy *sgmii_phy = devm_phy_get(&pdev->dev, "sgmii-phy");
> +
> +		if (IS_ERR(sgmii_phy)) {
> +			ret = PTR_ERR(sgmii_phy);
> +			dev_err_probe(&pdev->dev, ret,
> +				      "failed to get PS-GTR PHY\n");
> +			return ret;
> +		}
> +
> +		ret = phy_init(sgmii_phy);
> +		if (ret) {
> +			dev_err(&pdev->dev, "failed to init PS-GTR PHY: %d\n",
> +				ret);
> +			return ret;
> +		}

I was playing with it recently on u-boot side and device reset should happen 
between phy init and phy power on to finish calibration.
At least that's I was told and that's I use in u-boot driver.

Harini/Piyush: Please correct me if I am wrong.

Thanks,
Michal

Re: [PATCH net-next 2/3] net: macb: Added ZynqMP-specific initialization

[Harini Katakam]

+Radhey

Hi Robert,

On Thu, Jan 13, 2022 at 2:46 PM Michal Simek <michal.simek@xilinx.com> wrote:
>
>
>
> On 1/12/22 19:11, Robert Hancock wrote:
> > The GEM controllers on ZynqMP were missing some initialization steps which
> > are required in some cases when using SGMII mode, which uses the PS-GTR
> > transceivers managed by the phy-zynqmp driver.
> >
> > The GEM core appears to need a hardware-level reset in order to work
> > properly in SGMII mode in cases where the GT reference clock was not
> > present at initial power-on. This can be done using a reset mapped to
> > the zynqmp-reset driver in the device tree.
> >
> > Also, when in SGMII mode, the GEM driver needs to ensure the PHY is
> > initialized and powered on when it is initializing.
> >
> > Signed-off-by: Robert Hancock <robert.hancock@calian.com>
> > ---
> >   drivers/net/ethernet/cadence/macb_main.c | 47 +++++++++++++++++++++++-
> >   1 file changed, 46 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
> > index a363da928e8b..65b0360c487a 100644
> > --- a/drivers/net/ethernet/cadence/macb_main.c
> > +++ b/drivers/net/ethernet/cadence/macb_main.c
> > @@ -34,7 +34,9 @@
> >   #include <linux/udp.h>
> >   #include <linux/tcp.h>
> >   #include <linux/iopoll.h>
> > +#include <linux/phy/phy.h>
> >   #include <linux/pm_runtime.h>
> > +#include <linux/reset.h>
> >   #include "macb.h"
> >
> >   /* This structure is only used for MACB on SiFive FU540 devices */
> > @@ -4455,6 +4457,49 @@ static int fu540_c000_init(struct platform_device *pdev)
> >       return macb_init(pdev);
> >   }
> >
> > +static int zynqmp_init(struct platform_device *pdev)
> > +{
> > +     struct net_device *dev = platform_get_drvdata(pdev);
> > +     struct macb *bp = netdev_priv(dev);
> > +     int ret;
> > +
> > +     /* Fully reset GEM controller at hardware level using zynqmp-reset driver,
> > +      * if mapped in device tree.
> > +      */
> > +     ret = device_reset(&pdev->dev);
> > +     if (ret) {
> > +             dev_err_probe(&pdev->dev, ret, "failed to reset controller");
> > +             return ret;
> > +     }
> > +
> > +     if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII) {
> > +             /* Ensure PS-GTR PHY device used in SGMII mode is ready */
> > +             struct phy *sgmii_phy = devm_phy_get(&pdev->dev, "sgmii-phy");
> > +
> > +             if (IS_ERR(sgmii_phy)) {
> > +                     ret = PTR_ERR(sgmii_phy);
> > +                     dev_err_probe(&pdev->dev, ret,
> > +                                   "failed to get PS-GTR PHY\n");
> > +                     return ret;
> > +             }
> > +
> > +             ret = phy_init(sgmii_phy);
> > +             if (ret) {
> > +                     dev_err(&pdev->dev, "failed to init PS-GTR PHY: %d\n",
> > +                             ret);
> > +                     return ret;
> > +             }
>
> I was playing with it recently on u-boot side and device reset should happen
> between phy init and phy power on to finish calibration.
> At least that's I was told and that's I use in u-boot driver.
>
> Harini/Piyush: Please correct me if I am wrong.

Thanks for the patch.

GEM should definitely be reset once after the serdes init and power on is done.
It can be held in reset and released after serdes init or reset with a 1-0 after
serdes init. Either should be fine but a reset before phy init may not work.
I've added Radhey who worked on this recently and can add any further info.

Regards,
Harini

Re: [PATCH net-next 2/3] net: macb: Added ZynqMP-specific initialization

[Robert Hancock]

On Thu, 2022-01-13 at 15:44 +0530, Harini Katakam wrote:
> +Radhey
> 
> Hi Robert,
> 
> On Thu, Jan 13, 2022 at 2:46 PM Michal Simek <michal.simek@xilinx.com> wrote:
> > 
> > 
> > On 1/12/22 19:11, Robert Hancock wrote:
> > > The GEM controllers on ZynqMP were missing some initialization steps
> > > which
> > > are required in some cases when using SGMII mode, which uses the PS-GTR
> > > transceivers managed by the phy-zynqmp driver.
> > > 
> > > The GEM core appears to need a hardware-level reset in order to work
> > > properly in SGMII mode in cases where the GT reference clock was not
> > > present at initial power-on. This can be done using a reset mapped to
> > > the zynqmp-reset driver in the device tree.
> > > 
> > > Also, when in SGMII mode, the GEM driver needs to ensure the PHY is
> > > initialized and powered on when it is initializing.
> > > 
> > > Signed-off-by: Robert Hancock <robert.hancock@calian.com>
> > > ---
> > >   drivers/net/ethernet/cadence/macb_main.c | 47 +++++++++++++++++++++++-
> > >   1 file changed, 46 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/drivers/net/ethernet/cadence/macb_main.c
> > > b/drivers/net/ethernet/cadence/macb_main.c
> > > index a363da928e8b..65b0360c487a 100644
> > > --- a/drivers/net/ethernet/cadence/macb_main.c
> > > +++ b/drivers/net/ethernet/cadence/macb_main.c
> > > @@ -34,7 +34,9 @@
> > >   #include <linux/udp.h>
> > >   #include <linux/tcp.h>
> > >   #include <linux/iopoll.h>
> > > +#include <linux/phy/phy.h>
> > >   #include <linux/pm_runtime.h>
> > > +#include <linux/reset.h>
> > >   #include "macb.h"
> > > 
> > >   /* This structure is only used for MACB on SiFive FU540 devices */
> > > @@ -4455,6 +4457,49 @@ static int fu540_c000_init(struct platform_device
> > > *pdev)
> > >       return macb_init(pdev);
> > >   }
> > > 
> > > +static int zynqmp_init(struct platform_device *pdev)
> > > +{
> > > +     struct net_device *dev = platform_get_drvdata(pdev);
> > > +     struct macb *bp = netdev_priv(dev);
> > > +     int ret;
> > > +
> > > +     /* Fully reset GEM controller at hardware level using zynqmp-reset
> > > driver,
> > > +      * if mapped in device tree.
> > > +      */
> > > +     ret = device_reset(&pdev->dev);
> > > +     if (ret) {
> > > +             dev_err_probe(&pdev->dev, ret, "failed to reset
> > > controller");
> > > +             return ret;
> > > +     }
> > > +
> > > +     if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII) {
> > > +             /* Ensure PS-GTR PHY device used in SGMII mode is ready */
> > > +             struct phy *sgmii_phy = devm_phy_get(&pdev->dev, "sgmii-
> > > phy");
> > > +
> > > +             if (IS_ERR(sgmii_phy)) {
> > > +                     ret = PTR_ERR(sgmii_phy);
> > > +                     dev_err_probe(&pdev->dev, ret,
> > > +                                   "failed to get PS-GTR PHY\n");
> > > +                     return ret;
> > > +             }
> > > +
> > > +             ret = phy_init(sgmii_phy);
> > > +             if (ret) {
> > > +                     dev_err(&pdev->dev, "failed to init PS-GTR PHY:
> > > %d\n",
> > > +                             ret);
> > > +                     return ret;
> > > +             }
> > 
> > I was playing with it recently on u-boot side and device reset should
> > happen
> > between phy init and phy power on to finish calibration.
> > At least that's I was told and that's I use in u-boot driver.
> > 
> > Harini/Piyush: Please correct me if I am wrong.
> 
> Thanks for the patch.
> 
> GEM should definitely be reset once after the serdes init and power on is
> done.
> It can be held in reset and released after serdes init or reset with a 1-0
> after
> serdes init. Either should be fine but a reset before phy init may not work.
> I've added Radhey who worked on this recently and can add any further info.

Thanks for the feedback on this. I believe I pretty much arrived at the
sequence I had via trial and error when trying to get things to work and so it
might not quite be ideal. I've done some tests with moving the reset down to
after the PHY init and power on and that seems to work well, so if that's the
preferred sequence I can switch to that.

> 
> Regards,
> Harini
-- 
Robert Hancock
Senior Hardware Designer, Calian Advanced Technologies
www.calian.com

Re: [PATCH net-next 2/3] net: macb: Added ZynqMP-specific initialization

[Claudiu.Beznea]

On 12.01.2022 20:11, Robert Hancock wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> 
> The GEM controllers on ZynqMP were missing some initialization steps which
> are required in some cases when using SGMII mode, which uses the PS-GTR
> transceivers managed by the phy-zynqmp driver.
> 
> The GEM core appears to need a hardware-level reset in order to work
> properly in SGMII mode in cases where the GT reference clock was not
> present at initial power-on. This can be done using a reset mapped to
> the zynqmp-reset driver in the device tree.
> 
> Also, when in SGMII mode, the GEM driver needs to ensure the PHY is
> initialized and powered on when it is initializing.
> 
> Signed-off-by: Robert Hancock <robert.hancock@calian.com>
> ---
>  drivers/net/ethernet/cadence/macb_main.c | 47 +++++++++++++++++++++++-
>  1 file changed, 46 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
> index a363da928e8b..65b0360c487a 100644
> --- a/drivers/net/ethernet/cadence/macb_main.c
> +++ b/drivers/net/ethernet/cadence/macb_main.c
> @@ -34,7 +34,9 @@
>  #include <linux/udp.h>
>  #include <linux/tcp.h>
>  #include <linux/iopoll.h>
> +#include <linux/phy/phy.h>
>  #include <linux/pm_runtime.h>
> +#include <linux/reset.h>
>  #include "macb.h"
> 
>  /* This structure is only used for MACB on SiFive FU540 devices */
> @@ -4455,6 +4457,49 @@ static int fu540_c000_init(struct platform_device *pdev)
>         return macb_init(pdev);
>  }
> 
> +static int zynqmp_init(struct platform_device *pdev)
> +{
> +       struct net_device *dev = platform_get_drvdata(pdev);
> +       struct macb *bp = netdev_priv(dev);
> +       int ret;
> +
> +       /* Fully reset GEM controller at hardware level using zynqmp-reset driver,
> +        * if mapped in device tree.
> +        */
> +       ret = device_reset(&pdev->dev);
> +       if (ret) {
> +               dev_err_probe(&pdev->dev, ret, "failed to reset controller");
> +               return ret;

If using old device trees this will fail, right? If yes, you should take
care this code will also work with old device trees.

Thank you,
Claudiu Beznea

> +       }
> +
> +       if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII) {
> +               /* Ensure PS-GTR PHY device used in SGMII mode is ready */
> +               struct phy *sgmii_phy = devm_phy_get(&pdev->dev, "sgmii-phy");
> +
> +               if (IS_ERR(sgmii_phy)) {
> +                       ret = PTR_ERR(sgmii_phy);
> +                       dev_err_probe(&pdev->dev, ret,
> +                                     "failed to get PS-GTR PHY\n");
> +                       return ret;
> +               }
> +
> +               ret = phy_init(sgmii_phy);
> +               if (ret) {
> +                       dev_err(&pdev->dev, "failed to init PS-GTR PHY: %d\n",
> +                               ret);
> +                       return ret;
> +               }
> +
> +               ret = phy_power_on(sgmii_phy);
> +               if (ret) {
> +                       dev_err(&pdev->dev, "failed to power on PS-GTR PHY: %d\n",
> +                               ret);
> +                       return ret;
> +               }
> +       }
> +       return macb_init(pdev);
> +}
> +
>  static const struct macb_usrio_config sama7g5_usrio = {
>         .mii = 0,
>         .rmii = 1,
> @@ -4550,7 +4595,7 @@ static const struct macb_config zynqmp_config = {
>                         MACB_CAPS_GEM_HAS_PTP | MACB_CAPS_BD_RD_PREFETCH,
>         .dma_burst_length = 16,
>         .clk_init = macb_clk_init,
> -       .init = macb_init,
> +       .init = zynqmp_init,
>         .jumbo_max_len = 10240,
>         .usrio = &macb_default_usrio,
>  };
> --
> 2.31.1
> 

Re: [PATCH net-next 2/3] net: macb: Added ZynqMP-specific initialization

[Robert Hancock]

On Thu, 2022-01-13 at 08:04 +0000, Claudiu.Beznea@microchip.com wrote:
> On 12.01.2022 20:11, Robert Hancock wrote:
> > EXTERNAL EMAIL: Do not click links or open attachments unless you know the
> > content is safe
> > 
> > The GEM controllers on ZynqMP were missing some initialization steps which
> > are required in some cases when using SGMII mode, which uses the PS-GTR
> > transceivers managed by the phy-zynqmp driver.
> > 
> > The GEM core appears to need a hardware-level reset in order to work
> > properly in SGMII mode in cases where the GT reference clock was not
> > present at initial power-on. This can be done using a reset mapped to
> > the zynqmp-reset driver in the device tree.
> > 
> > Also, when in SGMII mode, the GEM driver needs to ensure the PHY is
> > initialized and powered on when it is initializing.
> > 
> > Signed-off-by: Robert Hancock <robert.hancock@calian.com>
> > ---
> >  drivers/net/ethernet/cadence/macb_main.c | 47 +++++++++++++++++++++++-
> >  1 file changed, 46 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/net/ethernet/cadence/macb_main.c
> > b/drivers/net/ethernet/cadence/macb_main.c
> > index a363da928e8b..65b0360c487a 100644
> > --- a/drivers/net/ethernet/cadence/macb_main.c
> > +++ b/drivers/net/ethernet/cadence/macb_main.c
> > @@ -34,7 +34,9 @@
> >  #include <linux/udp.h>
> >  #include <linux/tcp.h>
> >  #include <linux/iopoll.h>
> > +#include <linux/phy/phy.h>
> >  #include <linux/pm_runtime.h>
> > +#include <linux/reset.h>
> >  #include "macb.h"
> > 
> >  /* This structure is only used for MACB on SiFive FU540 devices */
> > @@ -4455,6 +4457,49 @@ static int fu540_c000_init(struct platform_device
> > *pdev)
> >         return macb_init(pdev);
> >  }
> > 
> > +static int zynqmp_init(struct platform_device *pdev)
> > +{
> > +       struct net_device *dev = platform_get_drvdata(pdev);
> > +       struct macb *bp = netdev_priv(dev);
> > +       int ret;
> > +
> > +       /* Fully reset GEM controller at hardware level using zynqmp-reset
> > driver,
> > +        * if mapped in device tree.
> > +        */
> > +       ret = device_reset(&pdev->dev);
> > +       if (ret) {
> > +               dev_err_probe(&pdev->dev, ret, "failed to reset
> > controller");
> > +               return ret;
> 
> If using old device trees this will fail, right? If yes, you should take
> care this code will also work with old device trees.

I think I had believed device_reset should just return a success without doing
anything if the device tree has no reset defined for the device, but it appears
to get that behavior we should be using device_reset_optional.

> 
> Thank you,
> Claudiu Beznea
> 
> > +       }
> > +
> > +       if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII) {
> > +               /* Ensure PS-GTR PHY device used in SGMII mode is ready */
> > +               struct phy *sgmii_phy = devm_phy_get(&pdev->dev, "sgmii-
> > phy");
> > +
> > +               if (IS_ERR(sgmii_phy)) {
> > +                       ret = PTR_ERR(sgmii_phy);
> > +                       dev_err_probe(&pdev->dev, ret,
> > +                                     "failed to get PS-GTR PHY\n");
> > +                       return ret;
> > +               }
> > +
> > +               ret = phy_init(sgmii_phy);
> > +               if (ret) {
> > +                       dev_err(&pdev->dev, "failed to init PS-GTR PHY:
> > %d\n",
> > +                               ret);
> > +                       return ret;
> > +               }
> > +
> > +               ret = phy_power_on(sgmii_phy);
> > +               if (ret) {
> > +                       dev_err(&pdev->dev, "failed to power on PS-GTR PHY:
> > %d\n",
> > +                               ret);
> > +                       return ret;
> > +               }
> > +       }
> > +       return macb_init(pdev);
> > +}
> > +
> >  static const struct macb_usrio_config sama7g5_usrio = {
> >         .mii = 0,
> >         .rmii = 1,
> > @@ -4550,7 +4595,7 @@ static const struct macb_config zynqmp_config = {
> >                         MACB_CAPS_GEM_HAS_PTP | MACB_CAPS_BD_RD_PREFETCH,
> >         .dma_burst_length = 16,
> >         .clk_init = macb_clk_init,
> > -       .init = macb_init,
> > +       .init = zynqmp_init,
> >         .jumbo_max_len = 10240,
> >         .usrio = &macb_default_usrio,
> >  };
> > --
> > 2.31.1
> > 

[PATCH net-next 3/3] arm64: dts: zynqmp: Added GEM reset definitions

[Robert Hancock]

The Cadence GEM/MACB driver now utilizes the platform-level reset on the
ZynqMP platform. Add reset definitions to the ZynqMP platform device
tree to allow this to be used.

Signed-off-by: Robert Hancock <robert.hancock@calian.com>
---
 arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
index 74e66443e4ce..9bec3ba20c69 100644
--- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
+++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
@@ -512,6 +512,8 @@ gem0: ethernet@ff0b0000 {
 			#stream-id-cells = <1>;
 			iommus = <&smmu 0x874>;
 			power-domains = <&zynqmp_firmware PD_ETH_0>;
+			resets = <&zynqmp_reset ZYNQMP_RESET_GEM0>;
+			reset-names = "gem0_rst";
 		};
 
 		gem1: ethernet@ff0c0000 {
@@ -526,6 +528,8 @@ gem1: ethernet@ff0c0000 {
 			#stream-id-cells = <1>;
 			iommus = <&smmu 0x875>;
 			power-domains = <&zynqmp_firmware PD_ETH_1>;
+			resets = <&zynqmp_reset ZYNQMP_RESET_GEM1>;
+			reset-names = "gem1_rst";
 		};
 
 		gem2: ethernet@ff0d0000 {
@@ -540,6 +544,8 @@ gem2: ethernet@ff0d0000 {
 			#stream-id-cells = <1>;
 			iommus = <&smmu 0x876>;
 			power-domains = <&zynqmp_firmware PD_ETH_2>;
+			resets = <&zynqmp_reset ZYNQMP_RESET_GEM2>;
+			reset-names = "gem2_rst";
 		};
 
 		gem3: ethernet@ff0e0000 {
@@ -554,6 +560,8 @@ gem3: ethernet@ff0e0000 {
 			#stream-id-cells = <1>;
 			iommus = <&smmu 0x877>;
 			power-domains = <&zynqmp_firmware PD_ETH_3>;
+			resets = <&zynqmp_reset ZYNQMP_RESET_GEM3>;
+			reset-names = "gem3_rst";
 		};
 
 		gpio: gpio@ff0a0000 {
-- 
2.31.1