[PATCH v1] PCI: imx6: Fix PERST# start-up sequence

[PATCH v1] PCI: imx6: Fix PERST# start-up sequence

[Francesco Dolcini]

According to the PCI-E standard the PERST# signal (reset-gpio in
fsl,imx6* compatible dts) should be kept asserted for at least 100 usec
before the PCI-E refclock is stable, should be kept asserted for at
least 100ms after the power rails are stable and the host should wait
at least 100 msec after it is de-asserted before accessing the
configuration space of any attached device.

From PCI Express Card Electromechanical Specification

  T-PVPERL: Power stable to PERST# inactive - 100 msec
  T-PERST-CLK: REFCLK stable before PERST# inactive 100 usec.

From PCI Express Base Specification:

  To allow components to perform internal initialization, system
  software must wait for at least 100 ms from the end of a Conventional
  Reset of one or more devices before it is permitted to issue
  Configuration Requests to those devices

Failure to do so could prevent PCI-E devices to be working correctly,
and this was experienced with real devices.

Move reset assert to imx6_pcie_assert_core_reset(), this way we ensure
that PERST# is asserted before enabling any clock, move de-assert to the
end of imx6_pcie_deassert_core_reset() after the clock is enabled and
deemed stable and add a new delay of 100 msec just afterward.

Fixes: bb38919ec56e ("PCI: imx6: Add support for i.MX6 PCIe controller")
Signed-off-by: Francesco Dolcini <francesco.dolcini@toradex.com>
---
 drivers/pci/controller/dwc/pci-imx6.c | 27 ++++++++++++++++++---------
 1 file changed, 18 insertions(+), 9 deletions(-)

diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
index 7b200b66114a..392803544364 100644
--- a/drivers/pci/controller/dwc/pci-imx6.c
+++ b/drivers/pci/controller/dwc/pci-imx6.c
@@ -408,6 +408,11 @@ static void imx6_pcie_assert_core_reset(struct imx6_pcie *imx6_pcie)
 			dev_err(dev, "failed to disable vpcie regulator: %d\n",
 				ret);
 	}
+
+	/* Some boards don't have PCIe reset GPIO. */
+	if (gpio_is_valid(imx6_pcie->reset_gpio))
+		gpio_set_value_cansleep(imx6_pcie->reset_gpio,
+					imx6_pcie->gpio_active_high);
 }
 
 static unsigned int imx6_pcie_grp_offset(const struct imx6_pcie *imx6_pcie)
@@ -544,15 +549,6 @@ static void imx6_pcie_deassert_core_reset(struct imx6_pcie *imx6_pcie)
 	/* allow the clocks to stabilize */
 	usleep_range(200, 500);
 
-	/* Some boards don't have PCIe reset GPIO. */
-	if (gpio_is_valid(imx6_pcie->reset_gpio)) {
-		gpio_set_value_cansleep(imx6_pcie->reset_gpio,
-					imx6_pcie->gpio_active_high);
-		msleep(100);
-		gpio_set_value_cansleep(imx6_pcie->reset_gpio,
-					!imx6_pcie->gpio_active_high);
-	}
-
 	switch (imx6_pcie->drvdata->variant) {
 	case IMX8MQ:
 		reset_control_deassert(imx6_pcie->pciephy_reset);
@@ -599,6 +595,19 @@ static void imx6_pcie_deassert_core_reset(struct imx6_pcie *imx6_pcie)
 		break;
 	}
 
+	/* Some boards don't have PCIe reset GPIO. */
+	if (gpio_is_valid(imx6_pcie->reset_gpio)) {
+		msleep(100);
+		gpio_set_value_cansleep(imx6_pcie->reset_gpio,
+					!imx6_pcie->gpio_active_high);
+		/*
+		 * PCI Express Base Specification:
+		 *   A delay of at least 100ms is required after PERST# is
+		 *   de-asserted before issuing any Configuration Requests
+		 */
+		msleep(100);
+	}
+
 	return;
 
 err_ref_clk:
-- 
2.25.1

Re: [PATCH v1] PCI: imx6: Fix PERST# start-up sequence

[Bjorn Helgaas]

On Fri, Feb 11, 2022 at 04:25:50PM +0100, Francesco Dolcini wrote:
> According to the PCI-E standard the PERST# signal (reset-gpio in

s/PCI-E/PCIe/ everywhere, to match conventional usage.

> fsl,imx6* compatible dts) should be kept asserted for at least 100 usec
> before the PCI-E refclock is stable, should be kept asserted for at
> least 100ms after the power rails are stable and the host should wait
> at least 100 msec after it is de-asserted before accessing the
> configuration space of any attached device.
> 
> From PCI Express Card Electromechanical Specification

Please include spec revision and section number.  I think this would
probably be "PCIe CEM r5.0, sec 2.9.2".

>   T-PVPERL: Power stable to PERST# inactive - 100 msec
>   T-PERST-CLK: REFCLK stable before PERST# inactive 100 usec.
> 
> From PCI Express Base Specification:

Similarly, "PCIe r6.0, sec xxx".  I don't see the text below in r6.0,
so I think it must have evolved a bit.  Probably something in sec
6.6.1.

>   To allow components to perform internal initialization, system
>   software must wait for at least 100 ms from the end of a Conventional
>   Reset of one or more devices before it is permitted to issue
>   Configuration Requests to those devices
> 
> Failure to do so could prevent PCI-E devices to be working correctly,
> and this was experienced with real devices.
> 
> Move reset assert to imx6_pcie_assert_core_reset(), this way we ensure
> that PERST# is asserted before enabling any clock, move de-assert to the
> end of imx6_pcie_deassert_core_reset() after the clock is enabled and
> deemed stable and add a new delay of 100 msec just afterward.
> 
> Fixes: bb38919ec56e ("PCI: imx6: Add support for i.MX6 PCIe controller")
> Signed-off-by: Francesco Dolcini <francesco.dolcini@toradex.com>
> ---
>  drivers/pci/controller/dwc/pci-imx6.c | 27 ++++++++++++++++++---------
>  1 file changed, 18 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c
> index 7b200b66114a..392803544364 100644
> --- a/drivers/pci/controller/dwc/pci-imx6.c
> +++ b/drivers/pci/controller/dwc/pci-imx6.c
> @@ -408,6 +408,11 @@ static void imx6_pcie_assert_core_reset(struct imx6_pcie *imx6_pcie)
>  			dev_err(dev, "failed to disable vpcie regulator: %d\n",
>  				ret);
>  	}
> +
> +	/* Some boards don't have PCIe reset GPIO. */
> +	if (gpio_is_valid(imx6_pcie->reset_gpio))
> +		gpio_set_value_cansleep(imx6_pcie->reset_gpio,
> +					imx6_pcie->gpio_active_high);
>  }
>  
>  static unsigned int imx6_pcie_grp_offset(const struct imx6_pcie *imx6_pcie)
> @@ -544,15 +549,6 @@ static void imx6_pcie_deassert_core_reset(struct imx6_pcie *imx6_pcie)
>  	/* allow the clocks to stabilize */
>  	usleep_range(200, 500);
>  
> -	/* Some boards don't have PCIe reset GPIO. */
> -	if (gpio_is_valid(imx6_pcie->reset_gpio)) {
> -		gpio_set_value_cansleep(imx6_pcie->reset_gpio,
> -					imx6_pcie->gpio_active_high);
> -		msleep(100);
> -		gpio_set_value_cansleep(imx6_pcie->reset_gpio,
> -					!imx6_pcie->gpio_active_high);
> -	}
> -
>  	switch (imx6_pcie->drvdata->variant) {
>  	case IMX8MQ:
>  		reset_control_deassert(imx6_pcie->pciephy_reset);
> @@ -599,6 +595,19 @@ static void imx6_pcie_deassert_core_reset(struct imx6_pcie *imx6_pcie)
>  		break;
>  	}
>  
> +	/* Some boards don't have PCIe reset GPIO. */
> +	if (gpio_is_valid(imx6_pcie->reset_gpio)) {
> +		msleep(100);
> +		gpio_set_value_cansleep(imx6_pcie->reset_gpio,
> +					!imx6_pcie->gpio_active_high);
> +		/*
> +		 * PCI Express Base Specification:

Mention spec rev & section here, too.  Someday we'll consolidate these
with #defines and identifying all the delays related to sec 6.6.1 (or
whatever it is) will be important.

> +		 *   A delay of at least 100ms is required after PERST# is
> +		 *   de-asserted before issuing any Configuration Requests
> +		 */
> +		msleep(100);
> +	}
> +
>  	return;
>  
>  err_ref_clk:
> -- 
> 2.25.1
> 

Re: [PATCH v1] PCI: imx6: Fix PERST# start-up sequence

[Francesco Dolcini]

Hello Bjorn,
thanks for the review.

On Fri, Feb 11, 2022 at 12:18:24PM -0600, Bjorn Helgaas wrote:
> Mention spec rev & section here, too.  Someday we'll consolidate these
> with #defines and identifying all the delays related to sec 6.6.1 (or
> whatever it is) will be important.
I added all the references and will send the v2 later today, however I
do not have access to the PCI-SIG specifications anymore, therefore I'll
mention the one I currently have access to (I did not copy the sections
you mentioned just because you wrote "probably" and I had no way to
double check).

I guess that this should not be a big deal, since we already have
different version of the standard mentioned thought the code with
reference to this topic.

Francesco