[PATCH V5 0/5] Enhancements to Tegra194 PCIe driver

[PATCH V5 0/5] Enhancements to Tegra194 PCIe driver

[Vidya Sagar]

This series of patches do some enhancements and some bug fixes to the
Tegra194 PCIe platform driver like
- Fix Vendor-ID corruption
- Update DWC IP version
- Continue with uninitialization sequence even if parts fail
- Check return value of tegra_pcie_init_controller()
- Disable LTSSM during link's L2 entry

V5:
* Rebased the first patch in the series
* Dropped the second patch
* Added Tested-by and Acked-by for rest of the patches

V4:
* Added a new patch to address link-up issues with some of the cards

V3:
* Addressed Bjorn's review comments
* Split earlier patch-4 into two
  - Continue with the uninitialization sequence even if some parts fail
  - Check return value of tegra_pcie_init_controller() and exit accordingly

V2:
* Addressed Rob's comments. Changed 'Strongly Ordered' to 'nGnRnE'

Vidya Sagar (5):
  PCI: tegra: Fix ASPM-L1SS advertisement disable code
  PCI: tegra: Set DesignWare IP version
  PCI: tegra: Continue unconfig sequence even if parts fail
  PCI: tegra: Check return value of tegra_pcie_init_controller()
  PCI: tegra: Disable LTSSM during L2 entry

 drivers/pci/controller/dwc/pcie-tegra194.c | 74 +++++++++++-----------
 1 file changed, 36 insertions(+), 38 deletions(-)

-- 
2.17.1

[PATCH V5 1/5] PCI: tegra: Fix ASPM-L1SS advertisement disable code

[Vidya Sagar]

If the absence of CLKREQ# signal is indicated by the absence of
"supports-clkreq" in the device-tree node, current driver is disabling
the advertisement of ASPM-L1 Sub-States *before* the ASPM-L1 Sub-States
offset is correctly initialized. Since default value of the ASPM-L1SS
offset is zero, this is causing the Vendor-ID wrongly programmed to 0x10d2
instead of Nvidia's 0x10de thereby the quirks applicable for Tegra194 are
not being applied. This patch fixes this issue by refactoring the
code that disables the ASPM-L1SS advertisement.

Fixes: 56e15a238d92 ("PCI: tegra: Add Tegra194 PCIe support")
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
---
V5:
* Rebased on top of the tree code

V4:
* None

V3:
* None

V2:
* None

 drivers/pci/controller/dwc/pcie-tegra194.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index 648e731bccfa..4c966e9adb2b 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -863,12 +863,6 @@ static void tegra_pcie_prepare_host(struct pcie_port *pp)
 		pcie->pcie_cap_base = dw_pcie_find_capability(&pcie->pci,
 							      PCI_CAP_ID_EXP);
 
-	/* Disable ASPM-L1SS advertisement if there is no CLKREQ routing */
-	if (!pcie->supports_clkreq) {
-		disable_aspm_l11(pcie);
-		disable_aspm_l12(pcie);
-	}
-
 	val = dw_pcie_readl_dbi(pci, PCI_IO_BASE);
 	val &= ~(IO_BASE_IO_DECODE | IO_BASE_IO_DECODE_BIT8);
 	dw_pcie_writel_dbi(pci, PCI_IO_BASE, val);
@@ -897,6 +891,12 @@ static void tegra_pcie_prepare_host(struct pcie_port *pp)
 
 	init_host_aspm(pcie);
 
+	/* Disable ASPM-L1SS advertisement if there is no CLKREQ routing */
+	if (!pcie->supports_clkreq) {
+		disable_aspm_l11(pcie);
+		disable_aspm_l12(pcie);
+	}
+
 	val = dw_pcie_readl_dbi(pci, GEN3_RELATED_OFF);
 	val &= ~GEN3_RELATED_OFF_GEN3_ZRXDC_NONCOMPL;
 	dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF, val);
-- 
2.17.1

[PATCH V5 2/5] PCI: tegra: Set DesignWare IP version

[Vidya Sagar]

Set the DesignWare IP version for Tegra194 to 0x490A. This would be used
by the DesigWare sub-system to do any version specific configuration
(Ex:- TD bit programming for ECRC).

Tested-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V5:
* Added Tested-by and Acked-by from Thierry Reding

V4:
* None

V3:
* None

V2:
* None

 drivers/pci/controller/dwc/pcie-tegra194.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index 4c966e9adb2b..59163b735c96 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -1984,6 +1984,7 @@ static int tegra_pcie_dw_probe(struct platform_device *pdev)
 	pci->ops = &tegra_dw_pcie_ops;
 	pci->n_fts[0] = N_FTS_VAL;
 	pci->n_fts[1] = FTS_VAL;
+	pci->version = 0x490A;
 
 	pp = &pci->pp;
 	pp->num_vectors = MAX_MSI_IRQS;
-- 
2.17.1

[PATCH V5 3/5] PCI: tegra: Continue unconfig sequence even if parts fail

[Vidya Sagar]

Currently the driver checks for error value of different APIs during the
uninitialization sequence. It just returns from there if there is any error
observed for one of those calls. Comparatively it is better to continue the
uninitialization sequence irrespective of whether some of them are
returning error. That way, it is more closer to complete uninitialization.

Tested-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V5:
* Added Tested-by and Acked-by from Thierry Reding

V4:
* None

V3:
* Modified subject as per Bjorn's suggestion
* Removed tegra_pcie_init_controller()'s error checking part and pushed
  a separate patch for it

V2:
* None

 drivers/pci/controller/dwc/pcie-tegra194.c | 39 +++++++++-------------
 1 file changed, 15 insertions(+), 24 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index 59163b735c96..471c6d725c70 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -1415,43 +1415,32 @@ static int tegra_pcie_config_controller(struct tegra_pcie_dw *pcie,
 	return ret;
 }
 
-static int __deinit_controller(struct tegra_pcie_dw *pcie)
+static void tegra_pcie_unconfig_controller(struct tegra_pcie_dw *pcie)
 {
 	int ret;
 
 	ret = reset_control_assert(pcie->core_rst);
-	if (ret) {
-		dev_err(pcie->dev, "Failed to assert \"core\" reset: %d\n",
-			ret);
-		return ret;
-	}
+	if (ret)
+		dev_err(pcie->dev, "Failed to assert \"core\" reset: %d\n", ret);
 
 	tegra_pcie_disable_phy(pcie);
 
 	ret = reset_control_assert(pcie->core_apb_rst);
-	if (ret) {
+	if (ret)
 		dev_err(pcie->dev, "Failed to assert APB reset: %d\n", ret);
-		return ret;
-	}
 
 	clk_disable_unprepare(pcie->core_clk);
 
 	ret = regulator_disable(pcie->pex_ctl_supply);
-	if (ret) {
+	if (ret)
 		dev_err(pcie->dev, "Failed to disable regulator: %d\n", ret);
-		return ret;
-	}
 
 	tegra_pcie_disable_slot_regulators(pcie);
 
 	ret = tegra_pcie_bpmp_set_ctrl_state(pcie, false);
-	if (ret) {
+	if (ret)
 		dev_err(pcie->dev, "Failed to disable controller %d: %d\n",
 			pcie->cid, ret);
-		return ret;
-	}
-
-	return ret;
 }
 
 static int tegra_pcie_init_controller(struct tegra_pcie_dw *pcie)
@@ -1475,7 +1464,8 @@ static int tegra_pcie_init_controller(struct tegra_pcie_dw *pcie)
 	return 0;
 
 fail_host_init:
-	return __deinit_controller(pcie);
+	tegra_pcie_unconfig_controller(pcie);
+	return ret;
 }
 
 static int tegra_pcie_try_link_l2(struct tegra_pcie_dw *pcie)
@@ -1544,13 +1534,12 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
 	appl_writel(pcie, data, APPL_PINMUX);
 }
 
-static int tegra_pcie_deinit_controller(struct tegra_pcie_dw *pcie)
+static void tegra_pcie_deinit_controller(struct tegra_pcie_dw *pcie)
 {
 	tegra_pcie_downstream_dev_to_D0(pcie);
 	dw_pcie_host_deinit(&pcie->pci.pp);
 	tegra_pcie_dw_pme_turnoff(pcie);
-
-	return __deinit_controller(pcie);
+	tegra_pcie_unconfig_controller(pcie);
 }
 
 static int tegra_pcie_config_rp(struct tegra_pcie_dw *pcie)
@@ -2197,8 +2186,9 @@ static int tegra_pcie_dw_suspend_noirq(struct device *dev)
 					       PORT_LOGIC_MSI_CTRL_INT_0_EN);
 	tegra_pcie_downstream_dev_to_D0(pcie);
 	tegra_pcie_dw_pme_turnoff(pcie);
+	tegra_pcie_unconfig_controller(pcie);
 
-	return __deinit_controller(pcie);
+	return 0;
 }
 
 static int tegra_pcie_dw_resume_noirq(struct device *dev)
@@ -2226,7 +2216,8 @@ static int tegra_pcie_dw_resume_noirq(struct device *dev)
 	return 0;
 
 fail_host_init:
-	return __deinit_controller(pcie);
+	tegra_pcie_unconfig_controller(pcie);
+	return ret;
 }
 
 static int tegra_pcie_dw_resume_early(struct device *dev)
@@ -2264,7 +2255,7 @@ static void tegra_pcie_dw_shutdown(struct platform_device *pdev)
 		disable_irq(pcie->pci.pp.msi_irq);
 
 	tegra_pcie_dw_pme_turnoff(pcie);
-	__deinit_controller(pcie);
+	tegra_pcie_unconfig_controller(pcie);
 }
 
 static const struct tegra_pcie_dw_of_data tegra_pcie_dw_rc_of_data = {
-- 
2.17.1

[PATCH V5 4/5] PCI: tegra: Check return value of tegra_pcie_init_controller()

[Vidya Sagar]

The return value of tegra_pcie_init_controller() must be checked before
PCIe link up check and registering debugfs entries subsequently as it
doesn't make sense to do these when the controller initialization itself
has failed.

Tested-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V5:
* Added Tested-by and Acked-by from Thierry Reding

V4:
* None

V3:
* New patch in this series

 drivers/pci/controller/dwc/pcie-tegra194.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index 471c6d725c70..f4109d71f20b 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -1563,7 +1563,11 @@ static int tegra_pcie_config_rp(struct tegra_pcie_dw *pcie)
 		goto fail_pm_get_sync;
 	}
 
-	tegra_pcie_init_controller(pcie);
+	ret = tegra_pcie_init_controller(pcie);
+	if (ret < 0) {
+		dev_err(dev, "Failed to initialize controller: %d\n", ret);
+		goto fail_pm_get_sync;
+	}
 
 	pcie->link_state = tegra_pcie_dw_link_up(&pcie->pci);
 	if (!pcie->link_state) {
-- 
2.17.1

[PATCH V5 5/5] PCI: tegra: Disable LTSSM during L2 entry

[Vidya Sagar]

PCIe cards like Marvell SATA controller and some of the Samsung NVMe
drives don't support taking the link to L2 state. When the link doesn't
go to L2 state, Tegra194 requires the LTSSM to be disabled to allow PHY
to start the next link up process cleanly during suspend/resume sequence.
Failing to disable LTSSM results in the PCIe link not coming up in the
next resume cycle.

Tested-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Acked-by: Thierry Reding <treding@nvidia.com>
---
V5:
* Added Tested-by and Acked-by from Thierry Reding

V4:
* New patch in this series

 drivers/pci/controller/dwc/pcie-tegra194.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index f4109d71f20b..5597b2a49598 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -1506,6 +1506,14 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
 		data &= ~APPL_PINMUX_PEX_RST;
 		appl_writel(pcie, data, APPL_PINMUX);
 
+		/*
+		 * Some cards do not go to detect state even after de-asserting
+		 * PERST#. So, de-assert LTSSM to bring link to detect state.
+		 */
+		data = readl(pcie->appl_base + APPL_CTRL);
+		data &= ~APPL_CTRL_LTSSM_EN;
+		writel(data, pcie->appl_base + APPL_CTRL);
+
 		err = readl_poll_timeout_atomic(pcie->appl_base + APPL_DEBUG,
 						data,
 						((data &
@@ -1513,14 +1521,8 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
 						APPL_DEBUG_LTSSM_STATE_SHIFT) ==
 						LTSSM_STATE_PRE_DETECT,
 						1, LTSSM_TIMEOUT);
-		if (err) {
+		if (err)
 			dev_info(pcie->dev, "Link didn't go to detect state\n");
-		} else {
-			/* Disable LTSSM after link is in detect state */
-			data = appl_readl(pcie, APPL_CTRL);
-			data &= ~APPL_CTRL_LTSSM_EN;
-			appl_writel(pcie, data, APPL_CTRL);
-		}
 	}
 	/*
 	 * DBI registers may not be accessible after this as PLL-E would be
-- 
2.17.1

Re: [PATCH V5 0/5] Enhancements to Tegra194 PCIe driver

[Lorenzo Pieralisi]

On Thu, 3 Dec 2020 19:04:46 +0530, Vidya Sagar wrote:
> This series of patches do some enhancements and some bug fixes to the
> Tegra194 PCIe platform driver like
> - Fix Vendor-ID corruption
> - Update DWC IP version
> - Continue with uninitialization sequence even if parts fail
> - Check return value of tegra_pcie_init_controller()
> - Disable LTSSM during link's L2 entry
> 
> [...]

Applied to pci/dwc, thanks!

[1/5] PCI: tegra: Fix ASPM-L1SS advertisement disable code
      https://git.kernel.org/lpieralisi/pci/c/6b6fafc1ab
[2/5] PCI: tegra: Set DesignWare IP version
      https://git.kernel.org/lpieralisi/pci/c/01254b6d6b
[3/5] PCI: tegra: Continue unconfig sequence even if parts fail
      https://git.kernel.org/lpieralisi/pci/c/b8f0d67149
[4/5] PCI: tegra: Check return value of tegra_pcie_init_controller()
      https://git.kernel.org/lpieralisi/pci/c/3d710af75b
[5/5] PCI: tegra: Disable LTSSM during L2 entry
      https://git.kernel.org/lpieralisi/pci/c/cf68e3b7a6

Thanks,
Lorenzo

Re: [PATCH V5 5/5] PCI: tegra: Disable LTSSM during L2 entry

[Bjorn Helgaas]

[+cc Jingoo, Gustavo]

On Thu, Dec 03, 2020 at 07:04:51PM +0530, Vidya Sagar wrote:
> PCIe cards like Marvell SATA controller and some of the Samsung NVMe
> drives don't support taking the link to L2 state. When the link doesn't
> go to L2 state, Tegra194 requires the LTSSM to be disabled to allow PHY
> to start the next link up process cleanly during suspend/resume sequence.
> Failing to disable LTSSM results in the PCIe link not coming up in the
> next resume cycle.

Is this a Tegra194-specific issue, or will other DWC-based controllers
need a similar change?

> Tested-by: Thierry Reding <treding@nvidia.com>
> Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
> Acked-by: Thierry Reding <treding@nvidia.com>
> ---
> V5:
> * Added Tested-by and Acked-by from Thierry Reding
> 
> V4:
> * New patch in this series
> 
>  drivers/pci/controller/dwc/pcie-tegra194.c | 16 +++++++++-------
>  1 file changed, 9 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
> index f4109d71f20b..5597b2a49598 100644
> --- a/drivers/pci/controller/dwc/pcie-tegra194.c
> +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
> @@ -1506,6 +1506,14 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
>  		data &= ~APPL_PINMUX_PEX_RST;
>  		appl_writel(pcie, data, APPL_PINMUX);
>  
> +		/*
> +		 * Some cards do not go to detect state even after de-asserting
> +		 * PERST#. So, de-assert LTSSM to bring link to detect state.
> +		 */
> +		data = readl(pcie->appl_base + APPL_CTRL);
> +		data &= ~APPL_CTRL_LTSSM_EN;
> +		writel(data, pcie->appl_base + APPL_CTRL);
> +
>  		err = readl_poll_timeout_atomic(pcie->appl_base + APPL_DEBUG,
>  						data,
>  						((data &
> @@ -1513,14 +1521,8 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
>  						APPL_DEBUG_LTSSM_STATE_SHIFT) ==
>  						LTSSM_STATE_PRE_DETECT,
>  						1, LTSSM_TIMEOUT);
> -		if (err) {
> +		if (err)
>  			dev_info(pcie->dev, "Link didn't go to detect state\n");
> -		} else {
> -			/* Disable LTSSM after link is in detect state */
> -			data = appl_readl(pcie, APPL_CTRL);
> -			data &= ~APPL_CTRL_LTSSM_EN;
> -			appl_writel(pcie, data, APPL_CTRL);
> -		}
>  	}
>  	/*
>  	 * DBI registers may not be accessible after this as PLL-E would be
> -- 
> 2.17.1
> 

Re: [PATCH V5 5/5] PCI: tegra: Disable LTSSM during L2 entry

[Vidya Sagar]

On 12/8/2020 2:07 AM, Bjorn Helgaas wrote:
> External email: Use caution opening links or attachments
> 
> 
> [+cc Jingoo, Gustavo]
> 
> On Thu, Dec 03, 2020 at 07:04:51PM +0530, Vidya Sagar wrote:
>> PCIe cards like Marvell SATA controller and some of the Samsung NVMe
>> drives don't support taking the link to L2 state. When the link doesn't
>> go to L2 state, Tegra194 requires the LTSSM to be disabled to allow PHY
>> to start the next link up process cleanly during suspend/resume sequence.
>> Failing to disable LTSSM results in the PCIe link not coming up in the
>> next resume cycle.
> 
> Is this a Tegra194-specific issue, or will other DWC-based controllers
> need a similar change?
This is a Tegra194 specific issue.

Thanks,
Vidya Sagar
> 
>> Tested-by: Thierry Reding <treding@nvidia.com>
>> Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
>> Acked-by: Thierry Reding <treding@nvidia.com>
>> ---
>> V5:
>> * Added Tested-by and Acked-by from Thierry Reding
>>
>> V4:
>> * New patch in this series
>>
>>   drivers/pci/controller/dwc/pcie-tegra194.c | 16 +++++++++-------
>>   1 file changed, 9 insertions(+), 7 deletions(-)
>>
>> diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
>> index f4109d71f20b..5597b2a49598 100644
>> --- a/drivers/pci/controller/dwc/pcie-tegra194.c
>> +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
>> @@ -1506,6 +1506,14 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
>>                data &= ~APPL_PINMUX_PEX_RST;
>>                appl_writel(pcie, data, APPL_PINMUX);
>>
>> +             /*
>> +              * Some cards do not go to detect state even after de-asserting
>> +              * PERST#. So, de-assert LTSSM to bring link to detect state.
>> +              */
>> +             data = readl(pcie->appl_base + APPL_CTRL);
>> +             data &= ~APPL_CTRL_LTSSM_EN;
>> +             writel(data, pcie->appl_base + APPL_CTRL);
>> +
>>                err = readl_poll_timeout_atomic(pcie->appl_base + APPL_DEBUG,
>>                                                data,
>>                                                ((data &
>> @@ -1513,14 +1521,8 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
>>                                                APPL_DEBUG_LTSSM_STATE_SHIFT) ==
>>                                                LTSSM_STATE_PRE_DETECT,
>>                                                1, LTSSM_TIMEOUT);
>> -             if (err) {
>> +             if (err)
>>                        dev_info(pcie->dev, "Link didn't go to detect state\n");
>> -             } else {
>> -                     /* Disable LTSSM after link is in detect state */
>> -                     data = appl_readl(pcie, APPL_CTRL);
>> -                     data &= ~APPL_CTRL_LTSSM_EN;
>> -                     appl_writel(pcie, data, APPL_CTRL);
>> -             }
>>        }
>>        /*
>>         * DBI registers may not be accessible after this as PLL-E would be
>> --
>> 2.17.1
>>