[RESEND PATCH V1 0/5] Update pcie-tegra194 driver

[RESEND PATCH V1 0/5] Update pcie-tegra194 driver

[Om Prakash Singh]

Update pcie-tegra194 driver with bug fixing and cleanup

Om Prakash Singh (5):
  PCI: tegra: Fix handling BME_CHGED event
  PCI: tegra: Fix MSI-X programming
  PCI: tegra: Disable interrupts before entering L2
  PCI: tegra: Don't allow suspend when Tegra PCIe is in EP mode
  PCI: tegra: Cleanup unused code

 drivers/pci/controller/dwc/pcie-tegra194.c | 36 +++++++++++++---------
 1 file changed, 22 insertions(+), 14 deletions(-)

-- 
2.17.1

[RESEND PATCH V1 1/5] PCI: tegra: Fix handling BME_CHGED event

[Om Prakash Singh]

In tegra_pcie_ep_hard_irq(), APPL_INTR_STATUS_L0 is stored in val and again
APPL_INTR_STATUS_L1_0_0 is also stored in val. So when execution reaches
"if (val & APPL_INTR_STATUS_L0_PCI_CMD_EN_INT)", val is not correct.

Signed-off-by: Om Prakash Singh <omp@nvidia.com>
---
 drivers/pci/controller/dwc/pcie-tegra194.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index bafd2c6ab3c2..c51d666c9d87 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -615,10 +615,10 @@ static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
 	struct tegra_pcie_dw *pcie = arg;
 	struct dw_pcie_ep *ep = &pcie->pci.ep;
 	int spurious = 1;
-	u32 val, tmp;
+	u32 val_l0, val, tmp;
 
-	val = appl_readl(pcie, APPL_INTR_STATUS_L0);
-	if (val & APPL_INTR_STATUS_L0_LINK_STATE_INT) {
+	val_l0 = appl_readl(pcie, APPL_INTR_STATUS_L0);
+	if (val_l0 & APPL_INTR_STATUS_L0_LINK_STATE_INT) {
 		val = appl_readl(pcie, APPL_INTR_STATUS_L1_0_0);
 		appl_writel(pcie, val, APPL_INTR_STATUS_L1_0_0);
 
@@ -636,7 +636,7 @@ static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
 		spurious = 0;
 	}
 
-	if (val & APPL_INTR_STATUS_L0_PCI_CMD_EN_INT) {
+	if (val_l0 & APPL_INTR_STATUS_L0_PCI_CMD_EN_INT) {
 		val = appl_readl(pcie, APPL_INTR_STATUS_L1_15);
 		appl_writel(pcie, val, APPL_INTR_STATUS_L1_15);
 
@@ -648,8 +648,8 @@ static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
 
 	if (spurious) {
 		dev_warn(pcie->dev, "Random interrupt (STATUS = 0x%08X)\n",
-			 val);
-		appl_writel(pcie, val, APPL_INTR_STATUS_L0);
+			 val_l0);
+		appl_writel(pcie, val_l0, APPL_INTR_STATUS_L0);
 	}
 
 	return IRQ_HANDLED;
-- 
2.17.1

Re: [RESEND PATCH V1 1/5] PCI: tegra: Fix handling BME_CHGED event

[Bjorn Helgaas]

On Thu, May 27, 2021 at 05:22:42PM +0530, Om Prakash Singh wrote:
> In tegra_pcie_ep_hard_irq(), APPL_INTR_STATUS_L0 is stored in val and again
> APPL_INTR_STATUS_L1_0_0 is also stored in val. So when execution reaches
> "if (val & APPL_INTR_STATUS_L0_PCI_CMD_EN_INT)", val is not correct.
> 
> Signed-off-by: Om Prakash Singh <omp@nvidia.com>
> ---
>  drivers/pci/controller/dwc/pcie-tegra194.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
> index bafd2c6ab3c2..c51d666c9d87 100644
> --- a/drivers/pci/controller/dwc/pcie-tegra194.c
> +++ b/drivers/pci/controller/dwc/pcie-tegra194.c
> @@ -615,10 +615,10 @@ static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
>  	struct tegra_pcie_dw *pcie = arg;
>  	struct dw_pcie_ep *ep = &pcie->pci.ep;
>  	int spurious = 1;
> -	u32 val, tmp;
> +	u32 val_l0, val, tmp;

Too bad this uses such bad variable names.  Names like "status_l0",
"status_l1", "link_status" would have avoided this in the first place.

"val" makes sense in places like config accessors where we're reading
or writing unspecified registers.  But when we're accessing specific
named registers?  Not so much.

[RESEND PATCH V1 2/5] PCI: tegra: Fix MSI-X programming

[Om Prakash Singh]

Lower order MSI-X address is programmed in MSIX_ADDR_MATCH_HIGH_OFF
DBI register instead of higher order address. This patch fixes this
programming mistake.

Signed-off-by: Om Prakash Singh <omp@nvidia.com>
---
 drivers/pci/controller/dwc/pcie-tegra194.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index c51d666c9d87..58fc2615014d 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -1863,7 +1863,7 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
 	val = (ep->msi_mem_phys & MSIX_ADDR_MATCH_LOW_OFF_MASK);
 	val |= MSIX_ADDR_MATCH_LOW_OFF_EN;
 	dw_pcie_writel_dbi(pci, MSIX_ADDR_MATCH_LOW_OFF, val);
-	val = (lower_32_bits(ep->msi_mem_phys) & MSIX_ADDR_MATCH_HIGH_OFF_MASK);
+	val = (upper_32_bits(ep->msi_mem_phys) & MSIX_ADDR_MATCH_HIGH_OFF_MASK);
 	dw_pcie_writel_dbi(pci, MSIX_ADDR_MATCH_HIGH_OFF, val);
 
 	ret = dw_pcie_ep_init_complete(ep);
-- 
2.17.1

Re: [RESEND PATCH V1 2/5] PCI: tegra: Fix MSI-X programming

[Krzysztof Wilczyński]

Hi Prakash,

[...]
> @@ -1863,7 +1863,7 @@ static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
>  	val = (ep->msi_mem_phys & MSIX_ADDR_MATCH_LOW_OFF_MASK);
>  	val |= MSIX_ADDR_MATCH_LOW_OFF_EN;
>  	dw_pcie_writel_dbi(pci, MSIX_ADDR_MATCH_LOW_OFF, val);
> -	val = (lower_32_bits(ep->msi_mem_phys) & MSIX_ADDR_MATCH_HIGH_OFF_MASK);
> +	val = (upper_32_bits(ep->msi_mem_phys) & MSIX_ADDR_MATCH_HIGH_OFF_MASK);

Oh!  Nice catch!

	Krzysztof

[RESEND PATCH V1 3/5] PCI: tegra: Disable interrupts before entering L2

[Om Prakash Singh]

Tegra194 implements suspend_noirq() hook and during the system suspend, the link
is taken to L2 state after PME_Turn_off handshake and if it doesn't go into L2,
PERST# is asserted. It is observed that with some of the endpoints (Ex:- Marvell
SATA controller), the link doesn't go into L2 state and asserting PERST# results
in Surprise Link Down error and the corresponding AER interrupt is also raised.
Since the system is in noirq phase, this interrupt is not served. Both PME and
AER interrupts are served by the same wire interrupt in Tegra194, and since the
PCIe sub-system enables wake capability for PME interrupt, having a pending AER
interrupt is treated as PME wake interrupt by the system and prevents the system
going into the suspend state. To address this issue, the interrupts are disabled
before taking the link into L2 state as the interrupts are not expected anyway
from the controller afterward.

Signed-off-by: Om Prakash Singh <omp@nvidia.com>
---
 drivers/pci/controller/dwc/pcie-tegra194.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index 58fc2615014d..ae62fdc840e6 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -1593,6 +1593,16 @@ static void tegra_pcie_dw_pme_turnoff(struct tegra_pcie_dw *pcie)
 		return;
 	}
 
+	/*
+	 * PCIe controller exits from L2 only if reset is applied, so
+	 * controller doesn't handle interrupts. But in cases where
+	 * L2 entry fails, PERST# is asserted which can trigger surprise
+	 * link down AER. However this function call happens in
+	 * suspend_noirq(), so AER interrupt will not be processed.
+	 * Disable all interrupts to avoid such a scenario.
+	 */
+	appl_writel(pcie, 0x0, APPL_INTR_EN_L0_0);
+
 	if (tegra_pcie_try_link_l2(pcie)) {
 		dev_info(pcie->dev, "Link didn't transition to L2 state\n");
 		/*
-- 
2.17.1

Re: [RESEND PATCH V1 3/5] PCI: tegra: Disable interrupts before entering L2

[Krzysztof Wilczyński]

Hi Prakash,

> Tegra194 implements suspend_noirq() hook and during the system suspend, the link
> is taken to L2 state after PME_Turn_off handshake and if it doesn't go into L2,
> PERST# is asserted. It is observed that with some of the endpoints (Ex:- Marvell
> SATA controller), the link doesn't go into L2 state and asserting PERST# results
> in Surprise Link Down error and the corresponding AER interrupt is also raised.
> Since the system is in noirq phase, this interrupt is not served. Both PME and
> AER interrupts are served by the same wire interrupt in Tegra194, and since the
> PCIe sub-system enables wake capability for PME interrupt, having a pending AER
> interrupt is treated as PME wake interrupt by the system and prevents the system
> going into the suspend state. To address this issue, the interrupts are disabled
> before taking the link into L2 state as the interrupts are not expected anyway
> from the controller afterward.

This commit could use some formatting, perhaps splitting it into few
paragraphs and also I believe your line length could use some
adjustment.  Having said that, I am not sure if the whole detailed
account of the problem is required to be included here in the commit
message.

> +	/*
> +	 * PCIe controller exits from L2 only if reset is applied, so
> +	 * controller doesn't handle interrupts. But in cases where
> +	 * L2 entry fails, PERST# is asserted which can trigger surprise
> +	 * link down AER. However this function call happens in
> +	 * suspend_noirq(), so AER interrupt will not be processed.
> +	 * Disable all interrupts to avoid such a scenario.
> +	 */
[...]

This code comment added below makes for a better commit message that the
commit message above - clear, concise and straight to the point of why
this was added.

	Krzysztof

[RESEND PATCH V1 4/5] PCI: tegra: Don't allow suspend when Tegra PCIe is in EP mode

[Om Prakash Singh]

When Tegra PCIe is in endpoint mode it should be available for root port.
PCIe link up by root port fails if it is in suspend state. So, don't allow
Tegra to suspend when endpoint mode is enabled.

Signed-off-by: Om Prakash Singh <omp@nvidia.com>
---
 drivers/pci/controller/dwc/pcie-tegra194.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index ae62fdc840e6..93c89f2084a7 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -2276,6 +2276,11 @@ static int tegra_pcie_dw_suspend_late(struct device *dev)
 	struct tegra_pcie_dw *pcie = dev_get_drvdata(dev);
 	u32 val;
 
+	if (pcie->mode == DW_PCIE_EP_TYPE) {
+		dev_err(dev, "Tegra PCIe is in EP mode, suspend not allowed");
+		return -EPERM;
+	}
+
 	if (!pcie->link_state)
 		return 0;
 
-- 
2.17.1

Re: [RESEND PATCH V1 4/5] PCI: tegra: Don't allow suspend when Tegra PCIe is in EP mode

[Krzysztof Wilczyński]

Hi Prakash

[...]
> @@ -2276,6 +2276,11 @@ static int tegra_pcie_dw_suspend_late(struct device *dev)
[...]
> +	if (pcie->mode == DW_PCIE_EP_TYPE) {
> +		dev_err(dev, "Tegra PCIe is in EP mode, suspend not allowed");
> +		return -EPERM;
> +	}

Would the -EINVAL be more appropriate here?  It seem more appropriate
when something is an invalid operation, rather than access to the
resource being denied (or something along these lines), what do you
think?

	Krzysztof

Re: [RESEND PATCH V1 4/5] PCI: tegra: Don't allow suspend when Tegra PCIe is in EP mode

[Krzysztof Wilczyński]

Hi,

> > +	if (pcie->mode == DW_PCIE_EP_TYPE) {
> > +		dev_err(dev, "Tegra PCIe is in EP mode, suspend not allowed");
> > +		return -EPERM;
> > +	}
> 
> Would the -EINVAL be more appropriate here?  It seem more appropriate
> when something is an invalid operation, rather than access to the
> resource being denied (or something along these lines), what do you
> think?

After looking at this again, perhaps -ENOTSUPP would be a better fit,
and then the message could say that the suspend is not supported.

	Krzysztof

[RESEND PATCH V1 5/5] PCI: tegra: Cleanup unused code

[Om Prakash Singh]

Remove unused code from function tegra_pcie_config_ep.

Signed-off-by: Om Prakash Singh <omp@nvidia.com>
---
 drivers/pci/controller/dwc/pcie-tegra194.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/drivers/pci/controller/dwc/pcie-tegra194.c b/drivers/pci/controller/dwc/pcie-tegra194.c
index 93c89f2084a7..096aa5306fda 100644
--- a/drivers/pci/controller/dwc/pcie-tegra194.c
+++ b/drivers/pci/controller/dwc/pcie-tegra194.c
@@ -2045,13 +2045,6 @@ static int tegra_pcie_config_ep(struct tegra_pcie_dw *pcie,
 		return ret;
 	}
 
-	name = devm_kasprintf(dev, GFP_KERNEL, "tegra_pcie_%u_ep_work",
-			      pcie->cid);
-	if (!name) {
-		dev_err(dev, "Failed to create PCIe EP work thread string\n");
-		return -ENOMEM;
-	}
-
 	pm_runtime_enable(dev);
 
 	ret = dw_pcie_ep_init(ep);
-- 
2.17.1

Re: [RESEND PATCH V1 0/5] Update pcie-tegra194 driver

[Krzysztof Wilczyński]

Hi Prakash,

Thank you for sending the patches over!

> Update pcie-tegra194 driver with bug fixing and cleanup
> 
> Om Prakash Singh (5):
>   PCI: tegra: Fix handling BME_CHGED event
>   PCI: tegra: Fix MSI-X programming
>   PCI: tegra: Disable interrupts before entering L2
>   PCI: tegra: Don't allow suspend when Tegra PCIe is in EP mode
>   PCI: tegra: Cleanup unused code
> 
>  drivers/pci/controller/dwc/pcie-tegra194.c | 36 +++++++++++++---------
>  1 file changed, 22 insertions(+), 14 deletions(-)

Why the resend?  I saw you send this series before, and now you are
resending it?  Help me understand what is going on here.

	Krzysztof

Re: [RESEND PATCH V1 0/5] Update pcie-tegra194 driver

[Jon Hunter]

On 27/05/2021 13:00, Krzysztof Wilczyński wrote:
> Hi Prakash,
> 
> Thank you for sending the patches over!
> 
>> Update pcie-tegra194 driver with bug fixing and cleanup
>>
>> Om Prakash Singh (5):
>>   PCI: tegra: Fix handling BME_CHGED event
>>   PCI: tegra: Fix MSI-X programming
>>   PCI: tegra: Disable interrupts before entering L2
>>   PCI: tegra: Don't allow suspend when Tegra PCIe is in EP mode
>>   PCI: tegra: Cleanup unused code
>>
>>  drivers/pci/controller/dwc/pcie-tegra194.c | 36 +++++++++++++---------
>>  1 file changed, 22 insertions(+), 14 deletions(-)
> 
> Why the resend?  I saw you send this series before, and now you are
> resending it?  Help me understand what is going on here.


I suggested resending it because the initial version had two cover
letters and two copies of the same patch. It took me a minute to figure
out what was what. So just ignore the first series.

Jon

-- 
nvpublic

Re: [RESEND PATCH V1 0/5] Update pcie-tegra194 driver

[Krzysztof Wilczyński]

Hi Jon,

[...]
> I suggested resending it because the initial version had two cover
> letters and two copies of the same patch. It took me a minute to figure
> out what was what. So just ignore the first series.

Ah, OK.  No worries then.

	Krzysztof