RE: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators after link is down

[Hongxing Zhu <hongxing.zhu@nxp.com>]

> -----Original Message-----
> From: Lucas Stach <l.stach@pengutronix.de>
> Sent: 2022年6月8日 15:35
> To: Hongxing Zhu <hongxing.zhu@nxp.com>; bhelgaas@google.com;
> robh+dt@kernel.org; broonie@kernel.org; lorenzo.pieralisi@arm.com;
> jingoohan1@gmail.com; festevam@gmail.com;
> francesco.dolcini@toradex.com
> Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> linux-kernel@vger.kernel.org; kernel@pengutronix.de; dl-linux-imx
> <linux-imx@nxp.com>
> Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators after link is
> down
> 
> Am Freitag, dem 06.05.2022 um 09:47 +0800 schrieb Richard Zhu:
> > Since i.MX PCIe doesn't support hot-plug, reduce power consumption as
> > much as possible by disabling clocks and regulators and returning
> > error when the link is down.
> >
> > Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com>
> > ---
> >  drivers/pci/controller/dwc/pci-imx6.c | 19 +++++++++++++++----
> >  1 file changed, 15 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/pci/controller/dwc/pci-imx6.c
> > b/drivers/pci/controller/dwc/pci-imx6.c
> > index 3ce3993d5797..d122c12193a6 100644
> > --- a/drivers/pci/controller/dwc/pci-imx6.c
> > +++ b/drivers/pci/controller/dwc/pci-imx6.c
> > @@ -845,7 +845,9 @@ static int imx6_pcie_start_link(struct dw_pcie *pci)
> >  	/* Start LTSSM. */
> >  	imx6_pcie_ltssm_enable(dev);
> >
> > -	dw_pcie_wait_for_link(pci);
> > +	ret = dw_pcie_wait_for_link(pci);
> > +	if (ret)
> > +		goto err_out;
> 
> This adds back error handling that has been intentionally removed in
> f81f095e8771 ("PCI: imx6: Allow to probe when dw_pcie_wait_for_link() fails").
> While I agree that disabling the clocks and regulators is the right thing to do
> when we don't manage to get a link, we should still allow the driver to probe,
> so please add a "ret = 0" to this newly added non-fatal error paths.
> 
Thanks for your review comments.
There would be a long latency if the link is down and probe is finished
 successfully.
Since the dw_pcie_wait_for_link() would be invoked twice in every driver probe
 and resume operation later. Each dw_pcie_wait_for_link() would consume about
 90,000*10 ~ 100,000*10 u-seconds. I'm afraid that such a long latency would
 bring bad user experience.

Here are the logs when probe is allowed when PCIe link is down:
[   55.045954][ T1835] imx6q-pcie 5f000000.pcie: PM: calling imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 @ 1835, parent: bus@5f000000
...
[   56.074566][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came up
[   57.074816][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came up
...
[   57.182300][ T1835] imx6q-pcie 5f000000.pcie: PM: imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 returned 0 after 2136334 usecs

[   57.182347][ T1835] imx6q-pcie 5f010000.pcie: PM: calling imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 @ 1835, parent: bus@5f000000
...
[   58.210584][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came up
[   59.210831][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came up
...
[   59.318313][ T1835] imx6q-pcie 5f010000.pcie: PM: imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 returned 0 after 2135949 usecs

So, I'm prefer that it's better to let the probe failed when link is down. 
How do you think about that?

> >
> >  	if (pci->link_gen == 2) {
> >  		/* Allow Gen2 mode after the link is up. */ @@ -876,12 +878,14
> @@
> > static int imx6_pcie_start_link(struct dw_pcie *pci)
> >  			ret = imx6_pcie_wait_for_speed_change(imx6_pcie);
> >  			if (ret) {
> >  				dev_err(dev, "Failed to bring link up!\n");
> > -				goto err_reset_phy;
> > +				goto err_out;
> >  			}
> >  		}
> >
> >  		/* Make sure link training is finished as well! */
> > -		dw_pcie_wait_for_link(pci);
> > +		ret = dw_pcie_wait_for_link(pci);
> > +		if (ret)
> > +			goto err_out;
> >  	} else {
> >  		dev_info(dev, "Link: Gen2 disabled\n");
> >  	}
> > @@ -890,11 +894,18 @@ static int imx6_pcie_start_link(struct dw_pcie
> *pci)
> >  	dev_info(dev, "Link up, Gen%i\n", tmp & PCI_EXP_LNKSTA_CLS);
> >  	return 0;
> >
> > -err_reset_phy:
> > +err_out:
> >  	dev_dbg(dev, "PHY DEBUG_R0=0x%08x DEBUG_R1=0x%08x\n",
> >  		dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG0),
> >  		dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG1));
> >  	imx6_pcie_reset_phy(imx6_pcie);
> > +	imx6_pcie_clk_disable(imx6_pcie);
> > +	if (imx6_pcie->phy != NULL) {
> 
> Please use the more common if (imx6_pcie->phy) here.
> 
Okay. Thanks.

Best Regards
Richard Zhu

> Regards,
> Lucas
> 
> > +		phy_power_off(imx6_pcie->phy);
> > +		phy_exit(imx6_pcie->phy);
> > +	}
> > +	if (imx6_pcie->vpcie)
> > +		regulator_disable(imx6_pcie->vpcie);
> >  	return ret;
> >  }
> >
>