RE: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators after link is down

[Hongxing Zhu <hongxing.zhu@nxp.com>]

> -----Original Message-----
> From: Francesco Dolcini <francesco.dolcini@toradex.com>
> Sent: 2022年6月9日 15:53
> To: Hongxing Zhu <hongxing.zhu@nxp.com>
> Cc: Lucas Stach <l.stach@pengutronix.de>; bhelgaas@google.com;
> robh+dt@kernel.org; broonie@kernel.org; lorenzo.pieralisi@arm.com;
> jingoohan1@gmail.com; festevam@gmail.com;
> francesco.dolcini@toradex.com; linux-pci@vger.kernel.org;
> linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org;
> kernel@pengutronix.de; dl-linux-imx <linux-imx@nxp.com>
> Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators after link is
> down
> 
> On Thu, Jun 09, 2022 at 06:17:46AM +0000, Hongxing Zhu wrote:
> > > -----Original Message-----
> > > From: Lucas Stach <l.stach@pengutronix.de>
> > > Sent: 2022年6月8日 15:35
> > > To: Hongxing Zhu <hongxing.zhu@nxp.com>; bhelgaas@google.com;
> > > robh+dt@kernel.org; broonie@kernel.org; lorenzo.pieralisi@arm.com;
> > > jingoohan1@gmail.com; festevam@gmail.com;
> > > francesco.dolcini@toradex.com
> > > Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org;
> > > linux-kernel@vger.kernel.org; kernel@pengutronix.de; dl-linux-imx
> > > <linux-imx@nxp.com>
> > > Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators
> > > after link is down
> > >
> > > Am Freitag, dem 06.05.2022 um 09:47 +0800 schrieb Richard Zhu:
> > > > Since i.MX PCIe doesn't support hot-plug, reduce power consumption
> > > > as much as possible by disabling clocks and regulators and
> > > > returning error when the link is down.
> > > >
> > > > Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com>
> > > > ---
> > > >  drivers/pci/controller/dwc/pci-imx6.c | 19 +++++++++++++++----
> > > >  1 file changed, 15 insertions(+), 4 deletions(-)
> > > >
> > > > diff --git a/drivers/pci/controller/dwc/pci-imx6.c
> > > > b/drivers/pci/controller/dwc/pci-imx6.c
> > > > index 3ce3993d5797..d122c12193a6 100644
> > > > --- a/drivers/pci/controller/dwc/pci-imx6.c
> > > > +++ b/drivers/pci/controller/dwc/pci-imx6.c
> > > > @@ -845,7 +845,9 @@ static int imx6_pcie_start_link(struct dw_pcie
> *pci)
> > > >  	/* Start LTSSM. */
> > > >  	imx6_pcie_ltssm_enable(dev);
> > > >
> > > > -	dw_pcie_wait_for_link(pci);
> > > > +	ret = dw_pcie_wait_for_link(pci);
> > > > +	if (ret)
> > > > +		goto err_out;
> > >
> > > This adds back error handling that has been intentionally removed in
> > > f81f095e8771 ("PCI: imx6: Allow to probe when dw_pcie_wait_for_link()
> fails").
> > > While I agree that disabling the clocks and regulators is the right
> > > thing to do when we don't manage to get a link, we should still
> > > allow the driver to probe, so please add a "ret = 0" to this newly added
> non-fatal error paths.
> > >
> > Thanks for your review comments.
> > There would be a long latency if the link is down and probe is
> > finished  successfully.
> > Since the dw_pcie_wait_for_link() would be invoked twice in every
> > driver probe  and resume operation later. Each dw_pcie_wait_for_link()
> > would consume about
> >  90,000*10 ~ 100,000*10 u-seconds. I'm afraid that such a long latency
> > would  bring bad user experience.
> >
> > Here are the logs when probe is allowed when PCIe link is down:
> > [   55.045954][ T1835] imx6q-pcie 5f000000.pcie: PM: calling
> imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x
> 8 @ 1835, parent: bus@5f000000
> > ...
> > [   56.074566][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came up
> > [   57.074816][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came up
> > ...
> > [   57.182300][ T1835] imx6q-pcie 5f000000.pcie: PM:
> imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x
> 8 returned 0 after 2136334 usecs
> >
> > [   57.182347][ T1835] imx6q-pcie 5f010000.pcie: PM: calling
> imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x
> 8 @ 1835, parent: bus@5f000000
> > ...
> > [   58.210584][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came up
> > [   59.210831][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came up
> > ...
> > [   59.318313][ T1835] imx6q-pcie 5f010000.pcie: PM:
> imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x
> 8 returned 0 after 2135949 usecs
> >
> > So, I'm prefer that it's better to let the probe failed when link is down.
> > How do you think about that?
> 
> I think that recently Bjorn mentioned some concern with this approach, and I
> agree with him.
> I think that the probe of the PCIe root port should not fail if the link is down.
> 
> What is the reason for such a long wait in dw_pcie_wait_for_link()? Is this
> slowing down the resume process as a whole? Why called twice? (I'm not
> familiar with that part of the code)
> 
Thanks for your concerns.
To avoid a corner link down issue, iMX PCIe driver force link up GEN1 speed
firstly, then try to link up the highest speed later with one speed exchange.
So, the dw_pcie_wait_for_link() would be invoked twice in iMX PCIe driver.

Lucas and I try to figure out one method to avoid the long latency when resume
back from suspend mode when link is down and probe is succeed.

Best Regards
Richard Zhu

> Francesco