Re: [PATCH] pci: aardvark: Wait for endpoint to be ready before training link

[Remi Pommarel <repk@triplefau.lt>]

On Wed, Apr 24, 2019 at 05:50:02PM +0100, Lorenzo Pieralisi wrote:
> On Wed, Apr 24, 2019 at 12:29:18AM +0200, Remi Pommarel wrote:
> > Hi,
> > 
> > On Tue, Apr 23, 2019 at 05:32:15PM +0100, Lorenzo Pieralisi wrote:
> > > On Wed, Mar 13, 2019 at 10:37:52PM +0100, Remi Pommarel wrote:
> > > > When configuring pcie reset pin from gpio (e.g. initially set by
> > > > u-boot) to pcie function this pin goes low for a brief moment
> > > > asserting the PERST# signal. Thus connected device enters fundamental
> > > > reset process and link configuration can only begin after a minimal
> > > > 100ms delay (see [1]).
> > > > 
> > > > This makes sure that link is configured after at least 100ms from
> > > > beginning of probe() callback (shortly after the reset pin function
> > > > configuration switch through pinctrl subsytem).
> 
> I am a bit lost, what's the connection between the probe() callback
> and the reset pin function configuration ?
> 
> Please elaborate.
> 

So currently u-boot configures the reset pin as a GPIO set to high. The
espressobin devicetree defines a default pinctrl to configure this pin
as a PCIe reset function.

As you can see in drivers/base/dd.c, driver_probe_device() calls
really_probe() which first calls pinctrl_bind_pins() then shortly after
drv->probe() callback. The pinctrl_bind_pins() function applies the
default state. So here, just before drv->probe() gets called our reset
pin goes from GPIO function to PCIe reset one making it going low for a
short time during this transition.

Because the pin goes low then gets back to high, PERST# signal is
asserted then deasserted and device enters fundamental reset process
just before drv->probe() is called. So in order to reduce the waiting
time to a minimum I sample jiffies at the very beginning of the probe
function, which is the closer spot from where PERST# is deasserted.

To sum up:

driver_probe_device() {
	...
	really_probe() {
		...
		pinctrl_bind_pins(); /* Here PERST# is asserted because pin configuration changes */
		...
		drv->probe();
		...
	}
	...
}

> > > > 
> > > > [1] "PCI Express Base Specification", REV. 2.1
> > > >         PCI Express, March 4 2009, 6.6.1 Conventional Reset
> > > > 
> > > > Signed-off-by: Remi Pommarel <repk@triplefau.lt>
> > > > ---
> > > >  drivers/pci/controller/pci-aardvark.c | 17 ++++++++++++++---
> > > >  1 file changed, 14 insertions(+), 3 deletions(-)
> > > > 
> > > > diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c
> > > > index a30ae7cf8e7e..70a1023d0ef1 100644
> > > > --- a/drivers/pci/controller/pci-aardvark.c
> > > > +++ b/drivers/pci/controller/pci-aardvark.c
> > > > @@ -177,6 +177,9 @@
> > > >  
> > > >  #define PIO_TIMEOUT_MS			1
> > > >  
> > > > +/* Endpoint can take up to 100ms to be ready after a reset */
> > > > +#define ENDPOINT_RST_MS			100
> > > > +
> > > >  #define LINK_WAIT_MAX_RETRIES		10
> > > >  #define LINK_WAIT_USLEEP_MIN		90000
> > > >  #define LINK_WAIT_USLEEP_MAX		100000
> > > > @@ -242,8 +245,10 @@ static int advk_pcie_wait_for_link(struct advk_pcie *pcie)
> > > >  	return -ETIMEDOUT;
> > > >  }
> > > >  
> > > > -static void advk_pcie_setup_hw(struct advk_pcie *pcie)
> > > > +static void
> > > > +advk_pcie_setup_hw(struct advk_pcie *pcie, unsigned long ep_rdy_time)
> > > 
> > > Nit: I prefer the prototype to be in one line, I wrap it for you.
> > > 
> > > I am wondering why you need to pass in ep_rdy_time parameter when you
> > > can easily compute it in the function itself.
> > > 
> > 
> > The only reason for that is because the sooner I get the jiffies the
> > lower the delay has to be. I was trying to reduce the impact of this
> > delay to a minimum, but maybe the improvement is not worth it.
> 
> That should just be (roughly) some microseconds unless there is
> something I am missing. Try to measure it :)

So doing that I do a msleep() of around 75-80ms instead of 100ms. So,
yes, are 20ms enough to justify that, or should we just go with a plain
msleep(100) to improve legibility.

-- 
Remi