[U-Boot] U-boot hangs on imx6 pci driver

[U-Boot] U-boot hangs on imx6 pci driver

[Fabio Estevam]

Hi,

Working on a mx6solo board with PCIe driver enabled in U-boot, I
notice that after doing several reboots a hang is seen on the PCIe
driver:

PCI Autoconfig: Bus Memory region: [0x1100000-0x1efffff],
                Physical Memory [1100000-1efffffx]
PCI Autoconfig: Bus I/O region: [0x1000000-0x10fffff],
                Physical Memory: [1000000-10fffff]
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc00c
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc000
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc000
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc008
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
  00:01.0     - 16c3:abcd - Bridge device
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc008
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc004
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc010
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
PCI Autoconfig: BAR 0, I/O, size=0xfff4, address=0x1000000
bus_lower=0x100fff4### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config

### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc014
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc004
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc00c
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc00c
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
PCI Autoconfig: Found P2P bridge, device 1
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc004
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc018
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc018
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc018
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc020
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc024
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc024
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc01c
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc030
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1ffc004
### 5 -> imx_pcie_read_config
### OUT -> imx_pcie_read_config
### IN -> imx_pcie_write_config
### OUT -> imx_pcie_write_config
### IN -> imx_pcie_read_config
### 1 -> imx_pcie_read_config
### 2 -> imx_pcie_read_config
### 3 -> imx_pcie_read_config
### 4 -> imx_pcie_read_config - va_address = 0x 1f0000c

(Hang)

As a quick workaround I tried the following:

--- a/drivers/pci/pcie_imx.c
+++ b/drivers/pci/pcie_imx.c
@@ -385,7 +385,8 @@ static int imx_pcie_read_config(struct pci_controller *hose,
         */
        imx_pcie_fix_dabt_handler(true);
        writel(0xffffffff, val);
-       *val = readl(va_address);
+       if (0x01ffc000 <= va_address && va_address <= 0x01ffffff)
+               *val = readl(va_address);
        imx_pcie_fix_dabt_handler(false);

and the hang does not happen. However, not all devices connected to
the PCI switch can be discovered.

Any suggestions?

Thanks,

Fabio Estevam

[U-Boot] U-boot hangs on imx6 pci driver

[Marek Vasut]

On Tuesday, May 27, 2014 at 02:30:27 PM, Fabio Estevam wrote:
> Hi,
> 
> Working on a mx6solo board with PCIe driver enabled in U-boot, I
> notice that after doing several reboots a hang is seen on the PCIe
> driver:

[...]

> Any suggestions?

Take a look at SR# 1-1347946851 in the FSL internal bug tracker. It looks like 
the PCIe IP core implementation in the MX6 is bugged in my opinion.

Best regards,
Marek Vasut

[U-Boot] U-boot hangs on imx6 pci driver

["David Müller (ELSOFT AG)"]

Marek Vasut wrote:
> Take a look at SR# 1-1347946851 in the FSL internal bug tracker. It
> looks like the PCIe IP core implementation in the MX6 is bugged in my
> opinion.

Are there any publicly available info regarding this SR?

Dave

[U-Boot] U-boot hangs on imx6 pci driver

[Marek Vasut]

On Tuesday, May 27, 2014 at 04:43:08 PM, David M?ller (ELSOFT AG) wrote:
> Marek Vasut wrote:
> > Take a look at SR# 1-1347946851 in the FSL internal bug tracker. It
> > looks like the PCIe IP core implementation in the MX6 is bugged in my
> > opinion.
> 
> Are there any publicly available info regarding this SR?

Scrubbed the irrelevant bits:

-->8--
Please find a defect report below for the i.MX6DL PCI express driver in current 
Linux 3.10.17-1.0.0-ga released by Freescale:

Priority: High

Issue Type: Defect Report
Problem Category: Freescale Software
Problem Domain: Serial Connectivity

Project Name: Freescale SabreSDP
Industry Segment: Industrial

Target Processor: i.MX6DL Processors
Target HW Platform: i.MX6DL SabreSDP
Target OS: Linux
Target Software Package: Linux imx_3.10.17_1.0.0_ga

PC Host System O/S, Version: Not Applicable in this case

Reproducibility: Rarely
Steps to Reproduce:
  Prep:
  1) Take MX6DL SabreSDP platform
  2) Attach Intel i210 PCI express card
  3) Install BSP with kernel 3.10.17-1.0.0-ga onto SD card
  4) Boot the platform and confirm the i210 is recognized

  Test:
  1) Power on the MX6DL SabreSDP
  2) Boot the kernel
  3) Verify the SabreSDP recognized the i210
  4) When the platform reaches the init process, trigger a software-reboot
  5) The software-reboot will put the platform back into U-Boot, so the SabreSDP
     will again continue from step 2) by booting the kernel automatically.

  Test steps 2)...5) must be performed at least 10000 times.

Expected Results:
  10000 of 10000 times, the PCI express i210 card will be recognized by the
  platform. The PCI express link will always reliably come up.

Observed Results:
  After some hundreds of the soft-reboot cycles, the i210 is not recognized. 
  This is caused by the PCI express link failing to come up. We see "link never 
  came up" message in the kernel log.

Description:
  We perform the test above -- soft-restarting the MX6DL SabreSDP platform with 
  Linux 3.10.17-1.0.0-ga in quick sequences. We would expect for the PCIe link 
  to reliably come up in all of the 10000 cycles, but in some of those cycles, 
  the link fails to come up.

  Can you please confirm/replicate the issue and provide us with a fix ?
--8<--

Best regards,
Marek Vasut

[U-Boot] U-boot hangs on imx6 pci driver

["David Müller (ELSOFT AG)"]

Marek Vasut wrote:
> Observed Results: After some hundreds of the soft-reboot cycles, the
> i210 is not recognized. This is caused by the PCI express link
> failing to come up. We see "link never came up" message in the kernel
> log.

Just a guest but maybe this is "errata #18" of the i210.

A little bit off-topic but i'm facing the problem that Linux (kernel
3.14 + some patches) hangs regularly during PCIe initialisation on our
custom iMX6 / i210 board.

I use an additional delay in imx6_add_pcie_port() as a workaround so far.


Dave

[U-Boot] U-boot hangs on imx6 pci driver

[Fabio Estevam]

On Tue, May 27, 2014 at 9:30 AM, Fabio Estevam <festevam@gmail.com> wrote:
> Hi,
>
> Working on a mx6solo board with PCIe driver enabled in U-boot, I
> notice that after doing several reboots a hang is seen on the PCIe
> driver:

Just an update: we are able now to run overnight tests without PCI hangs.

Sorry for the noise, but the hang was caused by a debug code.

We still get "link never came up" issue on some boots, but this is a
different one.

[U-Boot] U-boot hangs on imx6 pci driver

[Fabio Estevam]

Hi David,

On Wed, May 28, 2014 at 4:40 AM, "David M?ller (ELSOFT AG)"
<d.mueller@elsoft.ch> wrote:
> Marek Vasut wrote:
>> Observed Results: After some hundreds of the soft-reboot cycles, the
>> i210 is not recognized. This is caused by the PCI express link
>> failing to come up. We see "link never came up" message in the kernel
>> log.
>
> Just a guest but maybe this is "errata #18" of the i210.
>
> A little bit off-topic but i'm facing the problem that Linux (kernel
> 3.14 + some patches) hangs regularly during PCIe initialisation on our
> custom iMX6 / i210 board.
>
> I use an additional delay in imx6_add_pcie_port() as a workaround so far.

How much of additional delay? Could you please share your patch?

[U-Boot] U-boot hangs on imx6 pci driver

[Marek Vasut]

On Wednesday, May 28, 2014 at 06:42:41 PM, Fabio Estevam wrote:
> On Tue, May 27, 2014 at 9:30 AM, Fabio Estevam <festevam@gmail.com> wrote:
> > Hi,
> > 
> > Working on a mx6solo board with PCIe driver enabled in U-boot, I
> > notice that after doing several reboots a hang is seen on the PCIe
> 
> > driver:
> Just an update: we are able now to run overnight tests without PCI hangs.
> 
> Sorry for the noise, but the hang was caused by a debug code.
> 
> We still get "link never came up" issue on some boots, but this is a
> different one.

Does that mean FSL is unable to reliably bring PCIe link up on every boot on 
their platform please?

Best regards,
Marek Vasut

[U-Boot] U-boot hangs on imx6 pci driver

["David Müller (ELSOFT AG)"]

Fabio Estevam wrote:
> On Wed, May 28, 2014 at 4:40 AM, "David M?ller (ELSOFT AG)"
> <d.mueller@elsoft.ch> wrote:
>> I use an additional delay in imx6_add_pcie_port() as a workaround so far.
> 
> How much of additional delay? Could you please share your patch?

diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-imx6.c
index ee08250..1accc0e 100644
--- a/drivers/pci/host/pci-imx6.c
+++ b/drivers/pci/host/pci-imx6.c
@@ -503,6 +532,9 @@ static int imx6_add_pcie_port(struct pcie_port *pp,
 	pp->root_bus_nr = -1;
 	pp->ops = &imx6_pcie_host_ops;

+	/* FIXME  kernel hangs without this delay */
+	usleep_range(20000, 25000);
+
 	spin_lock_init(&pp->conf_lock);
 	ret = dw_pcie_host_init(pp);
 	if (ret) {


Dave

[U-Boot] U-boot hangs on imx6 pci driver

[Tim Harvey]

On Fri, May 30, 2014 at 12:04 AM, "David M?ller (ELSOFT AG)"
<d.mueller@elsoft.ch> wrote:
> Fabio Estevam wrote:
>> On Wed, May 28, 2014 at 4:40 AM, "David M?ller (ELSOFT AG)"
>> <d.mueller@elsoft.ch> wrote:
>>> I use an additional delay in imx6_add_pcie_port() as a workaround so far.
>>
>> How much of additional delay? Could you please share your patch?
>
> diff --git a/drivers/pci/host/pci-imx6.c b/drivers/pci/host/pci-imx6.c
> index ee08250..1accc0e 100644
> --- a/drivers/pci/host/pci-imx6.c
> +++ b/drivers/pci/host/pci-imx6.c
> @@ -503,6 +532,9 @@ static int imx6_add_pcie_port(struct pcie_port *pp,
>         pp->root_bus_nr = -1;
>         pp->ops = &imx6_pcie_host_ops;
>
> +       /* FIXME  kernel hangs without this delay */
> +       usleep_range(20000, 25000);
> +
>         spin_lock_init(&pp->conf_lock);
>         ret = dw_pcie_host_init(pp);
>         if (ret) {
>
>
> Dave
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot

David / Marek,

I run into this issue with our Gateworks Ventana IMX6 boards as well.

When enabling PCI support in u-boot my 3.14 kernel hangs somewhere
during PCI init or enumeration (can't tell as uart is not up yet) with
PCI devices behind a PCIe switch. Applying your patch does seem to
work-around the problem on my end as well. Note that I do not see this
issue unless I'm using one of our boards with a PCIe switch and If I
disable PCI support in u-boot this does not occur. This feels like a
PCIE_RST# issue however I do have a PCIE_RST# signal being pinmuxed
and configured properly in my u-boot board support as well as the
kernel.

There seems to be some sort of timing thing that we don't understand here.

David, on your board(s) do you have a PCIe switch as well?

Tim

[U-Boot] U-boot hangs on imx6 pci driver

["David Müller (ELSOFT AG)"]

Tim Harvey wrote:

> When enabling PCI support in u-boot my 3.14 kernel hangs somewhere 
> during PCI init or enumeration (can't tell as uart is not up yet)

Enabling "earlyprintk" support may help.

> David, on your board(s) do you have a PCIe switch as well?

Not yet, but this (using a PEX8603) is planned for the near future.

Dave

[U-Boot] U-boot hangs on imx6 pci driver

[Tim Harvey]

On Wed, Jun 4, 2014 at 11:30 PM, "David M?ller (ELSOFT AG)"
<d.mueller@elsoft.ch> wrote:
> Tim Harvey wrote:
>
>> When enabling PCI support in u-boot my 3.14 kernel hangs somewhere
>> during PCI init or enumeration (can't tell as uart is not up yet)
>
> Enabling "earlyprintk" support may help.

David,

This is definitely related to PCI_RST# as the delay you inserted is
essentially after imx6_pcie_probe() gets the reset_gpio from OF and
asserts it low. Moving the delay around I find that it must come
before imx6_pcie_assert_core_reset(), specifically before
IMOUX_GPR1:18 is set (phy power down request).

Enabling earlyprintk to try to debug the 'hang' on my boards further I
find that I hang in in imx6_pcie_link_up() during the
readl(pp->dbi_base + PCIE_PHY_DEBUG_R1) which is called after setting
IOMUXC_GPR12:1 to start LTSSM. If I add a udelay(55) (55us determined
via trial and error) directly after setting IOMUXC_GPR12:1 I get
passed 'this' hang however during pci_scan_bridge() I find that
PCI_PRIMARY_BUS config dword comes back as 0x00000000 instead of
0x00010100. This appears to cause the causes the pci code to think the
RC is a bridge, tries to scan behind it, and hangs (because its not a
bridge and those transactions are not valid).

All of this seems to indicate to me that the PLX bridge I have on my
boards requires a longer minimum time to be held in reset 'before' the
PCIe PHY is powered down 'if' it has already been enumerated (or
something to that nature) as I only see this if I enable PCI in uboot.
Why I also need the extra udelay after enabling LTSSM in this scenario
I can't say.

This may correlate with your findings as I believe you say you have an
i210 attached to the IMX6 RC and have found an errata indicating the
i210 needs a longer time in reset. Do you find that this is the case
even if you disable PCI in uboot? My first thought when I read about
that i210 errata you pointed out was that it wasn't an issue because
we do hold reset low for 100ms in the kernel but if the issue has
something to do with holding it in reset with the PHY being powered
down then perhaps this explains things.

Merek,

you've done much more work on IMX6 link than I... any of this make
sense to you? I believe you have never encountered this 100%
repeatable issue on your board(s) that David and I do encounter, but
you have encountered a 1% PCIe link failure rate (which I'm inclined
to say is something completely different?).

Regards,

Tim

>
>> David, on your board(s) do you have a PCIe switch as well?
>
> Not yet, but this (using a PEX8603) is planned for the near future.
>
> Dave
>

[U-Boot] U-boot hangs on imx6 pci driver

[Fabio Estevam]

Tim,

On Wed, Jun 4, 2014 at 9:16 PM, Tim Harvey <tharvey@gateworks.com> wrote:

> work-around the problem on my end as well. Note that I do not see this
> issue unless I'm using one of our boards with a PCIe switch and If I
> disable PCI support in u-boot this does not occur. This feels like a

Does this help?

 drivers/pci/pcie_imx.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/pci/pcie_imx.c b/drivers/pci/pcie_imx.c
index c48737e..190cdec 100644
--- a/drivers/pci/pcie_imx.c
+++ b/drivers/pci/pcie_imx.c
@@ -435,8 +435,7 @@ static int imx6_pcie_init_phy(void)
     clrbits_le32(&iomuxc_regs->gpr[12], IOMUXC_GPR12_APPS_LTSSM_ENABLE);

     clrsetbits_le32(&iomuxc_regs->gpr[12],
-            IOMUXC_GPR12_DEVICE_TYPE_MASK,
-            IOMUXC_GPR12_DEVICE_TYPE_RC);
+            IOMUXC_GPR12_DEVICE_TYPE_MASK, 4 << 12);
     clrsetbits_le32(&iomuxc_regs->gpr[12],
             IOMUXC_GPR12_LOS_LEVEL_MASK,
             IOMUXC_GPR12_LOS_LEVEL_9);

[U-Boot] U-boot hangs on imx6 pci driver

[Marek Vasut]

On Thursday, June 05, 2014 at 05:27:08 PM, Fabio Estevam wrote:
> Tim,
> 
> On Wed, Jun 4, 2014 at 9:16 PM, Tim Harvey <tharvey@gateworks.com> wrote:
> > work-around the problem on my end as well. Note that I do not see this
> > issue unless I'm using one of our boards with a PCIe switch and If I
> > disable PCI support in u-boot this does not occur. This feels like a
> 
> Does this help?
> 
>  drivers/pci/pcie_imx.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/pcie_imx.c b/drivers/pci/pcie_imx.c
> index c48737e..190cdec 100644
> --- a/drivers/pci/pcie_imx.c
> +++ b/drivers/pci/pcie_imx.c
> @@ -435,8 +435,7 @@ static int imx6_pcie_init_phy(void)
>      clrbits_le32(&iomuxc_regs->gpr[12], IOMUXC_GPR12_APPS_LTSSM_ENABLE);
> 
>      clrsetbits_le32(&iomuxc_regs->gpr[12],
> -            IOMUXC_GPR12_DEVICE_TYPE_MASK,
> -            IOMUXC_GPR12_DEVICE_TYPE_RC);
> +            IOMUXC_GPR12_DEVICE_TYPE_MASK, 4 << 12);

Is this the setting of RC-mode in GPR12 ? That's wrong in the FSL datasheet, not 
in the code IIRC ;-)

Best regards,
Marek Vasut

[U-Boot] U-boot hangs on imx6 pci driver

[Fabio Estevam]

On Thu, Jun 5, 2014 at 2:53 PM, Marek Vasut <marex@denx.de> wrote:

>> Does this help?
>>
>>  drivers/pci/pcie_imx.c | 3 +--
>>  1 file changed, 1 insertion(+), 2 deletions(-)
>>
>> diff --git a/drivers/pci/pcie_imx.c b/drivers/pci/pcie_imx.c
>> index c48737e..190cdec 100644
>> --- a/drivers/pci/pcie_imx.c
>> +++ b/drivers/pci/pcie_imx.c
>> @@ -435,8 +435,7 @@ static int imx6_pcie_init_phy(void)
>>      clrbits_le32(&iomuxc_regs->gpr[12], IOMUXC_GPR12_APPS_LTSSM_ENABLE);
>>
>>      clrsetbits_le32(&iomuxc_regs->gpr[12],
>> -            IOMUXC_GPR12_DEVICE_TYPE_MASK,
>> -            IOMUXC_GPR12_DEVICE_TYPE_RC);
>> +            IOMUXC_GPR12_DEVICE_TYPE_MASK, 4 << 12);
>
> Is this the setting of RC-mode in GPR12 ? That's wrong in the FSL datasheet, not
> in the code IIRC ;-)

Yes, RM is wrong. U-boot is setting it to 2. Kernel sets it to 4,
which is the correct value.

[U-Boot] U-boot hangs on imx6 pci driver

[Marek Vasut]

On Thursday, June 05, 2014 at 09:20:12 PM, Fabio Estevam wrote:
> On Thu, Jun 5, 2014 at 2:53 PM, Marek Vasut <marex@denx.de> wrote:
> >> Does this help?
> >> 
> >>  drivers/pci/pcie_imx.c | 3 +--
> >>  1 file changed, 1 insertion(+), 2 deletions(-)
> >> 
> >> diff --git a/drivers/pci/pcie_imx.c b/drivers/pci/pcie_imx.c
> >> index c48737e..190cdec 100644
> >> --- a/drivers/pci/pcie_imx.c
> >> +++ b/drivers/pci/pcie_imx.c
> >> @@ -435,8 +435,7 @@ static int imx6_pcie_init_phy(void)
> >> 
> >>      clrbits_le32(&iomuxc_regs->gpr[12],
> >>      IOMUXC_GPR12_APPS_LTSSM_ENABLE);
> >>      
> >>      clrsetbits_le32(&iomuxc_regs->gpr[12],
> >> 
> >> -            IOMUXC_GPR12_DEVICE_TYPE_MASK,
> >> -            IOMUXC_GPR12_DEVICE_TYPE_RC);
> >> +            IOMUXC_GPR12_DEVICE_TYPE_MASK, 4 << 12);
> > 
> > Is this the setting of RC-mode in GPR12 ? That's wrong in the FSL
> > datasheet, not in the code IIRC ;-)
> 
> Yes, RM is wrong. U-boot is setting it to 2. Kernel sets it to 4,
> which is the correct value.

Ah yes, 0x4 is correct, I stand corrected, sorry.

Best regards,
Marek Vasut

[U-Boot] U-boot hangs on imx6 pci driver

[Fabio Estevam]

On Thu, Jun 5, 2014 at 7:04 PM, Marek Vasut <marex@denx.de> wrote:

> Ah yes, 0x4 is correct, I stand corrected, sorry.

No problem. I will submit a patch for it soon.

[U-Boot] U-boot hangs on imx6 pci driver

[Marek Vasut]

On Friday, June 06, 2014 at 12:14:03 AM, Fabio Estevam wrote:
> On Thu, Jun 5, 2014 at 7:04 PM, Marek Vasut <marex@denx.de> wrote:
> > Ah yes, 0x4 is correct, I stand corrected, sorry.
> 
> No problem. I will submit a patch for it soon.

Thanks!

Best regards,
Marek Vasut

[U-Boot] U-boot hangs on imx6 pci driver

[Tim Harvey]

On Thu, Jun 5, 2014 at 8:27 AM, Fabio Estevam <festevam@gmail.com> wrote:
> Tim,
>
> On Wed, Jun 4, 2014 at 9:16 PM, Tim Harvey <tharvey@gateworks.com> wrote:
>
>> work-around the problem on my end as well. Note that I do not see this
>> issue unless I'm using one of our boards with a PCIe switch and If I
>> disable PCI support in u-boot this does not occur. This feels like a
>
> Does this help?
>
>  drivers/pci/pcie_imx.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/drivers/pci/pcie_imx.c b/drivers/pci/pcie_imx.c
> index c48737e..190cdec 100644
> --- a/drivers/pci/pcie_imx.c
> +++ b/drivers/pci/pcie_imx.c
> @@ -435,8 +435,7 @@ static int imx6_pcie_init_phy(void)
>      clrbits_le32(&iomuxc_regs->gpr[12], IOMUXC_GPR12_APPS_LTSSM_ENABLE);
>
>      clrsetbits_le32(&iomuxc_regs->gpr[12],
> -            IOMUXC_GPR12_DEVICE_TYPE_MASK,
> -            IOMUXC_GPR12_DEVICE_TYPE_RC);
> +            IOMUXC_GPR12_DEVICE_TYPE_MASK, 4 << 12);
>      clrsetbits_le32(&iomuxc_regs->gpr[12],
>              IOMUXC_GPR12_LOS_LEVEL_MASK,
>              IOMUXC_GPR12_LOS_LEVEL_9);

Fabio,

Good catch, but that doesn't resolve the issue i'm seeing here.

Any other ideas?

Regards,

Tim

[U-Boot] U-boot hangs on imx6 pci driver

[Fabio Estevam]

Hi Tim,

On Fri, Jun 6, 2014 at 1:35 AM, Tim Harvey <tharvey@gateworks.com> wrote:

> Fabio,
>
> Good catch, but that doesn't resolve the issue i'm seeing here.
>
> Any other ideas?

Do you still have issues after applying David's delay workaround?

On my mx6qsabresd I noticed that:

- if U-boot has PCI driver enabled, then kernel hangs 100% of time.

- if U-boot does not have PCI driver enabled, then the kernel boots
but does not detect my PCI card.

After applying David's workaround I can boot the kernel with PCI
driver enabled in U-boot and the kernel does detect the PCI Wifi
module.

[U-Boot] U-boot hangs on imx6 pci driver

[Marek Vasut]

On Thursday, June 05, 2014 at 12:13:39 PM, Tim Harvey wrote:
> On Wed, Jun 4, 2014 at 11:30 PM, "David M?ller (ELSOFT AG)"
> 
> <d.mueller@elsoft.ch> wrote:
> > Tim Harvey wrote:
> >> When enabling PCI support in u-boot my 3.14 kernel hangs somewhere
> >> during PCI init or enumeration (can't tell as uart is not up yet)
> > 
> > Enabling "earlyprintk" support may help.
> 
> David,
> 
> This is definitely related to PCI_RST# as the delay you inserted is
> essentially after imx6_pcie_probe() gets the reset_gpio from OF and
> asserts it low. Moving the delay around I find that it must come
> before imx6_pcie_assert_core_reset(), specifically before
> IMOUX_GPR1:18 is set (phy power down request).
> 
> Enabling earlyprintk to try to debug the 'hang' on my boards further I
> find that I hang in in imx6_pcie_link_up() during the
> readl(pp->dbi_base + PCIE_PHY_DEBUG_R1) which is called after setting
> IOMUXC_GPR12:1 to start LTSSM. If I add a udelay(55) (55us determined
> via trial and error) directly after setting IOMUXC_GPR12:1 I get
> passed 'this' hang however during pci_scan_bridge() I find that
> PCI_PRIMARY_BUS config dword comes back as 0x00000000 instead of
> 0x00010100. This appears to cause the causes the pci code to think the
> RC is a bridge, tries to scan behind it, and hangs (because its not a
> bridge and those transactions are not valid).
> 
> All of this seems to indicate to me that the PLX bridge I have on my
> boards requires a longer minimum time to be held in reset 'before' the
> PCIe PHY is powered down 'if' it has already been enumerated (or
> something to that nature) as I only see this if I enable PCI in uboot.
> Why I also need the extra udelay after enabling LTSSM in this scenario
> I can't say.
> 
> This may correlate with your findings as I believe you say you have an
> i210 attached to the IMX6 RC and have found an errata indicating the
> i210 needs a longer time in reset. Do you find that this is the case
> even if you disable PCI in uboot? My first thought when I read about
> that i210 errata you pointed out was that it wasn't an issue because
> we do hold reset low for 100ms in the kernel but if the issue has
> something to do with holding it in reset with the PHY being powered
> down then perhaps this explains things.
> 
> Merek,
> 
> you've done much more work on IMX6 link than I... any of this make
> sense to you? I believe you have never encountered this 100%
> repeatable issue on your board(s) that David and I do encounter, but
> you have encountered a 1% PCIe link failure rate (which I'm inclined
> to say is something completely different?).

Sorry for the later reply. Most certainly, I observe a failure rate of 1-out-
of-200 or so. I also use i210 though, so it might be related somehow. Can you 
tell me which i210 errata are you talking about here please?

I am currently discussing this with FSL, but it didn't yield any results yet.

Best regards,
Marek Vasut

[U-Boot] U-boot hangs on imx6 pci driver

[Marek Vasut]

On Tuesday, June 17, 2014 at 04:14:20 PM, Fabio Estevam wrote:
> Hi Tim,
> 
> On Fri, Jun 6, 2014 at 1:35 AM, Tim Harvey <tharvey@gateworks.com> wrote:
> > Fabio,
> > 
> > Good catch, but that doesn't resolve the issue i'm seeing here.
> > 
> > Any other ideas?
> 
> Do you still have issues after applying David's delay workaround?

Yes

Again, apologies for the late reply.

> On my mx6qsabresd I noticed that:
> 
> - if U-boot has PCI driver enabled, then kernel hangs 100% of time.

That's because the PCIe core and PCIe PIPE PHY are not properly restarted. Do we 
have any software way to put the PCIe core to an initial state ? Same pro PIPE 
PHY ?

> - if U-boot does not have PCI driver enabled, then the kernel boots
> but does not detect my PCI card.

Well, that's some other shitness. I can get MX6SabreSDP to detect my card, but I 
have that 1-out-of-200 "phy link never came up" problem.

> After applying David's workaround I can boot the kernel with PCI
> driver enabled in U-boot and the kernel does detect the PCI Wifi
> module.

OK. Even with David's patch or with FSL supplied patches, I still have the 1-
out-of-200 failure case :-/

Best regards,
Marek Vasut