Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

I am working on a couple of different MedaiTek MT7621 based platforms
and am having problems with the PCI bus on those.

Big picture is that the PCI bus on my boards worked in linux-4.20
(with the obvious compilation breakage fixed), and it does not work
in linux-5.0 or linux-5.1.

On linux-4.20 the PCI bus probe at kernel boot looks like this:

***** Xtal 40MHz *****
PCIE1 no card, disable it(RST&CLK)
PCIE2 no card, disable it(RST&CLK)
PCIE0 enabled
PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0xffffffff]
pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
pci_bus 0000:00: root bus resource [bus 00-ff]
pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:00.0: PCI bridge to [bus 01-ff]
pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
pci 0000:00:00.0: PCI bridge to [bus 01]
pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]

The PCI bus works, and devices on it are found and work as expected.

On linux-5.1 the PCI initialization and probe fails, with the kernel
locking up:

...
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
mt7621-pci 1e140000.pcie: Initiating port 0 failed
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
mt7621-pci 1e140000.pcie: Initiating port 1 failed
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)

The lockup is in mt7621_pci_phy_power_off(), at the phy_read() call.
If I modify that code and return immediately in that mt7621_pci_phy_power_off()
the systemboots - but obviously from the above you can see that the PCI bus
and no devices were detected.

Copying in the working linux-4.20 pci-mt7621.c into place on
linux-5.1 results in a working PCI bus also. I have 2 very different
MT7621 based boards, and they both exhibit this same problem.

I tried bisecting that back to find the problem commit.
It was not at all easy with quite a few of the mt7621 PCI related
patches not building in isolation while bisecting. But ultimately
I got to commit 745eeeac68d7 ("staging: mt7621-pci: factor out
'mt7621_pcie_enable_port' function").

Any idea what might be going wrong here?

Regards
Greg


_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

Hi Greg,

On Tue, May 21, 2019 at 8:44 AM Greg Ungerer <gerg@kernel.org> wrote:
>
>
> Hi Sergio,
>
> I am working on a couple of different MedaiTek MT7621 based platforms
> and am having problems with the PCI bus on those.
>
> Big picture is that the PCI bus on my boards worked in linux-4.20
> (with the obvious compilation breakage fixed), and it does not work
> in linux-5.0 or linux-5.1.
>
> On linux-4.20 the PCI bus probe at kernel boot looks like this:
>
> ***** Xtal 40MHz *****
> PCIE1 no card, disable it(RST&CLK)
> PCIE2 no card, disable it(RST&CLK)
> PCIE0 enabled
> PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> pci_bus 0000:00: root bus resource [io  0xffffffff]
> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> pci_bus 0000:00: root bus resource [bus 00-ff]
> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
> pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
> pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
> pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
> pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
> pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
> pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
> pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
> pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
> pci 0000:00:00.0: PCI bridge to [bus 01]
> pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
> pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
>
> The PCI bus works, and devices on it are found and work as expected.
>
> On linux-5.1 the PCI initialization and probe fails, with the kernel
> locking up:
>
> ...
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> mt7621-pci 1e140000.pcie: Initiating port 0 failed
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> mt7621-pci 1e140000.pcie: Initiating port 1 failed
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>
> The lockup is in mt7621_pci_phy_power_off(), at the phy_read() call.
> If I modify that code and return immediately in that mt7621_pci_phy_power_off()
> the systemboots - but obviously from the above you can see that the PCI bus
> and no devices were detected.

There are two changes with this two commits:

commit 36d657b011ef49b549aae44d0fe49ce845beb975
Author: Sergio Paracuellos <sergio.paracuellos@gmail.com>
Date:   Wed Apr 17 13:58:38 2019 +0200

    staging: mt7621-pci-phy: convert driver to use kernel regmap API's

    Instead of using writel and readl use regmap API which makes
    the driver maintainability easier.

    Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

commit 9445ccb3714c78c26a3a25fafed4d9d965080431
Author: Sergio Paracuellos <sergio.paracuellos@gmail.com>
Date:   Wed Apr 17 13:58:37 2019 +0200

    staging: mt7621-pci-phy: add quirks for 'E2' revision using
'soc_device_attribute'

    Depending on revision of the chip, 'mt7621_bypass_pipe_rst' function
    must be executed. Add better support for this using 'soc_device_match'
    in driver probe function.

    Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

So, I added a quirk for E2 revision of the board as I was suggested to
do by the phy
tree maintainer, and this is the only place I can think the problem could be.

I think all te changes before this was properly tested by Neil and
results in a working
PCI.

>
> Copying in the working linux-4.20 pci-mt7621.c into place on
> linux-5.1 results in a working PCI bus also. I have 2 very different
> MT7621 based boards, and they both exhibit this same problem.
>
> I tried bisecting that back to find the problem commit.
> It was not at all easy with quite a few of the mt7621 PCI related
> patches not building in isolation while bisecting. But ultimately
> I got to commit 745eeeac68d7 ("staging: mt7621-pci: factor out
> 'mt7621_pcie_enable_port' function").

Sorry for your time in this and for the problems with isolation patches. I'll
do my best from now to this don't happen again. The problem commit
you are pointing out here had problems at first because depending on the
revision of the chip the reset lines are inverted but this was properly fixed in
this other commit:

commit e51844bf825169024e0c743a92cf264e27f2366f
Author: Sergio Paracuellos <sergio.paracuellos@gmail.com>
Date:   Sat Nov 24 18:54:54 2018 +0100

    staging: mt7621-pci: fix reset lines for each pcie port

    Depending of chip revision reset lines are inverted. It is also
    necessary to read PCIE_FTS_NUM register before enabling the phy.
    Hence update the code to achieve this.

    Fixes: 745eeeac68d7 ("staging: mt7621-pci: factor out
'mt7621_pcie_enable_port' function")
    Reported-by: NeilBrown <neil@brown.name>
    Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
    Tested-by: NeilBrown <neil@brown.name>
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

>
> Any idea what might be going wrong here?

So I do believe that the problem could be with last phy changes I am
pointed out first with quirks
for your revision don't be executing function 'mt7621_bypass_pipe_rst'
inside 'mt7621_pci_phy_init'.
You should take care of quirks if revision of the board is 'E2'.

Hope this helps.

> Regards
> Greg

Best regards,
     Sergio Paracuellos
>
>
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

Thanks for the quick response.

On 21/5/19 6:14 pm, Sergio Paracuellos wrote:
> On Tue, May 21, 2019 at 8:44 AM Greg Ungerer <gerg@kernel.org> wrote:
>> I am working on a couple of different MedaiTek MT7621 based platforms
>> and am having problems with the PCI bus on those.
>>
>> Big picture is that the PCI bus on my boards worked in linux-4.20
>> (with the obvious compilation breakage fixed), and it does not work
>> in linux-5.0 or linux-5.1.
>>
>> On linux-4.20 the PCI bus probe at kernel boot looks like this:
>>
>> ***** Xtal 40MHz *****
>> PCIE1 no card, disable it(RST&CLK)
>> PCIE2 no card, disable it(RST&CLK)
>> PCIE0 enabled
>> PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>> pci_bus 0000:00: root bus resource [io  0xffffffff]
>> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>> pci_bus 0000:00: root bus resource [bus 00-ff]
>> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>> pci 0000:00:00.0: PCI bridge to [bus 01-ff]
>> pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
>> pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
>> pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
>> pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
>> pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
>> pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
>> pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
>> pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
>> pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
>> pci 0000:00:00.0: PCI bridge to [bus 01]
>> pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
>> pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
>>
>> The PCI bus works, and devices on it are found and work as expected.
>>
>> On linux-5.1 the PCI initialization and probe fails, with the kernel
>> locking up:
>>
>> ...
>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>> mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>> mt7621-pci 1e140000.pcie: Initiating port 0 failed
>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>> mt7621-pci 1e140000.pcie: Initiating port 1 failed
>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>
>> The lockup is in mt7621_pci_phy_power_off(), at the phy_read() call.
>> If I modify that code and return immediately in that mt7621_pci_phy_power_off()
>> the systemboots - but obviously from the above you can see that the PCI bus
>> and no devices were detected.
> 
> There are two changes with this two commits:
> 
> commit 36d657b011ef49b549aae44d0fe49ce845beb975
> Author: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> Date:   Wed Apr 17 13:58:38 2019 +0200
> 
>      staging: mt7621-pci-phy: convert driver to use kernel regmap API's
> 
>      Instead of using writel and readl use regmap API which makes
>      the driver maintainability easier.
> 
>      Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
>      Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> 
> commit 9445ccb3714c78c26a3a25fafed4d9d965080431
> Author: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> Date:   Wed Apr 17 13:58:37 2019 +0200
> 
>      staging: mt7621-pci-phy: add quirks for 'E2' revision using
> 'soc_device_attribute'
> 
>      Depending on revision of the chip, 'mt7621_bypass_pipe_rst' function
>      must be executed. Add better support for this using 'soc_device_match'
>      in driver probe function.
> 
>      Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
>      Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> 
> So, I added a quirk for E2 revision of the board as I was suggested to
> do by the phy
> tree maintainer, and this is the only place I can think the problem could be.

I took the pci-mt7621.c and pci-mt7621-phy.c from a linux-5.2-rc1,
which has both those commits. Same behavior, PCI probing locks up
kernel in mt7621_pci_phy_power_off().


> I think all te changes before this was properly tested by Neil and
> results in a working
> PCI.

Not sure what board Neil is using.
I am using a Digi/EX15 and I also tried a Digi/TX54 (very different
boards, very different designs).


>> Copying in the working linux-4.20 pci-mt7621.c into place on
>> linux-5.1 results in a working PCI bus also. I have 2 very different
>> MT7621 based boards, and they both exhibit this same problem.
>>
>> I tried bisecting that back to find the problem commit.
>> It was not at all easy with quite a few of the mt7621 PCI related
>> patches not building in isolation while bisecting. But ultimately
>> I got to commit 745eeeac68d7 ("staging: mt7621-pci: factor out
>> 'mt7621_pcie_enable_port' function").
> 
> Sorry for your time in this and for the problems with isolation patches. I'll

FWIW, I do like your changes, they really clean up that code a lot.


> do my best from now to this don't happen again. The problem commit
> you are pointing out here had problems at first because depending on the
> revision of the chip the reset lines are inverted but this was properly fixed in
> this other commit:
> 
> commit e51844bf825169024e0c743a92cf264e27f2366f
> Author: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> Date:   Sat Nov 24 18:54:54 2018 +0100
> 
>      staging: mt7621-pci: fix reset lines for each pcie port
> 
>      Depending of chip revision reset lines are inverted. It is also
>      necessary to read PCIE_FTS_NUM register before enabling the phy.
>      Hence update the code to achieve this.
> 
>      Fixes: 745eeeac68d7 ("staging: mt7621-pci: factor out
> 'mt7621_pcie_enable_port' function")
>      Reported-by: NeilBrown <neil@brown.name>
>      Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
>      Tested-by: NeilBrown <neil@brown.name>
>      Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> 
>>
>> Any idea what might be going wrong here?
> 
> So I do believe that the problem could be with last phy changes I am
> pointed out first with quirks
> for your revision don't be executing function 'mt7621_bypass_pipe_rst'
> inside 'mt7621_pci_phy_init'.
> You should take care of quirks if revision of the board is 'E2'.

I checked that revision register on my boards, my MT7621 parts report
a revision ID of 0x00030103 - so that means they do not match the
E2 ID. Tracing the code they do not go through mt7621_bypass_pipe_rst().
And that is true even in the older linux-4.20 driver, it does not
execute that bypass routine.

Although the lock up is bad, the real issue is that the probe of
PCI0 is not finding any cards - and it should.

I tried restoring some of the probe code from linux-4.20 and had some
inconsistent success. But I haven't been able to point to exactly what
the problem is.

Regards
Greg
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

Hi Greg,

On Wed, May 22, 2019 at 2:20 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> Thanks for the quick response.
>
> On 21/5/19 6:14 pm, Sergio Paracuellos wrote:
> > On Tue, May 21, 2019 at 8:44 AM Greg Ungerer <gerg@kernel.org> wrote:
> >> I am working on a couple of different MedaiTek MT7621 based platforms
> >> and am having problems with the PCI bus on those.
> >>
> >> Big picture is that the PCI bus on my boards worked in linux-4.20
> >> (with the obvious compilation breakage fixed), and it does not work
> >> in linux-5.0 or linux-5.1.
> >>
> >> On linux-4.20 the PCI bus probe at kernel boot looks like this:
> >>
> >> ***** Xtal 40MHz *****
> >> PCIE1 no card, disable it(RST&CLK)
> >> PCIE2 no card, disable it(RST&CLK)
> >> PCIE0 enabled
> >> PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >> pci_bus 0000:00: root bus resource [io  0xffffffff]
> >> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >> pci_bus 0000:00: root bus resource [bus 00-ff]
> >> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> >> pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> >> pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
> >> pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
> >> pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
> >> pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
> >> pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
> >> pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
> >> pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
> >> pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
> >> pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
> >> pci 0000:00:00.0: PCI bridge to [bus 01]
> >> pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
> >> pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
> >>
> >> The PCI bus works, and devices on it are found and work as expected.
> >>
> >> On linux-5.1 the PCI initialization and probe fails, with the kernel
> >> locking up:
> >>
> >> ...
> >> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >> mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> >> mt7621-pci 1e140000.pcie: Initiating port 0 failed
> >> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >> mt7621-pci 1e140000.pcie: Initiating port 1 failed
> >> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>
> >> The lockup is in mt7621_pci_phy_power_off(), at the phy_read() call.
> >> If I modify that code and return immediately in that mt7621_pci_phy_power_off()
> >> the systemboots - but obviously from the above you can see that the PCI bus
> >> and no devices were detected.
> >
> > There are two changes with this two commits:
> >
> > commit 36d657b011ef49b549aae44d0fe49ce845beb975
> > Author: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> > Date:   Wed Apr 17 13:58:38 2019 +0200
> >
> >      staging: mt7621-pci-phy: convert driver to use kernel regmap API's
> >
> >      Instead of using writel and readl use regmap API which makes
> >      the driver maintainability easier.
> >
> >      Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> >      Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> >
> > commit 9445ccb3714c78c26a3a25fafed4d9d965080431
> > Author: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> > Date:   Wed Apr 17 13:58:37 2019 +0200
> >
> >      staging: mt7621-pci-phy: add quirks for 'E2' revision using
> > 'soc_device_attribute'
> >
> >      Depending on revision of the chip, 'mt7621_bypass_pipe_rst' function
> >      must be executed. Add better support for this using 'soc_device_match'
> >      in driver probe function.
> >
> >      Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> >      Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> >
> > So, I added a quirk for E2 revision of the board as I was suggested to
> > do by the phy
> > tree maintainer, and this is the only place I can think the problem could be.
>
> I took the pci-mt7621.c and pci-mt7621-phy.c from a linux-5.2-rc1,
> which has both those commits. Same behavior, PCI probing locks up
> kernel in mt7621_pci_phy_power_off().
>

I see, thanks for testing these two also.

>
> > I think all te changes before this was properly tested by Neil and
> > results in a working
> > PCI.
>
> Not sure what board Neil is using.
> I am using a Digi/EX15 and I also tried a Digi/TX54 (very different
> boards, very different designs).

Neil's board Is a gnubee board.

>
>
> >> Copying in the working linux-4.20 pci-mt7621.c into place on
> >> linux-5.1 results in a working PCI bus also. I have 2 very different
> >> MT7621 based boards, and they both exhibit this same problem.
> >>
> >> I tried bisecting that back to find the problem commit.
> >> It was not at all easy with quite a few of the mt7621 PCI related
> >> patches not building in isolation while bisecting. But ultimately
> >> I got to commit 745eeeac68d7 ("staging: mt7621-pci: factor out
> >> 'mt7621_pcie_enable_port' function").
> >
> > Sorry for your time in this and for the problems with isolation patches. I'll
>
> FWIW, I do like your changes, they really clean up that code a lot.

Thanks, is cleaner now and should be ready to be upstreamed but it
seems has problems
on some boards yet :)). Let's make it working properly again in all
scenarios :-).

>
>
> > do my best from now to this don't happen again. The problem commit
> > you are pointing out here had problems at first because depending on the
> > revision of the chip the reset lines are inverted but this was properly fixed in
> > this other commit:
> >
> > commit e51844bf825169024e0c743a92cf264e27f2366f
> > Author: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> > Date:   Sat Nov 24 18:54:54 2018 +0100
> >
> >      staging: mt7621-pci: fix reset lines for each pcie port
> >
> >      Depending of chip revision reset lines are inverted. It is also
> >      necessary to read PCIE_FTS_NUM register before enabling the phy.
> >      Hence update the code to achieve this.
> >
> >      Fixes: 745eeeac68d7 ("staging: mt7621-pci: factor out
> > 'mt7621_pcie_enable_port' function")
> >      Reported-by: NeilBrown <neil@brown.name>
> >      Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
> >      Tested-by: NeilBrown <neil@brown.name>
> >      Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> >
> >>
> >> Any idea what might be going wrong here?
> >
> > So I do believe that the problem could be with last phy changes I am
> > pointed out first with quirks
> > for your revision don't be executing function 'mt7621_bypass_pipe_rst'
> > inside 'mt7621_pci_phy_init'.
> > You should take care of quirks if revision of the board is 'E2'.
>
> I checked that revision register on my boards, my MT7621 parts report
> a revision ID of 0x00030103 - so that means they do not match the
> E2 ID. Tracing the code they do not go through mt7621_bypass_pipe_rst().
> And that is true even in the older linux-4.20 driver, it does not
> execute that bypass routine.
>
> Although the lock up is bad, the real issue is that the probe of
> PCI0 is not finding any cards - and it should.
>
> I tried restoring some of the probe code from linux-4.20 and had some
> inconsistent success. But I haven't been able to point to exactly what
> the problem is.

There are some big changes between 4.20 and 5.x. One is the use of PERST_N
instead of using gpio. This PERT_N stuff is used now on enable ports
assuming the
link of PCI is properly detected after enabling the phy. And it seems
it is not according to
your dmesg traces. The previous 4.20 code used gpio before this was done.

This code is the one I am referring:

/* Use GPIO control instead of PERST_N */
*(unsigned int *)(0xbe000620) |= BIT(19) | BIT(8) | BIT(7); // set DATA
mdelay(1000);

I assume reset lines on your device tree are properly set up which is
other of the big changes here: use
reset lines instead of that hardcoding stuff. Also, the
mt7621_reset_port routine is also using msleep(100)
but maybe you can try a bigger value and change it into a mdelay, to
see if that changes anything.

Other big change is to use the new phy driver but I think the problem
seems to be related with resets.

>
> Regards
> Greg

Please, don't hesitate to ask me whatever you need to.

Hope this helps.

Best regards,
    Sergio Paracuellos
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 22/5/19 4:27 pm, Sergio Paracuellos wrote:
[snip]
> There are some big changes between 4.20 and 5.x. One is the use of PERST_N
> instead of using gpio. This PERT_N stuff is used now on enable ports
> assuming the
> link of PCI is properly detected after enabling the phy. And it seems
> it is not according to
> your dmesg traces. The previous 4.20 code used gpio before this was done.
> 
> This code is the one I am referring:
> 
> /* Use GPIO control instead of PERST_N */
> *(unsigned int *)(0xbe000620) |= BIT(19) | BIT(8) | BIT(7); // set DATA
> mdelay(1000);

I have been looking closely at those, wondering why the old code
drove that PERST line as a GPIO instead of using the built-in behavior.
(I have ignored bits 7 and 8 here since they are control of UART 3)


> I assume reset lines on your device tree are properly set up which is
> other of the big changes here: use
> reset lines instead of that hardcoding stuff. Also, the
> mt7621_reset_port routine is also using msleep(100)
> but maybe you can try a bigger value and change it into a mdelay, to
> see if that changes anything.

I see the reset line configuration in the pcie section of mt7621.dtsi,
is there any others absolutely required here?  I couldn't see the
gbpc1.dts devicetree do anything else with pcie - othe than enable it.
My device tree for the EX15 is similar in that regard.

I tried a couple of things with interesting results.

1. I made sure that the PERST_N line is set for PCIe operation (not GPIO).
    I forced it with:

        *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);

    I checked bits 10 and 11 at kernel PCI init and they were 00 anyway.
    So PERST_N was definitely in PCIe reset mode. No change in behavior,
    

2. I forced a GPIO style reset of that PERST line (using GPIO19) and got
    the following result on kernel boot:

     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
     mt7621-pci 1e140000.pcie: Initiating port 1 failed
     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
     mt7621-pci 1e140000.pcie: Initiating port 2 failed
     mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N
     mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
     mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
     pci_bus 0000:00: root bus resource [io  0xffffffff]
     pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
     pci_bus 0000:00: root bus resource [bus 00-ff]

And the system continued on to fully boot. So it looks like it thinks
pcie0 is active. Better, but the PCI bus probe didn't find any of the
devices it should have.

I inserted the quick hack code to do this at the top of mt7621_pcie_init_ports()
and it looked like this:

         /* Force PERST PCIe line into GPIO mode */
         *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
         *(unsigned int *)(0xbe000060) |=  BIT(10);
         mdelay(100);

         *(unsigned int *)(0xbe000600) |= BIT(19); // use GPIO19 (PERST_N/)
         mdelay(100);
         *(unsigned int *)(0xbe000620) &= ~(BIT(19)); // clear DATA
         mdelay(1000);

         /* Use GPIO control instead of PERST_N */
         *(unsigned int *)(0xbe000620) |= BIT(19); // set DATA
         mdelay(1000);


Regards
Greg


> Other big change is to use the new phy driver but I think the problem
> seems to be related with resets.
> 
>>
>> Regards
>> Greg
> 
> Please, don't hesitate to ask me whatever you need to.
> 
> Hope this helps.
> 
> Best regards,
>      Sergio Paracuellos
> 
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

Hi Greg,


On Thu, May 23, 2019 at 4:11 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> On 22/5/19 4:27 pm, Sergio Paracuellos wrote:
> [snip]
> > There are some big changes between 4.20 and 5.x. One is the use of PERST_N
> > instead of using gpio. This PERT_N stuff is used now on enable ports
> > assuming the
> > link of PCI is properly detected after enabling the phy. And it seems
> > it is not according to
> > your dmesg traces. The previous 4.20 code used gpio before this was done.
> >
> > This code is the one I am referring:
> >
> > /* Use GPIO control instead of PERST_N */
> > *(unsigned int *)(0xbe000620) |= BIT(19) | BIT(8) | BIT(7); // set DATA
> > mdelay(1000);
>
> I have been looking closely at those, wondering why the old code
> drove that PERST line as a GPIO instead of using the built-in behavior.
> (I have ignored bits 7 and 8 here since they are control of UART 3)

Yes, this was also at first one of my big concerns so I tried to change into
to use builtin behaviour (which is much more cleaner) and when the
code was tested
it worked. It seems it is not valid for every board.

>
>
> > I assume reset lines on your device tree are properly set up which is
> > other of the big changes here: use
> > reset lines instead of that hardcoding stuff. Also, the
> > mt7621_reset_port routine is also using msleep(100)
> > but maybe you can try a bigger value and change it into a mdelay, to
> > see if that changes anything.
>
> I see the reset line configuration in the pcie section of mt7621.dtsi,
> is there any others absolutely required here?  I couldn't see the
> gbpc1.dts devicetree do anything else with pcie - othe than enable it.
> My device tree for the EX15 is similar in that regard.
>
> I tried a couple of things with interesting results.
>
> 1. I made sure that the PERST_N line is set for PCIe operation (not GPIO).
>     I forced it with:
>
>         *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
>
>     I checked bits 10 and 11 at kernel PCI init and they were 00 anyway.
>     So PERST_N was definitely in PCIe reset mode. No change in behavior,
>
>
> 2. I forced a GPIO style reset of that PERST line (using GPIO19) and got
>     the following result on kernel boot:
>
>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>      mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>      mt7621-pci 1e140000.pcie: Initiating port 1 failed
>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>      mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>      mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>      mt7621-pci 1e140000.pcie: Initiating port 2 failed
>      mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N

This line seems to be the problem. When ports are init, (and with your
changes seems the are
init properly), the ports with pcie link are stored into a list to be
enabled afterwards. This code is
located into 'mt7621_pcie_enable_ports' which call simple
'mt7621_pcie_enable_port' to enable each port
on the list. In this process it uses the PERS_N built-in register
deasserting and asserting it. If enabling fails
(and this is ypour case now) the port is removed from the list and it
is not properly set up. You should try to
comment this code:

/* assert port PERST_N */
val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
val |= PCIE_PORT_PERST(slot);
pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);

/* de-assert port PERST_N */
val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
val &= ~PCIE_PORT_PERST(slot);
pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);

/* 100ms timeout value should be enough for Gen1 training */
err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
val, !!(val & PCIE_PORT_LINKUP),
20, 100 * USEC_PER_MSEC);
if (err)
return -ETIMEDOUT;

because on enabling, it seems it is getting ETIMEOUT and hence the
message '  mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N'.
Commenting
this code should end up into a properly configured pci?

>      mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>      mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>      pci_bus 0000:00: root bus resource [io  0xffffffff]
>      pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>      pci_bus 0000:00: root bus resource [bus 00-ff]
>
> And the system continued on to fully boot. So it looks like it thinks
> pcie0 is active. Better, but the PCI bus probe didn't find any of the
> devices it should have.

Yes, that seems what is happening because of my explanation above.

>
> I inserted the quick hack code to do this at the top of mt7621_pcie_init_ports()
> and it looked like this:
>
>          /* Force PERST PCIe line into GPIO mode */
>          *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
>          *(unsigned int *)(0xbe000060) |=  BIT(10);
>          mdelay(100);
>
>          *(unsigned int *)(0xbe000600) |= BIT(19); // use GPIO19 (PERST_N/)
>          mdelay(100);
>          *(unsigned int *)(0xbe000620) &= ~(BIT(19)); // clear DATA
>          mdelay(1000);
>
>          /* Use GPIO control instead of PERST_N */
>          *(unsigned int *)(0xbe000620) |= BIT(19); // set DATA
>          mdelay(1000);
>
>

I really hate the gpio way of doing this. If this works we have to
think in how to achieve this in a cleaner way :))

> Regards
> Greg

Thanks for your effort in this.

Best regards,
    Sergio Paracuellos
>
>
> > Other big change is to use the new phy driver but I think the problem
> > seems to be related with resets.
> >
> >>
> >> Regards
> >> Greg
> >
> > Please, don't hesitate to ask me whatever you need to.
> >
> > Hope this helps.
> >
> > Best regards,
> >      Sergio Paracuellos
> >
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 23/5/19 3:26 pm, Sergio Paracuellos wrote:
> On Thu, May 23, 2019 at 4:11 AM Greg Ungerer <gerg@kernel.org> wrote:
>> On 22/5/19 4:27 pm, Sergio Paracuellos wrote:
>> [snip]
>>> There are some big changes between 4.20 and 5.x. One is the use of PERST_N
>>> instead of using gpio. This PERT_N stuff is used now on enable ports
>>> assuming the
>>> link of PCI is properly detected after enabling the phy. And it seems
>>> it is not according to
>>> your dmesg traces. The previous 4.20 code used gpio before this was done.
>>>
>>> This code is the one I am referring:
>>>
>>> /* Use GPIO control instead of PERST_N */
>>> *(unsigned int *)(0xbe000620) |= BIT(19) | BIT(8) | BIT(7); // set DATA
>>> mdelay(1000);
>>
>> I have been looking closely at those, wondering why the old code
>> drove that PERST line as a GPIO instead of using the built-in behavior.
>> (I have ignored bits 7 and 8 here since they are control of UART 3)
> 
> Yes, this was also at first one of my big concerns so I tried to change into
> to use builtin behaviour (which is much more cleaner) and when the
> code was tested
> it worked. It seems it is not valid for every board.
> 
>>
>>
>>> I assume reset lines on your device tree are properly set up which is
>>> other of the big changes here: use
>>> reset lines instead of that hardcoding stuff. Also, the
>>> mt7621_reset_port routine is also using msleep(100)
>>> but maybe you can try a bigger value and change it into a mdelay, to
>>> see if that changes anything.
>>
>> I see the reset line configuration in the pcie section of mt7621.dtsi,
>> is there any others absolutely required here?  I couldn't see the
>> gbpc1.dts devicetree do anything else with pcie - othe than enable it.
>> My device tree for the EX15 is similar in that regard.
>>
>> I tried a couple of things with interesting results.
>>
>> 1. I made sure that the PERST_N line is set for PCIe operation (not GPIO).
>>      I forced it with:
>>
>>          *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
>>
>>      I checked bits 10 and 11 at kernel PCI init and they were 00 anyway.
>>      So PERST_N was definitely in PCIe reset mode. No change in behavior,
>>
>>
>> 2. I forced a GPIO style reset of that PERST line (using GPIO19) and got
>>      the following result on kernel boot:
>>
>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>       mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>       mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>       mt7621-pci 1e140000.pcie: Initiating port 1 failed
>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>       mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>       mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>       mt7621-pci 1e140000.pcie: Initiating port 2 failed
>>       mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N
> 
> This line seems to be the problem. When ports are init, (and with your
> changes seems the are
> init properly), the ports with pcie link are stored into a list to be
> enabled afterwards. This code is
> located into 'mt7621_pcie_enable_ports' which call simple
> 'mt7621_pcie_enable_port' to enable each port
> on the list. In this process it uses the PERS_N built-in register
> deasserting and asserting it. If enabling fails
> (and this is ypour case now) the port is removed from the list and it
> is not properly set up. You should try to
> comment this code:
> 
> /* assert port PERST_N */
> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
> val |= PCIE_PORT_PERST(slot);
> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
> 
> /* de-assert port PERST_N */
> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
> val &= ~PCIE_PORT_PERST(slot);
> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
> 
> /* 100ms timeout value should be enough for Gen1 training */
> err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
> val, !!(val & PCIE_PORT_LINKUP),
> 20, 100 * USEC_PER_MSEC);
> if (err)
> return -ETIMEDOUT;
> 
> because on enabling, it seems it is getting ETIMEOUT and hence the
> message '  mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N'.
> Commenting
> this code should end up into a properly configured pci?

No, unfortunately it doesn't. It does show PCIE0 enabled now though:

mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
mt7621-pci 1e140000.pcie: Initiating port 1 failed
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
mt7621-pci 1e140000.pcie: Initiating port 2 failed
mt7621-pci 1e140000.pcie: PCIE0 enabled
mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0xffffffff]
pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
pci_bus 0000:00: root bus resource [bus 00-ff]

And again no devices are found on the PCI bus.
(System did still boot too).

I'll keep digging.

Thanks
Greg


> 
>>       mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>       mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>       pci_bus 0000:00: root bus resource [io  0xffffffff]
>>       pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>       pci_bus 0000:00: root bus resource [bus 00-ff]
>>
>> And the system continued on to fully boot. So it looks like it thinks
>> pcie0 is active. Better, but the PCI bus probe didn't find any of the
>> devices it should have.
> 
> Yes, that seems what is happening because of my explanation above.
> 
>>
>> I inserted the quick hack code to do this at the top of mt7621_pcie_init_ports()
>> and it looked like this:
>>
>>           /* Force PERST PCIe line into GPIO mode */
>>           *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
>>           *(unsigned int *)(0xbe000060) |=  BIT(10);
>>           mdelay(100);
>>
>>           *(unsigned int *)(0xbe000600) |= BIT(19); // use GPIO19 (PERST_N/)
>>           mdelay(100);
>>           *(unsigned int *)(0xbe000620) &= ~(BIT(19)); // clear DATA
>>           mdelay(1000);
>>
>>           /* Use GPIO control instead of PERST_N */
>>           *(unsigned int *)(0xbe000620) |= BIT(19); // set DATA
>>           mdelay(1000);
>>
>>
> 
> I really hate the gpio way of doing this. If this works we have to
> think in how to achieve this in a cleaner way :))
> 
>> Regards
>> Greg
> 
> Thanks for your effort in this.
> 
> Best regards,
>      Sergio Paracuellos
>>
>>
>>> Other big change is to use the new phy driver but I think the problem
>>> seems to be related with resets.
>>>
>>>>
>>>> Regards
>>>> Greg
>>>
>>> Please, don't hesitate to ask me whatever you need to.
>>>
>>> Hope this helps.
>>>
>>> Best regards,
>>>       Sergio Paracuellos
>>>
> 
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

Hi Greg,

On Fri, May 24, 2019 at 2:35 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> On 23/5/19 3:26 pm, Sergio Paracuellos wrote:
> > On Thu, May 23, 2019 at 4:11 AM Greg Ungerer <gerg@kernel.org> wrote:
> >> On 22/5/19 4:27 pm, Sergio Paracuellos wrote:
> >> [snip]
> >>> There are some big changes between 4.20 and 5.x. One is the use of PERST_N
> >>> instead of using gpio. This PERT_N stuff is used now on enable ports
> >>> assuming the
> >>> link of PCI is properly detected after enabling the phy. And it seems
> >>> it is not according to
> >>> your dmesg traces. The previous 4.20 code used gpio before this was done.
> >>>
> >>> This code is the one I am referring:
> >>>
> >>> /* Use GPIO control instead of PERST_N */
> >>> *(unsigned int *)(0xbe000620) |= BIT(19) | BIT(8) | BIT(7); // set DATA
> >>> mdelay(1000);
> >>
> >> I have been looking closely at those, wondering why the old code
> >> drove that PERST line as a GPIO instead of using the built-in behavior.
> >> (I have ignored bits 7 and 8 here since they are control of UART 3)
> >
> > Yes, this was also at first one of my big concerns so I tried to change into
> > to use builtin behaviour (which is much more cleaner) and when the
> > code was tested
> > it worked. It seems it is not valid for every board.
> >
> >>
> >>
> >>> I assume reset lines on your device tree are properly set up which is
> >>> other of the big changes here: use
> >>> reset lines instead of that hardcoding stuff. Also, the
> >>> mt7621_reset_port routine is also using msleep(100)
> >>> but maybe you can try a bigger value and change it into a mdelay, to
> >>> see if that changes anything.
> >>
> >> I see the reset line configuration in the pcie section of mt7621.dtsi,
> >> is there any others absolutely required here?  I couldn't see the
> >> gbpc1.dts devicetree do anything else with pcie - othe than enable it.
> >> My device tree for the EX15 is similar in that regard.
> >>
> >> I tried a couple of things with interesting results.
> >>
> >> 1. I made sure that the PERST_N line is set for PCIe operation (not GPIO).
> >>      I forced it with:
> >>
> >>          *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
> >>
> >>      I checked bits 10 and 11 at kernel PCI init and they were 00 anyway.
> >>      So PERST_N was definitely in PCIe reset mode. No change in behavior,
> >>
> >>
> >> 2. I forced a GPIO style reset of that PERST line (using GPIO19) and got
> >>      the following result on kernel boot:
> >>
> >>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>       mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>       mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>       mt7621-pci 1e140000.pcie: Initiating port 1 failed
> >>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>       mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>       mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>       mt7621-pci 1e140000.pcie: Initiating port 2 failed
> >>       mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N
> >
> > This line seems to be the problem. When ports are init, (and with your
> > changes seems the are
> > init properly), the ports with pcie link are stored into a list to be
> > enabled afterwards. This code is
> > located into 'mt7621_pcie_enable_ports' which call simple
> > 'mt7621_pcie_enable_port' to enable each port
> > on the list. In this process it uses the PERS_N built-in register
> > deasserting and asserting it. If enabling fails
> > (and this is ypour case now) the port is removed from the list and it
> > is not properly set up. You should try to
> > comment this code:
> >
> > /* assert port PERST_N */
> > val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
> > val |= PCIE_PORT_PERST(slot);
> > pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
> >
> > /* de-assert port PERST_N */
> > val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
> > val &= ~PCIE_PORT_PERST(slot);
> > pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
> >
> > /* 100ms timeout value should be enough for Gen1 training */
> > err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
> > val, !!(val & PCIE_PORT_LINKUP),
> > 20, 100 * USEC_PER_MSEC);
> > if (err)
> > return -ETIMEDOUT;
> >
> > because on enabling, it seems it is getting ETIMEOUT and hence the
> > message '  mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N'.
> > Commenting
> > this code should end up into a properly configured pci?
>
> No, unfortunately it doesn't. It does show PCIE0 enabled now though:

That is a surprise :(

>
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> mt7621-pci 1e140000.pcie: Initiating port 1 failed
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> mt7621-pci 1e140000.pcie: Initiating port 2 failed
> mt7621-pci 1e140000.pcie: PCIE0 enabled
> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> pci_bus 0000:00: root bus resource [io  0xffffffff]
> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> pci_bus 0000:00: root bus resource [bus 00-ff]


>
> And again no devices are found on the PCI bus.
> (System did still boot too).

Looking to your original trace of linux-4.20 working the init traces
are pretty much the same... I don't really know what could be
happening there. Root resources
are correct, virtual bridge seems to be detected, the next should be
to reconfigure resources of
the bridge and this is done by the pci kernel apis.

Can you check that "mt7621_pcie_init_virtual_bridges" is getting link
up and configuring bridges
correctly?

>
> I'll keep digging.

Thanks, really appreciate it.

>
> Thanks
> Greg

Best regards,
    Sergio Paracuellos

>
>
> >
> >>       mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >>       mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >>       pci_bus 0000:00: root bus resource [io  0xffffffff]
> >>       pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >>       pci_bus 0000:00: root bus resource [bus 00-ff]
> >>
> >> And the system continued on to fully boot. So it looks like it thinks
> >> pcie0 is active. Better, but the PCI bus probe didn't find any of the
> >> devices it should have.
> >
> > Yes, that seems what is happening because of my explanation above.
> >
> >>
> >> I inserted the quick hack code to do this at the top of mt7621_pcie_init_ports()
> >> and it looked like this:
> >>
> >>           /* Force PERST PCIe line into GPIO mode */
> >>           *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
> >>           *(unsigned int *)(0xbe000060) |=  BIT(10);
> >>           mdelay(100);
> >>
> >>           *(unsigned int *)(0xbe000600) |= BIT(19); // use GPIO19 (PERST_N/)
> >>           mdelay(100);
> >>           *(unsigned int *)(0xbe000620) &= ~(BIT(19)); // clear DATA
> >>           mdelay(1000);
> >>
> >>           /* Use GPIO control instead of PERST_N */
> >>           *(unsigned int *)(0xbe000620) |= BIT(19); // set DATA
> >>           mdelay(1000);
> >>
> >>
> >
> > I really hate the gpio way of doing this. If this works we have to
> > think in how to achieve this in a cleaner way :))
> >
> >> Regards
> >> Greg
> >
> > Thanks for your effort in this.
> >
> > Best regards,
> >      Sergio Paracuellos
> >>
> >>
> >>> Other big change is to use the new phy driver but I think the problem
> >>> seems to be related with resets.
> >>>
> >>>>
> >>>> Regards
> >>>> Greg
> >>>
> >>> Please, don't hesitate to ask me whatever you need to.
> >>>
> >>> Hope this helps.
> >>>
> >>> Best regards,
> >>>       Sergio Paracuellos
> >>>
> >
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 24/5/19 3:35 pm, Sergio Paracuellos wrote:
> On Fri, May 24, 2019 at 2:35 AM Greg Ungerer <gerg@kernel.org> wrote:
>> On 23/5/19 3:26 pm, Sergio Paracuellos wrote:
>>> On Thu, May 23, 2019 at 4:11 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>> On 22/5/19 4:27 pm, Sergio Paracuellos wrote:
>>>> [snip]
>>>>> There are some big changes between 4.20 and 5.x. One is the use of PERST_N
>>>>> instead of using gpio. This PERT_N stuff is used now on enable ports
>>>>> assuming the
>>>>> link of PCI is properly detected after enabling the phy. And it seems
>>>>> it is not according to
>>>>> your dmesg traces. The previous 4.20 code used gpio before this was done.
>>>>>
>>>>> This code is the one I am referring:
>>>>>
>>>>> /* Use GPIO control instead of PERST_N */
>>>>> *(unsigned int *)(0xbe000620) |= BIT(19) | BIT(8) | BIT(7); // set DATA
>>>>> mdelay(1000);
>>>>
>>>> I have been looking closely at those, wondering why the old code
>>>> drove that PERST line as a GPIO instead of using the built-in behavior.
>>>> (I have ignored bits 7 and 8 here since they are control of UART 3)
>>>
>>> Yes, this was also at first one of my big concerns so I tried to change into
>>> to use builtin behaviour (which is much more cleaner) and when the
>>> code was tested
>>> it worked. It seems it is not valid for every board.
>>>
>>>>
>>>>
>>>>> I assume reset lines on your device tree are properly set up which is
>>>>> other of the big changes here: use
>>>>> reset lines instead of that hardcoding stuff. Also, the
>>>>> mt7621_reset_port routine is also using msleep(100)
>>>>> but maybe you can try a bigger value and change it into a mdelay, to
>>>>> see if that changes anything.
>>>>
>>>> I see the reset line configuration in the pcie section of mt7621.dtsi,
>>>> is there any others absolutely required here?  I couldn't see the
>>>> gbpc1.dts devicetree do anything else with pcie - othe than enable it.
>>>> My device tree for the EX15 is similar in that regard.
>>>>
>>>> I tried a couple of things with interesting results.
>>>>
>>>> 1. I made sure that the PERST_N line is set for PCIe operation (not GPIO).
>>>>       I forced it with:
>>>>
>>>>           *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
>>>>
>>>>       I checked bits 10 and 11 at kernel PCI init and they were 00 anyway.
>>>>       So PERST_N was definitely in PCIe reset mode. No change in behavior,
>>>>
>>>>
>>>> 2. I forced a GPIO style reset of that PERST line (using GPIO19) and got
>>>>       the following result on kernel boot:
>>>>
>>>>        mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>>        mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>        mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>>        mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>        mt7621-pci 1e140000.pcie: Initiating port 1 failed
>>>>        mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>>        mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>        mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>>        mt7621-pci 1e140000.pcie: Initiating port 2 failed
>>>>        mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N
>>>
>>> This line seems to be the problem. When ports are init, (and with your
>>> changes seems the are
>>> init properly), the ports with pcie link are stored into a list to be
>>> enabled afterwards. This code is
>>> located into 'mt7621_pcie_enable_ports' which call simple
>>> 'mt7621_pcie_enable_port' to enable each port
>>> on the list. In this process it uses the PERS_N built-in register
>>> deasserting and asserting it. If enabling fails
>>> (and this is ypour case now) the port is removed from the list and it
>>> is not properly set up. You should try to
>>> comment this code:
>>>
>>> /* assert port PERST_N */
>>> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
>>> val |= PCIE_PORT_PERST(slot);
>>> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
>>>
>>> /* de-assert port PERST_N */
>>> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
>>> val &= ~PCIE_PORT_PERST(slot);
>>> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
>>>
>>> /* 100ms timeout value should be enough for Gen1 training */
>>> err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
>>> val, !!(val & PCIE_PORT_LINKUP),
>>> 20, 100 * USEC_PER_MSEC);
>>> if (err)
>>> return -ETIMEDOUT;
>>>
>>> because on enabling, it seems it is getting ETIMEOUT and hence the
>>> message '  mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N'.
>>> Commenting
>>> this code should end up into a properly configured pci?
>>
>> No, unfortunately it doesn't. It does show PCIE0 enabled now though:
> 
> That is a surprise :(
> 
>>
>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>> mt7621-pci 1e140000.pcie: Initiating port 1 failed
>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>> mt7621-pci 1e140000.pcie: Initiating port 2 failed
>> mt7621-pci 1e140000.pcie: PCIE0 enabled
>> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>> pci_bus 0000:00: root bus resource [io  0xffffffff]
>> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>> pci_bus 0000:00: root bus resource [bus 00-ff]
> 
> 
>>
>> And again no devices are found on the PCI bus.
>> (System did still boot too).
> 
> Looking to your original trace of linux-4.20 working the init traces
> are pretty much the same... I don't really know what could be
> happening there. Root resources
> are correct, virtual bridge seems to be detected, the next should be
> to reconfigure resources of
> the bridge and this is done by the pci kernel apis.
> 
> Can you check that "mt7621_pcie_init_virtual_bridges" is getting link
> up and configuring bridges
> correctly?

Yes, it does get link there. It sees pcie_link_status as 0x1, so its getting
through that.

I threw a bit of trace in to see where we end up losing the ability to
read correct config data from slot 0 (my only valid slot). It gets to
the "err_no_link_up:" label for port/slot 2 still being able to read config
space, but then after executing the phy_power_off() and phy_exit()
calls for that port/slot we can no longer read config for slot 0.

If I comment out the code in phy_power_off() and phy_exit() so they
return doing nothing then I get further:

mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
mt7621-pci 1e140000.pcie: Initiating port 1 failed
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
mt7621-pci 1e140000.pcie: Initiating port 2 failed
mt7621-pci 1e140000.pcie: PCIE0 enabled
mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0xffffffff]
pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
pci_bus 0000:00: root bus resource [bus 00-ff]
pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:00.0: PCI bridge to [bus 01-ff]
pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
pci 0000:00:00.0: PCI bridge to [bus 01]
pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
pcieport 0000:00:00.0: enabling device (0004 -> 0006)


So devices found, but interrupt setup failed for some reason.
I have an atheros PCIe WIFI device on that bus which is detected, but
the interrupt failure means it still doesn't actually work.

Regards
Greg


>> I'll keep digging.
> 
> Thanks, really appreciate it.
> 
>>
>> Thanks
>> Greg
> 
> Best regards,
>      Sergio Paracuellos
> 
>>
>>
>>>
>>>>        mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>>>        mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>>>        pci_bus 0000:00: root bus resource [io  0xffffffff]
>>>>        pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>>>        pci_bus 0000:00: root bus resource [bus 00-ff]
>>>>
>>>> And the system continued on to fully boot. So it looks like it thinks
>>>> pcie0 is active. Better, but the PCI bus probe didn't find any of the
>>>> devices it should have.
>>>
>>> Yes, that seems what is happening because of my explanation above.
>>>
>>>>
>>>> I inserted the quick hack code to do this at the top of mt7621_pcie_init_ports()
>>>> and it looked like this:
>>>>
>>>>            /* Force PERST PCIe line into GPIO mode */
>>>>            *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
>>>>            *(unsigned int *)(0xbe000060) |=  BIT(10);
>>>>            mdelay(100);
>>>>
>>>>            *(unsigned int *)(0xbe000600) |= BIT(19); // use GPIO19 (PERST_N/)
>>>>            mdelay(100);
>>>>            *(unsigned int *)(0xbe000620) &= ~(BIT(19)); // clear DATA
>>>>            mdelay(1000);
>>>>
>>>>            /* Use GPIO control instead of PERST_N */
>>>>            *(unsigned int *)(0xbe000620) |= BIT(19); // set DATA
>>>>            mdelay(1000);
>>>>
>>>>
>>>
>>> I really hate the gpio way of doing this. If this works we have to
>>> think in how to achieve this in a cleaner way :))
>>>
>>>> Regards
>>>> Greg
>>>
>>> Thanks for your effort in this.
>>>
>>> Best regards,
>>>       Sergio Paracuellos
>>>>
>>>>
>>>>> Other big change is to use the new phy driver but I think the problem
>>>>> seems to be related with resets.
>>>>>
>>>>>>
>>>>>> Regards
>>>>>> Greg
>>>>>
>>>>> Please, don't hesitate to ask me whatever you need to.
>>>>>
>>>>> Hope this helps.
>>>>>
>>>>> Best regards,
>>>>>        Sergio Paracuellos
>>>>>
>>>
> 
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

Hi Greg,

On Mon, May 27, 2019 at 6:37 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> On 24/5/19 3:35 pm, Sergio Paracuellos wrote:
> > On Fri, May 24, 2019 at 2:35 AM Greg Ungerer <gerg@kernel.org> wrote:
> >> On 23/5/19 3:26 pm, Sergio Paracuellos wrote:
> >>> On Thu, May 23, 2019 at 4:11 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>> On 22/5/19 4:27 pm, Sergio Paracuellos wrote:
> >>>> [snip]
> >>>>> There are some big changes between 4.20 and 5.x. One is the use of PERST_N
> >>>>> instead of using gpio. This PERT_N stuff is used now on enable ports
> >>>>> assuming the
> >>>>> link of PCI is properly detected after enabling the phy. And it seems
> >>>>> it is not according to
> >>>>> your dmesg traces. The previous 4.20 code used gpio before this was done.
> >>>>>
> >>>>> This code is the one I am referring:
> >>>>>
> >>>>> /* Use GPIO control instead of PERST_N */
> >>>>> *(unsigned int *)(0xbe000620) |= BIT(19) | BIT(8) | BIT(7); // set DATA
> >>>>> mdelay(1000);
> >>>>
> >>>> I have been looking closely at those, wondering why the old code
> >>>> drove that PERST line as a GPIO instead of using the built-in behavior.
> >>>> (I have ignored bits 7 and 8 here since they are control of UART 3)
> >>>
> >>> Yes, this was also at first one of my big concerns so I tried to change into
> >>> to use builtin behaviour (which is much more cleaner) and when the
> >>> code was tested
> >>> it worked. It seems it is not valid for every board.
> >>>
> >>>>
> >>>>
> >>>>> I assume reset lines on your device tree are properly set up which is
> >>>>> other of the big changes here: use
> >>>>> reset lines instead of that hardcoding stuff. Also, the
> >>>>> mt7621_reset_port routine is also using msleep(100)
> >>>>> but maybe you can try a bigger value and change it into a mdelay, to
> >>>>> see if that changes anything.
> >>>>
> >>>> I see the reset line configuration in the pcie section of mt7621.dtsi,
> >>>> is there any others absolutely required here?  I couldn't see the
> >>>> gbpc1.dts devicetree do anything else with pcie - othe than enable it.
> >>>> My device tree for the EX15 is similar in that regard.
> >>>>
> >>>> I tried a couple of things with interesting results.
> >>>>
> >>>> 1. I made sure that the PERST_N line is set for PCIe operation (not GPIO).
> >>>>       I forced it with:
> >>>>
> >>>>           *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
> >>>>
> >>>>       I checked bits 10 and 11 at kernel PCI init and they were 00 anyway.
> >>>>       So PERST_N was definitely in PCIe reset mode. No change in behavior,
> >>>>
> >>>>
> >>>> 2. I forced a GPIO style reset of that PERST line (using GPIO19) and got
> >>>>       the following result on kernel boot:
> >>>>
> >>>>        mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>>        mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>        mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>>        mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>>        mt7621-pci 1e140000.pcie: Initiating port 1 failed
> >>>>        mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>>        mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>>        mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>>        mt7621-pci 1e140000.pcie: Initiating port 2 failed
> >>>>        mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N
> >>>
> >>> This line seems to be the problem. When ports are init, (and with your
> >>> changes seems the are
> >>> init properly), the ports with pcie link are stored into a list to be
> >>> enabled afterwards. This code is
> >>> located into 'mt7621_pcie_enable_ports' which call simple
> >>> 'mt7621_pcie_enable_port' to enable each port
> >>> on the list. In this process it uses the PERS_N built-in register
> >>> deasserting and asserting it. If enabling fails
> >>> (and this is ypour case now) the port is removed from the list and it
> >>> is not properly set up. You should try to
> >>> comment this code:
> >>>
> >>> /* assert port PERST_N */
> >>> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
> >>> val |= PCIE_PORT_PERST(slot);
> >>> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
> >>>
> >>> /* de-assert port PERST_N */
> >>> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
> >>> val &= ~PCIE_PORT_PERST(slot);
> >>> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
> >>>
> >>> /* 100ms timeout value should be enough for Gen1 training */
> >>> err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
> >>> val, !!(val & PCIE_PORT_LINKUP),
> >>> 20, 100 * USEC_PER_MSEC);
> >>> if (err)
> >>> return -ETIMEDOUT;
> >>>
> >>> because on enabling, it seems it is getting ETIMEOUT and hence the
> >>> message '  mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N'.
> >>> Commenting
> >>> this code should end up into a properly configured pci?
> >>
> >> No, unfortunately it doesn't. It does show PCIE0 enabled now though:
> >
> > That is a surprise :(
> >
> >>
> >> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >> mt7621-pci 1e140000.pcie: Initiating port 1 failed
> >> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >> mt7621-pci 1e140000.pcie: Initiating port 2 failed
> >> mt7621-pci 1e140000.pcie: PCIE0 enabled
> >> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >> pci_bus 0000:00: root bus resource [io  0xffffffff]
> >> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >> pci_bus 0000:00: root bus resource [bus 00-ff]
> >
> >
> >>
> >> And again no devices are found on the PCI bus.
> >> (System did still boot too).
> >
> > Looking to your original trace of linux-4.20 working the init traces
> > are pretty much the same... I don't really know what could be
> > happening there. Root resources
> > are correct, virtual bridge seems to be detected, the next should be
> > to reconfigure resources of
> > the bridge and this is done by the pci kernel apis.
> >
> > Can you check that "mt7621_pcie_init_virtual_bridges" is getting link
> > up and configuring bridges
> > correctly?
>
> Yes, it does get link there. It sees pcie_link_status as 0x1, so its getting
> through that.
>
> I threw a bit of trace in to see where we end up losing the ability to
> read correct config data from slot 0 (my only valid slot). It gets to
> the "err_no_link_up:" label for port/slot 2 still being able to read config
> space, but then after executing the phy_power_off() and phy_exit()
> calls for that port/slot we can no longer read config for slot 0.

Mmmm. I see. So phy instances for port 0 and 2 are different instances
of the phy, so it should not
have problems for the power_off function. Looking again to the version
which is in the 5.0 linux (but not in the last changes of
staging where no child nodes are being used) I can see the phy_exit
function is disabling the clock using PCIE_PORT_CLK_EN which is
defined as:

#define PCIE_PORT_CLK_EN(x) BIT(24 + (x))

On probe function index is being set to 0 for the port 2 also, instead
of 2 (which is the correct value). Just try to comment this line:

rt_sysc_m32(PCIE_PORT_CLK_EN(instance->index), 0, RALINK_CLKCFG1);

Does this enough to get the pci enumeration being done correctly?

>
> If I comment out the code in phy_power_off() and phy_exit() so they
> return doing nothing then I get further:
>
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> mt7621-pci 1e140000.pcie: Initiating port 1 failed
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> mt7621-pci 1e140000.pcie: Initiating port 2 failed
> mt7621-pci 1e140000.pcie: PCIE0 enabled
> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> pci_bus 0000:00: root bus resource [io  0xffffffff]
> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> pci_bus 0000:00: root bus resource [bus 00-ff]
> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
> pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
> pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
> pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
> pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
> pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
> pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
> pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
> pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
> pci 0000:00:00.0: PCI bridge to [bus 01]
> pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
> pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
>
>
> So devices found, but interrupt setup failed for some reason.
> I have an atheros PCIe WIFI device on that bus which is detected, but
> the interrupt failure means it still doesn't actually work.

Nothing has changed about interrupts from linux 4.20 version to this.
It is returning -EINVAL
for some reason. Irq is set using  "of_irq_parse_and_map_pci" function.

>
> Regards
> Greg

Best regards,
    Sergio Paracuellos
>
>
> >> I'll keep digging.
> >
> > Thanks, really appreciate it.
> >
> >>
> >> Thanks
> >> Greg
> >
> > Best regards,
> >      Sergio Paracuellos
> >
> >>
> >>
> >>>
> >>>>        mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >>>>        mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >>>>        pci_bus 0000:00: root bus resource [io  0xffffffff]
> >>>>        pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >>>>        pci_bus 0000:00: root bus resource [bus 00-ff]
> >>>>
> >>>> And the system continued on to fully boot. So it looks like it thinks
> >>>> pcie0 is active. Better, but the PCI bus probe didn't find any of the
> >>>> devices it should have.
> >>>
> >>> Yes, that seems what is happening because of my explanation above.
> >>>
> >>>>
> >>>> I inserted the quick hack code to do this at the top of mt7621_pcie_init_ports()
> >>>> and it looked like this:
> >>>>
> >>>>            /* Force PERST PCIe line into GPIO mode */
> >>>>            *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
> >>>>            *(unsigned int *)(0xbe000060) |=  BIT(10);
> >>>>            mdelay(100);
> >>>>
> >>>>            *(unsigned int *)(0xbe000600) |= BIT(19); // use GPIO19 (PERST_N/)
> >>>>            mdelay(100);
> >>>>            *(unsigned int *)(0xbe000620) &= ~(BIT(19)); // clear DATA
> >>>>            mdelay(1000);
> >>>>
> >>>>            /* Use GPIO control instead of PERST_N */
> >>>>            *(unsigned int *)(0xbe000620) |= BIT(19); // set DATA
> >>>>            mdelay(1000);
> >>>>
> >>>>
> >>>
> >>> I really hate the gpio way of doing this. If this works we have to
> >>> think in how to achieve this in a cleaner way :))
> >>>
> >>>> Regards
> >>>> Greg
> >>>
> >>> Thanks for your effort in this.
> >>>
> >>> Best regards,
> >>>       Sergio Paracuellos
> >>>>
> >>>>
> >>>>> Other big change is to use the new phy driver but I think the problem
> >>>>> seems to be related with resets.
> >>>>>
> >>>>>>
> >>>>>> Regards
> >>>>>> Greg
> >>>>>
> >>>>> Please, don't hesitate to ask me whatever you need to.
> >>>>>
> >>>>> Hope this helps.
> >>>>>
> >>>>> Best regards,
> >>>>>        Sergio Paracuellos
> >>>>>
> >>>
> >
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 27/5/19 4:35 pm, Sergio Paracuellos wrote:
> On Mon, May 27, 2019 at 6:37 AM Greg Ungerer <gerg@kernel.org> wrote:
>> On 24/5/19 3:35 pm, Sergio Paracuellos wrote:
>>> On Fri, May 24, 2019 at 2:35 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>> On 23/5/19 3:26 pm, Sergio Paracuellos wrote:
>>>>> On Thu, May 23, 2019 at 4:11 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>> On 22/5/19 4:27 pm, Sergio Paracuellos wrote:
>>>>>> [snip]
>>>>>>> There are some big changes between 4.20 and 5.x. One is the use of PERST_N
>>>>>>> instead of using gpio. This PERT_N stuff is used now on enable ports
>>>>>>> assuming the
>>>>>>> link of PCI is properly detected after enabling the phy. And it seems
>>>>>>> it is not according to
>>>>>>> your dmesg traces. The previous 4.20 code used gpio before this was done.
>>>>>>>
>>>>>>> This code is the one I am referring:
>>>>>>>
>>>>>>> /* Use GPIO control instead of PERST_N */
>>>>>>> *(unsigned int *)(0xbe000620) |= BIT(19) | BIT(8) | BIT(7); // set DATA
>>>>>>> mdelay(1000);
>>>>>>
>>>>>> I have been looking closely at those, wondering why the old code
>>>>>> drove that PERST line as a GPIO instead of using the built-in behavior.
>>>>>> (I have ignored bits 7 and 8 here since they are control of UART 3)
>>>>>
>>>>> Yes, this was also at first one of my big concerns so I tried to change into
>>>>> to use builtin behaviour (which is much more cleaner) and when the
>>>>> code was tested
>>>>> it worked. It seems it is not valid for every board.
>>>>>
>>>>>>
>>>>>>
>>>>>>> I assume reset lines on your device tree are properly set up which is
>>>>>>> other of the big changes here: use
>>>>>>> reset lines instead of that hardcoding stuff. Also, the
>>>>>>> mt7621_reset_port routine is also using msleep(100)
>>>>>>> but maybe you can try a bigger value and change it into a mdelay, to
>>>>>>> see if that changes anything.
>>>>>>
>>>>>> I see the reset line configuration in the pcie section of mt7621.dtsi,
>>>>>> is there any others absolutely required here?  I couldn't see the
>>>>>> gbpc1.dts devicetree do anything else with pcie - othe than enable it.
>>>>>> My device tree for the EX15 is similar in that regard.
>>>>>>
>>>>>> I tried a couple of things with interesting results.
>>>>>>
>>>>>> 1. I made sure that the PERST_N line is set for PCIe operation (not GPIO).
>>>>>>        I forced it with:
>>>>>>
>>>>>>            *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
>>>>>>
>>>>>>        I checked bits 10 and 11 at kernel PCI init and they were 00 anyway.
>>>>>>        So PERST_N was definitely in PCIe reset mode. No change in behavior,
>>>>>>
>>>>>>
>>>>>> 2. I forced a GPIO style reset of that PERST line (using GPIO19) and got
>>>>>>        the following result on kernel boot:
>>>>>>
>>>>>>         mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>>>>         mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>>         mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>>>>         mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>>>         mt7621-pci 1e140000.pcie: Initiating port 1 failed
>>>>>>         mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>>>>         mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>>>         mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>>>>         mt7621-pci 1e140000.pcie: Initiating port 2 failed
>>>>>>         mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N
>>>>>
>>>>> This line seems to be the problem. When ports are init, (and with your
>>>>> changes seems the are
>>>>> init properly), the ports with pcie link are stored into a list to be
>>>>> enabled afterwards. This code is
>>>>> located into 'mt7621_pcie_enable_ports' which call simple
>>>>> 'mt7621_pcie_enable_port' to enable each port
>>>>> on the list. In this process it uses the PERS_N built-in register
>>>>> deasserting and asserting it. If enabling fails
>>>>> (and this is ypour case now) the port is removed from the list and it
>>>>> is not properly set up. You should try to
>>>>> comment this code:
>>>>>
>>>>> /* assert port PERST_N */
>>>>> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
>>>>> val |= PCIE_PORT_PERST(slot);
>>>>> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
>>>>>
>>>>> /* de-assert port PERST_N */
>>>>> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
>>>>> val &= ~PCIE_PORT_PERST(slot);
>>>>> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
>>>>>
>>>>> /* 100ms timeout value should be enough for Gen1 training */
>>>>> err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
>>>>> val, !!(val & PCIE_PORT_LINKUP),
>>>>> 20, 100 * USEC_PER_MSEC);
>>>>> if (err)
>>>>> return -ETIMEDOUT;
>>>>>
>>>>> because on enabling, it seems it is getting ETIMEOUT and hence the
>>>>> message '  mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N'.
>>>>> Commenting
>>>>> this code should end up into a properly configured pci?
>>>>
>>>> No, unfortunately it doesn't. It does show PCIE0 enabled now though:
>>>
>>> That is a surprise :(
>>>
>>>>
>>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>> mt7621-pci 1e140000.pcie: Initiating port 1 failed
>>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>> mt7621-pci 1e140000.pcie: Initiating port 2 failed
>>>> mt7621-pci 1e140000.pcie: PCIE0 enabled
>>>> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>>> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>>> pci_bus 0000:00: root bus resource [io  0xffffffff]
>>>> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>>> pci_bus 0000:00: root bus resource [bus 00-ff]
>>>
>>>
>>>>
>>>> And again no devices are found on the PCI bus.
>>>> (System did still boot too).
>>>
>>> Looking to your original trace of linux-4.20 working the init traces
>>> are pretty much the same... I don't really know what could be
>>> happening there. Root resources
>>> are correct, virtual bridge seems to be detected, the next should be
>>> to reconfigure resources of
>>> the bridge and this is done by the pci kernel apis.
>>>
>>> Can you check that "mt7621_pcie_init_virtual_bridges" is getting link
>>> up and configuring bridges
>>> correctly?
>>
>> Yes, it does get link there. It sees pcie_link_status as 0x1, so its getting
>> through that.
>>
>> I threw a bit of trace in to see where we end up losing the ability to
>> read correct config data from slot 0 (my only valid slot). It gets to
>> the "err_no_link_up:" label for port/slot 2 still being able to read config
>> space, but then after executing the phy_power_off() and phy_exit()
>> calls for that port/slot we can no longer read config for slot 0.
> 
> Mmmm. I see. So phy instances for port 0 and 2 are different instances
> of the phy, so it should not
> have problems for the power_off function. Looking again to the version
> which is in the 5.0 linux (but not in the last changes of
> staging where no child nodes are being used) I can see the phy_exit
> function is disabling the clock using PCIE_PORT_CLK_EN which is
> defined as:
> 
> #define PCIE_PORT_CLK_EN(x) BIT(24 + (x))
> 
> On probe function index is being set to 0 for the port 2 also, instead
> of 2 (which is the correct value). Just try to comment this line:
> 
> rt_sysc_m32(PCIE_PORT_CLK_EN(instance->index), 0, RALINK_CLKCFG1);
> 
> Does this enough to get the pci enumeration being done correctly?

Yes, just that (and the gpio based reset hack) is enough to enumerate the bus.



>> If I comment out the code in phy_power_off() and phy_exit() so they
>> return doing nothing then I get further:
>>
>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>> mt7621-pci 1e140000.pcie: Initiating port 1 failed
>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>> mt7621-pci 1e140000.pcie: Initiating port 2 failed
>> mt7621-pci 1e140000.pcie: PCIE0 enabled
>> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>> pci_bus 0000:00: root bus resource [io  0xffffffff]
>> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>> pci_bus 0000:00: root bus resource [bus 00-ff]
>> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>> pci 0000:00:00.0: PCI bridge to [bus 01-ff]
>> pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
>> pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
>> pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
>> pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
>> pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
>> pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
>> pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
>> pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
>> pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
>> pci 0000:00:00.0: PCI bridge to [bus 01]
>> pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
>> pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
>> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
>> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
>>
>>
>> So devices found, but interrupt setup failed for some reason.
>> I have an atheros PCIe WIFI device on that bus which is detected, but
>> the interrupt failure means it still doesn't actually work.
> 
> Nothing has changed about interrupts from linux 4.20 version to this.
> It is returning -EINVAL
> for some reason. Irq is set using  "of_irq_parse_and_map_pci" function.

Not sure either why this would be different.
I'll dig into that tomorrow too.

Regards
Greg


>> Regards
>> Greg
> 
> Best regards,
>      Sergio Paracuellos
>>
>>
>>>> I'll keep digging.
>>>
>>> Thanks, really appreciate it.
>>>
>>>>
>>>> Thanks
>>>> Greg
>>>
>>> Best regards,
>>>       Sergio Paracuellos
>>>
>>>>
>>>>
>>>>>
>>>>>>         mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>>>>>         mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>>>>>         pci_bus 0000:00: root bus resource [io  0xffffffff]
>>>>>>         pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>>>>>         pci_bus 0000:00: root bus resource [bus 00-ff]
>>>>>>
>>>>>> And the system continued on to fully boot. So it looks like it thinks
>>>>>> pcie0 is active. Better, but the PCI bus probe didn't find any of the
>>>>>> devices it should have.
>>>>>
>>>>> Yes, that seems what is happening because of my explanation above.
>>>>>
>>>>>>
>>>>>> I inserted the quick hack code to do this at the top of mt7621_pcie_init_ports()
>>>>>> and it looked like this:
>>>>>>
>>>>>>             /* Force PERST PCIe line into GPIO mode */
>>>>>>             *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
>>>>>>             *(unsigned int *)(0xbe000060) |=  BIT(10);
>>>>>>             mdelay(100);
>>>>>>
>>>>>>             *(unsigned int *)(0xbe000600) |= BIT(19); // use GPIO19 (PERST_N/)
>>>>>>             mdelay(100);
>>>>>>             *(unsigned int *)(0xbe000620) &= ~(BIT(19)); // clear DATA
>>>>>>             mdelay(1000);
>>>>>>
>>>>>>             /* Use GPIO control instead of PERST_N */
>>>>>>             *(unsigned int *)(0xbe000620) |= BIT(19); // set DATA
>>>>>>             mdelay(1000);
>>>>>>
>>>>>>
>>>>>
>>>>> I really hate the gpio way of doing this. If this works we have to
>>>>> think in how to achieve this in a cleaner way :))
>>>>>
>>>>>> Regards
>>>>>> Greg
>>>>>
>>>>> Thanks for your effort in this.
>>>>>
>>>>> Best regards,
>>>>>        Sergio Paracuellos
>>>>>>
>>>>>>
>>>>>>> Other big change is to use the new phy driver but I think the problem
>>>>>>> seems to be related with resets.
>>>>>>>
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> Greg
>>>>>>>
>>>>>>> Please, don't hesitate to ask me whatever you need to.
>>>>>>>
>>>>>>> Hope this helps.
>>>>>>>
>>>>>>> Best regards,
>>>>>>>         Sergio Paracuellos
>>>>>>>
>>>>>
>>>
> 
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

Hi Greg,

On Mon, May 27, 2019 at 9:29 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> On 27/5/19 4:35 pm, Sergio Paracuellos wrote:
> > On Mon, May 27, 2019 at 6:37 AM Greg Ungerer <gerg@kernel.org> wrote:
> >> On 24/5/19 3:35 pm, Sergio Paracuellos wrote:
> >>> On Fri, May 24, 2019 at 2:35 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>> On 23/5/19 3:26 pm, Sergio Paracuellos wrote:
> >>>>> On Thu, May 23, 2019 at 4:11 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>> On 22/5/19 4:27 pm, Sergio Paracuellos wrote:
> >>>>>> [snip]
> >>>>>>> There are some big changes between 4.20 and 5.x. One is the use of PERST_N
> >>>>>>> instead of using gpio. This PERT_N stuff is used now on enable ports
> >>>>>>> assuming the
> >>>>>>> link of PCI is properly detected after enabling the phy. And it seems
> >>>>>>> it is not according to
> >>>>>>> your dmesg traces. The previous 4.20 code used gpio before this was done.
> >>>>>>>
> >>>>>>> This code is the one I am referring:
> >>>>>>>
> >>>>>>> /* Use GPIO control instead of PERST_N */
> >>>>>>> *(unsigned int *)(0xbe000620) |= BIT(19) | BIT(8) | BIT(7); // set DATA
> >>>>>>> mdelay(1000);
> >>>>>>
> >>>>>> I have been looking closely at those, wondering why the old code
> >>>>>> drove that PERST line as a GPIO instead of using the built-in behavior.
> >>>>>> (I have ignored bits 7 and 8 here since they are control of UART 3)
> >>>>>
> >>>>> Yes, this was also at first one of my big concerns so I tried to change into
> >>>>> to use builtin behaviour (which is much more cleaner) and when the
> >>>>> code was tested
> >>>>> it worked. It seems it is not valid for every board.
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>>> I assume reset lines on your device tree are properly set up which is
> >>>>>>> other of the big changes here: use
> >>>>>>> reset lines instead of that hardcoding stuff. Also, the
> >>>>>>> mt7621_reset_port routine is also using msleep(100)
> >>>>>>> but maybe you can try a bigger value and change it into a mdelay, to
> >>>>>>> see if that changes anything.
> >>>>>>
> >>>>>> I see the reset line configuration in the pcie section of mt7621.dtsi,
> >>>>>> is there any others absolutely required here?  I couldn't see the
> >>>>>> gbpc1.dts devicetree do anything else with pcie - othe than enable it.
> >>>>>> My device tree for the EX15 is similar in that regard.
> >>>>>>
> >>>>>> I tried a couple of things with interesting results.
> >>>>>>
> >>>>>> 1. I made sure that the PERST_N line is set for PCIe operation (not GPIO).
> >>>>>>        I forced it with:
> >>>>>>
> >>>>>>            *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
> >>>>>>
> >>>>>>        I checked bits 10 and 11 at kernel PCI init and they were 00 anyway.
> >>>>>>        So PERST_N was definitely in PCIe reset mode. No change in behavior,
> >>>>>>
> >>>>>>
> >>>>>> 2. I forced a GPIO style reset of that PERST line (using GPIO19) and got
> >>>>>>        the following result on kernel boot:
> >>>>>>
> >>>>>>         mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>>>>         mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>>>         mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>>>>         mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>>>>         mt7621-pci 1e140000.pcie: Initiating port 1 failed
> >>>>>>         mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>>>>         mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>>>>         mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>>>>         mt7621-pci 1e140000.pcie: Initiating port 2 failed
> >>>>>>         mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N
> >>>>>
> >>>>> This line seems to be the problem. When ports are init, (and with your
> >>>>> changes seems the are
> >>>>> init properly), the ports with pcie link are stored into a list to be
> >>>>> enabled afterwards. This code is
> >>>>> located into 'mt7621_pcie_enable_ports' which call simple
> >>>>> 'mt7621_pcie_enable_port' to enable each port
> >>>>> on the list. In this process it uses the PERS_N built-in register
> >>>>> deasserting and asserting it. If enabling fails
> >>>>> (and this is ypour case now) the port is removed from the list and it
> >>>>> is not properly set up. You should try to
> >>>>> comment this code:
> >>>>>
> >>>>> /* assert port PERST_N */
> >>>>> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
> >>>>> val |= PCIE_PORT_PERST(slot);
> >>>>> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
> >>>>>
> >>>>> /* de-assert port PERST_N */
> >>>>> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
> >>>>> val &= ~PCIE_PORT_PERST(slot);
> >>>>> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
> >>>>>
> >>>>> /* 100ms timeout value should be enough for Gen1 training */
> >>>>> err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
> >>>>> val, !!(val & PCIE_PORT_LINKUP),
> >>>>> 20, 100 * USEC_PER_MSEC);
> >>>>> if (err)
> >>>>> return -ETIMEDOUT;
> >>>>>
> >>>>> because on enabling, it seems it is getting ETIMEOUT and hence the
> >>>>> message '  mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N'.
> >>>>> Commenting
> >>>>> this code should end up into a properly configured pci?
> >>>>
> >>>> No, unfortunately it doesn't. It does show PCIE0 enabled now though:
> >>>
> >>> That is a surprise :(
> >>>
> >>>>
> >>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>> mt7621-pci 1e140000.pcie: Initiating port 1 failed
> >>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>> mt7621-pci 1e140000.pcie: Initiating port 2 failed
> >>>> mt7621-pci 1e140000.pcie: PCIE0 enabled
> >>>> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >>>> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >>>> pci_bus 0000:00: root bus resource [io  0xffffffff]
> >>>> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >>>> pci_bus 0000:00: root bus resource [bus 00-ff]
> >>>
> >>>
> >>>>
> >>>> And again no devices are found on the PCI bus.
> >>>> (System did still boot too).
> >>>
> >>> Looking to your original trace of linux-4.20 working the init traces
> >>> are pretty much the same... I don't really know what could be
> >>> happening there. Root resources
> >>> are correct, virtual bridge seems to be detected, the next should be
> >>> to reconfigure resources of
> >>> the bridge and this is done by the pci kernel apis.
> >>>
> >>> Can you check that "mt7621_pcie_init_virtual_bridges" is getting link
> >>> up and configuring bridges
> >>> correctly?
> >>
> >> Yes, it does get link there. It sees pcie_link_status as 0x1, so its getting
> >> through that.
> >>
> >> I threw a bit of trace in to see where we end up losing the ability to
> >> read correct config data from slot 0 (my only valid slot). It gets to
> >> the "err_no_link_up:" label for port/slot 2 still being able to read config
> >> space, but then after executing the phy_power_off() and phy_exit()
> >> calls for that port/slot we can no longer read config for slot 0.
> >
> > Mmmm. I see. So phy instances for port 0 and 2 are different instances
> > of the phy, so it should not
> > have problems for the power_off function. Looking again to the version
> > which is in the 5.0 linux (but not in the last changes of
> > staging where no child nodes are being used) I can see the phy_exit
> > function is disabling the clock using PCIE_PORT_CLK_EN which is
> > defined as:
> >
> > #define PCIE_PORT_CLK_EN(x) BIT(24 + (x))
> >
> > On probe function index is being set to 0 for the port 2 also, instead
> > of 2 (which is the correct value). Just try to comment this line:
> >
> > rt_sysc_m32(PCIE_PORT_CLK_EN(instance->index), 0, RALINK_CLKCFG1);
> >
> > Does this enough to get the pci enumeration being done correctly?
>
> Yes, just that (and the gpio based reset hack) is enough to enumerate the bus.

Ok, so this is problem shouldn't be present in the current staging and
linux tree at master where the
device tree is not using child nodes, which is the way to go.

I think we should add a gpio reset line in the device tree and use it
properly with
the gpio descriptor api. Maybe this is better for all the boards
instead of use the builtin stuff.

>
>
>
> >> If I comment out the code in phy_power_off() and phy_exit() so they
> >> return doing nothing then I get further:
> >>
> >> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >> mt7621-pci 1e140000.pcie: Initiating port 1 failed
> >> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >> mt7621-pci 1e140000.pcie: Initiating port 2 failed
> >> mt7621-pci 1e140000.pcie: PCIE0 enabled
> >> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >> pci_bus 0000:00: root bus resource [io  0xffffffff]
> >> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >> pci_bus 0000:00: root bus resource [bus 00-ff]
> >> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> >> pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> >> pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
> >> pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
> >> pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
> >> pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
> >> pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
> >> pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
> >> pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
> >> pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
> >> pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
> >> pci 0000:00:00.0: PCI bridge to [bus 01]
> >> pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
> >> pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
> >> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
> >> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
> >>
> >>
> >> So devices found, but interrupt setup failed for some reason.
> >> I have an atheros PCIe WIFI device on that bus which is detected, but
> >> the interrupt failure means it still doesn't actually work.
> >
> > Nothing has changed about interrupts from linux 4.20 version to this.
> > It is returning -EINVAL
> > for some reason. Irq is set using  "of_irq_parse_and_map_pci" function.
>
> Not sure either why this would be different.
> I'll dig into that tomorrow too.

Thanks, let me know any advance on this, please.

>
> Regards
> Greg

Best regards,
    Sergio Paracuellos

>
>
> >> Regards
> >> Greg
> >
> > Best regards,
> >      Sergio Paracuellos
> >>
> >>
> >>>> I'll keep digging.
> >>>
> >>> Thanks, really appreciate it.
> >>>
> >>>>
> >>>> Thanks
> >>>> Greg
> >>>
> >>> Best regards,
> >>>       Sergio Paracuellos
> >>>
> >>>>
> >>>>
> >>>>>
> >>>>>>         mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >>>>>>         mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >>>>>>         pci_bus 0000:00: root bus resource [io  0xffffffff]
> >>>>>>         pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >>>>>>         pci_bus 0000:00: root bus resource [bus 00-ff]
> >>>>>>
> >>>>>> And the system continued on to fully boot. So it looks like it thinks
> >>>>>> pcie0 is active. Better, but the PCI bus probe didn't find any of the
> >>>>>> devices it should have.
> >>>>>
> >>>>> Yes, that seems what is happening because of my explanation above.
> >>>>>
> >>>>>>
> >>>>>> I inserted the quick hack code to do this at the top of mt7621_pcie_init_ports()
> >>>>>> and it looked like this:
> >>>>>>
> >>>>>>             /* Force PERST PCIe line into GPIO mode */
> >>>>>>             *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
> >>>>>>             *(unsigned int *)(0xbe000060) |=  BIT(10);
> >>>>>>             mdelay(100);
> >>>>>>
> >>>>>>             *(unsigned int *)(0xbe000600) |= BIT(19); // use GPIO19 (PERST_N/)
> >>>>>>             mdelay(100);
> >>>>>>             *(unsigned int *)(0xbe000620) &= ~(BIT(19)); // clear DATA
> >>>>>>             mdelay(1000);
> >>>>>>
> >>>>>>             /* Use GPIO control instead of PERST_N */
> >>>>>>             *(unsigned int *)(0xbe000620) |= BIT(19); // set DATA
> >>>>>>             mdelay(1000);
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> I really hate the gpio way of doing this. If this works we have to
> >>>>> think in how to achieve this in a cleaner way :))
> >>>>>
> >>>>>> Regards
> >>>>>> Greg
> >>>>>
> >>>>> Thanks for your effort in this.
> >>>>>
> >>>>> Best regards,
> >>>>>        Sergio Paracuellos
> >>>>>>
> >>>>>>
> >>>>>>> Other big change is to use the new phy driver but I think the problem
> >>>>>>> seems to be related with resets.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Regards
> >>>>>>>> Greg
> >>>>>>>
> >>>>>>> Please, don't hesitate to ask me whatever you need to.
> >>>>>>>
> >>>>>>> Hope this helps.
> >>>>>>>
> >>>>>>> Best regards,
> >>>>>>>         Sergio Paracuellos
> >>>>>>>
> >>>>>
> >>>
> >
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 27/5/19 6:02 pm, Sergio Paracuellos wrote:
> On Mon, May 27, 2019 at 9:29 AM Greg Ungerer <gerg@kernel.org> wrote:
>> On 27/5/19 4:35 pm, Sergio Paracuellos wrote:
>>> On Mon, May 27, 2019 at 6:37 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>> On 24/5/19 3:35 pm, Sergio Paracuellos wrote:
>>>>> On Fri, May 24, 2019 at 2:35 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>> On 23/5/19 3:26 pm, Sergio Paracuellos wrote:
>>>>>>> On Thu, May 23, 2019 at 4:11 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>>> On 22/5/19 4:27 pm, Sergio Paracuellos wrote:
>>>>>>>> [snip]
>>>>>>>>> There are some big changes between 4.20 and 5.x. One is the use of PERST_N
>>>>>>>>> instead of using gpio. This PERT_N stuff is used now on enable ports
>>>>>>>>> assuming the
>>>>>>>>> link of PCI is properly detected after enabling the phy. And it seems
>>>>>>>>> it is not according to
>>>>>>>>> your dmesg traces. The previous 4.20 code used gpio before this was done.
>>>>>>>>>
>>>>>>>>> This code is the one I am referring:
>>>>>>>>>
>>>>>>>>> /* Use GPIO control instead of PERST_N */
>>>>>>>>> *(unsigned int *)(0xbe000620) |= BIT(19) | BIT(8) | BIT(7); // set DATA
>>>>>>>>> mdelay(1000);
>>>>>>>>
>>>>>>>> I have been looking closely at those, wondering why the old code
>>>>>>>> drove that PERST line as a GPIO instead of using the built-in behavior.
>>>>>>>> (I have ignored bits 7 and 8 here since they are control of UART 3)
>>>>>>>
>>>>>>> Yes, this was also at first one of my big concerns so I tried to change into
>>>>>>> to use builtin behaviour (which is much more cleaner) and when the
>>>>>>> code was tested
>>>>>>> it worked. It seems it is not valid for every board.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> I assume reset lines on your device tree are properly set up which is
>>>>>>>>> other of the big changes here: use
>>>>>>>>> reset lines instead of that hardcoding stuff. Also, the
>>>>>>>>> mt7621_reset_port routine is also using msleep(100)
>>>>>>>>> but maybe you can try a bigger value and change it into a mdelay, to
>>>>>>>>> see if that changes anything.
>>>>>>>>
>>>>>>>> I see the reset line configuration in the pcie section of mt7621.dtsi,
>>>>>>>> is there any others absolutely required here?  I couldn't see the
>>>>>>>> gbpc1.dts devicetree do anything else with pcie - othe than enable it.
>>>>>>>> My device tree for the EX15 is similar in that regard.
>>>>>>>>
>>>>>>>> I tried a couple of things with interesting results.
>>>>>>>>
>>>>>>>> 1. I made sure that the PERST_N line is set for PCIe operation (not GPIO).
>>>>>>>>         I forced it with:
>>>>>>>>
>>>>>>>>             *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
>>>>>>>>
>>>>>>>>         I checked bits 10 and 11 at kernel PCI init and they were 00 anyway.
>>>>>>>>         So PERST_N was definitely in PCIe reset mode. No change in behavior,
>>>>>>>>
>>>>>>>>
>>>>>>>> 2. I forced a GPIO style reset of that PERST line (using GPIO19) and got
>>>>>>>>         the following result on kernel boot:
>>>>>>>>
>>>>>>>>          mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>>>>>>          mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>>>>          mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>>>>>>          mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>>>>>          mt7621-pci 1e140000.pcie: Initiating port 1 failed
>>>>>>>>          mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>>>>>>          mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>>>>>          mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>>>>>>          mt7621-pci 1e140000.pcie: Initiating port 2 failed
>>>>>>>>          mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N
>>>>>>>
>>>>>>> This line seems to be the problem. When ports are init, (and with your
>>>>>>> changes seems the are
>>>>>>> init properly), the ports with pcie link are stored into a list to be
>>>>>>> enabled afterwards. This code is
>>>>>>> located into 'mt7621_pcie_enable_ports' which call simple
>>>>>>> 'mt7621_pcie_enable_port' to enable each port
>>>>>>> on the list. In this process it uses the PERS_N built-in register
>>>>>>> deasserting and asserting it. If enabling fails
>>>>>>> (and this is ypour case now) the port is removed from the list and it
>>>>>>> is not properly set up. You should try to
>>>>>>> comment this code:
>>>>>>>
>>>>>>> /* assert port PERST_N */
>>>>>>> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
>>>>>>> val |= PCIE_PORT_PERST(slot);
>>>>>>> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
>>>>>>>
>>>>>>> /* de-assert port PERST_N */
>>>>>>> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
>>>>>>> val &= ~PCIE_PORT_PERST(slot);
>>>>>>> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
>>>>>>>
>>>>>>> /* 100ms timeout value should be enough for Gen1 training */
>>>>>>> err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
>>>>>>> val, !!(val & PCIE_PORT_LINKUP),
>>>>>>> 20, 100 * USEC_PER_MSEC);
>>>>>>> if (err)
>>>>>>> return -ETIMEDOUT;
>>>>>>>
>>>>>>> because on enabling, it seems it is getting ETIMEOUT and hence the
>>>>>>> message '  mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N'.
>>>>>>> Commenting
>>>>>>> this code should end up into a properly configured pci?
>>>>>>
>>>>>> No, unfortunately it doesn't. It does show PCIE0 enabled now though:
>>>>>
>>>>> That is a surprise :(
>>>>>
>>>>>>
>>>>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>>>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>>>> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>>> mt7621-pci 1e140000.pcie: Initiating port 1 failed
>>>>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>>>> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>>> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>>>> mt7621-pci 1e140000.pcie: Initiating port 2 failed
>>>>>> mt7621-pci 1e140000.pcie: PCIE0 enabled
>>>>>> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>>>>> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>>>>> pci_bus 0000:00: root bus resource [io  0xffffffff]
>>>>>> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>>>>> pci_bus 0000:00: root bus resource [bus 00-ff]
>>>>>
>>>>>
>>>>>>
>>>>>> And again no devices are found on the PCI bus.
>>>>>> (System did still boot too).
>>>>>
>>>>> Looking to your original trace of linux-4.20 working the init traces
>>>>> are pretty much the same... I don't really know what could be
>>>>> happening there. Root resources
>>>>> are correct, virtual bridge seems to be detected, the next should be
>>>>> to reconfigure resources of
>>>>> the bridge and this is done by the pci kernel apis.
>>>>>
>>>>> Can you check that "mt7621_pcie_init_virtual_bridges" is getting link
>>>>> up and configuring bridges
>>>>> correctly?
>>>>
>>>> Yes, it does get link there. It sees pcie_link_status as 0x1, so its getting
>>>> through that.
>>>>
>>>> I threw a bit of trace in to see where we end up losing the ability to
>>>> read correct config data from slot 0 (my only valid slot). It gets to
>>>> the "err_no_link_up:" label for port/slot 2 still being able to read config
>>>> space, but then after executing the phy_power_off() and phy_exit()
>>>> calls for that port/slot we can no longer read config for slot 0.
>>>
>>> Mmmm. I see. So phy instances for port 0 and 2 are different instances
>>> of the phy, so it should not
>>> have problems for the power_off function. Looking again to the version
>>> which is in the 5.0 linux (but not in the last changes of
>>> staging where no child nodes are being used) I can see the phy_exit
>>> function is disabling the clock using PCIE_PORT_CLK_EN which is
>>> defined as:
>>>
>>> #define PCIE_PORT_CLK_EN(x) BIT(24 + (x))
>>>
>>> On probe function index is being set to 0 for the port 2 also, instead
>>> of 2 (which is the correct value). Just try to comment this line:
>>>
>>> rt_sysc_m32(PCIE_PORT_CLK_EN(instance->index), 0, RALINK_CLKCFG1);
>>>
>>> Does this enough to get the pci enumeration being done correctly?
>>
>> Yes, just that (and the gpio based reset hack) is enough to enumerate the bus.
> 
> Ok, so this is problem shouldn't be present in the current staging and
> linux tree at master where the
> device tree is not using child nodes, which is the way to go.

I cloned the staging tree from git.kernel.org and tried that.
It didn't work any better, I had to patch pci-mt7621.c and
pci-mt8721-phy.c in the same ways to get an almost working result.


> I think we should add a gpio reset line in the device tree and use it
> properly with
> the gpio descriptor api. Maybe this is better for all the boards
> instead of use the builtin stuff.

Would seem to be the best approach.


>>>> If I comment out the code in phy_power_off() and phy_exit() so they
>>>> return doing nothing then I get further:
>>>>
>>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>> mt7621-pci 1e140000.pcie: Initiating port 1 failed
>>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>> mt7621-pci 1e140000.pcie: Initiating port 2 failed
>>>> mt7621-pci 1e140000.pcie: PCIE0 enabled
>>>> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>>> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>>> pci_bus 0000:00: root bus resource [io  0xffffffff]
>>>> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>>> pci_bus 0000:00: root bus resource [bus 00-ff]
>>>> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>>>> pci 0000:00:00.0: PCI bridge to [bus 01-ff]
>>>> pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
>>>> pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
>>>> pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
>>>> pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
>>>> pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
>>>> pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
>>>> pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
>>>> pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
>>>> pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
>>>> pci 0000:00:00.0: PCI bridge to [bus 01]
>>>> pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
>>>> pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
>>>> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
>>>> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
>>>>
>>>>
>>>> So devices found, but interrupt setup failed for some reason.
>>>> I have an atheros PCIe WIFI device on that bus which is detected, but
>>>> the interrupt failure means it still doesn't actually work.
>>>
>>> Nothing has changed about interrupts from linux 4.20 version to this.
>>> It is returning -EINVAL
>>> for some reason. Irq is set using  "of_irq_parse_and_map_pci" function.
>>
>> Not sure either why this would be different.
>> I'll dig into that tomorrow too.
> 
> Thanks, let me know any advance on this, please.

I suspect that the bus or devices stop reading/writing valid data
at some point in this initialization process. What I see is that
dumping /sys/bus/pci/devices/0000:01:00.0/config on a running
system shows:

   00000000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff     ................
   00000010: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff     ................
   00000020: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff     ................
   ...

But if I replace the PCI code again with that from 4.20 then
I get a valid dump of the wifi card config space:

   00000000: 8c 16 3c 00 06 00 10 00 00 00 80 02 00 00 00 00     ..<.............
   00000010: 04 00 00 60 00 00 00 00 00 00 00 00 00 00 00 00     ...`............
   00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00     ................

Regards
Greg


>> Regards
>> Greg
> 
> Best regards,
>      Sergio Paracuellos
> 
>>
>>
>>>> Regards
>>>> Greg
>>>
>>> Best regards,
>>>       Sergio Paracuellos
>>>>
>>>>
>>>>>> I'll keep digging.
>>>>>
>>>>> Thanks, really appreciate it.
>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> Greg
>>>>>
>>>>> Best regards,
>>>>>        Sergio Paracuellos
>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>>          mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>>>>>>>          mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>>>>>>>          pci_bus 0000:00: root bus resource [io  0xffffffff]
>>>>>>>>          pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>>>>>>>          pci_bus 0000:00: root bus resource [bus 00-ff]
>>>>>>>>
>>>>>>>> And the system continued on to fully boot. So it looks like it thinks
>>>>>>>> pcie0 is active. Better, but the PCI bus probe didn't find any of the
>>>>>>>> devices it should have.
>>>>>>>
>>>>>>> Yes, that seems what is happening because of my explanation above.
>>>>>>>
>>>>>>>>
>>>>>>>> I inserted the quick hack code to do this at the top of mt7621_pcie_init_ports()
>>>>>>>> and it looked like this:
>>>>>>>>
>>>>>>>>              /* Force PERST PCIe line into GPIO mode */
>>>>>>>>              *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
>>>>>>>>              *(unsigned int *)(0xbe000060) |=  BIT(10);
>>>>>>>>              mdelay(100);
>>>>>>>>
>>>>>>>>              *(unsigned int *)(0xbe000600) |= BIT(19); // use GPIO19 (PERST_N/)
>>>>>>>>              mdelay(100);
>>>>>>>>              *(unsigned int *)(0xbe000620) &= ~(BIT(19)); // clear DATA
>>>>>>>>              mdelay(1000);
>>>>>>>>
>>>>>>>>              /* Use GPIO control instead of PERST_N */
>>>>>>>>              *(unsigned int *)(0xbe000620) |= BIT(19); // set DATA
>>>>>>>>              mdelay(1000);
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> I really hate the gpio way of doing this. If this works we have to
>>>>>>> think in how to achieve this in a cleaner way :))
>>>>>>>
>>>>>>>> Regards
>>>>>>>> Greg
>>>>>>>
>>>>>>> Thanks for your effort in this.
>>>>>>>
>>>>>>> Best regards,
>>>>>>>         Sergio Paracuellos
>>>>>>>>
>>>>>>>>
>>>>>>>>> Other big change is to use the new phy driver but I think the problem
>>>>>>>>> seems to be related with resets.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>>> Greg
>>>>>>>>>
>>>>>>>>> Please, don't hesitate to ask me whatever you need to.
>>>>>>>>>
>>>>>>>>> Hope this helps.
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>>          Sergio Paracuellos
>>>>>>>>>
>>>>>>>
>>>>>
>>>
> 
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

[-- Attachment #1: Type: text/plain, Size: 16040 bytes --]

Hi Greg,

On Wed, May 29, 2019 at 9:11 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> On 27/5/19 6:02 pm, Sergio Paracuellos wrote:
> > On Mon, May 27, 2019 at 9:29 AM Greg Ungerer <gerg@kernel.org> wrote:
> >> On 27/5/19 4:35 pm, Sergio Paracuellos wrote:
> >>> On Mon, May 27, 2019 at 6:37 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>> On 24/5/19 3:35 pm, Sergio Paracuellos wrote:
> >>>>> On Fri, May 24, 2019 at 2:35 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>> On 23/5/19 3:26 pm, Sergio Paracuellos wrote:
> >>>>>>> On Thu, May 23, 2019 at 4:11 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>>>> On 22/5/19 4:27 pm, Sergio Paracuellos wrote:
> >>>>>>>> [snip]
> >>>>>>>>> There are some big changes between 4.20 and 5.x. One is the use of PERST_N
> >>>>>>>>> instead of using gpio. This PERT_N stuff is used now on enable ports
> >>>>>>>>> assuming the
> >>>>>>>>> link of PCI is properly detected after enabling the phy. And it seems
> >>>>>>>>> it is not according to
> >>>>>>>>> your dmesg traces. The previous 4.20 code used gpio before this was done.
> >>>>>>>>>
> >>>>>>>>> This code is the one I am referring:
> >>>>>>>>>
> >>>>>>>>> /* Use GPIO control instead of PERST_N */
> >>>>>>>>> *(unsigned int *)(0xbe000620) |= BIT(19) | BIT(8) | BIT(7); // set DATA
> >>>>>>>>> mdelay(1000);
> >>>>>>>>
> >>>>>>>> I have been looking closely at those, wondering why the old code
> >>>>>>>> drove that PERST line as a GPIO instead of using the built-in behavior.
> >>>>>>>> (I have ignored bits 7 and 8 here since they are control of UART 3)
> >>>>>>>
> >>>>>>> Yes, this was also at first one of my big concerns so I tried to change into
> >>>>>>> to use builtin behaviour (which is much more cleaner) and when the
> >>>>>>> code was tested
> >>>>>>> it worked. It seems it is not valid for every board.
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> I assume reset lines on your device tree are properly set up which is
> >>>>>>>>> other of the big changes here: use
> >>>>>>>>> reset lines instead of that hardcoding stuff. Also, the
> >>>>>>>>> mt7621_reset_port routine is also using msleep(100)
> >>>>>>>>> but maybe you can try a bigger value and change it into a mdelay, to
> >>>>>>>>> see if that changes anything.
> >>>>>>>>
> >>>>>>>> I see the reset line configuration in the pcie section of mt7621.dtsi,
> >>>>>>>> is there any others absolutely required here?  I couldn't see the
> >>>>>>>> gbpc1.dts devicetree do anything else with pcie - othe than enable it.
> >>>>>>>> My device tree for the EX15 is similar in that regard.
> >>>>>>>>
> >>>>>>>> I tried a couple of things with interesting results.
> >>>>>>>>
> >>>>>>>> 1. I made sure that the PERST_N line is set for PCIe operation (not GPIO).
> >>>>>>>>         I forced it with:
> >>>>>>>>
> >>>>>>>>             *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
> >>>>>>>>
> >>>>>>>>         I checked bits 10 and 11 at kernel PCI init and they were 00 anyway.
> >>>>>>>>         So PERST_N was definitely in PCIe reset mode. No change in behavior,
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> 2. I forced a GPIO style reset of that PERST line (using GPIO19) and got
> >>>>>>>>         the following result on kernel boot:
> >>>>>>>>
> >>>>>>>>          mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>>>>>>          mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>>>>>          mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>>>>>>          mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>>>>>>          mt7621-pci 1e140000.pcie: Initiating port 1 failed
> >>>>>>>>          mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>>>>>>          mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>>>>>>          mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>>>>>>          mt7621-pci 1e140000.pcie: Initiating port 2 failed
> >>>>>>>>          mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N
> >>>>>>>
> >>>>>>> This line seems to be the problem. When ports are init, (and with your
> >>>>>>> changes seems the are
> >>>>>>> init properly), the ports with pcie link are stored into a list to be
> >>>>>>> enabled afterwards. This code is
> >>>>>>> located into 'mt7621_pcie_enable_ports' which call simple
> >>>>>>> 'mt7621_pcie_enable_port' to enable each port
> >>>>>>> on the list. In this process it uses the PERS_N built-in register
> >>>>>>> deasserting and asserting it. If enabling fails
> >>>>>>> (and this is ypour case now) the port is removed from the list and it
> >>>>>>> is not properly set up. You should try to
> >>>>>>> comment this code:
> >>>>>>>
> >>>>>>> /* assert port PERST_N */
> >>>>>>> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
> >>>>>>> val |= PCIE_PORT_PERST(slot);
> >>>>>>> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
> >>>>>>>
> >>>>>>> /* de-assert port PERST_N */
> >>>>>>> val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
> >>>>>>> val &= ~PCIE_PORT_PERST(slot);
> >>>>>>> pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
> >>>>>>>
> >>>>>>> /* 100ms timeout value should be enough for Gen1 training */
> >>>>>>> err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
> >>>>>>> val, !!(val & PCIE_PORT_LINKUP),
> >>>>>>> 20, 100 * USEC_PER_MSEC);
> >>>>>>> if (err)
> >>>>>>> return -ETIMEDOUT;
> >>>>>>>
> >>>>>>> because on enabling, it seems it is getting ETIMEOUT and hence the
> >>>>>>> message '  mt7621-pci 1e140000.pcie: de-assert port 0 PERST_N'.
> >>>>>>> Commenting
> >>>>>>> this code should end up into a properly configured pci?
> >>>>>>
> >>>>>> No, unfortunately it doesn't. It does show PCIE0 enabled now though:
> >>>>>
> >>>>> That is a surprise :(
> >>>>>
> >>>>>>
> >>>>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>>>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>>>> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>>>> mt7621-pci 1e140000.pcie: Initiating port 1 failed
> >>>>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>>>> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>>>> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>>>> mt7621-pci 1e140000.pcie: Initiating port 2 failed
> >>>>>> mt7621-pci 1e140000.pcie: PCIE0 enabled
> >>>>>> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >>>>>> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >>>>>> pci_bus 0000:00: root bus resource [io  0xffffffff]
> >>>>>> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >>>>>> pci_bus 0000:00: root bus resource [bus 00-ff]
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> And again no devices are found on the PCI bus.
> >>>>>> (System did still boot too).
> >>>>>
> >>>>> Looking to your original trace of linux-4.20 working the init traces
> >>>>> are pretty much the same... I don't really know what could be
> >>>>> happening there. Root resources
> >>>>> are correct, virtual bridge seems to be detected, the next should be
> >>>>> to reconfigure resources of
> >>>>> the bridge and this is done by the pci kernel apis.
> >>>>>
> >>>>> Can you check that "mt7621_pcie_init_virtual_bridges" is getting link
> >>>>> up and configuring bridges
> >>>>> correctly?
> >>>>
> >>>> Yes, it does get link there. It sees pcie_link_status as 0x1, so its getting
> >>>> through that.
> >>>>
> >>>> I threw a bit of trace in to see where we end up losing the ability to
> >>>> read correct config data from slot 0 (my only valid slot). It gets to
> >>>> the "err_no_link_up:" label for port/slot 2 still being able to read config
> >>>> space, but then after executing the phy_power_off() and phy_exit()
> >>>> calls for that port/slot we can no longer read config for slot 0.
> >>>
> >>> Mmmm. I see. So phy instances for port 0 and 2 are different instances
> >>> of the phy, so it should not
> >>> have problems for the power_off function. Looking again to the version
> >>> which is in the 5.0 linux (but not in the last changes of
> >>> staging where no child nodes are being used) I can see the phy_exit
> >>> function is disabling the clock using PCIE_PORT_CLK_EN which is
> >>> defined as:
> >>>
> >>> #define PCIE_PORT_CLK_EN(x) BIT(24 + (x))
> >>>
> >>> On probe function index is being set to 0 for the port 2 also, instead
> >>> of 2 (which is the correct value). Just try to comment this line:
> >>>
> >>> rt_sysc_m32(PCIE_PORT_CLK_EN(instance->index), 0, RALINK_CLKCFG1);
> >>>
> >>> Does this enough to get the pci enumeration being done correctly?
> >>
> >> Yes, just that (and the gpio based reset hack) is enough to enumerate the bus.
> >
> > Ok, so this is problem shouldn't be present in the current staging and
> > linux tree at master where the
> > device tree is not using child nodes, which is the way to go.
>
> I cloned the staging tree from git.kernel.org and tried that.
> It didn't work any better, I had to patch pci-mt7621.c and
> pci-mt8721-phy.c in the same ways to get an almost working result.

That's weird. I'll check that.

>
>
> > I think we should add a gpio reset line in the device tree and use it
> > properly with
> > the gpio descriptor api. Maybe this is better for all the boards
> > instead of use the builtin stuff.
>
> Would seem to be the best approach.
>
>
> >>>> If I comment out the code in phy_power_off() and phy_exit() so they
> >>>> return doing nothing then I get further:
> >>>>
> >>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>> mt7621-pci 1e140000.pcie: Initiating port 1 failed
> >>>> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>> mt7621-pci 1e140000.pcie: Initiating port 2 failed
> >>>> mt7621-pci 1e140000.pcie: PCIE0 enabled
> >>>> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >>>> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >>>> pci_bus 0000:00: root bus resource [io  0xffffffff]
> >>>> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >>>> pci_bus 0000:00: root bus resource [bus 00-ff]
> >>>> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> >>>> pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> >>>> pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
> >>>> pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
> >>>> pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
> >>>> pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
> >>>> pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
> >>>> pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
> >>>> pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
> >>>> pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
> >>>> pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
> >>>> pci 0000:00:00.0: PCI bridge to [bus 01]
> >>>> pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
> >>>> pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
> >>>> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
> >>>> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
> >>>>
> >>>>
> >>>> So devices found, but interrupt setup failed for some reason.
> >>>> I have an atheros PCIe WIFI device on that bus which is detected, but
> >>>> the interrupt failure means it still doesn't actually work.
> >>>
> >>> Nothing has changed about interrupts from linux 4.20 version to this.
> >>> It is returning -EINVAL
> >>> for some reason. Irq is set using  "of_irq_parse_and_map_pci" function.
> >>
> >> Not sure either why this would be different.
> >> I'll dig into that tomorrow too.
> >
> > Thanks, let me know any advance on this, please.
>
> I suspect that the bus or devices stop reading/writing valid data
> at some point in this initialization process. What I see is that
> dumping /sys/bus/pci/devices/0000:01:00.0/config on a running
> system shows:
>
>    00000000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff     ................
>    00000010: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff     ................
>    00000020: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff     ................
>    ...
>
> But if I replace the PCI code again with that from 4.20 then
> I get a valid dump of the wifi card config space:
>
>    00000000: 8c 16 3c 00 06 00 10 00 00 00 80 02 00 00 00 00     ..<.............
>    00000010: 04 00 00 60 00 00 00 00 00 00 00 00 00 00 00 00     ...`............
>    00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00     ................
>

I have added gpio consumer stuff and reorder a bit the code to be more
similar to 4.20.

I attach the patch. I have not try it to compile it, because my normal
environment is in another
computer and I am in the middle of moving from my current house and
don't have access to it, sorry.
So, please try this and let's see what happens.

> Regards
> Greg

Hope this helps.

Best regards,
    Sergio Paracuellos
>
>
> >> Regards
> >> Greg
> >
> > Best regards,
> >      Sergio Paracuellos
> >
> >>
> >>
> >>>> Regards
> >>>> Greg
> >>>
> >>> Best regards,
> >>>       Sergio Paracuellos
> >>>>
> >>>>
> >>>>>> I'll keep digging.
> >>>>>
> >>>>> Thanks, really appreciate it.
> >>>>>
> >>>>>>
> >>>>>> Thanks
> >>>>>> Greg
> >>>>>
> >>>>> Best regards,
> >>>>>        Sergio Paracuellos
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>>          mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >>>>>>>>          mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >>>>>>>>          pci_bus 0000:00: root bus resource [io  0xffffffff]
> >>>>>>>>          pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >>>>>>>>          pci_bus 0000:00: root bus resource [bus 00-ff]
> >>>>>>>>
> >>>>>>>> And the system continued on to fully boot. So it looks like it thinks
> >>>>>>>> pcie0 is active. Better, but the PCI bus probe didn't find any of the
> >>>>>>>> devices it should have.
> >>>>>>>
> >>>>>>> Yes, that seems what is happening because of my explanation above.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> I inserted the quick hack code to do this at the top of mt7621_pcie_init_ports()
> >>>>>>>> and it looked like this:
> >>>>>>>>
> >>>>>>>>              /* Force PERST PCIe line into GPIO mode */
> >>>>>>>>              *(unsigned int *)(0xbe000060) &= ~(0x3 << 10);
> >>>>>>>>              *(unsigned int *)(0xbe000060) |=  BIT(10);
> >>>>>>>>              mdelay(100);
> >>>>>>>>
> >>>>>>>>              *(unsigned int *)(0xbe000600) |= BIT(19); // use GPIO19 (PERST_N/)
> >>>>>>>>              mdelay(100);
> >>>>>>>>              *(unsigned int *)(0xbe000620) &= ~(BIT(19)); // clear DATA
> >>>>>>>>              mdelay(1000);
> >>>>>>>>
> >>>>>>>>              /* Use GPIO control instead of PERST_N */
> >>>>>>>>              *(unsigned int *)(0xbe000620) |= BIT(19); // set DATA
> >>>>>>>>              mdelay(1000);
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>> I really hate the gpio way of doing this. If this works we have to
> >>>>>>> think in how to achieve this in a cleaner way :))
> >>>>>>>
> >>>>>>>> Regards
> >>>>>>>> Greg
> >>>>>>>
> >>>>>>> Thanks for your effort in this.
> >>>>>>>
> >>>>>>> Best regards,
> >>>>>>>         Sergio Paracuellos
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> Other big change is to use the new phy driver but I think the problem
> >>>>>>>>> seems to be related with resets.
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Regards
> >>>>>>>>>> Greg
> >>>>>>>>>
> >>>>>>>>> Please, don't hesitate to ask me whatever you need to.
> >>>>>>>>>
> >>>>>>>>> Hope this helps.
> >>>>>>>>>
> >>>>>>>>> Best regards,
> >>>>>>>>>          Sergio Paracuellos
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> >

[-- Attachment #2: 0001-staging-mt7621-pci-use-perst-gpio-instead-of-builtin.patch --]
[-- Type: text/x-patch, Size: 8008 bytes --]

From b7d26e7cca7d354d83fc217ab04a179019e1ad47 Mon Sep 17 00:00:00 2001
From: Sergio Paracuellos <sergio.paracuellos@gmail.com>
Date: Wed, 29 May 2019 09:58:07 +0200
Subject: [PATCH] staging: mt7621-pci: use perst gpio instead of builtin perst

Some boards need gpio instead of builtin perst. Use gpio for all
of them which was the approach of the original code.

Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
---
 drivers/staging/mt7621-dts/mt7621.dtsi  |  3 +-
 drivers/staging/mt7621-pci/pci-mt7621.c | 98 ++++++++++++++++-----------------
 2 files changed, 51 insertions(+), 50 deletions(-)

diff --git a/drivers/staging/mt7621-dts/mt7621.dtsi b/drivers/staging/mt7621-dts/mt7621.dtsi
index 280ec33..aed2458 100644
--- a/drivers/staging/mt7621-dts/mt7621.dtsi
+++ b/drivers/staging/mt7621-dts/mt7621.dtsi
@@ -1,5 +1,5 @@
 #include <dt-bindings/interrupt-controller/mips-gic.h>
-
+#include <dt-bindings/gpio/gpio.h>
 / {
 	#address-cells = <1>;
 	#size-cells = <1>;
@@ -468,6 +468,7 @@
 		#address-cells = <3>;
 		#size-cells = <2>;
 
+		perst-gpio = <&gpio 19 GPIO_ACTIVE_HIGH>;
 		pinctrl-names = "default";
 		pinctrl-0 = <&pcie_pins>;
 
diff --git a/drivers/staging/mt7621-pci/pci-mt7621.c b/drivers/staging/mt7621-pci/pci-mt7621.c
index 03d919a..8464985 100644
--- a/drivers/staging/mt7621-pci/pci-mt7621.c
+++ b/drivers/staging/mt7621-pci/pci-mt7621.c
@@ -17,6 +17,7 @@
 
 #include <linux/bitops.h>
 #include <linux/delay.h>
+#include <linux/gpio/consumer.h>
 #include <linux/iopoll.h>
 #include <linux/module.h>
 #include <linux/of.h>
@@ -75,13 +76,13 @@
 #define RALINK_PCI_STATUS		0x0050
 
 /* Some definition values */
+#define RALINK_PCI_IO_MAP_BASE  0x1e160000
 #define PCIE_REVISION_ID		BIT(0)
 #define PCIE_CLASS_CODE			(0x60400 << 8)
 #define PCIE_BAR_MAP_MAX		GENMASK(30, 16)
 #define PCIE_BAR_ENABLE			BIT(0)
 #define PCIE_PORT_INT_EN(x)		BIT(20 + (x))
 #define PCIE_PORT_CLK_EN(x)		BIT(24 + (x))
-#define PCIE_PORT_PERST(x)		BIT(1 + (x))
 #define PCIE_PORT_LINKUP		BIT(0)
 
 #define PCIE_CLK_GEN_EN			BIT(31)
@@ -90,6 +91,8 @@
 #define PCIE_CLK_GEN1_EN		(BIT(27) | BIT(25))
 #define MEMORY_BASE			0x0
 
+#define PERST_DELAY_US          1000
+
 /**
  * struct mt7621_pcie_port - PCIe port information
  * @base: I/O mapped register base
@@ -119,6 +122,7 @@ struct mt7621_pcie_port {
  * @offset: IO / Memory offset
  * @dev: Pointer to PCIe device
  * @ports: pointer to PCIe port information
+ * @perst: gpio reset
  * @rst: pointer to pcie reset
  */
 struct mt7621_pcie {
@@ -132,6 +136,7 @@ struct mt7621_pcie {
 		resource_size_t io;
 	} offset;
 	struct list_head ports;
+    struct gpio_desc *perst;
 	struct reset_control *rst;
 };
 
@@ -198,6 +203,18 @@ static void write_config(struct mt7621_pcie *pcie, unsigned int dev,
 	pcie_write(pcie, val, RALINK_PCI_CONFIG_DATA);
 }
 
+static void mt7621_perst_gpio_pcie_assert(struct mt7621_pcie *pcie)
+{
+    gpiod_set_value(pcie->perst, 0);
+    mdelay(PERST_DELAY_US);
+}
+
+static void mt7621_perst_gpio_pcie_deassert(struct mt7621_pcie *pcie)
+{
+    gpiod_set_value(pcie->perst, 1);
+    mdelay(PERST_DELAY_US);
+}
+
 static inline void mt7621_control_assert(struct mt7621_pcie_port *port)
 {
 	u32 chip_rev_id = rt_sysc_r32(MT7621_CHIP_REV_ID);
@@ -344,6 +361,12 @@ static int mt7621_pcie_parse_dt(struct mt7621_pcie *pcie)
 	struct resource regs;
 	int err;
 
+    pcie->perst = devm_gpiod_get(dev, "perst", GPIOD_OUT_HIGH);
+    if (IS_ERR(pcie->perst)) {
+        dev_err(dev, "failed to get gpio perst\n");
+        return PTR_ERR(pcie->perst);
+    }
+
 	err = of_address_to_resource(node, 0, &regs);
 	if (err) {
 		dev_err(dev, "missing \"reg\" property\n");
@@ -384,7 +407,6 @@ static int mt7621_pcie_init_port(struct mt7621_pcie_port *port)
 	struct mt7621_pcie *pcie = port->pcie;
 	struct device *dev = pcie->dev;
 	u32 slot = port->slot;
-	u32 val = 0;
 	int err;
 
 	/*
@@ -393,47 +415,32 @@ static int mt7621_pcie_init_port(struct mt7621_pcie_port *port)
 	 */
 	mt7621_reset_port(port);
 
-	val = read_config(pcie, slot, PCIE_FTS_NUM);
-	dev_info(dev, "Port %d N_FTS = %x\n", (unsigned int)val, slot);
-
 	err = phy_init(port->phy);
 	if (err) {
 		dev_err(dev, "failed to initialize port%d phy\n", slot);
-		goto err_phy_init;
+		return err;
 	}
 
 	err = phy_power_on(port->phy);
 	if (err) {
 		dev_err(dev, "failed to power on port%d phy\n", slot);
-		goto err_phy_on;
-	}
-
-	if ((pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) == 0) {
-		dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", slot);
-		mt7621_control_assert(port);
-		port->enabled = false;
-		err = -ENODEV;
-		goto err_no_link_up;
+		return err;
 	}
 
 	port->enabled = true;
 
 	return 0;
-
-err_no_link_up:
-	phy_power_off(port->phy);
-err_phy_on:
-	phy_exit(port->phy);
-err_phy_init:
-	return err;
 }
 
 static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct mt7621_pcie_port *port, *tmp;
+	u32 val = 0;
 	int err;
 
+    mt7621_perst_gpio_pcie_assert(pcie);
+
 	list_for_each_entry_safe(port, tmp, &pcie->ports, list) {
 		u32 slot = port->slot;
 
@@ -441,7 +448,10 @@ static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 		if (err) {
 			dev_err(dev, "Initiating port %d failed\n", slot);
 			list_del(&port->list);
-		}
+		} else {
+	        val = read_config(pcie, slot, PCIE_FTS_NUM);
+	        dev_info(dev, "Port %d N_FTS = %x\n", (unsigned int)val, slot);
+        }
 	}
 
 	reset_control_assert(pcie->rst);
@@ -451,32 +461,25 @@ static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 	rt_sysc_m32(PCIE_CLK_GEN_DIS, PCIE_CLK_GEN_EN, RALINK_PCIE_CLK_GEN);
 	msleep(50);
 	reset_control_deassert(pcie->rst);
+
+    mt7621_perst_gpio_pcie_deassert(pcie);
+
+	list_for_each_entry(port, &pcie->ports, list) {
+	    if ((pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) == 0) {
+		    dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", slot);
+		    mt7621_control_assert(port);
+            phy_power_off(port->phy);
+		    port->enabled = false;
+        }
+	}
 }
 
-static int mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
+static void mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
 {
 	struct mt7621_pcie *pcie = port->pcie;
 	u32 slot = port->slot;
 	u32 offset = MT7621_PCIE_OFFSET + (slot * MT7621_NEXT_PORT);
 	u32 val;
-	int err;
-
-	/* assert port PERST_N */
-	val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
-	val |= PCIE_PORT_PERST(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
-
-	/* de-assert port PERST_N */
-	val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
-	val &= ~PCIE_PORT_PERST(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
-
-	/* 100ms timeout value should be enough for Gen1 training */
-	err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
-				 val, !!(val & PCIE_PORT_LINKUP),
-				 20, 100 * USEC_PER_MSEC);
-	if (err)
-		return -ETIMEDOUT;
 
 	/* enable pcie interrupt */
 	val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR);
@@ -492,8 +495,6 @@ static int mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
 	/* configure class code and revision ID */
 	pcie_write(pcie, PCIE_CLASS_CODE | PCIE_REVISION_ID,
 		   offset + RALINK_PCI_CLASS);
-
-	return 0;
 }
 
 static void mt7621_pcie_enable_ports(struct mt7621_pcie *pcie)
@@ -506,11 +507,7 @@ static void mt7621_pcie_enable_ports(struct mt7621_pcie *pcie)
 
 	list_for_each_entry(port, &pcie->ports, list) {
 		if (port->enabled) {
-			if (mt7621_pcie_enable_port(port)) {
-				dev_err(dev, "de-assert port %d PERST_N\n",
-					port->slot);
-				continue;
-			}
+			mt7621_pcie_enable_port(port);
 			dev_info(dev, "PCIE%d enabled\n", slot);
 			num_slots_enabled++;
 		}
@@ -665,6 +662,9 @@ static int mt7621_pci_probe(struct platform_device *pdev)
 		return 0;
 	}
 
+    pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
+    pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
+
 	mt7621_pcie_enable_ports(pcie);
 
 	err = mt7621_pci_parse_request_of_pci_ranges(pcie);
-- 
2.7.4


[-- Attachment #3: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
[snip]
> I have added gpio consumer stuff and reorder a bit the code to be more
> similar to 4.20.
> 
> I attach the patch. I have not try it to compile it, because my normal
> environment is in another
> computer and I am in the middle of moving from my current house and
> don't have access to it, sorry.
> So, please try this and let's see what happens.

No problem, thanks for the patch.

Unfortunately always locks up on kernel boot:

   ...
   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
   mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
   mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)

That was original linux-5.1 patched with your attached patch.

I'll try and dig down into that further today and get some
feedback on where it is failing.

Regards
Greg

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Brett,

On 30/5/19 12:44 am, Brett Neumeier wrote:
> On Wed, May 29, 2019 at 3:09 AM Sergio Paracuellos <sergio.paracuellos@gmail.com <mailto:sergio.paracuellos@gmail.com>> wrote:
> 
>     I have added gpio consumer stuff and reorder a bit the code to be more
>     similar to 4.20.
> 
>     I attach the patch. I have not try it to compile it, because my normal
>     environment is in another
>     computer and I am in the middle of moving from my current house and
>     don't have access to it, sorry.
>     So, please try this and let's see what happens.
> 
> I'm jumping in late here because I just recently became aware of this thread. I have a GnuBee PC2 on which I'm running a 5.1.4 kernel with Neil Brown's patches applied; I'm having an issue where approximately 2/3 of the time the kernel hangs from a cold boot while configuring PCIe. I'd be happy to test whatever patches might help disagnose or correct what's going on. (I am not an expert device driver engineer or anything, so I probably won't be much help in other ways.)
> 
> In case it is helpful -- the kernel messages logged regardless of whether or not the problem occurs are:
> 
> mt7621-pci 1e140000.pcie: Parsing DT failed
> mt7621_gpio 1e000600.gpio: registering 32 gpios
> mt7621_gpio 1e000600.gpio: registering 32 gpios
> mt7621_gpio 1e000600.gpio: registering 32 gpios
> spi-mt7621 1e000b00.spi: sys_freq: 225000000
> rt2880-pinmux pinctrl: pcie is already enabled
> mt7621-pci 1e140000.pcie: Error applying setting, reverse things back
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: PCIE0 enabled
> mt7621-pci 1e140000.pcie: PCIE0 enabled
> mt7621-pci 1e140000.pcie: PCIE0 enabled
> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> pci_bus 0000:00: root bus resource [io  0xffffffff]
> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> pci_bus 0000:00: root bus resource [bus 00-ff]
> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> 
> at that point the boot process sometimes hangs.

FWIW, I see this occasional hang here too. Sometimes it boots through,
sometimes hangs - with unchanged code.

Difference is when I get a good boot, I never get the PCI bus
probed, and never any devices found.

Regards
Greg


> When it does not hang, it proceeds with:
> 
> pci 0000:01:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:00.0 (capable of 4.000 Gb/s with 5 GT/s x1 link)
> pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> pci 0000:02:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:01.0 (capable of 4.000 Gb/s with 5 GT/s x1 link)
> pci 0000:00:01.0: PCI bridge to [bus 02-ff]
> pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:02.0 (capable of 4.000 Gb/s with 5 GT/s x1 link)
> pci 0000:00:02.0: PCI bridge to [bus 03-ff]
> 
> and then does a bunch of resource assignments and things and all is well.
> 
> I'm building a new kernel with the "use perst gpio instead of builtin perst" patch and will report back my results. If there's anything else I can do to help, please let me know!
> 
> -- 
> Brett Neumeier (bneumeier@gmail.com <mailto:bneumeier@gmail.com>)
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 30/5/19 10:44 am, Greg Ungerer wrote:
> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
> [snip]
>> I have added gpio consumer stuff and reorder a bit the code to be more
>> similar to 4.20.
>>
>> I attach the patch. I have not try it to compile it, because my normal
>> environment is in another
>> computer and I am in the middle of moving from my current house and
>> don't have access to it, sorry.
>> So, please try this and let's see what happens.
> 
> No problem, thanks for the patch.
> 
> Unfortunately always locks up on kernel boot:
> 
>    ...
>    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>    mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>    mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>    mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>    mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> 
> That was original linux-5.1 patched with your attached patch.
> 
> I'll try and dig down into that further today and get some
> feedback on where it is failing.

The first problem I see is that the GPIO MODE register bits for
PERST_MODE are set to 00, so in "PCIe Reset" mode. If I hack in
a register update for that with:

     /* Force PERST PCIe reset into GPIO mode */
     *(unsigned int *)(0xbe000060) |=  BIT(10);

I do that at the start of mt7621_pcie_init_ports(). With that in
place I get further:

   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
   mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
   mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: PCIE0 enabled
   mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
   mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
   pci_bus 0000:00: root bus resource [io  0xffffffff]
   pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
   pci_bus 0000:00: root bus resource [bus 00-ff]
   pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring

It hangs there...

Regards
Greg


_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

[-- Attachment #1: Type: text/plain, Size: 3446 bytes --]

Hi Greg,

On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> On 30/5/19 10:44 am, Greg Ungerer wrote:
> > On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
> > [snip]
> >> I have added gpio consumer stuff and reorder a bit the code to be more
> >> similar to 4.20.
> >>
> >> I attach the patch. I have not try it to compile it, because my normal
> >> environment is in another
> >> computer and I am in the middle of moving from my current house and
> >> don't have access to it, sorry.
> >> So, please try this and let's see what happens.
> >
> > No problem, thanks for the patch.
> >
> > Unfortunately always locks up on kernel boot:
> >
> >    ...
> >    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >    mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >    mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> >    mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >    mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >
> > That was original linux-5.1 patched with your attached patch.
> >
> > I'll try and dig down into that further today and get some
> > feedback on where it is failing.
>
> The first problem I see is that the GPIO MODE register bits for
> PERST_MODE are set to 00, so in "PCIe Reset" mode. If I hack in
> a register update for that with:
>
>      /* Force PERST PCIe reset into GPIO mode */
>      *(unsigned int *)(0xbe000060) |=  BIT(10);

I have set GPIO mode for this in the new attached patch.

>
> I do that at the start of mt7621_pcie_init_ports(). With that in
> place I get further:
>
>    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>    mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>    mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>    mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>    mt7621-pci 1e140000.pcie: PCIE0 enabled
>    mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>    mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>    pci_bus 0000:00: root bus resource [io  0xffffffff]
>    pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>    pci_bus 0000:00: root bus resource [bus 00-ff]
>    pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>
> It hangs there...

I had review the boot order is working for you in version 4.20 and the
order with the new patch applied. There were
only one difference, the place where interrupt bits are set. I have
changed that also in the new attached patch.

For me, the order now and how the different boot steps are being done
in v4.20 are the same.

One other thing I don't really understand why is needed but is in the
v4.20 code are this two lines:

pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);

These are added also in the current patch.

>
> Regards
> Greg

Hope this helps.

Best regards,
    Sergio Paracuellos
>
>

[-- Attachment #2: 0001-staging-mt7621-pci-use-perst-gpio-instead-of-builtin.patch --]
[-- Type: text/x-patch, Size: 8700 bytes --]

From be6e894e4592ac0ad7edca590ebff8f5e80a9733 Mon Sep 17 00:00:00 2001
From: Sergio Paracuellos <sergio.paracuellos@gmail.com>
Date: Wed, 29 May 2019 09:58:07 +0200
Subject: [PATCH] staging: mt7621-pci: use perst gpio instead of builtin perst

Some boards need gpio instead of builtin perst. Use gpio for all
of them which was the approach of the original code.

Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
---
 drivers/staging/mt7621-dts/mt7621.dtsi  |   3 +-
 drivers/staging/mt7621-pci/pci-mt7621.c | 113 +++++++++++++++++---------------
 2 files changed, 61 insertions(+), 55 deletions(-)

diff --git a/drivers/staging/mt7621-dts/mt7621.dtsi b/drivers/staging/mt7621-dts/mt7621.dtsi
index 280ec33..aed2458 100644
--- a/drivers/staging/mt7621-dts/mt7621.dtsi
+++ b/drivers/staging/mt7621-dts/mt7621.dtsi
@@ -1,5 +1,5 @@
 #include <dt-bindings/interrupt-controller/mips-gic.h>
-
+#include <dt-bindings/gpio/gpio.h>
 / {
 	#address-cells = <1>;
 	#size-cells = <1>;
@@ -468,6 +468,7 @@
 		#address-cells = <3>;
 		#size-cells = <2>;
 
+		perst-gpio = <&gpio 19 GPIO_ACTIVE_HIGH>;
 		pinctrl-names = "default";
 		pinctrl-0 = <&pcie_pins>;
 
diff --git a/drivers/staging/mt7621-pci/pci-mt7621.c b/drivers/staging/mt7621-pci/pci-mt7621.c
index 03d919a..25970f5 100644
--- a/drivers/staging/mt7621-pci/pci-mt7621.c
+++ b/drivers/staging/mt7621-pci/pci-mt7621.c
@@ -17,6 +17,7 @@
 
 #include <linux/bitops.h>
 #include <linux/delay.h>
+#include <linux/gpio/consumer.h>
 #include <linux/iopoll.h>
 #include <linux/module.h>
 #include <linux/of.h>
@@ -35,6 +36,7 @@
 
 /* sysctl */
 #define MT7621_CHIP_REV_ID		0x0c
+#define MT7621_GPIO_MODE        0x60
 #define CHIP_REV_MT7621_E2		0x0101
 
 /* MediaTek specific configuration registers */
@@ -75,13 +77,13 @@
 #define RALINK_PCI_STATUS		0x0050
 
 /* Some definition values */
+#define RALINK_PCI_IO_MAP_BASE  0x1e160000
 #define PCIE_REVISION_ID		BIT(0)
 #define PCIE_CLASS_CODE			(0x60400 << 8)
 #define PCIE_BAR_MAP_MAX		GENMASK(30, 16)
 #define PCIE_BAR_ENABLE			BIT(0)
 #define PCIE_PORT_INT_EN(x)		BIT(20 + (x))
 #define PCIE_PORT_CLK_EN(x)		BIT(24 + (x))
-#define PCIE_PORT_PERST(x)		BIT(1 + (x))
 #define PCIE_PORT_LINKUP		BIT(0)
 
 #define PCIE_CLK_GEN_EN			BIT(31)
@@ -90,6 +92,9 @@
 #define PCIE_CLK_GEN1_EN		(BIT(27) | BIT(25))
 #define MEMORY_BASE			0x0
 
+#define PERST_MODE_GPIO         BIT(10)
+#define PERST_DELAY_US          1000
+
 /**
  * struct mt7621_pcie_port - PCIe port information
  * @base: I/O mapped register base
@@ -119,6 +124,7 @@ struct mt7621_pcie_port {
  * @offset: IO / Memory offset
  * @dev: Pointer to PCIe device
  * @ports: pointer to PCIe port information
+ * @perst: gpio reset
  * @rst: pointer to pcie reset
  */
 struct mt7621_pcie {
@@ -132,6 +138,7 @@ struct mt7621_pcie {
 		resource_size_t io;
 	} offset;
 	struct list_head ports;
+    struct gpio_desc *perst;
 	struct reset_control *rst;
 };
 
@@ -198,6 +205,18 @@ static void write_config(struct mt7621_pcie *pcie, unsigned int dev,
 	pcie_write(pcie, val, RALINK_PCI_CONFIG_DATA);
 }
 
+static void mt7621_perst_gpio_pcie_assert(struct mt7621_pcie *pcie)
+{
+    gpiod_set_value(pcie->perst, 0);
+    mdelay(PERST_DELAY_US);
+}
+
+static void mt7621_perst_gpio_pcie_deassert(struct mt7621_pcie *pcie)
+{
+    gpiod_set_value(pcie->perst, 1);
+    mdelay(PERST_DELAY_US);
+}
+
 static inline void mt7621_control_assert(struct mt7621_pcie_port *port)
 {
 	u32 chip_rev_id = rt_sysc_r32(MT7621_CHIP_REV_ID);
@@ -344,6 +363,12 @@ static int mt7621_pcie_parse_dt(struct mt7621_pcie *pcie)
 	struct resource regs;
 	int err;
 
+    pcie->perst = devm_gpiod_get(dev, "perst", GPIOD_OUT_HIGH);
+    if (IS_ERR(pcie->perst)) {
+        dev_err(dev, "failed to get gpio perst\n");
+        return PTR_ERR(pcie->perst);
+    }
+
 	err = of_address_to_resource(node, 0, &regs);
 	if (err) {
 		dev_err(dev, "missing \"reg\" property\n");
@@ -384,7 +409,6 @@ static int mt7621_pcie_init_port(struct mt7621_pcie_port *port)
 	struct mt7621_pcie *pcie = port->pcie;
 	struct device *dev = pcie->dev;
 	u32 slot = port->slot;
-	u32 val = 0;
 	int err;
 
 	/*
@@ -393,47 +417,33 @@ static int mt7621_pcie_init_port(struct mt7621_pcie_port *port)
 	 */
 	mt7621_reset_port(port);
 
-	val = read_config(pcie, slot, PCIE_FTS_NUM);
-	dev_info(dev, "Port %d N_FTS = %x\n", (unsigned int)val, slot);
-
 	err = phy_init(port->phy);
 	if (err) {
 		dev_err(dev, "failed to initialize port%d phy\n", slot);
-		goto err_phy_init;
+		return err;
 	}
 
 	err = phy_power_on(port->phy);
 	if (err) {
 		dev_err(dev, "failed to power on port%d phy\n", slot);
-		goto err_phy_on;
-	}
-
-	if ((pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) == 0) {
-		dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", slot);
-		mt7621_control_assert(port);
-		port->enabled = false;
-		err = -ENODEV;
-		goto err_no_link_up;
+		return err;
 	}
 
 	port->enabled = true;
 
 	return 0;
-
-err_no_link_up:
-	phy_power_off(port->phy);
-err_phy_on:
-	phy_exit(port->phy);
-err_phy_init:
-	return err;
 }
 
 static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct mt7621_pcie_port *port, *tmp;
+	u32 val = 0;
 	int err;
 
+    rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);
+    mt7621_perst_gpio_pcie_assert(pcie);
+
 	list_for_each_entry_safe(port, tmp, &pcie->ports, list) {
 		u32 slot = port->slot;
 
@@ -441,7 +451,10 @@ static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 		if (err) {
 			dev_err(dev, "Initiating port %d failed\n", slot);
 			list_del(&port->list);
-		}
+		} else {
+	        val = read_config(pcie, slot, PCIE_FTS_NUM);
+	        dev_info(dev, "Port %d N_FTS = %x\n", (unsigned int)val, slot);
+        }
 	}
 
 	reset_control_assert(pcie->rst);
@@ -451,37 +464,32 @@ static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 	rt_sysc_m32(PCIE_CLK_GEN_DIS, PCIE_CLK_GEN_EN, RALINK_PCIE_CLK_GEN);
 	msleep(50);
 	reset_control_deassert(pcie->rst);
+
+    mt7621_perst_gpio_pcie_deassert(pcie);
+
+	list_for_each_entry(port, &pcie->ports, list) {
+		u32 slot = port->slot;
+
+	    if ((pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) == 0) {
+		    dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", slot);
+		    mt7621_control_assert(port);
+            phy_power_off(port->phy);
+		    port->enabled = false;
+        } else {
+	        /* enable pcie interrupt */
+	        val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR);
+	        val |= PCIE_PORT_INT_EN(slot);
+	        pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR);
+        }
+	}
 }
 
-static int mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
+static void mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
 {
 	struct mt7621_pcie *pcie = port->pcie;
 	u32 slot = port->slot;
 	u32 offset = MT7621_PCIE_OFFSET + (slot * MT7621_NEXT_PORT);
 	u32 val;
-	int err;
-
-	/* assert port PERST_N */
-	val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
-	val |= PCIE_PORT_PERST(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
-
-	/* de-assert port PERST_N */
-	val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
-	val &= ~PCIE_PORT_PERST(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
-
-	/* 100ms timeout value should be enough for Gen1 training */
-	err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
-				 val, !!(val & PCIE_PORT_LINKUP),
-				 20, 100 * USEC_PER_MSEC);
-	if (err)
-		return -ETIMEDOUT;
-
-	/* enable pcie interrupt */
-	val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR);
-	val |= PCIE_PORT_INT_EN(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR);
 
 	/* map 2G DDR region */
 	pcie_write(pcie, PCIE_BAR_MAP_MAX | PCIE_BAR_ENABLE,
@@ -492,8 +500,6 @@ static int mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
 	/* configure class code and revision ID */
 	pcie_write(pcie, PCIE_CLASS_CODE | PCIE_REVISION_ID,
 		   offset + RALINK_PCI_CLASS);
-
-	return 0;
 }
 
 static void mt7621_pcie_enable_ports(struct mt7621_pcie *pcie)
@@ -506,11 +512,7 @@ static void mt7621_pcie_enable_ports(struct mt7621_pcie *pcie)
 
 	list_for_each_entry(port, &pcie->ports, list) {
 		if (port->enabled) {
-			if (mt7621_pcie_enable_port(port)) {
-				dev_err(dev, "de-assert port %d PERST_N\n",
-					port->slot);
-				continue;
-			}
+			mt7621_pcie_enable_port(port);
 			dev_info(dev, "PCIE%d enabled\n", slot);
 			num_slots_enabled++;
 		}
@@ -665,6 +667,9 @@ static int mt7621_pci_probe(struct platform_device *pdev)
 		return 0;
 	}
 
+    pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
+    pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
+
 	mt7621_pcie_enable_ports(pcie);
 
 	err = mt7621_pci_parse_request_of_pci_ranges(pcie);
-- 
2.7.4


[-- Attachment #3: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
>> On 30/5/19 10:44 am, Greg Ungerer wrote:
>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
>>> [snip]
>>>> I have added gpio consumer stuff and reorder a bit the code to be more
>>>> similar to 4.20.
>>>>
>>>> I attach the patch. I have not try it to compile it, because my normal
>>>> environment is in another
>>>> computer and I am in the middle of moving from my current house and
>>>> don't have access to it, sorry.
>>>> So, please try this and let's see what happens.
>>>
>>> No problem, thanks for the patch.
>>>
>>> Unfortunately always locks up on kernel boot:
>>>
>>>     ...
>>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>     mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>>>     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>
>>> That was original linux-5.1 patched with your attached patch.
>>>
>>> I'll try and dig down into that further today and get some
>>> feedback on where it is failing.
>>
>> The first problem I see is that the GPIO MODE register bits for
>> PERST_MODE are set to 00, so in "PCIe Reset" mode. If I hack in
>> a register update for that with:
>>
>>       /* Force PERST PCIe reset into GPIO mode */
>>       *(unsigned int *)(0xbe000060) |=  BIT(10);
> 
> I have set GPIO mode for this in the new attached patch.
> 
>>
>> I do that at the start of mt7621_pcie_init_ports(). With that in
>> place I get further:
>>
>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>     mt7621-pci 1e140000.pcie: PCIE0 enabled
>>     mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>     mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>     pci_bus 0000:00: root bus resource [io  0xffffffff]
>>     pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>     pci_bus 0000:00: root bus resource [bus 00-ff]
>>     pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>>
>> It hangs there...
> 
> I had review the boot order is working for you in version 4.20 and the
> order with the new patch applied. There were
> only one difference, the place where interrupt bits are set. I have
> changed that also in the new attached patch.
> 
> For me, the order now and how the different boot steps are being done
> in v4.20 are the same.
> 
> One other thing I don't really understand why is needed but is in the
> v4.20 code are this two lines:
> 
> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
> 
> These are added also in the current patch.

Thats great, thanks for your efforts on this.
I will try first thing Monday morning my time and get back to you.

Regards
Greg



>> Regards
>> Greg
> 
> Hope this helps.
> 
> Best regards,
>      Sergio Paracuellos
>>
>>
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
>> On 30/5/19 10:44 am, Greg Ungerer wrote:
>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
>>> [snip]
>>>> I have added gpio consumer stuff and reorder a bit the code to be more
>>>> similar to 4.20.
>>>>
>>>> I attach the patch. I have not try it to compile it, because my normal
>>>> environment is in another
>>>> computer and I am in the middle of moving from my current house and
>>>> don't have access to it, sorry.
>>>> So, please try this and let's see what happens.
>>>
>>> No problem, thanks for the patch.
>>>
>>> Unfortunately always locks up on kernel boot:
>>>
>>>     ...
>>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>     mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>>>     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>
>>> That was original linux-5.1 patched with your attached patch.
>>>
>>> I'll try and dig down into that further today and get some
>>> feedback on where it is failing.
>>
>> The first problem I see is that the GPIO MODE register bits for
>> PERST_MODE are set to 00, so in "PCIe Reset" mode. If I hack in
>> a register update for that with:
>>
>>       /* Force PERST PCIe reset into GPIO mode */
>>       *(unsigned int *)(0xbe000060) |=  BIT(10);
> 
> I have set GPIO mode for this in the new attached patch.
> 
>>
>> I do that at the start of mt7621_pcie_init_ports(). With that in
>> place I get further:
>>
>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>     mt7621-pci 1e140000.pcie: PCIE0 enabled
>>     mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>     mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>     pci_bus 0000:00: root bus resource [io  0xffffffff]
>>     pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>     pci_bus 0000:00: root bus resource [bus 00-ff]
>>     pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>>
>> It hangs there...
> 
> I had review the boot order is working for you in version 4.20 and the
> order with the new patch applied. There were
> only one difference, the place where interrupt bits are set. I have
> changed that also in the new attached patch.
> 
> For me, the order now and how the different boot steps are being done
> in v4.20 are the same.
> 
> One other thing I don't really understand why is needed but is in the
> v4.20 code are this two lines:
> 
> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
> 
> These are added also in the current patch.

Tried out this latest patch. Unfortunately no good news.

Boot gets through the PCI bus scan, but does not find any devices:

   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
   mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
   mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: Nothing is connected in virtual bridges. Exiting...

And now in the completely weird and un-expected department the boot
continues on and appears to hang for me when it tries to attach a
UBI NAND flash partition. It hangs there for about a minute or so
and then dumps complaing about flash problems:

   ubi0: attaching mtd3
   ubi0: scanning is finished
   ubi0: empty MTD device detected
   ubi0 warning: do_sync_erase: error -5 while erasing PEB 0, retry
   ubi0 warning: do_sync_erase: error -5 while erasing PEB 0, retry
   ubi0 warning: do_sync_erase: error -5 while erasing PEB 0, retry
   ubi0 error: do_sync_erase: cannot erase PEB 0, error -5
   CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.1.0-ac0 #59
   Stack : 000000f0 00000000 00000000 808e0000 8ebf2000 80070584 808285b8 0000000b
         00000038 00000000 80827e00 8fc23b1c 80870000 00000001 8fc23ab0 aeddef9b
         00000000 00000000 80920000 00000000 00000000 808e7693 000000f1 00000000
         203a6d6d 00000000 00000000 00000000 80870000 00000000 00000000 8080a01c
         00000000 80870000 8ebf1014 8ebd2000 00000018 80361d24 00000004 808e0004
         ...
   Call Trace:
   [<8000cfc0>] show_stack+0x94/0x12c
   [<806e553c>] dump_stack+0x8c/0xd0
   [<803c73c8>] do_sync_erase+0xf4/0x208
   [<803c7694>] ubi_io_sync_erase+0x1b8/0x304
   [<803cbc90>] ubi_early_get_peb+0x148/0x1dc
   [<803ba930>] create_vtbl+0xb4/0x29c
   [<803bafc8>] ubi_read_volume_table+0x27c/0xae4
   [<803cc294>] ubi_attach+0x570/0x15dc
   [<803bf2a0>] ubi_attach_mtd_dev+0x5b0/0xbec
   [<808b0488>] ubi_init+0x1c0/0x274
   [<800015f4>] do_one_initcall+0x50/0x1ac
   [<80897e98>] kernel_init_freeable+0x184/0x26c
   [<807035b4>] kernel_init+0x14/0x110
   [<800071f8>] ret_from_kernel_thread+0x14/0x1c

I tried booting and running this several times, I always get the same
long hang and dump from USB/NAND. If I copy in the one linux-4.20
file, drivers/staging/mt7621-pci/pci-mt7621.c, then the system
boots, scans and finds PCI devices, and does not hang/dump on
UBI/NAND flash setup.

I'll try and dig into it some time today and get you some feedback.

Regards
Greg



_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

Hi Greg,

No luck for us also today :-(

On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
> > On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
> >> On 30/5/19 10:44 am, Greg Ungerer wrote:
> >>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
> >>> [snip]
> >>>> I have added gpio consumer stuff and reorder a bit the code to be more
> >>>> similar to 4.20.
> >>>>
> >>>> I attach the patch. I have not try it to compile it, because my normal
> >>>> environment is in another
> >>>> computer and I am in the middle of moving from my current house and
> >>>> don't have access to it, sorry.
> >>>> So, please try this and let's see what happens.
> >>>
> >>> No problem, thanks for the patch.
> >>>
> >>> Unfortunately always locks up on kernel boot:
> >>>
> >>>     ...
> >>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>     mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> >>>     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>
> >>> That was original linux-5.1 patched with your attached patch.
> >>>
> >>> I'll try and dig down into that further today and get some
> >>> feedback on where it is failing.
> >>
> >> The first problem I see is that the GPIO MODE register bits for
> >> PERST_MODE are set to 00, so in "PCIe Reset" mode. If I hack in
> >> a register update for that with:
> >>
> >>       /* Force PERST PCIe reset into GPIO mode */
> >>       *(unsigned int *)(0xbe000060) |=  BIT(10);
> >
> > I have set GPIO mode for this in the new attached patch.
> >
> >>
> >> I do that at the start of mt7621_pcie_init_ports(). With that in
> >> place I get further:
> >>
> >>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>     mt7621-pci 1e140000.pcie: PCIE0 enabled
> >>     mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >>     mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >>     pci_bus 0000:00: root bus resource [io  0xffffffff]
> >>     pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >>     pci_bus 0000:00: root bus resource [bus 00-ff]
> >>     pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> >>
> >> It hangs there...
> >
> > I had review the boot order is working for you in version 4.20 and the
> > order with the new patch applied. There were
> > only one difference, the place where interrupt bits are set. I have
> > changed that also in the new attached patch.
> >
> > For me, the order now and how the different boot steps are being done
> > in v4.20 are the same.
> >
> > One other thing I don't really understand why is needed but is in the
> > v4.20 code are this two lines:
> >
> > pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
> > pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
> >
> > These are added also in the current patch.
>
> Tried out this latest patch. Unfortunately no good news.
>
> Boot gets through the PCI bus scan, but does not find any devices:
>
>    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>    mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>    mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>    mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>    mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>    mt7621-pci 1e140000.pcie: Nothing is connected in virtual bridges. Exiting...



>
> And now in the completely weird and un-expected department the boot
> continues on and appears to hang for me when it tries to attach a
> UBI NAND flash partition. It hangs there for about a minute or so
> and then dumps complaing about flash problems:
>
>    ubi0: attaching mtd3
>    ubi0: scanning is finished
>    ubi0: empty MTD device detected
>    ubi0 warning: do_sync_erase: error -5 while erasing PEB 0, retry
>    ubi0 warning: do_sync_erase: error -5 while erasing PEB 0, retry
>    ubi0 warning: do_sync_erase: error -5 while erasing PEB 0, retry
>    ubi0 error: do_sync_erase: cannot erase PEB 0, error -5
>    CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.1.0-ac0 #59
>    Stack : 000000f0 00000000 00000000 808e0000 8ebf2000 80070584 808285b8 0000000b
>          00000038 00000000 80827e00 8fc23b1c 80870000 00000001 8fc23ab0 aeddef9b
>          00000000 00000000 80920000 00000000 00000000 808e7693 000000f1 00000000
>          203a6d6d 00000000 00000000 00000000 80870000 00000000 00000000 8080a01c
>          00000000 80870000 8ebf1014 8ebd2000 00000018 80361d24 00000004 808e0004
>          ...
>    Call Trace:
>    [<8000cfc0>] show_stack+0x94/0x12c
>    [<806e553c>] dump_stack+0x8c/0xd0
>    [<803c73c8>] do_sync_erase+0xf4/0x208
>    [<803c7694>] ubi_io_sync_erase+0x1b8/0x304
>    [<803cbc90>] ubi_early_get_peb+0x148/0x1dc
>    [<803ba930>] create_vtbl+0xb4/0x29c
>    [<803bafc8>] ubi_read_volume_table+0x27c/0xae4
>    [<803cc294>] ubi_attach+0x570/0x15dc
>    [<803bf2a0>] ubi_attach_mtd_dev+0x5b0/0xbec
>    [<808b0488>] ubi_init+0x1c0/0x274
>    [<800015f4>] do_one_initcall+0x50/0x1ac
>    [<80897e98>] kernel_init_freeable+0x184/0x26c
>    [<807035b4>] kernel_init+0x14/0x110
>    [<800071f8>] ret_from_kernel_thread+0x14/0x1c
>
> I tried booting and running this several times, I always get the same
> long hang and dump from USB/NAND. If I copy in the one linux-4.20
> file, drivers/staging/mt7621-pci/pci-mt7621.c, then the system
> boots, scans and finds PCI devices, and does not hang/dump on
> UBI/NAND flash setup.

Can you try to read and set BIT(10) instead of write it directly?:

Instead of:

rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);

Do:

u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
val |= PERST_MODE_GPIO;
rt_sysc_w32(val, MT7621_GPIO_MODE);


>
> I'll try and dig into it some time today and get you some feedback.

No other changes with the previous one, just the order of where
interrupts bits are set up in
the same place of 4.20.

Can you point me out to the link to your board of something to check
if I can acquire one and test
in my side?

>
> Regards
> Greg

Best regards,
    Sergio Paracuellos
>
>
>
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 3/6/19 3:34 pm, Sergio Paracuellos wrote:
> On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
>> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
>>> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>> On 30/5/19 10:44 am, Greg Ungerer wrote:
>>>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
>>>>> [snip]
>>>>>> I have added gpio consumer stuff and reorder a bit the code to be more
>>>>>> similar to 4.20.
>>>>>>
>>>>>> I attach the patch. I have not try it to compile it, because my normal
>>>>>> environment is in another
>>>>>> computer and I am in the middle of moving from my current house and
>>>>>> don't have access to it, sorry.
>>>>>> So, please try this and let's see what happens.
>>>>>
>>>>> No problem, thanks for the patch.
>>>>>
>>>>> Unfortunately always locks up on kernel boot:
>>>>>
>>>>>      ...
>>>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>>>      mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>>>      mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>>>>>      mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>>      mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>>>
>>>>> That was original linux-5.1 patched with your attached patch.
>>>>>
>>>>> I'll try and dig down into that further today and get some
>>>>> feedback on where it is failing.
>>>>
>>>> The first problem I see is that the GPIO MODE register bits for
>>>> PERST_MODE are set to 00, so in "PCIe Reset" mode. If I hack in
>>>> a register update for that with:
>>>>
>>>>        /* Force PERST PCIe reset into GPIO mode */
>>>>        *(unsigned int *)(0xbe000060) |=  BIT(10);
>>>
>>> I have set GPIO mode for this in the new attached patch.
>>>
>>>>
>>>> I do that at the start of mt7621_pcie_init_ports(). With that in
>>>> place I get further:
>>>>
>>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>>      mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>>      mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>      mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>>      mt7621-pci 1e140000.pcie: PCIE0 enabled
>>>>      mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>>>      mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>>>      pci_bus 0000:00: root bus resource [io  0xffffffff]
>>>>      pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>>>      pci_bus 0000:00: root bus resource [bus 00-ff]
>>>>      pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>>>>
>>>> It hangs there...
>>>
>>> I had review the boot order is working for you in version 4.20 and the
>>> order with the new patch applied. There were
>>> only one difference, the place where interrupt bits are set. I have
>>> changed that also in the new attached patch.
>>>
>>> For me, the order now and how the different boot steps are being done
>>> in v4.20 are the same.
>>>
>>> One other thing I don't really understand why is needed but is in the
>>> v4.20 code are this two lines:
>>>
>>> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
>>> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
>>>
>>> These are added also in the current patch.
>>
>> Tried out this latest patch. Unfortunately no good news.
>>
>> Boot gets through the PCI bus scan, but does not find any devices:
>>
>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>     mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>>     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>     mt7621-pci 1e140000.pcie: Nothing is connected in virtual bridges. Exiting...
> 
> 
> 
>>
>> And now in the completely weird and un-expected department the boot
>> continues on and appears to hang for me when it tries to attach a
>> UBI NAND flash partition. It hangs there for about a minute or so
>> and then dumps complaing about flash problems:
>>
>>     ubi0: attaching mtd3
>>     ubi0: scanning is finished
>>     ubi0: empty MTD device detected
>>     ubi0 warning: do_sync_erase: error -5 while erasing PEB 0, retry
>>     ubi0 warning: do_sync_erase: error -5 while erasing PEB 0, retry
>>     ubi0 warning: do_sync_erase: error -5 while erasing PEB 0, retry
>>     ubi0 error: do_sync_erase: cannot erase PEB 0, error -5
>>     CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.1.0-ac0 #59
>>     Stack : 000000f0 00000000 00000000 808e0000 8ebf2000 80070584 808285b8 0000000b
>>           00000038 00000000 80827e00 8fc23b1c 80870000 00000001 8fc23ab0 aeddef9b
>>           00000000 00000000 80920000 00000000 00000000 808e7693 000000f1 00000000
>>           203a6d6d 00000000 00000000 00000000 80870000 00000000 00000000 8080a01c
>>           00000000 80870000 8ebf1014 8ebd2000 00000018 80361d24 00000004 808e0004
>>           ...
>>     Call Trace:
>>     [<8000cfc0>] show_stack+0x94/0x12c
>>     [<806e553c>] dump_stack+0x8c/0xd0
>>     [<803c73c8>] do_sync_erase+0xf4/0x208
>>     [<803c7694>] ubi_io_sync_erase+0x1b8/0x304
>>     [<803cbc90>] ubi_early_get_peb+0x148/0x1dc
>>     [<803ba930>] create_vtbl+0xb4/0x29c
>>     [<803bafc8>] ubi_read_volume_table+0x27c/0xae4
>>     [<803cc294>] ubi_attach+0x570/0x15dc
>>     [<803bf2a0>] ubi_attach_mtd_dev+0x5b0/0xbec
>>     [<808b0488>] ubi_init+0x1c0/0x274
>>     [<800015f4>] do_one_initcall+0x50/0x1ac
>>     [<80897e98>] kernel_init_freeable+0x184/0x26c
>>     [<807035b4>] kernel_init+0x14/0x110
>>     [<800071f8>] ret_from_kernel_thread+0x14/0x1c
>>
>> I tried booting and running this several times, I always get the same
>> long hang and dump from USB/NAND. If I copy in the one linux-4.20
>> file, drivers/staging/mt7621-pci/pci-mt7621.c, then the system
>> boots, scans and finds PCI devices, and does not hang/dump on
>> UBI/NAND flash setup.
> 
> Can you try to read and set BIT(10) instead of write it directly?:
> 
> Instead of:
> 
> rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);

Oh, yeah, that is definitely not going to work. There is a bunch of
other GPIO control bits in there for other hardware blocks. Would
explain the broken NAND flash behavior...


> Do:
> 
> u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
> val |= PERST_MODE_GPIO;
> rt_sysc_w32(val, MT7621_GPIO_MODE);

Much better result with that. Though ultimately it should clear
bits 10 and 11 (both are PERST_MODE bits) and then OR in BIT(10).

Boot is successful and now shows:

mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
mt7621-pci 1e140000.pcie: PCIE0 enabled
mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0xffffffff]
pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
pci_bus 0000:00: root bus resource [bus 00-ff]
pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:00.0: PCI bridge to [bus 01-ff]
pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
pci 0000:00:00.0: PCI bridge to [bus 01]
pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
pcieport 0000:00:00.0: enabling device (0004 -> 0006)


So that is really good. Still now just some problem with the IRQ.

I also found that I could dump /sys/bus/pci/devices/0000:01:00.0/config
and get a good dump from the command line:

00000000: 8c 16 3c 00 06 00 10 00 00 00 80 02 00 00 00 00     ..<.............
00000010: 04 00 00 60 00 00 00 00 00 00 00 00 00 00 00 00     ...`............
00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00     ................
00000030: 00 00 00 00 40 00 00 00 00 00 00 00 17 01 00 00     ....@...........
...

Also good.


>> I'll try and dig into it some time today and get you some feedback.

Sorry, I didn't get any more time to look at this today.


> No other changes with the previous one, just the order of where
> interrupts bits are set up in
> the same place of 4.20.
> 
> Can you point me out to the link to your board of something to check
> if I can acquire one and test
> in my side?

I am using a Digi/EX15:
https://www.digi.com/products/networking/cellular-routers/enterprise/digi-ex15

FWIW, I think we are close now.

Regards
Greg
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

[-- Attachment #1: Type: text/plain, Size: 11443 bytes --]

Hi Greg,

On Mon, Jun 3, 2019 at 2:32 PM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> On 3/6/19 3:34 pm, Sergio Paracuellos wrote:
> > On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
> >> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
> >>> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>> On 30/5/19 10:44 am, Greg Ungerer wrote:
> >>>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
> >>>>> [snip]
> >>>>>> I have added gpio consumer stuff and reorder a bit the code to be more
> >>>>>> similar to 4.20.
> >>>>>>
> >>>>>> I attach the patch. I have not try it to compile it, because my normal
> >>>>>> environment is in another
> >>>>>> computer and I am in the middle of moving from my current house and
> >>>>>> don't have access to it, sorry.
> >>>>>> So, please try this and let's see what happens.
> >>>>>
> >>>>> No problem, thanks for the patch.
> >>>>>
> >>>>> Unfortunately always locks up on kernel boot:
> >>>>>
> >>>>>      ...
> >>>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>>>      mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>>>      mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> >>>>>      mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>>>      mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>>>
> >>>>> That was original linux-5.1 patched with your attached patch.
> >>>>>
> >>>>> I'll try and dig down into that further today and get some
> >>>>> feedback on where it is failing.
> >>>>
> >>>> The first problem I see is that the GPIO MODE register bits for
> >>>> PERST_MODE are set to 00, so in "PCIe Reset" mode. If I hack in
> >>>> a register update for that with:
> >>>>
> >>>>        /* Force PERST PCIe reset into GPIO mode */
> >>>>        *(unsigned int *)(0xbe000060) |=  BIT(10);
> >>>
> >>> I have set GPIO mode for this in the new attached patch.
> >>>
> >>>>
> >>>> I do that at the start of mt7621_pcie_init_ports(). With that in
> >>>> place I get further:
> >>>>
> >>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>>      mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>>      mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>>      mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>>      mt7621-pci 1e140000.pcie: PCIE0 enabled
> >>>>      mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >>>>      mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >>>>      pci_bus 0000:00: root bus resource [io  0xffffffff]
> >>>>      pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >>>>      pci_bus 0000:00: root bus resource [bus 00-ff]
> >>>>      pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> >>>>
> >>>> It hangs there...
> >>>
> >>> I had review the boot order is working for you in version 4.20 and the
> >>> order with the new patch applied. There were
> >>> only one difference, the place where interrupt bits are set. I have
> >>> changed that also in the new attached patch.
> >>>
> >>> For me, the order now and how the different boot steps are being done
> >>> in v4.20 are the same.
> >>>
> >>> One other thing I don't really understand why is needed but is in the
> >>> v4.20 code are this two lines:
> >>>
> >>> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
> >>> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
> >>>
> >>> These are added also in the current patch.
> >>
> >> Tried out this latest patch. Unfortunately no good news.
> >>
> >> Boot gets through the PCI bus scan, but does not find any devices:
> >>
> >>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>     mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> >>     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>     mt7621-pci 1e140000.pcie: Nothing is connected in virtual bridges. Exiting...
> >
> >
> >
> >>
> >> And now in the completely weird and un-expected department the boot
> >> continues on and appears to hang for me when it tries to attach a
> >> UBI NAND flash partition. It hangs there for about a minute or so
> >> and then dumps complaing about flash problems:
> >>
> >>     ubi0: attaching mtd3
> >>     ubi0: scanning is finished
> >>     ubi0: empty MTD device detected
> >>     ubi0 warning: do_sync_erase: error -5 while erasing PEB 0, retry
> >>     ubi0 warning: do_sync_erase: error -5 while erasing PEB 0, retry
> >>     ubi0 warning: do_sync_erase: error -5 while erasing PEB 0, retry
> >>     ubi0 error: do_sync_erase: cannot erase PEB 0, error -5
> >>     CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.1.0-ac0 #59
> >>     Stack : 000000f0 00000000 00000000 808e0000 8ebf2000 80070584 808285b8 0000000b
> >>           00000038 00000000 80827e00 8fc23b1c 80870000 00000001 8fc23ab0 aeddef9b
> >>           00000000 00000000 80920000 00000000 00000000 808e7693 000000f1 00000000
> >>           203a6d6d 00000000 00000000 00000000 80870000 00000000 00000000 8080a01c
> >>           00000000 80870000 8ebf1014 8ebd2000 00000018 80361d24 00000004 808e0004
> >>           ...
> >>     Call Trace:
> >>     [<8000cfc0>] show_stack+0x94/0x12c
> >>     [<806e553c>] dump_stack+0x8c/0xd0
> >>     [<803c73c8>] do_sync_erase+0xf4/0x208
> >>     [<803c7694>] ubi_io_sync_erase+0x1b8/0x304
> >>     [<803cbc90>] ubi_early_get_peb+0x148/0x1dc
> >>     [<803ba930>] create_vtbl+0xb4/0x29c
> >>     [<803bafc8>] ubi_read_volume_table+0x27c/0xae4
> >>     [<803cc294>] ubi_attach+0x570/0x15dc
> >>     [<803bf2a0>] ubi_attach_mtd_dev+0x5b0/0xbec
> >>     [<808b0488>] ubi_init+0x1c0/0x274
> >>     [<800015f4>] do_one_initcall+0x50/0x1ac
> >>     [<80897e98>] kernel_init_freeable+0x184/0x26c
> >>     [<807035b4>] kernel_init+0x14/0x110
> >>     [<800071f8>] ret_from_kernel_thread+0x14/0x1c
> >>
> >> I tried booting and running this several times, I always get the same
> >> long hang and dump from USB/NAND. If I copy in the one linux-4.20
> >> file, drivers/staging/mt7621-pci/pci-mt7621.c, then the system
> >> boots, scans and finds PCI devices, and does not hang/dump on
> >> UBI/NAND flash setup.
> >
> > Can you try to read and set BIT(10) instead of write it directly?:
> >
> > Instead of:
> >
> > rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);
>
> Oh, yeah, that is definitely not going to work. There is a bunch of
> other GPIO control bits in there for other hardware blocks. Would
> explain the broken NAND flash behavior...

Yes, my bad here. Sometimes is better to go to sleep :)).

>
>
> > Do:
> >
> > u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
> > val |= PERST_MODE_GPIO;
> > rt_sysc_w32(val, MT7621_GPIO_MODE);
>
> Much better result with that. Though ultimately it should clear
> bits 10 and 11 (both are PERST_MODE bits) and then OR in BIT(10).

Ok, so the following should do the trick:

rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);

with PERST_MODE_MASK defined as:

#define PERST_MODE_MASK         GENMASK(11, 10)

(patch attached with this changes)

It would be also good to know what happen if you comment the following lines:

pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);

Are they really needed?

>
> Boot is successful and now shows:
>
> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> mt7621-pci 1e140000.pcie: PCIE0 enabled
> mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> pci_bus 0000:00: root bus resource [io  0xffffffff]
> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> pci_bus 0000:00: root bus resource [bus 00-ff]
> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
> pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
> pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
> pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
> pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
> pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
> pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
> pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
> pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
> pci 0000:00:00.0: PCI bridge to [bus 01]
> pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
> pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
>
>
> So that is really good. Still now just some problem with the IRQ.

No idea at all why irq is failing there. The driver code related with
irq is the same for 4.20.
Some debug traces would be useful.

>
> I also found that I could dump /sys/bus/pci/devices/0000:01:00.0/config
> and get a good dump from the command line:
>
> 00000000: 8c 16 3c 00 06 00 10 00 00 00 80 02 00 00 00 00     ..<.............
> 00000010: 04 00 00 60 00 00 00 00 00 00 00 00 00 00 00 00     ...`............
> 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00     ................
> 00000030: 00 00 00 00 40 00 00 00 00 00 00 00 17 01 00 00     ....@...........
> ...
>
> Also good.

Yeah, That looks good. Much  better than getting all F's like the other day.

>
>
> >> I'll try and dig into it some time today and get you some feedback.
>
> Sorry, I didn't get any more time to look at this today.
>

No problem at all.

>
> > No other changes with the previous one, just the order of where
> > interrupts bits are set up in
> > the same place of 4.20.
> >
> > Can you point me out to the link to your board of something to check
> > if I can acquire one and test
> > in my side?
>
> I am using a Digi/EX15:
> https://www.digi.com/products/networking/cellular-routers/enterprise/digi-ex15
>
> FWIW, I think we are close now.

Only one step more to get this properly working.

>
> Regards
> Greg

Best regards,
    Sergio Paracuellos

[-- Attachment #2: 0001-staging-mt7621-pci-use-perst-gpio-instead-of-builtin.patch --]
[-- Type: text/x-patch, Size: 8761 bytes --]

From cb33266157579ecaa720e6b8385b972264f999f6 Mon Sep 17 00:00:00 2001
From: Sergio Paracuellos <sparacuellos@ikergune.com>
Date: Wed, 29 May 2019 09:58:07 +0200
Subject: [PATCH] staging: mt7621-pci: use perst gpio instead of builtin perst

Some boards need gpio instead of builtin perst. Use gpio for all
of them which was the approach of the original code.

Signed-off-by: Sergio Paracuellos <sparacuellos@ikergune.com>
---
 drivers/staging/mt7621-dts/mt7621.dtsi  |   3 +-
 drivers/staging/mt7621-pci/pci-mt7621.c | 114 +++++++++++++++++---------------
 2 files changed, 62 insertions(+), 55 deletions(-)

diff --git a/drivers/staging/mt7621-dts/mt7621.dtsi b/drivers/staging/mt7621-dts/mt7621.dtsi
index 280ec33..aed2458 100644
--- a/drivers/staging/mt7621-dts/mt7621.dtsi
+++ b/drivers/staging/mt7621-dts/mt7621.dtsi
@@ -1,5 +1,5 @@
 #include <dt-bindings/interrupt-controller/mips-gic.h>
-
+#include <dt-bindings/gpio/gpio.h>
 / {
 	#address-cells = <1>;
 	#size-cells = <1>;
@@ -468,6 +468,7 @@
 		#address-cells = <3>;
 		#size-cells = <2>;
 
+		perst-gpio = <&gpio 19 GPIO_ACTIVE_HIGH>;
 		pinctrl-names = "default";
 		pinctrl-0 = <&pcie_pins>;
 
diff --git a/drivers/staging/mt7621-pci/pci-mt7621.c b/drivers/staging/mt7621-pci/pci-mt7621.c
index 03d919a..9ff4a8f 100644
--- a/drivers/staging/mt7621-pci/pci-mt7621.c
+++ b/drivers/staging/mt7621-pci/pci-mt7621.c
@@ -17,6 +17,7 @@
 
 #include <linux/bitops.h>
 #include <linux/delay.h>
+#include <linux/gpio/consumer.h>
 #include <linux/iopoll.h>
 #include <linux/module.h>
 #include <linux/of.h>
@@ -35,6 +36,7 @@
 
 /* sysctl */
 #define MT7621_CHIP_REV_ID		0x0c
+#define MT7621_GPIO_MODE        0x60
 #define CHIP_REV_MT7621_E2		0x0101
 
 /* MediaTek specific configuration registers */
@@ -75,13 +77,13 @@
 #define RALINK_PCI_STATUS		0x0050
 
 /* Some definition values */
+#define RALINK_PCI_IO_MAP_BASE  0x1e160000
 #define PCIE_REVISION_ID		BIT(0)
 #define PCIE_CLASS_CODE			(0x60400 << 8)
 #define PCIE_BAR_MAP_MAX		GENMASK(30, 16)
 #define PCIE_BAR_ENABLE			BIT(0)
 #define PCIE_PORT_INT_EN(x)		BIT(20 + (x))
 #define PCIE_PORT_CLK_EN(x)		BIT(24 + (x))
-#define PCIE_PORT_PERST(x)		BIT(1 + (x))
 #define PCIE_PORT_LINKUP		BIT(0)
 
 #define PCIE_CLK_GEN_EN			BIT(31)
@@ -90,6 +92,10 @@
 #define PCIE_CLK_GEN1_EN		(BIT(27) | BIT(25))
 #define MEMORY_BASE			0x0
 
+#define PERST_MODE_MASK         GENMASK(11, 10)
+#define PERST_MODE_GPIO         BIT(10)
+#define PERST_DELAY_US          1000
+
 /**
  * struct mt7621_pcie_port - PCIe port information
  * @base: I/O mapped register base
@@ -119,6 +125,7 @@ struct mt7621_pcie_port {
  * @offset: IO / Memory offset
  * @dev: Pointer to PCIe device
  * @ports: pointer to PCIe port information
+ * @perst: gpio reset
  * @rst: pointer to pcie reset
  */
 struct mt7621_pcie {
@@ -132,6 +139,7 @@ struct mt7621_pcie {
 		resource_size_t io;
 	} offset;
 	struct list_head ports;
+    struct gpio_desc *perst;
 	struct reset_control *rst;
 };
 
@@ -198,6 +206,18 @@ static void write_config(struct mt7621_pcie *pcie, unsigned int dev,
 	pcie_write(pcie, val, RALINK_PCI_CONFIG_DATA);
 }
 
+static void mt7621_perst_gpio_pcie_assert(struct mt7621_pcie *pcie)
+{
+    gpiod_set_value(pcie->perst, 0);
+    mdelay(PERST_DELAY_US);
+}
+
+static void mt7621_perst_gpio_pcie_deassert(struct mt7621_pcie *pcie)
+{
+    gpiod_set_value(pcie->perst, 1);
+    mdelay(PERST_DELAY_US);
+}
+
 static inline void mt7621_control_assert(struct mt7621_pcie_port *port)
 {
 	u32 chip_rev_id = rt_sysc_r32(MT7621_CHIP_REV_ID);
@@ -344,6 +364,12 @@ static int mt7621_pcie_parse_dt(struct mt7621_pcie *pcie)
 	struct resource regs;
 	int err;
 
+    pcie->perst = devm_gpiod_get(dev, "perst", GPIOD_OUT_HIGH);
+    if (IS_ERR(pcie->perst)) {
+        dev_err(dev, "failed to get gpio perst\n");
+        return PTR_ERR(pcie->perst);
+    }
+
 	err = of_address_to_resource(node, 0, &regs);
 	if (err) {
 		dev_err(dev, "missing \"reg\" property\n");
@@ -384,7 +410,6 @@ static int mt7621_pcie_init_port(struct mt7621_pcie_port *port)
 	struct mt7621_pcie *pcie = port->pcie;
 	struct device *dev = pcie->dev;
 	u32 slot = port->slot;
-	u32 val = 0;
 	int err;
 
 	/*
@@ -393,47 +418,33 @@ static int mt7621_pcie_init_port(struct mt7621_pcie_port *port)
 	 */
 	mt7621_reset_port(port);
 
-	val = read_config(pcie, slot, PCIE_FTS_NUM);
-	dev_info(dev, "Port %d N_FTS = %x\n", (unsigned int)val, slot);
-
 	err = phy_init(port->phy);
 	if (err) {
 		dev_err(dev, "failed to initialize port%d phy\n", slot);
-		goto err_phy_init;
+		return err;
 	}
 
 	err = phy_power_on(port->phy);
 	if (err) {
 		dev_err(dev, "failed to power on port%d phy\n", slot);
-		goto err_phy_on;
-	}
-
-	if ((pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) == 0) {
-		dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", slot);
-		mt7621_control_assert(port);
-		port->enabled = false;
-		err = -ENODEV;
-		goto err_no_link_up;
+		return err;
 	}
 
 	port->enabled = true;
 
 	return 0;
-
-err_no_link_up:
-	phy_power_off(port->phy);
-err_phy_on:
-	phy_exit(port->phy);
-err_phy_init:
-	return err;
 }
 
 static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct mt7621_pcie_port *port, *tmp;
+	u32 val = 0;
 	int err;
 
+    rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
+    mt7621_perst_gpio_pcie_assert(pcie);
+
 	list_for_each_entry_safe(port, tmp, &pcie->ports, list) {
 		u32 slot = port->slot;
 
@@ -441,7 +452,10 @@ static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 		if (err) {
 			dev_err(dev, "Initiating port %d failed\n", slot);
 			list_del(&port->list);
-		}
+		} else {
+	        val = read_config(pcie, slot, PCIE_FTS_NUM);
+	        dev_info(dev, "Port %d N_FTS = %x\n", (unsigned int)val, slot);
+        }
 	}
 
 	reset_control_assert(pcie->rst);
@@ -451,37 +465,32 @@ static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 	rt_sysc_m32(PCIE_CLK_GEN_DIS, PCIE_CLK_GEN_EN, RALINK_PCIE_CLK_GEN);
 	msleep(50);
 	reset_control_deassert(pcie->rst);
+
+    mt7621_perst_gpio_pcie_deassert(pcie);
+
+	list_for_each_entry(port, &pcie->ports, list) {
+		u32 slot = port->slot;
+
+	    if ((pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) == 0) {
+		    dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", slot);
+		    mt7621_control_assert(port);
+            phy_power_off(port->phy);
+		    port->enabled = false;
+        } else {
+	        /* enable pcie interrupt */
+	        val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR);
+	        val |= PCIE_PORT_INT_EN(slot);
+	        pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR);
+        }
+	}
 }
 
-static int mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
+static void mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
 {
 	struct mt7621_pcie *pcie = port->pcie;
 	u32 slot = port->slot;
 	u32 offset = MT7621_PCIE_OFFSET + (slot * MT7621_NEXT_PORT);
 	u32 val;
-	int err;
-
-	/* assert port PERST_N */
-	val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
-	val |= PCIE_PORT_PERST(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
-
-	/* de-assert port PERST_N */
-	val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
-	val &= ~PCIE_PORT_PERST(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
-
-	/* 100ms timeout value should be enough for Gen1 training */
-	err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
-				 val, !!(val & PCIE_PORT_LINKUP),
-				 20, 100 * USEC_PER_MSEC);
-	if (err)
-		return -ETIMEDOUT;
-
-	/* enable pcie interrupt */
-	val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR);
-	val |= PCIE_PORT_INT_EN(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR);
 
 	/* map 2G DDR region */
 	pcie_write(pcie, PCIE_BAR_MAP_MAX | PCIE_BAR_ENABLE,
@@ -492,8 +501,6 @@ static int mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
 	/* configure class code and revision ID */
 	pcie_write(pcie, PCIE_CLASS_CODE | PCIE_REVISION_ID,
 		   offset + RALINK_PCI_CLASS);
-
-	return 0;
 }
 
 static void mt7621_pcie_enable_ports(struct mt7621_pcie *pcie)
@@ -506,11 +513,7 @@ static void mt7621_pcie_enable_ports(struct mt7621_pcie *pcie)
 
 	list_for_each_entry(port, &pcie->ports, list) {
 		if (port->enabled) {
-			if (mt7621_pcie_enable_port(port)) {
-				dev_err(dev, "de-assert port %d PERST_N\n",
-					port->slot);
-				continue;
-			}
+			mt7621_pcie_enable_port(port);
 			dev_info(dev, "PCIE%d enabled\n", slot);
 			num_slots_enabled++;
 		}
@@ -665,6 +668,9 @@ static int mt7621_pci_probe(struct platform_device *pdev)
 		return 0;
 	}
 
+    pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
+    pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
+
 	mt7621_pcie_enable_ports(pcie);
 
 	err = mt7621_pci_parse_request_of_pci_ranges(pcie);
-- 
2.7.4


[-- Attachment #3: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 4/6/19 5:59 am, Sergio Paracuellos wrote:
> On Mon, Jun 3, 2019 at 2:32 PM Greg Ungerer <gerg@kernel.org> wrote:
>> On 3/6/19 3:34 pm, Sergio Paracuellos wrote:
>>> On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
>>>>> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>> On 30/5/19 10:44 am, Greg Ungerer wrote:
>>>>>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
>>>>>>> [snip]
>>> Can you try to read and set BIT(10) instead of write it directly?:
>>>
>>> Instead of:
>>>
>>> rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);
>>
>> Oh, yeah, that is definitely not going to work. There is a bunch of
>> other GPIO control bits in there for other hardware blocks. Would
>> explain the broken NAND flash behavior...
> 
> Yes, my bad here. Sometimes is better to go to sleep :)).
>>
>>> Do:
>>>
>>> u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
>>> val |= PERST_MODE_GPIO;
>>> rt_sysc_w32(val, MT7621_GPIO_MODE);
>>
>> Much better result with that. Though ultimately it should clear
>> bits 10 and 11 (both are PERST_MODE bits) and then OR in BIT(10).
> 
> Ok, so the following should do the trick:
> 
> rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
> 
> with PERST_MODE_MASK defined as:
> 
> #define PERST_MODE_MASK         GENMASK(11, 10)
> 
> (patch attached with this changes)

I have mostly good news and a little bad news :-)

I should have tested more thoroughly last night. Anyway, the new patch
works, even the IRQ of the PCI device. My Wifi PCI device works just
as well now as it did with the 4.20 pci-mt7621.c/pci-mt7621-phy.c.
I still get the lines:

pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
pcieport 0000:00:00.0: enabling device (0004 -> 0006)

However that is referring to the root host PCI device (0000:00:00.0),
not my PCI peripheral device (which is 0000:01:00.0). It is actually
probed and working properly.

That is the good news.


> It would be also good to know what happen if you comment the following lines:
> 
> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
> 
> Are they really needed?

Had no effect for me. Everything worked the same with or without
those lines as far as I could tell.


[snip]
> Only one step more to get this properly working.

Ok, now the bad news.

I often get the boot hanging at various points in the PCI
initialization, setup and probing.

For example sometimes it hangs with boot up to:

   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz


Then sometimes it hangs at:

   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
   mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
   mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)


And then I also see it hang up to:

   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
   mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
   mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: PCIE0 enabled
   mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
   mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
   pci_bus 0000:00: root bus resource [io  0xffffffff]
   pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
   pci_bus 0000:00: root bus resource [bus 00-ff]
   pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring


It happens on cold or warm boots. Sometimes it boots up all the
way and everything works perfectly.

I had perfect reliable boot and operation with linux-5.1 using the
older 4.20 pci-mt7621.c and pci-mt7621-phy.c.

Regards
Greg

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

Hi Greg,

On Tue, Jun 4, 2019 at 3:31 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> On 4/6/19 5:59 am, Sergio Paracuellos wrote:
> > On Mon, Jun 3, 2019 at 2:32 PM Greg Ungerer <gerg@kernel.org> wrote:
> >> On 3/6/19 3:34 pm, Sergio Paracuellos wrote:
> >>> On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
> >>>>> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>> On 30/5/19 10:44 am, Greg Ungerer wrote:
> >>>>>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
> >>>>>>> [snip]
> >>> Can you try to read and set BIT(10) instead of write it directly?:
> >>>
> >>> Instead of:
> >>>
> >>> rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);
> >>
> >> Oh, yeah, that is definitely not going to work. There is a bunch of
> >> other GPIO control bits in there for other hardware blocks. Would
> >> explain the broken NAND flash behavior...
> >
> > Yes, my bad here. Sometimes is better to go to sleep :)).
> >>
> >>> Do:
> >>>
> >>> u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
> >>> val |= PERST_MODE_GPIO;
> >>> rt_sysc_w32(val, MT7621_GPIO_MODE);
> >>
> >> Much better result with that. Though ultimately it should clear
> >> bits 10 and 11 (both are PERST_MODE bits) and then OR in BIT(10).
> >
> > Ok, so the following should do the trick:
> >
> > rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
> >
> > with PERST_MODE_MASK defined as:
> >
> > #define PERST_MODE_MASK         GENMASK(11, 10)
> >
> > (patch attached with this changes)
>
> I have mostly good news and a little bad news :-)
>
> I should have tested more thoroughly last night. Anyway, the new patch
> works, even the IRQ of the PCI device. My Wifi PCI device works just
> as well now as it did with the 4.20 pci-mt7621.c/pci-mt7621-phy.c.
> I still get the lines:
>
> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
>
> However that is referring to the root host PCI device (0000:00:00.0),
> not my PCI peripheral device (which is 0000:01:00.0). It is actually
> probed and working properly.
>
> That is the good news.

That makes sense :). Good news are always good news even bads are
coming also :))

>
>
> > It would be also good to know what happen if you comment the following lines:
> >
> > pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
> > pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
> >
> > Are they really needed?
>
> Had no effect for me. Everything worked the same with or without
> those lines as far as I could tell.

Ok, I won't include them when I send a clean patch series with all of
this changes.

>
>
> [snip]
> > Only one step more to get this properly working.
>
> Ok, now the bad news.
>
> I often get the boot hanging at various points in the PCI
> initialization, setup and probing.
>
> For example sometimes it hangs with boot up to:
>
>    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>
>
> Then sometimes it hangs at:
>
>    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>    mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>    mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>    mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>    mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>

It is weird, here it is not initializing any port... All of them are disabled.

>
> And then I also see it hang up to:
>
>    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>    mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>    mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>    mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>    mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>    mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>    mt7621-pci 1e140000.pcie: PCIE0 enabled
>    mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>    mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>    pci_bus 0000:00: root bus resource [io  0xffffffff]
>    pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>    pci_bus 0000:00: root bus resource [bus 00-ff]
>    pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>
>
> It happens on cold or warm boots. Sometimes it boots up all the
> way and everything works perfectly.

Can you try to change all the msleeps of the code driver in favour of
mdelay's? Looks like
some timing problem.

If it doesn't work, it would be great to have a full trace of working
4.20 and no working current 5.x series
version just to carefully compare them.

>
> I had perfect reliable boot and operation with linux-5.1 using the
> older 4.20 pci-mt7621.c and pci-mt7621-phy.c.

AFAICT, there is not pci-mt7621-phy.c in the v4.20, isn't it? Do you
mean you put also that
code into that version and compile them? Or are you just using "pci-mt7621.c"?

>
> Regards
> Greg
>
Thanks for your effort in this.

Best regards,
    Sergio Paracuellos
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 4/6/19 3:06 pm, Sergio Paracuellos wrote:
> On Tue, Jun 4, 2019 at 3:31 AM Greg Ungerer <gerg@kernel.org> wrote:
>> On 4/6/19 5:59 am, Sergio Paracuellos wrote:
>>> On Mon, Jun 3, 2019 at 2:32 PM Greg Ungerer <gerg@kernel.org> wrote:
>>>> On 3/6/19 3:34 pm, Sergio Paracuellos wrote:
>>>>> On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
>>>>>>> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>>> On 30/5/19 10:44 am, Greg Ungerer wrote:
>>>>>>>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
>>>>>>>>> [snip]
>>>>> Can you try to read and set BIT(10) instead of write it directly?:
>>>>>
>>>>> Instead of:
>>>>>
>>>>> rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);
>>>>
>>>> Oh, yeah, that is definitely not going to work. There is a bunch of
>>>> other GPIO control bits in there for other hardware blocks. Would
>>>> explain the broken NAND flash behavior...
>>>
>>> Yes, my bad here. Sometimes is better to go to sleep :)).
>>>>
>>>>> Do:
>>>>>
>>>>> u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
>>>>> val |= PERST_MODE_GPIO;
>>>>> rt_sysc_w32(val, MT7621_GPIO_MODE);
>>>>
>>>> Much better result with that. Though ultimately it should clear
>>>> bits 10 and 11 (both are PERST_MODE bits) and then OR in BIT(10).
>>>
>>> Ok, so the following should do the trick:
>>>
>>> rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
>>>
>>> with PERST_MODE_MASK defined as:
>>>
>>> #define PERST_MODE_MASK         GENMASK(11, 10)
>>>
>>> (patch attached with this changes)
>>
>> I have mostly good news and a little bad news :-)
>>
>> I should have tested more thoroughly last night. Anyway, the new patch
>> works, even the IRQ of the PCI device. My Wifi PCI device works just
>> as well now as it did with the 4.20 pci-mt7621.c/pci-mt7621-phy.c.
>> I still get the lines:
>>
>> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
>> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
>>
>> However that is referring to the root host PCI device (0000:00:00.0),
>> not my PCI peripheral device (which is 0000:01:00.0). It is actually
>> probed and working properly.
>>
>> That is the good news.
> 
> That makes sense :). Good news are always good news even bads are
> coming also :))
> 
>>
>>
>>> It would be also good to know what happen if you comment the following lines:
>>>
>>> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
>>> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
>>>
>>> Are they really needed?
>>
>> Had no effect for me. Everything worked the same with or without
>> those lines as far as I could tell.
> 
> Ok, I won't include them when I send a clean patch series with all of
> this changes.
> 
>>
>>
>> [snip]
>>> Only one step more to get this properly working.
>>
>> Ok, now the bad news.
>>
>> I often get the boot hanging at various points in the PCI
>> initialization, setup and probing.
>>
>> For example sometimes it hangs with boot up to:
>>
>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>
>>
>> Then sometimes it hangs at:
>>
>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>     mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>>     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>
> 
> It is weird, here it is not initializing any port... All of them are disabled.
> 
>>
>> And then I also see it hang up to:
>>
>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>     mt7621-pci 1e140000.pcie: PCIE0 enabled
>>     mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>     mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>     pci_bus 0000:00: root bus resource [io  0xffffffff]
>>     pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>     pci_bus 0000:00: root bus resource [bus 00-ff]
>>     pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>>
>>
>> It happens on cold or warm boots. Sometimes it boots up all the
>> way and everything works perfectly.
> 
> Can you try to change all the msleeps of the code driver in favour of
> mdelay's? Looks like
> some timing problem.

Changed the msleeps to mdelays and it still hangs somtimes.
I doubled the delay times in the msleeps and tried that too -
that also sometimes hangs.


> If it doesn't work, it would be great to have a full trace of working
> 4.20 and no working current 5.x series
> version just to carefully compare them.

I'll attach at the end a full working boot trace from 5.1 with the 4.20 pci-mt7621.c.
I'll attach a 5.1 boot trace with your patch and hang below. Not that this very
same image doesn't always hang. Same binary will sometimes boot fine.


>> I had perfect reliable boot and operation with linux-5.1 using the
>> older 4.20 pci-mt7621.c and pci-mt7621-phy.c.
> 
> AFAICT, there is not pci-mt7621-phy.c in the v4.20, isn't it? Do you
> mean you put also that
> code into that version and compile them? Or are you just using "pci-mt7621.c"?

I just leave that pci-mt7621-phy.c code in place in linux-5.1 when I copy in
the linux-4.20 pci-mt7621.c - even though it isn't used in that case.

One thing I always notice is that the PCI probing happens much earlier
with the 4.20 pci-mt7621.c. Not sure that will have any impact though.

Regards
Greg


---------------linux-5.1-with-4.20-pci-mt7621.c-----------------------------------
Linux version 5.1.0-ac0 (gerg@goober) (gcc version 5.4.0 (GCC)) #73 SMP Tue Jun 4 17:06:36 AEST 2019
SoC Type: MediaTek MT7621 ver:1 eco:3
printk: bootconsole [early0] enabled
CPU0 revision is: 0001992f (MIPS 1004Kc)
OF: fdt: No chosen node found, continuing without
MIPS: machine is Digi EX15
Determined physical RAM map:
  memory: 10000000 @ 00000000 (usable)
Initial ramdisk at: 0x81000000 (16920576 bytes)
VPE topology {2,2} total 4
Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
Zone ranges:
   Normal   [mem 0x0000000000000000-0x000000000fffffff]
Movable zone start for each node
Early memory node ranges
   node   0: [mem 0x0000000000000000-0x000000000fffffff]
Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
random: get_random_bytes called from start_kernel+0xb0/0x524 with crng_init=0
percpu: Embedded 15 pages/cpu s30144 r8192 d23104 u61440
Built 1 zonelists, mobility grouping on.  Total pages: 65024
Kernel command line:  boot=network boot_ver=19.3.223.0-0c02c07312-dirty console=ttyS0,115200 ubi.mtd=3 root=/dev/ram0 rd_start=0x81000000 rd_size=0x1023000
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Writing ErrCtl register=0002051f
Readback ErrCtl register=0002051f
Memory: 233756K/262144K available (7220K kernel code, 251K rwdata, 1320K rodata, 284K init, 230K bss, 28388K reserved, 0K cma-reserved)
SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
rcu: Hierarchical RCU implementation.
rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
NR_IRQS: 256
clocksource: GIC: mask: 0xffffffffffffffff max_cycles: 0xcaf478abb4, max_idle_ns: 440795247997 ns
sched_clock: 32 bits at 100 Hz, resolution 10000000ns, wraps every 21474836475000000ns
Calibrating delay loop... 586.13 BogoMIPS (lpj=2930688)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
rcu: Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
CPU1 revision is: 0001992f (MIPS 1004Kc)
Synchronize counters for CPU 1: done.
Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
CPU2 revision is: 0001992f (MIPS 1004Kc)
Synchronize counters for CPU 2: done.
Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
CPU3 revision is: 0001992f (MIPS 1004Kc)
Synchronize counters for CPU 3: done.
smp: Brought up 1 node, 4 CPUs
devtmpfs: initialized
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 1024 (order: 3, 32768 bytes)
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
***** Xtal 40MHz *****
Port 0 N_FTS = 1b102800
Port 1 N_FTS = 1b102800
Port 2 N_FTS = 1b102800
PCIE1 no card, disable it(RST&CLK)
PCIE2 no card, disable it(RST&CLK)
PCIE0 enabled
PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0xffffffff]
pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
pci_bus 0000:00: root bus resource [bus 00-ff]
pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
pci 0000:00:00.0: PCI bridge to [bus 01-ff]
pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
pci 0000:00:00.0: PCI bridge to [bus 01]
pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
pcf857x 0-0026: probed
i2c-mt7621 1e000900.i2c: clock 400KHz, re-start not support
clocksource: Switched to clocksource GIC
NET: Registered protocol family 2
tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes)
TCP established hash table entries: 2048 (order: 1, 8192 bytes)
TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
UDP hash table entries: 256 (order: 1, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
NET: Registered protocol family 1
Trying to unpack rootfs image as initramfs...
rootfs image is not initramfs (invalid magic at start of compressed archive); looks like an initrd
Freeing initrd memory: 16524K
Initialise system trusted keyrings
workingset: timestamp_bits=30 max_order=16 bucket_order=0
squashfs: version 4.0 (2009/01/31) Phillip Lougher
random: fast init done
Key type asymmetric registered
Asymmetric key parser 'x509' registered
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
mt7621_gpio 1e000600.gpio: registering 32 gpios
mt7621_gpio 1e000600.gpio: registering 32 gpios
mt7621_gpio 1e000600.gpio: registering 32 gpios
pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
pcieport 0000:00:00.0: enabling device (0004 -> 0006)
Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
printk: console [ttyS0] disabled
1e000c00.uartlite: ttyS0 at MMIO 0x1e000c00 (irq = 18, base_baud = 3125000) is a 16550A
printk: console [ttyS0] enabled
printk: console [ttyS0] enabled
printk: bootconsole [early0] disabled
printk: bootconsole [early0] disabled
1e000d00.uartlite: ttyS1 at MMIO 0x1e000d00 (irq = 19, base_baud = 3125000) is a 16550A
ledman: Copyright (C) SnapGear, 2000-2010.
ledman: registered ERASE switch on IRQ28
snapdog: HW/SW watchdog timer for SnapGear/Others
cacheinfo: Failed to find cpu0 device node
cacheinfo: Unable to detect cache hierarchy for CPU 0
brd: module loaded
mt7621-nand: NAND register bank at 0xbe003000
mt7621-nand: NAND ECC register bank at 0xbe003800
nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xdc
nand: Micron NAND 512MiB 3,3V 8-bit
nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
nand: ecc bit: 4, spare_per_sector: 16
Bad block table found at page 262080, version 0x01
Bad block table found at page 262016, version 0x01
5 fixed-partitions partitions found on MTD device MT7621-NAND
Creating 5 MTD partitions on "MT7621-NAND":
0x000000000000-0x000000200000 : "u-boot"
0x000000200000-0x000000300000 : "u-boot-env"
0x000000300000-0x000000500000 : "log"
0x000000500000-0x000020000000 : "flash"
0x000000000000-0x000020000000 : "all"
libphy: Fixed MDIO Bus: probed
tun: Universal TUN/TAP device driver, 1.6
libphy: mdio: probed
mtk_soc_eth 1e100000.ethernet: generated random MAC address ca:3b:7c:59:df:b1
mtk_soc_eth 1e100000.ethernet: connected mac 0 to PHY at fixed-0:00 [uid=00000000, driver=Generic PHY]
mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
PPP generic driver version 2.4.2
PPP BSD Compression module registered
PPP Deflate Compression module registered
PPP MPPE Compression module registered
NET: Registered protocol family 24
SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
CSLIP: code copyright 1989 Regents of the University of California.
usbcore: registered new interface driver ar5523
usbcore: registered new interface driver asix
usbcore: registered new interface driver ax88179_178a
usbcore: registered new interface driver cdc_ether
usbcore: registered new interface driver cdc_eem
usbcore: registered new interface driver rndis_host
usbcore: registered new interface driver cdc_subset
usbcore: registered new interface driver cdc_ncm
usbcore: registered new interface driver qmi_wwan
usbcore: registered new interface driver cdc_mbim
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
ehci-platform: EHCI generic platform driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci-pci: OHCI PCI platform driver
xhci-mtk 1e1c0000.xhci: xHCI Host Controller
xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 1
xhci-mtk 1e1c0000.xhci: hcc params 0x01401198 hci version 0x96 quirks 0x0000000000210010
xhci-mtk 1e1c0000.xhci: irq 20, io mem 0x1e1c0000
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.01
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: xHCI Host Controller
usb usb1: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
usb usb1: SerialNumber: 1e1c0000.xhci
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
xhci-mtk 1e1c0000.xhci: xHCI Host Controller
xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 2
xhci-mtk 1e1c0000.xhci: Host supports USB 3.0  SuperSpeed
usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.01
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: xHCI Host Controller
usb usb2: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
usb usb2: SerialNumber: 1e1c0000.xhci
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 1 port detected
usbcore: registered new interface driver cdc_acm
cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
usbcore: registered new interface driver cdc_wdm
usbcore: registered new interface driver usb-storage
usbcore: registered new interface driver usbserial_generic
usbserial: USB Serial support registered for generic
usbcore: registered new interface driver ipw
usbserial: USB Serial support registered for IPWireless converter
usbcore: registered new interface driver option
usbserial: USB Serial support registered for GSM modem (1-port)
usbcore: registered new interface driver qcaux
usbserial: USB Serial support registered for qcaux
usbcore: registered new interface driver qcserial
usbserial: USB Serial support registered for Qualcomm USB modem
usbcore: registered new interface driver quatech2
usbserial: USB Serial support registered for Quatech 2nd gen USB to Serial Driver
usbcore: registered new interface driver usb_serial_simple
usbserial: USB Serial support registered for carelink
usbserial: USB Serial support registered for zio
usbserial: USB Serial support registered for funsoft
usbserial: USB Serial support registered for flashloader
usbserial: USB Serial support registered for google
usbserial: USB Serial support registered for libtransistor
usbserial: USB Serial support registered for vivopay
usbserial: USB Serial support registered for moto_modem
usbserial: USB Serial support registered for motorola_tetra
usbserial: USB Serial support registered for novatel_gps
usbserial: USB Serial support registered for hp4x
usbserial: USB Serial support registered for suunto
usbserial: USB Serial support registered for siemens_mpi
u32 classifier
     input device check on
ipip: IPv4 and MPLS over IPv4 tunneling driver
gre: GRE over IPv4 demultiplexor driver
ip_gre: GRE over IPv4 tunneling driver
Initializing XFRM netlink socket
NET: Registered protocol family 10
Segment Routing with IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
NET: Registered protocol family 17
NET: Registered protocol family 15
8021q: 802.1Q VLAN Support v1.8
Loading compiled-in X.509 certificates
libphy: dsa slave smi: probed
mt7530 mdio-bus:00 lan (uninitialized): PHY [dsa-0.0:00] driver [Generic PHY]
mt7530 mdio-bus:00 wan (uninitialized): PHY [dsa-0.0:01] driver [Generic PHY]
DSA: tree 0 setup
ubi0: attaching mtd3
ubi0: scanning is finished
ubi0: attached mtd3 (name "flash", size 507 MiB)
ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
ubi0: good PEBs: 4052, bad PEBs: 4, corrupted PEBs: 0
ubi0: user volume: 4, internal volumes: 1, max. volumes count: 128
ubi0: max/mean erase counter: 7/4, WL threshold: 4096, image sequence number: 1236486005
ubi0: available PEBs: 0, total reserved PEBs: 4052, PEBs reserved for bad PEB handling: 76
ubi0: background thread "ubi_bgt0d" started, PID 115
hctosys: unable to open rtc device (rtc0)
cfg80211: Loading compiled-in X.509 certificates for regulatory database
cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
cfg80211: failed to load regulatory.db
RAMDISK: squashfs filesystem found at block 0
RAMDISK: Loading 16521KiB [1 disk] into ram disk... /
/
done.
VFS: Mounted root (squashfs filesystem) readonly on device 1:0.
devtmpfs: mounted
Freeing unused kernel memory: 284K
This architecture does not have kernel memory protection.
Run /sbin/init as init process
Run /etc/init as init process
Run /bin/init as init process
Mounting filesystems...
Starting watchdog...
snapdog: user servicing enabled (short=60,long=300).
Initializing network interfaces...
Set eth0 to MAC address 00:27:04:03:02:01
Set lan to MAC address 00:27:04:39:05:a7
Set wan to MAC address 00:27:04:39:05:a8
Initializing CELL interface control...
Initializing WiFi interface control...
Mounting config filesystem...
UBIFS (ubi0:2): Mounting in unauthenticated mode
UBIFS (ubi0:2): background thread "ubifs_bgt0_2" started, PID 197
UBIFS (ubi0:2): recovery needed
UBIFS (ubi0:2): recovery completed
UBIFS (ubi0:2): UBIFS: mounted UBI device 0, volume 2, name "config"
UBIFS (ubi0:2): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
UBIFS (ubi0:2): FS size: 15618048 bytes (14 MiB, 123 LEBs), journal size 1015809 bytes (0 MiB, 6 LEBs)
UBIFS (ubi0:2): reserved for root: 737678 bytes (720 KiB)
UBIFS (ubi0:2): media format: w5/r0 (latest is w5/r0), UUID AFD2AF01-271A-41C0-8D8E-E615C64EA6ED, small LPT model
Mounting opt filesystem....
UBIFS (ubi0:3): Mounting in unauthenticated mode
UBIFS (ubi0:3): background thread "ubifs_bgt0_3" started, PID 203
UBIFS (ubi0:3): recovery needed
UBIFS (ubi0:3): recovery completed
UBIFS (ubi0:3): UBIFS: mounted UBI device 0, volume 3, name "opt"
UBIFS (ubi0:3): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
UBIFS (ubi0:3): FS size: 418258944 bytes (398 MiB, 3294 LEBs), journal size 20951040 bytes (19 MiB, 165 LEBs)
UBIFS (ubi0:3): reserved for root: 4952683 bytes (4836 KiB)
UBIFS (ubi0:3): media format: w5/r0 (latest is w5/r0), UUID 975D738A-1015-45AE-BD40-D7FD8D10756F, small LPT model

Digi International, Inc.
https://www.digi.com


Digi EX15W
EX15W login:
----------------------------------------------------------------------------------
---------------linux-5.1-with-sergios-latest-patch--------------------------------

Linux version 5.1.0-ac0 (gerg@goober) (gcc version 5.4.0 (GCC)) #74 SMP Tue Jun 4 17:09:50 AEST 2019
SoC Type: MediaTek MT7621 ver:1 eco:3
printk: bootconsole [early0] enabled
CPU0 revision is: 0001992f (MIPS 1004Kc)
OF: fdt: No chosen node found, continuing without
MIPS: machine is Digi EX15
Determined physical RAM map:
  memory: 10000000 @ 00000000 (usable)
Initial ramdisk at: 0x81000000 (16920576 bytes)
VPE topology {2,2} total 4
Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
Zone ranges:
   Normal   [mem 0x0000000000000000-0x000000000fffffff]
Movable zone start for each node
Early memory node ranges
   node   0: [mem 0x0000000000000000-0x000000000fffffff]
Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
random: get_random_bytes called from start_kernel+0xb0/0x524 with crng_init=0
percpu: Embedded 15 pages/cpu s30144 r8192 d23104 u61440
Built 1 zonelists, mobility grouping on.  Total pages: 65024
Kernel command line:  boot=network boot_ver=19.3.223.0-0c02c07312-dirty console=ttyS0,115200 ubi.mtd=3 root=/dev/ram0 rd_start=0x81000000 rd_size=0x1023000
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Writing ErrCtl register=00020510
Readback ErrCtl register=00020510
Memory: 233756K/262144K available (7216K kernel code, 247K rwdata, 1320K rodata, 292K init, 230K bss, 28388K reserved, 0K cma-reserved)
SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
rcu: Hierarchical RCU implementation.
rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
NR_IRQS: 256
clocksource: GIC: mask: 0xffffffffffffffff max_cycles: 0xcaf478abb4, max_idle_ns: 440795247997 ns
sched_clock: 32 bits at 100 Hz, resolution 10000000ns, wraps every 21474836475000000ns
Calibrating delay loop... 586.13 BogoMIPS (lpj=2930688)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
rcu: Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
CPU1 revision is: 0001992f (MIPS 1004Kc)
Synchronize counters for CPU 1: done.
Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
CPU2 revision is: 0001992f (MIPS 1004Kc)
Synchronize counters for CPU 2: done.
Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
CPU3 revision is: 0001992f (MIPS 1004Kc)
Synchronize counters for CPU 3: done.
smp: Brought up 1 node, 4 CPUs
devtmpfs: initialized
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 1024 (order: 3, 32768 bytes)
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
mt7621-pci 1e140000.pcie: failed to get gpio perst
mt7621-pci 1e140000.pcie: Parsing DT failed
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
pcf857x 0-0026: probed
i2c-mt7621 1e000900.i2c: clock 400KHz, re-start not support
clocksource: Switched to clocksource GIC
NET: Registered protocol family 2
tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes)
TCP established hash table entries: 2048 (order: 1, 8192 bytes)
TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
UDP hash table entries: 256 (order: 1, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
NET: Registered protocol family 1
Trying to unpack rootfs image as initramfs...
rootfs image is not initramfs (invalid magic at start of compressed archive); looks like an initrd
Freeing initrd memory: 16524K
Initialise system trusted keyrings
workingset: timestamp_bits=30 max_order=16 bucket_order=0
squashfs: version 4.0 (2009/01/31) Phillip Lougher
random: fast init done
Key type asymmetric registered
Asymmetric key parser 'x509' registered
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
mt7621_gpio 1e000600.gpio: registering 32 gpios
mt7621_gpio 1e000600.gpio: registering 32 gpios
mt7621_gpio 1e000600.gpio: registering 32 gpios
Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
printk: console [ttyS0] disabled
1e000c00.uartlite: ttyS0 at MMIO 0x1e000c00 (irq = 18, base_baud = 3125000) is a 16550A
printk: console [ttyS0] enabled
printk: console [ttyS0] enabled
printk: bootconsole [early0] disabled
printk: bootconsole [early0] disabled
1e000d00.uartlite: ttyS1 at MMIO 0x1e000d00 (irq = 19, base_baud = 3125000) is a 16550A
ledman: Copyright (C) SnapGear, 2000-2010.
ledman: registered ERASE switch on IRQ28
snapdog: HW/SW watchdog timer for SnapGear/Others
cacheinfo: Failed to find cpu0 device node
cacheinfo: Unable to detect cache hierarchy for CPU 0
brd: module loaded
mt7621-nand: NAND register bank at 0xbe003000
mt7621-nand: NAND ECC register bank at 0xbe003800
nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xdc
nand: Micron NAND 512MiB 3,3V 8-bit
nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
nand: ecc bit: 4, spare_per_sector: 16
Bad block table found at page 262080, version 0x01
Bad block table found at page 262016, version 0x01
5 fixed-partitions partitions found on MTD device MT7621-NAND
Creating 5 MTD partitions on "MT7621-NAND":
0x000000000000-0x000000200000 : "u-boot"
0x000000200000-0x000000300000 : "u-boot-env"
0x000000300000-0x000000500000 : "log"
0x000000500000-0x000020000000 : "flash"
0x000000000000-0x000020000000 : "all"
libphy: Fixed MDIO Bus: probed
tun: Universal TUN/TAP device driver, 1.6
libphy: mdio: probed
mtk_soc_eth 1e100000.ethernet: generated random MAC address a6:2b:8b:f3:dd:6a
mtk_soc_eth 1e100000.ethernet: connected mac 0 to PHY at fixed-0:00 [uid=00000000, driver=Generic PHY]
mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
PPP generic driver version 2.4.2
PPP BSD Compression module registered
PPP Deflate Compression module registered
PPP MPPE Compression module registered
NET: Registered protocol family 24
SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
CSLIP: code copyright 1989 Regents of the University of California.
usbcore: registered new interface driver ar5523
usbcore: registered new interface driver asix
usbcore: registered new interface driver ax88179_178a
usbcore: registered new interface driver cdc_ether
usbcore: registered new interface driver cdc_eem
usbcore: registered new interface driver rndis_host
usbcore: registered new interface driver cdc_subset
usbcore: registered new interface driver cdc_ncm
usbcore: registered new interface driver qmi_wwan
usbcore: registered new interface driver cdc_mbim
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
ehci-platform: EHCI generic platform driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci-pci: OHCI PCI platform driver
xhci-mtk 1e1c0000.xhci: xHCI Host Controller
xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 1
xhci-mtk 1e1c0000.xhci: hcc params 0x01401198 hci version 0x96 quirks 0x0000000000210010
xhci-mtk 1e1c0000.xhci: irq 20, io mem 0x1e1c0000
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.01
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: xHCI Host Controller
usb usb1: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
usb usb1: SerialNumber: 1e1c0000.xhci
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
xhci-mtk 1e1c0000.xhci: xHCI Host Controller
xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 2
xhci-mtk 1e1c0000.xhci: Host supports USB 3.0  SuperSpeed
usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.01
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: xHCI Host Controller
usb usb2: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
usb usb2: SerialNumber: 1e1c0000.xhci
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 1 port detected
usbcore: registered new interface driver cdc_acm
cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
usbcore: registered new interface driver cdc_wdm
usbcore: registered new interface driver usb-storage
usbcore: registered new interface driver usbserial_generic
usbserial: USB Serial support registered for generic
usbcore: registered new interface driver ipw
usbserial: USB Serial support registered for IPWireless converter
usbcore: registered new interface driver option
usbserial: USB Serial support registered for GSM modem (1-port)
usbcore: registered new interface driver qcaux
usbserial: USB Serial support registered for qcaux
usbcore: registered new interface driver qcserial
usbserial: USB Serial support registered for Qualcomm USB modem
usbcore: registered new interface driver quatech2
usbserial: USB Serial support registered for Quatech 2nd gen USB to Serial Driver
usbcore: registered new interface driver usb_serial_simple
usbserial: USB Serial support registered for carelink
usbserial: USB Serial support registered for zio
usbserial: USB Serial support registered for funsoft
usbserial: USB Serial support registered for flashloader
usbserial: USB Serial support registered for google
usbserial: USB Serial support registered for libtransistor
usbserial: USB Serial support registered for vivopay
usbserial: USB Serial support registered for moto_modem
usbserial: USB Serial support registered for motorola_tetra
usbserial: USB Serial support registered for novatel_gps
usbserial: USB Serial support registered for hp4x
usbserial: USB Serial support registered for suunto
usbserial: USB Serial support registered for siemens_mpi
u32 classifier
     input device check on
ipip: IPv4 and MPLS over IPv4 tunneling driver
gre: GRE over IPv4 demultiplexor driver
ip_gre: GRE over IPv4 tunneling driver
Initializing XFRM netlink socket
NET: Registered protocol family 10
Segment Routing with IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
NET: Registered protocol family 17
NET: Registered protocol family 15
8021q: 802.1Q VLAN Support v1.8
Loading compiled-in X.509 certificates
mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)

                [HUNG]

---------------------------------------------------------------------------------

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

[-- Attachment #1: Type: text/plain, Size: 35257 bytes --]

Hi Greg,

Thanks for the traces.


On Tue, Jun 4, 2019 at 9:14 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> On 4/6/19 3:06 pm, Sergio Paracuellos wrote:
> > On Tue, Jun 4, 2019 at 3:31 AM Greg Ungerer <gerg@kernel.org> wrote:
> >> On 4/6/19 5:59 am, Sergio Paracuellos wrote:
> >>> On Mon, Jun 3, 2019 at 2:32 PM Greg Ungerer <gerg@kernel.org> wrote:
> >>>> On 3/6/19 3:34 pm, Sergio Paracuellos wrote:
> >>>>> On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
> >>>>>>> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>>>> On 30/5/19 10:44 am, Greg Ungerer wrote:
> >>>>>>>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
> >>>>>>>>> [snip]
> >>>>> Can you try to read and set BIT(10) instead of write it directly?:
> >>>>>
> >>>>> Instead of:
> >>>>>
> >>>>> rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);
> >>>>
> >>>> Oh, yeah, that is definitely not going to work. There is a bunch of
> >>>> other GPIO control bits in there for other hardware blocks. Would
> >>>> explain the broken NAND flash behavior...
> >>>
> >>> Yes, my bad here. Sometimes is better to go to sleep :)).
> >>>>
> >>>>> Do:
> >>>>>
> >>>>> u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
> >>>>> val |= PERST_MODE_GPIO;
> >>>>> rt_sysc_w32(val, MT7621_GPIO_MODE);
> >>>>
> >>>> Much better result with that. Though ultimately it should clear
> >>>> bits 10 and 11 (both are PERST_MODE bits) and then OR in BIT(10).
> >>>
> >>> Ok, so the following should do the trick:
> >>>
> >>> rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
> >>>
> >>> with PERST_MODE_MASK defined as:
> >>>
> >>> #define PERST_MODE_MASK         GENMASK(11, 10)
> >>>
> >>> (patch attached with this changes)
> >>
> >> I have mostly good news and a little bad news :-)
> >>
> >> I should have tested more thoroughly last night. Anyway, the new patch
> >> works, even the IRQ of the PCI device. My Wifi PCI device works just
> >> as well now as it did with the 4.20 pci-mt7621.c/pci-mt7621-phy.c.
> >> I still get the lines:
> >>
> >> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
> >> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
> >>
> >> However that is referring to the root host PCI device (0000:00:00.0),
> >> not my PCI peripheral device (which is 0000:01:00.0). It is actually
> >> probed and working properly.
> >>
> >> That is the good news.
> >
> > That makes sense :). Good news are always good news even bads are
> > coming also :))
> >
> >>
> >>
> >>> It would be also good to know what happen if you comment the following lines:
> >>>
> >>> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
> >>> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
> >>>
> >>> Are they really needed?
> >>
> >> Had no effect for me. Everything worked the same with or without
> >> those lines as far as I could tell.
> >
> > Ok, I won't include them when I send a clean patch series with all of
> > this changes.
> >
> >>
> >>
> >> [snip]
> >>> Only one step more to get this properly working.
> >>
> >> Ok, now the bad news.
> >>
> >> I often get the boot hanging at various points in the PCI
> >> initialization, setup and probing.
> >>
> >> For example sometimes it hangs with boot up to:
> >>
> >>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>
> >>
> >> Then sometimes it hangs at:
> >>
> >>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>     mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> >>     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>
> >
> > It is weird, here it is not initializing any port... All of them are disabled.
> >
> >>
> >> And then I also see it hang up to:
> >>
> >>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>     mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>     mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>     mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>     mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>     mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>     mt7621-pci 1e140000.pcie: PCIE0 enabled
> >>     mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >>     mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >>     pci_bus 0000:00: root bus resource [io  0xffffffff]
> >>     pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >>     pci_bus 0000:00: root bus resource [bus 00-ff]
> >>     pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> >>
> >>
> >> It happens on cold or warm boots. Sometimes it boots up all the
> >> way and everything works perfectly.
> >
> > Can you try to change all the msleeps of the code driver in favour of
> > mdelay's? Looks like
> > some timing problem.
>
> Changed the msleeps to mdelays and it still hangs somtimes.
> I doubled the delay times in the msleeps and tried that too -
> that also sometimes hangs.
>
>
> > If it doesn't work, it would be great to have a full trace of working
> > 4.20 and no working current 5.x series
> > version just to carefully compare them.
>
> I'll attach at the end a full working boot trace from 5.1 with the 4.20 pci-mt7621.c.
> I'll attach a 5.1 boot trace with your patch and hang below. Not that this very
> same image doesn't always hang. Same binary will sometimes boot fine.
>
>
> >> I had perfect reliable boot and operation with linux-5.1 using the
> >> older 4.20 pci-mt7621.c and pci-mt7621-phy.c.
> >
> > AFAICT, there is not pci-mt7621-phy.c in the v4.20, isn't it? Do you
> > mean you put also that
> > code into that version and compile them? Or are you just using "pci-mt7621.c"?
>
> I just leave that pci-mt7621-phy.c code in place in linux-5.1 when I copy in
> the linux-4.20 pci-mt7621.c - even though it isn't used in that case.
>
> One thing I always notice is that the PCI probing happens much earlier
> with the 4.20 pci-mt7621.c. Not sure that will have any impact though.
>

It seems it is related because, looking at the trace which hangs I can see:

mt7621-pci 1e140000.pcie: failed to get gpio perst
mt7621-pci 1e140000.pcie: Parsing DT failed

At that point gpio's driver are not being probed yet. This not happen
with the original
patch because it just access to memory to set up all the stuff intead
of use the gpio's
consumers apis which, IMHO, is the way to go.

Can you try to return "-EPROBE_DEFER" if getting the gpio fails in
initialization?

(patch attached with this change and removing the two lines which
seems to not be needed
commented in previous patch)

> Regards
> Greg

Best regards,
    Sergio Paracuellos

>
>
> ---------------linux-5.1-with-4.20-pci-mt7621.c-----------------------------------
> Linux version 5.1.0-ac0 (gerg@goober) (gcc version 5.4.0 (GCC)) #73 SMP Tue Jun 4 17:06:36 AEST 2019
> SoC Type: MediaTek MT7621 ver:1 eco:3
> printk: bootconsole [early0] enabled
> CPU0 revision is: 0001992f (MIPS 1004Kc)
> OF: fdt: No chosen node found, continuing without
> MIPS: machine is Digi EX15
> Determined physical RAM map:
>   memory: 10000000 @ 00000000 (usable)
> Initial ramdisk at: 0x81000000 (16920576 bytes)
> VPE topology {2,2} total 4
> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> Zone ranges:
>    Normal   [mem 0x0000000000000000-0x000000000fffffff]
> Movable zone start for each node
> Early memory node ranges
>    node   0: [mem 0x0000000000000000-0x000000000fffffff]
> Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
> random: get_random_bytes called from start_kernel+0xb0/0x524 with crng_init=0
> percpu: Embedded 15 pages/cpu s30144 r8192 d23104 u61440
> Built 1 zonelists, mobility grouping on.  Total pages: 65024
> Kernel command line:  boot=network boot_ver=19.3.223.0-0c02c07312-dirty console=ttyS0,115200 ubi.mtd=3 root=/dev/ram0 rd_start=0x81000000 rd_size=0x1023000
> Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
> Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
> Writing ErrCtl register=0002051f
> Readback ErrCtl register=0002051f
> Memory: 233756K/262144K available (7220K kernel code, 251K rwdata, 1320K rodata, 284K init, 230K bss, 28388K reserved, 0K cma-reserved)
> SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
> rcu: Hierarchical RCU implementation.
> rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
> NR_IRQS: 256
> clocksource: GIC: mask: 0xffffffffffffffff max_cycles: 0xcaf478abb4, max_idle_ns: 440795247997 ns
> sched_clock: 32 bits at 100 Hz, resolution 10000000ns, wraps every 21474836475000000ns
> Calibrating delay loop... 586.13 BogoMIPS (lpj=2930688)
> pid_max: default: 32768 minimum: 301
> Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
> Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
> rcu: Hierarchical SRCU implementation.
> smp: Bringing up secondary CPUs ...
> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> CPU1 revision is: 0001992f (MIPS 1004Kc)
> Synchronize counters for CPU 1: done.
> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> CPU2 revision is: 0001992f (MIPS 1004Kc)
> Synchronize counters for CPU 2: done.
> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> CPU3 revision is: 0001992f (MIPS 1004Kc)
> Synchronize counters for CPU 3: done.
> smp: Brought up 1 node, 4 CPUs
> devtmpfs: initialized
> clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> futex hash table entries: 1024 (order: 3, 32768 bytes)
> pinctrl core: initialized pinctrl subsystem
> NET: Registered protocol family 16
> ***** Xtal 40MHz *****
> Port 0 N_FTS = 1b102800
> Port 1 N_FTS = 1b102800
> Port 2 N_FTS = 1b102800
> PCIE1 no card, disable it(RST&CLK)
> PCIE2 no card, disable it(RST&CLK)
> PCIE0 enabled
> PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> pci_bus 0000:00: root bus resource [io  0xffffffff]
> pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> pci_bus 0000:00: root bus resource [bus 00-ff]
> pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
> pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
> pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
> pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
> pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
> pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
> pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
> pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]
> pci 0000:01:00.0: BAR 6: assigned [mem 0x60200000-0x6020ffff pref]
> pci 0000:00:00.0: PCI bridge to [bus 01]
> pci 0000:00:00.0:   bridge window [mem 0x60000000-0x601fffff]
> pci 0000:00:00.0:   bridge window [mem 0x60200000-0x602fffff pref]
> SCSI subsystem initialized
> usbcore: registered new interface driver usbfs
> usbcore: registered new interface driver hub
> usbcore: registered new device driver usb
> pcf857x 0-0026: probed
> i2c-mt7621 1e000900.i2c: clock 400KHz, re-start not support
> clocksource: Switched to clocksource GIC
> NET: Registered protocol family 2
> tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes)
> TCP established hash table entries: 2048 (order: 1, 8192 bytes)
> TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
> TCP: Hash tables configured (established 2048 bind 2048)
> UDP hash table entries: 256 (order: 1, 8192 bytes)
> UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
> NET: Registered protocol family 1
> Trying to unpack rootfs image as initramfs...
> rootfs image is not initramfs (invalid magic at start of compressed archive); looks like an initrd
> Freeing initrd memory: 16524K
> Initialise system trusted keyrings
> workingset: timestamp_bits=30 max_order=16 bucket_order=0
> squashfs: version 4.0 (2009/01/31) Phillip Lougher
> random: fast init done
> Key type asymmetric registered
> Asymmetric key parser 'x509' registered
> Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
> mt7621_gpio 1e000600.gpio: registering 32 gpios
> mt7621_gpio 1e000600.gpio: registering 32 gpios
> mt7621_gpio 1e000600.gpio: registering 32 gpios
> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
> Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
> printk: console [ttyS0] disabled
> 1e000c00.uartlite: ttyS0 at MMIO 0x1e000c00 (irq = 18, base_baud = 3125000) is a 16550A
> printk: console [ttyS0] enabled
> printk: console [ttyS0] enabled
> printk: bootconsole [early0] disabled
> printk: bootconsole [early0] disabled
> 1e000d00.uartlite: ttyS1 at MMIO 0x1e000d00 (irq = 19, base_baud = 3125000) is a 16550A
> ledman: Copyright (C) SnapGear, 2000-2010.
> ledman: registered ERASE switch on IRQ28
> snapdog: HW/SW watchdog timer for SnapGear/Others
> cacheinfo: Failed to find cpu0 device node
> cacheinfo: Unable to detect cache hierarchy for CPU 0
> brd: module loaded
> mt7621-nand: NAND register bank at 0xbe003000
> mt7621-nand: NAND ECC register bank at 0xbe003800
> nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xdc
> nand: Micron NAND 512MiB 3,3V 8-bit
> nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
> nand: ecc bit: 4, spare_per_sector: 16
> Bad block table found at page 262080, version 0x01
> Bad block table found at page 262016, version 0x01
> 5 fixed-partitions partitions found on MTD device MT7621-NAND
> Creating 5 MTD partitions on "MT7621-NAND":
> 0x000000000000-0x000000200000 : "u-boot"
> 0x000000200000-0x000000300000 : "u-boot-env"
> 0x000000300000-0x000000500000 : "log"
> 0x000000500000-0x000020000000 : "flash"
> 0x000000000000-0x000020000000 : "all"
> libphy: Fixed MDIO Bus: probed
> tun: Universal TUN/TAP device driver, 1.6
> libphy: mdio: probed
> mtk_soc_eth 1e100000.ethernet: generated random MAC address ca:3b:7c:59:df:b1
> mtk_soc_eth 1e100000.ethernet: connected mac 0 to PHY at fixed-0:00 [uid=00000000, driver=Generic PHY]
> mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
> PPP generic driver version 2.4.2
> PPP BSD Compression module registered
> PPP Deflate Compression module registered
> PPP MPPE Compression module registered
> NET: Registered protocol family 24
> SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
> CSLIP: code copyright 1989 Regents of the University of California.
> usbcore: registered new interface driver ar5523
> usbcore: registered new interface driver asix
> usbcore: registered new interface driver ax88179_178a
> usbcore: registered new interface driver cdc_ether
> usbcore: registered new interface driver cdc_eem
> usbcore: registered new interface driver rndis_host
> usbcore: registered new interface driver cdc_subset
> usbcore: registered new interface driver cdc_ncm
> usbcore: registered new interface driver qmi_wwan
> usbcore: registered new interface driver cdc_mbim
> ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> ehci-pci: EHCI PCI platform driver
> ehci-platform: EHCI generic platform driver
> ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
> ohci-pci: OHCI PCI platform driver
> xhci-mtk 1e1c0000.xhci: xHCI Host Controller
> xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 1
> xhci-mtk 1e1c0000.xhci: hcc params 0x01401198 hci version 0x96 quirks 0x0000000000210010
> xhci-mtk 1e1c0000.xhci: irq 20, io mem 0x1e1c0000
> usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.01
> usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> usb usb1: Product: xHCI Host Controller
> usb usb1: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
> usb usb1: SerialNumber: 1e1c0000.xhci
> hub 1-0:1.0: USB hub found
> hub 1-0:1.0: 2 ports detected
> xhci-mtk 1e1c0000.xhci: xHCI Host Controller
> xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 2
> xhci-mtk 1e1c0000.xhci: Host supports USB 3.0  SuperSpeed
> usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
> usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.01
> usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> usb usb2: Product: xHCI Host Controller
> usb usb2: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
> usb usb2: SerialNumber: 1e1c0000.xhci
> hub 2-0:1.0: USB hub found
> hub 2-0:1.0: 1 port detected
> usbcore: registered new interface driver cdc_acm
> cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
> usbcore: registered new interface driver cdc_wdm
> usbcore: registered new interface driver usb-storage
> usbcore: registered new interface driver usbserial_generic
> usbserial: USB Serial support registered for generic
> usbcore: registered new interface driver ipw
> usbserial: USB Serial support registered for IPWireless converter
> usbcore: registered new interface driver option
> usbserial: USB Serial support registered for GSM modem (1-port)
> usbcore: registered new interface driver qcaux
> usbserial: USB Serial support registered for qcaux
> usbcore: registered new interface driver qcserial
> usbserial: USB Serial support registered for Qualcomm USB modem
> usbcore: registered new interface driver quatech2
> usbserial: USB Serial support registered for Quatech 2nd gen USB to Serial Driver
> usbcore: registered new interface driver usb_serial_simple
> usbserial: USB Serial support registered for carelink
> usbserial: USB Serial support registered for zio
> usbserial: USB Serial support registered for funsoft
> usbserial: USB Serial support registered for flashloader
> usbserial: USB Serial support registered for google
> usbserial: USB Serial support registered for libtransistor
> usbserial: USB Serial support registered for vivopay
> usbserial: USB Serial support registered for moto_modem
> usbserial: USB Serial support registered for motorola_tetra
> usbserial: USB Serial support registered for novatel_gps
> usbserial: USB Serial support registered for hp4x
> usbserial: USB Serial support registered for suunto
> usbserial: USB Serial support registered for siemens_mpi
> u32 classifier
>      input device check on
> ipip: IPv4 and MPLS over IPv4 tunneling driver
> gre: GRE over IPv4 demultiplexor driver
> ip_gre: GRE over IPv4 tunneling driver
> Initializing XFRM netlink socket
> NET: Registered protocol family 10
> Segment Routing with IPv6
> sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
> NET: Registered protocol family 17
> NET: Registered protocol family 15
> 8021q: 802.1Q VLAN Support v1.8
> Loading compiled-in X.509 certificates
> libphy: dsa slave smi: probed
> mt7530 mdio-bus:00 lan (uninitialized): PHY [dsa-0.0:00] driver [Generic PHY]
> mt7530 mdio-bus:00 wan (uninitialized): PHY [dsa-0.0:01] driver [Generic PHY]
> DSA: tree 0 setup
> ubi0: attaching mtd3
> ubi0: scanning is finished
> ubi0: attached mtd3 (name "flash", size 507 MiB)
> ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
> ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
> ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
> ubi0: good PEBs: 4052, bad PEBs: 4, corrupted PEBs: 0
> ubi0: user volume: 4, internal volumes: 1, max. volumes count: 128
> ubi0: max/mean erase counter: 7/4, WL threshold: 4096, image sequence number: 1236486005
> ubi0: available PEBs: 0, total reserved PEBs: 4052, PEBs reserved for bad PEB handling: 76
> ubi0: background thread "ubi_bgt0d" started, PID 115
> hctosys: unable to open rtc device (rtc0)
> cfg80211: Loading compiled-in X.509 certificates for regulatory database
> cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
> platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
> cfg80211: failed to load regulatory.db
> RAMDISK: squashfs filesystem found at block 0
> RAMDISK: Loading 16521KiB [1 disk] into ram disk... /
> /
> done.
> VFS: Mounted root (squashfs filesystem) readonly on device 1:0.
> devtmpfs: mounted
> Freeing unused kernel memory: 284K
> This architecture does not have kernel memory protection.
> Run /sbin/init as init process
> Run /etc/init as init process
> Run /bin/init as init process
> Mounting filesystems...
> Starting watchdog...
> snapdog: user servicing enabled (short=60,long=300).
> Initializing network interfaces...
> Set eth0 to MAC address 00:27:04:03:02:01
> Set lan to MAC address 00:27:04:39:05:a7
> Set wan to MAC address 00:27:04:39:05:a8
> Initializing CELL interface control...
> Initializing WiFi interface control...
> Mounting config filesystem...
> UBIFS (ubi0:2): Mounting in unauthenticated mode
> UBIFS (ubi0:2): background thread "ubifs_bgt0_2" started, PID 197
> UBIFS (ubi0:2): recovery needed
> UBIFS (ubi0:2): recovery completed
> UBIFS (ubi0:2): UBIFS: mounted UBI device 0, volume 2, name "config"
> UBIFS (ubi0:2): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
> UBIFS (ubi0:2): FS size: 15618048 bytes (14 MiB, 123 LEBs), journal size 1015809 bytes (0 MiB, 6 LEBs)
> UBIFS (ubi0:2): reserved for root: 737678 bytes (720 KiB)
> UBIFS (ubi0:2): media format: w5/r0 (latest is w5/r0), UUID AFD2AF01-271A-41C0-8D8E-E615C64EA6ED, small LPT model
> Mounting opt filesystem....
> UBIFS (ubi0:3): Mounting in unauthenticated mode
> UBIFS (ubi0:3): background thread "ubifs_bgt0_3" started, PID 203
> UBIFS (ubi0:3): recovery needed
> UBIFS (ubi0:3): recovery completed
> UBIFS (ubi0:3): UBIFS: mounted UBI device 0, volume 3, name "opt"
> UBIFS (ubi0:3): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
> UBIFS (ubi0:3): FS size: 418258944 bytes (398 MiB, 3294 LEBs), journal size 20951040 bytes (19 MiB, 165 LEBs)
> UBIFS (ubi0:3): reserved for root: 4952683 bytes (4836 KiB)
> UBIFS (ubi0:3): media format: w5/r0 (latest is w5/r0), UUID 975D738A-1015-45AE-BD40-D7FD8D10756F, small LPT model
>
> Digi International, Inc.
> https://www.digi.com
>
>
> Digi EX15W
> EX15W login:
> ----------------------------------------------------------------------------------
> ---------------linux-5.1-with-sergios-latest-patch--------------------------------
>
> Linux version 5.1.0-ac0 (gerg@goober) (gcc version 5.4.0 (GCC)) #74 SMP Tue Jun 4 17:09:50 AEST 2019
> SoC Type: MediaTek MT7621 ver:1 eco:3
> printk: bootconsole [early0] enabled
> CPU0 revision is: 0001992f (MIPS 1004Kc)
> OF: fdt: No chosen node found, continuing without
> MIPS: machine is Digi EX15
> Determined physical RAM map:
>   memory: 10000000 @ 00000000 (usable)
> Initial ramdisk at: 0x81000000 (16920576 bytes)
> VPE topology {2,2} total 4
> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> Zone ranges:
>    Normal   [mem 0x0000000000000000-0x000000000fffffff]
> Movable zone start for each node
> Early memory node ranges
>    node   0: [mem 0x0000000000000000-0x000000000fffffff]
> Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
> random: get_random_bytes called from start_kernel+0xb0/0x524 with crng_init=0
> percpu: Embedded 15 pages/cpu s30144 r8192 d23104 u61440
> Built 1 zonelists, mobility grouping on.  Total pages: 65024
> Kernel command line:  boot=network boot_ver=19.3.223.0-0c02c07312-dirty console=ttyS0,115200 ubi.mtd=3 root=/dev/ram0 rd_start=0x81000000 rd_size=0x1023000
> Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
> Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
> Writing ErrCtl register=00020510
> Readback ErrCtl register=00020510
> Memory: 233756K/262144K available (7216K kernel code, 247K rwdata, 1320K rodata, 292K init, 230K bss, 28388K reserved, 0K cma-reserved)
> SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
> rcu: Hierarchical RCU implementation.
> rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
> NR_IRQS: 256
> clocksource: GIC: mask: 0xffffffffffffffff max_cycles: 0xcaf478abb4, max_idle_ns: 440795247997 ns
> sched_clock: 32 bits at 100 Hz, resolution 10000000ns, wraps every 21474836475000000ns
> Calibrating delay loop... 586.13 BogoMIPS (lpj=2930688)
> pid_max: default: 32768 minimum: 301
> Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
> Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
> rcu: Hierarchical SRCU implementation.
> smp: Bringing up secondary CPUs ...
> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> CPU1 revision is: 0001992f (MIPS 1004Kc)
> Synchronize counters for CPU 1: done.
> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> CPU2 revision is: 0001992f (MIPS 1004Kc)
> Synchronize counters for CPU 2: done.
> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> CPU3 revision is: 0001992f (MIPS 1004Kc)
> Synchronize counters for CPU 3: done.
> smp: Brought up 1 node, 4 CPUs
> devtmpfs: initialized
> clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> futex hash table entries: 1024 (order: 3, 32768 bytes)
> pinctrl core: initialized pinctrl subsystem
> NET: Registered protocol family 16
> mt7621-pci 1e140000.pcie: failed to get gpio perst
> mt7621-pci 1e140000.pcie: Parsing DT failed
> SCSI subsystem initialized
> usbcore: registered new interface driver usbfs
> usbcore: registered new interface driver hub
> usbcore: registered new device driver usb
> pcf857x 0-0026: probed
> i2c-mt7621 1e000900.i2c: clock 400KHz, re-start not support
> clocksource: Switched to clocksource GIC
> NET: Registered protocol family 2
> tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes)
> TCP established hash table entries: 2048 (order: 1, 8192 bytes)
> TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
> TCP: Hash tables configured (established 2048 bind 2048)
> UDP hash table entries: 256 (order: 1, 8192 bytes)
> UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
> NET: Registered protocol family 1
> Trying to unpack rootfs image as initramfs...
> rootfs image is not initramfs (invalid magic at start of compressed archive); looks like an initrd
> Freeing initrd memory: 16524K
> Initialise system trusted keyrings
> workingset: timestamp_bits=30 max_order=16 bucket_order=0
> squashfs: version 4.0 (2009/01/31) Phillip Lougher
> random: fast init done
> Key type asymmetric registered
> Asymmetric key parser 'x509' registered
> Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
> mt7621_gpio 1e000600.gpio: registering 32 gpios
> mt7621_gpio 1e000600.gpio: registering 32 gpios
> mt7621_gpio 1e000600.gpio: registering 32 gpios
> Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
> printk: console [ttyS0] disabled
> 1e000c00.uartlite: ttyS0 at MMIO 0x1e000c00 (irq = 18, base_baud = 3125000) is a 16550A
> printk: console [ttyS0] enabled
> printk: console [ttyS0] enabled
> printk: bootconsole [early0] disabled
> printk: bootconsole [early0] disabled
> 1e000d00.uartlite: ttyS1 at MMIO 0x1e000d00 (irq = 19, base_baud = 3125000) is a 16550A
> ledman: Copyright (C) SnapGear, 2000-2010.
> ledman: registered ERASE switch on IRQ28
> snapdog: HW/SW watchdog timer for SnapGear/Others
> cacheinfo: Failed to find cpu0 device node
> cacheinfo: Unable to detect cache hierarchy for CPU 0
> brd: module loaded
> mt7621-nand: NAND register bank at 0xbe003000
> mt7621-nand: NAND ECC register bank at 0xbe003800
> nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xdc
> nand: Micron NAND 512MiB 3,3V 8-bit
> nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
> nand: ecc bit: 4, spare_per_sector: 16
> Bad block table found at page 262080, version 0x01
> Bad block table found at page 262016, version 0x01
> 5 fixed-partitions partitions found on MTD device MT7621-NAND
> Creating 5 MTD partitions on "MT7621-NAND":
> 0x000000000000-0x000000200000 : "u-boot"
> 0x000000200000-0x000000300000 : "u-boot-env"
> 0x000000300000-0x000000500000 : "log"
> 0x000000500000-0x000020000000 : "flash"
> 0x000000000000-0x000020000000 : "all"
> libphy: Fixed MDIO Bus: probed
> tun: Universal TUN/TAP device driver, 1.6
> libphy: mdio: probed
> mtk_soc_eth 1e100000.ethernet: generated random MAC address a6:2b:8b:f3:dd:6a
> mtk_soc_eth 1e100000.ethernet: connected mac 0 to PHY at fixed-0:00 [uid=00000000, driver=Generic PHY]
> mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
> PPP generic driver version 2.4.2
> PPP BSD Compression module registered
> PPP Deflate Compression module registered
> PPP MPPE Compression module registered
> NET: Registered protocol family 24
> SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
> CSLIP: code copyright 1989 Regents of the University of California.
> usbcore: registered new interface driver ar5523
> usbcore: registered new interface driver asix
> usbcore: registered new interface driver ax88179_178a
> usbcore: registered new interface driver cdc_ether
> usbcore: registered new interface driver cdc_eem
> usbcore: registered new interface driver rndis_host
> usbcore: registered new interface driver cdc_subset
> usbcore: registered new interface driver cdc_ncm
> usbcore: registered new interface driver qmi_wwan
> usbcore: registered new interface driver cdc_mbim
> ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> ehci-pci: EHCI PCI platform driver
> ehci-platform: EHCI generic platform driver
> ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
> ohci-pci: OHCI PCI platform driver
> xhci-mtk 1e1c0000.xhci: xHCI Host Controller
> xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 1
> xhci-mtk 1e1c0000.xhci: hcc params 0x01401198 hci version 0x96 quirks 0x0000000000210010
> xhci-mtk 1e1c0000.xhci: irq 20, io mem 0x1e1c0000
> usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.01
> usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> usb usb1: Product: xHCI Host Controller
> usb usb1: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
> usb usb1: SerialNumber: 1e1c0000.xhci
> hub 1-0:1.0: USB hub found
> hub 1-0:1.0: 2 ports detected
> xhci-mtk 1e1c0000.xhci: xHCI Host Controller
> xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 2
> xhci-mtk 1e1c0000.xhci: Host supports USB 3.0  SuperSpeed
> usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
> usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.01
> usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> usb usb2: Product: xHCI Host Controller
> usb usb2: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
> usb usb2: SerialNumber: 1e1c0000.xhci
> hub 2-0:1.0: USB hub found
> hub 2-0:1.0: 1 port detected
> usbcore: registered new interface driver cdc_acm
> cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
> usbcore: registered new interface driver cdc_wdm
> usbcore: registered new interface driver usb-storage
> usbcore: registered new interface driver usbserial_generic
> usbserial: USB Serial support registered for generic
> usbcore: registered new interface driver ipw
> usbserial: USB Serial support registered for IPWireless converter
> usbcore: registered new interface driver option
> usbserial: USB Serial support registered for GSM modem (1-port)
> usbcore: registered new interface driver qcaux
> usbserial: USB Serial support registered for qcaux
> usbcore: registered new interface driver qcserial
> usbserial: USB Serial support registered for Qualcomm USB modem
> usbcore: registered new interface driver quatech2
> usbserial: USB Serial support registered for Quatech 2nd gen USB to Serial Driver
> usbcore: registered new interface driver usb_serial_simple
> usbserial: USB Serial support registered for carelink
> usbserial: USB Serial support registered for zio
> usbserial: USB Serial support registered for funsoft
> usbserial: USB Serial support registered for flashloader
> usbserial: USB Serial support registered for google
> usbserial: USB Serial support registered for libtransistor
> usbserial: USB Serial support registered for vivopay
> usbserial: USB Serial support registered for moto_modem
> usbserial: USB Serial support registered for motorola_tetra
> usbserial: USB Serial support registered for novatel_gps
> usbserial: USB Serial support registered for hp4x
> usbserial: USB Serial support registered for suunto
> usbserial: USB Serial support registered for siemens_mpi
> u32 classifier
>      input device check on
> ipip: IPv4 and MPLS over IPv4 tunneling driver
> gre: GRE over IPv4 demultiplexor driver
> ip_gre: GRE over IPv4 tunneling driver
> Initializing XFRM netlink socket
> NET: Registered protocol family 10
> Segment Routing with IPv6
> sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
> NET: Registered protocol family 17
> NET: Registered protocol family 15
> 8021q: 802.1Q VLAN Support v1.8
> Loading compiled-in X.509 certificates
> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>
>                 [HUNG]
>
> ---------------------------------------------------------------------------------
>

[-- Attachment #2: 0001-staging-mt7621-pci-use-perst-gpio-instead-of-builtin.patch --]
[-- Type: text/x-patch, Size: 8218 bytes --]

From 438047027c132628800c4a840e598a4bad8f7ca3 Mon Sep 17 00:00:00 2001
From: Sergio Paracuellos <sergio.paracuellos@gmail.com>
Date: Wed, 29 May 2019 09:58:07 +0200
Subject: [PATCH] staging: mt7621-pci: use perst gpio instead of builtin perst

Some boards need gpio instead of builtin perst. Use gpio for all
of them which was the approach of the original code.

Signed-off-by: Sergio Paracuellos <sergio.paracuellos@gmail.com>
---
 drivers/staging/mt7621-dts/mt7621.dtsi  |   3 +-
 drivers/staging/mt7621-pci/pci-mt7621.c | 110 ++++++++++++++++----------------
 2 files changed, 58 insertions(+), 55 deletions(-)

diff --git a/drivers/staging/mt7621-dts/mt7621.dtsi b/drivers/staging/mt7621-dts/mt7621.dtsi
index 280ec33..aed2458 100644
--- a/drivers/staging/mt7621-dts/mt7621.dtsi
+++ b/drivers/staging/mt7621-dts/mt7621.dtsi
@@ -1,5 +1,5 @@
 #include <dt-bindings/interrupt-controller/mips-gic.h>
-
+#include <dt-bindings/gpio/gpio.h>
 / {
 	#address-cells = <1>;
 	#size-cells = <1>;
@@ -468,6 +468,7 @@
 		#address-cells = <3>;
 		#size-cells = <2>;
 
+		perst-gpio = <&gpio 19 GPIO_ACTIVE_HIGH>;
 		pinctrl-names = "default";
 		pinctrl-0 = <&pcie_pins>;
 
diff --git a/drivers/staging/mt7621-pci/pci-mt7621.c b/drivers/staging/mt7621-pci/pci-mt7621.c
index 03d919a..ca91934 100644
--- a/drivers/staging/mt7621-pci/pci-mt7621.c
+++ b/drivers/staging/mt7621-pci/pci-mt7621.c
@@ -17,6 +17,7 @@
 
 #include <linux/bitops.h>
 #include <linux/delay.h>
+#include <linux/gpio/consumer.h>
 #include <linux/iopoll.h>
 #include <linux/module.h>
 #include <linux/of.h>
@@ -35,6 +36,7 @@
 
 /* sysctl */
 #define MT7621_CHIP_REV_ID		0x0c
+#define MT7621_GPIO_MODE        0x60
 #define CHIP_REV_MT7621_E2		0x0101
 
 /* MediaTek specific configuration registers */
@@ -81,7 +83,6 @@
 #define PCIE_BAR_ENABLE			BIT(0)
 #define PCIE_PORT_INT_EN(x)		BIT(20 + (x))
 #define PCIE_PORT_CLK_EN(x)		BIT(24 + (x))
-#define PCIE_PORT_PERST(x)		BIT(1 + (x))
 #define PCIE_PORT_LINKUP		BIT(0)
 
 #define PCIE_CLK_GEN_EN			BIT(31)
@@ -90,6 +91,10 @@
 #define PCIE_CLK_GEN1_EN		(BIT(27) | BIT(25))
 #define MEMORY_BASE			0x0
 
+#define PERST_MODE_MASK         GENMASK(11, 10)
+#define PERST_MODE_GPIO         BIT(10)
+#define PERST_DELAY_US          1000
+
 /**
  * struct mt7621_pcie_port - PCIe port information
  * @base: I/O mapped register base
@@ -119,6 +124,7 @@ struct mt7621_pcie_port {
  * @offset: IO / Memory offset
  * @dev: Pointer to PCIe device
  * @ports: pointer to PCIe port information
+ * @perst: gpio reset
  * @rst: pointer to pcie reset
  */
 struct mt7621_pcie {
@@ -132,6 +138,7 @@ struct mt7621_pcie {
 		resource_size_t io;
 	} offset;
 	struct list_head ports;
+    struct gpio_desc *perst;
 	struct reset_control *rst;
 };
 
@@ -198,6 +205,18 @@ static void write_config(struct mt7621_pcie *pcie, unsigned int dev,
 	pcie_write(pcie, val, RALINK_PCI_CONFIG_DATA);
 }
 
+static void mt7621_perst_gpio_pcie_assert(struct mt7621_pcie *pcie)
+{
+    gpiod_set_value(pcie->perst, 0);
+    mdelay(PERST_DELAY_US);
+}
+
+static void mt7621_perst_gpio_pcie_deassert(struct mt7621_pcie *pcie)
+{
+    gpiod_set_value(pcie->perst, 1);
+    mdelay(PERST_DELAY_US);
+}
+
 static inline void mt7621_control_assert(struct mt7621_pcie_port *port)
 {
 	u32 chip_rev_id = rt_sysc_r32(MT7621_CHIP_REV_ID);
@@ -344,6 +363,12 @@ static int mt7621_pcie_parse_dt(struct mt7621_pcie *pcie)
 	struct resource regs;
 	int err;
 
+    pcie->perst = devm_gpiod_get(dev, "perst", GPIOD_OUT_HIGH);
+    if (IS_ERR(pcie->perst)) {
+        dev_err(dev, "failed to get gpio perst\n");
+        return -EPROBE_DEFER;
+    }
+
 	err = of_address_to_resource(node, 0, &regs);
 	if (err) {
 		dev_err(dev, "missing \"reg\" property\n");
@@ -384,7 +409,6 @@ static int mt7621_pcie_init_port(struct mt7621_pcie_port *port)
 	struct mt7621_pcie *pcie = port->pcie;
 	struct device *dev = pcie->dev;
 	u32 slot = port->slot;
-	u32 val = 0;
 	int err;
 
 	/*
@@ -393,47 +417,33 @@ static int mt7621_pcie_init_port(struct mt7621_pcie_port *port)
 	 */
 	mt7621_reset_port(port);
 
-	val = read_config(pcie, slot, PCIE_FTS_NUM);
-	dev_info(dev, "Port %d N_FTS = %x\n", (unsigned int)val, slot);
-
 	err = phy_init(port->phy);
 	if (err) {
 		dev_err(dev, "failed to initialize port%d phy\n", slot);
-		goto err_phy_init;
+		return err;
 	}
 
 	err = phy_power_on(port->phy);
 	if (err) {
 		dev_err(dev, "failed to power on port%d phy\n", slot);
-		goto err_phy_on;
-	}
-
-	if ((pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) == 0) {
-		dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", slot);
-		mt7621_control_assert(port);
-		port->enabled = false;
-		err = -ENODEV;
-		goto err_no_link_up;
+		return err;
 	}
 
 	port->enabled = true;
 
 	return 0;
-
-err_no_link_up:
-	phy_power_off(port->phy);
-err_phy_on:
-	phy_exit(port->phy);
-err_phy_init:
-	return err;
 }
 
 static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct mt7621_pcie_port *port, *tmp;
+	u32 val = 0;
 	int err;
 
+    rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
+    mt7621_perst_gpio_pcie_assert(pcie);
+
 	list_for_each_entry_safe(port, tmp, &pcie->ports, list) {
 		u32 slot = port->slot;
 
@@ -441,7 +451,10 @@ static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 		if (err) {
 			dev_err(dev, "Initiating port %d failed\n", slot);
 			list_del(&port->list);
-		}
+		} else {
+	        val = read_config(pcie, slot, PCIE_FTS_NUM);
+	        dev_info(dev, "Port %d N_FTS = %x\n", slot, (unsigned int)val);
+        }
 	}
 
 	reset_control_assert(pcie->rst);
@@ -451,37 +464,32 @@ static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 	rt_sysc_m32(PCIE_CLK_GEN_DIS, PCIE_CLK_GEN_EN, RALINK_PCIE_CLK_GEN);
 	msleep(50);
 	reset_control_deassert(pcie->rst);
+
+    mt7621_perst_gpio_pcie_deassert(pcie);
+
+	list_for_each_entry(port, &pcie->ports, list) {
+		u32 slot = port->slot;
+
+	    if ((pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) == 0) {
+		    dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", slot);
+		    mt7621_control_assert(port);
+            phy_power_off(port->phy);
+		    port->enabled = false;
+        } else {
+	        /* enable pcie interrupt */
+	        val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR);
+	        val |= PCIE_PORT_INT_EN(slot);
+	        pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR);
+        }
+	}
 }
 
-static int mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
+static void mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
 {
 	struct mt7621_pcie *pcie = port->pcie;
 	u32 slot = port->slot;
 	u32 offset = MT7621_PCIE_OFFSET + (slot * MT7621_NEXT_PORT);
 	u32 val;
-	int err;
-
-	/* assert port PERST_N */
-	val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
-	val |= PCIE_PORT_PERST(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
-
-	/* de-assert port PERST_N */
-	val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
-	val &= ~PCIE_PORT_PERST(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
-
-	/* 100ms timeout value should be enough for Gen1 training */
-	err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
-				 val, !!(val & PCIE_PORT_LINKUP),
-				 20, 100 * USEC_PER_MSEC);
-	if (err)
-		return -ETIMEDOUT;
-
-	/* enable pcie interrupt */
-	val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR);
-	val |= PCIE_PORT_INT_EN(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR);
 
 	/* map 2G DDR region */
 	pcie_write(pcie, PCIE_BAR_MAP_MAX | PCIE_BAR_ENABLE,
@@ -492,8 +500,6 @@ static int mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
 	/* configure class code and revision ID */
 	pcie_write(pcie, PCIE_CLASS_CODE | PCIE_REVISION_ID,
 		   offset + RALINK_PCI_CLASS);
-
-	return 0;
 }
 
 static void mt7621_pcie_enable_ports(struct mt7621_pcie *pcie)
@@ -506,11 +512,7 @@ static void mt7621_pcie_enable_ports(struct mt7621_pcie *pcie)
 
 	list_for_each_entry(port, &pcie->ports, list) {
 		if (port->enabled) {
-			if (mt7621_pcie_enable_port(port)) {
-				dev_err(dev, "de-assert port %d PERST_N\n",
-					port->slot);
-				continue;
-			}
+			mt7621_pcie_enable_port(port);
 			dev_info(dev, "PCIE%d enabled\n", slot);
 			num_slots_enabled++;
 		}
-- 
2.7.4


[-- Attachment #3: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 4/6/19 7:36 pm, Sergio Paracuellos wrote:
> On Tue, Jun 4, 2019 at 9:14 AM Greg Ungerer <gerg@kernel.org> wrote:
>> On 4/6/19 3:06 pm, Sergio Paracuellos wrote:
>>> On Tue, Jun 4, 2019 at 3:31 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>> On 4/6/19 5:59 am, Sergio Paracuellos wrote:
>>>>> On Mon, Jun 3, 2019 at 2:32 PM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>> On 3/6/19 3:34 pm, Sergio Paracuellos wrote:
>>>>>>> On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>>> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
>>>>>>>>> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>>>>> On 30/5/19 10:44 am, Greg Ungerer wrote:
>>>>>>>>>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
>>>>>>>>>>> [snip]
>>>>>>> Can you try to read and set BIT(10) instead of write it directly?:
>>>>>>>
>>>>>>> Instead of:
>>>>>>>
>>>>>>> rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);
>>>>>>
>>>>>> Oh, yeah, that is definitely not going to work. There is a bunch of
>>>>>> other GPIO control bits in there for other hardware blocks. Would
>>>>>> explain the broken NAND flash behavior...
>>>>>
>>>>> Yes, my bad here. Sometimes is better to go to sleep :)).
>>>>>>
>>>>>>> Do:
>>>>>>>
>>>>>>> u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
>>>>>>> val |= PERST_MODE_GPIO;
>>>>>>> rt_sysc_w32(val, MT7621_GPIO_MODE);
>>>>>>
>>>>>> Much better result with that. Though ultimately it should clear
>>>>>> bits 10 and 11 (both are PERST_MODE bits) and then OR in BIT(10).
>>>>>
>>>>> Ok, so the following should do the trick:
>>>>>
>>>>> rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
>>>>>
>>>>> with PERST_MODE_MASK defined as:
>>>>>
>>>>> #define PERST_MODE_MASK         GENMASK(11, 10)
>>>>>
>>>>> (patch attached with this changes)
>>>>
>>>> I have mostly good news and a little bad news :-)
>>>>
>>>> I should have tested more thoroughly last night. Anyway, the new patch
>>>> works, even the IRQ of the PCI device. My Wifi PCI device works just
>>>> as well now as it did with the 4.20 pci-mt7621.c/pci-mt7621-phy.c.
>>>> I still get the lines:
>>>>
>>>> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
>>>> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
>>>>
>>>> However that is referring to the root host PCI device (0000:00:00.0),
>>>> not my PCI peripheral device (which is 0000:01:00.0). It is actually
>>>> probed and working properly.
>>>>
>>>> That is the good news.
>>>
>>> That makes sense :). Good news are always good news even bads are
>>> coming also :))
>>>
>>>>
>>>>
>>>>> It would be also good to know what happen if you comment the following lines:
>>>>>
>>>>> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
>>>>> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
>>>>>
>>>>> Are they really needed?
>>>>
>>>> Had no effect for me. Everything worked the same with or without
>>>> those lines as far as I could tell.
>>>
>>> Ok, I won't include them when I send a clean patch series with all of
>>> this changes.
>>>
>>>>
>>>>
>>>> [snip]
>>>>> Only one step more to get this properly working.
>>>>
>>>> Ok, now the bad news.
>>>>
>>>> I often get the boot hanging at various points in the PCI
>>>> initialization, setup and probing.
>>>>
>>>> For example sometimes it hangs with boot up to:
>>>>
>>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>
>>>>
>>>> Then sometimes it hangs at:
>>>>
>>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>>      mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>>      mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>>>>      mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>      mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>>
>>>
>>> It is weird, here it is not initializing any port... All of them are disabled.
>>>
>>>>
>>>> And then I also see it hang up to:
>>>>
>>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>>      mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>>      mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>      mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>>      mt7621-pci 1e140000.pcie: PCIE0 enabled
>>>>      mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>>>      mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>>>      pci_bus 0000:00: root bus resource [io  0xffffffff]
>>>>      pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>>>      pci_bus 0000:00: root bus resource [bus 00-ff]
>>>>      pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>>>>
>>>>
>>>> It happens on cold or warm boots. Sometimes it boots up all the
>>>> way and everything works perfectly.
>>>
>>> Can you try to change all the msleeps of the code driver in favour of
>>> mdelay's? Looks like
>>> some timing problem.
>>
>> Changed the msleeps to mdelays and it still hangs somtimes.
>> I doubled the delay times in the msleeps and tried that too -
>> that also sometimes hangs.
>>
>>
>>> If it doesn't work, it would be great to have a full trace of working
>>> 4.20 and no working current 5.x series
>>> version just to carefully compare them.
>>
>> I'll attach at the end a full working boot trace from 5.1 with the 4.20 pci-mt7621.c.
>> I'll attach a 5.1 boot trace with your patch and hang below. Not that this very
>> same image doesn't always hang. Same binary will sometimes boot fine.
>>
>>
>>>> I had perfect reliable boot and operation with linux-5.1 using the
>>>> older 4.20 pci-mt7621.c and pci-mt7621-phy.c.
>>>
>>> AFAICT, there is not pci-mt7621-phy.c in the v4.20, isn't it? Do you
>>> mean you put also that
>>> code into that version and compile them? Or are you just using "pci-mt7621.c"?
>>
>> I just leave that pci-mt7621-phy.c code in place in linux-5.1 when I copy in
>> the linux-4.20 pci-mt7621.c - even though it isn't used in that case.
>>
>> One thing I always notice is that the PCI probing happens much earlier
>> with the 4.20 pci-mt7621.c. Not sure that will have any impact though.
>>
> 
> It seems it is related because, looking at the trace which hangs I can see:
> 
> mt7621-pci 1e140000.pcie: failed to get gpio perst
> mt7621-pci 1e140000.pcie: Parsing DT failed
> 
> At that point gpio's driver are not being probed yet. This not happen
> with the original
> patch because it just access to memory to set up all the stuff intead
> of use the gpio's
> consumers apis which, IMHO, is the way to go.
> 
> Can you try to return "-EPROBE_DEFER" if getting the gpio fails in
> initialization?
> 
> (patch attached with this change and removing the two lines which
> seems to not be needed
> commented in previous patch)

New patch tried. Behavior is no different. Occasional hang during boot.
I can see that it does the probe, and now deferr. Full boot log of a
hung boot below.

Regards
Greg


--------------------------------------------------------------------------------
Linux version 5.1.0-ac0 (gerg@goober) (gcc version 5.4.0 (GCC)) #78 SMP Wed Jun 5 13:45:19 AEST 2019
SoC Type: MediaTek MT7621 ver:1 eco:3
printk: bootconsole [early0] enabled
CPU0 revision is: 0001992f (MIPS 1004Kc)
OF: fdt: No chosen node found, continuing without
MIPS: machine is Digi EX15
Determined physical RAM map:
  memory: 10000000 @ 00000000 (usable)
Initial ramdisk at: 0x81000000 (16920576 bytes)
VPE topology {2,2} total 4
Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
Zone ranges:
   Normal   [mem 0x0000000000000000-0x000000000fffffff]
Movable zone start for each node
Early memory node ranges
   node   0: [mem 0x0000000000000000-0x000000000fffffff]
Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
random: get_random_bytes called from start_kernel+0xb0/0x524 with crng_init=0
percpu: Embedded 15 pages/cpu s30144 r8192 d23104 u61440
Built 1 zonelists, mobility grouping on.  Total pages: 65024
Kernel command line:  boot=network boot_ver=19.3.223.0-0c02c07312-dirty console=ttyS0,115200 ubi.mtd=3 root=/dev/ram0 rd_start=0x81000000 rd_size=0x1023000
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Writing ErrCtl register=00020513
Readback ErrCtl register=00020513
Memory: 233756K/262144K available (7216K kernel code, 247K rwdata, 1320K rodata, 292K init, 230K bss, 28388K reserved, 0K cma-reserved)
SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
rcu: Hierarchical RCU implementation.
rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
NR_IRQS: 256
clocksource: GIC: mask: 0xffffffffffffffff max_cycles: 0xcaf478abb4, max_idle_ns: 440795247997 ns
sched_clock: 32 bits at 100 Hz, resolution 10000000ns, wraps every 21474836475000000ns
Calibrating delay loop... 586.13 BogoMIPS (lpj=2930688)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
rcu: Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
CPU1 revision is: 0001992f (MIPS 1004Kc)
Synchronize counters for CPU 1: done.
Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
CPU2 revision is: 0001992f (MIPS 1004Kc)
Synchronize counters for CPU 2: done.
Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
CPU3 revision is: 0001992f (MIPS 1004Kc)
Synchronize counters for CPU 3: done.
smp: Brought up 1 node, 4 CPUs
devtmpfs: initialized
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 1024 (order: 3, 32768 bytes)
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
mt7621-pci 1e140000.pcie: failed to get gpio perst
mt7621-pci 1e140000.pcie: Parsing DT failed
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
pcf857x 0-0026: probed
i2c-mt7621 1e000900.i2c: clock 400KHz, re-start not support
clocksource: Switched to clocksource GIC
NET: Registered protocol family 2
tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes)
TCP established hash table entries: 2048 (order: 1, 8192 bytes)
TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
UDP hash table entries: 256 (order: 1, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
NET: Registered protocol family 1
Trying to unpack rootfs image as initramfs...
rootfs image is not initramfs (invalid magic at start of compressed archive); looks like an initrd
Freeing initrd memory: 16524K
Initialise system trusted keyrings
workingset: timestamp_bits=30 max_order=16 bucket_order=0
squashfs: version 4.0 (2009/01/31) Phillip Lougher
random: fast init done
Key type asymmetric registered
Asymmetric key parser 'x509' registered
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
mt7621_gpio 1e000600.gpio: registering 32 gpios
mt7621_gpio 1e000600.gpio: registering 32 gpios
mt7621_gpio 1e000600.gpio: registering 32 gpios
Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
printk: console [ttyS0] disabled
1e000c00.uartlite: ttyS0 at MMIO 0x1e000c00 (irq = 18, base_baud = 3125000) is a 16550A
printk: console [ttyS0] enabled
printk: console [ttyS0] enabled
printk: bootconsole [early0] disabled
printk: bootconsole [early0] disabled
1e000d00.uartlite: ttyS1 at MMIO 0x1e000d00 (irq = 19, base_baud = 3125000) is a 16550A
ledman: Copyright (C) SnapGear, 2000-2010.
ledman: registered ERASE switch on IRQ28
snapdog: HW/SW watchdog timer for SnapGear/Others
cacheinfo: Failed to find cpu0 device node
cacheinfo: Unable to detect cache hierarchy for CPU 0
brd: module loaded
mt7621-nand: NAND register bank at 0xbe003000
mt7621-nand: NAND ECC register bank at 0xbe003800
nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xdc
nand: Micron NAND 512MiB 3,3V 8-bit
nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
nand: ecc bit: 4, spare_per_sector: 16
Bad block table found at page 262080, version 0x01
Bad block table found at page 262016, version 0x01
5 fixed-partitions partitions found on MTD device MT7621-NAND
Creating 5 MTD partitions on "MT7621-NAND":
0x000000000000-0x000000200000 : "u-boot"
0x000000200000-0x000000300000 : "u-boot-env"
0x000000300000-0x000000500000 : "log"
0x000000500000-0x000020000000 : "flash"
0x000000000000-0x000020000000 : "all"
libphy: Fixed MDIO Bus: probed
tun: Universal TUN/TAP device driver, 1.6
libphy: mdio: probed
mtk_soc_eth 1e100000.ethernet: generated random MAC address 8a:8c:8b:24:60:92
mtk_soc_eth 1e100000.ethernet: connected mac 0 to PHY at fixed-0:00 [uid=00000000, driver=Generic PHY]
mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
PPP generic driver version 2.4.2
PPP BSD Compression module registered
PPP Deflate Compression module registered
PPP MPPE Compression module registered
NET: Registered protocol family 24
SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
CSLIP: code copyright 1989 Regents of the University of California.
usbcore: registered new interface driver ar5523
usbcore: registered new interface driver asix
usbcore: registered new interface driver ax88179_178a
usbcore: registered new interface driver cdc_ether
usbcore: registered new interface driver cdc_eem
usbcore: registered new interface driver rndis_host
usbcore: registered new interface driver cdc_subset
usbcore: registered new interface driver cdc_ncm
usbcore: registered new interface driver qmi_wwan
usbcore: registered new interface driver cdc_mbim
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
ehci-platform: EHCI generic platform driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci-pci: OHCI PCI platform driver
xhci-mtk 1e1c0000.xhci: xHCI Host Controller
xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 1
xhci-mtk 1e1c0000.xhci: hcc params 0x01401198 hci version 0x96 quirks 0x0000000000210010
xhci-mtk 1e1c0000.xhci: irq 20, io mem 0x1e1c0000
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.01
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: xHCI Host Controller
usb usb1: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
usb usb1: SerialNumber: 1e1c0000.xhci
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
xhci-mtk 1e1c0000.xhci: xHCI Host Controller
xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 2
xhci-mtk 1e1c0000.xhci: Host supports USB 3.0  SuperSpeed
usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.01
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: xHCI Host Controller
usb usb2: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
usb usb2: SerialNumber: 1e1c0000.xhci
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 1 port detected
usbcore: registered new interface driver cdc_acm
cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
usbcore: registered new interface driver cdc_wdm
usbcore: registered new interface driver usb-storage
usbcore: registered new interface driver usbserial_generic
usbserial: USB Serial support registered for generic
usbcore: registered new interface driver ipw
usbserial: USB Serial support registered for IPWireless converter
usbcore: registered new interface driver option
usbserial: USB Serial support registered for GSM modem (1-port)
usbcore: registered new interface driver qcaux
usbserial: USB Serial support registered for qcaux
usbcore: registered new interface driver qcserial
usbserial: USB Serial support registered for Qualcomm USB modem
usbcore: registered new interface driver quatech2
usbserial: USB Serial support registered for Quatech 2nd gen USB to Serial Driver
usbcore: registered new interface driver usb_serial_simple
usbserial: USB Serial support registered for carelink
usbserial: USB Serial support registered for zio
usbserial: USB Serial support registered for funsoft
usbserial: USB Serial support registered for flashloader
usbserial: USB Serial support registered for google
usbserial: USB Serial support registered for libtransistor
usbserial: USB Serial support registered for vivopay
usbserial: USB Serial support registered for moto_modem
usbserial: USB Serial support registered for motorola_tetra
usbserial: USB Serial support registered for novatel_gps
usbserial: USB Serial support registered for hp4x
usbserial: USB Serial support registered for suunto
usbserial: USB Serial support registered for siemens_mpi
u32 classifier
     input device check on
ipip: IPv4 and MPLS over IPv4 tunneling driver
gre: GRE over IPv4 demultiplexor driver
ip_gre: GRE over IPv4 tunneling driver
Initializing XFRM netlink socket
NET: Registered protocol family 10
Segment Routing with IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
NET: Registered protocol family 17
NET: Registered protocol family 15
8021q: 802.1Q VLAN Support v1.8
Loading compiled-in X.509 certificates
mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: Port 0 N_FTS = 1b102800
mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: Port 1 N_FTS = 1b102800
mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
mt7621-pci 1e140000.pcie: Port 2 N_FTS = 1b102800
mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)

          [HUNG]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

Hi Greg,

On Wed, Jun 5, 2019 at 6:01 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> On 4/6/19 7:36 pm, Sergio Paracuellos wrote:
> > On Tue, Jun 4, 2019 at 9:14 AM Greg Ungerer <gerg@kernel.org> wrote:
> >> On 4/6/19 3:06 pm, Sergio Paracuellos wrote:
> >>> On Tue, Jun 4, 2019 at 3:31 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>> On 4/6/19 5:59 am, Sergio Paracuellos wrote:
> >>>>> On Mon, Jun 3, 2019 at 2:32 PM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>> On 3/6/19 3:34 pm, Sergio Paracuellos wrote:
> >>>>>>> On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>>>> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
> >>>>>>>>> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>>>>>> On 30/5/19 10:44 am, Greg Ungerer wrote:
> >>>>>>>>>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
> >>>>>>>>>>> [snip]
> >>>>>>> Can you try to read and set BIT(10) instead of write it directly?:
> >>>>>>>
> >>>>>>> Instead of:
> >>>>>>>
> >>>>>>> rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);
> >>>>>>
> >>>>>> Oh, yeah, that is definitely not going to work. There is a bunch of
> >>>>>> other GPIO control bits in there for other hardware blocks. Would
> >>>>>> explain the broken NAND flash behavior...
> >>>>>
> >>>>> Yes, my bad here. Sometimes is better to go to sleep :)).
> >>>>>>
> >>>>>>> Do:
> >>>>>>>
> >>>>>>> u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
> >>>>>>> val |= PERST_MODE_GPIO;
> >>>>>>> rt_sysc_w32(val, MT7621_GPIO_MODE);
> >>>>>>
> >>>>>> Much better result with that. Though ultimately it should clear
> >>>>>> bits 10 and 11 (both are PERST_MODE bits) and then OR in BIT(10).
> >>>>>
> >>>>> Ok, so the following should do the trick:
> >>>>>
> >>>>> rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
> >>>>>
> >>>>> with PERST_MODE_MASK defined as:
> >>>>>
> >>>>> #define PERST_MODE_MASK         GENMASK(11, 10)
> >>>>>
> >>>>> (patch attached with this changes)
> >>>>
> >>>> I have mostly good news and a little bad news :-)
> >>>>
> >>>> I should have tested more thoroughly last night. Anyway, the new patch
> >>>> works, even the IRQ of the PCI device. My Wifi PCI device works just
> >>>> as well now as it did with the 4.20 pci-mt7621.c/pci-mt7621-phy.c.
> >>>> I still get the lines:
> >>>>
> >>>> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
> >>>> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
> >>>>
> >>>> However that is referring to the root host PCI device (0000:00:00.0),
> >>>> not my PCI peripheral device (which is 0000:01:00.0). It is actually
> >>>> probed and working properly.
> >>>>
> >>>> That is the good news.
> >>>
> >>> That makes sense :). Good news are always good news even bads are
> >>> coming also :))
> >>>
> >>>>
> >>>>
> >>>>> It would be also good to know what happen if you comment the following lines:
> >>>>>
> >>>>> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
> >>>>> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
> >>>>>
> >>>>> Are they really needed?
> >>>>
> >>>> Had no effect for me. Everything worked the same with or without
> >>>> those lines as far as I could tell.
> >>>
> >>> Ok, I won't include them when I send a clean patch series with all of
> >>> this changes.
> >>>
> >>>>
> >>>>
> >>>> [snip]
> >>>>> Only one step more to get this properly working.
> >>>>
> >>>> Ok, now the bad news.
> >>>>
> >>>> I often get the boot hanging at various points in the PCI
> >>>> initialization, setup and probing.
> >>>>
> >>>> For example sometimes it hangs with boot up to:
> >>>>
> >>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>
> >>>>
> >>>> Then sometimes it hangs at:
> >>>>
> >>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>>      mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>>      mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> >>>>      mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>>      mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>>
> >>>
> >>> It is weird, here it is not initializing any port... All of them are disabled.
> >>>
> >>>>
> >>>> And then I also see it hang up to:
> >>>>
> >>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>>      mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>>      mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>>      mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>>      mt7621-pci 1e140000.pcie: PCIE0 enabled
> >>>>      mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >>>>      mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >>>>      pci_bus 0000:00: root bus resource [io  0xffffffff]
> >>>>      pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >>>>      pci_bus 0000:00: root bus resource [bus 00-ff]
> >>>>      pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> >>>>
> >>>>
> >>>> It happens on cold or warm boots. Sometimes it boots up all the
> >>>> way and everything works perfectly.
> >>>
> >>> Can you try to change all the msleeps of the code driver in favour of
> >>> mdelay's? Looks like
> >>> some timing problem.
> >>
> >> Changed the msleeps to mdelays and it still hangs somtimes.
> >> I doubled the delay times in the msleeps and tried that too -
> >> that also sometimes hangs.
> >>
> >>
> >>> If it doesn't work, it would be great to have a full trace of working
> >>> 4.20 and no working current 5.x series
> >>> version just to carefully compare them.
> >>
> >> I'll attach at the end a full working boot trace from 5.1 with the 4.20 pci-mt7621.c.
> >> I'll attach a 5.1 boot trace with your patch and hang below. Not that this very
> >> same image doesn't always hang. Same binary will sometimes boot fine.
> >>
> >>
> >>>> I had perfect reliable boot and operation with linux-5.1 using the
> >>>> older 4.20 pci-mt7621.c and pci-mt7621-phy.c.
> >>>
> >>> AFAICT, there is not pci-mt7621-phy.c in the v4.20, isn't it? Do you
> >>> mean you put also that
> >>> code into that version and compile them? Or are you just using "pci-mt7621.c"?
> >>
> >> I just leave that pci-mt7621-phy.c code in place in linux-5.1 when I copy in
> >> the linux-4.20 pci-mt7621.c - even though it isn't used in that case.
> >>
> >> One thing I always notice is that the PCI probing happens much earlier
> >> with the 4.20 pci-mt7621.c. Not sure that will have any impact though.
> >>
> >
> > It seems it is related because, looking at the trace which hangs I can see:
> >
> > mt7621-pci 1e140000.pcie: failed to get gpio perst
> > mt7621-pci 1e140000.pcie: Parsing DT failed
> >
> > At that point gpio's driver are not being probed yet. This not happen
> > with the original
> > patch because it just access to memory to set up all the stuff intead
> > of use the gpio's
> > consumers apis which, IMHO, is the way to go.
> >
> > Can you try to return "-EPROBE_DEFER" if getting the gpio fails in
> > initialization?
> >
> > (patch attached with this change and removing the two lines which
> > seems to not be needed
> > commented in previous patch)
>
> New patch tried. Behavior is no different. Occasional hang during boot.
> I can see that it does the probe, and now deferr. Full boot log of a
> hung boot below.

Ok, so that means it was properly being deferred with the previous patch also.

The original code set up all at a very early stage (arch_initcall)
just after the pinctrl
driver is being set up. The new one without my last patches was also
being executed
later because of the phy. At arch_initcall the 'phy_create()' is
called before the phy module is initialized, so 'phy_class' is NULL,
the new phy isn't placed in the right class, and it cannot be found.
So the phy driver uses 'module_init' instead of 'arch_initcall' and it
worked for the gnubee board. But according to your traces when it
hangs the problem seems to be with gpio's being resetting or something
so the pcie link is not in up
state for any reason I cannot understand. So we have to determine if
the problem is because of the phy and gpio init being done in later
stages.

We can try to change 'arch_initcall' to a later stage changing it to
'module_init' to see if it is a deferring problem or something. The
other thing we can try to determine if it is a gpio access problem is
to change in the last patch
the gpio descriptor uses with you initial memory accessign hack to see
if it eventually hungs or not. If this both tests result in eventually
hung boot I'll try to prepare a patch with the current phy code in the
'arch_initcall' stage putting in the the same pci-mt7621.c file to see
if order matters. No more ideas by now, sorry :(

> Regards
> Greg

Best regards,
    Sergio Paracuellos
>
>
> --------------------------------------------------------------------------------
> Linux version 5.1.0-ac0 (gerg@goober) (gcc version 5.4.0 (GCC)) #78 SMP Wed Jun 5 13:45:19 AEST 2019
> SoC Type: MediaTek MT7621 ver:1 eco:3
> printk: bootconsole [early0] enabled
> CPU0 revision is: 0001992f (MIPS 1004Kc)
> OF: fdt: No chosen node found, continuing without
> MIPS: machine is Digi EX15
> Determined physical RAM map:
>   memory: 10000000 @ 00000000 (usable)
> Initial ramdisk at: 0x81000000 (16920576 bytes)
> VPE topology {2,2} total 4
> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> Zone ranges:
>    Normal   [mem 0x0000000000000000-0x000000000fffffff]
> Movable zone start for each node
> Early memory node ranges
>    node   0: [mem 0x0000000000000000-0x000000000fffffff]
> Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
> random: get_random_bytes called from start_kernel+0xb0/0x524 with crng_init=0
> percpu: Embedded 15 pages/cpu s30144 r8192 d23104 u61440
> Built 1 zonelists, mobility grouping on.  Total pages: 65024
> Kernel command line:  boot=network boot_ver=19.3.223.0-0c02c07312-dirty console=ttyS0,115200 ubi.mtd=3 root=/dev/ram0 rd_start=0x81000000 rd_size=0x1023000
> Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
> Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
> Writing ErrCtl register=00020513
> Readback ErrCtl register=00020513
> Memory: 233756K/262144K available (7216K kernel code, 247K rwdata, 1320K rodata, 292K init, 230K bss, 28388K reserved, 0K cma-reserved)
> SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
> rcu: Hierarchical RCU implementation.
> rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
> NR_IRQS: 256
> clocksource: GIC: mask: 0xffffffffffffffff max_cycles: 0xcaf478abb4, max_idle_ns: 440795247997 ns
> sched_clock: 32 bits at 100 Hz, resolution 10000000ns, wraps every 21474836475000000ns
> Calibrating delay loop... 586.13 BogoMIPS (lpj=2930688)
> pid_max: default: 32768 minimum: 301
> Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
> Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
> rcu: Hierarchical SRCU implementation.
> smp: Bringing up secondary CPUs ...
> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> CPU1 revision is: 0001992f (MIPS 1004Kc)
> Synchronize counters for CPU 1: done.
> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> CPU2 revision is: 0001992f (MIPS 1004Kc)
> Synchronize counters for CPU 2: done.
> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> CPU3 revision is: 0001992f (MIPS 1004Kc)
> Synchronize counters for CPU 3: done.
> smp: Brought up 1 node, 4 CPUs
> devtmpfs: initialized
> clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> futex hash table entries: 1024 (order: 3, 32768 bytes)
> pinctrl core: initialized pinctrl subsystem
> NET: Registered protocol family 16
> mt7621-pci 1e140000.pcie: failed to get gpio perst
> mt7621-pci 1e140000.pcie: Parsing DT failed
> SCSI subsystem initialized
> usbcore: registered new interface driver usbfs
> usbcore: registered new interface driver hub
> usbcore: registered new device driver usb
> pcf857x 0-0026: probed
> i2c-mt7621 1e000900.i2c: clock 400KHz, re-start not support
> clocksource: Switched to clocksource GIC
> NET: Registered protocol family 2
> tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes)
> TCP established hash table entries: 2048 (order: 1, 8192 bytes)
> TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
> TCP: Hash tables configured (established 2048 bind 2048)
> UDP hash table entries: 256 (order: 1, 8192 bytes)
> UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
> NET: Registered protocol family 1
> Trying to unpack rootfs image as initramfs...
> rootfs image is not initramfs (invalid magic at start of compressed archive); looks like an initrd
> Freeing initrd memory: 16524K
> Initialise system trusted keyrings
> workingset: timestamp_bits=30 max_order=16 bucket_order=0
> squashfs: version 4.0 (2009/01/31) Phillip Lougher
> random: fast init done
> Key type asymmetric registered
> Asymmetric key parser 'x509' registered
> Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
> mt7621_gpio 1e000600.gpio: registering 32 gpios
> mt7621_gpio 1e000600.gpio: registering 32 gpios
> mt7621_gpio 1e000600.gpio: registering 32 gpios
> Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
> printk: console [ttyS0] disabled
> 1e000c00.uartlite: ttyS0 at MMIO 0x1e000c00 (irq = 18, base_baud = 3125000) is a 16550A
> printk: console [ttyS0] enabled
> printk: console [ttyS0] enabled
> printk: bootconsole [early0] disabled
> printk: bootconsole [early0] disabled
> 1e000d00.uartlite: ttyS1 at MMIO 0x1e000d00 (irq = 19, base_baud = 3125000) is a 16550A
> ledman: Copyright (C) SnapGear, 2000-2010.
> ledman: registered ERASE switch on IRQ28
> snapdog: HW/SW watchdog timer for SnapGear/Others
> cacheinfo: Failed to find cpu0 device node
> cacheinfo: Unable to detect cache hierarchy for CPU 0
> brd: module loaded
> mt7621-nand: NAND register bank at 0xbe003000
> mt7621-nand: NAND ECC register bank at 0xbe003800
> nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xdc
> nand: Micron NAND 512MiB 3,3V 8-bit
> nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
> nand: ecc bit: 4, spare_per_sector: 16
> Bad block table found at page 262080, version 0x01
> Bad block table found at page 262016, version 0x01
> 5 fixed-partitions partitions found on MTD device MT7621-NAND
> Creating 5 MTD partitions on "MT7621-NAND":
> 0x000000000000-0x000000200000 : "u-boot"
> 0x000000200000-0x000000300000 : "u-boot-env"
> 0x000000300000-0x000000500000 : "log"
> 0x000000500000-0x000020000000 : "flash"
> 0x000000000000-0x000020000000 : "all"
> libphy: Fixed MDIO Bus: probed
> tun: Universal TUN/TAP device driver, 1.6
> libphy: mdio: probed
> mtk_soc_eth 1e100000.ethernet: generated random MAC address 8a:8c:8b:24:60:92
> mtk_soc_eth 1e100000.ethernet: connected mac 0 to PHY at fixed-0:00 [uid=00000000, driver=Generic PHY]
> mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
> PPP generic driver version 2.4.2
> PPP BSD Compression module registered
> PPP Deflate Compression module registered
> PPP MPPE Compression module registered
> NET: Registered protocol family 24
> SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
> CSLIP: code copyright 1989 Regents of the University of California.
> usbcore: registered new interface driver ar5523
> usbcore: registered new interface driver asix
> usbcore: registered new interface driver ax88179_178a
> usbcore: registered new interface driver cdc_ether
> usbcore: registered new interface driver cdc_eem
> usbcore: registered new interface driver rndis_host
> usbcore: registered new interface driver cdc_subset
> usbcore: registered new interface driver cdc_ncm
> usbcore: registered new interface driver qmi_wwan
> usbcore: registered new interface driver cdc_mbim
> ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> ehci-pci: EHCI PCI platform driver
> ehci-platform: EHCI generic platform driver
> ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
> ohci-pci: OHCI PCI platform driver
> xhci-mtk 1e1c0000.xhci: xHCI Host Controller
> xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 1
> xhci-mtk 1e1c0000.xhci: hcc params 0x01401198 hci version 0x96 quirks 0x0000000000210010
> xhci-mtk 1e1c0000.xhci: irq 20, io mem 0x1e1c0000
> usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.01
> usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> usb usb1: Product: xHCI Host Controller
> usb usb1: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
> usb usb1: SerialNumber: 1e1c0000.xhci
> hub 1-0:1.0: USB hub found
> hub 1-0:1.0: 2 ports detected
> xhci-mtk 1e1c0000.xhci: xHCI Host Controller
> xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 2
> xhci-mtk 1e1c0000.xhci: Host supports USB 3.0  SuperSpeed
> usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
> usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.01
> usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> usb usb2: Product: xHCI Host Controller
> usb usb2: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
> usb usb2: SerialNumber: 1e1c0000.xhci
> hub 2-0:1.0: USB hub found
> hub 2-0:1.0: 1 port detected
> usbcore: registered new interface driver cdc_acm
> cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
> usbcore: registered new interface driver cdc_wdm
> usbcore: registered new interface driver usb-storage
> usbcore: registered new interface driver usbserial_generic
> usbserial: USB Serial support registered for generic
> usbcore: registered new interface driver ipw
> usbserial: USB Serial support registered for IPWireless converter
> usbcore: registered new interface driver option
> usbserial: USB Serial support registered for GSM modem (1-port)
> usbcore: registered new interface driver qcaux
> usbserial: USB Serial support registered for qcaux
> usbcore: registered new interface driver qcserial
> usbserial: USB Serial support registered for Qualcomm USB modem
> usbcore: registered new interface driver quatech2
> usbserial: USB Serial support registered for Quatech 2nd gen USB to Serial Driver
> usbcore: registered new interface driver usb_serial_simple
> usbserial: USB Serial support registered for carelink
> usbserial: USB Serial support registered for zio
> usbserial: USB Serial support registered for funsoft
> usbserial: USB Serial support registered for flashloader
> usbserial: USB Serial support registered for google
> usbserial: USB Serial support registered for libtransistor
> usbserial: USB Serial support registered for vivopay
> usbserial: USB Serial support registered for moto_modem
> usbserial: USB Serial support registered for motorola_tetra
> usbserial: USB Serial support registered for novatel_gps
> usbserial: USB Serial support registered for hp4x
> usbserial: USB Serial support registered for suunto
> usbserial: USB Serial support registered for siemens_mpi
> u32 classifier
>      input device check on
> ipip: IPv4 and MPLS over IPv4 tunneling driver
> gre: GRE over IPv4 demultiplexor driver
> ip_gre: GRE over IPv4 tunneling driver
> Initializing XFRM netlink socket
> NET: Registered protocol family 10
> Segment Routing with IPv6
> sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
> NET: Registered protocol family 17
> NET: Registered protocol family 15
> 8021q: 802.1Q VLAN Support v1.8
> Loading compiled-in X.509 certificates
> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 0 N_FTS = 1b102800
> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 1 N_FTS = 1b102800
> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> mt7621-pci 1e140000.pcie: Port 2 N_FTS = 1b102800
> mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>
>           [HUNG]
>
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

[-- Attachment #1: Type: text/plain, Size: 22450 bytes --]

Hi Greg,

On Wed, Jun 5, 2019 at 7:47 AM Sergio Paracuellos
<sergio.paracuellos@gmail.com> wrote:
>
> Hi Greg,
>
> On Wed, Jun 5, 2019 at 6:01 AM Greg Ungerer <gerg@kernel.org> wrote:
> >
> > Hi Sergio,
> >
> > On 4/6/19 7:36 pm, Sergio Paracuellos wrote:
> > > On Tue, Jun 4, 2019 at 9:14 AM Greg Ungerer <gerg@kernel.org> wrote:
> > >> On 4/6/19 3:06 pm, Sergio Paracuellos wrote:
> > >>> On Tue, Jun 4, 2019 at 3:31 AM Greg Ungerer <gerg@kernel.org> wrote:
> > >>>> On 4/6/19 5:59 am, Sergio Paracuellos wrote:
> > >>>>> On Mon, Jun 3, 2019 at 2:32 PM Greg Ungerer <gerg@kernel.org> wrote:
> > >>>>>> On 3/6/19 3:34 pm, Sergio Paracuellos wrote:
> > >>>>>>> On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
> > >>>>>>>> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
> > >>>>>>>>> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
> > >>>>>>>>>> On 30/5/19 10:44 am, Greg Ungerer wrote:
> > >>>>>>>>>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
> > >>>>>>>>>>> [snip]
> > >>>>>>> Can you try to read and set BIT(10) instead of write it directly?:
> > >>>>>>>
> > >>>>>>> Instead of:
> > >>>>>>>
> > >>>>>>> rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);
> > >>>>>>
> > >>>>>> Oh, yeah, that is definitely not going to work. There is a bunch of
> > >>>>>> other GPIO control bits in there for other hardware blocks. Would
> > >>>>>> explain the broken NAND flash behavior...
> > >>>>>
> > >>>>> Yes, my bad here. Sometimes is better to go to sleep :)).
> > >>>>>>
> > >>>>>>> Do:
> > >>>>>>>
> > >>>>>>> u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
> > >>>>>>> val |= PERST_MODE_GPIO;
> > >>>>>>> rt_sysc_w32(val, MT7621_GPIO_MODE);
> > >>>>>>
> > >>>>>> Much better result with that. Though ultimately it should clear
> > >>>>>> bits 10 and 11 (both are PERST_MODE bits) and then OR in BIT(10).
> > >>>>>
> > >>>>> Ok, so the following should do the trick:
> > >>>>>
> > >>>>> rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
> > >>>>>
> > >>>>> with PERST_MODE_MASK defined as:
> > >>>>>
> > >>>>> #define PERST_MODE_MASK         GENMASK(11, 10)
> > >>>>>
> > >>>>> (patch attached with this changes)
> > >>>>
> > >>>> I have mostly good news and a little bad news :-)
> > >>>>
> > >>>> I should have tested more thoroughly last night. Anyway, the new patch
> > >>>> works, even the IRQ of the PCI device. My Wifi PCI device works just
> > >>>> as well now as it did with the 4.20 pci-mt7621.c/pci-mt7621-phy.c.
> > >>>> I still get the lines:
> > >>>>
> > >>>> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
> > >>>> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
> > >>>>
> > >>>> However that is referring to the root host PCI device (0000:00:00.0),
> > >>>> not my PCI peripheral device (which is 0000:01:00.0). It is actually
> > >>>> probed and working properly.
> > >>>>
> > >>>> That is the good news.
> > >>>
> > >>> That makes sense :). Good news are always good news even bads are
> > >>> coming also :))
> > >>>
> > >>>>
> > >>>>
> > >>>>> It would be also good to know what happen if you comment the following lines:
> > >>>>>
> > >>>>> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
> > >>>>> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
> > >>>>>
> > >>>>> Are they really needed?
> > >>>>
> > >>>> Had no effect for me. Everything worked the same with or without
> > >>>> those lines as far as I could tell.
> > >>>
> > >>> Ok, I won't include them when I send a clean patch series with all of
> > >>> this changes.
> > >>>
> > >>>>
> > >>>>
> > >>>> [snip]
> > >>>>> Only one step more to get this properly working.
> > >>>>
> > >>>> Ok, now the bad news.
> > >>>>
> > >>>> I often get the boot hanging at various points in the PCI
> > >>>> initialization, setup and probing.
> > >>>>
> > >>>> For example sometimes it hangs with boot up to:
> > >>>>
> > >>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> > >>>>
> > >>>>
> > >>>> Then sometimes it hangs at:
> > >>>>
> > >>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> > >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> > >>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> > >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> > >>>>      mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> > >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> > >>>>      mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> > >>>>      mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> > >>>>      mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> > >>>>
> > >>>
> > >>> It is weird, here it is not initializing any port... All of them are disabled.
> > >>>
> > >>>>
> > >>>> And then I also see it hang up to:
> > >>>>
> > >>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> > >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> > >>>>      mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> > >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> > >>>>      mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> > >>>>      mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> > >>>>      mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> > >>>>      mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> > >>>>      mt7621-pci 1e140000.pcie: PCIE0 enabled
> > >>>>      mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> > >>>>      mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> > >>>>      pci_bus 0000:00: root bus resource [io  0xffffffff]
> > >>>>      pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> > >>>>      pci_bus 0000:00: root bus resource [bus 00-ff]
> > >>>>      pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> > >>>>
> > >>>>
> > >>>> It happens on cold or warm boots. Sometimes it boots up all the
> > >>>> way and everything works perfectly.
> > >>>
> > >>> Can you try to change all the msleeps of the code driver in favour of
> > >>> mdelay's? Looks like
> > >>> some timing problem.
> > >>
> > >> Changed the msleeps to mdelays and it still hangs somtimes.
> > >> I doubled the delay times in the msleeps and tried that too -
> > >> that also sometimes hangs.
> > >>
> > >>
> > >>> If it doesn't work, it would be great to have a full trace of working
> > >>> 4.20 and no working current 5.x series
> > >>> version just to carefully compare them.
> > >>
> > >> I'll attach at the end a full working boot trace from 5.1 with the 4.20 pci-mt7621.c.
> > >> I'll attach a 5.1 boot trace with your patch and hang below. Not that this very
> > >> same image doesn't always hang. Same binary will sometimes boot fine.
> > >>
> > >>
> > >>>> I had perfect reliable boot and operation with linux-5.1 using the
> > >>>> older 4.20 pci-mt7621.c and pci-mt7621-phy.c.
> > >>>
> > >>> AFAICT, there is not pci-mt7621-phy.c in the v4.20, isn't it? Do you
> > >>> mean you put also that
> > >>> code into that version and compile them? Or are you just using "pci-mt7621.c"?
> > >>
> > >> I just leave that pci-mt7621-phy.c code in place in linux-5.1 when I copy in
> > >> the linux-4.20 pci-mt7621.c - even though it isn't used in that case.
> > >>
> > >> One thing I always notice is that the PCI probing happens much earlier
> > >> with the 4.20 pci-mt7621.c. Not sure that will have any impact though.
> > >>
> > >
> > > It seems it is related because, looking at the trace which hangs I can see:
> > >
> > > mt7621-pci 1e140000.pcie: failed to get gpio perst
> > > mt7621-pci 1e140000.pcie: Parsing DT failed
> > >
> > > At that point gpio's driver are not being probed yet. This not happen
> > > with the original
> > > patch because it just access to memory to set up all the stuff intead
> > > of use the gpio's
> > > consumers apis which, IMHO, is the way to go.
> > >
> > > Can you try to return "-EPROBE_DEFER" if getting the gpio fails in
> > > initialization?
> > >
> > > (patch attached with this change and removing the two lines which
> > > seems to not be needed
> > > commented in previous patch)
> >
> > New patch tried. Behavior is no different. Occasional hang during boot.
> > I can see that it does the probe, and now deferr. Full boot log of a
> > hung boot below.
>
> Ok, so that means it was properly being deferred with the previous patch also.
>
> The original code set up all at a very early stage (arch_initcall)
> just after the pinctrl
> driver is being set up. The new one without my last patches was also
> being executed
> later because of the phy. At arch_initcall the 'phy_create()' is
> called before the phy module is initialized, so 'phy_class' is NULL,
> the new phy isn't placed in the right class, and it cannot be found.
> So the phy driver uses 'module_init' instead of 'arch_initcall' and it
> worked for the gnubee board. But according to your traces when it
> hangs the problem seems to be with gpio's being resetting or something
> so the pcie link is not in up
> state for any reason I cannot understand. So we have to determine if
> the problem is because of the phy and gpio init being done in later
> stages.
>
> We can try to change 'arch_initcall' to a later stage changing it to
> 'module_init' to see if it is a deferring problem or something. The
> other thing we can try to determine if it is a gpio access problem is
> to change in the last patch
> the gpio descriptor uses with you initial memory accessign hack to see
> if it eventually hungs or not. If this both tests result in eventually
> hung boot I'll try to prepare a patch with the current phy code in the
> 'arch_initcall' stage putting in the the same pci-mt7621.c file to see
> if order matters. No more ideas by now, sorry :(
>
> > Regards
> > Greg
>
> Best regards,
>     Sergio Paracuellos
> >
> >
> > --------------------------------------------------------------------------------
> > Linux version 5.1.0-ac0 (gerg@goober) (gcc version 5.4.0 (GCC)) #78 SMP Wed Jun 5 13:45:19 AEST 2019
> > SoC Type: MediaTek MT7621 ver:1 eco:3
> > printk: bootconsole [early0] enabled
> > CPU0 revision is: 0001992f (MIPS 1004Kc)
> > OF: fdt: No chosen node found, continuing without
> > MIPS: machine is Digi EX15
> > Determined physical RAM map:
> >   memory: 10000000 @ 00000000 (usable)
> > Initial ramdisk at: 0x81000000 (16920576 bytes)
> > VPE topology {2,2} total 4
> > Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> > Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> > MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> > Zone ranges:
> >    Normal   [mem 0x0000000000000000-0x000000000fffffff]
> > Movable zone start for each node
> > Early memory node ranges
> >    node   0: [mem 0x0000000000000000-0x000000000fffffff]
> > Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
> > random: get_random_bytes called from start_kernel+0xb0/0x524 with crng_init=0
> > percpu: Embedded 15 pages/cpu s30144 r8192 d23104 u61440
> > Built 1 zonelists, mobility grouping on.  Total pages: 65024
> > Kernel command line:  boot=network boot_ver=19.3.223.0-0c02c07312-dirty console=ttyS0,115200 ubi.mtd=3 root=/dev/ram0 rd_start=0x81000000 rd_size=0x1023000
> > Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
> > Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
> > Writing ErrCtl register=00020513
> > Readback ErrCtl register=00020513
> > Memory: 233756K/262144K available (7216K kernel code, 247K rwdata, 1320K rodata, 292K init, 230K bss, 28388K reserved, 0K cma-reserved)
> > SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
> > rcu: Hierarchical RCU implementation.
> > rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
> > NR_IRQS: 256
> > clocksource: GIC: mask: 0xffffffffffffffff max_cycles: 0xcaf478abb4, max_idle_ns: 440795247997 ns
> > sched_clock: 32 bits at 100 Hz, resolution 10000000ns, wraps every 21474836475000000ns
> > Calibrating delay loop... 586.13 BogoMIPS (lpj=2930688)
> > pid_max: default: 32768 minimum: 301
> > Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
> > Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
> > rcu: Hierarchical SRCU implementation.
> > smp: Bringing up secondary CPUs ...
> > Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> > Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> > MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> > CPU1 revision is: 0001992f (MIPS 1004Kc)
> > Synchronize counters for CPU 1: done.
> > Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> > Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> > MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> > CPU2 revision is: 0001992f (MIPS 1004Kc)
> > Synchronize counters for CPU 2: done.
> > Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> > Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> > MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> > CPU3 revision is: 0001992f (MIPS 1004Kc)
> > Synchronize counters for CPU 3: done.
> > smp: Brought up 1 node, 4 CPUs
> > devtmpfs: initialized
> > clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> > futex hash table entries: 1024 (order: 3, 32768 bytes)
> > pinctrl core: initialized pinctrl subsystem
> > NET: Registered protocol family 16
> > mt7621-pci 1e140000.pcie: failed to get gpio perst
> > mt7621-pci 1e140000.pcie: Parsing DT failed
> > SCSI subsystem initialized
> > usbcore: registered new interface driver usbfs
> > usbcore: registered new interface driver hub
> > usbcore: registered new device driver usb
> > pcf857x 0-0026: probed
> > i2c-mt7621 1e000900.i2c: clock 400KHz, re-start not support
> > clocksource: Switched to clocksource GIC
> > NET: Registered protocol family 2
> > tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes)
> > TCP established hash table entries: 2048 (order: 1, 8192 bytes)
> > TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
> > TCP: Hash tables configured (established 2048 bind 2048)
> > UDP hash table entries: 256 (order: 1, 8192 bytes)
> > UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
> > NET: Registered protocol family 1
> > Trying to unpack rootfs image as initramfs...
> > rootfs image is not initramfs (invalid magic at start of compressed archive); looks like an initrd
> > Freeing initrd memory: 16524K
> > Initialise system trusted keyrings
> > workingset: timestamp_bits=30 max_order=16 bucket_order=0
> > squashfs: version 4.0 (2009/01/31) Phillip Lougher
> > random: fast init done
> > Key type asymmetric registered
> > Asymmetric key parser 'x509' registered
> > Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
> > mt7621_gpio 1e000600.gpio: registering 32 gpios
> > mt7621_gpio 1e000600.gpio: registering 32 gpios
> > mt7621_gpio 1e000600.gpio: registering 32 gpios
> > Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
> > printk: console [ttyS0] disabled
> > 1e000c00.uartlite: ttyS0 at MMIO 0x1e000c00 (irq = 18, base_baud = 3125000) is a 16550A
> > printk: console [ttyS0] enabled
> > printk: console [ttyS0] enabled
> > printk: bootconsole [early0] disabled
> > printk: bootconsole [early0] disabled
> > 1e000d00.uartlite: ttyS1 at MMIO 0x1e000d00 (irq = 19, base_baud = 3125000) is a 16550A
> > ledman: Copyright (C) SnapGear, 2000-2010.
> > ledman: registered ERASE switch on IRQ28
> > snapdog: HW/SW watchdog timer for SnapGear/Others
> > cacheinfo: Failed to find cpu0 device node
> > cacheinfo: Unable to detect cache hierarchy for CPU 0
> > brd: module loaded
> > mt7621-nand: NAND register bank at 0xbe003000
> > mt7621-nand: NAND ECC register bank at 0xbe003800
> > nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xdc
> > nand: Micron NAND 512MiB 3,3V 8-bit
> > nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
> > nand: ecc bit: 4, spare_per_sector: 16
> > Bad block table found at page 262080, version 0x01
> > Bad block table found at page 262016, version 0x01
> > 5 fixed-partitions partitions found on MTD device MT7621-NAND
> > Creating 5 MTD partitions on "MT7621-NAND":
> > 0x000000000000-0x000000200000 : "u-boot"
> > 0x000000200000-0x000000300000 : "u-boot-env"
> > 0x000000300000-0x000000500000 : "log"
> > 0x000000500000-0x000020000000 : "flash"
> > 0x000000000000-0x000020000000 : "all"
> > libphy: Fixed MDIO Bus: probed
> > tun: Universal TUN/TAP device driver, 1.6
> > libphy: mdio: probed
> > mtk_soc_eth 1e100000.ethernet: generated random MAC address 8a:8c:8b:24:60:92
> > mtk_soc_eth 1e100000.ethernet: connected mac 0 to PHY at fixed-0:00 [uid=00000000, driver=Generic PHY]
> > mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
> > PPP generic driver version 2.4.2
> > PPP BSD Compression module registered
> > PPP Deflate Compression module registered
> > PPP MPPE Compression module registered
> > NET: Registered protocol family 24
> > SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
> > CSLIP: code copyright 1989 Regents of the University of California.
> > usbcore: registered new interface driver ar5523
> > usbcore: registered new interface driver asix
> > usbcore: registered new interface driver ax88179_178a
> > usbcore: registered new interface driver cdc_ether
> > usbcore: registered new interface driver cdc_eem
> > usbcore: registered new interface driver rndis_host
> > usbcore: registered new interface driver cdc_subset
> > usbcore: registered new interface driver cdc_ncm
> > usbcore: registered new interface driver qmi_wwan
> > usbcore: registered new interface driver cdc_mbim
> > ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> > ehci-pci: EHCI PCI platform driver
> > ehci-platform: EHCI generic platform driver
> > ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
> > ohci-pci: OHCI PCI platform driver
> > xhci-mtk 1e1c0000.xhci: xHCI Host Controller
> > xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 1
> > xhci-mtk 1e1c0000.xhci: hcc params 0x01401198 hci version 0x96 quirks 0x0000000000210010
> > xhci-mtk 1e1c0000.xhci: irq 20, io mem 0x1e1c0000
> > usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.01
> > usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> > usb usb1: Product: xHCI Host Controller
> > usb usb1: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
> > usb usb1: SerialNumber: 1e1c0000.xhci
> > hub 1-0:1.0: USB hub found
> > hub 1-0:1.0: 2 ports detected
> > xhci-mtk 1e1c0000.xhci: xHCI Host Controller
> > xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 2
> > xhci-mtk 1e1c0000.xhci: Host supports USB 3.0  SuperSpeed
> > usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
> > usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.01
> > usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> > usb usb2: Product: xHCI Host Controller
> > usb usb2: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
> > usb usb2: SerialNumber: 1e1c0000.xhci
> > hub 2-0:1.0: USB hub found
> > hub 2-0:1.0: 1 port detected
> > usbcore: registered new interface driver cdc_acm
> > cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
> > usbcore: registered new interface driver cdc_wdm
> > usbcore: registered new interface driver usb-storage
> > usbcore: registered new interface driver usbserial_generic
> > usbserial: USB Serial support registered for generic
> > usbcore: registered new interface driver ipw
> > usbserial: USB Serial support registered for IPWireless converter
> > usbcore: registered new interface driver option
> > usbserial: USB Serial support registered for GSM modem (1-port)
> > usbcore: registered new interface driver qcaux
> > usbserial: USB Serial support registered for qcaux
> > usbcore: registered new interface driver qcserial
> > usbserial: USB Serial support registered for Qualcomm USB modem
> > usbcore: registered new interface driver quatech2
> > usbserial: USB Serial support registered for Quatech 2nd gen USB to Serial Driver
> > usbcore: registered new interface driver usb_serial_simple
> > usbserial: USB Serial support registered for carelink
> > usbserial: USB Serial support registered for zio
> > usbserial: USB Serial support registered for funsoft
> > usbserial: USB Serial support registered for flashloader
> > usbserial: USB Serial support registered for google
> > usbserial: USB Serial support registered for libtransistor
> > usbserial: USB Serial support registered for vivopay
> > usbserial: USB Serial support registered for moto_modem
> > usbserial: USB Serial support registered for motorola_tetra
> > usbserial: USB Serial support registered for novatel_gps
> > usbserial: USB Serial support registered for hp4x
> > usbserial: USB Serial support registered for suunto
> > usbserial: USB Serial support registered for siemens_mpi
> > u32 classifier
> >      input device check on
> > ipip: IPv4 and MPLS over IPv4 tunneling driver
> > gre: GRE over IPv4 demultiplexor driver
> > ip_gre: GRE over IPv4 tunneling driver
> > Initializing XFRM netlink socket
> > NET: Registered protocol family 10
> > Segment Routing with IPv6
> > sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
> > NET: Registered protocol family 17
> > NET: Registered protocol family 15
> > 8021q: 802.1Q VLAN Support v1.8
> > Loading compiled-in X.509 certificates
> > mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> > mt7621-pci 1e140000.pcie: Port 0 N_FTS = 1b102800
> > mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> > mt7621-pci 1e140000.pcie: Port 1 N_FTS = 1b102800
> > mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> > mt7621-pci 1e140000.pcie: Port 2 N_FTS = 1b102800
> > mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> > mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> > mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)

I changed also the way port resets are done. Now all of ports are
being reset before
enabling the phy. Also, power_off the phy is done before reseting the
port so it should
not hang now. The real problem here is if the pcie link is properly
detected to get a working
PCI.

Patch attached.

> >
> >           [HUNG]
> >

Best regards,
    Sergio Paracuellos

[-- Attachment #2: 0001-staging-mt7621-pci-use-perst-gpio-instead-of-builtin.patch --]
[-- Type: text/x-patch, Size: 8426 bytes --]

From a07e1c8167f0b71759ad44b42f1250bbe821f366 Mon Sep 17 00:00:00 2001
From: Sergio Paracuellos <sparacuellos@ikergune.com>
Date: Wed, 29 May 2019 09:58:07 +0200
Subject: [PATCH] staging: mt7621-pci: use perst gpio instead of builtin perst

Some boards need gpio instead of builtin perst. Use gpio for all
of them which was the approach of the original code.

Signed-off-by: Sergio Paracuellos <sparacuellos@ikergune.com>
---
 drivers/staging/mt7621-dts/mt7621.dtsi  |   3 +-
 drivers/staging/mt7621-pci/pci-mt7621.c | 122 ++++++++++++++++----------------
 2 files changed, 64 insertions(+), 61 deletions(-)

diff --git a/drivers/staging/mt7621-dts/mt7621.dtsi b/drivers/staging/mt7621-dts/mt7621.dtsi
index 280ec33..aed2458 100644
--- a/drivers/staging/mt7621-dts/mt7621.dtsi
+++ b/drivers/staging/mt7621-dts/mt7621.dtsi
@@ -1,5 +1,5 @@
 #include <dt-bindings/interrupt-controller/mips-gic.h>
-
+#include <dt-bindings/gpio/gpio.h>
 / {
 	#address-cells = <1>;
 	#size-cells = <1>;
@@ -468,6 +468,7 @@
 		#address-cells = <3>;
 		#size-cells = <2>;
 
+		perst-gpio = <&gpio 19 GPIO_ACTIVE_HIGH>;
 		pinctrl-names = "default";
 		pinctrl-0 = <&pcie_pins>;
 
diff --git a/drivers/staging/mt7621-pci/pci-mt7621.c b/drivers/staging/mt7621-pci/pci-mt7621.c
index 03d919a..b13920a 100644
--- a/drivers/staging/mt7621-pci/pci-mt7621.c
+++ b/drivers/staging/mt7621-pci/pci-mt7621.c
@@ -17,6 +17,7 @@
 
 #include <linux/bitops.h>
 #include <linux/delay.h>
+#include <linux/gpio/consumer.h>
 #include <linux/iopoll.h>
 #include <linux/module.h>
 #include <linux/of.h>
@@ -35,6 +36,7 @@
 
 /* sysctl */
 #define MT7621_CHIP_REV_ID		0x0c
+#define MT7621_GPIO_MODE        0x60
 #define CHIP_REV_MT7621_E2		0x0101
 
 /* MediaTek specific configuration registers */
@@ -81,7 +83,6 @@
 #define PCIE_BAR_ENABLE			BIT(0)
 #define PCIE_PORT_INT_EN(x)		BIT(20 + (x))
 #define PCIE_PORT_CLK_EN(x)		BIT(24 + (x))
-#define PCIE_PORT_PERST(x)		BIT(1 + (x))
 #define PCIE_PORT_LINKUP		BIT(0)
 
 #define PCIE_CLK_GEN_EN			BIT(31)
@@ -90,6 +91,10 @@
 #define PCIE_CLK_GEN1_EN		(BIT(27) | BIT(25))
 #define MEMORY_BASE			0x0
 
+#define PERST_MODE_MASK         GENMASK(11, 10)
+#define PERST_MODE_GPIO         BIT(10)
+#define PERST_DELAY_US          1000
+
 /**
  * struct mt7621_pcie_port - PCIe port information
  * @base: I/O mapped register base
@@ -119,6 +124,7 @@ struct mt7621_pcie_port {
  * @offset: IO / Memory offset
  * @dev: Pointer to PCIe device
  * @ports: pointer to PCIe port information
+ * @perst: gpio reset
  * @rst: pointer to pcie reset
  */
 struct mt7621_pcie {
@@ -132,6 +138,7 @@ struct mt7621_pcie {
 		resource_size_t io;
 	} offset;
 	struct list_head ports;
+    struct gpio_desc *perst;
 	struct reset_control *rst;
 };
 
@@ -198,6 +205,18 @@ static void write_config(struct mt7621_pcie *pcie, unsigned int dev,
 	pcie_write(pcie, val, RALINK_PCI_CONFIG_DATA);
 }
 
+static void mt7621_perst_gpio_pcie_assert(struct mt7621_pcie *pcie)
+{
+    gpiod_set_value(pcie->perst, 0);
+    mdelay(PERST_DELAY_US);
+}
+
+static void mt7621_perst_gpio_pcie_deassert(struct mt7621_pcie *pcie)
+{
+    gpiod_set_value(pcie->perst, 1);
+    mdelay(PERST_DELAY_US);
+}
+
 static inline void mt7621_control_assert(struct mt7621_pcie_port *port)
 {
 	u32 chip_rev_id = rt_sysc_r32(MT7621_CHIP_REV_ID);
@@ -344,6 +363,12 @@ static int mt7621_pcie_parse_dt(struct mt7621_pcie *pcie)
 	struct resource regs;
 	int err;
 
+    pcie->perst = devm_gpiod_get(dev, "perst", GPIOD_OUT_HIGH);
+    if (IS_ERR(pcie->perst)) {
+        dev_err(dev, "failed to get gpio perst\n");
+        return -EPROBE_DEFER;
+    }
+
 	err = of_address_to_resource(node, 0, &regs);
 	if (err) {
 		dev_err(dev, "missing \"reg\" property\n");
@@ -384,56 +409,41 @@ static int mt7621_pcie_init_port(struct mt7621_pcie_port *port)
 	struct mt7621_pcie *pcie = port->pcie;
 	struct device *dev = pcie->dev;
 	u32 slot = port->slot;
-	u32 val = 0;
 	int err;
 
-	/*
-	 * Any MT7621 Ralink pcie controller that doesn't have 0x0101 at
-	 * the end of the chip_id has inverted PCI resets.
-	 */
-	mt7621_reset_port(port);
-
-	val = read_config(pcie, slot, PCIE_FTS_NUM);
-	dev_info(dev, "Port %d N_FTS = %x\n", (unsigned int)val, slot);
-
 	err = phy_init(port->phy);
 	if (err) {
 		dev_err(dev, "failed to initialize port%d phy\n", slot);
-		goto err_phy_init;
+		return err;
 	}
 
 	err = phy_power_on(port->phy);
 	if (err) {
 		dev_err(dev, "failed to power on port%d phy\n", slot);
-		goto err_phy_on;
-	}
-
-	if ((pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) == 0) {
-		dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", slot);
-		mt7621_control_assert(port);
-		port->enabled = false;
-		err = -ENODEV;
-		goto err_no_link_up;
+		return err;
 	}
 
 	port->enabled = true;
 
 	return 0;
-
-err_no_link_up:
-	phy_power_off(port->phy);
-err_phy_on:
-	phy_exit(port->phy);
-err_phy_init:
-	return err;
 }
 
 static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct mt7621_pcie_port *port, *tmp;
+	u32 val = 0;
 	int err;
 
+	list_for_each_entry(port, &pcie->ports, list)
+        mt7621_control_assert(port);
+
+    rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
+    mt7621_perst_gpio_pcie_assert(pcie);
+
+	list_for_each_entry(port, &pcie->ports, list)
+        mt7621_control_deassert(port);
+
 	list_for_each_entry_safe(port, tmp, &pcie->ports, list) {
 		u32 slot = port->slot;
 
@@ -441,7 +451,10 @@ static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 		if (err) {
 			dev_err(dev, "Initiating port %d failed\n", slot);
 			list_del(&port->list);
-		}
+		} else {
+	        val = read_config(pcie, slot, PCIE_FTS_NUM);
+	        dev_info(dev, "Port %d N_FTS = %x\n", slot, (unsigned int)val);
+        }
 	}
 
 	reset_control_assert(pcie->rst);
@@ -451,37 +464,32 @@ static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 	rt_sysc_m32(PCIE_CLK_GEN_DIS, PCIE_CLK_GEN_EN, RALINK_PCIE_CLK_GEN);
 	msleep(50);
 	reset_control_deassert(pcie->rst);
+
+    mt7621_perst_gpio_pcie_deassert(pcie);
+
+	list_for_each_entry(port, &pcie->ports, list) {
+		u32 slot = port->slot;
+
+	    if ((pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) == 0) {
+		    dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", slot);
+            phy_power_off(port->phy);
+		    mt7621_control_assert(port);
+		    port->enabled = false;
+        } else {
+	        /* enable pcie interrupt */
+	        val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR);
+	        val |= PCIE_PORT_INT_EN(slot);
+	        pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR);
+        }
+	}
 }
 
-static int mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
+static void mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
 {
 	struct mt7621_pcie *pcie = port->pcie;
 	u32 slot = port->slot;
 	u32 offset = MT7621_PCIE_OFFSET + (slot * MT7621_NEXT_PORT);
 	u32 val;
-	int err;
-
-	/* assert port PERST_N */
-	val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
-	val |= PCIE_PORT_PERST(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
-
-	/* de-assert port PERST_N */
-	val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
-	val &= ~PCIE_PORT_PERST(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
-
-	/* 100ms timeout value should be enough for Gen1 training */
-	err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
-				 val, !!(val & PCIE_PORT_LINKUP),
-				 20, 100 * USEC_PER_MSEC);
-	if (err)
-		return -ETIMEDOUT;
-
-	/* enable pcie interrupt */
-	val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR);
-	val |= PCIE_PORT_INT_EN(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR);
 
 	/* map 2G DDR region */
 	pcie_write(pcie, PCIE_BAR_MAP_MAX | PCIE_BAR_ENABLE,
@@ -492,8 +500,6 @@ static int mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
 	/* configure class code and revision ID */
 	pcie_write(pcie, PCIE_CLASS_CODE | PCIE_REVISION_ID,
 		   offset + RALINK_PCI_CLASS);
-
-	return 0;
 }
 
 static void mt7621_pcie_enable_ports(struct mt7621_pcie *pcie)
@@ -506,11 +512,7 @@ static void mt7621_pcie_enable_ports(struct mt7621_pcie *pcie)
 
 	list_for_each_entry(port, &pcie->ports, list) {
 		if (port->enabled) {
-			if (mt7621_pcie_enable_port(port)) {
-				dev_err(dev, "de-assert port %d PERST_N\n",
-					port->slot);
-				continue;
-			}
+			mt7621_pcie_enable_port(port);
 			dev_info(dev, "PCIE%d enabled\n", slot);
 			num_slots_enabled++;
 		}
-- 
2.7.4


[-- Attachment #3: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 5/6/19 10:28 pm, Sergio Paracuellos wrote:
> On Wed, Jun 5, 2019 at 7:47 AM Sergio Paracuellos
> <sergio.paracuellos@gmail.com> wrote:
>> On Wed, Jun 5, 2019 at 6:01 AM Greg Ungerer <gerg@kernel.org> wrote:
>>> On 4/6/19 7:36 pm, Sergio Paracuellos wrote:
>>>> On Tue, Jun 4, 2019 at 9:14 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>> On 4/6/19 3:06 pm, Sergio Paracuellos wrote:
>>>>>> On Tue, Jun 4, 2019 at 3:31 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>> On 4/6/19 5:59 am, Sergio Paracuellos wrote:
>>>>>>>> On Mon, Jun 3, 2019 at 2:32 PM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>>>> On 3/6/19 3:34 pm, Sergio Paracuellos wrote:
>>>>>>>>>> On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>>>>>> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
>>>>>>>>>>>> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>>>>>>>> On 30/5/19 10:44 am, Greg Ungerer wrote:
>>>>>>>>>>>>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
>>>>>>>>>>>>>> [snip]
>>>>>>>>>> Can you try to read and set BIT(10) instead of write it directly?:
>>>>>>>>>>
>>>>>>>>>> Instead of:
>>>>>>>>>>
>>>>>>>>>> rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);
>>>>>>>>>
>>>>>>>>> Oh, yeah, that is definitely not going to work. There is a bunch of
>>>>>>>>> other GPIO control bits in there for other hardware blocks. Would
>>>>>>>>> explain the broken NAND flash behavior...
>>>>>>>>
>>>>>>>> Yes, my bad here. Sometimes is better to go to sleep :)).
>>>>>>>>>
>>>>>>>>>> Do:
>>>>>>>>>>
>>>>>>>>>> u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
>>>>>>>>>> val |= PERST_MODE_GPIO;
>>>>>>>>>> rt_sysc_w32(val, MT7621_GPIO_MODE);
>>>>>>>>>
>>>>>>>>> Much better result with that. Though ultimately it should clear
>>>>>>>>> bits 10 and 11 (both are PERST_MODE bits) and then OR in BIT(10).
>>>>>>>>
>>>>>>>> Ok, so the following should do the trick:
>>>>>>>>
>>>>>>>> rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
>>>>>>>>
>>>>>>>> with PERST_MODE_MASK defined as:
>>>>>>>>
>>>>>>>> #define PERST_MODE_MASK         GENMASK(11, 10)
>>>>>>>>
>>>>>>>> (patch attached with this changes)
>>>>>>>
>>>>>>> I have mostly good news and a little bad news :-)
>>>>>>>
>>>>>>> I should have tested more thoroughly last night. Anyway, the new patch
>>>>>>> works, even the IRQ of the PCI device. My Wifi PCI device works just
>>>>>>> as well now as it did with the 4.20 pci-mt7621.c/pci-mt7621-phy.c.
>>>>>>> I still get the lines:
>>>>>>>
>>>>>>> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
>>>>>>> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
>>>>>>>
>>>>>>> However that is referring to the root host PCI device (0000:00:00.0),
>>>>>>> not my PCI peripheral device (which is 0000:01:00.0). It is actually
>>>>>>> probed and working properly.
>>>>>>>
>>>>>>> That is the good news.
>>>>>>
>>>>>> That makes sense :). Good news are always good news even bads are
>>>>>> coming also :))
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> It would be also good to know what happen if you comment the following lines:
>>>>>>>>
>>>>>>>> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
>>>>>>>> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
>>>>>>>>
>>>>>>>> Are they really needed?
>>>>>>>
>>>>>>> Had no effect for me. Everything worked the same with or without
>>>>>>> those lines as far as I could tell.
>>>>>>
>>>>>> Ok, I won't include them when I send a clean patch series with all of
>>>>>> this changes.
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [snip]
>>>>>>>> Only one step more to get this properly working.
>>>>>>>
>>>>>>> Ok, now the bad news.
>>>>>>>
>>>>>>> I often get the boot hanging at various points in the PCI
>>>>>>> initialization, setup and probing.
>>>>>>>
>>>>>>> For example sometimes it hangs with boot up to:
>>>>>>>
>>>>>>>       mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>>>
>>>>>>>
>>>>>>> Then sometimes it hangs at:
>>>>>>>
>>>>>>>       mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>>>>>       mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>>>>>       mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>>>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>>>>>       mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>>>>>>>       mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>>>>       mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>>>>>
>>>>>>
>>>>>> It is weird, here it is not initializing any port... All of them are disabled.
>>>>>>
>>>>>>>
>>>>>>> And then I also see it hang up to:
>>>>>>>
>>>>>>>       mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>>>>>       mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>>>>>       mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>>>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>>>>>       mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>>>>       mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>>>>>       mt7621-pci 1e140000.pcie: PCIE0 enabled
>>>>>>>       mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>>>>>>       mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>>>>>>       pci_bus 0000:00: root bus resource [io  0xffffffff]
>>>>>>>       pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>>>>>>       pci_bus 0000:00: root bus resource [bus 00-ff]
>>>>>>>       pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>>>>>>>
>>>>>>>
>>>>>>> It happens on cold or warm boots. Sometimes it boots up all the
>>>>>>> way and everything works perfectly.
>>>>>>
>>>>>> Can you try to change all the msleeps of the code driver in favour of
>>>>>> mdelay's? Looks like
>>>>>> some timing problem.
>>>>>
>>>>> Changed the msleeps to mdelays and it still hangs somtimes.
>>>>> I doubled the delay times in the msleeps and tried that too -
>>>>> that also sometimes hangs.
>>>>>
>>>>>
>>>>>> If it doesn't work, it would be great to have a full trace of working
>>>>>> 4.20 and no working current 5.x series
>>>>>> version just to carefully compare them.
>>>>>
>>>>> I'll attach at the end a full working boot trace from 5.1 with the 4.20 pci-mt7621.c.
>>>>> I'll attach a 5.1 boot trace with your patch and hang below. Not that this very
>>>>> same image doesn't always hang. Same binary will sometimes boot fine.
>>>>>
>>>>>
>>>>>>> I had perfect reliable boot and operation with linux-5.1 using the
>>>>>>> older 4.20 pci-mt7621.c and pci-mt7621-phy.c.
>>>>>>
>>>>>> AFAICT, there is not pci-mt7621-phy.c in the v4.20, isn't it? Do you
>>>>>> mean you put also that
>>>>>> code into that version and compile them? Or are you just using "pci-mt7621.c"?
>>>>>
>>>>> I just leave that pci-mt7621-phy.c code in place in linux-5.1 when I copy in
>>>>> the linux-4.20 pci-mt7621.c - even though it isn't used in that case.
>>>>>
>>>>> One thing I always notice is that the PCI probing happens much earlier
>>>>> with the 4.20 pci-mt7621.c. Not sure that will have any impact though.
>>>>>
>>>>
>>>> It seems it is related because, looking at the trace which hangs I can see:
>>>>
>>>> mt7621-pci 1e140000.pcie: failed to get gpio perst
>>>> mt7621-pci 1e140000.pcie: Parsing DT failed
>>>>
>>>> At that point gpio's driver are not being probed yet. This not happen
>>>> with the original
>>>> patch because it just access to memory to set up all the stuff intead
>>>> of use the gpio's
>>>> consumers apis which, IMHO, is the way to go.
>>>>
>>>> Can you try to return "-EPROBE_DEFER" if getting the gpio fails in
>>>> initialization?
>>>>
>>>> (patch attached with this change and removing the two lines which
>>>> seems to not be needed
>>>> commented in previous patch)
>>>
>>> New patch tried. Behavior is no different. Occasional hang during boot.
>>> I can see that it does the probe, and now deferr. Full boot log of a
>>> hung boot below.
>>
>> Ok, so that means it was properly being deferred with the previous patch also.
>>
>> The original code set up all at a very early stage (arch_initcall)
>> just after the pinctrl
>> driver is being set up. The new one without my last patches was also
>> being executed
>> later because of the phy. At arch_initcall the 'phy_create()' is
>> called before the phy module is initialized, so 'phy_class' is NULL,
>> the new phy isn't placed in the right class, and it cannot be found.
>> So the phy driver uses 'module_init' instead of 'arch_initcall' and it
>> worked for the gnubee board. But according to your traces when it
>> hangs the problem seems to be with gpio's being resetting or something
>> so the pcie link is not in up
>> state for any reason I cannot understand. So we have to determine if
>> the problem is because of the phy and gpio init being done in later
>> stages.
>>
>> We can try to change 'arch_initcall' to a later stage changing it to
>> 'module_init' to see if it is a deferring problem or something. The
>> other thing we can try to determine if it is a gpio access problem is
>> to change in the last patch
>> the gpio descriptor uses with you initial memory accessign hack to see
>> if it eventually hungs or not. If this both tests result in eventually
>> hung boot I'll try to prepare a patch with the current phy code in the
>> 'arch_initcall' stage putting in the the same pci-mt7621.c file to see
>> if order matters. No more ideas by now, sorry :(
>>
>>> Regards
>>> Greg
>>
>> Best regards,
>>      Sergio Paracuellos
>>>
>>>
>>> --------------------------------------------------------------------------------
>>> Linux version 5.1.0-ac0 (gerg@goober) (gcc version 5.4.0 (GCC)) #78 SMP Wed Jun 5 13:45:19 AEST 2019
>>> SoC Type: MediaTek MT7621 ver:1 eco:3
>>> printk: bootconsole [early0] enabled
>>> CPU0 revision is: 0001992f (MIPS 1004Kc)
>>> OF: fdt: No chosen node found, continuing without
>>> MIPS: machine is Digi EX15
>>> Determined physical RAM map:
>>>    memory: 10000000 @ 00000000 (usable)
>>> Initial ramdisk at: 0x81000000 (16920576 bytes)
>>> VPE topology {2,2} total 4
>>> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
>>> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
>>> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
>>> Zone ranges:
>>>     Normal   [mem 0x0000000000000000-0x000000000fffffff]
>>> Movable zone start for each node
>>> Early memory node ranges
>>>     node   0: [mem 0x0000000000000000-0x000000000fffffff]
>>> Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
>>> random: get_random_bytes called from start_kernel+0xb0/0x524 with crng_init=0
>>> percpu: Embedded 15 pages/cpu s30144 r8192 d23104 u61440
>>> Built 1 zonelists, mobility grouping on.  Total pages: 65024
>>> Kernel command line:  boot=network boot_ver=19.3.223.0-0c02c07312-dirty console=ttyS0,115200 ubi.mtd=3 root=/dev/ram0 rd_start=0x81000000 rd_size=0x1023000
>>> Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
>>> Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
>>> Writing ErrCtl register=00020513
>>> Readback ErrCtl register=00020513
>>> Memory: 233756K/262144K available (7216K kernel code, 247K rwdata, 1320K rodata, 292K init, 230K bss, 28388K reserved, 0K cma-reserved)
>>> SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
>>> rcu: Hierarchical RCU implementation.
>>> rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
>>> NR_IRQS: 256
>>> clocksource: GIC: mask: 0xffffffffffffffff max_cycles: 0xcaf478abb4, max_idle_ns: 440795247997 ns
>>> sched_clock: 32 bits at 100 Hz, resolution 10000000ns, wraps every 21474836475000000ns
>>> Calibrating delay loop... 586.13 BogoMIPS (lpj=2930688)
>>> pid_max: default: 32768 minimum: 301
>>> Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
>>> Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
>>> rcu: Hierarchical SRCU implementation.
>>> smp: Bringing up secondary CPUs ...
>>> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
>>> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
>>> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
>>> CPU1 revision is: 0001992f (MIPS 1004Kc)
>>> Synchronize counters for CPU 1: done.
>>> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
>>> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
>>> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
>>> CPU2 revision is: 0001992f (MIPS 1004Kc)
>>> Synchronize counters for CPU 2: done.
>>> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
>>> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
>>> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
>>> CPU3 revision is: 0001992f (MIPS 1004Kc)
>>> Synchronize counters for CPU 3: done.
>>> smp: Brought up 1 node, 4 CPUs
>>> devtmpfs: initialized
>>> clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
>>> futex hash table entries: 1024 (order: 3, 32768 bytes)
>>> pinctrl core: initialized pinctrl subsystem
>>> NET: Registered protocol family 16
>>> mt7621-pci 1e140000.pcie: failed to get gpio perst
>>> mt7621-pci 1e140000.pcie: Parsing DT failed
>>> SCSI subsystem initialized
>>> usbcore: registered new interface driver usbfs
>>> usbcore: registered new interface driver hub
>>> usbcore: registered new device driver usb
>>> pcf857x 0-0026: probed
>>> i2c-mt7621 1e000900.i2c: clock 400KHz, re-start not support
>>> clocksource: Switched to clocksource GIC
>>> NET: Registered protocol family 2
>>> tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes)
>>> TCP established hash table entries: 2048 (order: 1, 8192 bytes)
>>> TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
>>> TCP: Hash tables configured (established 2048 bind 2048)
>>> UDP hash table entries: 256 (order: 1, 8192 bytes)
>>> UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
>>> NET: Registered protocol family 1
>>> Trying to unpack rootfs image as initramfs...
>>> rootfs image is not initramfs (invalid magic at start of compressed archive); looks like an initrd
>>> Freeing initrd memory: 16524K
>>> Initialise system trusted keyrings
>>> workingset: timestamp_bits=30 max_order=16 bucket_order=0
>>> squashfs: version 4.0 (2009/01/31) Phillip Lougher
>>> random: fast init done
>>> Key type asymmetric registered
>>> Asymmetric key parser 'x509' registered
>>> Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
>>> mt7621_gpio 1e000600.gpio: registering 32 gpios
>>> mt7621_gpio 1e000600.gpio: registering 32 gpios
>>> mt7621_gpio 1e000600.gpio: registering 32 gpios
>>> Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
>>> printk: console [ttyS0] disabled
>>> 1e000c00.uartlite: ttyS0 at MMIO 0x1e000c00 (irq = 18, base_baud = 3125000) is a 16550A
>>> printk: console [ttyS0] enabled
>>> printk: console [ttyS0] enabled
>>> printk: bootconsole [early0] disabled
>>> printk: bootconsole [early0] disabled
>>> 1e000d00.uartlite: ttyS1 at MMIO 0x1e000d00 (irq = 19, base_baud = 3125000) is a 16550A
>>> ledman: Copyright (C) SnapGear, 2000-2010.
>>> ledman: registered ERASE switch on IRQ28
>>> snapdog: HW/SW watchdog timer for SnapGear/Others
>>> cacheinfo: Failed to find cpu0 device node
>>> cacheinfo: Unable to detect cache hierarchy for CPU 0
>>> brd: module loaded
>>> mt7621-nand: NAND register bank at 0xbe003000
>>> mt7621-nand: NAND ECC register bank at 0xbe003800
>>> nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xdc
>>> nand: Micron NAND 512MiB 3,3V 8-bit
>>> nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
>>> nand: ecc bit: 4, spare_per_sector: 16
>>> Bad block table found at page 262080, version 0x01
>>> Bad block table found at page 262016, version 0x01
>>> 5 fixed-partitions partitions found on MTD device MT7621-NAND
>>> Creating 5 MTD partitions on "MT7621-NAND":
>>> 0x000000000000-0x000000200000 : "u-boot"
>>> 0x000000200000-0x000000300000 : "u-boot-env"
>>> 0x000000300000-0x000000500000 : "log"
>>> 0x000000500000-0x000020000000 : "flash"
>>> 0x000000000000-0x000020000000 : "all"
>>> libphy: Fixed MDIO Bus: probed
>>> tun: Universal TUN/TAP device driver, 1.6
>>> libphy: mdio: probed
>>> mtk_soc_eth 1e100000.ethernet: generated random MAC address 8a:8c:8b:24:60:92
>>> mtk_soc_eth 1e100000.ethernet: connected mac 0 to PHY at fixed-0:00 [uid=00000000, driver=Generic PHY]
>>> mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
>>> PPP generic driver version 2.4.2
>>> PPP BSD Compression module registered
>>> PPP Deflate Compression module registered
>>> PPP MPPE Compression module registered
>>> NET: Registered protocol family 24
>>> SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
>>> CSLIP: code copyright 1989 Regents of the University of California.
>>> usbcore: registered new interface driver ar5523
>>> usbcore: registered new interface driver asix
>>> usbcore: registered new interface driver ax88179_178a
>>> usbcore: registered new interface driver cdc_ether
>>> usbcore: registered new interface driver cdc_eem
>>> usbcore: registered new interface driver rndis_host
>>> usbcore: registered new interface driver cdc_subset
>>> usbcore: registered new interface driver cdc_ncm
>>> usbcore: registered new interface driver qmi_wwan
>>> usbcore: registered new interface driver cdc_mbim
>>> ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
>>> ehci-pci: EHCI PCI platform driver
>>> ehci-platform: EHCI generic platform driver
>>> ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
>>> ohci-pci: OHCI PCI platform driver
>>> xhci-mtk 1e1c0000.xhci: xHCI Host Controller
>>> xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 1
>>> xhci-mtk 1e1c0000.xhci: hcc params 0x01401198 hci version 0x96 quirks 0x0000000000210010
>>> xhci-mtk 1e1c0000.xhci: irq 20, io mem 0x1e1c0000
>>> usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.01
>>> usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
>>> usb usb1: Product: xHCI Host Controller
>>> usb usb1: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
>>> usb usb1: SerialNumber: 1e1c0000.xhci
>>> hub 1-0:1.0: USB hub found
>>> hub 1-0:1.0: 2 ports detected
>>> xhci-mtk 1e1c0000.xhci: xHCI Host Controller
>>> xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 2
>>> xhci-mtk 1e1c0000.xhci: Host supports USB 3.0  SuperSpeed
>>> usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
>>> usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.01
>>> usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
>>> usb usb2: Product: xHCI Host Controller
>>> usb usb2: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
>>> usb usb2: SerialNumber: 1e1c0000.xhci
>>> hub 2-0:1.0: USB hub found
>>> hub 2-0:1.0: 1 port detected
>>> usbcore: registered new interface driver cdc_acm
>>> cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
>>> usbcore: registered new interface driver cdc_wdm
>>> usbcore: registered new interface driver usb-storage
>>> usbcore: registered new interface driver usbserial_generic
>>> usbserial: USB Serial support registered for generic
>>> usbcore: registered new interface driver ipw
>>> usbserial: USB Serial support registered for IPWireless converter
>>> usbcore: registered new interface driver option
>>> usbserial: USB Serial support registered for GSM modem (1-port)
>>> usbcore: registered new interface driver qcaux
>>> usbserial: USB Serial support registered for qcaux
>>> usbcore: registered new interface driver qcserial
>>> usbserial: USB Serial support registered for Qualcomm USB modem
>>> usbcore: registered new interface driver quatech2
>>> usbserial: USB Serial support registered for Quatech 2nd gen USB to Serial Driver
>>> usbcore: registered new interface driver usb_serial_simple
>>> usbserial: USB Serial support registered for carelink
>>> usbserial: USB Serial support registered for zio
>>> usbserial: USB Serial support registered for funsoft
>>> usbserial: USB Serial support registered for flashloader
>>> usbserial: USB Serial support registered for google
>>> usbserial: USB Serial support registered for libtransistor
>>> usbserial: USB Serial support registered for vivopay
>>> usbserial: USB Serial support registered for moto_modem
>>> usbserial: USB Serial support registered for motorola_tetra
>>> usbserial: USB Serial support registered for novatel_gps
>>> usbserial: USB Serial support registered for hp4x
>>> usbserial: USB Serial support registered for suunto
>>> usbserial: USB Serial support registered for siemens_mpi
>>> u32 classifier
>>>       input device check on
>>> ipip: IPv4 and MPLS over IPv4 tunneling driver
>>> gre: GRE over IPv4 demultiplexor driver
>>> ip_gre: GRE over IPv4 tunneling driver
>>> Initializing XFRM netlink socket
>>> NET: Registered protocol family 10
>>> Segment Routing with IPv6
>>> sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
>>> NET: Registered protocol family 17
>>> NET: Registered protocol family 15
>>> 8021q: 802.1Q VLAN Support v1.8
>>> Loading compiled-in X.509 certificates
>>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>> mt7621-pci 1e140000.pcie: Port 0 N_FTS = 1b102800
>>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>> mt7621-pci 1e140000.pcie: Port 1 N_FTS = 1b102800
>>> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>> mt7621-pci 1e140000.pcie: Port 2 N_FTS = 1b102800
>>> mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>>> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> 
> I changed also the way port resets are done. Now all of ports are
> being reset before
> enabling the phy. Also, power_off the phy is done before reseting the
> port so it should
> not hang now. The real problem here is if the pcie link is properly
> detected to get a working
> PCI.
> 
> Patch attached.

Tested with this patch. I still see the same occasional hang behavior.

Sometimes boot up and scan work, sometimes it hangs.
It is probably around 1 in 3 boots is good, 2 out of 3 hang
(roughly speaking).

I won't have much time today or tomorrow to dig deeper.
But I will try and do some more proper debugging early next week.

Regards
Greg


_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

[-- Attachment #1: Type: text/plain, Size: 23915 bytes --]

Hi Greg,


On Thu, Jun 6, 2019 at 5:06 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Sergio,
>
> On 5/6/19 10:28 pm, Sergio Paracuellos wrote:
> > On Wed, Jun 5, 2019 at 7:47 AM Sergio Paracuellos
> > <sergio.paracuellos@gmail.com> wrote:
> >> On Wed, Jun 5, 2019 at 6:01 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>> On 4/6/19 7:36 pm, Sergio Paracuellos wrote:
> >>>> On Tue, Jun 4, 2019 at 9:14 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>> On 4/6/19 3:06 pm, Sergio Paracuellos wrote:
> >>>>>> On Tue, Jun 4, 2019 at 3:31 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>>> On 4/6/19 5:59 am, Sergio Paracuellos wrote:
> >>>>>>>> On Mon, Jun 3, 2019 at 2:32 PM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>>>>> On 3/6/19 3:34 pm, Sergio Paracuellos wrote:
> >>>>>>>>>> On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>>>>>>> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
> >>>>>>>>>>>> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
> >>>>>>>>>>>>> On 30/5/19 10:44 am, Greg Ungerer wrote:
> >>>>>>>>>>>>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
> >>>>>>>>>>>>>> [snip]
> >>>>>>>>>> Can you try to read and set BIT(10) instead of write it directly?:
> >>>>>>>>>>
> >>>>>>>>>> Instead of:
> >>>>>>>>>>
> >>>>>>>>>> rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);
> >>>>>>>>>
> >>>>>>>>> Oh, yeah, that is definitely not going to work. There is a bunch of
> >>>>>>>>> other GPIO control bits in there for other hardware blocks. Would
> >>>>>>>>> explain the broken NAND flash behavior...
> >>>>>>>>
> >>>>>>>> Yes, my bad here. Sometimes is better to go to sleep :)).
> >>>>>>>>>
> >>>>>>>>>> Do:
> >>>>>>>>>>
> >>>>>>>>>> u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
> >>>>>>>>>> val |= PERST_MODE_GPIO;
> >>>>>>>>>> rt_sysc_w32(val, MT7621_GPIO_MODE);
> >>>>>>>>>
> >>>>>>>>> Much better result with that. Though ultimately it should clear
> >>>>>>>>> bits 10 and 11 (both are PERST_MODE bits) and then OR in BIT(10).
> >>>>>>>>
> >>>>>>>> Ok, so the following should do the trick:
> >>>>>>>>
> >>>>>>>> rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
> >>>>>>>>
> >>>>>>>> with PERST_MODE_MASK defined as:
> >>>>>>>>
> >>>>>>>> #define PERST_MODE_MASK         GENMASK(11, 10)
> >>>>>>>>
> >>>>>>>> (patch attached with this changes)
> >>>>>>>
> >>>>>>> I have mostly good news and a little bad news :-)
> >>>>>>>
> >>>>>>> I should have tested more thoroughly last night. Anyway, the new patch
> >>>>>>> works, even the IRQ of the PCI device. My Wifi PCI device works just
> >>>>>>> as well now as it did with the 4.20 pci-mt7621.c/pci-mt7621-phy.c.
> >>>>>>> I still get the lines:
> >>>>>>>
> >>>>>>> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
> >>>>>>> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
> >>>>>>>
> >>>>>>> However that is referring to the root host PCI device (0000:00:00.0),
> >>>>>>> not my PCI peripheral device (which is 0000:01:00.0). It is actually
> >>>>>>> probed and working properly.
> >>>>>>>
> >>>>>>> That is the good news.
> >>>>>>
> >>>>>> That makes sense :). Good news are always good news even bads are
> >>>>>> coming also :))
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> It would be also good to know what happen if you comment the following lines:
> >>>>>>>>
> >>>>>>>> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
> >>>>>>>> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
> >>>>>>>>
> >>>>>>>> Are they really needed?
> >>>>>>>
> >>>>>>> Had no effect for me. Everything worked the same with or without
> >>>>>>> those lines as far as I could tell.
> >>>>>>
> >>>>>> Ok, I won't include them when I send a clean patch series with all of
> >>>>>> this changes.
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> [snip]
> >>>>>>>> Only one step more to get this properly working.
> >>>>>>>
> >>>>>>> Ok, now the bad news.
> >>>>>>>
> >>>>>>> I often get the boot hanging at various points in the PCI
> >>>>>>> initialization, setup and probing.
> >>>>>>>
> >>>>>>> For example sometimes it hangs with boot up to:
> >>>>>>>
> >>>>>>>       mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>>>>
> >>>>>>>
> >>>>>>> Then sometimes it hangs at:
> >>>>>>>
> >>>>>>>       mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>>>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>>>>>       mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>>>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>>>>>       mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>>>>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>>>>>       mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> >>>>>>>       mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>>>>>       mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>>>>>
> >>>>>>
> >>>>>> It is weird, here it is not initializing any port... All of them are disabled.
> >>>>>>
> >>>>>>>
> >>>>>>> And then I also see it hang up to:
> >>>>>>>
> >>>>>>>       mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>>>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> >>>>>>>       mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>>>>>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> >>>>>>>       mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>>>>>>       mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> >>>>>>>       mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>>>>>>       mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >>>>>>>       mt7621-pci 1e140000.pcie: PCIE0 enabled
> >>>>>>>       mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> >>>>>>>       mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> >>>>>>>       pci_bus 0000:00: root bus resource [io  0xffffffff]
> >>>>>>>       pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> >>>>>>>       pci_bus 0000:00: root bus resource [bus 00-ff]
> >>>>>>>       pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> >>>>>>>
> >>>>>>>
> >>>>>>> It happens on cold or warm boots. Sometimes it boots up all the
> >>>>>>> way and everything works perfectly.
> >>>>>>
> >>>>>> Can you try to change all the msleeps of the code driver in favour of
> >>>>>> mdelay's? Looks like
> >>>>>> some timing problem.
> >>>>>
> >>>>> Changed the msleeps to mdelays and it still hangs somtimes.
> >>>>> I doubled the delay times in the msleeps and tried that too -
> >>>>> that also sometimes hangs.
> >>>>>
> >>>>>
> >>>>>> If it doesn't work, it would be great to have a full trace of working
> >>>>>> 4.20 and no working current 5.x series
> >>>>>> version just to carefully compare them.
> >>>>>
> >>>>> I'll attach at the end a full working boot trace from 5.1 with the 4.20 pci-mt7621.c.
> >>>>> I'll attach a 5.1 boot trace with your patch and hang below. Not that this very
> >>>>> same image doesn't always hang. Same binary will sometimes boot fine.
> >>>>>
> >>>>>
> >>>>>>> I had perfect reliable boot and operation with linux-5.1 using the
> >>>>>>> older 4.20 pci-mt7621.c and pci-mt7621-phy.c.
> >>>>>>
> >>>>>> AFAICT, there is not pci-mt7621-phy.c in the v4.20, isn't it? Do you
> >>>>>> mean you put also that
> >>>>>> code into that version and compile them? Or are you just using "pci-mt7621.c"?
> >>>>>
> >>>>> I just leave that pci-mt7621-phy.c code in place in linux-5.1 when I copy in
> >>>>> the linux-4.20 pci-mt7621.c - even though it isn't used in that case.
> >>>>>
> >>>>> One thing I always notice is that the PCI probing happens much earlier
> >>>>> with the 4.20 pci-mt7621.c. Not sure that will have any impact though.
> >>>>>
> >>>>
> >>>> It seems it is related because, looking at the trace which hangs I can see:
> >>>>
> >>>> mt7621-pci 1e140000.pcie: failed to get gpio perst
> >>>> mt7621-pci 1e140000.pcie: Parsing DT failed
> >>>>
> >>>> At that point gpio's driver are not being probed yet. This not happen
> >>>> with the original
> >>>> patch because it just access to memory to set up all the stuff intead
> >>>> of use the gpio's
> >>>> consumers apis which, IMHO, is the way to go.
> >>>>
> >>>> Can you try to return "-EPROBE_DEFER" if getting the gpio fails in
> >>>> initialization?
> >>>>
> >>>> (patch attached with this change and removing the two lines which
> >>>> seems to not be needed
> >>>> commented in previous patch)
> >>>
> >>> New patch tried. Behavior is no different. Occasional hang during boot.
> >>> I can see that it does the probe, and now deferr. Full boot log of a
> >>> hung boot below.
> >>
> >> Ok, so that means it was properly being deferred with the previous patch also.
> >>
> >> The original code set up all at a very early stage (arch_initcall)
> >> just after the pinctrl
> >> driver is being set up. The new one without my last patches was also
> >> being executed
> >> later because of the phy. At arch_initcall the 'phy_create()' is
> >> called before the phy module is initialized, so 'phy_class' is NULL,
> >> the new phy isn't placed in the right class, and it cannot be found.
> >> So the phy driver uses 'module_init' instead of 'arch_initcall' and it
> >> worked for the gnubee board. But according to your traces when it
> >> hangs the problem seems to be with gpio's being resetting or something
> >> so the pcie link is not in up
> >> state for any reason I cannot understand. So we have to determine if
> >> the problem is because of the phy and gpio init being done in later
> >> stages.
> >>
> >> We can try to change 'arch_initcall' to a later stage changing it to
> >> 'module_init' to see if it is a deferring problem or something. The
> >> other thing we can try to determine if it is a gpio access problem is
> >> to change in the last patch
> >> the gpio descriptor uses with you initial memory accessign hack to see
> >> if it eventually hungs or not. If this both tests result in eventually
> >> hung boot I'll try to prepare a patch with the current phy code in the
> >> 'arch_initcall' stage putting in the the same pci-mt7621.c file to see
> >> if order matters. No more ideas by now, sorry :(
> >>
> >>> Regards
> >>> Greg
> >>
> >> Best regards,
> >>      Sergio Paracuellos
> >>>
> >>>
> >>> --------------------------------------------------------------------------------
> >>> Linux version 5.1.0-ac0 (gerg@goober) (gcc version 5.4.0 (GCC)) #78 SMP Wed Jun 5 13:45:19 AEST 2019
> >>> SoC Type: MediaTek MT7621 ver:1 eco:3
> >>> printk: bootconsole [early0] enabled
> >>> CPU0 revision is: 0001992f (MIPS 1004Kc)
> >>> OF: fdt: No chosen node found, continuing without
> >>> MIPS: machine is Digi EX15
> >>> Determined physical RAM map:
> >>>    memory: 10000000 @ 00000000 (usable)
> >>> Initial ramdisk at: 0x81000000 (16920576 bytes)
> >>> VPE topology {2,2} total 4
> >>> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> >>> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> >>> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> >>> Zone ranges:
> >>>     Normal   [mem 0x0000000000000000-0x000000000fffffff]
> >>> Movable zone start for each node
> >>> Early memory node ranges
> >>>     node   0: [mem 0x0000000000000000-0x000000000fffffff]
> >>> Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
> >>> random: get_random_bytes called from start_kernel+0xb0/0x524 with crng_init=0
> >>> percpu: Embedded 15 pages/cpu s30144 r8192 d23104 u61440
> >>> Built 1 zonelists, mobility grouping on.  Total pages: 65024
> >>> Kernel command line:  boot=network boot_ver=19.3.223.0-0c02c07312-dirty console=ttyS0,115200 ubi.mtd=3 root=/dev/ram0 rd_start=0x81000000 rd_size=0x1023000
> >>> Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
> >>> Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
> >>> Writing ErrCtl register=00020513
> >>> Readback ErrCtl register=00020513
> >>> Memory: 233756K/262144K available (7216K kernel code, 247K rwdata, 1320K rodata, 292K init, 230K bss, 28388K reserved, 0K cma-reserved)
> >>> SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
> >>> rcu: Hierarchical RCU implementation.
> >>> rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
> >>> NR_IRQS: 256
> >>> clocksource: GIC: mask: 0xffffffffffffffff max_cycles: 0xcaf478abb4, max_idle_ns: 440795247997 ns
> >>> sched_clock: 32 bits at 100 Hz, resolution 10000000ns, wraps every 21474836475000000ns
> >>> Calibrating delay loop... 586.13 BogoMIPS (lpj=2930688)
> >>> pid_max: default: 32768 minimum: 301
> >>> Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
> >>> Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
> >>> rcu: Hierarchical SRCU implementation.
> >>> smp: Bringing up secondary CPUs ...
> >>> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> >>> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> >>> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> >>> CPU1 revision is: 0001992f (MIPS 1004Kc)
> >>> Synchronize counters for CPU 1: done.
> >>> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> >>> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> >>> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> >>> CPU2 revision is: 0001992f (MIPS 1004Kc)
> >>> Synchronize counters for CPU 2: done.
> >>> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
> >>> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
> >>> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
> >>> CPU3 revision is: 0001992f (MIPS 1004Kc)
> >>> Synchronize counters for CPU 3: done.
> >>> smp: Brought up 1 node, 4 CPUs
> >>> devtmpfs: initialized
> >>> clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> >>> futex hash table entries: 1024 (order: 3, 32768 bytes)
> >>> pinctrl core: initialized pinctrl subsystem
> >>> NET: Registered protocol family 16
> >>> mt7621-pci 1e140000.pcie: failed to get gpio perst
> >>> mt7621-pci 1e140000.pcie: Parsing DT failed
> >>> SCSI subsystem initialized
> >>> usbcore: registered new interface driver usbfs
> >>> usbcore: registered new interface driver hub
> >>> usbcore: registered new device driver usb
> >>> pcf857x 0-0026: probed
> >>> i2c-mt7621 1e000900.i2c: clock 400KHz, re-start not support
> >>> clocksource: Switched to clocksource GIC
> >>> NET: Registered protocol family 2
> >>> tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes)
> >>> TCP established hash table entries: 2048 (order: 1, 8192 bytes)
> >>> TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
> >>> TCP: Hash tables configured (established 2048 bind 2048)
> >>> UDP hash table entries: 256 (order: 1, 8192 bytes)
> >>> UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
> >>> NET: Registered protocol family 1
> >>> Trying to unpack rootfs image as initramfs...
> >>> rootfs image is not initramfs (invalid magic at start of compressed archive); looks like an initrd
> >>> Freeing initrd memory: 16524K
> >>> Initialise system trusted keyrings
> >>> workingset: timestamp_bits=30 max_order=16 bucket_order=0
> >>> squashfs: version 4.0 (2009/01/31) Phillip Lougher
> >>> random: fast init done
> >>> Key type asymmetric registered
> >>> Asymmetric key parser 'x509' registered
> >>> Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
> >>> mt7621_gpio 1e000600.gpio: registering 32 gpios
> >>> mt7621_gpio 1e000600.gpio: registering 32 gpios
> >>> mt7621_gpio 1e000600.gpio: registering 32 gpios
> >>> Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
> >>> printk: console [ttyS0] disabled
> >>> 1e000c00.uartlite: ttyS0 at MMIO 0x1e000c00 (irq = 18, base_baud = 3125000) is a 16550A
> >>> printk: console [ttyS0] enabled
> >>> printk: console [ttyS0] enabled
> >>> printk: bootconsole [early0] disabled
> >>> printk: bootconsole [early0] disabled
> >>> 1e000d00.uartlite: ttyS1 at MMIO 0x1e000d00 (irq = 19, base_baud = 3125000) is a 16550A
> >>> ledman: Copyright (C) SnapGear, 2000-2010.
> >>> ledman: registered ERASE switch on IRQ28
> >>> snapdog: HW/SW watchdog timer for SnapGear/Others
> >>> cacheinfo: Failed to find cpu0 device node
> >>> cacheinfo: Unable to detect cache hierarchy for CPU 0
> >>> brd: module loaded
> >>> mt7621-nand: NAND register bank at 0xbe003000
> >>> mt7621-nand: NAND ECC register bank at 0xbe003800
> >>> nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xdc
> >>> nand: Micron NAND 512MiB 3,3V 8-bit
> >>> nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
> >>> nand: ecc bit: 4, spare_per_sector: 16
> >>> Bad block table found at page 262080, version 0x01
> >>> Bad block table found at page 262016, version 0x01
> >>> 5 fixed-partitions partitions found on MTD device MT7621-NAND
> >>> Creating 5 MTD partitions on "MT7621-NAND":
> >>> 0x000000000000-0x000000200000 : "u-boot"
> >>> 0x000000200000-0x000000300000 : "u-boot-env"
> >>> 0x000000300000-0x000000500000 : "log"
> >>> 0x000000500000-0x000020000000 : "flash"
> >>> 0x000000000000-0x000020000000 : "all"
> >>> libphy: Fixed MDIO Bus: probed
> >>> tun: Universal TUN/TAP device driver, 1.6
> >>> libphy: mdio: probed
> >>> mtk_soc_eth 1e100000.ethernet: generated random MAC address 8a:8c:8b:24:60:92
> >>> mtk_soc_eth 1e100000.ethernet: connected mac 0 to PHY at fixed-0:00 [uid=00000000, driver=Generic PHY]
> >>> mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
> >>> PPP generic driver version 2.4.2
> >>> PPP BSD Compression module registered
> >>> PPP Deflate Compression module registered
> >>> PPP MPPE Compression module registered
> >>> NET: Registered protocol family 24
> >>> SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
> >>> CSLIP: code copyright 1989 Regents of the University of California.
> >>> usbcore: registered new interface driver ar5523
> >>> usbcore: registered new interface driver asix
> >>> usbcore: registered new interface driver ax88179_178a
> >>> usbcore: registered new interface driver cdc_ether
> >>> usbcore: registered new interface driver cdc_eem
> >>> usbcore: registered new interface driver rndis_host
> >>> usbcore: registered new interface driver cdc_subset
> >>> usbcore: registered new interface driver cdc_ncm
> >>> usbcore: registered new interface driver qmi_wwan
> >>> usbcore: registered new interface driver cdc_mbim
> >>> ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> >>> ehci-pci: EHCI PCI platform driver
> >>> ehci-platform: EHCI generic platform driver
> >>> ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
> >>> ohci-pci: OHCI PCI platform driver
> >>> xhci-mtk 1e1c0000.xhci: xHCI Host Controller
> >>> xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 1
> >>> xhci-mtk 1e1c0000.xhci: hcc params 0x01401198 hci version 0x96 quirks 0x0000000000210010
> >>> xhci-mtk 1e1c0000.xhci: irq 20, io mem 0x1e1c0000
> >>> usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.01
> >>> usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> >>> usb usb1: Product: xHCI Host Controller
> >>> usb usb1: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
> >>> usb usb1: SerialNumber: 1e1c0000.xhci
> >>> hub 1-0:1.0: USB hub found
> >>> hub 1-0:1.0: 2 ports detected
> >>> xhci-mtk 1e1c0000.xhci: xHCI Host Controller
> >>> xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 2
> >>> xhci-mtk 1e1c0000.xhci: Host supports USB 3.0  SuperSpeed
> >>> usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
> >>> usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.01
> >>> usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> >>> usb usb2: Product: xHCI Host Controller
> >>> usb usb2: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
> >>> usb usb2: SerialNumber: 1e1c0000.xhci
> >>> hub 2-0:1.0: USB hub found
> >>> hub 2-0:1.0: 1 port detected
> >>> usbcore: registered new interface driver cdc_acm
> >>> cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
> >>> usbcore: registered new interface driver cdc_wdm
> >>> usbcore: registered new interface driver usb-storage
> >>> usbcore: registered new interface driver usbserial_generic
> >>> usbserial: USB Serial support registered for generic
> >>> usbcore: registered new interface driver ipw
> >>> usbserial: USB Serial support registered for IPWireless converter
> >>> usbcore: registered new interface driver option
> >>> usbserial: USB Serial support registered for GSM modem (1-port)
> >>> usbcore: registered new interface driver qcaux
> >>> usbserial: USB Serial support registered for qcaux
> >>> usbcore: registered new interface driver qcserial
> >>> usbserial: USB Serial support registered for Qualcomm USB modem
> >>> usbcore: registered new interface driver quatech2
> >>> usbserial: USB Serial support registered for Quatech 2nd gen USB to Serial Driver
> >>> usbcore: registered new interface driver usb_serial_simple
> >>> usbserial: USB Serial support registered for carelink
> >>> usbserial: USB Serial support registered for zio
> >>> usbserial: USB Serial support registered for funsoft
> >>> usbserial: USB Serial support registered for flashloader
> >>> usbserial: USB Serial support registered for google
> >>> usbserial: USB Serial support registered for libtransistor
> >>> usbserial: USB Serial support registered for vivopay
> >>> usbserial: USB Serial support registered for moto_modem
> >>> usbserial: USB Serial support registered for motorola_tetra
> >>> usbserial: USB Serial support registered for novatel_gps
> >>> usbserial: USB Serial support registered for hp4x
> >>> usbserial: USB Serial support registered for suunto
> >>> usbserial: USB Serial support registered for siemens_mpi
> >>> u32 classifier
> >>>       input device check on
> >>> ipip: IPv4 and MPLS over IPv4 tunneling driver
> >>> gre: GRE over IPv4 demultiplexor driver
> >>> ip_gre: GRE over IPv4 tunneling driver
> >>> Initializing XFRM netlink socket
> >>> NET: Registered protocol family 10
> >>> Segment Routing with IPv6
> >>> sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
> >>> NET: Registered protocol family 17
> >>> NET: Registered protocol family 15
> >>> 8021q: 802.1Q VLAN Support v1.8
> >>> Loading compiled-in X.509 certificates
> >>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>> mt7621-pci 1e140000.pcie: Port 0 N_FTS = 1b102800
> >>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> >>> mt7621-pci 1e140000.pcie: Port 1 N_FTS = 1b102800
> >>> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> >>> mt7621-pci 1e140000.pcie: Port 2 N_FTS = 1b102800
> >>> mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> >>> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
> >>> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
> >
> > I changed also the way port resets are done. Now all of ports are
> > being reset before
> > enabling the phy. Also, power_off the phy is done before reseting the
> > port so it should
> > not hang now. The real problem here is if the pcie link is properly
> > detected to get a working
> > PCI.
> >
> > Patch attached.
>
> Tested with this patch. I still see the same occasional hang behavior.
>
> Sometimes boot up and scan work, sometimes it hangs.
> It is probably around 1 in 3 boots is good, 2 out of 3 hang
> (roughly speaking).

No good numbers for us, yet :)

>
> I won't have much time today or tomorrow to dig deeper.
> But I will try and do some more proper debugging early next week.

Just to be sure that the hang is because of powering off the phy on
fail I directly unset clock
bits for each port in the attached patch.

Debugging would be very helpful.

Thanks.

>
> Regards
> Greg
>

Best regards,
    Sergio Paracuellos
>

[-- Attachment #2: 0001-staging-mt7621-pci-use-perst-gpio-instead-of-builtin.patch --]
[-- Type: text/x-patch, Size: 8984 bytes --]

From 2e4fda85e64f282c5ac934ab3cbaf1eb5df74e81 Mon Sep 17 00:00:00 2001
From: Sergio Paracuellos <sparacuellos@ikergune.com>
Date: Wed, 29 May 2019 09:58:07 +0200
Subject: [PATCH] staging: mt7621-pci: use perst gpio instead of builtin perst

Some boards need gpio instead of builtin perst. Use gpio for all
of them which was the approach of the original code.

Signed-off-by: Sergio Paracuellos <sparacuellos@ikergune.com>
---
 drivers/staging/mt7621-dts/mt7621.dtsi  |   3 +-
 drivers/staging/mt7621-pci/pci-mt7621.c | 131 ++++++++++++++++----------------
 2 files changed, 69 insertions(+), 65 deletions(-)

diff --git a/drivers/staging/mt7621-dts/mt7621.dtsi b/drivers/staging/mt7621-dts/mt7621.dtsi
index 280ec33..aed2458 100644
--- a/drivers/staging/mt7621-dts/mt7621.dtsi
+++ b/drivers/staging/mt7621-dts/mt7621.dtsi
@@ -1,5 +1,5 @@
 #include <dt-bindings/interrupt-controller/mips-gic.h>
-
+#include <dt-bindings/gpio/gpio.h>
 / {
 	#address-cells = <1>;
 	#size-cells = <1>;
@@ -468,6 +468,7 @@
 		#address-cells = <3>;
 		#size-cells = <2>;
 
+		perst-gpio = <&gpio 19 GPIO_ACTIVE_HIGH>;
 		pinctrl-names = "default";
 		pinctrl-0 = <&pcie_pins>;
 
diff --git a/drivers/staging/mt7621-pci/pci-mt7621.c b/drivers/staging/mt7621-pci/pci-mt7621.c
index 03d919a..4972367 100644
--- a/drivers/staging/mt7621-pci/pci-mt7621.c
+++ b/drivers/staging/mt7621-pci/pci-mt7621.c
@@ -17,6 +17,7 @@
 
 #include <linux/bitops.h>
 #include <linux/delay.h>
+#include <linux/gpio/consumer.h>
 #include <linux/iopoll.h>
 #include <linux/module.h>
 #include <linux/of.h>
@@ -35,6 +36,8 @@
 
 /* sysctl */
 #define MT7621_CHIP_REV_ID		0x0c
+#define MT7621_GPIO_MODE        0x60
+#define RALINK_CLKCFG1          0x30
 #define CHIP_REV_MT7621_E2		0x0101
 
 /* MediaTek specific configuration registers */
@@ -81,7 +84,6 @@
 #define PCIE_BAR_ENABLE			BIT(0)
 #define PCIE_PORT_INT_EN(x)		BIT(20 + (x))
 #define PCIE_PORT_CLK_EN(x)		BIT(24 + (x))
-#define PCIE_PORT_PERST(x)		BIT(1 + (x))
 #define PCIE_PORT_LINKUP		BIT(0)
 
 #define PCIE_CLK_GEN_EN			BIT(31)
@@ -90,6 +92,10 @@
 #define PCIE_CLK_GEN1_EN		(BIT(27) | BIT(25))
 #define MEMORY_BASE			0x0
 
+#define PERST_MODE_MASK         GENMASK(11, 10)
+#define PERST_MODE_GPIO         BIT(10)
+#define PERST_DELAY_US          1000
+
 /**
  * struct mt7621_pcie_port - PCIe port information
  * @base: I/O mapped register base
@@ -119,6 +125,7 @@ struct mt7621_pcie_port {
  * @offset: IO / Memory offset
  * @dev: Pointer to PCIe device
  * @ports: pointer to PCIe port information
+ * @perst: gpio reset
  * @rst: pointer to pcie reset
  */
 struct mt7621_pcie {
@@ -132,6 +139,7 @@ struct mt7621_pcie {
 		resource_size_t io;
 	} offset;
 	struct list_head ports;
+    struct gpio_desc *perst;
 	struct reset_control *rst;
 };
 
@@ -198,6 +206,18 @@ static void write_config(struct mt7621_pcie *pcie, unsigned int dev,
 	pcie_write(pcie, val, RALINK_PCI_CONFIG_DATA);
 }
 
+static void mt7621_perst_gpio_pcie_assert(struct mt7621_pcie *pcie)
+{
+    gpiod_set_value(pcie->perst, 0);
+    mdelay(PERST_DELAY_US);
+}
+
+static void mt7621_perst_gpio_pcie_deassert(struct mt7621_pcie *pcie)
+{
+    gpiod_set_value(pcie->perst, 1);
+    mdelay(PERST_DELAY_US);
+}
+
 static inline void mt7621_control_assert(struct mt7621_pcie_port *port)
 {
 	u32 chip_rev_id = rt_sysc_r32(MT7621_CHIP_REV_ID);
@@ -221,7 +241,7 @@ static inline void mt7621_control_deassert(struct mt7621_pcie_port *port)
 static void mt7621_reset_port(struct mt7621_pcie_port *port)
 {
 	mt7621_control_assert(port);
-	msleep(100);
+	mdelay(100);
 	mt7621_control_deassert(port);
 }
 
@@ -344,6 +364,12 @@ static int mt7621_pcie_parse_dt(struct mt7621_pcie *pcie)
 	struct resource regs;
 	int err;
 
+    pcie->perst = devm_gpiod_get(dev, "perst", GPIOD_OUT_HIGH);
+    if (IS_ERR(pcie->perst)) {
+        dev_err(dev, "failed to get gpio perst\n");
+        return -EPROBE_DEFER;
+    }
+
 	err = of_address_to_resource(node, 0, &regs);
 	if (err) {
 		dev_err(dev, "missing \"reg\" property\n");
@@ -384,56 +410,42 @@ static int mt7621_pcie_init_port(struct mt7621_pcie_port *port)
 	struct mt7621_pcie *pcie = port->pcie;
 	struct device *dev = pcie->dev;
 	u32 slot = port->slot;
-	u32 val = 0;
 	int err;
 
-	/*
-	 * Any MT7621 Ralink pcie controller that doesn't have 0x0101 at
-	 * the end of the chip_id has inverted PCI resets.
-	 */
-	mt7621_reset_port(port);
-
-	val = read_config(pcie, slot, PCIE_FTS_NUM);
-	dev_info(dev, "Port %d N_FTS = %x\n", (unsigned int)val, slot);
-
 	err = phy_init(port->phy);
 	if (err) {
 		dev_err(dev, "failed to initialize port%d phy\n", slot);
-		goto err_phy_init;
+		return err;
 	}
 
 	err = phy_power_on(port->phy);
 	if (err) {
 		dev_err(dev, "failed to power on port%d phy\n", slot);
-		goto err_phy_on;
-	}
-
-	if ((pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) == 0) {
-		dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", slot);
-		mt7621_control_assert(port);
-		port->enabled = false;
-		err = -ENODEV;
-		goto err_no_link_up;
+		return err;
 	}
 
 	port->enabled = true;
 
 	return 0;
-
-err_no_link_up:
-	phy_power_off(port->phy);
-err_phy_on:
-	phy_exit(port->phy);
-err_phy_init:
-	return err;
 }
 
 static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 {
 	struct device *dev = pcie->dev;
 	struct mt7621_pcie_port *port, *tmp;
+	u32 val = 0;
 	int err;
 
+	list_for_each_entry(port, &pcie->ports, list)
+        mt7621_control_assert(port);
+
+    rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
+    mdelay(100);
+    mt7621_perst_gpio_pcie_assert(pcie);
+
+	list_for_each_entry(port, &pcie->ports, list)
+        mt7621_control_deassert(port);
+
 	list_for_each_entry_safe(port, tmp, &pcie->ports, list) {
 		u32 slot = port->slot;
 
@@ -441,7 +453,10 @@ static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 		if (err) {
 			dev_err(dev, "Initiating port %d failed\n", slot);
 			list_del(&port->list);
-		}
+		} else {
+	        val = read_config(pcie, slot, PCIE_FTS_NUM);
+	        dev_info(dev, "Port %d N_FTS = %x\n", slot, (unsigned int)val);
+        }
 	}
 
 	reset_control_assert(pcie->rst);
@@ -449,39 +464,33 @@ static void mt7621_pcie_init_ports(struct mt7621_pcie *pcie)
 	rt_sysc_m32(PCIE_CLK_GEN_EN, PCIE_CLK_GEN_DIS, RALINK_PCIE_CLK_GEN);
 	rt_sysc_m32(PCIE_CLK_GEN1_DIS, PCIE_CLK_GEN1_EN, RALINK_PCIE_CLK_GEN1);
 	rt_sysc_m32(PCIE_CLK_GEN_DIS, PCIE_CLK_GEN_EN, RALINK_PCIE_CLK_GEN);
-	msleep(50);
+	mdelay(50);
 	reset_control_deassert(pcie->rst);
+
+    mt7621_perst_gpio_pcie_deassert(pcie);
+
+	list_for_each_entry(port, &pcie->ports, list) {
+		u32 slot = port->slot;
+
+	    if ((pcie_port_read(port, RALINK_PCI_STATUS) & PCIE_PORT_LINKUP) == 0) {
+		    dev_err(dev, "pcie%d no card, disable it (RST & CLK)\n", slot);
+		    mt7621_control_assert(port);
+            rt_sysc_m32(PCIE_PORT_CLK_EN(slot), 0, RALINK_CLKCFG1);
+		    port->enabled = false;
+        } else {
+	        /* enable pcie interrupt */
+	        val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR);
+	        val |= PCIE_PORT_INT_EN(slot);
+	        pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR);
+        }
+	}
 }
 
-static int mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
+static void mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
 {
 	struct mt7621_pcie *pcie = port->pcie;
 	u32 slot = port->slot;
 	u32 offset = MT7621_PCIE_OFFSET + (slot * MT7621_NEXT_PORT);
-	u32 val;
-	int err;
-
-	/* assert port PERST_N */
-	val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
-	val |= PCIE_PORT_PERST(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
-
-	/* de-assert port PERST_N */
-	val = pcie_read(pcie, RALINK_PCI_PCICFG_ADDR);
-	val &= ~PCIE_PORT_PERST(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCICFG_ADDR);
-
-	/* 100ms timeout value should be enough for Gen1 training */
-	err = readl_poll_timeout(port->base + RALINK_PCI_STATUS,
-				 val, !!(val & PCIE_PORT_LINKUP),
-				 20, 100 * USEC_PER_MSEC);
-	if (err)
-		return -ETIMEDOUT;
-
-	/* enable pcie interrupt */
-	val = pcie_read(pcie, RALINK_PCI_PCIMSK_ADDR);
-	val |= PCIE_PORT_INT_EN(slot);
-	pcie_write(pcie, val, RALINK_PCI_PCIMSK_ADDR);
 
 	/* map 2G DDR region */
 	pcie_write(pcie, PCIE_BAR_MAP_MAX | PCIE_BAR_ENABLE,
@@ -492,8 +501,6 @@ static int mt7621_pcie_enable_port(struct mt7621_pcie_port *port)
 	/* configure class code and revision ID */
 	pcie_write(pcie, PCIE_CLASS_CODE | PCIE_REVISION_ID,
 		   offset + RALINK_PCI_CLASS);
-
-	return 0;
 }
 
 static void mt7621_pcie_enable_ports(struct mt7621_pcie *pcie)
@@ -506,12 +513,8 @@ static void mt7621_pcie_enable_ports(struct mt7621_pcie *pcie)
 
 	list_for_each_entry(port, &pcie->ports, list) {
 		if (port->enabled) {
-			if (mt7621_pcie_enable_port(port)) {
-				dev_err(dev, "de-assert port %d PERST_N\n",
-					port->slot);
-				continue;
-			}
-			dev_info(dev, "PCIE%d enabled\n", slot);
+			mt7621_pcie_enable_port(port);
+			dev_info(dev, "PCIE%d enabled\n", port->slot);
 			num_slots_enabled++;
 		}
 	}
-- 
2.7.4


[-- Attachment #3: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Sergio,

On 6/6/19 3:07 pm, Sergio Paracuellos wrote:
> On Thu, Jun 6, 2019 at 5:06 AM Greg Ungerer <gerg@kernel.org> wrote:
>> On 5/6/19 10:28 pm, Sergio Paracuellos wrote:
>>> On Wed, Jun 5, 2019 at 7:47 AM Sergio Paracuellos
>>> <sergio.paracuellos@gmail.com> wrote:
>>>> On Wed, Jun 5, 2019 at 6:01 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>> On 4/6/19 7:36 pm, Sergio Paracuellos wrote:
>>>>>> On Tue, Jun 4, 2019 at 9:14 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>> On 4/6/19 3:06 pm, Sergio Paracuellos wrote:
>>>>>>>> On Tue, Jun 4, 2019 at 3:31 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>>>> On 4/6/19 5:59 am, Sergio Paracuellos wrote:
>>>>>>>>>> On Mon, Jun 3, 2019 at 2:32 PM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>>>>>> On 3/6/19 3:34 pm, Sergio Paracuellos wrote:
>>>>>>>>>>>> On Mon, Jun 3, 2019 at 3:26 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>>>>>>>> On 31/5/19 10:37 pm, Sergio Paracuellos wrote:
>>>>>>>>>>>>>> On Thu, May 30, 2019 at 3:46 AM Greg Ungerer <gerg@kernel.org> wrote:
>>>>>>>>>>>>>>> On 30/5/19 10:44 am, Greg Ungerer wrote:
>>>>>>>>>>>>>>>> On 29/5/19 6:08 pm, Sergio Paracuellos wrote:
>>>>>>>>>>>>>>>> [snip]
>>>>>>>>>>>> Can you try to read and set BIT(10) instead of write it directly?:
>>>>>>>>>>>>
>>>>>>>>>>>> Instead of:
>>>>>>>>>>>>
>>>>>>>>>>>> rt_sysc_w32(PERST_MODE_GPIO, MT7621_GPIO_MODE);
>>>>>>>>>>>
>>>>>>>>>>> Oh, yeah, that is definitely not going to work. There is a bunch of
>>>>>>>>>>> other GPIO control bits in there for other hardware blocks. Would
>>>>>>>>>>> explain the broken NAND flash behavior...
>>>>>>>>>>
>>>>>>>>>> Yes, my bad here. Sometimes is better to go to sleep :)).
>>>>>>>>>>>
>>>>>>>>>>>> Do:
>>>>>>>>>>>>
>>>>>>>>>>>> u32 val = rt_sysc_r32(MT7621_GPIO_MODE);
>>>>>>>>>>>> val |= PERST_MODE_GPIO;
>>>>>>>>>>>> rt_sysc_w32(val, MT7621_GPIO_MODE);
>>>>>>>>>>>
>>>>>>>>>>> Much better result with that. Though ultimately it should clear
>>>>>>>>>>> bits 10 and 11 (both are PERST_MODE bits) and then OR in BIT(10).
>>>>>>>>>>
>>>>>>>>>> Ok, so the following should do the trick:
>>>>>>>>>>
>>>>>>>>>> rt_sysc_m32(PERST_MODE_MASK, PERST_MODE_GPIO, MT7621_GPIO_MODE);
>>>>>>>>>>
>>>>>>>>>> with PERST_MODE_MASK defined as:
>>>>>>>>>>
>>>>>>>>>> #define PERST_MODE_MASK         GENMASK(11, 10)
>>>>>>>>>>
>>>>>>>>>> (patch attached with this changes)
>>>>>>>>>
>>>>>>>>> I have mostly good news and a little bad news :-)
>>>>>>>>>
>>>>>>>>> I should have tested more thoroughly last night. Anyway, the new patch
>>>>>>>>> works, even the IRQ of the PCI device. My Wifi PCI device works just
>>>>>>>>> as well now as it did with the 4.20 pci-mt7621.c/pci-mt7621-phy.c.
>>>>>>>>> I still get the lines:
>>>>>>>>>
>>>>>>>>> pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
>>>>>>>>> pcieport 0000:00:00.0: enabling device (0004 -> 0006)
>>>>>>>>>
>>>>>>>>> However that is referring to the root host PCI device (0000:00:00.0),
>>>>>>>>> not my PCI peripheral device (which is 0000:01:00.0). It is actually
>>>>>>>>> probed and working properly.
>>>>>>>>>
>>>>>>>>> That is the good news.
>>>>>>>>
>>>>>>>> That makes sense :). Good news are always good news even bads are
>>>>>>>> coming also :))
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> It would be also good to know what happen if you comment the following lines:
>>>>>>>>>>
>>>>>>>>>> pcie_write(pcie, 0xffffffff, RALINK_PCI_MEMBASE);
>>>>>>>>>> pcie_write(pcie, RALINK_PCI_IO_MAP_BASE, RALINK_PCI_IOBASE);
>>>>>>>>>>
>>>>>>>>>> Are they really needed?
>>>>>>>>>
>>>>>>>>> Had no effect for me. Everything worked the same with or without
>>>>>>>>> those lines as far as I could tell.
>>>>>>>>
>>>>>>>> Ok, I won't include them when I send a clean patch series with all of
>>>>>>>> this changes.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [snip]
>>>>>>>>>> Only one step more to get this properly working.
>>>>>>>>>
>>>>>>>>> Ok, now the bad news.
>>>>>>>>>
>>>>>>>>> I often get the boot hanging at various points in the PCI
>>>>>>>>> initialization, setup and probing.
>>>>>>>>>
>>>>>>>>> For example sometimes it hangs with boot up to:
>>>>>>>>>
>>>>>>>>>        mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Then sometimes it hangs at:
>>>>>>>>>
>>>>>>>>>        mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>>>>>        mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>>>>>>>        mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>>>>>        mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>>>>>>>        mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>>>>>>        mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>>>>>>>        mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>>>>>>>>>        mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>>>>>>        mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>>>>>>>
>>>>>>>>
>>>>>>>> It is weird, here it is not initializing any port... All of them are disabled.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> And then I also see it hang up to:
>>>>>>>>>
>>>>>>>>>        mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>>>>>        mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
>>>>>>>>>        mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>>>>>>        mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
>>>>>>>>>        mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>>>>>>        mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
>>>>>>>>>        mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>>>>>>        mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>>>>>>>        mt7621-pci 1e140000.pcie: PCIE0 enabled
>>>>>>>>>        mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
>>>>>>>>>        mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
>>>>>>>>>        pci_bus 0000:00: root bus resource [io  0xffffffff]
>>>>>>>>>        pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
>>>>>>>>>        pci_bus 0000:00: root bus resource [bus 00-ff]
>>>>>>>>>        pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It happens on cold or warm boots. Sometimes it boots up all the
>>>>>>>>> way and everything works perfectly.
>>>>>>>>
>>>>>>>> Can you try to change all the msleeps of the code driver in favour of
>>>>>>>> mdelay's? Looks like
>>>>>>>> some timing problem.
>>>>>>>
>>>>>>> Changed the msleeps to mdelays and it still hangs somtimes.
>>>>>>> I doubled the delay times in the msleeps and tried that too -
>>>>>>> that also sometimes hangs.
>>>>>>>
>>>>>>>
>>>>>>>> If it doesn't work, it would be great to have a full trace of working
>>>>>>>> 4.20 and no working current 5.x series
>>>>>>>> version just to carefully compare them.
>>>>>>>
>>>>>>> I'll attach at the end a full working boot trace from 5.1 with the 4.20 pci-mt7621.c.
>>>>>>> I'll attach a 5.1 boot trace with your patch and hang below. Not that this very
>>>>>>> same image doesn't always hang. Same binary will sometimes boot fine.
>>>>>>>
>>>>>>>
>>>>>>>>> I had perfect reliable boot and operation with linux-5.1 using the
>>>>>>>>> older 4.20 pci-mt7621.c and pci-mt7621-phy.c.
>>>>>>>>
>>>>>>>> AFAICT, there is not pci-mt7621-phy.c in the v4.20, isn't it? Do you
>>>>>>>> mean you put also that
>>>>>>>> code into that version and compile them? Or are you just using "pci-mt7621.c"?
>>>>>>>
>>>>>>> I just leave that pci-mt7621-phy.c code in place in linux-5.1 when I copy in
>>>>>>> the linux-4.20 pci-mt7621.c - even though it isn't used in that case.
>>>>>>>
>>>>>>> One thing I always notice is that the PCI probing happens much earlier
>>>>>>> with the 4.20 pci-mt7621.c. Not sure that will have any impact though.
>>>>>>>
>>>>>>
>>>>>> It seems it is related because, looking at the trace which hangs I can see:
>>>>>>
>>>>>> mt7621-pci 1e140000.pcie: failed to get gpio perst
>>>>>> mt7621-pci 1e140000.pcie: Parsing DT failed
>>>>>>
>>>>>> At that point gpio's driver are not being probed yet. This not happen
>>>>>> with the original
>>>>>> patch because it just access to memory to set up all the stuff intead
>>>>>> of use the gpio's
>>>>>> consumers apis which, IMHO, is the way to go.
>>>>>>
>>>>>> Can you try to return "-EPROBE_DEFER" if getting the gpio fails in
>>>>>> initialization?
>>>>>>
>>>>>> (patch attached with this change and removing the two lines which
>>>>>> seems to not be needed
>>>>>> commented in previous patch)
>>>>>
>>>>> New patch tried. Behavior is no different. Occasional hang during boot.
>>>>> I can see that it does the probe, and now deferr. Full boot log of a
>>>>> hung boot below.
>>>>
>>>> Ok, so that means it was properly being deferred with the previous patch also.
>>>>
>>>> The original code set up all at a very early stage (arch_initcall)
>>>> just after the pinctrl
>>>> driver is being set up. The new one without my last patches was also
>>>> being executed
>>>> later because of the phy. At arch_initcall the 'phy_create()' is
>>>> called before the phy module is initialized, so 'phy_class' is NULL,
>>>> the new phy isn't placed in the right class, and it cannot be found.
>>>> So the phy driver uses 'module_init' instead of 'arch_initcall' and it
>>>> worked for the gnubee board. But according to your traces when it
>>>> hangs the problem seems to be with gpio's being resetting or something
>>>> so the pcie link is not in up
>>>> state for any reason I cannot understand. So we have to determine if
>>>> the problem is because of the phy and gpio init being done in later
>>>> stages.
>>>>
>>>> We can try to change 'arch_initcall' to a later stage changing it to
>>>> 'module_init' to see if it is a deferring problem or something. The
>>>> other thing we can try to determine if it is a gpio access problem is
>>>> to change in the last patch
>>>> the gpio descriptor uses with you initial memory accessign hack to see
>>>> if it eventually hungs or not. If this both tests result in eventually
>>>> hung boot I'll try to prepare a patch with the current phy code in the
>>>> 'arch_initcall' stage putting in the the same pci-mt7621.c file to see
>>>> if order matters. No more ideas by now, sorry :(
>>>>
>>>>> Regards
>>>>> Greg
>>>>
>>>> Best regards,
>>>>       Sergio Paracuellos
>>>>>
>>>>>
>>>>> --------------------------------------------------------------------------------
>>>>> Linux version 5.1.0-ac0 (gerg@goober) (gcc version 5.4.0 (GCC)) #78 SMP Wed Jun 5 13:45:19 AEST 2019
>>>>> SoC Type: MediaTek MT7621 ver:1 eco:3
>>>>> printk: bootconsole [early0] enabled
>>>>> CPU0 revision is: 0001992f (MIPS 1004Kc)
>>>>> OF: fdt: No chosen node found, continuing without
>>>>> MIPS: machine is Digi EX15
>>>>> Determined physical RAM map:
>>>>>     memory: 10000000 @ 00000000 (usable)
>>>>> Initial ramdisk at: 0x81000000 (16920576 bytes)
>>>>> VPE topology {2,2} total 4
>>>>> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
>>>>> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
>>>>> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
>>>>> Zone ranges:
>>>>>      Normal   [mem 0x0000000000000000-0x000000000fffffff]
>>>>> Movable zone start for each node
>>>>> Early memory node ranges
>>>>>      node   0: [mem 0x0000000000000000-0x000000000fffffff]
>>>>> Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
>>>>> random: get_random_bytes called from start_kernel+0xb0/0x524 with crng_init=0
>>>>> percpu: Embedded 15 pages/cpu s30144 r8192 d23104 u61440
>>>>> Built 1 zonelists, mobility grouping on.  Total pages: 65024
>>>>> Kernel command line:  boot=network boot_ver=19.3.223.0-0c02c07312-dirty console=ttyS0,115200 ubi.mtd=3 root=/dev/ram0 rd_start=0x81000000 rd_size=0x1023000
>>>>> Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
>>>>> Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
>>>>> Writing ErrCtl register=00020513
>>>>> Readback ErrCtl register=00020513
>>>>> Memory: 233756K/262144K available (7216K kernel code, 247K rwdata, 1320K rodata, 292K init, 230K bss, 28388K reserved, 0K cma-reserved)
>>>>> SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
>>>>> rcu: Hierarchical RCU implementation.
>>>>> rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
>>>>> NR_IRQS: 256
>>>>> clocksource: GIC: mask: 0xffffffffffffffff max_cycles: 0xcaf478abb4, max_idle_ns: 440795247997 ns
>>>>> sched_clock: 32 bits at 100 Hz, resolution 10000000ns, wraps every 21474836475000000ns
>>>>> Calibrating delay loop... 586.13 BogoMIPS (lpj=2930688)
>>>>> pid_max: default: 32768 minimum: 301
>>>>> Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
>>>>> Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
>>>>> rcu: Hierarchical SRCU implementation.
>>>>> smp: Bringing up secondary CPUs ...
>>>>> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
>>>>> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
>>>>> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
>>>>> CPU1 revision is: 0001992f (MIPS 1004Kc)
>>>>> Synchronize counters for CPU 1: done.
>>>>> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
>>>>> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
>>>>> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
>>>>> CPU2 revision is: 0001992f (MIPS 1004Kc)
>>>>> Synchronize counters for CPU 2: done.
>>>>> Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
>>>>> Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
>>>>> MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
>>>>> CPU3 revision is: 0001992f (MIPS 1004Kc)
>>>>> Synchronize counters for CPU 3: done.
>>>>> smp: Brought up 1 node, 4 CPUs
>>>>> devtmpfs: initialized
>>>>> clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
>>>>> futex hash table entries: 1024 (order: 3, 32768 bytes)
>>>>> pinctrl core: initialized pinctrl subsystem
>>>>> NET: Registered protocol family 16
>>>>> mt7621-pci 1e140000.pcie: failed to get gpio perst
>>>>> mt7621-pci 1e140000.pcie: Parsing DT failed
>>>>> SCSI subsystem initialized
>>>>> usbcore: registered new interface driver usbfs
>>>>> usbcore: registered new interface driver hub
>>>>> usbcore: registered new device driver usb
>>>>> pcf857x 0-0026: probed
>>>>> i2c-mt7621 1e000900.i2c: clock 400KHz, re-start not support
>>>>> clocksource: Switched to clocksource GIC
>>>>> NET: Registered protocol family 2
>>>>> tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes)
>>>>> TCP established hash table entries: 2048 (order: 1, 8192 bytes)
>>>>> TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
>>>>> TCP: Hash tables configured (established 2048 bind 2048)
>>>>> UDP hash table entries: 256 (order: 1, 8192 bytes)
>>>>> UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
>>>>> NET: Registered protocol family 1
>>>>> Trying to unpack rootfs image as initramfs...
>>>>> rootfs image is not initramfs (invalid magic at start of compressed archive); looks like an initrd
>>>>> Freeing initrd memory: 16524K
>>>>> Initialise system trusted keyrings
>>>>> workingset: timestamp_bits=30 max_order=16 bucket_order=0
>>>>> squashfs: version 4.0 (2009/01/31) Phillip Lougher
>>>>> random: fast init done
>>>>> Key type asymmetric registered
>>>>> Asymmetric key parser 'x509' registered
>>>>> Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
>>>>> mt7621_gpio 1e000600.gpio: registering 32 gpios
>>>>> mt7621_gpio 1e000600.gpio: registering 32 gpios
>>>>> mt7621_gpio 1e000600.gpio: registering 32 gpios
>>>>> Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
>>>>> printk: console [ttyS0] disabled
>>>>> 1e000c00.uartlite: ttyS0 at MMIO 0x1e000c00 (irq = 18, base_baud = 3125000) is a 16550A
>>>>> printk: console [ttyS0] enabled
>>>>> printk: console [ttyS0] enabled
>>>>> printk: bootconsole [early0] disabled
>>>>> printk: bootconsole [early0] disabled
>>>>> 1e000d00.uartlite: ttyS1 at MMIO 0x1e000d00 (irq = 19, base_baud = 3125000) is a 16550A
>>>>> ledman: Copyright (C) SnapGear, 2000-2010.
>>>>> ledman: registered ERASE switch on IRQ28
>>>>> snapdog: HW/SW watchdog timer for SnapGear/Others
>>>>> cacheinfo: Failed to find cpu0 device node
>>>>> cacheinfo: Unable to detect cache hierarchy for CPU 0
>>>>> brd: module loaded
>>>>> mt7621-nand: NAND register bank at 0xbe003000
>>>>> mt7621-nand: NAND ECC register bank at 0xbe003800
>>>>> nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xdc
>>>>> nand: Micron NAND 512MiB 3,3V 8-bit
>>>>> nand: 512 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
>>>>> nand: ecc bit: 4, spare_per_sector: 16
>>>>> Bad block table found at page 262080, version 0x01
>>>>> Bad block table found at page 262016, version 0x01
>>>>> 5 fixed-partitions partitions found on MTD device MT7621-NAND
>>>>> Creating 5 MTD partitions on "MT7621-NAND":
>>>>> 0x000000000000-0x000000200000 : "u-boot"
>>>>> 0x000000200000-0x000000300000 : "u-boot-env"
>>>>> 0x000000300000-0x000000500000 : "log"
>>>>> 0x000000500000-0x000020000000 : "flash"
>>>>> 0x000000000000-0x000020000000 : "all"
>>>>> libphy: Fixed MDIO Bus: probed
>>>>> tun: Universal TUN/TAP device driver, 1.6
>>>>> libphy: mdio: probed
>>>>> mtk_soc_eth 1e100000.ethernet: generated random MAC address 8a:8c:8b:24:60:92
>>>>> mtk_soc_eth 1e100000.ethernet: connected mac 0 to PHY at fixed-0:00 [uid=00000000, driver=Generic PHY]
>>>>> mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
>>>>> PPP generic driver version 2.4.2
>>>>> PPP BSD Compression module registered
>>>>> PPP Deflate Compression module registered
>>>>> PPP MPPE Compression module registered
>>>>> NET: Registered protocol family 24
>>>>> SLIP: version 0.8.4-NET3.019-NEWTTY (dynamic channels, max=256).
>>>>> CSLIP: code copyright 1989 Regents of the University of California.
>>>>> usbcore: registered new interface driver ar5523
>>>>> usbcore: registered new interface driver asix
>>>>> usbcore: registered new interface driver ax88179_178a
>>>>> usbcore: registered new interface driver cdc_ether
>>>>> usbcore: registered new interface driver cdc_eem
>>>>> usbcore: registered new interface driver rndis_host
>>>>> usbcore: registered new interface driver cdc_subset
>>>>> usbcore: registered new interface driver cdc_ncm
>>>>> usbcore: registered new interface driver qmi_wwan
>>>>> usbcore: registered new interface driver cdc_mbim
>>>>> ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
>>>>> ehci-pci: EHCI PCI platform driver
>>>>> ehci-platform: EHCI generic platform driver
>>>>> ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
>>>>> ohci-pci: OHCI PCI platform driver
>>>>> xhci-mtk 1e1c0000.xhci: xHCI Host Controller
>>>>> xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 1
>>>>> xhci-mtk 1e1c0000.xhci: hcc params 0x01401198 hci version 0x96 quirks 0x0000000000210010
>>>>> xhci-mtk 1e1c0000.xhci: irq 20, io mem 0x1e1c0000
>>>>> usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.01
>>>>> usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
>>>>> usb usb1: Product: xHCI Host Controller
>>>>> usb usb1: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
>>>>> usb usb1: SerialNumber: 1e1c0000.xhci
>>>>> hub 1-0:1.0: USB hub found
>>>>> hub 1-0:1.0: 2 ports detected
>>>>> xhci-mtk 1e1c0000.xhci: xHCI Host Controller
>>>>> xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 2
>>>>> xhci-mtk 1e1c0000.xhci: Host supports USB 3.0  SuperSpeed
>>>>> usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
>>>>> usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.01
>>>>> usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
>>>>> usb usb2: Product: xHCI Host Controller
>>>>> usb usb2: Manufacturer: Linux 5.1.0-ac0 xhci-hcd
>>>>> usb usb2: SerialNumber: 1e1c0000.xhci
>>>>> hub 2-0:1.0: USB hub found
>>>>> hub 2-0:1.0: 1 port detected
>>>>> usbcore: registered new interface driver cdc_acm
>>>>> cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
>>>>> usbcore: registered new interface driver cdc_wdm
>>>>> usbcore: registered new interface driver usb-storage
>>>>> usbcore: registered new interface driver usbserial_generic
>>>>> usbserial: USB Serial support registered for generic
>>>>> usbcore: registered new interface driver ipw
>>>>> usbserial: USB Serial support registered for IPWireless converter
>>>>> usbcore: registered new interface driver option
>>>>> usbserial: USB Serial support registered for GSM modem (1-port)
>>>>> usbcore: registered new interface driver qcaux
>>>>> usbserial: USB Serial support registered for qcaux
>>>>> usbcore: registered new interface driver qcserial
>>>>> usbserial: USB Serial support registered for Qualcomm USB modem
>>>>> usbcore: registered new interface driver quatech2
>>>>> usbserial: USB Serial support registered for Quatech 2nd gen USB to Serial Driver
>>>>> usbcore: registered new interface driver usb_serial_simple
>>>>> usbserial: USB Serial support registered for carelink
>>>>> usbserial: USB Serial support registered for zio
>>>>> usbserial: USB Serial support registered for funsoft
>>>>> usbserial: USB Serial support registered for flashloader
>>>>> usbserial: USB Serial support registered for google
>>>>> usbserial: USB Serial support registered for libtransistor
>>>>> usbserial: USB Serial support registered for vivopay
>>>>> usbserial: USB Serial support registered for moto_modem
>>>>> usbserial: USB Serial support registered for motorola_tetra
>>>>> usbserial: USB Serial support registered for novatel_gps
>>>>> usbserial: USB Serial support registered for hp4x
>>>>> usbserial: USB Serial support registered for suunto
>>>>> usbserial: USB Serial support registered for siemens_mpi
>>>>> u32 classifier
>>>>>        input device check on
>>>>> ipip: IPv4 and MPLS over IPv4 tunneling driver
>>>>> gre: GRE over IPv4 demultiplexor driver
>>>>> ip_gre: GRE over IPv4 tunneling driver
>>>>> Initializing XFRM netlink socket
>>>>> NET: Registered protocol family 10
>>>>> Segment Routing with IPv6
>>>>> sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
>>>>> NET: Registered protocol family 17
>>>>> NET: Registered protocol family 15
>>>>> 8021q: 802.1Q VLAN Support v1.8
>>>>> Loading compiled-in X.509 certificates
>>>>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>> mt7621-pci 1e140000.pcie: Port 0 N_FTS = 1b102800
>>>>> mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
>>>>> mt7621-pci 1e140000.pcie: Port 1 N_FTS = 1b102800
>>>>> mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
>>>>> mt7621-pci 1e140000.pcie: Port 2 N_FTS = 1b102800
>>>>> mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
>>>>> mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
>>>>> mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
>>>
>>> I changed also the way port resets are done. Now all of ports are
>>> being reset before
>>> enabling the phy. Also, power_off the phy is done before reseting the
>>> port so it should
>>> not hang now. The real problem here is if the pcie link is properly
>>> detected to get a working
>>> PCI.
>>>
>>> Patch attached.
>>
>> Tested with this patch. I still see the same occasional hang behavior.
>>
>> Sometimes boot up and scan work, sometimes it hangs.
>> It is probably around 1 in 3 boots is good, 2 out of 3 hang
>> (roughly speaking).
> 
> No good numbers for us, yet :)
> 
>>
>> I won't have much time today or tomorrow to dig deeper.
>> But I will try and do some more proper debugging early next week.
> 
> Just to be sure that the hang is because of powering off the phy on
> fail I directly unset clock
> bits for each port in the attached patch.

Applied this patch, and ran with it. I still get some boot time hangs.
It does seem less often, for whatever that is worth.

Sample trace hangs (just the last part of the kernel boot trace):

   Segment Routing with IPv6
   sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
   NET: Registered protocol family 17
   NET: Registered protocol family 15
   8021q: 802.1Q VLAN Support v1.8
   Loading compiled-in X.509 certificates
   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   [HUNG]

And another:

   Segment Routing with IPv6
   sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
   NET: Registered protocol family 17
   NET: Registered protocol family 15
   8021q: 802.1Q VLAN Support v1.8
   Loading compiled-in X.509 certificates
   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 0 N_FTS = 1b102800
   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 1 N_FTS = 1b102800
   mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 2 N_FTS = 1b102800
   mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
   [HUNG]

And even another:

   Segment Routing with IPv6
   sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
   NET: Registered protocol family 17
   NET: Registered protocol family 15
   8021q: 802.1Q VLAN Support v1.8
   Loading compiled-in X.509 certificates
   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 0 N_FTS = 1b102800
   mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 1 N_FTS = 1b102800
   mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
   mt7621-pci 1e140000.pcie: Port 2 N_FTS = 1b102800
   mt7621-pci 1e140000.pcie: pcie1 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: pcie2 no card, disable it (RST & CLK)
   mt7621-pci 1e140000.pcie: PCIE0 enabled
   mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
   mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
   pci_bus 0000:00: root bus resource [io  0xffffffff]
   pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
   pci_bus 0000:00: root bus resource [bus 00-ff]
   pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
   pci 0000:00:00.0: PCI bridge to [bus 01-ff]
   pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
   pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
   pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x601fffff]
   pci 0000:00:00.0: BAR 9: assigned [mem 0x60200000-0x602fffff pref]
   pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
   pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
   pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
   pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x601fffff 64bit]


> Debugging would be very helpful.

I will try and get some time to really look deeper.
Thanks for your efforts so far.

Regards
Greg

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Greg Ungerer]

Hi Brett,

On 15/6/19 11:53 pm, Brett Neumeier wrote:
[snip]
> For what it's worth -- possibly nothing? -- I grabbed the most recent six versions of the patch from this thread, including the one here, and did test boots of all of them on both a GnuBee PC1 and PC2 (based on 5.1.7). There were no perfect results, which is probably not surprising. Here I'm referring to different versions of the patch based on the date of the email where it was attached, I don't know if there's a better approach.
> 
> 2019-05-29: 9 success, 1 hang
> 2019-05-31: 8 success, 2 hang
> 2019-06-03: 7 success, 3 hang
> 2019-06-04: 8 success, 2 hang
> 2019-06-05: 6 success, 4 hang
> 2019-06-06: 7 success, 3 hang

That is valuable feedback, thanks for taking the time to run
through those variations.

Your results pretty much mirror what I see. Very inconsistent
booting behavior. Sometimes boots, sometimes doesn't. When it
doesn't it is always somewhere in the PCI startup.

Regards
Greg



> I've put all the boot message logs (for both successes and hangs) at https://repo.freesa.org/gnubee/ in case they are interesting. There are a few different common sequences of boot messages before the hangs occur, which I'll summarize here:
> 
> This happened 10 times:
> 
> [    9.056069] mt7621-pci 1e140000.pcie: Error applying setting, reverse things back
> [   10.187679] mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> [   10.198796] mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> [   10.327667] mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> [   10.338771] mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> [   10.467668] mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> [   10.480904] mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> [   11.556073] mt7621-pci 1e140000.pcie: PCIE0 enabled
> [   11.565784] mt7621-pci 1e140000.pcie: PCIE0 enabled
> [   11.575497] mt7621-pci 1e140000.pcie: PCIE0 enabled
> [   11.585244] mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> [   11.603982] mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> [   11.616664] pci_bus 0000:00: root bus resource [io  0xffffffff]
> [   11.628457] pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> [   11.642155] pci_bus 0000:00: root bus resource [bus 00-ff]
> [   11.655286] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> [   11.671259] pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> [   11.687206] pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> 
> This happened 4 times, but only when using the 2019-06-04 and 2019-06-05 patches:
> 
> [    9.071852] mt7621-pci 1e140000.pcie: Error applying setting, reverse things back
> [   10.197138] mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> [   10.208254] mt7621-pci 1e140000.pcie: Port 0 N_FTS = 1b102800
> [   10.337129] mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> [   10.348232] mt7621-pci 1e140000.pcie: Port 1 N_FTS = 1b102800
> [   10.477129] mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> [   10.490365] mt7621-pci 1e140000.pcie: Port 2 N_FTS = 1b102800
> [   11.565525] mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> [   11.579392] mt7621-pci 1e140000.pcie: PCIE0 enabled
> [   11.589105] mt7621-pci 1e140000.pcie: PCIE0 enabled
> [   11.598853] mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> [   11.617590] mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> [   11.630266] pci_bus 0000:00: root bus resource [io  0xffffffff]
> [   11.642059] pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> [   11.655757] pci_bus 0000:00: root bus resource [bus 00-ff]
> [   11.668437] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> [   11.684402] pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> [   11.700805] pci 0000:01:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:00.0 (capable of 4.000
> Gb/s with 5 GT/s x1 link)
> [   11.729362] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> [   11.740320] pci 0000:02:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:01.0 (capable of 4.000
> Gb/s with 5 GT/s x1 link)
> [   11.768887] pci 0000:00:01.0: PCI bridge to [bus 02-ff]
> [   11.779414] pci 0000:00:01.0: BAR 0: no space for [mem size 0x80000000]
> [   11.792587] pci 0000:00:01.0: BAR 0: failed to assign [mem size 0x80000000]
> [   11.806461] pci 0000:00:00.0: BAR 0: assigned [mem 0x60000000-0x61ffffff]
> [   11.819988] pci 0000:00:00.0: BAR 8: assigned [mem 0x62000000-0x620fffff]
> [   11.833516] pci 0000:00:00.0: BAR 9: assigned [mem 0x62100000-0x621fffff pref]
> [   11.847902] pci 0000:00:01.0: BAR 8: assigned [mem 0x62200000-0x622fffff]
> [   11.861431] pci 0000:00:01.0: BAR 9: assigned [mem 0x62300000-0x623fffff pref]
> [   11.875819] pci 0000:00:00.0: BAR 1: assigned [mem 0x62400000-0x6240ffff]
> [   11.889350] pci 0000:00:01.0: BAR 1: assigned [mem 0x62410000-0x6241ffff]
> [   11.902884] pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
> [   11.915364] pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
> [   11.928541] pci 0000:00:01.0: BAR 7: no space for [io  size 0x1000]
> [   11.941026] pci 0000:00:01.0: BAR 7: failed to assign [io  size 0x1000]
> [   11.954211] pci 0000:01:00.0: BAR 5: assigned [mem 0x62000000-0x620001ff]
> [   11.967737] pci 0000:01:00.0: BAR 4: no space for [io  size 0x0010]
> [   11.980222] pci 0000:01:00.0: BAR 4: failed to assign [io  size 0x0010]
> [   11.993395] pci 0000:01:00.0: BAR 0: no space for [io  size 0x0008]
> [   12.005879] pci 0000:01:00.0: BAR 0: failed to assign [io  size 0x0008]
> [   12.019052] pci 0000:01:00.0: BAR 2: no space for [io  size 0x0008]
> [   12.031543] pci 0000:01:00.0: BAR 2: failed to assign [io  size 0x0008]
> [   12.044717] pci 0000:01:00.0: BAR 1: no space for [io  size 0x0004]
> [   12.057199] pci 0000:01:00.0: BAR 1: failed to assign [io  size 0x0004]
> [   12.070378] pci 0000:01:00.0: BAR 3: no space for [io  size 0x0004]
> [   12.082858] pci 0000:01:00.0: BAR 3: failed to assign [io  size 0x0004]
> [   12.096036] pci 0000:00:00.0: PCI bridge to [bus 01]
> [   12.105930] pci 0000:00:00.0:   bridge window [mem 0x62000000-0x620fffff]
> [   12.119452] pci 0000:00:00.0:   bridge window [mem 0x62100000-0x621fffff pref]
> [   12.133850] pci 0000:02:00.0: BAR 5: assigned [mem 0x62200000-0x622001ff]
> [   12.147381] pci 0000:02:00.0: BAR 4: no space for [io  size 0x0010]
> [   12.159872] pci 0000:02:00.0: BAR 4: failed to assign [io  size 0x0010]
> [   12.173045] pci 0000:02:00.0: BAR 0: no space for [io  size 0x0008]
> [   12.185528] pci 0000:02:00.0: BAR 0: failed to assign [io  size 0x0008]
> [   12.198701] pci 0000:02:00.0: BAR 2: no space for [io  size 0x0008]
> [   12.211186] pci 0000:02:00.0: BAR 2: failed to assign [io  size 0x0008]
> [   12.224359] pci 0000:02:00.0: BAR 1: no space for [io  size 0x0004]
> [   12.236844] pci 0000:02:00.0: BAR 1: failed to assign [io  size 0x0004]
> [   12.250016] pci 0000:02:00.0: BAR 3: no space for [io  size 0x0004]
> [   12.262501] pci 0000:02:00.0: BAR 3: failed to assign [io  size 0x0004]
> [   12.275675] pci 0000:00:01.0: PCI bridge to [bus 02]
> [   12.285570] pci 0000:00:01.0:   bridge window [mem 0x62200000-0x622fffff]
> [   12.299095] pci 0000:00:01.0:   bridge window [mem 0x62300000-0x623fffff pref]
> [   12.313830] pci 0000:00:00.0: enabling device (0000 -> 0002)
> [   12.325116] ahci 0000:01:00.0: enabling device (0000 -> 0002)
> I have no idea how to do any debugging of this stuff, I'm afraid, but if there is anything else I can do to help please let me know!
> 
> 
> -- 
> Brett Neumeier (bneumeier@gmail.com <mailto:bneumeier@gmail.com>)
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: staging: mt7621-pci: factor out 'mt7621_pcie_enable_port' function

[Sergio Paracuellos]

Hi Brett, Greg,

I am a little busy with some work stuff, so sorry for my late response.

On Mon, Jun 17, 2019 at 5:57 AM Greg Ungerer <gerg@kernel.org> wrote:
>
> Hi Brett,
>
> On 15/6/19 11:53 pm, Brett Neumeier wrote:
> [snip]
> > For what it's worth -- possibly nothing? -- I grabbed the most recent six versions of the patch from this thread, including the one here, and did test boots of all of them on both a GnuBee PC1 and PC2 (based on 5.1.7). There were no perfect results, which is probably not surprising. Here I'm referring to different versions of the patch based on the date of the email where it was attached, I don't know if there's a better approach.
> >
> > 2019-05-29: 9 success, 1 hang
> > 2019-05-31: 8 success, 2 hang
> > 2019-06-03: 7 success, 3 hang
> > 2019-06-04: 8 success, 2 hang
> > 2019-06-05: 6 success, 4 hang
> > 2019-06-06: 7 success, 3 hang
>

Thank you very much for your time in this.

> That is valuable feedback, thanks for taking the time to run
> through those variations.
>
> Your results pretty much mirror what I see. Very inconsistent
> booting behavior. Sometimes boots, sometimes doesn't. When it
> doesn't it is always somewhere in the PCI startup.

First I am going to officially send clean patches to achieve only the
PERST gpio stuff
to get those properly applied in the official trees.

In order to try to know what is happening some deeper debugging would
be helpful.

Does not using gpio descriptor API but accessing memory also sometimes hangs?
Does moving pci driver to module_init instead of arch_initcall changes
something?
Does moving pci phy code into pci driver with arch_initcall changes something?

If it is not a reset issue it might be something with the boot order,
according to the driver changes with properly working version.

> Regards
> Gregç

Best regards,
    Sergio Paracuellos
>
>
>
> > I've put all the boot message logs (for both successes and hangs) at https://repo.freesa.org/gnubee/ in case they are interesting. There are a few different common sequences of boot messages before the hangs occur, which I'll summarize here:
> >
> > This happened 10 times:
> >
> > [    9.056069] mt7621-pci 1e140000.pcie: Error applying setting, reverse things back
> > [   10.187679] mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> > [   10.198796] mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 0
> > [   10.327667] mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> > [   10.338771] mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 1
> > [   10.467668] mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> > [   10.480904] mt7621-pci 1e140000.pcie: Port 454043648 N_FTS = 2
> > [   11.556073] mt7621-pci 1e140000.pcie: PCIE0 enabled
> > [   11.565784] mt7621-pci 1e140000.pcie: PCIE0 enabled
> > [   11.575497] mt7621-pci 1e140000.pcie: PCIE0 enabled
> > [   11.585244] mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> > [   11.603982] mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> > [   11.616664] pci_bus 0000:00: root bus resource [io  0xffffffff]
> > [   11.628457] pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> > [   11.642155] pci_bus 0000:00: root bus resource [bus 00-ff]
> > [   11.655286] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> > [   11.671259] pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> > [   11.687206] pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> >
> > This happened 4 times, but only when using the 2019-06-04 and 2019-06-05 patches:
> >
> > [    9.071852] mt7621-pci 1e140000.pcie: Error applying setting, reverse things back
> > [   10.197138] mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> > [   10.208254] mt7621-pci 1e140000.pcie: Port 0 N_FTS = 1b102800
> > [   10.337129] mt7621-pci-phy 1e149000.pcie-phy: Xtal is 40MHz
> > [   10.348232] mt7621-pci 1e140000.pcie: Port 1 N_FTS = 1b102800
> > [   10.477129] mt7621-pci-phy 1e14a000.pcie-phy: Xtal is 40MHz
> > [   10.490365] mt7621-pci 1e140000.pcie: Port 2 N_FTS = 1b102800
> > [   11.565525] mt7621-pci 1e140000.pcie: pcie0 no card, disable it (RST & CLK)
> > [   11.579392] mt7621-pci 1e140000.pcie: PCIE0 enabled
> > [   11.589105] mt7621-pci 1e140000.pcie: PCIE0 enabled
> > [   11.598853] mt7621-pci 1e140000.pcie: PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
> > [   11.617590] mt7621-pci 1e140000.pcie: PCI host bridge to bus 0000:00
> > [   11.630266] pci_bus 0000:00: root bus resource [io  0xffffffff]
> > [   11.642059] pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
> > [   11.655757] pci_bus 0000:00: root bus resource [bus 00-ff]
> > [   11.668437] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> > [   11.684402] pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> > [   11.700805] pci 0000:01:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:00.0 (capable of 4.000
> > Gb/s with 5 GT/s x1 link)
> > [   11.729362] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
> > [   11.740320] pci 0000:02:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s x1 link at 0000:00:01.0 (capable of 4.000
> > Gb/s with 5 GT/s x1 link)
> > [   11.768887] pci 0000:00:01.0: PCI bridge to [bus 02-ff]
> > [   11.779414] pci 0000:00:01.0: BAR 0: no space for [mem size 0x80000000]
> > [   11.792587] pci 0000:00:01.0: BAR 0: failed to assign [mem size 0x80000000]
> > [   11.806461] pci 0000:00:00.0: BAR 0: assigned [mem 0x60000000-0x61ffffff]
> > [   11.819988] pci 0000:00:00.0: BAR 8: assigned [mem 0x62000000-0x620fffff]
> > [   11.833516] pci 0000:00:00.0: BAR 9: assigned [mem 0x62100000-0x621fffff pref]
> > [   11.847902] pci 0000:00:01.0: BAR 8: assigned [mem 0x62200000-0x622fffff]
> > [   11.861431] pci 0000:00:01.0: BAR 9: assigned [mem 0x62300000-0x623fffff pref]
> > [   11.875819] pci 0000:00:00.0: BAR 1: assigned [mem 0x62400000-0x6240ffff]
> > [   11.889350] pci 0000:00:01.0: BAR 1: assigned [mem 0x62410000-0x6241ffff]
> > [   11.902884] pci 0000:00:00.0: BAR 7: no space for [io  size 0x1000]
> > [   11.915364] pci 0000:00:00.0: BAR 7: failed to assign [io  size 0x1000]
> > [   11.928541] pci 0000:00:01.0: BAR 7: no space for [io  size 0x1000]
> > [   11.941026] pci 0000:00:01.0: BAR 7: failed to assign [io  size 0x1000]
> > [   11.954211] pci 0000:01:00.0: BAR 5: assigned [mem 0x62000000-0x620001ff]
> > [   11.967737] pci 0000:01:00.0: BAR 4: no space for [io  size 0x0010]
> > [   11.980222] pci 0000:01:00.0: BAR 4: failed to assign [io  size 0x0010]
> > [   11.993395] pci 0000:01:00.0: BAR 0: no space for [io  size 0x0008]
> > [   12.005879] pci 0000:01:00.0: BAR 0: failed to assign [io  size 0x0008]
> > [   12.019052] pci 0000:01:00.0: BAR 2: no space for [io  size 0x0008]
> > [   12.031543] pci 0000:01:00.0: BAR 2: failed to assign [io  size 0x0008]
> > [   12.044717] pci 0000:01:00.0: BAR 1: no space for [io  size 0x0004]
> > [   12.057199] pci 0000:01:00.0: BAR 1: failed to assign [io  size 0x0004]
> > [   12.070378] pci 0000:01:00.0: BAR 3: no space for [io  size 0x0004]
> > [   12.082858] pci 0000:01:00.0: BAR 3: failed to assign [io  size 0x0004]
> > [   12.096036] pci 0000:00:00.0: PCI bridge to [bus 01]
> > [   12.105930] pci 0000:00:00.0:   bridge window [mem 0x62000000-0x620fffff]
> > [   12.119452] pci 0000:00:00.0:   bridge window [mem 0x62100000-0x621fffff pref]
> > [   12.133850] pci 0000:02:00.0: BAR 5: assigned [mem 0x62200000-0x622001ff]
> > [   12.147381] pci 0000:02:00.0: BAR 4: no space for [io  size 0x0010]
> > [   12.159872] pci 0000:02:00.0: BAR 4: failed to assign [io  size 0x0010]
> > [   12.173045] pci 0000:02:00.0: BAR 0: no space for [io  size 0x0008]
> > [   12.185528] pci 0000:02:00.0: BAR 0: failed to assign [io  size 0x0008]
> > [   12.198701] pci 0000:02:00.0: BAR 2: no space for [io  size 0x0008]
> > [   12.211186] pci 0000:02:00.0: BAR 2: failed to assign [io  size 0x0008]
> > [   12.224359] pci 0000:02:00.0: BAR 1: no space for [io  size 0x0004]
> > [   12.236844] pci 0000:02:00.0: BAR 1: failed to assign [io  size 0x0004]
> > [   12.250016] pci 0000:02:00.0: BAR 3: no space for [io  size 0x0004]
> > [   12.262501] pci 0000:02:00.0: BAR 3: failed to assign [io  size 0x0004]
> > [   12.275675] pci 0000:00:01.0: PCI bridge to [bus 02]
> > [   12.285570] pci 0000:00:01.0:   bridge window [mem 0x62200000-0x622fffff]
> > [   12.299095] pci 0000:00:01.0:   bridge window [mem 0x62300000-0x623fffff pref]
> > [   12.313830] pci 0000:00:00.0: enabling device (0000 -> 0002)
> > [   12.325116] ahci 0000:01:00.0: enabling device (0000 -> 0002)
> > I have no idea how to do any debugging of this stuff, I'm afraid, but if there is anything else I can do to help please let me know!
> >
> >
> > --
> > Brett Neumeier (bneumeier@gmail.com <mailto:bneumeier@gmail.com>)
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel