orion/kirkwood pcie issue still open with rc6

orion/kirkwood pcie issue still open with rc6

[Dieter Kiermaier]

Hi Nicolas,

I've tried your latest kernel from orion.git and there is still a PCIe issue left.
Please see my patch below.
It would be very nice if someone could tell me why this patch helped me
to boot my kernel :)


---
>From 6ce3d877a96186e0dda0c319b9b7eb8df5d10541 Mon Sep 17 00:00:00 2001
From: Dieter Kiermaier <dk-arm-linux@gmx.de>
Date: Mon, 9 Nov 2009 09:15:12 +0100
Subject: [PATCH] Without this patch my kernel hangs during boot up while scanning the pci bus behind
 a marvell 88SB2211 stepping A0 pcie to pci bridge.

Ronan Shitrit from marvell gave me the information to clear bit 2 of physical
address 0xf1020100 to enable bus scanning.
I don't know what this really does but it helped to get my kernel up and running.

Signed-off-by: Dieter Kiermaier <dk-arm-linux@gmx.de>
---
 arch/arm/mach-kirkwood/openrd_base-setup.c |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/arch/arm/mach-kirkwood/openrd_base-setup.c b/arch/arm/mach-kirkwood/openrd_base-setup.c
index 77617c7..21a1cf7 100644
--- a/arch/arm/mach-kirkwood/openrd_base-setup.c
+++ b/arch/arm/mach-kirkwood/openrd_base-setup.c
@@ -14,6 +14,7 @@
 #include <linux/mtd/partitions.h>
 #include <linux/ata_platform.h>
 #include <linux/mv643xx_eth.h>
+#include <asm/io.h>
 #include <asm/mach-types.h>
 #include <asm/mach/arch.h>
 #include <mach/kirkwood.h>
@@ -76,6 +77,19 @@ static void __init openrd_base_init(void)
 
 static int __init openrd_base_pci_init(void)
 {
+	u32 cpu_config_reg;
+	void __iomem *base;
+	
+	base = ioremap(0xf1020100, 4);
+	if (base)
+	{
+		cpu_config_reg = readl(base);
+		cpu_config_reg &= ~(1 << 2);
+		writel(cpu_config_reg, base);
+		printk("register 0x20100: %x\n", readl(base));
+	}
+	iounmap(base);
+	
 	if (machine_is_openrd_base())
 		kirkwood_pcie_init();
 
-- 
1.5.4.3

Dieter

orion/kirkwood pcie issue still open with rc6

[Simon Kagstrom]

On Mon, 9 Nov 2009 09:33:46 +0100
Dieter Kiermaier <dk-arm-linux@gmx.de> wrote:

> Ronan Shitrit from marvell gave me the information to clear bit 2 of physical
> address 0xf1020100 to enable bus scanning.
> I don't know what this really does but it helped to get my kernel up and running.

You have that in the functional specification for the kirkwood
processors:

  http://www.marvell.com/files/products/embedded_processors/kirkwood/FS_88F6180_9x_6281_OpenSource.pdf

See Appendix A for the register set and lookup address 0x20100 (the CPU
configuration register). Bit 2 in that register turns on/off error
propagation:

              Mbus-L Error propogation
              For a list of errors see "Error Handling".
              0 = NoErrProp: Error indications are not propagated to Mbus-L. The
                  transactions are completed normally.
              1 = ErrProp: Error indications are propagated to Mbus-L.

and clearing it sounds quite a bit like hiding the real problem :-).


Probably it's best to try to find out what's causing this error (see
the Error Handling section of the PCIe chapter in the reference manual
above) so that that a proper fix can be implemented!

// Simon

orion/kirkwood pcie issue still open with rc6

[Dieter Kiermaier]

Am Montag 09 November 2009 09:49:58 schrieb Simon Kagstrom:
> On Mon, 9 Nov 2009 09:33:46 +0100
> Dieter Kiermaier <dk-arm-linux@gmx.de> wrote:
> 
> > Ronan Shitrit from marvell gave me the information to clear bit 2 of physical
> > address 0xf1020100 to enable bus scanning.
> > I don't know what this really does but it helped to get my kernel up and running.
> 
> You have that in the functional specification for the kirkwood
> processors:
> 
>   http://www.marvell.com/files/products/embedded_processors/kirkwood/FS_88F6180_9x_6281_OpenSource.pdf
> 
> See Appendix A for the register set and lookup address 0x20100 (the CPU
> configuration register). Bit 2 in that register turns on/off error
> propagation:
> 
>               Mbus-L Error propogation
>               For a list of errors see "Error Handling".
>               0 = NoErrProp: Error indications are not propagated to Mbus-L. The
>                   transactions are completed normally.
>               1 = ErrProp: Error indications are propagated to Mbus-L.
> 
> and clearing it sounds quite a bit like hiding the real problem :-).
> 

I've seen this and you're absolutely right. Thats why I've posted!

> 
> Probably it's best to try to find out what's causing this error (see
> the Error Handling section of the PCIe chapter in the reference manual
> above) so that that a proper fix can be implemented!

And here my problems are starting ;)
How can I implement error handling for that kind of reported errors?
I'm sorry but I've absolutely no idea.

> 
> // Simon

Many thanks,
Dieter
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

orion/kirkwood pcie issue still open with rc6

[Ronen Shitrit]

Hi

The bit below indicates whether to propagate external errors on the PCIE as external abort or not.
In your case the external error is a PCI master abort, this is a legitimate error in case of issuing a configuration read to a device which doesn't exist on the bus.

To overcome the kernel fault and still enable the error propagation, there is the option to add handler which will look for the error cause and ignore it in case of master abort or to disable this bit before doing PCI config read.

Regards


-----Original Message-----
From: linux-arm-kernel-bounces@lists.infradead.org [mailto:linux-arm-kernel-bounces at lists.infradead.org] On Behalf Of Dieter Kiermaier
Sent: Monday, November 09, 2009 11:06 AM
To: linux-arm-kernel at lists.infradead.org
Cc: Simon Kagstrom; Nicolas Pitre
Subject: Re: orion/kirkwood pcie issue still open with rc6

Am Montag 09 November 2009 09:49:58 schrieb Simon Kagstrom:
> On Mon, 9 Nov 2009 09:33:46 +0100
> Dieter Kiermaier <dk-arm-linux@gmx.de> wrote:
>
> > Ronan Shitrit from marvell gave me the information to clear bit 2 of physical
> > address 0xf1020100 to enable bus scanning.
> > I don't know what this really does but it helped to get my kernel up and running.
>
> You have that in the functional specification for the kirkwood
> processors:
>
>   http://www.marvell.com/files/products/embedded_processors/kirkwood/FS_88F6180_9x_6281_OpenSource.pdf
>
> See Appendix A for the register set and lookup address 0x20100 (the CPU
> configuration register). Bit 2 in that register turns on/off error
> propagation:
>
>               Mbus-L Error propogation
>               For a list of errors see "Error Handling".
>               0 = NoErrProp: Error indications are not propagated to Mbus-L. The
>                   transactions are completed normally.
>               1 = ErrProp: Error indications are propagated to Mbus-L.
>
> and clearing it sounds quite a bit like hiding the real problem :-).
>

I've seen this and you're absolutely right. Thats why I've posted!

>
> Probably it's best to try to find out what's causing this error (see
> the Error Handling section of the PCIe chapter in the reference manual
> above) so that that a proper fix can be implemented!

And here my problems are starting ;)
How can I implement error handling for that kind of reported errors?
I'm sorry but I've absolutely no idea.

>
> // Simon

Many thanks,
Dieter
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!)

[Dieter Kiermaier]

Hi again,

> Am Montag 09 November 2009 09:49:58 schrieb Simon Kagstrom:
> > On Mon, 9 Nov 2009 09:33:46 +0100
> > Dieter Kiermaier <dk-arm-linux@gmx.de> wrote:
> > 
> > > Ronan Shitrit from marvell gave me the information to clear bit 2 of physical
> > > address 0xf1020100 to enable bus scanning.
> > > I don't know what this really does but it helped to get my kernel up and running.
> > 

I've tested again:

with 2.6.22.18 marvell stock kernel my fpga behing the pcie->pci bridge works as expected.
Here is the output of lscpi:

sh-3.2# uname -a
Linux DB88FXX81 2.6.22.18 #1 Wed Apr 22 20:31:28 IST 2009 armv5tejl GNU/Linux
sh-3.2#


sh-3.2# lspci -vv
00:01.0 Class 0604: Device 11ab:2211 (rev 01)
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 32 bytes
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=240
        I/O behind bridge: 00001000-00004fff
        Memory behind bridge: e8000000-ebffffff
        Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity+ SERR- NoISA+ VGA- MAbort+ >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
                Bridge: PM- B3+
        Capabilities: [48] Express (v1) PCI/PCI-X Bridge, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <256ns, L1 unlimited
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        Capabilities: [100] #11ab
        Capabilities: [221] #f0e8
        Capabilities: [f1e] #2a0
        Capabilities: [e80] #187

01:08.0 Class ff00: Device 1731:0101 (rev 10)
        Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B- ParErr+ DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Region 0: Memory at e8000000 (32-bit, non-prefetchable) [size=64M]
        Region 1: I/O ports@<ignored>
        Kernel driver in use: ArtistaNET-III frame buffer driver


If I switch to mainline linux kernel (there is no difference between openrd, vanilla, orion.git kernels!) my problems start:
I have to patch openrd_init() as follows:

diff --git a/arch/arm/mach-kirkwood/openrd_base-setup.c b/arch/arm/mach-kirkwood/openrd_base-setup.c
index 77617c7..670312b 100644
--- a/arch/arm/mach-kirkwood/openrd_base-setup.c
+++ b/arch/arm/mach-kirkwood/openrd_base-setup.c
@@ -14,6 +14,7 @@
 #include <linux/mtd/partitions.h>
 #include <linux/ata_platform.h>
 #include <linux/mv643xx_eth.h>
+#include <asm/io.h>
 #include <asm/mach-types.h>
 #include <asm/mach/arch.h>
 #include <mach/kirkwood.h>
@@ -76,9 +77,21 @@ static void __init openrd_base_init(void)

 static int __init openrd_base_pci_init(void)
 {
+       u32 cpu_config_reg;
+       void __iomem *base;
+
+       base = ioremap(0xf1020100, 4);
+       if (base)
+       {
+               cpu_config_reg = readl(base);
+               cpu_config_reg &= ~(1 << 2);
+               writel(cpu_config_reg, base);
+               printk("register 0x20100: %x\n", readl(base));
+       }
+       iounmap(base);
        if (machine_is_openrd_base())
                kirkwood_pcie_init();
-
+
        return 0;
  }
 subsys_initcall(openrd_base_pci_init);
----
to allow my system to boot up and after succesfully boot my pci device isn't reachable - without any errors / warnings!
I've enabled printk (echo 8 > /proc/sys/kernel/printk) and still no console output.


sh-3.2# uname -a
Linux DB88FXX81 2.6.32-rc6-00014-g92bdfb9-dirty #3 PREEMPT Wed Nov 11 15:10:49 CET 2009 armv5tel GNU/Linux
sh-3.2#
sh-3.2# lspci -vv
00:00.0 Class 0580: Device 11ab:6281 (rev 02)
        Subsystem: Device 11ab:11ab
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 32 bytes
        Interrupt: pin A routed to IRQ 9
        Region 0: Memory@<ignored> (64-bit, prefetchable)
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [60] Express (v1) Root Port (Slot-), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <256ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <256ns, L1 unlimited
                        ClockPM+ Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
        Capabilities: [100] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-

00:01.0 Class 0604: Device 11ab:2211 (rev 01)
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: 00001000-00004fff
        Memory behind bridge: e0000000-e3ffffff
        Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity+ SERR- NoISA- VGA- MAbort+ >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
                Bridge: PM- B3+
        Capabilities: [48] Express (v1) PCI/PCI-X Bridge, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <256ns, L1 unlimited
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        Capabilities: [100] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-

01:08.0 Class ff00: Device 1731:0101 (rev 10)
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B- ParErr+ DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Region 0: Memory at e0000000 (32-bit, non-prefetchable) [disabled] [size=64M]
        Region 1: I/O ports@<unassigned> [disabled]

sh-3.2#



I hope someone from marvell will help me to solve this problem. I don't want to stick with 2.6.22 kernel!

It looks like the problem during boot up will show me that something went wrong but I've absolutely no idea
how to solve this problem :(


Many thanks for helping,
Dieter


> > You have that in the functional specification for the kirkwood
> > processors:
> > 
> >   http://www.marvell.com/files/products/embedded_processors/kirkwood/FS_88F6180_9x_6281_OpenSource.pdf
> > 
> > See Appendix A for the register set and lookup address 0x20100 (the CPU
> > configuration register). Bit 2 in that register turns on/off error
> > propagation:
> > 
> >               Mbus-L Error propogation
> >               For a list of errors see "Error Handling".
> >               0 = NoErrProp: Error indications are not propagated to Mbus-L. The
> >                   transactions are completed normally.
> >               1 = ErrProp: Error indications are propagated to Mbus-L.
> > 
> > and clearing it sounds quite a bit like hiding the real problem :-).
> > 
> 
> I've seen this and you're absolutely right. Thats why I've posted!
> 
> > 
> > Probably it's best to try to find out what's causing this error (see
> > the Error Handling section of the PCIe chapter in the reference manual
> > above) so that that a proper fix can be implemented!
> 
> And here my problems are starting ;)
> How can I implement error handling for that kind of reported errors?
> I'm sorry but I've absolutely no idea.
> 
> > 
> > // Simon
> 
> Many thanks,
> Dieter
> > 
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> > 
> 
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!)

[Lennert Buytenhek]

On Wed, Nov 11, 2009 at 06:29:22AM -0800, Dieter Kiermaier wrote:

> > > > Ronan Shitrit from marvell gave me the information to clear bit 2 of physical
> > > > address 0xf1020100 to enable bus scanning.
> > > > I don't know what this really does but it helped to get my kernel up and running.
> > >
> 
> I've tested again:
> 
> with 2.6.22.18 marvell stock kernel my fpga behing the pcie->pci bridge works as expected.
> Here is the output of lscpi:
> 
> sh-3.2# uname -a
> Linux DB88FXX81 2.6.22.18 #1 Wed Apr 22 20:31:28 IST 2009 armv5tejl GNU/Linux
> sh-3.2#
> 
> 
> sh-3.2# lspci -vv
> 00:01.0 Class 0604: Device 11ab:2211 (rev 01)
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
>         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         Latency: 0, Cache Line Size: 32 bytes
>         Bus: primary=00, secondary=01, subordinate=01, sec-latency=240
>         I/O behind bridge: 00001000-00004fff
>         Memory behind bridge: e8000000-ebffffff
>         Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
>         BridgeCtl: Parity+ SERR- NoISA+ VGA- MAbort+ >Reset- FastB2B-
>                 PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
>         Capabilities: [40] Power Management version 2
>                 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
>                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
>                 Bridge: PM- B3+
>         Capabilities: [48] Express (v1) PCI/PCI-X Bridge, MSI 00
>                 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
>                         ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
>                 DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
>                         RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
>                         MaxPayload 128 bytes, MaxReadReq 512 bytes
>                 DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq- AuxPwr- TransPend-
>                 LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <256ns, L1 unlimited
>                         ClockPM- Surprise- LLActRep- BwNot-
>                 LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
>                         ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
>                 LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
>         Capabilities: [100] #11ab
>         Capabilities: [221] #f0e8
>         Capabilities: [f1e] #2a0
>         Capabilities: [e80] #187
> 
> 01:08.0 Class ff00: Device 1731:0101 (rev 10)
>         Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
>         Status: Cap- 66MHz- UDF- FastB2B- ParErr+ DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         Region 0: Memory at e8000000 (32-bit, non-prefetchable) [size=64M]
>         Region 1: I/O ports at <ignored>
>         Kernel driver in use: ArtistaNET-III frame buffer driver
> 
> 
> If I switch to mainline linux kernel (there is no difference between openrd, vanilla, orion.git kernels!) my problems start:
> I have to patch openrd_init() as follows:
> 
> diff --git a/arch/arm/mach-kirkwood/openrd_base-setup.c b/arch/arm/mach-kirkwood/openrd_base-setup.c
> index 77617c7..670312b 100644
> --- a/arch/arm/mach-kirkwood/openrd_base-setup.c
> +++ b/arch/arm/mach-kirkwood/openrd_base-setup.c
> @@ -14,6 +14,7 @@
>  #include <linux/mtd/partitions.h>
>  #include <linux/ata_platform.h>
>  #include <linux/mv643xx_eth.h>
> +#include <asm/io.h>
>  #include <asm/mach-types.h>
>  #include <asm/mach/arch.h>
>  #include <mach/kirkwood.h>
> @@ -76,9 +77,21 @@ static void __init openrd_base_init(void)
> 
>  static int __init openrd_base_pci_init(void)
>  {
> +       u32 cpu_config_reg;
> +       void __iomem *base;
> +
> +       base = ioremap(0xf1020100, 4);
> +       if (base)
> +       {
> +               cpu_config_reg = readl(base);
> +               cpu_config_reg &= ~(1 << 2);
> +               writel(cpu_config_reg, base);
> +               printk("register 0x20100: %x\n", readl(base));
> +       }
> +       iounmap(base);
>         if (machine_is_openrd_base())
>                 kirkwood_pcie_init();
> -
> +
>         return 0;
>   }
>  subsys_initcall(openrd_base_pci_init);
> ----
> to allow my system to boot up

What's likely happening is that your boot loader either enables or
does not disable this bit, and the 2.6.22.18 kernel disables the bit,
while the upstream kernel leaves the bit untouched.

What this bit does is to decide whether or not aborts on the PCI
interface are translated into processor aborts.  It's not really
necessary to have this enabled, as the transaction will return
0xffffffff to the CPU anyway, which is then handled appropriately
as well.

What uboot version are you using?  The uboot versions I have on my
Kirkwood boards all jump to the OS with this bit already cleared.
Perhaps we should clear it explicitly from Linux.


> and after succesfully boot my pci device isn't reachable - without any errors / warnings!
> I've enabled printk (echo 8 > /proc/sys/kernel/printk) and still no console output.

The device shows up on the bus, the bridge primary/secondary bus numbers
look good, and the secondary memory address range on the bridge looks
properly programmed.  It all looks good to me.

What do you mean by 'not reachable'?  I'm guessing that you're trying
to access the memory BAR on the 01:08.0 device directly from userland
by reading from the e000_0000 - e3ff_ffff address range from /dev/mem
and only getting 0xffffffff back because you don't have an actual kernel
driver for this FPGA board and thus you're not calling pci_enable_device()
on your device, causing MEM/IO decoding not to have been enabled on
the device as seems to be the case in your dump?

For what it's worth, I have various mv78xx0 and Kirkwood boards with
88SB2211 PCIe-to-PCI bridges on them, and they all work fine, and
the devices behind those bridges do too.

orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!)

[Dieter Kiermaier]

Hi Lennert,


> On Wed, Nov 11, 2009 at 06:29:22AM -0800, Dieter Kiermaier wrote:
> 
> > > > > Ronan Shitrit from marvell gave me the information to clear bit 2 of physical
> > > > > address 0xf1020100 to enable bus scanning.
> > > > > I don't know what this really does but it helped to get my kernel up and running.
> > > >
> > 
> > I've tested again:
> > 
> > with 2.6.22.18 marvell stock kernel my fpga behing the pcie->pci bridge works as expected.
> > Here is the output of lscpi:
> > 
> > sh-3.2# uname -a
> > Linux DB88FXX81 2.6.22.18 #1 Wed Apr 22 20:31:28 IST 2009 armv5tejl GNU/Linux
> > sh-3.2#
> > 
> > 
> > sh-3.2# lspci -vv
> > 00:01.0 Class 0604: Device 11ab:2211 (rev 01)
> >         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> >         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >         Latency: 0, Cache Line Size: 32 bytes
> >         Bus: primary=00, secondary=01, subordinate=01, sec-latency=240
> >         I/O behind bridge: 00001000-00004fff
> >         Memory behind bridge: e8000000-ebffffff
> >         Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> >         BridgeCtl: Parity+ SERR- NoISA+ VGA- MAbort+ >Reset- FastB2B-
> >                 PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> >         Capabilities: [40] Power Management version 2
> >                 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
> >                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> >                 Bridge: PM- B3+
> >         Capabilities: [48] Express (v1) PCI/PCI-X Bridge, MSI 00
> >                 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
> >                         ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
> >                 DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
> >                         RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
> >                         MaxPayload 128 bytes, MaxReadReq 512 bytes
> >                 DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq- AuxPwr- TransPend-
> >                 LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <256ns, L1 unlimited
> >                         ClockPM- Surprise- LLActRep- BwNot-
> >                 LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
> >                         ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> >                 LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> >         Capabilities: [100] #11ab
> >         Capabilities: [221] #f0e8
> >         Capabilities: [f1e] #2a0
> >         Capabilities: [e80] #187
> > 
> > 01:08.0 Class ff00: Device 1731:0101 (rev 10)
> >         Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> >         Status: Cap- 66MHz- UDF- FastB2B- ParErr+ DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >         Region 0: Memory at e8000000 (32-bit, non-prefetchable) [size=64M]
> >         Region 1: I/O ports at <ignored>
> >         Kernel driver in use: ArtistaNET-III frame buffer driver
> > 
> > 
> > If I switch to mainline linux kernel (there is no difference between openrd, vanilla, orion.git kernels!) my problems start:
> > I have to patch openrd_init() as follows:
> > 
> > diff --git a/arch/arm/mach-kirkwood/openrd_base-setup.c b/arch/arm/mach-kirkwood/openrd_base-setup.c
> > index 77617c7..670312b 100644
> > --- a/arch/arm/mach-kirkwood/openrd_base-setup.c
> > +++ b/arch/arm/mach-kirkwood/openrd_base-setup.c
> > @@ -14,6 +14,7 @@
> >  #include <linux/mtd/partitions.h>
> >  #include <linux/ata_platform.h>
> >  #include <linux/mv643xx_eth.h>
> > +#include <asm/io.h>
> >  #include <asm/mach-types.h>
> >  #include <asm/mach/arch.h>
> >  #include <mach/kirkwood.h>
> > @@ -76,9 +77,21 @@ static void __init openrd_base_init(void)
> > 
> >  static int __init openrd_base_pci_init(void)
> >  {
> > +       u32 cpu_config_reg;
> > +       void __iomem *base;
> > +
> > +       base = ioremap(0xf1020100, 4);
> > +       if (base)
> > +       {
> > +               cpu_config_reg = readl(base);
> > +               cpu_config_reg &= ~(1 << 2);
> > +               writel(cpu_config_reg, base);
> > +               printk("register 0x20100: %x\n", readl(base));
> > +       }
> > +       iounmap(base);
> >         if (machine_is_openrd_base())
> >                 kirkwood_pcie_init();
> > -
> > +
> >         return 0;
> >   }
> >  subsys_initcall(openrd_base_pci_init);
> > ----
> > to allow my system to boot up
> 
> What's likely happening is that your boot loader either enables or
> does not disable this bit, and the 2.6.22.18 kernel disables the bit,
> while the upstream kernel leaves the bit untouched.
> 
> What this bit does is to decide whether or not aborts on the PCI
> interface are translated into processor aborts.  It's not really
> necessary to have this enabled, as the transaction will return
> 0xffffffff to the CPU anyway, which is then handled appropriately
> as well.
> 
> What uboot version are you using?  The uboot versions I have on my
> Kirkwood boards all jump to the OS with this bit already cleared.
> Perhaps we should clear it explicitly from Linux.
> 
I'm using prafullas latest u-boot from u-boot-marvell at denx.de.
What u-boot are you using?
> 
> > and after succesfully boot my pci device isn't reachable - without any errors / warnings!
> > I've enabled printk (echo 8 > /proc/sys/kernel/printk) and still no console output.
> 
> The device shows up on the bus, the bridge primary/secondary bus numbers
> look good, and the secondary memory address range on the bridge looks
> properly programmed.  It all looks good to me.
> 
> What do you mean by 'not reachable'?  I'm guessing that you're trying
> to access the memory BAR on the 01:08.0 device directly from userland
> by reading from the e000_0000 - e3ff_ffff address range from /dev/mem
> and only getting 0xffffffff back because you don't have an actual kernel
> driver for this FPGA board and thus you're not calling pci_enable_device()
> on your device, causing MEM/IO decoding not to have been enabled on
> the device as seems to be the case in your dump?
> 
No - I've allready a minimal driver here - I will check. It looks like I've forgotten to load it for the dump, sorry :(
Please see my driver probe below
(the FPGA supports simple memory mapped LEDs at the moment):

#define ARTISTA_MEM_SIZE (1024*1024)		/* I don't want to use all 64MBytes write for testing*/


static int probe(struct pci_dev *dev, const struct pci_device_id *id)
{
	int result;
	int i=0, y=0;

	printk(KERN_DEBUG "anetfb probe() called\n");
	
	if ((result = pci_enable_device(dev)) < 0) {
		return result;
	}

	memstart = pci_resource_start(dev, 0);
	memlength = pci_resource_len(dev, 0);

	printk(KERN_DEBUG "anetfb probe() request_mem_region() for BAR0\n");
	printk(KERN_DEBUG "anetfb probe() BAR0 start address:%lx BAR0 length:%lx\n", memstart, memlength);
	if (!request_mem_region(memstart, ARTISTA_MEM_SIZE, "mv_video MMIO"))
	{
		printk(KERN_DEBUG "anetfb probe() request_mem_region() for BAR0 failed\n");
		return -EIO;
	}

	iomem_pointer=ioremap_nocache(memstart, ARTISTA_MEM_SIZE);

	printk(KERN_DEBUG "anetfb trying to access iomem@address %x with length:%d...\n", iomem_pointer, ARTISTA_MEM_SIZE);

	iowrite32(0x0, iomem_pointer);
	printk(KERN_DEBUG "read register value via readl: %x\n", ioread32(iomem_pointer));
	printk(KERN_DEBUG"write 1s to mem space...\n");
	iowrite32(0xffffffff, iomem_pointer);
	printk(KERN_DEBUG "read register value via readl: %x\n", ioread32(iomem_pointer));

}

This driver code works fine with 2.6.22.18 and doesn't with latest git kernels (back to 2.6.30 where sheevaplug support starts).
With latest git kernels everytime 0xffffffff is returned by read calls.

> For what it's worth, I have various mv78xx0 and Kirkwood boards with
> 88SB2211 PCIe-to-PCI bridges on them, and they all work fine, and
> the devices behind those bridges do too.
> 
Do you use marvell 88SB2211 PCIe-to-PCI bridge evalboards? I use this one: DB-88SB2211-B-Pex2PCI


> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!)

[Dieter Kiermaier]

Am Mittwoch 11 November 2009 16:21:34 schrieb Lennert Buytenhek:
> On Wed, Nov 11, 2009 at 06:29:22AM -0800, Dieter Kiermaier wrote:
> 
> > > > > Ronan Shitrit from marvell gave me the information to clear bit 2 of physical
> > > > > address 0xf1020100 to enable bus scanning.
> > > > > I don't know what this really does but it helped to get my kernel up and running.
> > > >
> > 
> > I've tested again:
> > 
> > with 2.6.22.18 marvell stock kernel my fpga behing the pcie->pci bridge works as expected.
> > Here is the output of lscpi:
> > 
> > sh-3.2# uname -a
> > Linux DB88FXX81 2.6.22.18 #1 Wed Apr 22 20:31:28 IST 2009 armv5tejl GNU/Linux
> > sh-3.2#
> > 
> > 
> > sh-3.2# lspci -vv
> > 00:01.0 Class 0604: Device 11ab:2211 (rev 01)
> >         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> >         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >         Latency: 0, Cache Line Size: 32 bytes
> >         Bus: primary=00, secondary=01, subordinate=01, sec-latency=240
> >         I/O behind bridge: 00001000-00004fff
> >         Memory behind bridge: e8000000-ebffffff
> >         Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
> >         BridgeCtl: Parity+ SERR- NoISA+ VGA- MAbort+ >Reset- FastB2B-
> >                 PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
> >         Capabilities: [40] Power Management version 2
> >                 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
> >                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
> >                 Bridge: PM- B3+
> >         Capabilities: [48] Express (v1) PCI/PCI-X Bridge, MSI 00
> >                 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
> >                         ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
> >                 DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
> >                         RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
> >                         MaxPayload 128 bytes, MaxReadReq 512 bytes
> >                 DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq- AuxPwr- TransPend-
> >                 LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <256ns, L1 unlimited
> >                         ClockPM- Surprise- LLActRep- BwNot-
> >                 LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
> >                         ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> >                 LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> >         Capabilities: [100] #11ab
> >         Capabilities: [221] #f0e8
> >         Capabilities: [f1e] #2a0
> >         Capabilities: [e80] #187
> > 
> > 01:08.0 Class ff00: Device 1731:0101 (rev 10)
> >         Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
> >         Status: Cap- 66MHz- UDF- FastB2B- ParErr+ DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >         Region 0: Memory at e8000000 (32-bit, non-prefetchable) [size=64M]
> >         Region 1: I/O ports at <ignored>
> >         Kernel driver in use: ArtistaNET-III frame buffer driver
> > 
> > 
> > If I switch to mainline linux kernel (there is no difference between openrd, vanilla, orion.git kernels!) my problems start:
> > I have to patch openrd_init() as follows:
> > 
> > diff --git a/arch/arm/mach-kirkwood/openrd_base-setup.c b/arch/arm/mach-kirkwood/openrd_base-setup.c
> > index 77617c7..670312b 100644
> > --- a/arch/arm/mach-kirkwood/openrd_base-setup.c
> > +++ b/arch/arm/mach-kirkwood/openrd_base-setup.c
> > @@ -14,6 +14,7 @@
> >  #include <linux/mtd/partitions.h>
> >  #include <linux/ata_platform.h>
> >  #include <linux/mv643xx_eth.h>
> > +#include <asm/io.h>
> >  #include <asm/mach-types.h>
> >  #include <asm/mach/arch.h>
> >  #include <mach/kirkwood.h>
> > @@ -76,9 +77,21 @@ static void __init openrd_base_init(void)
> > 
> >  static int __init openrd_base_pci_init(void)
> >  {
> > +       u32 cpu_config_reg;
> > +       void __iomem *base;
> > +
> > +       base = ioremap(0xf1020100, 4);
> > +       if (base)
> > +       {
> > +               cpu_config_reg = readl(base);
> > +               cpu_config_reg &= ~(1 << 2);
> > +               writel(cpu_config_reg, base);
> > +               printk("register 0x20100: %x\n", readl(base));
> > +       }
> > +       iounmap(base);
> >         if (machine_is_openrd_base())
> >                 kirkwood_pcie_init();
> > -
> > +
> >         return 0;
> >   }
> >  subsys_initcall(openrd_base_pci_init);
> > ----
> > to allow my system to boot up
> 
> What's likely happening is that your boot loader either enables or
> does not disable this bit, and the 2.6.22.18 kernel disables the bit,
> while the upstream kernel leaves the bit untouched.
> 
> What this bit does is to decide whether or not aborts on the PCI
> interface are translated into processor aborts.  It's not really
> necessary to have this enabled, as the transaction will return
> 0xffffffff to the CPU anyway, which is then handled appropriately
> as well.
> 
> What uboot version are you using?  The uboot versions I have on my
> Kirkwood boards all jump to the OS with this bit already cleared.
> Perhaps we should clear it explicitly from Linux.
> 
> 
> > and after succesfully boot my pci device isn't reachable - without any errors / warnings!
> > I've enabled printk (echo 8 > /proc/sys/kernel/printk) and still no console output.
> 
> The device shows up on the bus, the bridge primary/secondary bus numbers
> look good, and the secondary memory address range on the bridge looks
> properly programmed.  It all looks good to me.
> 
> What do you mean by 'not reachable'?  I'm guessing that you're trying
> to access the memory BAR on the 01:08.0 device directly from userland
> by reading from the e000_0000 - e3ff_ffff address range from /dev/mem
> and only getting 0xffffffff back because you don't have an actual kernel
> driver for this FPGA board and thus you're not calling pci_enable_device()
> on your device, causing MEM/IO decoding not to have been enabled on
> the device as seems to be the case in your dump?
> 

I've tested once again (I really forgotten to load my kernel module at the last mail, sorry again!).
Here you can see the result (with a non working driver!);

sh-3.2# uname -a
Linux DB88FXX81 2.6.32-rc6-00014-g92bdfb9-dirty #3 PREEMPT Wed Nov 11 15:10:49 CET 2009 armv5tel GNU/Linux
sh-3.2# cat /proc/iomem
00000000-0fffffff : System RAM
  00026000-00464fff : Kernel text
  0048a000-00579c7b : Kernel data
10000000-1fffffff : System RAM
e0000000-e7ffffff : PCIe Memory Space
  e0000000-e3ffffff : PCI Bus 0000:01
    e0000000-e3ffffff : 0000:01:08.0
      e0000000-e00fffff : mv_video MMIO
f1010300-f101030f : rtc-mv
  f1010300-f101030f : rtc-mv
f1011000-f101101f : mv64xxx_i2c.0
  f1011000-f101101f : mv64xxx_i2c adapter
f1012000-f10120ff : serial8250.0
  f1012000-f101201f : serial
f1030000-f103ffff : regs
f1050000-f1050fff : orion-ehci.0
  f1050000-f1050fff : ehci_hcd
f1060800-f10608ff : xor 0 low
f1060900-f10609ff : xor 1 low
f1060a00-f1060aff : xor 0 high
f1060b00-f1060bff : xor 1 high
f1072000-f1073fff : ge00 base
f1080000-f1084fff : sata base
f1090000-f10903ff : mvsdio
  f1090000-f10903ff : mvsdio
f3000000-f30003ff : orion_nand
f4000000-f40007ff : sram
sh-3.2# lspci -vv
00:00.0 Class 0580: Device 11ab:6281 (rev 02)
        Subsystem: Device 11ab:11ab
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 32 bytes
        Interrupt: pin A routed to IRQ 9
        Region 0: Memory@<ignored> (64-bit, prefetchable)
        Capabilities: [40] Power Management version 3
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [60] Express (v1) Root Port (Slot-), MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <256ns, L1 <1us
                        ExtTag- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <256ns, L1 unlimited
                        ClockPM+ Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
                RootCap: CRSVisible-
                RootSta: PME ReqID 0000, PMEStatus- PMEPending-
        Capabilities: [100] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-

00:01.0 Class 0604: Device 11ab:2211 (rev 01)
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR+ <PERR- INTx-
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: 00001000-00004fff
        Memory behind bridge: e0000000-e3ffffff
        Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
        BridgeCtl: Parity+ SERR- NoISA- VGA- MAbort+ >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [40] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
                Bridge: PM- B3+
        Capabilities: [48] Express (v1) PCI/PCI-X Bridge, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, Latency L0 <256ns, L1 unlimited
                        ClockPM- Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        Capabilities: [100] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                AERCap: First Error Pointer: 14, GenCap- CGenEn- ChkCap- ChkEn-

01:08.0 Class ff00: Device 1731:0101 (rev 10)
        Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B- ParErr+ DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Region 0: Memory at e0000000 (32-bit, non-prefetchable) [size=64M]
        Region 1: I/O ports@<unassigned> [disabled]
        Kernel driver in use: ArtistaNET-III frame buffer driver

sh-3.2#
sh-3.2# lsmod
mv_video 3616 0 - Live 0xbf000000
sh-3.2#

Again, many thanks for helping!

Dieter


> For what it's worth, I have various mv78xx0 and Kirkwood boards with
> 88SB2211 PCIe-to-PCI bridges on them, and they all work fine, and
> the devices behind those bridges do too.
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!)

[Lennert Buytenhek]

On Wed, Nov 11, 2009 at 05:50:35PM +0100, Dieter Kiermaier wrote:

> 00:01.0 Class 0604: Device 11ab:2211 (rev 01)
>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-

The bridge has been told not to forward MEM transactions to the secondary
side, which explains what you're seeing.

It's odd that this is happening, as normally pci_enable_bridges()
takes care of enabling I/O and MEM decoding on bridges.

What other kernel patches are you using?  And can you put a full boot
log somewhere?

orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!)

[Lennert Buytenhek]

On Wed, Nov 11, 2009 at 05:43:06PM +0100, Dieter Kiermaier wrote:

> > What's likely happening is that your boot loader either enables or
> > does not disable this bit, and the 2.6.22.18 kernel disables the bit,
> > while the upstream kernel leaves the bit untouched.
> > 
> > What this bit does is to decide whether or not aborts on the PCI
> > interface are translated into processor aborts.  It's not really
> > necessary to have this enabled, as the transaction will return
> > 0xffffffff to the CPU anyway, which is then handled appropriately
> > as well.
> > 
> > What uboot version are you using?  The uboot versions I have on my
> > Kirkwood boards all jump to the OS with this bit already cleared.
> > Perhaps we should clear it explicitly from Linux.
> 
> I'm using prafullas latest u-boot from u-boot-marvell at denx.de.
> What u-boot are you using?

I think my boards are still on the stock uboot version.


> > > and after succesfully boot my pci device isn't reachable - without any errors / warnings!
> > > I've enabled printk (echo 8 > /proc/sys/kernel/printk) and still no console output.
> > 
> > The device shows up on the bus, the bridge primary/secondary bus numbers
> > look good, and the secondary memory address range on the bridge looks
> > properly programmed.  It all looks good to me.
> > 
> > What do you mean by 'not reachable'?  I'm guessing that you're trying
> > to access the memory BAR on the 01:08.0 device directly from userland
> > by reading from the e000_0000 - e3ff_ffff address range from /dev/mem
> > and only getting 0xffffffff back because you don't have an actual kernel
> > driver for this FPGA board and thus you're not calling pci_enable_device()
> > on your device, causing MEM/IO decoding not to have been enabled on
> > the device as seems to be the case in your dump?
> > 
> No - I've allready a minimal driver here - I will check. It looks like I've forgotten to load it for the dump, sorry :(
> Please see my driver probe below
> (the FPGA supports simple memory mapped LEDs at the moment):
> 
> #define ARTISTA_MEM_SIZE (1024*1024)		/* I don't want to use all 64MBytes write for testing*/
> 
> 
> static int probe(struct pci_dev *dev, const struct pci_device_id *id)
> {
> 	int result;
> 	int i=0, y=0;
> 
> 	printk(KERN_DEBUG "anetfb probe() called\n");
> 	
> 	if ((result = pci_enable_device(dev)) < 0) {
> 		return result;
> 	}
> 
> 	memstart = pci_resource_start(dev, 0);
> 	memlength = pci_resource_len(dev, 0);
> 
> 	printk(KERN_DEBUG "anetfb probe() request_mem_region() for BAR0\n");
> 	printk(KERN_DEBUG "anetfb probe() BAR0 start address:%lx BAR0 length:%lx\n", memstart, memlength);
> 	if (!request_mem_region(memstart, ARTISTA_MEM_SIZE, "mv_video MMIO"))
> 	{
> 		printk(KERN_DEBUG "anetfb probe() request_mem_region() for BAR0 failed\n");
> 		return -EIO;
> 	}
> 
> 	iomem_pointer=ioremap_nocache(memstart, ARTISTA_MEM_SIZE);

Here you should probably use pci_iomap().


> This driver code works fine with 2.6.22.18 and doesn't with latest
> git kernels (back to 2.6.30 where sheevaplug support starts).
> With latest git kernels everytime 0xffffffff is returned by read calls.

The fact that the bridge has transaction forwarding disables explains
this -- see my other reply.


> > For what it's worth, I have various mv78xx0 and Kirkwood boards with
> > 88SB2211 PCIe-to-PCI bridges on them, and they all work fine, and
> > the devices behind those bridges do too.
> 
> Do you use marvell 88SB2211 PCIe-to-PCI bridge evalboards? I use this
> one: DB-88SB2211-B-Pex2PCI

No, the mv78xx0 and Kirkwood development boards (ATX sized) come with
88SB2211 bridges integrated on the board, which is what I use.

orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!)

[Dieter Kiermaier]

Am Mittwoch 11 November 2009 17:53:53 schrieb Lennert Buytenhek:
> On Wed, Nov 11, 2009 at 05:50:35PM +0100, Dieter Kiermaier wrote:
> 
> > 00:01.0 Class 0604: Device 11ab:2211 (rev 01)
> >         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 
> The bridge has been told not to forward MEM transactions to the secondary
> side, which explains what you're seeing.

Where can I see that the bridge doesn't want to forward mem transactions?
Please could you point me to the entry?
> 
> It's odd that this is happening, as normally pci_enable_bridges()
> takes care of enabling I/O and MEM decoding on bridges.
> 
> What other kernel patches are you using?  And can you put a full boot
> log somewhere?
I don't use other kernel patches (except changed openrd_base_pci_init() as described in my last post).

You can find my boot log at
http://pastebin.com/m6e118098

Dieter
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!)

[Dieter Kiermaier]

Am Mittwoch 11 November 2009 17:59:27 schrieb Lennert Buytenhek:
> On Wed, Nov 11, 2009 at 05:43:06PM +0100, Dieter Kiermaier wrote:
> 
> > > What's likely happening is that your boot loader either enables or
> > > does not disable this bit, and the 2.6.22.18 kernel disables the bit,
> > > while the upstream kernel leaves the bit untouched.
> > > 
> > > What this bit does is to decide whether or not aborts on the PCI
> > > interface are translated into processor aborts.  It's not really
> > > necessary to have this enabled, as the transaction will return
> > > 0xffffffff to the CPU anyway, which is then handled appropriately
> > > as well.
> > > 
> > > What uboot version are you using?  The uboot versions I have on my
> > > Kirkwood boards all jump to the OS with this bit already cleared.
> > > Perhaps we should clear it explicitly from Linux.
> > 
> > I'm using prafullas latest u-boot from u-boot-marvell at denx.de.
> > What u-boot are you using?
> 
> I think my boards are still on the stock uboot version.
> 

Very strange - with stock u-boot everything works!?!

Hm, now my problem changed to not stick on stock u-boot :)

How can we go on to find out what is different?

Dieter



> 
> > > > and after succesfully boot my pci device isn't reachable - without any errors / warnings!
> > > > I've enabled printk (echo 8 > /proc/sys/kernel/printk) and still no console output.
> > > 
> > > The device shows up on the bus, the bridge primary/secondary bus numbers
> > > look good, and the secondary memory address range on the bridge looks
> > > properly programmed.  It all looks good to me.
> > > 
> > > What do you mean by 'not reachable'?  I'm guessing that you're trying
> > > to access the memory BAR on the 01:08.0 device directly from userland
> > > by reading from the e000_0000 - e3ff_ffff address range from /dev/mem
> > > and only getting 0xffffffff back because you don't have an actual kernel
> > > driver for this FPGA board and thus you're not calling pci_enable_device()
> > > on your device, causing MEM/IO decoding not to have been enabled on
> > > the device as seems to be the case in your dump?
> > > 
> > No - I've allready a minimal driver here - I will check. It looks like I've forgotten to load it for the dump, sorry :(
> > Please see my driver probe below
> > (the FPGA supports simple memory mapped LEDs at the moment):
> > 
> > #define ARTISTA_MEM_SIZE (1024*1024)		/* I don't want to use all 64MBytes write for testing*/
> > 
> > 
> > static int probe(struct pci_dev *dev, const struct pci_device_id *id)
> > {
> > 	int result;
> > 	int i=0, y=0;
> > 
> > 	printk(KERN_DEBUG "anetfb probe() called\n");
> > 	
> > 	if ((result = pci_enable_device(dev)) < 0) {
> > 		return result;
> > 	}
> > 
> > 	memstart = pci_resource_start(dev, 0);
> > 	memlength = pci_resource_len(dev, 0);
> > 
> > 	printk(KERN_DEBUG "anetfb probe() request_mem_region() for BAR0\n");
> > 	printk(KERN_DEBUG "anetfb probe() BAR0 start address:%lx BAR0 length:%lx\n", memstart, memlength);
> > 	if (!request_mem_region(memstart, ARTISTA_MEM_SIZE, "mv_video MMIO"))
> > 	{
> > 		printk(KERN_DEBUG "anetfb probe() request_mem_region() for BAR0 failed\n");
> > 		return -EIO;
> > 	}
> > 
> > 	iomem_pointer=ioremap_nocache(memstart, ARTISTA_MEM_SIZE);
> 
> Here you should probably use pci_iomap().
> 
> 
> > This driver code works fine with 2.6.22.18 and doesn't with latest
> > git kernels (back to 2.6.30 where sheevaplug support starts).
> > With latest git kernels everytime 0xffffffff is returned by read calls.
> 
> The fact that the bridge has transaction forwarding disables explains
> this -- see my other reply.
> 
> 
> > > For what it's worth, I have various mv78xx0 and Kirkwood boards with
> > > 88SB2211 PCIe-to-PCI bridges on them, and they all work fine, and
> > > the devices behind those bridges do too.
> > 
> > Do you use marvell 88SB2211 PCIe-to-PCI bridge evalboards? I use this
> > one: DB-88SB2211-B-Pex2PCI
> 
> No, the mv78xx0 and Kirkwood development boards (ATX sized) come with
> 88SB2211 bridges integrated on the board, which is what I use.
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!)

[Lennert Buytenhek]

On Wed, Nov 11, 2009 at 06:11:07PM +0100, Dieter Kiermaier wrote:

> > > 00:01.0 Class 0604: Device 11ab:2211 (rev 01)
> > >         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> > 
> > The bridge has been told not to forward MEM transactions to the secondary
> > side, which explains what you're seeing.
> 
> Where can I see that the bridge doesn't want to forward mem transactions?
> Please could you point me to the entry?

It's the "Mem" one (surprise! :)).  The minus in "Mem-" indicates that
the bit is disabled in the Control register.

The PCIe bridge spec says this about this bit:

	Controls the bridge?s response as a target to memory accesses
	on the primary interface that address a device that resides
	behind the bridge in both the non-prefetchable and prefetchable
	memory ranges (see Section 3.1) or targets a memory-mapped
	location within the bridge itself (see Section 5.1.2.1).


> > It's odd that this is happening, as normally pci_enable_bridges()
> > takes care of enabling I/O and MEM decoding on bridges.
> > 
> > What other kernel patches are you using?  And can you put a full boot
> > log somewhere?
>
> I don't use other kernel patches (except changed openrd_base_pci_init()
> as described in my last post).

Can you apply this patch and make another boot log?



diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index cef28a7..fb06be0 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -187,9 +187,14 @@ void pci_enable_bridges(struct pci_bus *bus)
 	struct pci_dev *dev;
 	int retval;
 
+	printk(KERN_INFO "pci_enable_bridges(%p)\n", bus);
+
 	list_for_each_entry(dev, &bus->devices, bus_list) {
+		printk(KERN_INFO "..considering device %p\n", dev);
 		if (dev->subordinate) {
+			printk(KERN_INFO "..has subordinate\n", dev);
 			if (!pci_is_enabled(dev)) {
+				printk(KERN_INFO "..enabling\n", dev);
 				retval = pci_enable_device(dev);
 				pci_set_master(dev);
 			}
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index b636e24..5ab9028 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -599,6 +599,8 @@ pci_assign_unassigned_resources(void)
 {
 	struct pci_bus *bus;
 
+	printk(KERN_INFO "pci_assign_unassigned_resources()\n");
+
 	/* Depth first, calculate sizes and alignments of all
 	   subordinate buses. */
 	list_for_each_entry(bus, &pci_root_buses, node) {
@@ -606,6 +608,7 @@ pci_assign_unassigned_resources(void)
 	}
 	/* Depth last, allocate resources and update the hardware. */
 	list_for_each_entry(bus, &pci_root_buses, node) {
+		printk(KERN_INFO "..allocating resources for bus %p\n", bus);
 		pci_bus_assign_resources(bus);
 		pci_enable_bridges(bus);
 	}

orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!)

[Dieter Kiermaier]

Am Mittwoch 11 November 2009 18:19:16 schrieb Lennert Buytenhek:
> On Wed, Nov 11, 2009 at 06:11:07PM +0100, Dieter Kiermaier wrote:
> 
> > > > 00:01.0 Class 0604: Device 11ab:2211 (rev 01)
> > > >         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> > > 
> > > The bridge has been told not to forward MEM transactions to the secondary
> > > side, which explains what you're seeing.
> > 
> > Where can I see that the bridge doesn't want to forward mem transactions?
> > Please could you point me to the entry?
> 
> It's the "Mem" one (surprise! :)).  The minus in "Mem-" indicates that
> the bit is disabled in the Control register.
> 
> The PCIe bridge spec says this about this bit:
> 
> 	Controls the bridge?s response as a target to memory accesses
> 	on the primary interface that address a device that resides
> 	behind the bridge in both the non-prefetchable and prefetchable
> 	memory ranges (see Section 3.1) or targets a memory-mapped
> 	location within the bridge itself (see Section 5.1.2.1).
> 
> 
> > > It's odd that this is happening, as normally pci_enable_bridges()
> > > takes care of enabling I/O and MEM decoding on bridges.
> > > 
> > > What other kernel patches are you using?  And can you put a full boot
> > > log somewhere?
> >
> > I don't use other kernel patches (except changed openrd_base_pci_init()
> > as described in my last post).
> 
> Can you apply this patch and make another boot log?
> 
Bootlog from git u-boot:
http://pastebin.com/d129d38a8

Bootlog from stock u-boot:
http://pastebin.com/d2b91d601

But it looks as the printk's will never reached!

Dieter

> 
> diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
> index cef28a7..fb06be0 100644
> --- a/drivers/pci/bus.c
> +++ b/drivers/pci/bus.c
> @@ -187,9 +187,14 @@ void pci_enable_bridges(struct pci_bus *bus)
>  	struct pci_dev *dev;
>  	int retval;
>  
> +	printk(KERN_INFO "pci_enable_bridges(%p)\n", bus);
> +
>  	list_for_each_entry(dev, &bus->devices, bus_list) {
> +		printk(KERN_INFO "..considering device %p\n", dev);
>  		if (dev->subordinate) {
> +			printk(KERN_INFO "..has subordinate\n", dev);
>  			if (!pci_is_enabled(dev)) {
> +				printk(KERN_INFO "..enabling\n", dev);
>  				retval = pci_enable_device(dev);
>  				pci_set_master(dev);
>  			}
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index b636e24..5ab9028 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -599,6 +599,8 @@ pci_assign_unassigned_resources(void)
>  {
>  	struct pci_bus *bus;
>  
> +	printk(KERN_INFO "pci_assign_unassigned_resources()\n");
> +
>  	/* Depth first, calculate sizes and alignments of all
>  	   subordinate buses. */
>  	list_for_each_entry(bus, &pci_root_buses, node) {
> @@ -606,6 +608,7 @@ pci_assign_unassigned_resources(void)
>  	}
>  	/* Depth last, allocate resources and update the hardware. */
>  	list_for_each_entry(bus, &pci_root_buses, node) {
> +		printk(KERN_INFO "..allocating resources for bus %p\n", bus);
>  		pci_bus_assign_resources(bus);
>  		pci_enable_bridges(bus);
>  	}
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

[PATCH] ARM: enable PCI bridges after assigning resources (was orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!))

[Maxime Bizon]

On Wed, 2009-11-11 at 18:19 +0100, Lennert Buytenhek wrote:

Hi Lennert & Dieter,

> It's the "Mem" one (surprise! :)).  The minus in "Mem-" indicates that
> the bit is disabled in the Control register.

I sent this patch twice on arm mailing list but it went unnoticed. I
think you're hitting the exact same problem.



When kernel is doing PCI resource assignment on ARM, nothing ever
enables PCI bridges, preventing any device behind them from working. The
following patch fixes this.


Signed-off-by: Maxime Bizon <mbizon@freebox.fr>

--- linux-2.6.29/arch/arm/kernel/bios32.c.old	2009-04-16 22:43:18.000000000 +0200
+++ linux-2.6.29/arch/arm/kernel/bios32.c	2009-04-16 22:43:25.000000000 +0200
@@ -580,6 +580,7 @@
 			 * Assign resources.
 			 */
 			pci_bus_assign_resources(bus);
+			pci_enable_bridges(bus);
 		}
 
 		/*




-- 
Maxime

[PATCH] ARM: enable PCI bridges after assigning resources (was orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!))

[Dieter Kiermaier]

Hi Maxime, hi Lennert,

> 
> On Wed, 2009-11-11 at 18:19 +0100, Lennert Buytenhek wrote:
> 
> Hi Lennert & Dieter,
> 
> > It's the "Mem" one (surprise! :)).  The minus in "Mem-" indicates that
> > the bit is disabled in the Control register.
> 
> I sent this patch twice on arm mailing list but it went unnoticed. I
> think you're hitting the exact same problem.
> 
> 
> 
> When kernel is doing PCI resource assignment on ARM, nothing ever
> enables PCI bridges, preventing any device behind them from working. The
> following patch fixes this.
> 

You hit it :)
With your patch my FPGA works behind the bridge together with the marvell git u-boot.

@Lennert: now your debug printk's are shown during boot up!

To see things clear:
Is it possible that the marvell stock u-boot enables all devices before it starts the kernel
and that's the reason why it worked if I booted the kernel from stock u-boot and it didn't
work with the git u-boot?

Here is my:
Acked-by: Dieter Kiermaier <dk-arm-linux@gmx.de>

Additionally I will cook up a patch which clears the bit for M-Bus error reporting against most recent kernel
during the course of the day.

Many thanks to all for helping,
Dieter


> 
> Signed-off-by: Maxime Bizon <mbizon@freebox.fr>
> 
> --- linux-2.6.29/arch/arm/kernel/bios32.c.old	2009-04-16 22:43:18.000000000 +0200
> +++ linux-2.6.29/arch/arm/kernel/bios32.c	2009-04-16 22:43:25.000000000 +0200
> @@ -580,6 +580,7 @@
>  			 * Assign resources.
>  			 */
>  			pci_bus_assign_resources(bus);
> +			pci_enable_bridges(bus);
>  		}
>  
>  		/*
> 
> 
> 
> 

[PATCH] ARM: enable PCI bridges after assigning resources (was orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!))

[Simon Kagstrom]

On Thu, 12 Nov 2009 08:23:44 +0100
Dieter Kiermaier <dk-arm-linux@gmx.de> wrote:

> > When kernel is doing PCI resource assignment on ARM, nothing ever
> > enables PCI bridges, preventing any device behind them from working. The
> > following patch fixes this.
> 
> You hit it :)
> With your patch my FPGA works behind the bridge together with the marvell git u-boot.
>
> To see things clear:
> Is it possible that the marvell stock u-boot enables all devices before it starts the kernel
> and that's the reason why it worked if I booted the kernel from stock u-boot and it didn't
> work with the git u-boot?

The upstream (I suppose this is what you mean with "git"!) U-boot
doesn't touch the PCIe configuration at all as far as I know, but
probably the old Marvell U-boot does, and I guess that's the difference
we're seeing here.

Anyway, if U-boot isn't using PCIe, it shouldn't touch it, so it's good
to get the fixes into the kernel.

// Simon

[PATCH] ARM: enable PCI bridges after assigning resources (was orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!))

[Dieter Kiermaier]

Am Donnerstag 12 November 2009 08:31:26 schrieb Simon Kagstrom:
> On Thu, 12 Nov 2009 08:23:44 +0100
> Dieter Kiermaier <dk-arm-linux@gmx.de> wrote:
> 
> > > When kernel is doing PCI resource assignment on ARM, nothing ever
> > > enables PCI bridges, preventing any device behind them from working. The
> > > following patch fixes this.
> > 
> > You hit it :)
> > With your patch my FPGA works behind the bridge together with the marvell git u-boot.
> >
> > To see things clear:
> > Is it possible that the marvell stock u-boot enables all devices before it starts the kernel
> > and that's the reason why it worked if I booted the kernel from stock u-boot and it didn't
> > work with the git u-boot?
> 
> The upstream (I suppose this is what you mean with "git"!) U-boot
> doesn't touch the PCIe configuration at all as far as I know, but
> probably the old Marvell U-boot does, and I guess that's the difference
> we're seeing here.
> 
Yes i meant upstream u-boot (u-boot-marvell tree from denx git repository).

> Anyway, if U-boot isn't using PCIe, it shouldn't touch it, so it's good
> to get the fixes into the kernel.

Yes. IMHO, too.

> 
> // Simon
> 

[PATCH] ARM: enable PCI bridges after assigning resources (was orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!))

[Lennert Buytenhek]

On Thu, Nov 12, 2009 at 02:35:09AM +0100, Maxime Bizon wrote:

> Hi Lennert & Dieter,

Hi Maxime,


> > It's the "Mem" one (surprise! :)).  The minus in "Mem-" indicates that
> > the bit is disabled in the Control register.
> 
> I sent this patch twice on arm mailing list but it went unnoticed. I
> think you're hitting the exact same problem.
> 
> 
> 
> When kernel is doing PCI resource assignment on ARM, nothing ever
> enables PCI bridges, preventing any device behind them from working. The
> following patch fixes this.

As normal devices aren't enabled (in the MEM/IO decoding sense) by
default on boot either, I wonder if we should just have pci_enable_device()
walk through the PCI tree up to the root, enabling each of the bridges
on its path?

(pci_disable_device() doesn't disable MEM/IO decoding (just the initiator
bit), so there's no symmetry problem with not disabling those bridges
in pci_disable_device() again.)

[PATCH] ARM: enable PCI bridges after assigning resources (was orion/kirkwood pcie issue still open with 2.6.32-rc6 (marvell stock 2.6.22.18 works!))

[Maxime Bizon]

On Thu, 2009-11-12 at 20:56 +0100, Lennert Buytenhek wrote:

> As normal devices aren't enabled (in the MEM/IO decoding sense) by
> default on boot either, I wonder if we should just have pci_enable_device()
> walk through the PCI tree up to the root, enabling each of the bridges
> on its path?

My patch is what (at least) MIPS pci code does, but your solution sounds
fine too.

-- 
Maxime