[PATCH] PCI: PCI_PROBE_ONLY clean-up

[PATCH] PCI: PCI_PROBE_ONLY clean-up

[Bjorn Helgaas]

Hi Ralf, et al,

Lorenzo is doing some nice cleanup of the PCI_PROBE_ONLY case.  Previous
situation:

  - PCI_PROBE_ONLY: all PCI BARs and windows are immutable, and we don't
    put those resources in the iomem_resource tree, and they don't show up
    in /proc/iomem.

  - !PCI_PROBE_ONLY: we use whatever existing BAR settings may have been
    done by firmware, we change them if we find conflicts, and we insert
    them all in the iomem_resource tree.

Lorenzo is changing the PCI_PROBE_ONLY case so the BARs and windows remain
immutable, but we insert the resources into the iomem_resource tree.

The ideal thing would be to remove the use of PCI_PROBE_ONLY completely,
and allow Linux to program BARs as necessary.  If the firmware *has*
programmed the BARs, we don't change them unless we find something broken,
so in most cases PCI_PROBE_ONLY is unnecessary.

There are several MIPS platforms (bcm1480, ip27, sb1250, virtio_guest, xlp,
xlr) that set PCI_PROBE_ONLY for reasons I don't know.  These were added
by:

  bcm1480
    Andrew Isaacson <adi@broadcom.com>
    dc41f94f7709 ("Support for the BCM1480 on-chip PCI-X bridge.")

  ip27
    Thomas Bogendoerfer <tsbogend@alpha.franken.de>
    96173a6c4ebc ("[MIPS] IP27: misc fixes")

  virtio
    David Daney <david.daney@cavium.com>
    ae6e7e635c2c ("MIPS: paravirt: Add pci controller for virtio")

  xlp
    Ganesan Ramalingam <ganesanr@netlogicmicro.com>
    9bac624b0fe0 ("MIPS: Netlogic: XLP PCIe controller support.")

  xlr
    Jayachandran C <jayachandranc@netlogicmicro.com>
    9b130f8004e5 ("MIPS: XLR, XLS: Add PCI support.")

I suspect some of these uses are copied and not actually necessary.  If we
need to keep them, that's fine, but I would like to at least insert the
resources into the tree, as with the following patch.

Any thoughts on this would be appreciated.

The whole series as I propose to merge it is on my pci/resource branch
at [2].

[1] http://lkml.kernel.org/r/1465383890-13538-1-git-send-email-lorenzo.pieralisi@arm.com
[2] https://git.kernel.org/cgit/linux/kernel/git/helgaas/pci.git/log/?h=pci/resource

---

Bjorn Helgaas (1):
      MIPS/PCI: Claim bus resources on PCI_PROBE_ONLY set-ups


 arch/mips/pci/pci.c |    9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

[PATCH] MIPS/PCI: Claim bus resources on PCI_PROBE_ONLY set-ups

[Bjorn Helgaas]

We claim PCI BAR and bridge window resources in pci_bus_assign_resources(),
but when PCI_PROBE_ONLY is set, we treat those resources as immutable and
don't call pci_bus_assign_resources(), so the resources aren't put in the
resource tree.

When the resources aren't in the tree, they don't show up in /proc/iomem,
we can't detect conflicts, and we need special cases elsewhere for
PCI_PROBE_ONLY or resources without a parent pointer.

Claim all PCI BAR and window resources in the PCI_PROBE_ONLY case.

If a PCI_PROBE_ONLY platform assigns conflicting resources, Linux can't fix
the conflicts.  Previously we didn't notice the conflicts, but now we will,
which may expose new failures.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/mips/pci/pci.c |    9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/arch/mips/pci/pci.c b/arch/mips/pci/pci.c
index 5717384..b4c02f2 100644
--- a/arch/mips/pci/pci.c
+++ b/arch/mips/pci/pci.c
@@ -112,7 +112,14 @@ static void pcibios_scanbus(struct pci_controller *hose)
 		need_domain_info = 1;
 	}
 
-	if (!pci_has_flag(PCI_PROBE_ONLY)) {
+	/*
+	 * We insert PCI resources into the iomem_resource and
+	 * ioport_resource trees in either pci_bus_claim_resources()
+	 * or pci_bus_assign_resources().
+	 */
+	if (pci_has_flag(PCI_PROBE_ONLY)) {
+		pci_bus_claim_resources(bus);
+	} else {
 		pci_bus_size_bridges(bus);
 		pci_bus_assign_resources(bus);
 	}

Re: [PATCH] MIPS/PCI: Claim bus resources on PCI_PROBE_ONLY set-ups

[Joshua Kinard]

On 06/23/2016 18:16, Bjorn Helgaas wrote:
> We claim PCI BAR and bridge window resources in pci_bus_assign_resources(),
> but when PCI_PROBE_ONLY is set, we treat those resources as immutable and
> don't call pci_bus_assign_resources(), so the resources aren't put in the
> resource tree.
> 
> When the resources aren't in the tree, they don't show up in /proc/iomem,
> we can't detect conflicts, and we need special cases elsewhere for
> PCI_PROBE_ONLY or resources without a parent pointer.
> 
> Claim all PCI BAR and window resources in the PCI_PROBE_ONLY case.
> 
> If a PCI_PROBE_ONLY platform assigns conflicting resources, Linux can't fix
> the conflicts.  Previously we didn't notice the conflicts, but now we will,
> which may expose new failures.
> 
> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> ---
>  arch/mips/pci/pci.c |    9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/mips/pci/pci.c b/arch/mips/pci/pci.c
> index 5717384..b4c02f2 100644
> --- a/arch/mips/pci/pci.c
> +++ b/arch/mips/pci/pci.c
> @@ -112,7 +112,14 @@ static void pcibios_scanbus(struct pci_controller *hose)
>  		need_domain_info = 1;
>  	}
>  
> -	if (!pci_has_flag(PCI_PROBE_ONLY)) {
> +	/*
> +	 * We insert PCI resources into the iomem_resource and
> +	 * ioport_resource trees in either pci_bus_claim_resources()
> +	 * or pci_bus_assign_resources().
> +	 */
> +	if (pci_has_flag(PCI_PROBE_ONLY)) {
> +		pci_bus_claim_resources(bus);
> +	} else {
>  		pci_bus_size_bridges(bus);
>  		pci_bus_assign_resources(bus);
>  	}

So I finally tested this on two SGI platforms, IP30 and IP27.  For IP30, it
works fine once I drop the "pci_set_flags(PCI_PROBE_ONLY);" call.  IP30 uses
standard virtual addressing via ioremap and the like for all of its PCI stuff,
so it was trivial to fix it.

IP27, on the other hand, is not happy with this change.  I don't know about
mainline IP27, as that's generally bitrotted and didn't boot the last time I
tried it, but under the refactored code I'm using that actually boots, this
platform still needs to bypass *all* attempts to claim or assign the PCI addresses.

I suspect the difference with IP27 is unlike normal systems, IP27 uses special
physical addresses in the 0x92xxxxxxxxxxxxxx block to address I/O devices, so
there's no proper ioremap or use of virtual addressing for I/O.  Ralf, does
this sound correct?

If I use the patches I got plus this change and set PCI_PROBE_ONLY, this is
what happens:

[   47.299511] PCI host bridge to bus 0001:00
[   47.348155] pci_bus 0001:00: root bus resource [mem
0x920000000f200000-0x920000000f9fffff]
[   47.447663] pci_bus 0001:00: root bus resource [io
0x920000000fa00000-0x920000000fbfffff]
[   47.547183] pci_bus 0001:00: root bus resource [bus 01-ff]
[   47.616581] pci 0001:00:00.0: can't claim BAR 0 [io  0xf200000-0xf2000ff]:
no compatible bridge window
[   47.726766] pci 0001:00:00.0: can't claim BAR 1 [mem 0x0f200000-0x0f200fff]:
no compatible bridge window
[   47.840937] pci 0001:00:00.0: can't claim BAR 6 [mem 0x00000000-0x0000ffff
pref]: no compatible bridge window
[   47.960317] pci 0001:00:01.0: can't claim BAR 0 [io  0xf400000-0xf4000ff]:
no compatible bridge window
[   48.072399] pci 0001:00:01.0: can't claim BAR 1 [mem 0x0f400000-0x0f400fff]:
no compatible bridge window
[   48.186560] pci 0001:00:01.0: can't claim BAR 6 [mem 0x00000000-0x0000ffff
pref]: no compatible bridge window
[   48.305957] pci 0001:00:02.0: can't claim BAR 0 [mem 0x0f600000-0x0f6fffff]:
no compatible bridge window
[   48.420098] pci 0001:00:06.0: can't claim BAR 0 [mem 0x0fa00000-0x0fafffff]:
no compatible bridge window
[   48.534294] pci 0001:00:07.0: can't claim BAR 0 [mem 0x00000000-0x00001fff]:
no compatible bridge window
[   48.648541] qla1280: QLA1040 found on PCI bus 0, dev 0
[   48.710232] PCI: Enabling device 0001:00:00.0 (0006 -> 0007)
[   48.779231] qla1280 0001:00:00.0: can't ioremap BAR 1: [mem size 0x00001000]
[   48.863145] qla1280: Unable to map I/O memory
[   48.916520] qla1280: QLA1040 found on PCI bus 0, dev 1
[   48.977298] PCI: Enabling device 0001:00:01.0 (0006 -> 0007)
[   49.046128] qla1280 0001:00:01.0: can't ioremap BAR 1: [mem size 0x00001000]
[   49.130201] qla1280: Unable to map I/O memory
[   49.183068] IOC3 0001:00:02.0: can't ioremap BAR 0: [mem size 0x00100000]
[   49.264292] ioc3: Unable to remap PCI BAR for 0001:00:02.0.
[   49.331533] IOC3 0001:00:06.0: can't ioremap BAR 0: [mem size 0x00100000]
[   49.412984] ioc3: Unable to remap PCI BAR for 0001:00:06.0.


If I disable PCI_PROBE_ONLY, then this:

[   47.588033] PCI host bridge to bus 0001:00
[   47.636649] pci_bus 0001:00: root bus resource [mem
0x920000000f200000-0x920000000f9fffff]
[   47.736144] pci_bus 0001:00: root bus resource [io
0x920000000fa00000-0x920000000fbfffff]
[   47.835621] pci_bus 0001:00: root bus resource [bus 01-ff]
[   47.905023] pci 0001:00:02.0: BAR 0: no space for [mem size 0x00100000]
[   47.982775] pci 0001:00:02.0: BAR 0: failed to assign [mem size 0x00100000]
[   48.066586] pci 0001:00:06.0: BAR 0: no space for [mem size 0x00100000]
[   48.146173] pci 0001:00:06.0: BAR 0: failed to assign [mem size 0x00100000]
[   48.229972] pci 0001:00:00.0: BAR 6: no space for [mem size 0x00010000 pref]
[   48.314783] pci 0001:00:00.0: BAR 6: failed to assign [mem size 0x00010000 pref]
[   48.403825] pci 0001:00:01.0: BAR 6: no space for [mem size 0x00010000 pref]
[   48.488655] pci 0001:00:01.0: BAR 6: failed to assign [mem size 0x00010000 pref]
[   48.577695] pci 0001:00:00.0: BAR 1: no space for [mem size 0x00001000]
[   48.657260] pci 0001:00:00.0: BAR 1: failed to assign [mem size 0x00001000]
[   48.741068] pci 0001:00:01.0: BAR 1: no space for [mem size 0x00001000]
[   48.820644] pci 0001:00:01.0: BAR 1: failed to assign [mem size 0x00001000]
[   48.904452] pci 0001:00:00.0: BAR 0: no space for [io  size 0x0100]
[   48.979840] pci 0001:00:00.0: BAR 0: failed to assign [io  size 0x0100]
[   49.059459] pci 0001:00:01.0: BAR 0: no space for [io  size 0x0100]
[   49.134855] pci 0001:00:01.0: BAR 0: failed to assign [io  size 0x0100]
[   49.214588] qla1280: QLA1040 found on PCI bus 0, dev 0
[   49.277249] qla1280 0001:00:00.0: can't ioremap BAR 1: [??? 0x00000000 flags
0x0]
[   49.366330] qla1280: Unable to map I/O memory
[   49.419832] qla1280: QLA1040 found on PCI bus 0, dev 1
[   49.481211] qla1280 0001:00:01.0: can't ioremap BAR 1: [??? 0x00000000 flags
0x0]
[   49.570560] qla1280: Unable to map I/O memory
[   49.623439] IOC3 0001:00:02.0: can't ioremap BAR 0: [??? 0x00000000 flags 0x0]
[   49.709857] ioc3: Unable to remap PCI BAR for 0001:00:02.0.
[   49.777113] IOC3 0001:00:06.0: can't ioremap BAR 0: [??? 0x00000000 flags 0x0]
[   49.863796] ioc3: Unable to remap PCI BAR for 0001:00:06.0.


The only fix I could come up with was more IP27 #ifdef hackery to retain the
original code for IP27, but use the new conditional logic for all other
platforms.  Would a patch for this be acceptable (with verbose comment), or is
there a better way to deal with IP27 that doesn't involve re-writing its I/O
addressing support entirely?

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
6144R/F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And our
lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [PATCH] MIPS/PCI: Claim bus resources on PCI_PROBE_ONLY set-ups

[Lorenzo Pieralisi]

On Mon, Oct 17, 2016 at 12:36:34AM -0400, Joshua Kinard wrote:
> On 06/23/2016 18:16, Bjorn Helgaas wrote:
> > We claim PCI BAR and bridge window resources in pci_bus_assign_resources(),
> > but when PCI_PROBE_ONLY is set, we treat those resources as immutable and
> > don't call pci_bus_assign_resources(), so the resources aren't put in the
> > resource tree.
> > 
> > When the resources aren't in the tree, they don't show up in /proc/iomem,
> > we can't detect conflicts, and we need special cases elsewhere for
> > PCI_PROBE_ONLY or resources without a parent pointer.
> > 
> > Claim all PCI BAR and window resources in the PCI_PROBE_ONLY case.
> > 
> > If a PCI_PROBE_ONLY platform assigns conflicting resources, Linux can't fix
> > the conflicts.  Previously we didn't notice the conflicts, but now we will,
> > which may expose new failures.
> > 
> > Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> > ---
> >  arch/mips/pci/pci.c |    9 ++++++++-
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> > 
> > diff --git a/arch/mips/pci/pci.c b/arch/mips/pci/pci.c
> > index 5717384..b4c02f2 100644
> > --- a/arch/mips/pci/pci.c
> > +++ b/arch/mips/pci/pci.c
> > @@ -112,7 +112,14 @@ static void pcibios_scanbus(struct pci_controller *hose)
> >  		need_domain_info = 1;
> >  	}
> >  
> > -	if (!pci_has_flag(PCI_PROBE_ONLY)) {
> > +	/*
> > +	 * We insert PCI resources into the iomem_resource and
> > +	 * ioport_resource trees in either pci_bus_claim_resources()
> > +	 * or pci_bus_assign_resources().
> > +	 */
> > +	if (pci_has_flag(PCI_PROBE_ONLY)) {
> > +		pci_bus_claim_resources(bus);
> > +	} else {
> >  		pci_bus_size_bridges(bus);
> >  		pci_bus_assign_resources(bus);
> >  	}
> 
> So I finally tested this on two SGI platforms, IP30 and IP27.  For IP30, it
> works fine once I drop the "pci_set_flags(PCI_PROBE_ONLY);" call.  IP30 uses
> standard virtual addressing via ioremap and the like for all of its PCI stuff,
> so it was trivial to fix it.
> 
> IP27, on the other hand, is not happy with this change.  I don't know about
> mainline IP27, as that's generally bitrotted and didn't boot the last time I
> tried it, but under the refactored code I'm using that actually boots, this
> platform still needs to bypass *all* attempts to claim or assign the PCI addresses.
> 
> I suspect the difference with IP27 is unlike normal systems, IP27 uses special
> physical addresses in the 0x92xxxxxxxxxxxxxx block to address I/O devices, so
> there's no proper ioremap or use of virtual addressing for I/O.  Ralf, does
> this sound correct?
> 
> If I use the patches I got plus this change and set PCI_PROBE_ONLY, this is
> what happens:
> 
> [   47.299511] PCI host bridge to bus 0001:00
> [   47.348155] pci_bus 0001:00: root bus resource [mem
> 0x920000000f200000-0x920000000f9fffff]
> [   47.447663] pci_bus 0001:00: root bus resource [io
> 0x920000000fa00000-0x920000000fbfffff]
> [   47.547183] pci_bus 0001:00: root bus resource [bus 01-ff]
> [   47.616581] pci 0001:00:00.0: can't claim BAR 0 [io  0xf200000-0xf2000ff]:
> no compatible bridge window
> [   47.726766] pci 0001:00:00.0: can't claim BAR 1 [mem 0x0f200000-0x0f200fff]:
> no compatible bridge window
> [   47.840937] pci 0001:00:00.0: can't claim BAR 6 [mem 0x00000000-0x0000ffff
> pref]: no compatible bridge window
> [   47.960317] pci 0001:00:01.0: can't claim BAR 0 [io  0xf400000-0xf4000ff]:
> no compatible bridge window
> [   48.072399] pci 0001:00:01.0: can't claim BAR 1 [mem 0x0f400000-0x0f400fff]:
> no compatible bridge window
> [   48.186560] pci 0001:00:01.0: can't claim BAR 6 [mem 0x00000000-0x0000ffff
> pref]: no compatible bridge window
> [   48.305957] pci 0001:00:02.0: can't claim BAR 0 [mem 0x0f600000-0x0f6fffff]:
> no compatible bridge window
> [   48.420098] pci 0001:00:06.0: can't claim BAR 0 [mem 0x0fa00000-0x0fafffff]:
> no compatible bridge window
> [   48.534294] pci 0001:00:07.0: can't claim BAR 0 [mem 0x00000000-0x00001fff]:
> no compatible bridge window
> [   48.648541] qla1280: QLA1040 found on PCI bus 0, dev 0
> [   48.710232] PCI: Enabling device 0001:00:00.0 (0006 -> 0007)
> [   48.779231] qla1280 0001:00:00.0: can't ioremap BAR 1: [mem size 0x00001000]
> [   48.863145] qla1280: Unable to map I/O memory
> [   48.916520] qla1280: QLA1040 found on PCI bus 0, dev 1
> [   48.977298] PCI: Enabling device 0001:00:01.0 (0006 -> 0007)
> [   49.046128] qla1280 0001:00:01.0: can't ioremap BAR 1: [mem size 0x00001000]
> [   49.130201] qla1280: Unable to map I/O memory
> [   49.183068] IOC3 0001:00:02.0: can't ioremap BAR 0: [mem size 0x00100000]
> [   49.264292] ioc3: Unable to remap PCI BAR for 0001:00:02.0.
> [   49.331533] IOC3 0001:00:06.0: can't ioremap BAR 0: [mem size 0x00100000]
> [   49.412984] ioc3: Unable to remap PCI BAR for 0001:00:06.0.
> 
> 
> If I disable PCI_PROBE_ONLY, then this:
> 
> [   47.588033] PCI host bridge to bus 0001:00
> [   47.636649] pci_bus 0001:00: root bus resource [mem
> 0x920000000f200000-0x920000000f9fffff]
> [   47.736144] pci_bus 0001:00: root bus resource [io
> 0x920000000fa00000-0x920000000fbfffff]
> [   47.835621] pci_bus 0001:00: root bus resource [bus 01-ff]
> [   47.905023] pci 0001:00:02.0: BAR 0: no space for [mem size 0x00100000]
> [   47.982775] pci 0001:00:02.0: BAR 0: failed to assign [mem size 0x00100000]
> [   48.066586] pci 0001:00:06.0: BAR 0: no space for [mem size 0x00100000]
> [   48.146173] pci 0001:00:06.0: BAR 0: failed to assign [mem size 0x00100000]
> [   48.229972] pci 0001:00:00.0: BAR 6: no space for [mem size 0x00010000 pref]
> [   48.314783] pci 0001:00:00.0: BAR 6: failed to assign [mem size 0x00010000 pref]
> [   48.403825] pci 0001:00:01.0: BAR 6: no space for [mem size 0x00010000 pref]
> [   48.488655] pci 0001:00:01.0: BAR 6: failed to assign [mem size 0x00010000 pref]
> [   48.577695] pci 0001:00:00.0: BAR 1: no space for [mem size 0x00001000]
> [   48.657260] pci 0001:00:00.0: BAR 1: failed to assign [mem size 0x00001000]
> [   48.741068] pci 0001:00:01.0: BAR 1: no space for [mem size 0x00001000]
> [   48.820644] pci 0001:00:01.0: BAR 1: failed to assign [mem size 0x00001000]
> [   48.904452] pci 0001:00:00.0: BAR 0: no space for [io  size 0x0100]
> [   48.979840] pci 0001:00:00.0: BAR 0: failed to assign [io  size 0x0100]
> [   49.059459] pci 0001:00:01.0: BAR 0: no space for [io  size 0x0100]
> [   49.134855] pci 0001:00:01.0: BAR 0: failed to assign [io  size 0x0100]
> [   49.214588] qla1280: QLA1040 found on PCI bus 0, dev 0
> [   49.277249] qla1280 0001:00:00.0: can't ioremap BAR 1: [??? 0x00000000 flags
> 0x0]
> [   49.366330] qla1280: Unable to map I/O memory
> [   49.419832] qla1280: QLA1040 found on PCI bus 0, dev 1
> [   49.481211] qla1280 0001:00:01.0: can't ioremap BAR 1: [??? 0x00000000 flags
> 0x0]
> [   49.570560] qla1280: Unable to map I/O memory
> [   49.623439] IOC3 0001:00:02.0: can't ioremap BAR 0: [??? 0x00000000 flags 0x0]
> [   49.709857] ioc3: Unable to remap PCI BAR for 0001:00:02.0.
> [   49.777113] IOC3 0001:00:06.0: can't ioremap BAR 0: [??? 0x00000000 flags 0x0]
> [   49.863796] ioc3: Unable to remap PCI BAR for 0001:00:06.0.
> 
> 
> The only fix I could come up with was more IP27 #ifdef hackery to retain the
> original code for IP27, but use the new conditional logic for all other
> platforms.  Would a patch for this be acceptable (with verbose comment), or is
> there a better way to deal with IP27 that doesn't involve re-writing its I/O
> addressing support entirely?

Would you mind providing a kernel boot log, lspci output and /proc/iomem
/proc/ioports for your "working" system boot ? That would help debugging
what's the expected resources allocation for the system and consequently
where the problem currently lies.

Thanks,
Lorenzo

Re: [PATCH] MIPS/PCI: Claim bus resources on PCI_PROBE_ONLY set-ups

[Joshua Kinard]

On 10/17/2016 14:09, Lorenzo Pieralisi wrote:
> On Mon, Oct 17, 2016 at 12:36:34AM -0400, Joshua Kinard wrote:
>> On 06/23/2016 18:16, Bjorn Helgaas wrote:
>>> We claim PCI BAR and bridge window resources in pci_bus_assign_resources(),
>>> but when PCI_PROBE_ONLY is set, we treat those resources as immutable and
>>> don't call pci_bus_assign_resources(), so the resources aren't put in the
>>> resource tree.
>>>
>>> When the resources aren't in the tree, they don't show up in /proc/iomem,
>>> we can't detect conflicts, and we need special cases elsewhere for
>>> PCI_PROBE_ONLY or resources without a parent pointer.
>>>
>>> Claim all PCI BAR and window resources in the PCI_PROBE_ONLY case.
>>>
>>> If a PCI_PROBE_ONLY platform assigns conflicting resources, Linux can't fix
>>> the conflicts.  Previously we didn't notice the conflicts, but now we will,
>>> which may expose new failures.
>>>
>>> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>>> ---

[snip]

>>
>> So I finally tested this on two SGI platforms, IP30 and IP27.  For IP30, it
>> works fine once I drop the "pci_set_flags(PCI_PROBE_ONLY);" call.  IP30 uses
>> standard virtual addressing via ioremap and the like for all of its PCI stuff,
>> so it was trivial to fix it.
>>
>> IP27, on the other hand, is not happy with this change.  I don't know about
>> mainline IP27, as that's generally bitrotted and didn't boot the last time I
>> tried it, but under the refactored code I'm using that actually boots, this
>> platform still needs to bypass *all* attempts to claim or assign the PCI addresses.
>>
>> I suspect the difference with IP27 is unlike normal systems, IP27 uses special
>> physical addresses in the 0x92xxxxxxxxxxxxxx block to address I/O devices, so
>> there's no proper ioremap or use of virtual addressing for I/O.  Ralf, does
>> this sound correct?
>>
>> If I use the patches I got plus this change and set PCI_PROBE_ONLY, this is
>> what happens:
>>
>> [   47.299511] PCI host bridge to bus 0001:00
>> [   47.348155] pci_bus 0001:00: root bus resource [mem
>> 0x920000000f200000-0x920000000f9fffff]
>> [   47.447663] pci_bus 0001:00: root bus resource [io
>> 0x920000000fa00000-0x920000000fbfffff]
>> [   47.547183] pci_bus 0001:00: root bus resource [bus 01-ff]
>> [   47.616581] pci 0001:00:00.0: can't claim BAR 0 [io  0xf200000-0xf2000ff]:
>> no compatible bridge window
>> [   47.726766] pci 0001:00:00.0: can't claim BAR 1 [mem 0x0f200000-0x0f200fff]:
>> no compatible bridge window
>> [   47.840937] pci 0001:00:00.0: can't claim BAR 6 [mem 0x00000000-0x0000ffff
>> pref]: no compatible bridge window
>> [   47.960317] pci 0001:00:01.0: can't claim BAR 0 [io  0xf400000-0xf4000ff]:
>> no compatible bridge window
>> [   48.072399] pci 0001:00:01.0: can't claim BAR 1 [mem 0x0f400000-0x0f400fff]:
>> no compatible bridge window
>> [   48.186560] pci 0001:00:01.0: can't claim BAR 6 [mem 0x00000000-0x0000ffff
>> pref]: no compatible bridge window
>> [   48.305957] pci 0001:00:02.0: can't claim BAR 0 [mem 0x0f600000-0x0f6fffff]:
>> no compatible bridge window
>> [   48.420098] pci 0001:00:06.0: can't claim BAR 0 [mem 0x0fa00000-0x0fafffff]:
>> no compatible bridge window
>> [   48.534294] pci 0001:00:07.0: can't claim BAR 0 [mem 0x00000000-0x00001fff]:
>> no compatible bridge window
>> [   48.648541] qla1280: QLA1040 found on PCI bus 0, dev 0
>> [   48.710232] PCI: Enabling device 0001:00:00.0 (0006 -> 0007)
>> [   48.779231] qla1280 0001:00:00.0: can't ioremap BAR 1: [mem size 0x00001000]
>> [   48.863145] qla1280: Unable to map I/O memory
>> [   48.916520] qla1280: QLA1040 found on PCI bus 0, dev 1
>> [   48.977298] PCI: Enabling device 0001:00:01.0 (0006 -> 0007)
>> [   49.046128] qla1280 0001:00:01.0: can't ioremap BAR 1: [mem size 0x00001000]
>> [   49.130201] qla1280: Unable to map I/O memory
>> [   49.183068] IOC3 0001:00:02.0: can't ioremap BAR 0: [mem size 0x00100000]
>> [   49.264292] ioc3: Unable to remap PCI BAR for 0001:00:02.0.
>> [   49.331533] IOC3 0001:00:06.0: can't ioremap BAR 0: [mem size 0x00100000]
>> [   49.412984] ioc3: Unable to remap PCI BAR for 0001:00:06.0.
>>
>>
>> If I disable PCI_PROBE_ONLY, then this:
>>
>> [   47.588033] PCI host bridge to bus 0001:00
>> [   47.636649] pci_bus 0001:00: root bus resource [mem
>> 0x920000000f200000-0x920000000f9fffff]
>> [   47.736144] pci_bus 0001:00: root bus resource [io
>> 0x920000000fa00000-0x920000000fbfffff]
>> [   47.835621] pci_bus 0001:00: root bus resource [bus 01-ff]
>> [   47.905023] pci 0001:00:02.0: BAR 0: no space for [mem size 0x00100000]
>> [   47.982775] pci 0001:00:02.0: BAR 0: failed to assign [mem size 0x00100000]
>> [   48.066586] pci 0001:00:06.0: BAR 0: no space for [mem size 0x00100000]
>> [   48.146173] pci 0001:00:06.0: BAR 0: failed to assign [mem size 0x00100000]
>> [   48.229972] pci 0001:00:00.0: BAR 6: no space for [mem size 0x00010000 pref]
>> [   48.314783] pci 0001:00:00.0: BAR 6: failed to assign [mem size 0x00010000 pref]
>> [   48.403825] pci 0001:00:01.0: BAR 6: no space for [mem size 0x00010000 pref]
>> [   48.488655] pci 0001:00:01.0: BAR 6: failed to assign [mem size 0x00010000 pref]
>> [   48.577695] pci 0001:00:00.0: BAR 1: no space for [mem size 0x00001000]
>> [   48.657260] pci 0001:00:00.0: BAR 1: failed to assign [mem size 0x00001000]
>> [   48.741068] pci 0001:00:01.0: BAR 1: no space for [mem size 0x00001000]
>> [   48.820644] pci 0001:00:01.0: BAR 1: failed to assign [mem size 0x00001000]
>> [   48.904452] pci 0001:00:00.0: BAR 0: no space for [io  size 0x0100]
>> [   48.979840] pci 0001:00:00.0: BAR 0: failed to assign [io  size 0x0100]
>> [   49.059459] pci 0001:00:01.0: BAR 0: no space for [io  size 0x0100]
>> [   49.134855] pci 0001:00:01.0: BAR 0: failed to assign [io  size 0x0100]
>> [   49.214588] qla1280: QLA1040 found on PCI bus 0, dev 0
>> [   49.277249] qla1280 0001:00:00.0: can't ioremap BAR 1: [??? 0x00000000 flags
>> 0x0]
>> [   49.366330] qla1280: Unable to map I/O memory
>> [   49.419832] qla1280: QLA1040 found on PCI bus 0, dev 1
>> [   49.481211] qla1280 0001:00:01.0: can't ioremap BAR 1: [??? 0x00000000 flags
>> 0x0]
>> [   49.570560] qla1280: Unable to map I/O memory
>> [   49.623439] IOC3 0001:00:02.0: can't ioremap BAR 0: [??? 0x00000000 flags 0x0]
>> [   49.709857] ioc3: Unable to remap PCI BAR for 0001:00:02.0.
>> [   49.777113] IOC3 0001:00:06.0: can't ioremap BAR 0: [??? 0x00000000 flags 0x0]
>> [   49.863796] ioc3: Unable to remap PCI BAR for 0001:00:06.0.
>>
>>
>> The only fix I could come up with was more IP27 #ifdef hackery to retain the
>> original code for IP27, but use the new conditional logic for all other
>> platforms.  Would a patch for this be acceptable (with verbose comment), or is
>> there a better way to deal with IP27 that doesn't involve re-writing its I/O
>> addressing support entirely?
> 
> Would you mind providing a kernel boot log, lspci output and /proc/iomem
> /proc/ioports for your "working" system boot ? That would help debugging
> what's the expected resources allocation for the system and consequently
> where the problem currently lies.

Sure.  I've disabled line-wrapping on this message so my mail client doesn't
mangle the output.

First, is the boot output from my serial console:
[    0.000000] Linux version 4.8.2-mipsgit-20161016 (root@helcaraxe) (gcc version 5.4.0 (Gentoo Hardened 5.4.0 p1.0, pie-0.6.5) ) #4 SMP Mon Oct 17 19:48:30 EDT 2016
[    0.000000] ARCH: SGI-IP27
[    0.000000] PROMLIB: ARC firmware Version 64 Revision 0
[    0.000000] SMP: Discovered 4 cpus on 2 nodes
[    0.000000] ************** Topology ********************
[    0.000000]     00 01
[    0.000000] 00  255 255
[    0.000000] 01  255 255
[    0.000000] bootconsole [early0] enabled
[    0.000000] CPU0 revision is: 00000f14 (R14000)
[    0.000000] FPU revision is: 00000900
[    0.000000] Checking for the multiply/shift bug... no.
[    0.000000] Checking for the daddiu bug... no.
[    0.000000] IP27: Running on node 0.
[    0.000000] IP27: Node 0 has a primary CPU, CPU is running.
[    0.000000] IP27: Node 0 has a secondary CPU, CPU is running.
[    0.000000] IP27: Machine is in M mode.
[    0.000000] IP27: CPU 1A (CPU0): 500 MHz CPU detected.
[    0.000000] Determined physical RAM map:
[    0.000000]  memory: 0000000000838000 @ 000000000001c000 (usable)
[    0.000000]  memory: 000000000057c000 @ 0000000000854000 (usable after init)
[    0.000000] Initrd not found or empty - disabling initrd
[    0.000000] cma: Reserved 32 MiB at 0x00000001fe000000
[    0.000000] REPLICATION: ON nasid 0, ktext from nasid 0, kdata from nasid 0
[    0.000000] REPLICATION: ON nasid 1, ktext from nasid 0, kdata from nasid 0
[    0.000000] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
[    0.000000] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
[    0.000000] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000000000000-0x00000001ffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x00000000ffffffff]
[    0.000000]   node   1: [mem 0x0000000100000000-0x00000001ffffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000ffffffff]
[    0.000000] Initmem setup node 1 [mem 0x0000000100000000-0x00000001ffffffff]
[    0.000000] percpu: Embedded 4 pages/cpu @a800000002010000 s36096 r0 d29440 u65536
[    0.000000] Built 2 zonelists in Node order, mobility grouping on.  Total pages: 522240
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: root=dksc(0,1,0) console=ttyS0,9600
[    0.000000] PID hash table entries: 4096 (order: 1, 32768 bytes)
[    0.000000] Memory: 8307776K/8388608K available (6498K kernel code, 454K rwdata, 1436K rodata, 5616K init, 543K bss, 48064K reserved, 32768K cma-reserved)
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  Build-time adjustment of leaf fanout to 64.
[    0.000000]  RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=4
[    0.000000] NR_IRQS:256
[    0.000000] clocksource: HUB: mask: 0xfffffffffffff max_cycles: 0x127350b88, max_idle_ns: 1763180808480 ns
[    0.000019] sched_clock: 52 bits at 1250kHz, resolution 800ns, wraps every 4398046510800ns
[    0.102422] Console: colour dummy device 80x25
[    0.154975] Calibrating delay loop... 749.56 BogoMIPS (lpj=3747840)
[    0.317670] pid_max: default: 32768 minimum: 301
[    0.377938] Dentry cache hash table entries: 1048576 (order: 9, 8388608 bytes)
[    0.592303] Inode-cache hash table entries: 524288 (order: 8, 4194304 bytes)
[    0.736316] Mount-cache hash table entries: 16384 (order: 3, 131072 bytes)
[    0.817310] Mountpoint-cache hash table entries: 16384 (order: 3, 131072 bytes)
[    0.916094] Checking for the daddi bug... no.
[    0.976495] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
[    0.976520] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
[    0.976532] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
[    0.979792] IP27: CPU 1B (CPU1): 500 MHz CPU detected.
[    0.979824] CPU1 revision is: 00000f14 (R14000)
[    0.979831] FPU revision is: 00000900
[    1.014463] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
[    1.014493] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
[    1.014506] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
[    1.017888] IP27: CPU 2A (CPU2): 500 MHz CPU detected.
[    1.017931] CPU2 revision is: 00000f14 (R14000)
[    1.017938] FPU revision is: 00000900
[    1.055420] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
[    1.055452] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
[    1.055467] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
[    1.058810] IP27: CPU 2B (CPU3): 500 MHz CPU detected.
[    1.058851] CPU3 revision is: 00000f14 (R14000)
[    1.058858] FPU revision is: 00000900
[    1.092672] Brought up 4 CPUs
[    2.484801] devtmpfs: initialized
[    2.603700] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    2.892716] random: fast init done
[    2.990752] xor: measuring software checksum speed
[    3.146730]    8regs     :   486.400 MB/sec
[    3.295350]    8regs_prefetch:   486.400 MB/sec
[    3.448144]    32regs    :   486.400 MB/sec
[    3.596768]    32regs_prefetch:   486.400 MB/sec
[    3.650415] xor: using function: 32regs_prefetch (486.400 MB/sec)
[    3.834852] NET: Registered protocol family 16
[    3.946535] cpuidle: using governor ladder
[    4.043955] cpuidle: using governor menu
[    4.089355] xtalk:n0/0 xbow widget (rev 1.3)
[    4.140796] xtalk:n0/8 kona widget (rev unknown) registered as a platform device.
[    4.230796] xtalk:n0/b bridge widget (rev D) registered as a platform device.
[    4.316712] xtalk:n0/c bridge widget (rev D) registered as a platform device.
[    4.402590] xtalk:n0/d impact widget (rev B) registered as a platform device.
[    4.488444] xtalk:n0/f bridge widget (rev C) registered as a platform device.
[    4.574178] xtalk:n1/0 xbow widget (rev 1.3)
[    5.247396] raid6: int64x1  gen()    84 MB/s
[    5.467370] raid6: int64x1  xor()    80 MB/s
[    5.687324] raid6: int64x2  gen()   151 MB/s
[    5.907243] raid6: int64x2  xor()   116 MB/s
[    6.127125] raid6: int64x4  gen()   235 MB/s
[    6.347155] raid6: int64x4  xor()   142 MB/s
[    6.567117] raid6: int64x8  gen()   251 MB/s
[    6.787018] raid6: int64x8  xor()   175 MB/s
[    6.836420] raid6: using algorithm int64x8 gen() 251 MB/s
[    6.901352] raid6: .... xor() 175 MB/s, rmw enabled
[    6.960002] raid6: using intx1 recovery algorithm
[    7.017544] SCSI subsystem initialized
[    7.063853] clocksource: Switched to clocksource HUB
[    7.363862] FS-Cache: Loaded
[   31.643872] NET: Registered protocol family 2
[   31.696149] TCP established hash table entries: 65536 (order: 5, 524288 bytes)
[   31.787294] TCP bind hash table entries: 65536 (order: 6, 1048576 bytes)
[   31.874832] TCP: Hash tables configured (established 65536 bind 65536)
[   31.952084] UDP hash table entries: 4096 (order: 3, 131072 bytes)
[   32.026304] UDP-Lite hash table entries: 4096 (order: 3, 131072 bytes)
[   32.403836] NET: Registered protocol family 1
[   32.583876] RPC: Registered named UNIX socket transport module.
[   32.653282] RPC: Registered udp transport module.
[   32.709785] RPC: Registered tcp transport module.
[   32.766346] RPC: Registered tcp NFSv4.1 backchannel transport module.
[   34.045622] futex hash table entries: 1024 (order: 3, 131072 bytes)
[   34.275238] workingset: timestamp_bits=40 max_order=19 bucket_order=0
[   34.350961] zbud: loaded
[   37.383916] NFS: Registering the id_resolver key type
[   37.442917] Key type id_resolver registered
[   37.493101] Key type id_legacy registered
[   37.541388] efs: 1.0a - http://aeschi.ch.eu.org/efs/
[   37.873853] SGI XFS with ACLs, security attributes, realtime, no debug enabled
[   39.453936] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[   39.541066] io scheduler noop registered
[   39.588223] io scheduler deadline registered
[   39.639639] io scheduler cfq registered (default)
[   39.973936] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[   40.148640] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[   40.245916] loop: module loaded
[   40.283288] mousedev: PS/2 mouse device common for all mice
[   40.349460] md: linear personality registered for level -1
[   40.414802] md: raid0 personality registered for level 0
[   40.478677] md: raid1 personality registered for level 1
[   40.542549] md: raid10 personality registered for level 10
[   40.609068] md: raid6 personality registered for level 6
[   40.672411] md: raid5 personality registered for level 5
[   40.736315] md: raid4 personality registered for level 4
[   40.800808] PCI host bridge to bus 0000:00
[   40.849424] pci_bus 0000:00: root bus resource [mem 0x920000000b200000-0x920000000b9fffff]
[   40.948920] pci_bus 0000:00: root bus resource [io  0x920000000ba00000-0x920000000bbfffff]
[   41.048408] pci_bus 0000:00: root bus resource [bus 00-ff]
[   41.114880] PCI host bridge to bus 0001:00
[   41.163629] pci_bus 0001:00: root bus resource [mem 0x920000000c200000-0x920000000c9fffff]
[   41.263152] pci_bus 0001:00: root bus resource [io  0x920000000ca00000-0x920000000cbfffff]
[   41.362645] pci_bus 0001:00: root bus resource [bus 01-ff]
[   41.430847] PCI host bridge to bus 0002:00
[   41.478263] pci_bus 0002:00: root bus resource [mem 0x920000000f200000-0x920000000f9fffff]
[   41.577771] pci_bus 0002:00: root bus resource [io  0x920000000fa00000-0x920000000fbfffff]
[   41.677250] pci_bus 0002:00: root bus resource [bus 02-ff]
[   41.746742] qla1280: QLA1040 found on PCI bus 0, dev 0
[   41.806687] PCI: Enabling device 0002:00:00.0 (0006 -> 0007)
[   41.875796] scsi(0): Enabling vchannel on BRIDGE for SGI/MIPS
[   42.324520] random: crng init done
[   42.839160] scsi(0:0): Resetting SCSI BUS
[   45.964232] scsi host0: QLogic QLA1040 PCI to SCSI Host Adapter
[   45.964232]        Firmware version:  7.65.06, Driver version 3.27.1
[   46.118149] qla1280: QLA1040 found on PCI bus 0, dev 1
[   46.178192] PCI: Enabling device 0002:00:01.0 (0006 -> 0007)
[   46.289985] scsi 0:0:1:0: Direct-Access     SEAGATE  SX150176LC       BA12 PQ: 0 ANSI: 2
[   46.387174] scsi(0:0:1:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
[   46.486875] scsi 0:0:2:0: Direct-Access     SEAGATE  SX150176LC       BA08 PQ: 0 ANSI: 2
[   46.583893] scsi(0:0:2:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
[   46.683736] scsi 0:0:3:0: Direct-Access     SEAGATE  SX150176LC       BA08 PQ: 0 ANSI: 2
[   46.780696] scsi(0:0:3:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
[   46.880821] scsi 0:0:4:0: Direct-Access     SEAGATE  SX150176LC       BA11 PQ: 0 ANSI: 2
[   46.977716] scsi(0:0:4:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
[   47.077797] scsi 0:0:5:0: Direct-Access     SEAGATE  SX150176LC       BA11 PQ: 0 ANSI: 2
[   47.174768] scsi(0:0:5:0):[   47.181219] scsi(1:0): Resetting SCSI BUS
[   47.251839]  Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
[   47.639912] scsi 0:0:6:0: CD-ROM            TOSHIBA  CD-ROM XM-5701TA 0167 PQ: 0 ANSI: 2
[   47.736890] scsi(0:0:6:0): Sync: period 10, offset 12
[   51.553976] scsi host1: QLogic QLA1040 PCI to SCSI Host Adapter
[   51.553976]        Firmware version:  7.65.06, Driver version 3.27.1
[   51.998859] ioc3: part: [], serial: [] => class IP27 BaseIO
[   52.068783] ioc3-eth: Ethernet address is 08:00:69:05:64:74.
[   52.136831] IOC3 0002:00:02.0 eth0: link up, 100Mbps, full-duplex, lpa 0x8DE1
[   52.220897] eth0: Using PHY 31, vendor 0x20005c0, model 0, rev 0.
[   52.294232] eth0: IOC3 SSRAM has 128 kbyte.
[   52.345852] rtc-m48t35 rtc-m48t35: rtc core: registered m48t35 as rtc0
[   52.588471] console [ttyS0] enabled
[   52.489601] 0002:00:02.0: ttyS0 at IOC3 0xf620178 (irq = 0, base_baud = 458333) is a 16550A
[   52.671141] bootconsole [early0] disabled
[   54.146680] 0002:00:02.0: ttyS1 at IOC3 0xf620170 (irq = 0, base_baud = 458333) is a 16550A
[   54.248408] IOC3 Master Driver loaded for 0002:00:02.0
[   54.313895] ioc3: NIC search failed.
[   54.356992] ioc3: part: [], serial: [] => class IP27 BaseIO
[   54.424356] ioc3: request_irq fails for IRQ 0xffffffff
[   54.490948] ioc3-eth: Ethernet address is 00:00:00:00:00:00.
[   54.561038] IOC3 0002:00:06.0 eth1: link down
[   54.613597] eth1: Using PHY 31, vendor 0x0, model 0, rev 0.
[   54.680825] eth1: IOC3 SSRAM has 64 kbyte.
[   54.765266] 0002:00:06.0: ttyS2 at IOC3 0xfa20178 (irq = 0, base_baud = 458333) is a 16550A
[   54.888655] 0002:00:06.0: ttyS3 at IOC3 0xfa20170 (irq = 0, base_baud = 458333) is a 16550A
[   54.990198] IOC3 Master Driver loaded for 0002:00:06.0
[   55.052840] NET: Registered protocol family 26
[   55.509832] sd 0:0:1:0: [sda] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
[   55.515697] sr 0:0:6:0: [sr0] scsi-1 drive
[   55.515709] cdrom: Uniform CD-ROM driver Revision: 3.20
[   55.520834] sd 0:0:4:0: [sdd] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
[   55.523611] sd 0:0:4:0: [sdd] Write Protect is off
[   55.526511] sd 0:0:4:0: [sdd] Write cache: disabled, read cache: enabled, supports DPO and FUA
[   55.550956]  sdd: sdd1 sdd2 sdd3 sdd4 sdd5 sdd6 sdd9 sdd11
[   55.562326] sd 0:0:4:0: [sdd] Attached SCSI disk
[   55.564803] sd 0:0:5:0: [sde] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
[   55.567564] sd 0:0:5:0: [sde] Write Protect is off
[   55.570463] sd 0:0:5:0: [sde] Write cache: disabled, read cache: enabled, supports DPO and FUA
[   55.600908] sd 0:0:5:0: [sde] Attached SCSI disk
[   55.604715] sd 0:0:2:0: [sdb] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
[   55.605297] sd 0:0:3:0: [sdc] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
[   55.607054] sd 0:0:2:0: [sdb] Write Protect is off
[   55.607592] sd 0:0:3:0: [sdc] Write Protect is off
[   55.609959] sd 0:0:2:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
[   55.610460] sd 0:0:3:0: [sdc] Write cache: disabled, read cache: enabled, supports DPO and FUA
[   55.631199]  sdb: sdb1 sdb2 sdb3 sdb4 sdb5 sdb6 sdb9 sdb11
[   55.634048]  sdc: sdc1 sdc2 sdc3 sdc4 sdc5 sdc6 sdc9 sdc11
[   55.646437] sd 0:0:3:0: [sdc] Attached SCSI disk
[   55.647162] sd 0:0:2:0: [sdb] Attached SCSI disk
[   57.150140] sd 0:0:1:0: [sda] Write Protect is off
[   57.173988] NET: Registered protocol family 10
[   57.264566] sd 0:0:1:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
[   57.386960]  sda: sda1 sda2 sda3 sda4 sda5 sda6 sda9 sda11
[   57.464000] sd 0:0:1:0: [sda] Attached SCSI disk
[   57.544064] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[   57.617430] NET: Registered protocol family 17
[   57.671360] 8021q: 802.1Q VLAN Support v1.8
[   57.722075] sctp: Hash tables configured (bind 1024/1024)
[   58.093962] Key type dns_resolver registered
[   58.148015] rtc-m48t35 rtc-m48t35: setting system clock to 2016-10-17 23:49:58 UTC (1476748198)
[   59.626451] Freeing unused kernel memory: 5616K (a800000000854000 - a800000000dd0000)
[   59.720950] This architecture does not have kernel memory protection.


Next is /proc/iomem and /proc/ioports:
# cat /proc/iomem
0f600000-0f6fffff : ioc3
  0f680000-0f687fff : rtc-m48t35
0fa00000-0fafffff : ioc3
920000000b200000-920000000b9fffff : Bridge MEM
920000000c200000-920000000c9fffff : Bridge MEM
920000000f200000-920000000f9fffff : Bridge MEM

/ # cat /proc/ioports
920000000ba00000-920000000bbfffff : Bridge IO
920000000ca00000-920000000cbfffff : Bridge IO
920000000fa00000-920000000fbfffff : Bridge IO


And lspci -vvvx:

/ # /usr/sbin/lspci -vvvx
0001:00:00.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. Linc I/O controller
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        NUMA node: 0
        Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=134217729]
        Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Expansion ROM at <unassigned> [disabled] [size=2]
lspci: Unable to load libkmod resources: error -12
00: a9 10 02 00 00 00 00 02 00 00 00 ff 00 f8 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0001:00:01.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. Linc I/O controller
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        NUMA node: 0
        Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=134217729]
        Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Expansion ROM at <unassigned> [disabled] [size=2]
00: a9 10 02 00 00 00 00 02 00 00 00 ff 00 f8 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0002:00:00.0 SCSI storage controller: QLogic Corp. ISP1020 Fast-wide SCSI (rev 05)
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 64, Cache Line Size: 256 bytes
        Interrupt: pin A routed to IRQ 1
        NUMA node: 0
        Region 0: I/O ports at f200000 [size=257]
        Region 1: Memory at 0f200000 (32-bit, non-prefetchable) [size=4097]
        Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Expansion ROM at <unassigned> [disabled] [size=65537]
        Kernel driver in use: qla1280
00: 77 10 20 10 07 00 00 02 05 00 00 01 40 40 00 00
10: 01 00 20 0f 00 00 20 0f 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00

0002:00:01.0 SCSI storage controller: QLogic Corp. ISP1020 Fast-wide SCSI (rev 05)
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 64, Cache Line Size: 256 bytes
        Interrupt: pin A routed to IRQ 2
        NUMA node: 0
        Region 0: I/O ports at f400000 [size=257]
        Region 1: Memory at 0f400000 (32-bit, non-prefetchable) [size=4097]
        Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Expansion ROM at <unassigned> [disabled] [size=65537]
        Kernel driver in use: qla1280
00: 77 10 20 10 07 00 00 02 05 00 00 01 40 40 00 00
10: 01 00 40 0f 00 00 40 0f 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00

0002:00:02.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. IOC3 I/O controller (rev 01)
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 64
        Interrupt: pin A routed to IRQ 3
        NUMA node: 0
        Region 0: Memory at 0f600000 (32-bit, non-prefetchable) [size=1048577]
        Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Expansion ROM at <unassigned> [disabled] [size=2]
        Kernel driver in use: IOC3
00: a9 10 03 00 06 00 80 02 01 00 00 ff 00 40 00 00
10: 00 00 60 0f 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00

0002:00:06.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. IOC3 I/O controller (rev 01)
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 64
        Interrupt: pin A routed to IRQ 5
        NUMA node: 0
        Region 0: Memory at 0fa00000 (32-bit, non-prefetchable) [size=1048577]
        Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
        Expansion ROM at <unassigned> [disabled] [size=2]
        Kernel driver in use: IOC3
00: a9 10 03 00 06 00 80 02 01 00 00 ff 00 40 00 00
10: 00 00 a0 0f 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00

0002:00:07.0 Non-VGA unclassified device: Silicon Graphics Intl. Corp. RAD Audio (rev 70)
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        NUMA node: 0
        Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=8193]
        Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
        Expansion ROM at <unassigned> [disabled] [size=2]
00: a9 10 05 00 00 00 80 04 70 00 00 00 00 00 00 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00


IP27 hardware does some interesting things with address spaces, depending on
which "mode" you run the machine in.  "M mode" is the only one that's really
usable these days (the other, "N mode", doesn't boot).  Under M mode,
there's five address spaces, of which, "I/O Space" is selectable by turning on
specific bits in the 64-bit memory address.

So, given 0x920000000b200000, bits 63:62 select physical addressing (10) and
bits 58:57 select I/O space (01), resulting in all PCI addresses (or any I/O
address) appearing in the 0x92xxxxxxxxxxxxxx range.  The rest of the bits go
on to select which NUMA node the I/O is targeted at and which Crosstalk widget
to access, and so on.  I still haven't fully understood it all myself just
yet.

You can google for document 007-3410-001, "Origin and Onyx2 Programmer's
Reference Manual" for more details (it's no longer available via SGI's
TechPubs, but it is archived on plenty of other sites).

This particular machine that I have has three "BRIDGE" chips in it, all
attached to the Crosstalk bus.  Each BRIDGE can address up to 8 PCI devices,
but some crosstalk boards slap a BRIDGE chip on for just a single device.  In
this instance, the BRIDGE w/ address 0x920000000fxxxxxx is the IO6 at xtalk
widget 0xf (this has the basic I/O devices for this machine).  The BRIDGE at
0x920000000cxxxxxx (widget 0xc) is a PCI "shoebox" that can hold up to three
PCI-X cards, and the BRIDGE at 0x920000000bxxxxxx (widget 0xb) is an
SGI-specific HIPPI board (ID'ed as a LINC I/O controller by pci.ids).

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
6144R/F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [PATCH] MIPS/PCI: Claim bus resources on PCI_PROBE_ONLY set-ups

[Lorenzo Pieralisi]

On Mon, Oct 17, 2016 at 08:30:38PM -0400, Joshua Kinard wrote:
> On 10/17/2016 14:09, Lorenzo Pieralisi wrote:
> > On Mon, Oct 17, 2016 at 12:36:34AM -0400, Joshua Kinard wrote:
> >> On 06/23/2016 18:16, Bjorn Helgaas wrote:
> >>> We claim PCI BAR and bridge window resources in pci_bus_assign_resources(),
> >>> but when PCI_PROBE_ONLY is set, we treat those resources as immutable and
> >>> don't call pci_bus_assign_resources(), so the resources aren't put in the
> >>> resource tree.
> >>>
> >>> When the resources aren't in the tree, they don't show up in /proc/iomem,
> >>> we can't detect conflicts, and we need special cases elsewhere for
> >>> PCI_PROBE_ONLY or resources without a parent pointer.
> >>>
> >>> Claim all PCI BAR and window resources in the PCI_PROBE_ONLY case.
> >>>
> >>> If a PCI_PROBE_ONLY platform assigns conflicting resources, Linux can't fix
> >>> the conflicts.  Previously we didn't notice the conflicts, but now we will,
> >>> which may expose new failures.
> >>>
> >>> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
> >>> ---
> 
> [snip]
> 
> >>
> >> So I finally tested this on two SGI platforms, IP30 and IP27.  For IP30, it
> >> works fine once I drop the "pci_set_flags(PCI_PROBE_ONLY);" call.  IP30 uses
> >> standard virtual addressing via ioremap and the like for all of its PCI stuff,
> >> so it was trivial to fix it.
> >>
> >> IP27, on the other hand, is not happy with this change.  I don't know about
> >> mainline IP27, as that's generally bitrotted and didn't boot the last time I
> >> tried it, but under the refactored code I'm using that actually boots, this
> >> platform still needs to bypass *all* attempts to claim or assign the PCI addresses.
> >>
> >> I suspect the difference with IP27 is unlike normal systems, IP27 uses special
> >> physical addresses in the 0x92xxxxxxxxxxxxxx block to address I/O devices, so
> >> there's no proper ioremap or use of virtual addressing for I/O.  Ralf, does
> >> this sound correct?
> >>
> >> If I use the patches I got plus this change and set PCI_PROBE_ONLY, this is
> >> what happens:
> >>
> >> [   47.299511] PCI host bridge to bus 0001:00
> >> [   47.348155] pci_bus 0001:00: root bus resource [mem
> >> 0x920000000f200000-0x920000000f9fffff]
> >> [   47.447663] pci_bus 0001:00: root bus resource [io
> >> 0x920000000fa00000-0x920000000fbfffff]
> >> [   47.547183] pci_bus 0001:00: root bus resource [bus 01-ff]
> >> [   47.616581] pci 0001:00:00.0: can't claim BAR 0 [io  0xf200000-0xf2000ff]:
> >> no compatible bridge window
> >> [   47.726766] pci 0001:00:00.0: can't claim BAR 1 [mem 0x0f200000-0x0f200fff]:
> >> no compatible bridge window
> >> [   47.840937] pci 0001:00:00.0: can't claim BAR 6 [mem 0x00000000-0x0000ffff
> >> pref]: no compatible bridge window
> >> [   47.960317] pci 0001:00:01.0: can't claim BAR 0 [io  0xf400000-0xf4000ff]:
> >> no compatible bridge window
> >> [   48.072399] pci 0001:00:01.0: can't claim BAR 1 [mem 0x0f400000-0x0f400fff]:
> >> no compatible bridge window
> >> [   48.186560] pci 0001:00:01.0: can't claim BAR 6 [mem 0x00000000-0x0000ffff
> >> pref]: no compatible bridge window
> >> [   48.305957] pci 0001:00:02.0: can't claim BAR 0 [mem 0x0f600000-0x0f6fffff]:
> >> no compatible bridge window
> >> [   48.420098] pci 0001:00:06.0: can't claim BAR 0 [mem 0x0fa00000-0x0fafffff]:
> >> no compatible bridge window
> >> [   48.534294] pci 0001:00:07.0: can't claim BAR 0 [mem 0x00000000-0x00001fff]:
> >> no compatible bridge window
> >> [   48.648541] qla1280: QLA1040 found on PCI bus 0, dev 0
> >> [   48.710232] PCI: Enabling device 0001:00:00.0 (0006 -> 0007)
> >> [   48.779231] qla1280 0001:00:00.0: can't ioremap BAR 1: [mem size 0x00001000]
> >> [   48.863145] qla1280: Unable to map I/O memory
> >> [   48.916520] qla1280: QLA1040 found on PCI bus 0, dev 1
> >> [   48.977298] PCI: Enabling device 0001:00:01.0 (0006 -> 0007)
> >> [   49.046128] qla1280 0001:00:01.0: can't ioremap BAR 1: [mem size 0x00001000]
> >> [   49.130201] qla1280: Unable to map I/O memory
> >> [   49.183068] IOC3 0001:00:02.0: can't ioremap BAR 0: [mem size 0x00100000]
> >> [   49.264292] ioc3: Unable to remap PCI BAR for 0001:00:02.0.
> >> [   49.331533] IOC3 0001:00:06.0: can't ioremap BAR 0: [mem size 0x00100000]
> >> [   49.412984] ioc3: Unable to remap PCI BAR for 0001:00:06.0.
> >>
> >>
> >> If I disable PCI_PROBE_ONLY, then this:
> >>
> >> [   47.588033] PCI host bridge to bus 0001:00
> >> [   47.636649] pci_bus 0001:00: root bus resource [mem
> >> 0x920000000f200000-0x920000000f9fffff]
> >> [   47.736144] pci_bus 0001:00: root bus resource [io
> >> 0x920000000fa00000-0x920000000fbfffff]
> >> [   47.835621] pci_bus 0001:00: root bus resource [bus 01-ff]
> >> [   47.905023] pci 0001:00:02.0: BAR 0: no space for [mem size 0x00100000]
> >> [   47.982775] pci 0001:00:02.0: BAR 0: failed to assign [mem size 0x00100000]
> >> [   48.066586] pci 0001:00:06.0: BAR 0: no space for [mem size 0x00100000]
> >> [   48.146173] pci 0001:00:06.0: BAR 0: failed to assign [mem size 0x00100000]
> >> [   48.229972] pci 0001:00:00.0: BAR 6: no space for [mem size 0x00010000 pref]
> >> [   48.314783] pci 0001:00:00.0: BAR 6: failed to assign [mem size 0x00010000 pref]
> >> [   48.403825] pci 0001:00:01.0: BAR 6: no space for [mem size 0x00010000 pref]
> >> [   48.488655] pci 0001:00:01.0: BAR 6: failed to assign [mem size 0x00010000 pref]
> >> [   48.577695] pci 0001:00:00.0: BAR 1: no space for [mem size 0x00001000]
> >> [   48.657260] pci 0001:00:00.0: BAR 1: failed to assign [mem size 0x00001000]
> >> [   48.741068] pci 0001:00:01.0: BAR 1: no space for [mem size 0x00001000]
> >> [   48.820644] pci 0001:00:01.0: BAR 1: failed to assign [mem size 0x00001000]
> >> [   48.904452] pci 0001:00:00.0: BAR 0: no space for [io  size 0x0100]
> >> [   48.979840] pci 0001:00:00.0: BAR 0: failed to assign [io  size 0x0100]
> >> [   49.059459] pci 0001:00:01.0: BAR 0: no space for [io  size 0x0100]
> >> [   49.134855] pci 0001:00:01.0: BAR 0: failed to assign [io  size 0x0100]
> >> [   49.214588] qla1280: QLA1040 found on PCI bus 0, dev 0
> >> [   49.277249] qla1280 0001:00:00.0: can't ioremap BAR 1: [??? 0x00000000 flags
> >> 0x0]
> >> [   49.366330] qla1280: Unable to map I/O memory
> >> [   49.419832] qla1280: QLA1040 found on PCI bus 0, dev 1
> >> [   49.481211] qla1280 0001:00:01.0: can't ioremap BAR 1: [??? 0x00000000 flags
> >> 0x0]
> >> [   49.570560] qla1280: Unable to map I/O memory
> >> [   49.623439] IOC3 0001:00:02.0: can't ioremap BAR 0: [??? 0x00000000 flags 0x0]
> >> [   49.709857] ioc3: Unable to remap PCI BAR for 0001:00:02.0.
> >> [   49.777113] IOC3 0001:00:06.0: can't ioremap BAR 0: [??? 0x00000000 flags 0x0]
> >> [   49.863796] ioc3: Unable to remap PCI BAR for 0001:00:06.0.
> >>
> >>
> >> The only fix I could come up with was more IP27 #ifdef hackery to retain the
> >> original code for IP27, but use the new conditional logic for all other
> >> platforms.  Would a patch for this be acceptable (with verbose comment), or is
> >> there a better way to deal with IP27 that doesn't involve re-writing its I/O
> >> addressing support entirely?
> > 
> > Would you mind providing a kernel boot log, lspci output and /proc/iomem
> > /proc/ioports for your "working" system boot ? That would help debugging
> > what's the expected resources allocation for the system and consequently
> > where the problem currently lies.
> 
> Sure.  I've disabled line-wrapping on this message so my mail client doesn't
> mangle the output.

Ok, I think this exactly showed what Bjorn wanted to see by claiming
resources because the problem is that the value in the BARs is not
contained in any bridge window, for the reason that you just mentioned
(ie the "special" physical addresses flags).

In the PCI_PROBE_ONLY case, claiming resources fail, which means that
we set the resource flags res->flags |= IORESOURCE_UNSET, which causes
pci_ioremap_bar() to fail.

So, it is correct to claim the resources if PCI_PROBE_ONLY, this fails
because the values in the BARs and the PCI host bridge window resources
have different formats (ie the PCI bridge window resources have addresses
that contain the special flags you mentioned) the question I have is
why the PCI root bridge window should be set with the flags in it
instead of a "pure" IO address.

Is my reading of the problem correct ?

Why do the PCI host bridge windows have to contain that 0x92 tag in the
upper bits ?

By not claiming resources, as you can see in the log below
/proc/iomem and /proc/ioports do not report the PCI devices resource
usages and the respective resources are not reserved in the kernel,
which is wrong and that's exactly why Bjorn patched the kernel.

Please let me know.

Thanks,
Lorenzo

> First, is the boot output from my serial console:
> [    0.000000] Linux version 4.8.2-mipsgit-20161016 (root@helcaraxe) (gcc version 5.4.0 (Gentoo Hardened 5.4.0 p1.0, pie-0.6.5) ) #4 SMP Mon Oct 17 19:48:30 EDT 2016
> [    0.000000] ARCH: SGI-IP27
> [    0.000000] PROMLIB: ARC firmware Version 64 Revision 0
> [    0.000000] SMP: Discovered 4 cpus on 2 nodes
> [    0.000000] ************** Topology ********************
> [    0.000000]     00 01
> [    0.000000] 00  255 255
> [    0.000000] 01  255 255
> [    0.000000] bootconsole [early0] enabled
> [    0.000000] CPU0 revision is: 00000f14 (R14000)
> [    0.000000] FPU revision is: 00000900
> [    0.000000] Checking for the multiply/shift bug... no.
> [    0.000000] Checking for the daddiu bug... no.
> [    0.000000] IP27: Running on node 0.
> [    0.000000] IP27: Node 0 has a primary CPU, CPU is running.
> [    0.000000] IP27: Node 0 has a secondary CPU, CPU is running.
> [    0.000000] IP27: Machine is in M mode.
> [    0.000000] IP27: CPU 1A (CPU0): 500 MHz CPU detected.
> [    0.000000] Determined physical RAM map:
> [    0.000000]  memory: 0000000000838000 @ 000000000001c000 (usable)
> [    0.000000]  memory: 000000000057c000 @ 0000000000854000 (usable after init)
> [    0.000000] Initrd not found or empty - disabling initrd
> [    0.000000] cma: Reserved 32 MiB at 0x00000001fe000000
> [    0.000000] REPLICATION: ON nasid 0, ktext from nasid 0, kdata from nasid 0
> [    0.000000] REPLICATION: ON nasid 1, ktext from nasid 0, kdata from nasid 0
> [    0.000000] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
> [    0.000000] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
> [    0.000000] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
> [    0.000000] Zone ranges:
> [    0.000000]   Normal   [mem 0x0000000000000000-0x00000001ffffffff]
> [    0.000000] Movable zone start for each node
> [    0.000000] Early memory node ranges
> [    0.000000]   node   0: [mem 0x0000000000000000-0x00000000ffffffff]
> [    0.000000]   node   1: [mem 0x0000000100000000-0x00000001ffffffff]
> [    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000ffffffff]
> [    0.000000] Initmem setup node 1 [mem 0x0000000100000000-0x00000001ffffffff]
> [    0.000000] percpu: Embedded 4 pages/cpu @a800000002010000 s36096 r0 d29440 u65536
> [    0.000000] Built 2 zonelists in Node order, mobility grouping on.  Total pages: 522240
> [    0.000000] Policy zone: Normal
> [    0.000000] Kernel command line: root=dksc(0,1,0) console=ttyS0,9600
> [    0.000000] PID hash table entries: 4096 (order: 1, 32768 bytes)
> [    0.000000] Memory: 8307776K/8388608K available (6498K kernel code, 454K rwdata, 1436K rodata, 5616K init, 543K bss, 48064K reserved, 32768K cma-reserved)
> [    0.000000] Hierarchical RCU implementation.
> [    0.000000]  Build-time adjustment of leaf fanout to 64.
> [    0.000000]  RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4.
> [    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=4
> [    0.000000] NR_IRQS:256
> [    0.000000] clocksource: HUB: mask: 0xfffffffffffff max_cycles: 0x127350b88, max_idle_ns: 1763180808480 ns
> [    0.000019] sched_clock: 52 bits at 1250kHz, resolution 800ns, wraps every 4398046510800ns
> [    0.102422] Console: colour dummy device 80x25
> [    0.154975] Calibrating delay loop... 749.56 BogoMIPS (lpj=3747840)
> [    0.317670] pid_max: default: 32768 minimum: 301
> [    0.377938] Dentry cache hash table entries: 1048576 (order: 9, 8388608 bytes)
> [    0.592303] Inode-cache hash table entries: 524288 (order: 8, 4194304 bytes)
> [    0.736316] Mount-cache hash table entries: 16384 (order: 3, 131072 bytes)
> [    0.817310] Mountpoint-cache hash table entries: 16384 (order: 3, 131072 bytes)
> [    0.916094] Checking for the daddi bug... no.
> [    0.976495] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
> [    0.976520] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
> [    0.976532] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
> [    0.979792] IP27: CPU 1B (CPU1): 500 MHz CPU detected.
> [    0.979824] CPU1 revision is: 00000f14 (R14000)
> [    0.979831] FPU revision is: 00000900
> [    1.014463] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
> [    1.014493] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
> [    1.014506] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
> [    1.017888] IP27: CPU 2A (CPU2): 500 MHz CPU detected.
> [    1.017931] CPU2 revision is: 00000f14 (R14000)
> [    1.017938] FPU revision is: 00000900
> [    1.055420] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
> [    1.055452] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
> [    1.055467] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
> [    1.058810] IP27: CPU 2B (CPU3): 500 MHz CPU detected.
> [    1.058851] CPU3 revision is: 00000f14 (R14000)
> [    1.058858] FPU revision is: 00000900
> [    1.092672] Brought up 4 CPUs
> [    2.484801] devtmpfs: initialized
> [    2.603700] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> [    2.892716] random: fast init done
> [    2.990752] xor: measuring software checksum speed
> [    3.146730]    8regs     :   486.400 MB/sec
> [    3.295350]    8regs_prefetch:   486.400 MB/sec
> [    3.448144]    32regs    :   486.400 MB/sec
> [    3.596768]    32regs_prefetch:   486.400 MB/sec
> [    3.650415] xor: using function: 32regs_prefetch (486.400 MB/sec)
> [    3.834852] NET: Registered protocol family 16
> [    3.946535] cpuidle: using governor ladder
> [    4.043955] cpuidle: using governor menu
> [    4.089355] xtalk:n0/0 xbow widget (rev 1.3)
> [    4.140796] xtalk:n0/8 kona widget (rev unknown) registered as a platform device.
> [    4.230796] xtalk:n0/b bridge widget (rev D) registered as a platform device.
> [    4.316712] xtalk:n0/c bridge widget (rev D) registered as a platform device.
> [    4.402590] xtalk:n0/d impact widget (rev B) registered as a platform device.
> [    4.488444] xtalk:n0/f bridge widget (rev C) registered as a platform device.
> [    4.574178] xtalk:n1/0 xbow widget (rev 1.3)
> [    5.247396] raid6: int64x1  gen()    84 MB/s
> [    5.467370] raid6: int64x1  xor()    80 MB/s
> [    5.687324] raid6: int64x2  gen()   151 MB/s
> [    5.907243] raid6: int64x2  xor()   116 MB/s
> [    6.127125] raid6: int64x4  gen()   235 MB/s
> [    6.347155] raid6: int64x4  xor()   142 MB/s
> [    6.567117] raid6: int64x8  gen()   251 MB/s
> [    6.787018] raid6: int64x8  xor()   175 MB/s
> [    6.836420] raid6: using algorithm int64x8 gen() 251 MB/s
> [    6.901352] raid6: .... xor() 175 MB/s, rmw enabled
> [    6.960002] raid6: using intx1 recovery algorithm
> [    7.017544] SCSI subsystem initialized
> [    7.063853] clocksource: Switched to clocksource HUB
> [    7.363862] FS-Cache: Loaded
> [   31.643872] NET: Registered protocol family 2
> [   31.696149] TCP established hash table entries: 65536 (order: 5, 524288 bytes)
> [   31.787294] TCP bind hash table entries: 65536 (order: 6, 1048576 bytes)
> [   31.874832] TCP: Hash tables configured (established 65536 bind 65536)
> [   31.952084] UDP hash table entries: 4096 (order: 3, 131072 bytes)
> [   32.026304] UDP-Lite hash table entries: 4096 (order: 3, 131072 bytes)
> [   32.403836] NET: Registered protocol family 1
> [   32.583876] RPC: Registered named UNIX socket transport module.
> [   32.653282] RPC: Registered udp transport module.
> [   32.709785] RPC: Registered tcp transport module.
> [   32.766346] RPC: Registered tcp NFSv4.1 backchannel transport module.
> [   34.045622] futex hash table entries: 1024 (order: 3, 131072 bytes)
> [   34.275238] workingset: timestamp_bits=40 max_order=19 bucket_order=0
> [   34.350961] zbud: loaded
> [   37.383916] NFS: Registering the id_resolver key type
> [   37.442917] Key type id_resolver registered
> [   37.493101] Key type id_legacy registered
> [   37.541388] efs: 1.0a - http://aeschi.ch.eu.org/efs/
> [   37.873853] SGI XFS with ACLs, security attributes, realtime, no debug enabled
> [   39.453936] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
> [   39.541066] io scheduler noop registered
> [   39.588223] io scheduler deadline registered
> [   39.639639] io scheduler cfq registered (default)
> [   39.973936] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> [   40.148640] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [   40.245916] loop: module loaded
> [   40.283288] mousedev: PS/2 mouse device common for all mice
> [   40.349460] md: linear personality registered for level -1
> [   40.414802] md: raid0 personality registered for level 0
> [   40.478677] md: raid1 personality registered for level 1
> [   40.542549] md: raid10 personality registered for level 10
> [   40.609068] md: raid6 personality registered for level 6
> [   40.672411] md: raid5 personality registered for level 5
> [   40.736315] md: raid4 personality registered for level 4
> [   40.800808] PCI host bridge to bus 0000:00
> [   40.849424] pci_bus 0000:00: root bus resource [mem 0x920000000b200000-0x920000000b9fffff]
> [   40.948920] pci_bus 0000:00: root bus resource [io  0x920000000ba00000-0x920000000bbfffff]
> [   41.048408] pci_bus 0000:00: root bus resource [bus 00-ff]
> [   41.114880] PCI host bridge to bus 0001:00
> [   41.163629] pci_bus 0001:00: root bus resource [mem 0x920000000c200000-0x920000000c9fffff]
> [   41.263152] pci_bus 0001:00: root bus resource [io  0x920000000ca00000-0x920000000cbfffff]
> [   41.362645] pci_bus 0001:00: root bus resource [bus 01-ff]
> [   41.430847] PCI host bridge to bus 0002:00
> [   41.478263] pci_bus 0002:00: root bus resource [mem 0x920000000f200000-0x920000000f9fffff]
> [   41.577771] pci_bus 0002:00: root bus resource [io  0x920000000fa00000-0x920000000fbfffff]
> [   41.677250] pci_bus 0002:00: root bus resource [bus 02-ff]
> [   41.746742] qla1280: QLA1040 found on PCI bus 0, dev 0
> [   41.806687] PCI: Enabling device 0002:00:00.0 (0006 -> 0007)
> [   41.875796] scsi(0): Enabling vchannel on BRIDGE for SGI/MIPS
> [   42.324520] random: crng init done
> [   42.839160] scsi(0:0): Resetting SCSI BUS
> [   45.964232] scsi host0: QLogic QLA1040 PCI to SCSI Host Adapter
> [   45.964232]        Firmware version:  7.65.06, Driver version 3.27.1
> [   46.118149] qla1280: QLA1040 found on PCI bus 0, dev 1
> [   46.178192] PCI: Enabling device 0002:00:01.0 (0006 -> 0007)
> [   46.289985] scsi 0:0:1:0: Direct-Access     SEAGATE  SX150176LC       BA12 PQ: 0 ANSI: 2
> [   46.387174] scsi(0:0:1:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
> [   46.486875] scsi 0:0:2:0: Direct-Access     SEAGATE  SX150176LC       BA08 PQ: 0 ANSI: 2
> [   46.583893] scsi(0:0:2:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
> [   46.683736] scsi 0:0:3:0: Direct-Access     SEAGATE  SX150176LC       BA08 PQ: 0 ANSI: 2
> [   46.780696] scsi(0:0:3:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
> [   46.880821] scsi 0:0:4:0: Direct-Access     SEAGATE  SX150176LC       BA11 PQ: 0 ANSI: 2
> [   46.977716] scsi(0:0:4:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
> [   47.077797] scsi 0:0:5:0: Direct-Access     SEAGATE  SX150176LC       BA11 PQ: 0 ANSI: 2
> [   47.174768] scsi(0:0:5:0):[   47.181219] scsi(1:0): Resetting SCSI BUS
> [   47.251839]  Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
> [   47.639912] scsi 0:0:6:0: CD-ROM            TOSHIBA  CD-ROM XM-5701TA 0167 PQ: 0 ANSI: 2
> [   47.736890] scsi(0:0:6:0): Sync: period 10, offset 12
> [   51.553976] scsi host1: QLogic QLA1040 PCI to SCSI Host Adapter
> [   51.553976]        Firmware version:  7.65.06, Driver version 3.27.1
> [   51.998859] ioc3: part: [], serial: [] => class IP27 BaseIO
> [   52.068783] ioc3-eth: Ethernet address is 08:00:69:05:64:74.
> [   52.136831] IOC3 0002:00:02.0 eth0: link up, 100Mbps, full-duplex, lpa 0x8DE1
> [   52.220897] eth0: Using PHY 31, vendor 0x20005c0, model 0, rev 0.
> [   52.294232] eth0: IOC3 SSRAM has 128 kbyte.
> [   52.345852] rtc-m48t35 rtc-m48t35: rtc core: registered m48t35 as rtc0
> [   52.588471] console [ttyS0] enabled
> [   52.489601] 0002:00:02.0: ttyS0 at IOC3 0xf620178 (irq = 0, base_baud = 458333) is a 16550A
> [   52.671141] bootconsole [early0] disabled
> [   54.146680] 0002:00:02.0: ttyS1 at IOC3 0xf620170 (irq = 0, base_baud = 458333) is a 16550A
> [   54.248408] IOC3 Master Driver loaded for 0002:00:02.0
> [   54.313895] ioc3: NIC search failed.
> [   54.356992] ioc3: part: [], serial: [] => class IP27 BaseIO
> [   54.424356] ioc3: request_irq fails for IRQ 0xffffffff
> [   54.490948] ioc3-eth: Ethernet address is 00:00:00:00:00:00.
> [   54.561038] IOC3 0002:00:06.0 eth1: link down
> [   54.613597] eth1: Using PHY 31, vendor 0x0, model 0, rev 0.
> [   54.680825] eth1: IOC3 SSRAM has 64 kbyte.
> [   54.765266] 0002:00:06.0: ttyS2 at IOC3 0xfa20178 (irq = 0, base_baud = 458333) is a 16550A
> [   54.888655] 0002:00:06.0: ttyS3 at IOC3 0xfa20170 (irq = 0, base_baud = 458333) is a 16550A
> [   54.990198] IOC3 Master Driver loaded for 0002:00:06.0
> [   55.052840] NET: Registered protocol family 26
> [   55.509832] sd 0:0:1:0: [sda] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
> [   55.515697] sr 0:0:6:0: [sr0] scsi-1 drive
> [   55.515709] cdrom: Uniform CD-ROM driver Revision: 3.20
> [   55.520834] sd 0:0:4:0: [sdd] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
> [   55.523611] sd 0:0:4:0: [sdd] Write Protect is off
> [   55.526511] sd 0:0:4:0: [sdd] Write cache: disabled, read cache: enabled, supports DPO and FUA
> [   55.550956]  sdd: sdd1 sdd2 sdd3 sdd4 sdd5 sdd6 sdd9 sdd11
> [   55.562326] sd 0:0:4:0: [sdd] Attached SCSI disk
> [   55.564803] sd 0:0:5:0: [sde] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
> [   55.567564] sd 0:0:5:0: [sde] Write Protect is off
> [   55.570463] sd 0:0:5:0: [sde] Write cache: disabled, read cache: enabled, supports DPO and FUA
> [   55.600908] sd 0:0:5:0: [sde] Attached SCSI disk
> [   55.604715] sd 0:0:2:0: [sdb] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
> [   55.605297] sd 0:0:3:0: [sdc] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
> [   55.607054] sd 0:0:2:0: [sdb] Write Protect is off
> [   55.607592] sd 0:0:3:0: [sdc] Write Protect is off
> [   55.609959] sd 0:0:2:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
> [   55.610460] sd 0:0:3:0: [sdc] Write cache: disabled, read cache: enabled, supports DPO and FUA
> [   55.631199]  sdb: sdb1 sdb2 sdb3 sdb4 sdb5 sdb6 sdb9 sdb11
> [   55.634048]  sdc: sdc1 sdc2 sdc3 sdc4 sdc5 sdc6 sdc9 sdc11
> [   55.646437] sd 0:0:3:0: [sdc] Attached SCSI disk
> [   55.647162] sd 0:0:2:0: [sdb] Attached SCSI disk
> [   57.150140] sd 0:0:1:0: [sda] Write Protect is off
> [   57.173988] NET: Registered protocol family 10
> [   57.264566] sd 0:0:1:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
> [   57.386960]  sda: sda1 sda2 sda3 sda4 sda5 sda6 sda9 sda11
> [   57.464000] sd 0:0:1:0: [sda] Attached SCSI disk
> [   57.544064] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
> [   57.617430] NET: Registered protocol family 17
> [   57.671360] 8021q: 802.1Q VLAN Support v1.8
> [   57.722075] sctp: Hash tables configured (bind 1024/1024)
> [   58.093962] Key type dns_resolver registered
> [   58.148015] rtc-m48t35 rtc-m48t35: setting system clock to 2016-10-17 23:49:58 UTC (1476748198)
> [   59.626451] Freeing unused kernel memory: 5616K (a800000000854000 - a800000000dd0000)
> [   59.720950] This architecture does not have kernel memory protection.
> 
> 
> Next is /proc/iomem and /proc/ioports:
> # cat /proc/iomem
> 0f600000-0f6fffff : ioc3
>   0f680000-0f687fff : rtc-m48t35
> 0fa00000-0fafffff : ioc3
> 920000000b200000-920000000b9fffff : Bridge MEM
> 920000000c200000-920000000c9fffff : Bridge MEM
> 920000000f200000-920000000f9fffff : Bridge MEM
> 
> / # cat /proc/ioports
> 920000000ba00000-920000000bbfffff : Bridge IO
> 920000000ca00000-920000000cbfffff : Bridge IO
> 920000000fa00000-920000000fbfffff : Bridge IO
> 
> 
> And lspci -vvvx:
> 
> / # /usr/sbin/lspci -vvvx
> 0001:00:00.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. Linc I/O controller
>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         NUMA node: 0
>         Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=134217729]
>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Expansion ROM at <unassigned> [disabled] [size=2]
> lspci: Unable to load libkmod resources: error -12
> 00: a9 10 02 00 00 00 00 02 00 00 00 ff 00 f8 80 00
> 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 
> 0001:00:01.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. Linc I/O controller
>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         NUMA node: 0
>         Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=134217729]
>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Expansion ROM at <unassigned> [disabled] [size=2]
> 00: a9 10 02 00 00 00 00 02 00 00 00 ff 00 f8 80 00
> 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 
> 0002:00:00.0 SCSI storage controller: QLogic Corp. ISP1020 Fast-wide SCSI (rev 05)
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         Latency: 64, Cache Line Size: 256 bytes
>         Interrupt: pin A routed to IRQ 1
>         NUMA node: 0
>         Region 0: I/O ports at f200000 [size=257]
>         Region 1: Memory at 0f200000 (32-bit, non-prefetchable) [size=4097]
>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Expansion ROM at <unassigned> [disabled] [size=65537]
>         Kernel driver in use: qla1280
> 00: 77 10 20 10 07 00 00 02 05 00 00 01 40 40 00 00
> 10: 01 00 20 0f 00 00 20 0f 00 00 00 00 00 00 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
> 
> 0002:00:01.0 SCSI storage controller: QLogic Corp. ISP1020 Fast-wide SCSI (rev 05)
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         Latency: 64, Cache Line Size: 256 bytes
>         Interrupt: pin A routed to IRQ 2
>         NUMA node: 0
>         Region 0: I/O ports at f400000 [size=257]
>         Region 1: Memory at 0f400000 (32-bit, non-prefetchable) [size=4097]
>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Expansion ROM at <unassigned> [disabled] [size=65537]
>         Kernel driver in use: qla1280
> 00: 77 10 20 10 07 00 00 02 05 00 00 01 40 40 00 00
> 10: 01 00 40 0f 00 00 40 0f 00 00 00 00 00 00 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
> 
> 0002:00:02.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. IOC3 I/O controller (rev 01)
>         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>         Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         Latency: 64
>         Interrupt: pin A routed to IRQ 3
>         NUMA node: 0
>         Region 0: Memory at 0f600000 (32-bit, non-prefetchable) [size=1048577]
>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Expansion ROM at <unassigned> [disabled] [size=2]
>         Kernel driver in use: IOC3
> 00: a9 10 03 00 06 00 80 02 01 00 00 ff 00 40 00 00
> 10: 00 00 60 0f 00 00 00 00 00 00 00 00 00 00 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
> 
> 0002:00:06.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. IOC3 I/O controller (rev 01)
>         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>         Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         Latency: 64
>         Interrupt: pin A routed to IRQ 5
>         NUMA node: 0
>         Region 0: Memory at 0fa00000 (32-bit, non-prefetchable) [size=1048577]
>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>         Expansion ROM at <unassigned> [disabled] [size=2]
>         Kernel driver in use: IOC3
> 00: a9 10 03 00 06 00 80 02 01 00 00 ff 00 40 00 00
> 10: 00 00 a0 0f 00 00 00 00 00 00 00 00 00 00 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
> 
> 0002:00:07.0 Non-VGA unclassified device: Silicon Graphics Intl. Corp. RAD Audio (rev 70)
>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>         Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         NUMA node: 0
>         Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=8193]
>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>         Expansion ROM at <unassigned> [disabled] [size=2]
> 00: a9 10 05 00 00 00 80 04 70 00 00 00 00 00 00 00
> 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 
> 
> IP27 hardware does some interesting things with address spaces, depending on
> which "mode" you run the machine in.  "M mode" is the only one that's really
> usable these days (the other, "N mode", doesn't boot).  Under M mode,
> there's five address spaces, of which, "I/O Space" is selectable by turning on
> specific bits in the 64-bit memory address.
> 
> So, given 0x920000000b200000, bits 63:62 select physical addressing (10) and
> bits 58:57 select I/O space (01), resulting in all PCI addresses (or any I/O
> address) appearing in the 0x92xxxxxxxxxxxxxx range.  The rest of the bits go
> on to select which NUMA node the I/O is targeted at and which Crosstalk widget
> to access, and so on.  I still haven't fully understood it all myself just
> yet.
> 
> You can google for document 007-3410-001, "Origin and Onyx2 Programmer's
> Reference Manual" for more details (it's no longer available via SGI's
> TechPubs, but it is archived on plenty of other sites).
> 
> This particular machine that I have has three "BRIDGE" chips in it, all
> attached to the Crosstalk bus.  Each BRIDGE can address up to 8 PCI devices,
> but some crosstalk boards slap a BRIDGE chip on for just a single device.  In
> this instance, the BRIDGE w/ address 0x920000000fxxxxxx is the IO6 at xtalk
> widget 0xf (this has the basic I/O devices for this machine).  The BRIDGE at
> 0x920000000cxxxxxx (widget 0xc) is a PCI "shoebox" that can hold up to three
> PCI-X cards, and the BRIDGE at 0x920000000bxxxxxx (widget 0xb) is an
> SGI-specific HIPPI board (ID'ed as a LINC I/O controller by pci.ids).
> 
> -- 
> Joshua Kinard
> Gentoo/MIPS
> kumba@gentoo.org
> 6144R/F5C6C943 2015-04-27
> 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943
> 
> "The past tempts us, the present confuses us, the future frightens us.  And
> our lives slip away, moment by moment, lost in that vast, terrible in-between."
> 
> --Emperor Turhan, Centauri Republic
> 

Re: [PATCH] MIPS/PCI: Claim bus resources on PCI_PROBE_ONLY set-ups

[Joshua Kinard]

On 10/19/2016 07:04, Lorenzo Pieralisi wrote:
> On Mon, Oct 17, 2016 at 08:30:38PM -0400, Joshua Kinard wrote:
>> On 10/17/2016 14:09, Lorenzo Pieralisi wrote:
>>> On Mon, Oct 17, 2016 at 12:36:34AM -0400, Joshua Kinard wrote:
>>>> On 06/23/2016 18:16, Bjorn Helgaas wrote:
>>>>> We claim PCI BAR and bridge window resources in pci_bus_assign_resources(),
>>>>> but when PCI_PROBE_ONLY is set, we treat those resources as immutable and
>>>>> don't call pci_bus_assign_resources(), so the resources aren't put in the
>>>>> resource tree.
>>>>>
>>>>> When the resources aren't in the tree, they don't show up in /proc/iomem,
>>>>> we can't detect conflicts, and we need special cases elsewhere for
>>>>> PCI_PROBE_ONLY or resources without a parent pointer.
>>>>>
>>>>> Claim all PCI BAR and window resources in the PCI_PROBE_ONLY case.
>>>>>
>>>>> If a PCI_PROBE_ONLY platform assigns conflicting resources, Linux can't fix
>>>>> the conflicts.  Previously we didn't notice the conflicts, but now we will,
>>>>> which may expose new failures.
>>>>>
>>>>> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>>>>> ---
>>
>> [snip]
>>
>>>>
>>>> So I finally tested this on two SGI platforms, IP30 and IP27.  For IP30, it
>>>> works fine once I drop the "pci_set_flags(PCI_PROBE_ONLY);" call.  IP30 uses
>>>> standard virtual addressing via ioremap and the like for all of its PCI stuff,
>>>> so it was trivial to fix it.
>>>>
>>>> IP27, on the other hand, is not happy with this change.  I don't know about
>>>> mainline IP27, as that's generally bitrotted and didn't boot the last time I
>>>> tried it, but under the refactored code I'm using that actually boots, this
>>>> platform still needs to bypass *all* attempts to claim or assign the PCI addresses.
>>>>
>>>> I suspect the difference with IP27 is unlike normal systems, IP27 uses special
>>>> physical addresses in the 0x92xxxxxxxxxxxxxx block to address I/O devices, so
>>>> there's no proper ioremap or use of virtual addressing for I/O.  Ralf, does
>>>> this sound correct?
>>>>
>>>> If I use the patches I got plus this change and set PCI_PROBE_ONLY, this is
>>>> what happens:
>>>>
>>>> [   47.299511] PCI host bridge to bus 0001:00
>>>> [   47.348155] pci_bus 0001:00: root bus resource [mem
>>>> 0x920000000f200000-0x920000000f9fffff]
>>>> [   47.447663] pci_bus 0001:00: root bus resource [io
>>>> 0x920000000fa00000-0x920000000fbfffff]
>>>> [   47.547183] pci_bus 0001:00: root bus resource [bus 01-ff]
>>>> [   47.616581] pci 0001:00:00.0: can't claim BAR 0 [io  0xf200000-0xf2000ff]:
>>>> no compatible bridge window
>>>> [   47.726766] pci 0001:00:00.0: can't claim BAR 1 [mem 0x0f200000-0x0f200fff]:
>>>> no compatible bridge window
>>>> [   47.840937] pci 0001:00:00.0: can't claim BAR 6 [mem 0x00000000-0x0000ffff
>>>> pref]: no compatible bridge window
>>>> [   47.960317] pci 0001:00:01.0: can't claim BAR 0 [io  0xf400000-0xf4000ff]:
>>>> no compatible bridge window
>>>> [   48.072399] pci 0001:00:01.0: can't claim BAR 1 [mem 0x0f400000-0x0f400fff]:
>>>> no compatible bridge window
>>>> [   48.186560] pci 0001:00:01.0: can't claim BAR 6 [mem 0x00000000-0x0000ffff
>>>> pref]: no compatible bridge window
>>>> [   48.305957] pci 0001:00:02.0: can't claim BAR 0 [mem 0x0f600000-0x0f6fffff]:
>>>> no compatible bridge window
>>>> [   48.420098] pci 0001:00:06.0: can't claim BAR 0 [mem 0x0fa00000-0x0fafffff]:
>>>> no compatible bridge window
>>>> [   48.534294] pci 0001:00:07.0: can't claim BAR 0 [mem 0x00000000-0x00001fff]:
>>>> no compatible bridge window
>>>> [   48.648541] qla1280: QLA1040 found on PCI bus 0, dev 0
>>>> [   48.710232] PCI: Enabling device 0001:00:00.0 (0006 -> 0007)
>>>> [   48.779231] qla1280 0001:00:00.0: can't ioremap BAR 1: [mem size 0x00001000]
>>>> [   48.863145] qla1280: Unable to map I/O memory
>>>> [   48.916520] qla1280: QLA1040 found on PCI bus 0, dev 1
>>>> [   48.977298] PCI: Enabling device 0001:00:01.0 (0006 -> 0007)
>>>> [   49.046128] qla1280 0001:00:01.0: can't ioremap BAR 1: [mem size 0x00001000]
>>>> [   49.130201] qla1280: Unable to map I/O memory
>>>> [   49.183068] IOC3 0001:00:02.0: can't ioremap BAR 0: [mem size 0x00100000]
>>>> [   49.264292] ioc3: Unable to remap PCI BAR for 0001:00:02.0.
>>>> [   49.331533] IOC3 0001:00:06.0: can't ioremap BAR 0: [mem size 0x00100000]
>>>> [   49.412984] ioc3: Unable to remap PCI BAR for 0001:00:06.0.
>>>>
>>>>
>>>> If I disable PCI_PROBE_ONLY, then this:
>>>>
>>>> [   47.588033] PCI host bridge to bus 0001:00
>>>> [   47.636649] pci_bus 0001:00: root bus resource [mem
>>>> 0x920000000f200000-0x920000000f9fffff]
>>>> [   47.736144] pci_bus 0001:00: root bus resource [io
>>>> 0x920000000fa00000-0x920000000fbfffff]
>>>> [   47.835621] pci_bus 0001:00: root bus resource [bus 01-ff]
>>>> [   47.905023] pci 0001:00:02.0: BAR 0: no space for [mem size 0x00100000]
>>>> [   47.982775] pci 0001:00:02.0: BAR 0: failed to assign [mem size 0x00100000]
>>>> [   48.066586] pci 0001:00:06.0: BAR 0: no space for [mem size 0x00100000]
>>>> [   48.146173] pci 0001:00:06.0: BAR 0: failed to assign [mem size 0x00100000]
>>>> [   48.229972] pci 0001:00:00.0: BAR 6: no space for [mem size 0x00010000 pref]
>>>> [   48.314783] pci 0001:00:00.0: BAR 6: failed to assign [mem size 0x00010000 pref]
>>>> [   48.403825] pci 0001:00:01.0: BAR 6: no space for [mem size 0x00010000 pref]
>>>> [   48.488655] pci 0001:00:01.0: BAR 6: failed to assign [mem size 0x00010000 pref]
>>>> [   48.577695] pci 0001:00:00.0: BAR 1: no space for [mem size 0x00001000]
>>>> [   48.657260] pci 0001:00:00.0: BAR 1: failed to assign [mem size 0x00001000]
>>>> [   48.741068] pci 0001:00:01.0: BAR 1: no space for [mem size 0x00001000]
>>>> [   48.820644] pci 0001:00:01.0: BAR 1: failed to assign [mem size 0x00001000]
>>>> [   48.904452] pci 0001:00:00.0: BAR 0: no space for [io  size 0x0100]
>>>> [   48.979840] pci 0001:00:00.0: BAR 0: failed to assign [io  size 0x0100]
>>>> [   49.059459] pci 0001:00:01.0: BAR 0: no space for [io  size 0x0100]
>>>> [   49.134855] pci 0001:00:01.0: BAR 0: failed to assign [io  size 0x0100]
>>>> [   49.214588] qla1280: QLA1040 found on PCI bus 0, dev 0
>>>> [   49.277249] qla1280 0001:00:00.0: can't ioremap BAR 1: [??? 0x00000000 flags
>>>> 0x0]
>>>> [   49.366330] qla1280: Unable to map I/O memory
>>>> [   49.419832] qla1280: QLA1040 found on PCI bus 0, dev 1
>>>> [   49.481211] qla1280 0001:00:01.0: can't ioremap BAR 1: [??? 0x00000000 flags
>>>> 0x0]
>>>> [   49.570560] qla1280: Unable to map I/O memory
>>>> [   49.623439] IOC3 0001:00:02.0: can't ioremap BAR 0: [??? 0x00000000 flags 0x0]
>>>> [   49.709857] ioc3: Unable to remap PCI BAR for 0001:00:02.0.
>>>> [   49.777113] IOC3 0001:00:06.0: can't ioremap BAR 0: [??? 0x00000000 flags 0x0]
>>>> [   49.863796] ioc3: Unable to remap PCI BAR for 0001:00:06.0.
>>>>
>>>>
>>>> The only fix I could come up with was more IP27 #ifdef hackery to retain the
>>>> original code for IP27, but use the new conditional logic for all other
>>>> platforms.  Would a patch for this be acceptable (with verbose comment), or is
>>>> there a better way to deal with IP27 that doesn't involve re-writing its I/O
>>>> addressing support entirely?
>>>
>>> Would you mind providing a kernel boot log, lspci output and /proc/iomem
>>> /proc/ioports for your "working" system boot ? That would help debugging
>>> what's the expected resources allocation for the system and consequently
>>> where the problem currently lies.
>>
>> Sure.  I've disabled line-wrapping on this message so my mail client doesn't
>> mangle the output.
> 
> Ok, I think this exactly showed what Bjorn wanted to see by claiming
> resources because the problem is that the value in the BARs is not
> contained in any bridge window, for the reason that you just mentioned
> (ie the "special" physical addresses flags).
> 
> In the PCI_PROBE_ONLY case, claiming resources fail, which means that
> we set the resource flags res->flags |= IORESOURCE_UNSET, which causes
> pci_ioremap_bar() to fail.
> 
> So, it is correct to claim the resources if PCI_PROBE_ONLY, this fails
> because the values in the BARs and the PCI host bridge window resources
> have different formats (ie the PCI bridge window resources have addresses
> that contain the special flags you mentioned) the question I have is
> why the PCI root bridge window should be set with the flags in it
> instead of a "pure" IO address.
> 
> Is my reading of the problem correct ?

Sorry, been tied up lately.  I haven't done a deep-dive of the way the PCI
code actually assigns/claims addresses, but this sounds mostly correct to
me.  Reading in the document I cited, these addresses are actually "virtual
addresses", but I am uncertain if they're "virtual" in the same sense that
the Linux kernel regards as virtual (as in allocated by the kernel's memory
manager).


> Why do the PCI host bridge windows have to contain that 0x92 tag in the
> upper bits ?

It's a hardware-specific mechanism.  The IP27 platform was a
massively-scalable (for its time) system that supported up to 128 CPUs
spread across 64 nodeboards in 8 racks (4 nodeboards per system module).  So
SGI designed the address space big enough such that all nodes and all
devices assigned to each node could be addressed by using offsets from the
address space of Node 0.

The basic bit assignment for an address runs like this:

63:62 - address space selection
61:59 - cache algorithm
58:57 - uncached attribute
56:44 - not translated by TLB
39:00 - index to physical space when 63:62 == 10b (physical addressing)

IP27 uses this bit assignment to layout five address spaces:
  - Cached (63:62 are any value other than 10b)
  - HUB Special (58:57 == 00b)
  - I/O Special (58:57 == 01b)
  - Memory Special (58:57 == 10b)
  - Uncached (58:57 == 11b)

So if 63:62 are 10b (physical address), 61:59 are 010b (uncached), and 58:57
are 01b (I/O space), that results in the address 0x92xxxxxxxxxxxxxx, which
tells the hardware you're trying to access uncached I/O space, and thus, an
I/O device.  The remaining bits in the address are used to specify the
particular node the device is attached to all the way down to the
device-specific address value itself.  There's also things like "little
windows" and "big windows" that I don't fully understand just yet, as
they're a function of the Crosstalk (XIO) bus that the PCI busses are
actually attached to within the system.


> By not claiming resources, as you can see in the log below
> /proc/iomem and /proc/ioports do not report the PCI devices resource
> usages and the respective resources are not reserved in the kernel,
> which is wrong and that's exactly why Bjorn patched the kernel.
> 
> Please let me know.
> 
> Thanks,
> Lorenzo

Yup, I can see that on my SGI Octane machine (IP30 platform), which is
architecturally-similar to IP27, but uses a more "normal" addressing layout.
 Previously, it was also setting the PCI_PROBE_ONLY flag and thus
/proc/iomem looked very similar to IP27's output.  Now, however, Octane's
/proc/iomem looks more like this:

# cat /proc/iomem
00000000-00003fff : reserved
1d200000-1d9fffff : Bridge MEM
  d080000000-d080003fff : 0001:00:03.0
  1d204000-1d204fff : 0001:00:02.0
  1d205000-1d205fff : 0001:00:02.1
  1d206000-1d206fff : 0001:00:02.2
  1d207000-1d2070ff : 0001:00:02.3
  1d207100-1d20717f : 0001:00:01.0
1f200000-1f9fffff : Bridge MEM
  f080100000-f0801fffff : 0000:00:02.0
  f080010000-f08001ffff : 0000:00:00.0
  f080030000-f08003ffff : 0000:00:01.0
  f080000000-f080000fff : 0000:00:00.0
  f080020000-f080020fff : 0000:00:01.0
20004000-20a27fff : reserved
  20004000-207180b7 : Kernel code
  207180b8-2090ffff : Kernel data
20a28000-20efffff : System RAM
20f00000-20ffffff : System RAM
21000000-9fffffff : System RAM
f080100000-f0801fffff : ioc3

And /proc/ioports:
# cat /proc/ioports
1da00000-1dbfffff : Bridge IO
  1da00000-1da000ff : 0001:00:01.0
  1da00400-1da0047f : 0001:00:01.0
1fa00000-1fbfffff : Bridge IO
  f100000000-f1000000ff : 0000:00:00.0
  f100000100-f1000001ff : 0000:00:01.0

PCI support on IP27 has been a bit weird for a long time, mostly due to it
using a lot of "IRIX'isms" in the code to get it working initially.  Some of
these are actually short-circuits for things that would instead use normal
kernel mechanisms, and no one's really had the time or interest to try and
clean it up.  I've done a significant amount of refactoring of the IP27's
machine-specific code so far the last few months, but haven't spent much
time understanding PCI and XIO some more to make further progress.  Lack of
time has made it difficult to break those code changes down into bite-sized
patches to send back upstream (I should've planned things a bit better...).

--J


>> First, is the boot output from my serial console:
>> [    0.000000] Linux version 4.8.2-mipsgit-20161016 (root@helcaraxe) (gcc version 5.4.0 (Gentoo Hardened 5.4.0 p1.0, pie-0.6.5) ) #4 SMP Mon Oct 17 19:48:30 EDT 2016
>> [    0.000000] ARCH: SGI-IP27
>> [    0.000000] PROMLIB: ARC firmware Version 64 Revision 0
>> [    0.000000] SMP: Discovered 4 cpus on 2 nodes
>> [    0.000000] ************** Topology ********************
>> [    0.000000]     00 01
>> [    0.000000] 00  255 255
>> [    0.000000] 01  255 255
>> [    0.000000] bootconsole [early0] enabled
>> [    0.000000] CPU0 revision is: 00000f14 (R14000)
>> [    0.000000] FPU revision is: 00000900
>> [    0.000000] Checking for the multiply/shift bug... no.
>> [    0.000000] Checking for the daddiu bug... no.
>> [    0.000000] IP27: Running on node 0.
>> [    0.000000] IP27: Node 0 has a primary CPU, CPU is running.
>> [    0.000000] IP27: Node 0 has a secondary CPU, CPU is running.
>> [    0.000000] IP27: Machine is in M mode.
>> [    0.000000] IP27: CPU 1A (CPU0): 500 MHz CPU detected.
>> [    0.000000] Determined physical RAM map:
>> [    0.000000]  memory: 0000000000838000 @ 000000000001c000 (usable)
>> [    0.000000]  memory: 000000000057c000 @ 0000000000854000 (usable after init)
>> [    0.000000] Initrd not found or empty - disabling initrd
>> [    0.000000] cma: Reserved 32 MiB at 0x00000001fe000000
>> [    0.000000] REPLICATION: ON nasid 0, ktext from nasid 0, kdata from nasid 0
>> [    0.000000] REPLICATION: ON nasid 1, ktext from nasid 0, kdata from nasid 0
>> [    0.000000] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
>> [    0.000000] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
>> [    0.000000] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
>> [    0.000000] Zone ranges:
>> [    0.000000]   Normal   [mem 0x0000000000000000-0x00000001ffffffff]
>> [    0.000000] Movable zone start for each node
>> [    0.000000] Early memory node ranges
>> [    0.000000]   node   0: [mem 0x0000000000000000-0x00000000ffffffff]
>> [    0.000000]   node   1: [mem 0x0000000100000000-0x00000001ffffffff]
>> [    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000ffffffff]
>> [    0.000000] Initmem setup node 1 [mem 0x0000000100000000-0x00000001ffffffff]
>> [    0.000000] percpu: Embedded 4 pages/cpu @a800000002010000 s36096 r0 d29440 u65536
>> [    0.000000] Built 2 zonelists in Node order, mobility grouping on.  Total pages: 522240
>> [    0.000000] Policy zone: Normal
>> [    0.000000] Kernel command line: root=dksc(0,1,0) console=ttyS0,9600
>> [    0.000000] PID hash table entries: 4096 (order: 1, 32768 bytes)
>> [    0.000000] Memory: 8307776K/8388608K available (6498K kernel code, 454K rwdata, 1436K rodata, 5616K init, 543K bss, 48064K reserved, 32768K cma-reserved)
>> [    0.000000] Hierarchical RCU implementation.
>> [    0.000000]  Build-time adjustment of leaf fanout to 64.
>> [    0.000000]  RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4.
>> [    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=4
>> [    0.000000] NR_IRQS:256
>> [    0.000000] clocksource: HUB: mask: 0xfffffffffffff max_cycles: 0x127350b88, max_idle_ns: 1763180808480 ns
>> [    0.000019] sched_clock: 52 bits at 1250kHz, resolution 800ns, wraps every 4398046510800ns
>> [    0.102422] Console: colour dummy device 80x25
>> [    0.154975] Calibrating delay loop... 749.56 BogoMIPS (lpj=3747840)
>> [    0.317670] pid_max: default: 32768 minimum: 301
>> [    0.377938] Dentry cache hash table entries: 1048576 (order: 9, 8388608 bytes)
>> [    0.592303] Inode-cache hash table entries: 524288 (order: 8, 4194304 bytes)
>> [    0.736316] Mount-cache hash table entries: 16384 (order: 3, 131072 bytes)
>> [    0.817310] Mountpoint-cache hash table entries: 16384 (order: 3, 131072 bytes)
>> [    0.916094] Checking for the daddi bug... no.
>> [    0.976495] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
>> [    0.976520] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
>> [    0.976532] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
>> [    0.979792] IP27: CPU 1B (CPU1): 500 MHz CPU detected.
>> [    0.979824] CPU1 revision is: 00000f14 (R14000)
>> [    0.979831] FPU revision is: 00000900
>> [    1.014463] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
>> [    1.014493] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
>> [    1.014506] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
>> [    1.017888] IP27: CPU 2A (CPU2): 500 MHz CPU detected.
>> [    1.017931] CPU2 revision is: 00000f14 (R14000)
>> [    1.017938] FPU revision is: 00000900
>> [    1.055420] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
>> [    1.055452] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
>> [    1.055467] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
>> [    1.058810] IP27: CPU 2B (CPU3): 500 MHz CPU detected.
>> [    1.058851] CPU3 revision is: 00000f14 (R14000)
>> [    1.058858] FPU revision is: 00000900
>> [    1.092672] Brought up 4 CPUs
>> [    2.484801] devtmpfs: initialized
>> [    2.603700] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
>> [    2.892716] random: fast init done
>> [    2.990752] xor: measuring software checksum speed
>> [    3.146730]    8regs     :   486.400 MB/sec
>> [    3.295350]    8regs_prefetch:   486.400 MB/sec
>> [    3.448144]    32regs    :   486.400 MB/sec
>> [    3.596768]    32regs_prefetch:   486.400 MB/sec
>> [    3.650415] xor: using function: 32regs_prefetch (486.400 MB/sec)
>> [    3.834852] NET: Registered protocol family 16
>> [    3.946535] cpuidle: using governor ladder
>> [    4.043955] cpuidle: using governor menu
>> [    4.089355] xtalk:n0/0 xbow widget (rev 1.3)
>> [    4.140796] xtalk:n0/8 kona widget (rev unknown) registered as a platform device.
>> [    4.230796] xtalk:n0/b bridge widget (rev D) registered as a platform device.
>> [    4.316712] xtalk:n0/c bridge widget (rev D) registered as a platform device.
>> [    4.402590] xtalk:n0/d impact widget (rev B) registered as a platform device.
>> [    4.488444] xtalk:n0/f bridge widget (rev C) registered as a platform device.
>> [    4.574178] xtalk:n1/0 xbow widget (rev 1.3)
>> [    5.247396] raid6: int64x1  gen()    84 MB/s
>> [    5.467370] raid6: int64x1  xor()    80 MB/s
>> [    5.687324] raid6: int64x2  gen()   151 MB/s
>> [    5.907243] raid6: int64x2  xor()   116 MB/s
>> [    6.127125] raid6: int64x4  gen()   235 MB/s
>> [    6.347155] raid6: int64x4  xor()   142 MB/s
>> [    6.567117] raid6: int64x8  gen()   251 MB/s
>> [    6.787018] raid6: int64x8  xor()   175 MB/s
>> [    6.836420] raid6: using algorithm int64x8 gen() 251 MB/s
>> [    6.901352] raid6: .... xor() 175 MB/s, rmw enabled
>> [    6.960002] raid6: using intx1 recovery algorithm
>> [    7.017544] SCSI subsystem initialized
>> [    7.063853] clocksource: Switched to clocksource HUB
>> [    7.363862] FS-Cache: Loaded
>> [   31.643872] NET: Registered protocol family 2
>> [   31.696149] TCP established hash table entries: 65536 (order: 5, 524288 bytes)
>> [   31.787294] TCP bind hash table entries: 65536 (order: 6, 1048576 bytes)
>> [   31.874832] TCP: Hash tables configured (established 65536 bind 65536)
>> [   31.952084] UDP hash table entries: 4096 (order: 3, 131072 bytes)
>> [   32.026304] UDP-Lite hash table entries: 4096 (order: 3, 131072 bytes)
>> [   32.403836] NET: Registered protocol family 1
>> [   32.583876] RPC: Registered named UNIX socket transport module.
>> [   32.653282] RPC: Registered udp transport module.
>> [   32.709785] RPC: Registered tcp transport module.
>> [   32.766346] RPC: Registered tcp NFSv4.1 backchannel transport module.
>> [   34.045622] futex hash table entries: 1024 (order: 3, 131072 bytes)
>> [   34.275238] workingset: timestamp_bits=40 max_order=19 bucket_order=0
>> [   34.350961] zbud: loaded
>> [   37.383916] NFS: Registering the id_resolver key type
>> [   37.442917] Key type id_resolver registered
>> [   37.493101] Key type id_legacy registered
>> [   37.541388] efs: 1.0a - http://aeschi.ch.eu.org/efs/
>> [   37.873853] SGI XFS with ACLs, security attributes, realtime, no debug enabled
>> [   39.453936] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
>> [   39.541066] io scheduler noop registered
>> [   39.588223] io scheduler deadline registered
>> [   39.639639] io scheduler cfq registered (default)
>> [   39.973936] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
>> [   40.148640] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
>> [   40.245916] loop: module loaded
>> [   40.283288] mousedev: PS/2 mouse device common for all mice
>> [   40.349460] md: linear personality registered for level -1
>> [   40.414802] md: raid0 personality registered for level 0
>> [   40.478677] md: raid1 personality registered for level 1
>> [   40.542549] md: raid10 personality registered for level 10
>> [   40.609068] md: raid6 personality registered for level 6
>> [   40.672411] md: raid5 personality registered for level 5
>> [   40.736315] md: raid4 personality registered for level 4
>> [   40.800808] PCI host bridge to bus 0000:00
>> [   40.849424] pci_bus 0000:00: root bus resource [mem 0x920000000b200000-0x920000000b9fffff]
>> [   40.948920] pci_bus 0000:00: root bus resource [io  0x920000000ba00000-0x920000000bbfffff]
>> [   41.048408] pci_bus 0000:00: root bus resource [bus 00-ff]
>> [   41.114880] PCI host bridge to bus 0001:00
>> [   41.163629] pci_bus 0001:00: root bus resource [mem 0x920000000c200000-0x920000000c9fffff]
>> [   41.263152] pci_bus 0001:00: root bus resource [io  0x920000000ca00000-0x920000000cbfffff]
>> [   41.362645] pci_bus 0001:00: root bus resource [bus 01-ff]
>> [   41.430847] PCI host bridge to bus 0002:00
>> [   41.478263] pci_bus 0002:00: root bus resource [mem 0x920000000f200000-0x920000000f9fffff]
>> [   41.577771] pci_bus 0002:00: root bus resource [io  0x920000000fa00000-0x920000000fbfffff]
>> [   41.677250] pci_bus 0002:00: root bus resource [bus 02-ff]
>> [   41.746742] qla1280: QLA1040 found on PCI bus 0, dev 0
>> [   41.806687] PCI: Enabling device 0002:00:00.0 (0006 -> 0007)
>> [   41.875796] scsi(0): Enabling vchannel on BRIDGE for SGI/MIPS
>> [   42.324520] random: crng init done
>> [   42.839160] scsi(0:0): Resetting SCSI BUS
>> [   45.964232] scsi host0: QLogic QLA1040 PCI to SCSI Host Adapter
>> [   45.964232]        Firmware version:  7.65.06, Driver version 3.27.1
>> [   46.118149] qla1280: QLA1040 found on PCI bus 0, dev 1
>> [   46.178192] PCI: Enabling device 0002:00:01.0 (0006 -> 0007)
>> [   46.289985] scsi 0:0:1:0: Direct-Access     SEAGATE  SX150176LC       BA12 PQ: 0 ANSI: 2
>> [   46.387174] scsi(0:0:1:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
>> [   46.486875] scsi 0:0:2:0: Direct-Access     SEAGATE  SX150176LC       BA08 PQ: 0 ANSI: 2
>> [   46.583893] scsi(0:0:2:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
>> [   46.683736] scsi 0:0:3:0: Direct-Access     SEAGATE  SX150176LC       BA08 PQ: 0 ANSI: 2
>> [   46.780696] scsi(0:0:3:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
>> [   46.880821] scsi 0:0:4:0: Direct-Access     SEAGATE  SX150176LC       BA11 PQ: 0 ANSI: 2
>> [   46.977716] scsi(0:0:4:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
>> [   47.077797] scsi 0:0:5:0: Direct-Access     SEAGATE  SX150176LC       BA11 PQ: 0 ANSI: 2
>> [   47.174768] scsi(0:0:5:0):[   47.181219] scsi(1:0): Resetting SCSI BUS
>> [   47.251839]  Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
>> [   47.639912] scsi 0:0:6:0: CD-ROM            TOSHIBA  CD-ROM XM-5701TA 0167 PQ: 0 ANSI: 2
>> [   47.736890] scsi(0:0:6:0): Sync: period 10, offset 12
>> [   51.553976] scsi host1: QLogic QLA1040 PCI to SCSI Host Adapter
>> [   51.553976]        Firmware version:  7.65.06, Driver version 3.27.1
>> [   51.998859] ioc3: part: [], serial: [] => class IP27 BaseIO
>> [   52.068783] ioc3-eth: Ethernet address is 08:00:69:05:64:74.
>> [   52.136831] IOC3 0002:00:02.0 eth0: link up, 100Mbps, full-duplex, lpa 0x8DE1
>> [   52.220897] eth0: Using PHY 31, vendor 0x20005c0, model 0, rev 0.
>> [   52.294232] eth0: IOC3 SSRAM has 128 kbyte.
>> [   52.345852] rtc-m48t35 rtc-m48t35: rtc core: registered m48t35 as rtc0
>> [   52.588471] console [ttyS0] enabled
>> [   52.489601] 0002:00:02.0: ttyS0 at IOC3 0xf620178 (irq = 0, base_baud = 458333) is a 16550A
>> [   52.671141] bootconsole [early0] disabled
>> [   54.146680] 0002:00:02.0: ttyS1 at IOC3 0xf620170 (irq = 0, base_baud = 458333) is a 16550A
>> [   54.248408] IOC3 Master Driver loaded for 0002:00:02.0
>> [   54.313895] ioc3: NIC search failed.
>> [   54.356992] ioc3: part: [], serial: [] => class IP27 BaseIO
>> [   54.424356] ioc3: request_irq fails for IRQ 0xffffffff
>> [   54.490948] ioc3-eth: Ethernet address is 00:00:00:00:00:00.
>> [   54.561038] IOC3 0002:00:06.0 eth1: link down
>> [   54.613597] eth1: Using PHY 31, vendor 0x0, model 0, rev 0.
>> [   54.680825] eth1: IOC3 SSRAM has 64 kbyte.
>> [   54.765266] 0002:00:06.0: ttyS2 at IOC3 0xfa20178 (irq = 0, base_baud = 458333) is a 16550A
>> [   54.888655] 0002:00:06.0: ttyS3 at IOC3 0xfa20170 (irq = 0, base_baud = 458333) is a 16550A
>> [   54.990198] IOC3 Master Driver loaded for 0002:00:06.0
>> [   55.052840] NET: Registered protocol family 26
>> [   55.509832] sd 0:0:1:0: [sda] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
>> [   55.515697] sr 0:0:6:0: [sr0] scsi-1 drive
>> [   55.515709] cdrom: Uniform CD-ROM driver Revision: 3.20
>> [   55.520834] sd 0:0:4:0: [sdd] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
>> [   55.523611] sd 0:0:4:0: [sdd] Write Protect is off
>> [   55.526511] sd 0:0:4:0: [sdd] Write cache: disabled, read cache: enabled, supports DPO and FUA
>> [   55.550956]  sdd: sdd1 sdd2 sdd3 sdd4 sdd5 sdd6 sdd9 sdd11
>> [   55.562326] sd 0:0:4:0: [sdd] Attached SCSI disk
>> [   55.564803] sd 0:0:5:0: [sde] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
>> [   55.567564] sd 0:0:5:0: [sde] Write Protect is off
>> [   55.570463] sd 0:0:5:0: [sde] Write cache: disabled, read cache: enabled, supports DPO and FUA
>> [   55.600908] sd 0:0:5:0: [sde] Attached SCSI disk
>> [   55.604715] sd 0:0:2:0: [sdb] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
>> [   55.605297] sd 0:0:3:0: [sdc] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
>> [   55.607054] sd 0:0:2:0: [sdb] Write Protect is off
>> [   55.607592] sd 0:0:3:0: [sdc] Write Protect is off
>> [   55.609959] sd 0:0:2:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
>> [   55.610460] sd 0:0:3:0: [sdc] Write cache: disabled, read cache: enabled, supports DPO and FUA
>> [   55.631199]  sdb: sdb1 sdb2 sdb3 sdb4 sdb5 sdb6 sdb9 sdb11
>> [   55.634048]  sdc: sdc1 sdc2 sdc3 sdc4 sdc5 sdc6 sdc9 sdc11
>> [   55.646437] sd 0:0:3:0: [sdc] Attached SCSI disk
>> [   55.647162] sd 0:0:2:0: [sdb] Attached SCSI disk
>> [   57.150140] sd 0:0:1:0: [sda] Write Protect is off
>> [   57.173988] NET: Registered protocol family 10
>> [   57.264566] sd 0:0:1:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
>> [   57.386960]  sda: sda1 sda2 sda3 sda4 sda5 sda6 sda9 sda11
>> [   57.464000] sd 0:0:1:0: [sda] Attached SCSI disk
>> [   57.544064] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
>> [   57.617430] NET: Registered protocol family 17
>> [   57.671360] 8021q: 802.1Q VLAN Support v1.8
>> [   57.722075] sctp: Hash tables configured (bind 1024/1024)
>> [   58.093962] Key type dns_resolver registered
>> [   58.148015] rtc-m48t35 rtc-m48t35: setting system clock to 2016-10-17 23:49:58 UTC (1476748198)
>> [   59.626451] Freeing unused kernel memory: 5616K (a800000000854000 - a800000000dd0000)
>> [   59.720950] This architecture does not have kernel memory protection.
>>
>>
>> Next is /proc/iomem and /proc/ioports:
>> # cat /proc/iomem
>> 0f600000-0f6fffff : ioc3
>>   0f680000-0f687fff : rtc-m48t35
>> 0fa00000-0fafffff : ioc3
>> 920000000b200000-920000000b9fffff : Bridge MEM
>> 920000000c200000-920000000c9fffff : Bridge MEM
>> 920000000f200000-920000000f9fffff : Bridge MEM
>>
>> / # cat /proc/ioports
>> 920000000ba00000-920000000bbfffff : Bridge IO
>> 920000000ca00000-920000000cbfffff : Bridge IO
>> 920000000fa00000-920000000fbfffff : Bridge IO
>>
>>
>> And lspci -vvvx:
>>
>> / # /usr/sbin/lspci -vvvx
>> 0001:00:00.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. Linc I/O controller
>>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         NUMA node: 0
>>         Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=134217729]
>>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Expansion ROM at <unassigned> [disabled] [size=2]
>> lspci: Unable to load libkmod resources: error -12
>> 00: a9 10 02 00 00 00 00 02 00 00 00 ff 00 f8 80 00
>> 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>
>> 0001:00:01.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. Linc I/O controller
>>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         NUMA node: 0
>>         Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=134217729]
>>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Expansion ROM at <unassigned> [disabled] [size=2]
>> 00: a9 10 02 00 00 00 00 02 00 00 00 ff 00 f8 80 00
>> 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>
>> 0002:00:00.0 SCSI storage controller: QLogic Corp. ISP1020 Fast-wide SCSI (rev 05)
>>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 64, Cache Line Size: 256 bytes
>>         Interrupt: pin A routed to IRQ 1
>>         NUMA node: 0
>>         Region 0: I/O ports at f200000 [size=257]
>>         Region 1: Memory at 0f200000 (32-bit, non-prefetchable) [size=4097]
>>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Expansion ROM at <unassigned> [disabled] [size=65537]
>>         Kernel driver in use: qla1280
>> 00: 77 10 20 10 07 00 00 02 05 00 00 01 40 40 00 00
>> 10: 01 00 20 0f 00 00 20 0f 00 00 00 00 00 00 00 00
>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
>>
>> 0002:00:01.0 SCSI storage controller: QLogic Corp. ISP1020 Fast-wide SCSI (rev 05)
>>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 64, Cache Line Size: 256 bytes
>>         Interrupt: pin A routed to IRQ 2
>>         NUMA node: 0
>>         Region 0: I/O ports at f400000 [size=257]
>>         Region 1: Memory at 0f400000 (32-bit, non-prefetchable) [size=4097]
>>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Expansion ROM at <unassigned> [disabled] [size=65537]
>>         Kernel driver in use: qla1280
>> 00: 77 10 20 10 07 00 00 02 05 00 00 01 40 40 00 00
>> 10: 01 00 40 0f 00 00 40 0f 00 00 00 00 00 00 00 00
>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
>>
>> 0002:00:02.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. IOC3 I/O controller (rev 01)
>>         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 64
>>         Interrupt: pin A routed to IRQ 3
>>         NUMA node: 0
>>         Region 0: Memory at 0f600000 (32-bit, non-prefetchable) [size=1048577]
>>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Expansion ROM at <unassigned> [disabled] [size=2]
>>         Kernel driver in use: IOC3
>> 00: a9 10 03 00 06 00 80 02 01 00 00 ff 00 40 00 00
>> 10: 00 00 60 0f 00 00 00 00 00 00 00 00 00 00 00 00
>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
>>
>> 0002:00:06.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. IOC3 I/O controller (rev 01)
>>         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 64
>>         Interrupt: pin A routed to IRQ 5
>>         NUMA node: 0
>>         Region 0: Memory at 0fa00000 (32-bit, non-prefetchable) [size=1048577]
>>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
>>         Expansion ROM at <unassigned> [disabled] [size=2]
>>         Kernel driver in use: IOC3
>> 00: a9 10 03 00 06 00 80 02 01 00 00 ff 00 40 00 00
>> 10: 00 00 a0 0f 00 00 00 00 00 00 00 00 00 00 00 00
>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
>>
>> 0002:00:07.0 Non-VGA unclassified device: Silicon Graphics Intl. Corp. RAD Audio (rev 70)
>>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         NUMA node: 0
>>         Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=8193]
>>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
>>         Expansion ROM at <unassigned> [disabled] [size=2]
>> 00: a9 10 05 00 00 00 80 04 70 00 00 00 00 00 00 00
>> 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>
>>
>> IP27 hardware does some interesting things with address spaces, depending on
>> which "mode" you run the machine in.  "M mode" is the only one that's really
>> usable these days (the other, "N mode", doesn't boot).  Under M mode,
>> there's five address spaces, of which, "I/O Space" is selectable by turning on
>> specific bits in the 64-bit memory address.
>>
>> So, given 0x920000000b200000, bits 63:62 select physical addressing (10) and
>> bits 58:57 select I/O space (01), resulting in all PCI addresses (or any I/O
>> address) appearing in the 0x92xxxxxxxxxxxxxx range.  The rest of the bits go
>> on to select which NUMA node the I/O is targeted at and which Crosstalk widget
>> to access, and so on.  I still haven't fully understood it all myself just
>> yet.
>>
>> You can google for document 007-3410-001, "Origin and Onyx2 Programmer's
>> Reference Manual" for more details (it's no longer available via SGI's
>> TechPubs, but it is archived on plenty of other sites).
>>
>> This particular machine that I have has three "BRIDGE" chips in it, all
>> attached to the Crosstalk bus.  Each BRIDGE can address up to 8 PCI devices,
>> but some crosstalk boards slap a BRIDGE chip on for just a single device.  In
>> this instance, the BRIDGE w/ address 0x920000000fxxxxxx is the IO6 at xtalk
>> widget 0xf (this has the basic I/O devices for this machine).  The BRIDGE at
>> 0x920000000cxxxxxx (widget 0xc) is a PCI "shoebox" that can hold up to three
>> PCI-X cards, and the BRIDGE at 0x920000000bxxxxxx (widget 0xb) is an
>> SGI-specific HIPPI board (ID'ed as a LINC I/O controller by pci.ids).

Re: [PATCH] MIPS/PCI: Claim bus resources on PCI_PROBE_ONLY set-ups

[Lorenzo Pieralisi]

On Mon, Oct 31, 2016 at 03:36:52PM -0400, Joshua Kinard wrote:

[...]

> > By not claiming resources, as you can see in the log below
> > /proc/iomem and /proc/ioports do not report the PCI devices resource
> > usages and the respective resources are not reserved in the kernel,
> > which is wrong and that's exactly why Bjorn patched the kernel.
> > 
> > Please let me know.
> > 
> > Thanks,
> > Lorenzo
> 
> Yup, I can see that on my SGI Octane machine (IP30 platform), which is
> architecturally-similar to IP27, but uses a more "normal" addressing
> layout.  Previously, it was also setting the PCI_PROBE_ONLY flag and
> thus /proc/iomem looked very similar to IP27's output.  Now, however,
> Octane's /proc/iomem looks more like this:
> 
> # cat /proc/iomem
> 00000000-00003fff : reserved
> 1d200000-1d9fffff : Bridge MEM
>   d080000000-d080003fff : 0001:00:03.0
>   1d204000-1d204fff : 0001:00:02.0
>   1d205000-1d205fff : 0001:00:02.1
>   1d206000-1d206fff : 0001:00:02.2
>   1d207000-1d2070ff : 0001:00:02.3
>   1d207100-1d20717f : 0001:00:01.0
> 1f200000-1f9fffff : Bridge MEM
>   f080100000-f0801fffff : 0000:00:02.0
>   f080010000-f08001ffff : 0000:00:00.0
>   f080030000-f08003ffff : 0000:00:01.0
>   f080000000-f080000fff : 0000:00:00.0
>   f080020000-f080020fff : 0000:00:01.0

Is this the resource hierarchy you get with the patch reverted ?

This resource hierarchy is weird (and IIUC it should be the end result
of the kernel assigning the whole resource space). I need kernel log
with mainline and mainline with this patch reverted please on IP30.
Still, without digging into the platform code that sets-up the PCI
resources I doubt I can get to the bottom of this so if you can give
me a pointer to the source code files that set-up the PCI bridges
I am keen to help.

Thanks a lot,
Lorenzo

> 20004000-20a27fff : reserved
>   20004000-207180b7 : Kernel code
>   207180b8-2090ffff : Kernel data
> 20a28000-20efffff : System RAM
> 20f00000-20ffffff : System RAM
> 21000000-9fffffff : System RAM
> f080100000-f0801fffff : ioc3
> 
> And /proc/ioports:
> # cat /proc/ioports
> 1da00000-1dbfffff : Bridge IO
>   1da00000-1da000ff : 0001:00:01.0
>   1da00400-1da0047f : 0001:00:01.0
> 1fa00000-1fbfffff : Bridge IO
>   f100000000-f1000000ff : 0000:00:00.0
>   f100000100-f1000001ff : 0000:00:01.0
> 
> PCI support on IP27 has been a bit weird for a long time, mostly due to it
> using a lot of "IRIX'isms" in the code to get it working initially.  Some of
> these are actually short-circuits for things that would instead use normal
> kernel mechanisms, and no one's really had the time or interest to try and
> clean it up.  I've done a significant amount of refactoring of the IP27's
> machine-specific code so far the last few months, but haven't spent much
> time understanding PCI and XIO some more to make further progress.  Lack of
> time has made it difficult to break those code changes down into bite-sized
> patches to send back upstream (I should've planned things a bit better...).
> 
> --J
> 
> 
> >> First, is the boot output from my serial console:
> >> [    0.000000] Linux version 4.8.2-mipsgit-20161016 (root@helcaraxe) (gcc version 5.4.0 (Gentoo Hardened 5.4.0 p1.0, pie-0.6.5) ) #4 SMP Mon Oct 17 19:48:30 EDT 2016
> >> [    0.000000] ARCH: SGI-IP27
> >> [    0.000000] PROMLIB: ARC firmware Version 64 Revision 0
> >> [    0.000000] SMP: Discovered 4 cpus on 2 nodes
> >> [    0.000000] ************** Topology ********************
> >> [    0.000000]     00 01
> >> [    0.000000] 00  255 255
> >> [    0.000000] 01  255 255
> >> [    0.000000] bootconsole [early0] enabled
> >> [    0.000000] CPU0 revision is: 00000f14 (R14000)
> >> [    0.000000] FPU revision is: 00000900
> >> [    0.000000] Checking for the multiply/shift bug... no.
> >> [    0.000000] Checking for the daddiu bug... no.
> >> [    0.000000] IP27: Running on node 0.
> >> [    0.000000] IP27: Node 0 has a primary CPU, CPU is running.
> >> [    0.000000] IP27: Node 0 has a secondary CPU, CPU is running.
> >> [    0.000000] IP27: Machine is in M mode.
> >> [    0.000000] IP27: CPU 1A (CPU0): 500 MHz CPU detected.
> >> [    0.000000] Determined physical RAM map:
> >> [    0.000000]  memory: 0000000000838000 @ 000000000001c000 (usable)
> >> [    0.000000]  memory: 000000000057c000 @ 0000000000854000 (usable after init)
> >> [    0.000000] Initrd not found or empty - disabling initrd
> >> [    0.000000] cma: Reserved 32 MiB at 0x00000001fe000000
> >> [    0.000000] REPLICATION: ON nasid 0, ktext from nasid 0, kdata from nasid 0
> >> [    0.000000] REPLICATION: ON nasid 1, ktext from nasid 0, kdata from nasid 0
> >> [    0.000000] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
> >> [    0.000000] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
> >> [    0.000000] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
> >> [    0.000000] Zone ranges:
> >> [    0.000000]   Normal   [mem 0x0000000000000000-0x00000001ffffffff]
> >> [    0.000000] Movable zone start for each node
> >> [    0.000000] Early memory node ranges
> >> [    0.000000]   node   0: [mem 0x0000000000000000-0x00000000ffffffff]
> >> [    0.000000]   node   1: [mem 0x0000000100000000-0x00000001ffffffff]
> >> [    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000ffffffff]
> >> [    0.000000] Initmem setup node 1 [mem 0x0000000100000000-0x00000001ffffffff]
> >> [    0.000000] percpu: Embedded 4 pages/cpu @a800000002010000 s36096 r0 d29440 u65536
> >> [    0.000000] Built 2 zonelists in Node order, mobility grouping on.  Total pages: 522240
> >> [    0.000000] Policy zone: Normal
> >> [    0.000000] Kernel command line: root=dksc(0,1,0) console=ttyS0,9600
> >> [    0.000000] PID hash table entries: 4096 (order: 1, 32768 bytes)
> >> [    0.000000] Memory: 8307776K/8388608K available (6498K kernel code, 454K rwdata, 1436K rodata, 5616K init, 543K bss, 48064K reserved, 32768K cma-reserved)
> >> [    0.000000] Hierarchical RCU implementation.
> >> [    0.000000]  Build-time adjustment of leaf fanout to 64.
> >> [    0.000000]  RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4.
> >> [    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=4
> >> [    0.000000] NR_IRQS:256
> >> [    0.000000] clocksource: HUB: mask: 0xfffffffffffff max_cycles: 0x127350b88, max_idle_ns: 1763180808480 ns
> >> [    0.000019] sched_clock: 52 bits at 1250kHz, resolution 800ns, wraps every 4398046510800ns
> >> [    0.102422] Console: colour dummy device 80x25
> >> [    0.154975] Calibrating delay loop... 749.56 BogoMIPS (lpj=3747840)
> >> [    0.317670] pid_max: default: 32768 minimum: 301
> >> [    0.377938] Dentry cache hash table entries: 1048576 (order: 9, 8388608 bytes)
> >> [    0.592303] Inode-cache hash table entries: 524288 (order: 8, 4194304 bytes)
> >> [    0.736316] Mount-cache hash table entries: 16384 (order: 3, 131072 bytes)
> >> [    0.817310] Mountpoint-cache hash table entries: 16384 (order: 3, 131072 bytes)
> >> [    0.916094] Checking for the daddi bug... no.
> >> [    0.976495] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
> >> [    0.976520] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
> >> [    0.976532] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
> >> [    0.979792] IP27: CPU 1B (CPU1): 500 MHz CPU detected.
> >> [    0.979824] CPU1 revision is: 00000f14 (R14000)
> >> [    0.979831] FPU revision is: 00000900
> >> [    1.014463] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
> >> [    1.014493] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
> >> [    1.014506] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
> >> [    1.017888] IP27: CPU 2A (CPU2): 500 MHz CPU detected.
> >> [    1.017931] CPU2 revision is: 00000f14 (R14000)
> >> [    1.017938] FPU revision is: 00000900
> >> [    1.055420] Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
> >> [    1.055452] Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
> >> [    1.055467] Unified secondary cache 8192kB 2-way, linesize 128 bytes.
> >> [    1.058810] IP27: CPU 2B (CPU3): 500 MHz CPU detected.
> >> [    1.058851] CPU3 revision is: 00000f14 (R14000)
> >> [    1.058858] FPU revision is: 00000900
> >> [    1.092672] Brought up 4 CPUs
> >> [    2.484801] devtmpfs: initialized
> >> [    2.603700] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> >> [    2.892716] random: fast init done
> >> [    2.990752] xor: measuring software checksum speed
> >> [    3.146730]    8regs     :   486.400 MB/sec
> >> [    3.295350]    8regs_prefetch:   486.400 MB/sec
> >> [    3.448144]    32regs    :   486.400 MB/sec
> >> [    3.596768]    32regs_prefetch:   486.400 MB/sec
> >> [    3.650415] xor: using function: 32regs_prefetch (486.400 MB/sec)
> >> [    3.834852] NET: Registered protocol family 16
> >> [    3.946535] cpuidle: using governor ladder
> >> [    4.043955] cpuidle: using governor menu
> >> [    4.089355] xtalk:n0/0 xbow widget (rev 1.3)
> >> [    4.140796] xtalk:n0/8 kona widget (rev unknown) registered as a platform device.
> >> [    4.230796] xtalk:n0/b bridge widget (rev D) registered as a platform device.
> >> [    4.316712] xtalk:n0/c bridge widget (rev D) registered as a platform device.
> >> [    4.402590] xtalk:n0/d impact widget (rev B) registered as a platform device.
> >> [    4.488444] xtalk:n0/f bridge widget (rev C) registered as a platform device.
> >> [    4.574178] xtalk:n1/0 xbow widget (rev 1.3)
> >> [    5.247396] raid6: int64x1  gen()    84 MB/s
> >> [    5.467370] raid6: int64x1  xor()    80 MB/s
> >> [    5.687324] raid6: int64x2  gen()   151 MB/s
> >> [    5.907243] raid6: int64x2  xor()   116 MB/s
> >> [    6.127125] raid6: int64x4  gen()   235 MB/s
> >> [    6.347155] raid6: int64x4  xor()   142 MB/s
> >> [    6.567117] raid6: int64x8  gen()   251 MB/s
> >> [    6.787018] raid6: int64x8  xor()   175 MB/s
> >> [    6.836420] raid6: using algorithm int64x8 gen() 251 MB/s
> >> [    6.901352] raid6: .... xor() 175 MB/s, rmw enabled
> >> [    6.960002] raid6: using intx1 recovery algorithm
> >> [    7.017544] SCSI subsystem initialized
> >> [    7.063853] clocksource: Switched to clocksource HUB
> >> [    7.363862] FS-Cache: Loaded
> >> [   31.643872] NET: Registered protocol family 2
> >> [   31.696149] TCP established hash table entries: 65536 (order: 5, 524288 bytes)
> >> [   31.787294] TCP bind hash table entries: 65536 (order: 6, 1048576 bytes)
> >> [   31.874832] TCP: Hash tables configured (established 65536 bind 65536)
> >> [   31.952084] UDP hash table entries: 4096 (order: 3, 131072 bytes)
> >> [   32.026304] UDP-Lite hash table entries: 4096 (order: 3, 131072 bytes)
> >> [   32.403836] NET: Registered protocol family 1
> >> [   32.583876] RPC: Registered named UNIX socket transport module.
> >> [   32.653282] RPC: Registered udp transport module.
> >> [   32.709785] RPC: Registered tcp transport module.
> >> [   32.766346] RPC: Registered tcp NFSv4.1 backchannel transport module.
> >> [   34.045622] futex hash table entries: 1024 (order: 3, 131072 bytes)
> >> [   34.275238] workingset: timestamp_bits=40 max_order=19 bucket_order=0
> >> [   34.350961] zbud: loaded
> >> [   37.383916] NFS: Registering the id_resolver key type
> >> [   37.442917] Key type id_resolver registered
> >> [   37.493101] Key type id_legacy registered
> >> [   37.541388] efs: 1.0a - http://aeschi.ch.eu.org/efs/
> >> [   37.873853] SGI XFS with ACLs, security attributes, realtime, no debug enabled
> >> [   39.453936] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
> >> [   39.541066] io scheduler noop registered
> >> [   39.588223] io scheduler deadline registered
> >> [   39.639639] io scheduler cfq registered (default)
> >> [   39.973936] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> >> [   40.148640] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> >> [   40.245916] loop: module loaded
> >> [   40.283288] mousedev: PS/2 mouse device common for all mice
> >> [   40.349460] md: linear personality registered for level -1
> >> [   40.414802] md: raid0 personality registered for level 0
> >> [   40.478677] md: raid1 personality registered for level 1
> >> [   40.542549] md: raid10 personality registered for level 10
> >> [   40.609068] md: raid6 personality registered for level 6
> >> [   40.672411] md: raid5 personality registered for level 5
> >> [   40.736315] md: raid4 personality registered for level 4
> >> [   40.800808] PCI host bridge to bus 0000:00
> >> [   40.849424] pci_bus 0000:00: root bus resource [mem 0x920000000b200000-0x920000000b9fffff]
> >> [   40.948920] pci_bus 0000:00: root bus resource [io  0x920000000ba00000-0x920000000bbfffff]
> >> [   41.048408] pci_bus 0000:00: root bus resource [bus 00-ff]
> >> [   41.114880] PCI host bridge to bus 0001:00
> >> [   41.163629] pci_bus 0001:00: root bus resource [mem 0x920000000c200000-0x920000000c9fffff]
> >> [   41.263152] pci_bus 0001:00: root bus resource [io  0x920000000ca00000-0x920000000cbfffff]
> >> [   41.362645] pci_bus 0001:00: root bus resource [bus 01-ff]
> >> [   41.430847] PCI host bridge to bus 0002:00
> >> [   41.478263] pci_bus 0002:00: root bus resource [mem 0x920000000f200000-0x920000000f9fffff]
> >> [   41.577771] pci_bus 0002:00: root bus resource [io  0x920000000fa00000-0x920000000fbfffff]
> >> [   41.677250] pci_bus 0002:00: root bus resource [bus 02-ff]
> >> [   41.746742] qla1280: QLA1040 found on PCI bus 0, dev 0
> >> [   41.806687] PCI: Enabling device 0002:00:00.0 (0006 -> 0007)
> >> [   41.875796] scsi(0): Enabling vchannel on BRIDGE for SGI/MIPS
> >> [   42.324520] random: crng init done
> >> [   42.839160] scsi(0:0): Resetting SCSI BUS
> >> [   45.964232] scsi host0: QLogic QLA1040 PCI to SCSI Host Adapter
> >> [   45.964232]        Firmware version:  7.65.06, Driver version 3.27.1
> >> [   46.118149] qla1280: QLA1040 found on PCI bus 0, dev 1
> >> [   46.178192] PCI: Enabling device 0002:00:01.0 (0006 -> 0007)
> >> [   46.289985] scsi 0:0:1:0: Direct-Access     SEAGATE  SX150176LC       BA12 PQ: 0 ANSI: 2
> >> [   46.387174] scsi(0:0:1:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
> >> [   46.486875] scsi 0:0:2:0: Direct-Access     SEAGATE  SX150176LC       BA08 PQ: 0 ANSI: 2
> >> [   46.583893] scsi(0:0:2:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
> >> [   46.683736] scsi 0:0:3:0: Direct-Access     SEAGATE  SX150176LC       BA08 PQ: 0 ANSI: 2
> >> [   46.780696] scsi(0:0:3:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
> >> [   46.880821] scsi 0:0:4:0: Direct-Access     SEAGATE  SX150176LC       BA11 PQ: 0 ANSI: 2
> >> [   46.977716] scsi(0:0:4:0): Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
> >> [   47.077797] scsi 0:0:5:0: Direct-Access     SEAGATE  SX150176LC       BA11 PQ: 0 ANSI: 2
> >> [   47.174768] scsi(0:0:5:0):[   47.181219] scsi(1:0): Resetting SCSI BUS
> >> [   47.251839]  Sync: period 10, offset 12, Wide, Tagged queuing: depth 31
> >> [   47.639912] scsi 0:0:6:0: CD-ROM            TOSHIBA  CD-ROM XM-5701TA 0167 PQ: 0 ANSI: 2
> >> [   47.736890] scsi(0:0:6:0): Sync: period 10, offset 12
> >> [   51.553976] scsi host1: QLogic QLA1040 PCI to SCSI Host Adapter
> >> [   51.553976]        Firmware version:  7.65.06, Driver version 3.27.1
> >> [   51.998859] ioc3: part: [], serial: [] => class IP27 BaseIO
> >> [   52.068783] ioc3-eth: Ethernet address is 08:00:69:05:64:74.
> >> [   52.136831] IOC3 0002:00:02.0 eth0: link up, 100Mbps, full-duplex, lpa 0x8DE1
> >> [   52.220897] eth0: Using PHY 31, vendor 0x20005c0, model 0, rev 0.
> >> [   52.294232] eth0: IOC3 SSRAM has 128 kbyte.
> >> [   52.345852] rtc-m48t35 rtc-m48t35: rtc core: registered m48t35 as rtc0
> >> [   52.588471] console [ttyS0] enabled
> >> [   52.489601] 0002:00:02.0: ttyS0 at IOC3 0xf620178 (irq = 0, base_baud = 458333) is a 16550A
> >> [   52.671141] bootconsole [early0] disabled
> >> [   54.146680] 0002:00:02.0: ttyS1 at IOC3 0xf620170 (irq = 0, base_baud = 458333) is a 16550A
> >> [   54.248408] IOC3 Master Driver loaded for 0002:00:02.0
> >> [   54.313895] ioc3: NIC search failed.
> >> [   54.356992] ioc3: part: [], serial: [] => class IP27 BaseIO
> >> [   54.424356] ioc3: request_irq fails for IRQ 0xffffffff
> >> [   54.490948] ioc3-eth: Ethernet address is 00:00:00:00:00:00.
> >> [   54.561038] IOC3 0002:00:06.0 eth1: link down
> >> [   54.613597] eth1: Using PHY 31, vendor 0x0, model 0, rev 0.
> >> [   54.680825] eth1: IOC3 SSRAM has 64 kbyte.
> >> [   54.765266] 0002:00:06.0: ttyS2 at IOC3 0xfa20178 (irq = 0, base_baud = 458333) is a 16550A
> >> [   54.888655] 0002:00:06.0: ttyS3 at IOC3 0xfa20170 (irq = 0, base_baud = 458333) is a 16550A
> >> [   54.990198] IOC3 Master Driver loaded for 0002:00:06.0
> >> [   55.052840] NET: Registered protocol family 26
> >> [   55.509832] sd 0:0:1:0: [sda] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
> >> [   55.515697] sr 0:0:6:0: [sr0] scsi-1 drive
> >> [   55.515709] cdrom: Uniform CD-ROM driver Revision: 3.20
> >> [   55.520834] sd 0:0:4:0: [sdd] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
> >> [   55.523611] sd 0:0:4:0: [sdd] Write Protect is off
> >> [   55.526511] sd 0:0:4:0: [sdd] Write cache: disabled, read cache: enabled, supports DPO and FUA
> >> [   55.550956]  sdd: sdd1 sdd2 sdd3 sdd4 sdd5 sdd6 sdd9 sdd11
> >> [   55.562326] sd 0:0:4:0: [sdd] Attached SCSI disk
> >> [   55.564803] sd 0:0:5:0: [sde] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
> >> [   55.567564] sd 0:0:5:0: [sde] Write Protect is off
> >> [   55.570463] sd 0:0:5:0: [sde] Write cache: disabled, read cache: enabled, supports DPO and FUA
> >> [   55.600908] sd 0:0:5:0: [sde] Attached SCSI disk
> >> [   55.604715] sd 0:0:2:0: [sdb] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
> >> [   55.605297] sd 0:0:3:0: [sdc] 97693755 512-byte logical blocks: (50.0 GB/46.6 GiB)
> >> [   55.607054] sd 0:0:2:0: [sdb] Write Protect is off
> >> [   55.607592] sd 0:0:3:0: [sdc] Write Protect is off
> >> [   55.609959] sd 0:0:2:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
> >> [   55.610460] sd 0:0:3:0: [sdc] Write cache: disabled, read cache: enabled, supports DPO and FUA
> >> [   55.631199]  sdb: sdb1 sdb2 sdb3 sdb4 sdb5 sdb6 sdb9 sdb11
> >> [   55.634048]  sdc: sdc1 sdc2 sdc3 sdc4 sdc5 sdc6 sdc9 sdc11
> >> [   55.646437] sd 0:0:3:0: [sdc] Attached SCSI disk
> >> [   55.647162] sd 0:0:2:0: [sdb] Attached SCSI disk
> >> [   57.150140] sd 0:0:1:0: [sda] Write Protect is off
> >> [   57.173988] NET: Registered protocol family 10
> >> [   57.264566] sd 0:0:1:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
> >> [   57.386960]  sda: sda1 sda2 sda3 sda4 sda5 sda6 sda9 sda11
> >> [   57.464000] sd 0:0:1:0: [sda] Attached SCSI disk
> >> [   57.544064] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
> >> [   57.617430] NET: Registered protocol family 17
> >> [   57.671360] 8021q: 802.1Q VLAN Support v1.8
> >> [   57.722075] sctp: Hash tables configured (bind 1024/1024)
> >> [   58.093962] Key type dns_resolver registered
> >> [   58.148015] rtc-m48t35 rtc-m48t35: setting system clock to 2016-10-17 23:49:58 UTC (1476748198)
> >> [   59.626451] Freeing unused kernel memory: 5616K (a800000000854000 - a800000000dd0000)
> >> [   59.720950] This architecture does not have kernel memory protection.
> >>
> >>
> >> Next is /proc/iomem and /proc/ioports:
> >> # cat /proc/iomem
> >> 0f600000-0f6fffff : ioc3
> >>   0f680000-0f687fff : rtc-m48t35
> >> 0fa00000-0fafffff : ioc3
> >> 920000000b200000-920000000b9fffff : Bridge MEM
> >> 920000000c200000-920000000c9fffff : Bridge MEM
> >> 920000000f200000-920000000f9fffff : Bridge MEM
> >>
> >> / # cat /proc/ioports
> >> 920000000ba00000-920000000bbfffff : Bridge IO
> >> 920000000ca00000-920000000cbfffff : Bridge IO
> >> 920000000fa00000-920000000fbfffff : Bridge IO
> >>
> >>
> >> And lspci -vvvx:
> >>
> >> / # /usr/sbin/lspci -vvvx
> >> 0001:00:00.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. Linc I/O controller
> >>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
> >>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >>         NUMA node: 0
> >>         Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=134217729]
> >>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Expansion ROM at <unassigned> [disabled] [size=2]
> >> lspci: Unable to load libkmod resources: error -12
> >> 00: a9 10 02 00 00 00 00 02 00 00 00 ff 00 f8 80 00
> >> 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >>
> >> 0001:00:01.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. Linc I/O controller
> >>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
> >>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >>         NUMA node: 0
> >>         Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=134217729]
> >>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Expansion ROM at <unassigned> [disabled] [size=2]
> >> 00: a9 10 02 00 00 00 00 02 00 00 00 ff 00 f8 80 00
> >> 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >>
> >> 0002:00:00.0 SCSI storage controller: QLogic Corp. ISP1020 Fast-wide SCSI (rev 05)
> >>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
> >>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >>         Latency: 64, Cache Line Size: 256 bytes
> >>         Interrupt: pin A routed to IRQ 1
> >>         NUMA node: 0
> >>         Region 0: I/O ports at f200000 [size=257]
> >>         Region 1: Memory at 0f200000 (32-bit, non-prefetchable) [size=4097]
> >>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Expansion ROM at <unassigned> [disabled] [size=65537]
> >>         Kernel driver in use: qla1280
> >> 00: 77 10 20 10 07 00 00 02 05 00 00 01 40 40 00 00
> >> 10: 01 00 20 0f 00 00 20 0f 00 00 00 00 00 00 00 00
> >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
> >>
> >> 0002:00:01.0 SCSI storage controller: QLogic Corp. ISP1020 Fast-wide SCSI (rev 05)
> >>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
> >>         Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >>         Latency: 64, Cache Line Size: 256 bytes
> >>         Interrupt: pin A routed to IRQ 2
> >>         NUMA node: 0
> >>         Region 0: I/O ports at f400000 [size=257]
> >>         Region 1: Memory at 0f400000 (32-bit, non-prefetchable) [size=4097]
> >>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Expansion ROM at <unassigned> [disabled] [size=65537]
> >>         Kernel driver in use: qla1280
> >> 00: 77 10 20 10 07 00 00 02 05 00 00 01 40 40 00 00
> >> 10: 01 00 40 0f 00 00 40 0f 00 00 00 00 00 00 00 00
> >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
> >>
> >> 0002:00:02.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. IOC3 I/O controller (rev 01)
> >>         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
> >>         Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >>         Latency: 64
> >>         Interrupt: pin A routed to IRQ 3
> >>         NUMA node: 0
> >>         Region 0: Memory at 0f600000 (32-bit, non-prefetchable) [size=1048577]
> >>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Expansion ROM at <unassigned> [disabled] [size=2]
> >>         Kernel driver in use: IOC3
> >> 00: a9 10 03 00 06 00 80 02 01 00 00 ff 00 40 00 00
> >> 10: 00 00 60 0f 00 00 00 00 00 00 00 00 00 00 00 00
> >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
> >>
> >> 0002:00:06.0 Unassigned class [ff00]: Silicon Graphics Intl. Corp. IOC3 I/O controller (rev 01)
> >>         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
> >>         Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >>         Latency: 64
> >>         Interrupt: pin A routed to IRQ 5
> >>         NUMA node: 0
> >>         Region 0: Memory at 0fa00000 (32-bit, non-prefetchable) [size=1048577]
> >>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [size=2]
> >>         Expansion ROM at <unassigned> [disabled] [size=2]
> >>         Kernel driver in use: IOC3
> >> 00: a9 10 03 00 06 00 80 02 01 00 00 ff 00 40 00 00
> >> 10: 00 00 a0 0f 00 00 00 00 00 00 00 00 00 00 00 00
> >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00
> >>
> >> 0002:00:07.0 Non-VGA unclassified device: Silicon Graphics Intl. Corp. RAD Audio (rev 70)
> >>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
> >>         Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=slow >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> >>         NUMA node: 0
> >>         Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=8193]
> >>         Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Region 3: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Region 4: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Region 5: Memory at <unassigned> (32-bit, non-prefetchable) [disabled] [size=2]
> >>         Expansion ROM at <unassigned> [disabled] [size=2]
> >> 00: a9 10 05 00 00 00 80 04 70 00 00 00 00 00 00 00
> >> 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >> 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >> 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >>
> >>
> >> IP27 hardware does some interesting things with address spaces, depending on
> >> which "mode" you run the machine in.  "M mode" is the only one that's really
> >> usable these days (the other, "N mode", doesn't boot).  Under M mode,
> >> there's five address spaces, of which, "I/O Space" is selectable by turning on
> >> specific bits in the 64-bit memory address.
> >>
> >> So, given 0x920000000b200000, bits 63:62 select physical addressing (10) and
> >> bits 58:57 select I/O space (01), resulting in all PCI addresses (or any I/O
> >> address) appearing in the 0x92xxxxxxxxxxxxxx range.  The rest of the bits go
> >> on to select which NUMA node the I/O is targeted at and which Crosstalk widget
> >> to access, and so on.  I still haven't fully understood it all myself just
> >> yet.
> >>
> >> You can google for document 007-3410-001, "Origin and Onyx2 Programmer's
> >> Reference Manual" for more details (it's no longer available via SGI's
> >> TechPubs, but it is archived on plenty of other sites).
> >>
> >> This particular machine that I have has three "BRIDGE" chips in it, all
> >> attached to the Crosstalk bus.  Each BRIDGE can address up to 8 PCI devices,
> >> but some crosstalk boards slap a BRIDGE chip on for just a single device.  In
> >> this instance, the BRIDGE w/ address 0x920000000fxxxxxx is the IO6 at xtalk
> >> widget 0xf (this has the basic I/O devices for this machine).  The BRIDGE at
> >> 0x920000000cxxxxxx (widget 0xc) is a PCI "shoebox" that can hold up to three
> >> PCI-X cards, and the BRIDGE at 0x920000000bxxxxxx (widget 0xb) is an
> >> SGI-specific HIPPI board (ID'ed as a LINC I/O controller by pci.ids).

Re: [PATCH] PCI: PCI_PROBE_ONLY clean-up

[Ralf Baechle]

On Thu, Jun 23, 2016 at 05:16:47PM -0500, Bjorn Helgaas wrote:

> Lorenzo is changing the PCI_PROBE_ONLY case so the BARs and windows remain
> immutable, but we insert the resources into the iomem_resource tree.
> 
> The ideal thing would be to remove the use of PCI_PROBE_ONLY completely,
> and allow Linux to program BARs as necessary.  If the firmware *has*
> programmed the BARs, we don't change them unless we find something broken,
> so in most cases PCI_PROBE_ONLY is unnecessary.
> 
> There are several MIPS platforms (bcm1480, ip27, sb1250, virtio_guest, xlp,
> xlr) that set PCI_PROBE_ONLY for reasons I don't know.  These were added
> by:
> 
>   bcm1480
>     Andrew Isaacson <adi@broadcom.com>
>     dc41f94f7709 ("Support for the BCM1480 on-chip PCI-X bridge.")
> 
>   ip27
>     Thomas Bogendoerfer <tsbogend@alpha.franken.de>
>     96173a6c4ebc ("[MIPS] IP27: misc fixes")

Afair  I did originally implement the IP27 use of PCI_PROBE_ONLY.  The
problem is that it was not possible for the kernel to assign resources
properly on an IP27.  Also that would invalidate firmware configuration
information, so we had to live with whatever the firmware (mis)configured
for us.  Afair - it's a darn long time ...  I think the reasoning for
the BCM1480 was similar but Andy will hopefully recall the details.

  Ralf

Re: [PATCH] PCI: PCI_PROBE_ONLY clean-up

[Andy Isaacson]

On Fri, Jun 24, 2016 at 05:50:21PM +0200, Ralf Baechle wrote:
> >   bcm1480
> >     Andrew Isaacson <adi@broadcom.com>
> >     dc41f94f7709 ("Support for the BCM1480 on-chip PCI-X bridge.")
> > 
> >   ip27
> >     Thomas Bogendoerfer <tsbogend@alpha.franken.de>
> >     96173a6c4ebc ("[MIPS] IP27: misc fixes")
> 
> Afair  I did originally implement the IP27 use of PCI_PROBE_ONLY.  The
> problem is that it was not possible for the kernel to assign resources
> properly on an IP27.  Also that would invalidate firmware configuration
> information, so we had to live with whatever the firmware (mis)configured
> for us.  Afair - it's a darn long time ...  I think the reasoning for
> the BCM1480 was similar but Andy will hopefully recall the details.

I don't remember the details, unfortunately, but my general recollection
is similar -- CFE on bcm1480 was doing a fairly good job of configuring
PCI, and I think we needed to preserve the configuration for Reasons
rather than letting the kernel change things away from the CFE setup.

I unfortunately no longer have any 1480 hardware to test on, nor do I
know who's left at BCM who might be able to validate this change.

-andy

Re: [PATCH] PCI: PCI_PROBE_ONLY clean-up

[Ralf Baechle]

On Fri, Jun 24, 2016 at 11:46:02AM -0700, Andy Isaacson wrote:

> I don't remember the details, unfortunately, but my general recollection
> is similar -- CFE on bcm1480 was doing a fairly good job of configuring
> PCI, and I think we needed to preserve the configuration for Reasons
> rather than letting the kernel change things away from the CFE setup.
> 
> I unfortunately no longer have any 1480 hardware to test on, nor do I
> know who's left at BCM who might be able to validate this change.

I still have a small pile of Sibyte hardware of various description,
including BCM91480 and can test.

  Ralf

Re: [PATCH] PCI: PCI_PROBE_ONLY clean-up

[Joshua Kinard]

On 06/24/2016 11:50, Ralf Baechle wrote:
> On Thu, Jun 23, 2016 at 05:16:47PM -0500, Bjorn Helgaas wrote:
> 
>> Lorenzo is changing the PCI_PROBE_ONLY case so the BARs and windows remain
>> immutable, but we insert the resources into the iomem_resource tree.
>>
>> The ideal thing would be to remove the use of PCI_PROBE_ONLY completely,
>> and allow Linux to program BARs as necessary.  If the firmware *has*
>> programmed the BARs, we don't change them unless we find something broken,
>> so in most cases PCI_PROBE_ONLY is unnecessary.
>>
>> There are several MIPS platforms (bcm1480, ip27, sb1250, virtio_guest, xlp,
>> xlr) that set PCI_PROBE_ONLY for reasons I don't know.  These were added
>> by:
>>
>>   bcm1480
>>     Andrew Isaacson <adi@broadcom.com>
>>     dc41f94f7709 ("Support for the BCM1480 on-chip PCI-X bridge.")
>>
>>   ip27
>>     Thomas Bogendoerfer <tsbogend@alpha.franken.de>
>>     96173a6c4ebc ("[MIPS] IP27: misc fixes")
> 
> Afair  I did originally implement the IP27 use of PCI_PROBE_ONLY.  The
> problem is that it was not possible for the kernel to assign resources
> properly on an IP27.  Also that would invalidate firmware configuration
> information, so we had to live with whatever the firmware (mis)configured
> for us.  Afair - it's a darn long time ...  I think the reasoning for
> the BCM1480 was similar but Andy will hopefully recall the details.

I'm not onto 4.7 yet, but I've got some major rework of the BRIDGE code that's
solved the resource assignment bit in IP27 (including being able to kzalloc the
bridge_controller struct now).  Can try this change out with the changes I've
done to see if everything still works, or if IP27 (and maybe IP30) still
require this flag, or if it can be implemented in another way.


-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
6144R/F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And our
lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic