[PATCH v1 0/2] PCI: Sparc 64-bit resource fixups

[PATCH v1 0/2] PCI: Sparc 64-bit resource fixups

[Bjorn Helgaas]

I'm trying to make some progress on this old series from Yinghai [1].
There's a lot more to do than just these first two patches, but maybe a
tiny bit of incremental progress is better than none.

The first patch ("Support arbitrary host bridge address offset") is
unchanged from Yinghai's posting except for the changelog.

I'm interested in feedback about the second ("Reserve System ROM and Video
ROM") patch because it changes something that *looks* wrong to me, but I
don't know anything about the Sparc address map.

If this is on the right track, I'd like to merge these via my PCI tree in
the hope that I can include more of Yinghai's patches as well.

[1] https://lkml.kernel.org/r/20170421050500.13957-1-yinghai@kernel.org

---

Bjorn Helgaas (1):
      sparc/PCI: Reserve System ROM and Video ROM outside of PCI space

Yinghai Lu (1):
      sparc/PCI: Support arbitrary host bridge address offset


 arch/sparc/kernel/pci.c        |    6 ++--
 arch/sparc/kernel/pci_common.c |   54 +++++++++++++++++++---------------------
 arch/sparc/kernel/pci_impl.h   |    4 +++
 arch/sparc/mm/init_64.c        |   17 +++++++++++++
 4 files changed, 50 insertions(+), 31 deletions(-)

[PATCH v1 1/2] sparc/PCI: Support arbitrary host bridge address offset

[Bjorn Helgaas]

From: Yinghai Lu <yinghai@kernel.org>

Add support for arbitrary bus address offset.  Previously we ignored the
child (PCI) address in the "ranges" property and assumed it was always
zero.  That means every host bridge window mapped to PCI bus address zero,
e.g.,

  pci_bus 0000:00: root bus resource [mem 0x2000000000000-0x200007fffffff] (bus address [0x00000000-0x7fffffff])

But some systems have host bridge windows with non-zero child addresses, so
parse the child address and compute the offset between the parent (CPU) and
child (PCI) addresses.  This allows windows like these:

  /pci@305: PCI MEM [mem 0x2000000100000-0x200007effffff] offset 2000000000000
  pci_sun4v f02ae7f8: PCI host bridge to bus 0000:00
  pci_bus 0000:00: root bus resource [mem 0x2000000100000-0x200007effffff] (bus address [0x00100000-0x7effffff])

[bhelgaas: changelog]
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: sparclinux@vger.kernel.org
---
 arch/sparc/kernel/pci.c        |    6 +++---
 arch/sparc/kernel/pci_common.c |   34 ++++++++++++++++++++++++++--------
 arch/sparc/kernel/pci_impl.h   |    4 ++++
 3 files changed, 33 insertions(+), 11 deletions(-)

diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index 220d0f36560a..41b20edb427d 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -664,12 +664,12 @@ struct pci_bus *pci_scan_one_pbm(struct pci_pbm_info *pbm,
 	printk("PCI: Scanning PBM %s\n", node->full_name);
 
 	pci_add_resource_offset(&resources, &pbm->io_space,
-				pbm->io_space.start);
+				pbm->io_offset);
 	pci_add_resource_offset(&resources, &pbm->mem_space,
-				pbm->mem_space.start);
+				pbm->mem_offset);
 	if (pbm->mem64_space.flags)
 		pci_add_resource_offset(&resources, &pbm->mem64_space,
-					pbm->mem_space.start);
+					pbm->mem64_offset);
 	pbm->busn.start = pbm->pci_first_busno;
 	pbm->busn.end	= pbm->pci_last_busno;
 	pbm->busn.flags	= IORESOURCE_BUS;
diff --git a/arch/sparc/kernel/pci_common.c b/arch/sparc/kernel/pci_common.c
index 1e10fb26fa88..14fa74f90fdb 100644
--- a/arch/sparc/kernel/pci_common.c
+++ b/arch/sparc/kernel/pci_common.c
@@ -397,6 +397,8 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 	int i, saw_mem, saw_io;
 	int num_pbm_ranges;
 
+	/* Corresponding generic code in of_pci_get_host_bridge_resources() */
+
 	saw_mem = saw_io = 0;
 	pbm_ranges = of_get_property(pbm->op->dev.of_node, "ranges", &i);
 	if (!pbm_ranges) {
@@ -411,13 +413,16 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 
 	for (i = 0; i < num_pbm_ranges; i++) {
 		const struct linux_prom_pci_ranges *pr = &pbm_ranges[i];
-		unsigned long a, size;
+		unsigned long a, size, region_a;
 		u32 parent_phys_hi, parent_phys_lo;
+		u32 child_phys_mid, child_phys_lo;
 		u32 size_hi, size_lo;
 		int type;
 
 		parent_phys_hi = pr->parent_phys_hi;
 		parent_phys_lo = pr->parent_phys_lo;
+		child_phys_mid = pr->child_phys_mid;
+		child_phys_lo = pr->child_phys_lo;
 		if (tlb_type == hypervisor)
 			parent_phys_hi &= 0x0fffffff;
 
@@ -427,6 +432,8 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 		type = (pr->child_phys_hi >> 24) & 0x3;
 		a = (((unsigned long)parent_phys_hi << 32UL) |
 		     ((unsigned long)parent_phys_lo  <<  0UL));
+		region_a = (((unsigned long)child_phys_mid << 32UL) |
+		     ((unsigned long)child_phys_lo  <<  0UL));
 		size = (((unsigned long)size_hi << 32UL) |
 			((unsigned long)size_lo  <<  0UL));
 
@@ -441,6 +448,7 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 			pbm->io_space.start = a;
 			pbm->io_space.end = a + size - 1UL;
 			pbm->io_space.flags = IORESOURCE_IO;
+			pbm->io_offset = a - region_a;
 			saw_io = 1;
 			break;
 
@@ -449,6 +457,7 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 			pbm->mem_space.start = a;
 			pbm->mem_space.end = a + size - 1UL;
 			pbm->mem_space.flags = IORESOURCE_MEM;
+			pbm->mem_offset = a - region_a;
 			saw_mem = 1;
 			break;
 
@@ -457,6 +466,7 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 			pbm->mem64_space.start = a;
 			pbm->mem64_space.end = a + size - 1UL;
 			pbm->mem64_space.flags = IORESOURCE_MEM;
+			pbm->mem64_offset = a - region_a;
 			saw_mem = 1;
 			break;
 
@@ -472,14 +482,22 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 		prom_halt();
 	}
 
-	printk("%s: PCI IO[%llx] MEM[%llx]",
-	       pbm->name,
-	       pbm->io_space.start,
-	       pbm->mem_space.start);
+	if (pbm->io_space.flags)
+		printk("%s: PCI IO %pR offset %llx\n",
+		       pbm->name, &pbm->io_space, pbm->io_offset);
+	if (pbm->mem_space.flags)
+		printk("%s: PCI MEM %pR offset %llx\n",
+		       pbm->name, &pbm->mem_space, pbm->mem_offset);
+	if (pbm->mem64_space.flags && pbm->mem_space.flags) {
+		if (pbm->mem64_space.start <= pbm->mem_space.end)
+			pbm->mem64_space.start = pbm->mem_space.end + 1;
+		if (pbm->mem64_space.start > pbm->mem64_space.end)
+			pbm->mem64_space.flags = 0;
+	}
+
 	if (pbm->mem64_space.flags)
-		printk(" MEM64[%llx]",
-		       pbm->mem64_space.start);
-	printk("\n");
+		printk("%s: PCI MEM64 %pR offset %llx\n",
+		       pbm->name, &pbm->mem64_space, pbm->mem64_offset);
 
 	pbm->io_space.name = pbm->mem_space.name = pbm->name;
 	pbm->mem64_space.name = pbm->name;
diff --git a/arch/sparc/kernel/pci_impl.h b/arch/sparc/kernel/pci_impl.h
index ac172961d276..4e3d15189fa9 100644
--- a/arch/sparc/kernel/pci_impl.h
+++ b/arch/sparc/kernel/pci_impl.h
@@ -100,6 +100,10 @@ struct pci_pbm_info {
 	struct resource			mem_space;
 	struct resource			mem64_space;
 	struct resource			busn;
+	/* offset */
+	resource_size_t			io_offset;
+	resource_size_t			mem_offset;
+	resource_size_t			mem64_offset;
 
 	/* Base of PCI Config space, can be per-PBM or shared. */
 	unsigned long			config_space;

[PATCH v1 2/2] sparc/PCI: Reserve System ROM and Video ROM outside of PCI space

[Bjorn Helgaas]

From: Bjorn Helgaas <bhelgaas@google.com>

Previously, pci_register_legacy_regions() reserved PCI address space under
every PCI host bridge for the System ROM and the Video ROM, but these
regions are not part of PCI address space.

PCI address space usage is discoverable via PCI device BARs, PCI bridge
window registers, or legacy IDE or VGA devices that use the hardwired
legacy resources, e.g., memory 0xa0000-0xbffff, io 0x3c0-0x3df, etc.

The System ROM and Video ROM areas are none of these, so they should be
reserved as part of the system address space, not the PCI address space.

Reserve the System ROM and Video ROM areas once per system.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
---
 arch/sparc/kernel/pci_common.c |   20 --------------------
 arch/sparc/mm/init_64.c        |   17 +++++++++++++++++
 2 files changed, 17 insertions(+), 20 deletions(-)

diff --git a/arch/sparc/kernel/pci_common.c b/arch/sparc/kernel/pci_common.c
index 14fa74f90fdb..38d46bcc8634 100644
--- a/arch/sparc/kernel/pci_common.c
+++ b/arch/sparc/kernel/pci_common.c
@@ -344,26 +344,6 @@ static void pci_register_legacy_regions(struct resource *io_res,
 	p->end = p->start + 0x1ffffUL;
 	p->flags = IORESOURCE_BUSY;
 	request_resource(mem_res, p);
-
-	p = kzalloc(sizeof(*p), GFP_KERNEL);
-	if (!p)
-		return;
-
-	p->name = "System ROM";
-	p->start = mem_res->start + 0xf0000UL;
-	p->end = p->start + 0xffffUL;
-	p->flags = IORESOURCE_BUSY;
-	request_resource(mem_res, p);
-
-	p = kzalloc(sizeof(*p), GFP_KERNEL);
-	if (!p)
-		return;
-
-	p->name = "Video ROM";
-	p->start = mem_res->start + 0xc0000UL;
-	p->end = p->start + 0x7fffUL;
-	p->flags = IORESOURCE_BUSY;
-	request_resource(mem_res, p);
 }
 
 static void pci_register_iommu_region(struct pci_pbm_info *pbm)
diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c
index 995f9490334d..a9f94e911e0a 100644
--- a/arch/sparc/mm/init_64.c
+++ b/arch/sparc/mm/init_64.c
@@ -3089,6 +3089,20 @@ static struct resource bss_resource = {
 	.flags	= IORESOURCE_BUSY | IORESOURCE_SYSTEM_RAM
 };
 
+static struct resource system_rom_resource = {
+	.name	= "System ROM",
+	.start  = 0xf0000,
+	.end    = 0xfffff,
+	.flags	= IORESOURCE_BUSY | IORESOURCE_MEM,
+};
+
+static struct resource video_rom_resource = {
+	.name	= "Video ROM",
+	.start  = 0xc0000,
+	.end    = 0xc7fff,
+	.flags	= IORESOURCE_BUSY | IORESOURCE_MEM,
+};
+
 static inline resource_size_t compute_kern_paddr(void *addr)
 {
 	return (resource_size_t) (addr - KERNBASE + kern_base);
@@ -3134,6 +3148,9 @@ static int __init report_memory(void)
 		insert_resource(res, &bss_resource);
 	}
 
+	request_resource(&iomem_resource, &system_rom_resource);
+	request_resource(&iomem_resource, &video_rom_resource);
+
 	return 0;
 }
 arch_initcall(report_memory);

Re: [PATCH v1 0/2] PCI: Sparc 64-bit resource fixups

[Bjorn Helgaas]

[+cc David Ahern, TJ]

On Thu, Feb 15, 2018 at 09:17:58AM -0600, Bjorn Helgaas wrote:
> I'm trying to make some progress on this old series from Yinghai [1].
> There's a lot more to do than just these first two patches, but maybe a
> tiny bit of incremental progress is better than none.
> 
> The first patch ("Support arbitrary host bridge address offset") is
> unchanged from Yinghai's posting except for the changelog.
> 
> I'm interested in feedback about the second ("Reserve System ROM and Video
> ROM") patch because it changes something that *looks* wrong to me, but I
> don't know anything about the Sparc address map.
> 
> If this is on the right track, I'd like to merge these via my PCI tree in
> the hope that I can include more of Yinghai's patches as well.
> 
> [1] https://lkml.kernel.org/r/20170421050500.13957-1-yinghai@kernel.org
> 
> ---
> 
> Bjorn Helgaas (1):
>       sparc/PCI: Reserve System ROM and Video ROM outside of PCI space
> 
> Yinghai Lu (1):
>       sparc/PCI: Support arbitrary host bridge address offset
> 
> 
>  arch/sparc/kernel/pci.c        |    6 ++--
>  arch/sparc/kernel/pci_common.c |   54 +++++++++++++++++++---------------------
>  arch/sparc/kernel/pci_impl.h   |    4 +++
>  arch/sparc/mm/init_64.c        |   17 +++++++++++++
>  4 files changed, 50 insertions(+), 31 deletions(-)

Both these patches are on my pci/sparc branch and appeared in the
Feb 19 linux-next tree.

Any testing and feedback (especially on the second patch, which should
change /proc/iomem) would be great.

They're headed for v4.17 unless I hear about issues.

It would be useful to hear about what's still broken so I can try to
pull in the other patches.

IIUC, the first patch should fix something on T4.

I think patch 04/13 of the original series adds IORESOURCE_MEM_64 to
host bridge windows.  We don't do that for other arches, so it's not
clear to me why we need it for sparc.

Patch 09/13 of the original series fixes something on T5-8.  I haven't
included that yet, so it's probably still broken.

Bjorn

Re: [PATCH v1 0/2] PCI: Sparc 64-bit resource fixups

[Khalid Aziz]

[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]

On 02/20/2018 04:39 PM, Bjorn Helgaas wrote:
> 
> Both these patches are on my pci/sparc branch and appeared in the
> Feb 19 linux-next tree.
> 
> Any testing and feedback (especially on the second patch, which should
> change /proc/iomem) would be great.
> 
> They're headed for v4.17 unless I hear about issues.
> 
> It would be useful to hear about what's still broken so I can try to
> pull in the other patches.
> 

Hi Bjorn,

I tested these two patches on an M7 machine. With these two patches, one 
of the PCI devices was able to claim BAR 0 which it fails to without 
these patches. This device happens to be a NIC, so with these two 
patches, the NIC finally works on my test machine. I have attached log 
showing all the failures to claim a BAR. First file is from 4.16-rc2 
while the second one is from 4.16-rc2+these two patches. There still are 
a number of failures to claim BAR even with these two patches.

I am also attaching /proc/iomem for the 4.16-rc2 kernel and 4.16-rc2 
with these two patches.

--
Khalid

[-- Attachment #2: log.rc2 --]
[-- Type: text/plain, Size: 1831 bytes --]

pci 0000:03:00.0: can't claim BAR 0 [mem 0x2000100000000-0x20001007fffff 64bit pref]: no compatible bridge window
pci 0000:03:00.0: can't claim BAR 6 [mem 0x2000000100000-0x200000017ffff]: no compatible bridge window
pci 0000:03:00.1: can't claim BAR 6 [mem 0x2000000180000-0x20000001fffff]: no compatible bridge window
pci 0000:03:00.2: can't claim BAR 3 [mem 0x2000100d10000-0x2000100d17fff 64bit pref]: address conflict with 0000:03:00.1 [mem 0x2000100b90000-0x2000100d8ffff 64bit pref]
pci 0000:03:00.2: can't claim BAR 7 [mem 0x2000100e20000-0x200010101ffff 64bit pref]: address conflict with 0000:03:00.1 [mem 0x2000101000000-0x20001017fffff 64bit pref]
pci 0000:03:00.2: can't claim BAR 10 [mem 0x2000101020000-0x200010109ffff 64bit pref]: address conflict with 0000:03:00.1 [mem 0x2000101000000-0x20001017fffff 64bit pref]
pci 0000:03:00.3: can't claim BAR 3 [mem 0x2000100d18000-0x2000100d1ffff 64bit pref]: address conflict with 0000:03:00.1 [mem 0x2000100b90000-0x2000100d8ffff 64bit pref]
pci 0000:04:00.0: can't claim BAR 0 [mem 0x2000110000000-0x200011000ffff 64bit]: no compatible bridge window
pci 0000:04:00.0: can't claim BAR 2 [mem 0x2000110010000-0x2000110011fff 64bit]: no compatible bridge window
pci 0001:54:00.0: can't claim BAR 1 [mem 0x2020120000000-0x202012000ffff 64bit]: no compatible bridge window
pci 0001:54:00.0: can't claim BAR 3 [mem 0x2020120040000-0x202012007ffff 64bit]: no compatible bridge window
pci 0001:54:00.0: can't claim BAR 6 [mem 0x2020010100000-0x20200101fffff]: no compatible bridge window
pci 0001:54:00.0: can't claim BAR 7 [mem 0x2020120180000-0x202012027ffff 64bit]: no compatible bridge window
pci 0001:54:00.0: can't claim BAR 9 [mem 0x2020120280000-0x202012067ffff 64bit]: no compatible bridge window
i40e 0000:03:00.0: BAR 0: can't reserve [mem size 0x00800000 64bit pref]

[-- Attachment #3: log.rc2-bjorn --]
[-- Type: text/plain, Size: 2129 bytes --]

pci 0000:00:01.0: can't claim BAR 14 [mem 0x2000000100000-0x20000002fffff]: address conflict with Video RAM area [??? 0x20000001a0000-0x20000001bffff flags 0x80000000]
pci 0000:01:00.0: can't claim BAR 14 [mem 0x2000000100000-0x20000002fffff]: no compatible bridge window
pci 0000:02:01.0: can't claim BAR 14 [mem 0x2000000100000-0x20000002fffff]: no compatible bridge window
pci 0000:03:00.0: can't claim BAR 6 [mem 0x2000000100000-0x200000017ffff]: no compatible bridge window
pci 0000:03:00.1: can't claim BAR 6 [mem 0x2000000180000-0x20000001fffff]: no compatible bridge window
pci 0000:03:00.2: can't claim BAR 6 [mem 0x2000000200000-0x200000027ffff]: no compatible bridge window
pci 0000:03:00.3: can't claim BAR 6 [mem 0x2000000280000-0x20000002fffff]: no compatible bridge window
pci 0000:04:00.0: can't claim BAR 0 [mem 0x2000110000000-0x200011000ffff 64bit]: no compatible bridge window
pci 0000:04:00.0: can't claim BAR 2 [mem 0x2000110010000-0x2000110011fff 64bit]: no compatible bridge window
pci 0001:00:02.0: can't claim BAR 14 [mem 0x2020000100000-0x20200101fffff]: address conflict with Video RAM area [??? 0x20200001a0000-0x20200001bffff flags 0x80000000]
pci 0001:05:00.0: can't claim BAR 14 [mem 0x2020000100000-0x20200101fffff]: no compatible bridge window
pci 0001:06:04.0: can't claim BAR 14 [mem 0x2020000100000-0x20200080fffff]: no compatible bridge window
pci 0001:06:05.0: can't claim BAR 14 [mem 0x2020008100000-0x20200100fffff]: no compatible bridge window
pci 0001:06:17.0: can't claim BAR 14 [mem 0x2020010100000-0x20200101fffff]: no compatible bridge window
pci 0001:54:00.0: can't claim BAR 1 [mem 0x2020120000000-0x202012000ffff 64bit]: no compatible bridge window
pci 0001:54:00.0: can't claim BAR 3 [mem 0x2020120040000-0x202012007ffff 64bit]: no compatible bridge window
pci 0001:54:00.0: can't claim BAR 6 [mem 0x2020010100000-0x20200101fffff]: no compatible bridge window
pci 0001:54:00.0: can't claim BAR 7 [mem 0x2020120080000-0x202012017ffff 64bit]: no compatible bridge window
pci 0001:54:00.0: can't claim BAR 9 [mem 0x2020120180000-0x202012057ffff 64bit]: no compatible bridge window

[-- Attachment #4: iomem.rc2 --]
[-- Type: text/plain, Size: 3488 bytes --]

50400000-8fec9fff : System RAM
90000000-1fefffffff : System RAM
400000000000-401fef5cffff : System RAM
  400000004000-4000006da93f : Kernel code
  4000006da940-400000a9985f : Kernel data
  400000b44000-400000d04517 : Kernel bss
2000000100000-200007effffff : /pci@300
  20000001a0000-20000001bffff : Video RAM area
  20000001c0000-20000001c7fff : Video ROM
  20000001f0000-20000001fffff : System ROM
  2000000200000-20000003fffff : PCI Bus 0000:01
    2000000200000-20000003fffff : PCI Bus 0000:02
      2000000200000-20000003fffff : PCI Bus 0000:03
        2000000200000-200000027ffff : 0000:03:00.2
        2000000280000-20000002fffff : 0000:03:00.3
  2000000400000-20000103fffff : PCI Bus 0000:06
    2000000400000-20000103fffff : PCI Bus 0000:07
      2000000400000-20000083fffff : PCI Bus 0000:08
      2000008400000-20000103fffff : PCI Bus 0000:28
2000100000000-2000dffffffff : /pci@300
  2000100100000-20001200fffff : PCI Bus 0000:01
    2000100100000-20001200fffff : PCI Bus 0000:02
      2000100100000-20001100fffff : PCI Bus 0000:03
        2000100800000-2000100807fff : 0000:03:00.0
        2000100808000-200010080ffff : 0000:03:00.1
          2000100808000-200010080ffff : i40e
        2000100910000-2000100b0ffff : 0000:03:00.0
        2000100b10000-2000100b8ffff : 0000:03:00.0
        2000100b90000-2000100d8ffff : 0000:03:00.1
          2000100d10000-2000100d17fff : i40e
          2000100d18000-2000100d1ffff : i40e
        2000100d90000-2000100e0ffff : 0000:03:00.1
        2000101000000-20001017fffff : 0000:03:00.1
          2000101000000-20001017fffff : i40e
        2000101800000-2000101ffffff : 0000:03:00.2
          2000101800000-2000101ffffff : i40e
        2000102000000-20001027fffff : 0000:03:00.3
          2000102000000-20001027fffff : i40e
        2000102900000-2000102afffff : 0000:03:00.3
        2000102b00000-2000102b7ffff : 0000:03:00.3
        2000110000000-200011000ffff : xhci-hcd
      2000110100000-20001200fffff : PCI Bus 0000:04
  2000120100000-20001400fffff : PCI Bus 0000:06
    2000120100000-20001400fffff : PCI Bus 0000:07
      2000120100000-20001300fffff : PCI Bus 0000:08
      2000130100000-20001400fffff : PCI Bus 0000:28
2020000100000-202007effffff : /pci@302
  20200001a0000-20200001bffff : Video RAM area
  20200001c0000-20200001c7fff : Video ROM
  20200001f0000-20200001fffff : System ROM
  2020000200000-20200002fffff : PCI Bus 0001:01
    2020000200000-20200002fffff : PCI Bus 0001:02
      2020000200000-20200002fffff : PCI Bus 0001:03
  2020000300000-20200103fffff : PCI Bus 0001:05
    2020000300000-20200103fffff : PCI Bus 0001:06
      2020000300000-20200082fffff : PCI Bus 0001:07
      2020008300000-20200102fffff : PCI Bus 0001:2d
      2020010300000-20200103fffff : PCI Bus 0001:54
2020100000000-2020dffffffff : /pci@302
  2020100100000-20202100fffff : PCI Bus 0001:01
    2020100100000-20202100fffff : PCI Bus 0001:02
      2020100100000-20202100fffff : PCI Bus 0001:03
        2020100108000-2020100187fff : 0001:03:00.0
        2020100188000-2020100207fff : 0001:03:00.0
        2020108000000-202010fffffff : 0001:03:00.0
        2020110100000-20202100fffff : 0001:03:00.0
  2020210100000-20202400fffff : PCI Bus 0001:05
    2020210100000-20202400fffff : PCI Bus 0001:06
      2020210100000-20202200fffff : PCI Bus 0001:07
      2020220100000-20202300fffff : PCI Bus 0001:2d
        2020230000000-202023000ffff : mpt3sas
        2020230040000-202023007ffff : mpt3sas
      2020230100000-20202400fffff : PCI Bus 0001:54

[-- Attachment #5: iomem.rc2-bjorn --]
[-- Type: text/plain, Size: 3475 bytes --]

000c0000-000c7fff : Video ROM
000f0000-000fffff : System ROM
50400000-8fec9fff : System RAM
90000000-1fefffffff : System RAM
400000000000-401fef5cffff : System RAM
  400000004000-4000006da93f : Kernel code
  4000006da940-400000a998df : Kernel data
  400000b44000-400000d04517 : Kernel bss
2000000100000-200007effffff : /pci@300
  20000001a0000-20000001bffff : Video RAM area
  2000000300000-20000102fffff : PCI Bus 0000:06
    2000000300000-20000102fffff : PCI Bus 0000:07
      2000000300000-20000082fffff : PCI Bus 0000:08
      2000008300000-20000102fffff : PCI Bus 0000:28
2000100000000-2000dffffffff : /pci@300
  2000100000000-200011fffffff : PCI Bus 0000:01
    2000100000000-200011fffffff : PCI Bus 0000:02
      2000100000000-200010fffffff : PCI Bus 0000:03
        2000100000000-20001007fffff : 0000:03:00.0
          2000100000000-20001007fffff : i40e
        2000100800000-2000100807fff : 0000:03:00.0
          2000100800000-2000100807fff : i40e
        2000100808000-200010080ffff : 0000:03:00.1
          2000100808000-200010080ffff : i40e
        2000100810000-2000100a0ffff : 0000:03:00.0
        2000100a10000-2000100a8ffff : 0000:03:00.0
        2000100a90000-2000100c8ffff : 0000:03:00.1
        2000100c90000-2000100d0ffff : 0000:03:00.1
        2000100d10000-2000100d17fff : 0000:03:00.2
          2000100d10000-2000100d17fff : i40e
        2000100d18000-2000100d1ffff : 0000:03:00.3
          2000100d18000-2000100d1ffff : i40e
        2000100d20000-2000100f1ffff : 0000:03:00.2
        2000100f20000-2000100f9ffff : 0000:03:00.2
        2000101000000-20001017fffff : 0000:03:00.1
          2000101000000-20001017fffff : i40e
        2000101800000-2000101ffffff : 0000:03:00.2
          2000101800000-2000101ffffff : i40e
        2000102000000-20001027fffff : 0000:03:00.3
          2000102000000-20001027fffff : i40e
        2000102800000-20001029fffff : 0000:03:00.3
        2000102a00000-2000102a7ffff : 0000:03:00.3
      2000110000000-200011fffffff : PCI Bus 0000:04
        2000110000000-200011000ffff : xhci-hcd
  2000120000000-200013fffffff : PCI Bus 0000:06
    2000120000000-200013fffffff : PCI Bus 0000:07
      2000120000000-200012fffffff : PCI Bus 0000:08
      2000130000000-200013fffffff : PCI Bus 0000:28
2020000100000-202007effffff : /pci@302
  20200001a0000-20200001bffff : Video RAM area
  2020000200000-20200102fffff : PCI Bus 0001:05
    2020000200000-20200102fffff : PCI Bus 0001:06
      2020000200000-20200081fffff : PCI Bus 0001:07
      2020008200000-20200101fffff : PCI Bus 0001:2d
      2020010200000-20200102fffff : PCI Bus 0001:54
        2020010200000-20200102fffff : 0001:54:00.0
2020100000000-2020dffffffff : /pci@302
  2020100000000-202020fffffff : PCI Bus 0001:01
    2020100000000-202020fffffff : PCI Bus 0001:02
      2020100000000-202020fffffff : PCI Bus 0001:03
        2020100000000-2020100003fff : 0001:03:00.0
        2020100004000-2020100007fff : 0001:03:00.0
        2020100008000-2020100087fff : 0001:03:00.0
        2020100088000-2020100107fff : 0001:03:00.0
        2020108000000-202010fffffff : 0001:03:00.0
        2020110000000-202020fffffff : 0001:03:00.0
  2020210000000-202023fffffff : PCI Bus 0001:05
    2020210000000-202023fffffff : PCI Bus 0001:06
      2020210000000-202021fffffff : PCI Bus 0001:07
      2020220000000-202022fffffff : PCI Bus 0001:2d
      2020230000000-202023fffffff : PCI Bus 0001:54
        2020230000000-202023000ffff : mpt3sas
        2020230040000-202023007ffff : mpt3sas

Re: [PATCH v1 0/2] PCI: Sparc 64-bit resource fixups

[David Miller]

From: Bjorn Helgaas <helgaas@kernel.org>
Date: Tue, 20 Feb 2018 17:39:35 -0600

> Both these patches are on my pci/sparc branch and appeared in the
> Feb 19 linux-next tree.
> 
> Any testing and feedback (especially on the second patch, which should
> change /proc/iomem) would be great.
> 
> They're headed for v4.17 unless I hear about issues.
> 
> It would be useful to hear about what's still broken so I can try to
> pull in the other patches.

I don't understand why you put the SYSTEM and Video ROM at addresses
relative to absolute zero.  The result of this is that it might
overlap real physical memory.

If a VGA card is present, it's ROM will respond to those addresses
relative to the PCI controller's MEM space area.

Their legacy resources are "hardwired", but those hardwired addresses
need to be relative to the PCI host controller's MEM areas in order
for them to be accessible.

I could understand removing the System ROM resource altogether, that
makes a lot of sense to me.

Thanks.

Re: [PATCH v1 0/2] PCI: Sparc 64-bit resource fixups

[Bjorn Helgaas]

On Sun, Mar 18, 2018 at 12:07:34PM -0400, David Miller wrote:
> From: Bjorn Helgaas <helgaas@kernel.org>
> Date: Tue, 20 Feb 2018 17:39:35 -0600
> 
> > Both these patches are on my pci/sparc branch and appeared in the
> > Feb 19 linux-next tree.
> > 
> > Any testing and feedback (especially on the second patch, which should
> > change /proc/iomem) would be great.
> > 
> > They're headed for v4.17 unless I hear about issues.
> > 
> > It would be useful to hear about what's still broken so I can try to
> > pull in the other patches.
> 
> I don't understand why you put the SYSTEM and Video ROM at addresses
> relative to absolute zero.  The result of this is that it might
> overlap real physical memory.
> 
> If a VGA card is present, it's ROM will respond to those addresses
> relative to the PCI controller's MEM space area.

I could have worded the changelog better.  This is about reserving PCI
bus addresses 0xc0000-0xc7fff, not the VGA framebuffer at
0xa0000-0xbffff.

If I understand correctly, a VGA device will respond to the
framebuffer at 0xa0000-0xbffff, but the device itself will not respond
to the 0xc0000-0xc7fff range.  I think the typical x86 PC arrangement
is that the BIOS reads the VGA option ROM using the normal relocatable
expansion ROM BAR and copies it into system memory at 0xc0000, so it
is in real physical memory.

I don't know what sparc firmware does, but I'm assuming the VGA PCI
device behaves the same in that it wouldn't respond to 0xc0000 itself.

> Their legacy resources are "hardwired", but those hardwired addresses
> need to be relative to the PCI host controller's MEM areas in order
> for them to be accessible.
> 
> I could understand removing the System ROM resource altogether, that
> makes a lot of sense to me.

Do you want me to remove the System ROM resource?  If so, I'll
make a separate patch to remove it, followed by one that does
whatever we figure out is the right thing for the video ROM.

Thanks for the feedback.

Bjorn

Re: [PATCH v1 0/2] PCI: Sparc 64-bit resource fixups

[David Miller]

From: Bjorn Helgaas <helgaas@kernel.org>
Date: Mon, 19 Mar 2018 12:11:40 -0500

> I could have worded the changelog better.  This is about reserving PCI
> bus addresses 0xc0000-0xc7fff, not the VGA framebuffer at
> 0xa0000-0xbffff.
> 
> If I understand correctly, a VGA device will respond to the
> framebuffer at 0xa0000-0xbffff, but the device itself will not respond
> to the 0xc0000-0xc7fff range.  I think the typical x86 PC arrangement
> is that the BIOS reads the VGA option ROM using the normal relocatable
> expansion ROM BAR and copies it into system memory at 0xc0000, so it
> is in real physical memory.
> 
> I don't know what sparc firmware does, but I'm assuming the VGA PCI
> device behaves the same in that it wouldn't respond to 0xc0000 itself.

The Sparc firmware doesn't copy the VGA option ROM.

That physical address location 0xc0000 in system memory is just
normal memory and most likely the kernel image itself is residing
there.

>> I could understand removing the System ROM resource altogether, that
>> makes a lot of sense to me.
> 
> Do you want me to remove the System ROM resource?  If so, I'll
> make a separate patch to remove it, followed by one that does
> whatever we figure out is the right thing for the video ROM.

Porbably it makes the most sense to remove both, given the above.

Re: [PATCH v1 0/2] PCI: Sparc 64-bit resource fixups

[Bjorn Helgaas]

[+cc Dave, Alex, dri-devel]

On Mon, Mar 19, 2018 at 02:28:31PM -0400, David Miller wrote:
> From: Bjorn Helgaas <helgaas@kernel.org>
> Date: Mon, 19 Mar 2018 12:11:40 -0500
> 
> > I could have worded the changelog better.  This is about reserving PCI
> > bus addresses 0xc0000-0xc7fff, not the VGA framebuffer at
> > 0xa0000-0xbffff.
> > 
> > If I understand correctly, a VGA device will respond to the
> > framebuffer at 0xa0000-0xbffff, but the device itself will not respond
> > to the 0xc0000-0xc7fff range.  I think the typical x86 PC arrangement
> > is that the BIOS reads the VGA option ROM using the normal relocatable
> > expansion ROM BAR and copies it into system memory at 0xc0000, so it
> > is in real physical memory.
> > 
> > I don't know what sparc firmware does, but I'm assuming the VGA PCI
> > device behaves the same in that it wouldn't respond to 0xc0000 itself.
> 
> The Sparc firmware doesn't copy the VGA option ROM.
> 
> That physical address location 0xc0000 in system memory is just
> normal memory and most likely the kernel image itself is residing
> there.
> 
> >> I could understand removing the System ROM resource altogether, that
> >> makes a lot of sense to me.
> > 
> > Do you want me to remove the System ROM resource?  If so, I'll
> > make a separate patch to remove it, followed by one that does
> > whatever we figure out is the right thing for the video ROM.
> 
> Porbably it makes the most sense to remove both, given the above.

I'd be happy to remove both.  But I wonder how X init works on sparc.

I thought X had code to run the option ROM under emulation and I
wondered if that might rely on the ROM image being at 0xc0000.  Maybe
it reads it from the sysfs "rom" file?  That would be available
regardless of what firmware does, but I have a dim recollection of 
something being more complicated than just reading the image directly
from the ROM.

Maybe Dave or Alex know the details?

Bjorn

Re: [PATCH v1 0/2] PCI: Sparc 64-bit resource fixups

[David Miller]

From: Bjorn Helgaas <helgaas@kernel.org>
Date: Mon, 19 Mar 2018 14:38:54 -0500

> I'd be happy to remove both.  But I wonder how X init works on
> sparc.

It's been many years since I worked on this, but I think it
mapped and used the PCI ROM resource for this.

Re: [PATCH v1 0/2] PCI: Sparc 64-bit resource fixups

[Bjorn Helgaas]

On Mon, Mar 19, 2018 at 07:33:41PM -0400, David Miller wrote:
> From: Bjorn Helgaas <helgaas@kernel.org>
> Date: Mon, 19 Mar 2018 14:38:54 -0500
> 
> > I'd be happy to remove both.  But I wonder how X init works on
> > sparc.
> 
> It's been many years since I worked on this, but I think it
> mapped and used the PCI ROM resource for this.

Thanks, I posted a v2 series that removes the System ROM and Video ROM
areas completely.