[PATCH v8 00/61] PCI: Resource allocation cleanup for v4.4

[PATCH v8 00/61] PCI: Resource allocation cleanup for v4.4

[Yinghai Lu]

Hi,

After 5b28541552ef (PCI: Restrict 64-bit prefetchable bridge windows
to 64-bit resources), we have several reports on resource allocation
failure, and we try to fix the problem with resource clip, and find
more problems.

One is realloc fail with two graphics cards above 4G.
One is from sparc that have problem with clip as we don't parse
mem64 for it.

Other report is about pci remove/rescan does not work on some setup
when BIOS tend to allocate small bus size.

This patchset enhance resource allocation to address those problems.

patch 1-11: parse MEM64 for sparc and other system with OF
patch 12-16: MMIO64 allocation enhancement
	treat non-pref mmio64 if parent bridges are all pcie.
	restore old pref allocation logic if hostbridge does not support mmio64.
patch 17-19: FIXED resource handling during realloc
	don't realloc resource if device firmware does not support bar change.
patch 20-23: bridge MMIO allocation with hotplug and last try.
	treat optional as required on first try when hotplug.
	MMIO size set to 0 for last try during realloc
patch 24-57: enhancement for mmio resource allocation:
	optimize bus mmio alignment calculation.
	optimize bus mmio optional alignment calculation.
	add support for alt size to prefer small bus size to small bus alignment.
	treat ROM bar as optional resource.
	during allocation, will pick up best fit resource, and allocate near end.
patch 58: add pci=assign_pref_bars to clear and assign pref bars.
patch 59-60: don't clear resource when allocation fails
patch 61: don't try io port allocation if root bus does not have io port.

I put latest copy at:
  git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git for-pci-v4.4-next

That is based on v4.3-rc4+.

Yijing and Wei Yang both tested the branch, and the patchset fixs rescan
problem, and is working well on powerpc setup.

Khalid Aziz tested on some sparc platforms that have different offset to
mem space start or support 64bit mmio.

Thanks

Yinghai

v2:
    - fix resource_disabled allocation from powerpc.
    - other warnings that were found by Fengguang's build robot.
    - io port alignment clean up.
    - rebased to v4.2-rc2.
    - better for two alt_size support.

v3:
    - fix reset bridge resource on last try: only with bridge resources.
    - remove/rescan path checking with old_size.

v4:
    - Update changelog according to Bjorn.
    - split calculate_size patch to two.
    - use required instead of must.

v5:
    - update changelog.
    - add one debug patch.
    - add one patch that get resource allocation near end of avialable window.

v6:
    - reorder patches according to Bjorn.

v7:
    - add some patches that fix problems on sparc that Khalid Aziz found.

v8:
    - update patch 2, 3 to remove the assumpation about mem64_offset == mem_offset.
    - add one patch for pci_find_root_bus_resource() to support above change.
    - update pref_compatible flags handling after pci_find_root_bus_resource().
    - add patch for quirk that ignore BAR for M1533 PCI-ISA bridge.

Yinghai Lu (61):
  sparc/PCI: Add mem64 resource parsing for root bus
  PCI: Add pci_find_root_bus_resource()
  sparc/PCI: Use correct bus address to resource offset
  sparc/PCI: Reserve legacy mmio after PCI mmio
  sparc/PCI: Add IORESOURCE_MEM_64 for 64-bit resource in OF parsing
  sparc/PCI: Keep resource idx order with bridge register number
  PCI: Kill wrong quirk about M7101
  PCI: Ignore BAR for ALi M1533 PCI-ISA bridge
  powerpc/PCI: Keep resource idx order with bridge register number
  powerpc/PCI: Add IORESOURCE_MEM_64 for 64-bit resource in OF parsing
  OF/PCI: Add IORESOURCE_MEM_64 for 64-bit resource
  PCI: Check pref compatible bit for mem64 resource of PCIe device
  PCI: Only treat non-pref mmio64 as pref if all bridges have MEM_64
  PCI: Add has_mem64 for struct host_bridge
  PCI: Only treat non-pref mmio64 as pref if host bridge has mmio64
  PCI: Restore pref MMIO allocation logic for host bridge without mmio64
  PCI: Don't release fixed resource for realloc
  PCI: Claim fixed resource during remove/rescan path
  PCI: Set resource to FIXED for LSI devices
  PCI: Separate realloc list checking after allocation
  PCI: Treat optional as required in first try for bridge rescan
  PCI: Get new realloc size for bridge for last try
  PCI: Don't release sibling bridge resources during hotplug
  PCI: Cleanup res_to_dev_res() printout
  PCI: Reuse res_to_dev_res() in reassign_resources_sorted()
  PCI: Use correct align for optional only resources during sorting
  PCI: Optimize bus min_align/size calculation during sizing
  PCI: Optimize bus align/size calculation for optional during sizing
  PCI: Don't add too much optional size for hotplug bridge MMIO
  PCI: Reorder resources list for required/optional resources
  PCI: Remove duplicated code for resource sorting
  PCI: Rename pdev_sort_resources() to pdev_assign_resources_prepare()
  PCI: Treat ROM resource as optional during realloc
  PCI: Add debug printout during releasing partial assigned resources
  PCI: Simplify res reference using in __assign_resources_sorted()
  PCI: Add __add_to_list()
  PCI: Cache window alignment value during bus sizing
  PCI: Check if resource is allocated before trying to assign one
  PCI: Separate out save_resources()/restore_resources()
  PCI: Move comment to pci_need_to_release()
  PCI: Separate required+optional assigning to another function
  PCI: Skip required+optional if there is no optional
  PCI: Move saved required resource list out of required+optional assigning
  PCI: Add alt_size ressource allocation support
  PCI: Add support for more than two alt_size entries under same bridge
  PCI: Fix size calculation with old_size on rescan path
  PCI: Don't add too much optional size for hotplug bridge io
  PCI: Move ISA io port align out of calculate_iosize()
  PCI: Don't add too much io port for hotplug bridge with old size
  PCI: Unify calculate_size() for io port and MMIO
  PCI: Allow bridge optional only io port resource required size to be 0
  PCI: Unify skip_ioresource_align()
  PCI: Kill macro checking for bus io port sizing
  resources: Split out __allocate_resource()
  resources: Make allocate_resource() return best fit resource
  PCI, x86: Allocate from high in available window for MMIO
  PCI: Add debug print out for min_align and alt_size
  PCI, x86: Add pci=assign_pref_bars to reallocate pref BARs
  PCI: Introduce resource_disabled()
  PCI: Don't set flags to 0 when assign resource fail
  PCI: Only try to assign io port only for root bus that support it

 arch/alpha/kernel/pci.c                   |    2 +-
 arch/ia64/pci/pci.c                       |    4 +-
 arch/microblaze/pci/pci-common.c          |   23 +-
 arch/mn10300/unit-asb2305/pci-asb2305.c   |    4 +-
 arch/mn10300/unit-asb2305/pci.c           |    4 +-
 arch/powerpc/kernel/pci-common.c          |   27 +-
 arch/powerpc/kernel/pci_of_scan.c         |   12 +-
 arch/powerpc/platforms/powernv/pci-ioda.c |   12 +-
 arch/s390/pci/pci.c                       |    2 +-
 arch/sparc/kernel/of_device_32.c          |    5 +-
 arch/sparc/kernel/of_device_64.c          |    5 +-
 arch/sparc/kernel/pci.c                   |   66 +-
 arch/sparc/kernel/pci_common.c            |  100 ++-
 arch/sparc/kernel/pci_impl.h              |    6 +
 arch/x86/include/asm/pci_x86.h            |    2 +-
 arch/x86/pci/common.c                     |    7 +-
 arch/x86/pci/i386.c                       |   86 +-
 arch/xtensa/kernel/pci.c                  |    4 +-
 drivers/iommu/intel-iommu.c               |    3 +-
 drivers/of/address.c                      |    4 +-
 drivers/pci/bus.c                         |    6 +-
 drivers/pci/host/pcie-rcar.c              |    2 +-
 drivers/pci/hotplug/acpiphp_glue.c        |    1 +
 drivers/pci/iov.c                         |    2 +-
 drivers/pci/pci.c                         |   40 +-
 drivers/pci/pci.h                         |    4 +
 drivers/pci/probe.c                       |   48 +-
 drivers/pci/quirks.c                      |   60 +-
 drivers/pci/rom.c                         |    2 +-
 drivers/pci/setup-bus.c                   | 1312 +++++++++++++++++++++--------
 drivers/pci/setup-res.c                   |   18 +-
 include/asm-generic/pci-bridge.h          |    2 +
 include/linux/ioport.h                    |    6 +-
 include/linux/pci.h                       |    8 +
 kernel/resource.c                         |  175 +++-
 35 files changed, 1508 insertions(+), 556 deletions(-)

-- 
1.8.4.5

[PATCH v8 01/61] sparc/PCI: Add mem64 resource parsing for root bus

[Yinghai Lu]

Found "no compatible bridge window" warning in boot log from T5-8.

pci 0000:00:01.0: can't claim BAR 15 [mem 0x100000000-0x4afffffff pref]: no compatible bridge window

That resource is above 4G, but does not get offset correctly as
root bus only report io and mem32.

pci_sun4v f02dbcfc: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x804000000000-0x80400fffffff] (bus address [0x0000-0xfffffff])
pci_bus 0000:00: root bus resource [mem 0x800000000000-0x80007effffff] (bus address [0x00000000-0x7effffff])
pci_bus 0000:00: root bus resource [bus 00-77]

Add mem64 handling in pci_common for sparc, so we can have 64bit resource
registered for root bus at first.

After patch, will have:
pci_sun4v f02dbcfc: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x804000000000-0x80400fffffff] (bus address [0x0000-0xfffffff])
pci_bus 0000:00: root bus resource [mem 0x800000000000-0x80007effffff] (bus address [0x00000000-0x7effffff])
pci_bus 0000:00: root bus resource [mem 0x800100000000-0x8007ffffffff] (bus address [0x100000000-0x7ffffffff])
pci_bus 0000:00: root bus resource [bus 00-77]

-v2: mem64_space should use mem_space.start as offset.
-v3: add IORESOURCE_MEM_64 flag
-v4: set name for mem64_space, otherwise /proc/iomem will have <bad> for name

Fixes: commit d63e2e1f3df9 ("sparc/PCI: Clip bridge windows to fit in upstream windows")
Link: http://lkml.kernel.org/r/CAE9FiQU1gJY1LYrxs+ma5LCTEEe4xmtjRG0aXJ9K_Tsu+m9Wuw@mail.gmail.com
Reported-by: David Ahern <david.ahern@oracle.com>
Tested-by: David Ahern <david.ahern@oracle.com>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>

 arch/sparc/kernel/pci.c        |    7 ++++++-
 arch/sparc/kernel/pci_common.c |   17 +++++++++++++++--
 arch/sparc/kernel/pci_impl.h   |    1 +
 3 files changed, 22 insertions(+), 3 deletions(-)
---
 arch/sparc/kernel/pci.c        |  7 ++++++-
 arch/sparc/kernel/pci_common.c | 17 +++++++++++++++--
 arch/sparc/kernel/pci_impl.h   |  1 +
 3 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index b91d7f1..badf095 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -185,8 +185,10 @@ static unsigned long pci_parse_of_flags(u32 addr0)
 
 	if (addr0 & 0x02000000) {
 		flags = IORESOURCE_MEM | PCI_BASE_ADDRESS_SPACE_MEMORY;
-		flags |= (addr0 >> 22) & PCI_BASE_ADDRESS_MEM_TYPE_64;
 		flags |= (addr0 >> 28) & PCI_BASE_ADDRESS_MEM_TYPE_1M;
+		if (addr0 & 0x01000000)
+			flags |= IORESOURCE_MEM_64
+				 | PCI_BASE_ADDRESS_MEM_TYPE_64;
 		if (addr0 & 0x40000000)
 			flags |= IORESOURCE_PREFETCH
 				 | PCI_BASE_ADDRESS_MEM_PREFETCH;
@@ -655,6 +657,9 @@ struct pci_bus *pci_scan_one_pbm(struct pci_pbm_info *pbm,
 				pbm->io_space.start);
 	pci_add_resource_offset(&resources, &pbm->mem_space,
 				pbm->mem_space.start);
+	if (pbm->mem64_space.flags)
+		pci_add_resource_offset(&resources, &pbm->mem64_space,
+					pbm->mem_space.start);
 	pbm->busn.start = pbm->pci_first_busno;
 	pbm->busn.end	= pbm->pci_last_busno;
 	pbm->busn.flags	= IORESOURCE_BUS;
diff --git a/arch/sparc/kernel/pci_common.c b/arch/sparc/kernel/pci_common.c
index 944a065..33524c1 100644
--- a/arch/sparc/kernel/pci_common.c
+++ b/arch/sparc/kernel/pci_common.c
@@ -406,6 +406,7 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 	}
 
 	num_pbm_ranges = i / sizeof(*pbm_ranges);
+	memset(&pbm->mem64_space, 0, sizeof(struct resource));
 
 	for (i = 0; i < num_pbm_ranges; i++) {
 		const struct linux_prom_pci_ranges *pr = &pbm_ranges[i];
@@ -451,7 +452,12 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 			break;
 
 		case 3:
-			/* XXX 64-bit MEM handling XXX */
+			/* 64-bit MEM handling */
+			pbm->mem64_space.start = a;
+			pbm->mem64_space.end = a + size - 1UL;
+			pbm->mem64_space.flags = IORESOURCE_MEM;
+			saw_mem = 1;
+			break;
 
 		default:
 			break;
@@ -465,15 +471,22 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 		prom_halt();
 	}
 
-	printk("%s: PCI IO[%llx] MEM[%llx]\n",
+	printk("%s: PCI IO[%llx] MEM[%llx]",
 	       pbm->name,
 	       pbm->io_space.start,
 	       pbm->mem_space.start);
+	if (pbm->mem64_space.flags)
+		printk(" MEM64[%llx]",
+		       pbm->mem64_space.start);
+	printk("\n");
 
 	pbm->io_space.name = pbm->mem_space.name = pbm->name;
+	pbm->mem64_space.name = pbm->name;
 
 	request_resource(&ioport_resource, &pbm->io_space);
 	request_resource(&iomem_resource, &pbm->mem_space);
+	if (pbm->mem64_space.flags)
+		request_resource(&iomem_resource, &pbm->mem64_space);
 
 	pci_register_legacy_regions(&pbm->io_space,
 				    &pbm->mem_space);
diff --git a/arch/sparc/kernel/pci_impl.h b/arch/sparc/kernel/pci_impl.h
index 75803c7..37222ca 100644
--- a/arch/sparc/kernel/pci_impl.h
+++ b/arch/sparc/kernel/pci_impl.h
@@ -97,6 +97,7 @@ struct pci_pbm_info {
 	/* PBM I/O and Memory space resources. */
 	struct resource			io_space;
 	struct resource			mem_space;
+	struct resource			mem64_space;
 	struct resource			busn;
 
 	/* Base of PCI Config space, can be per-PBM or shared. */
-- 
1.8.4.5

[PATCH v8 02/61] PCI: Add pci_find_root_bus_resource()

[Yinghai Lu]

Add pci_find_root_bus_resource() to return root bus res
for resource.

For sparc mem64 support, We need that host bridge window res,
  1. we need direct parent for request_resource_conflict() calling in
      pci_register_legacy_regions().
  2. check if return is NULL to decide if region is valid or not.

Convert old pci_find_parent_resource() to pci_find_bus_resource(),
and reuse it in pci_find_root_bus_resource().
So we avoid changing pcibios_bus_to_resource().

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/pci.c   | 36 +++++++++++++++++++++++++-----------
 include/linux/pci.h |  2 ++
 2 files changed, 27 insertions(+), 11 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 6a9a111..613eef1 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -414,18 +414,9 @@ int pci_find_ht_capability(struct pci_dev *dev, int ht_cap)
 }
 EXPORT_SYMBOL_GPL(pci_find_ht_capability);
 
-/**
- * pci_find_parent_resource - return resource region of parent bus of given region
- * @dev: PCI device structure contains resources to be searched
- * @res: child resource record for which parent is sought
- *
- *  For given resource region of given device, return the resource
- *  region of parent bus the given region is contained in.
- */
-struct resource *pci_find_parent_resource(const struct pci_dev *dev,
-					  struct resource *res)
+static struct resource *pci_find_bus_resource(const struct pci_bus *bus,
+					      struct resource *res)
 {
-	const struct pci_bus *bus = dev->bus;
 	struct resource *r;
 	int i;
 
@@ -455,8 +446,31 @@ struct resource *pci_find_parent_resource(const struct pci_dev *dev,
 	}
 	return NULL;
 }
+
+/**
+ * pci_find_parent_resource - return resource region of parent bus of given region
+ * @dev: PCI device structure contains resources to be searched
+ * @res: child resource record for which parent is sought
+ *
+ *  For given resource region of given device, return the resource
+ *  region of parent bus the given region is contained in.
+ */
+struct resource *pci_find_parent_resource(const struct pci_dev *dev,
+					  struct resource *res)
+{
+	return pci_find_bus_resource(dev->bus, res);
+}
 EXPORT_SYMBOL(pci_find_parent_resource);
 
+struct resource *pci_find_root_bus_resource(struct pci_bus *bus,
+					    struct resource *res)
+{
+	while (bus->parent)
+		bus = bus->parent;
+
+	return pci_find_bus_resource(bus, res);
+}
+
 /**
  * pci_wait_for_pending - wait for @mask bit(s) to clear in status word @pos
  * @dev: the PCI device to operate on
diff --git a/include/linux/pci.h b/include/linux/pci.h
index e90eb22..05f5eac 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -783,6 +783,8 @@ void pcibios_resource_to_bus(struct pci_bus *bus, struct pci_bus_region *region,
 			     struct resource *res);
 void pcibios_bus_to_resource(struct pci_bus *bus, struct resource *res,
 			     struct pci_bus_region *region);
+struct resource *pci_find_root_bus_resource(struct pci_bus *bus,
+					    struct resource *res);
 void pcibios_scan_specific_bus(int busn);
 struct pci_bus *pci_find_bus(int domain, int busnr);
 void pci_bus_add_devices(const struct pci_bus *bus);
-- 
1.8.4.5

[PATCH v8 03/61] sparc/PCI: Use correct bus address to resource offset

[Yinghai Lu]

After we add 64bit mmio parsing, we got some "no compatible bridge window"
warning on anther new model that support 64bit resource.

It turns out that we can not use mem_space.start as 64bit mem space
offset, aka mem_space.start != offset.

Use child_phys_addr to calculate exact offset and recorde offset in
pbm.

After patch we get correct offset.

/pci@305: PCI IO [io  0x2007e00000000-0x2007e0fffffff] offset 2007e00000000
/pci@305: PCI MEM [mem 0x2000000100000-0x200007effffff] offset 2000000000000
/pci@305: PCI MEM64 [mem 0x2000100000000-0x2000dffffffff] offset 2000000000000
...
pci_sun4v f02ae7f8: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x2007e00000000-0x2007e0fffffff] (bus address [0x0000-0xfffffff])
pci_bus 0000:00: root bus resource [mem 0x2000000100000-0x200007effffff] (bus address [0x00100000-0x7effffff])
pci_bus 0000:00: root bus resource [mem 0x2000100000000-0x2000dffffffff] (bus address [0x100000000-0xdffffffff])

-v2: to make it simple, do not add mem64_offset, and assume
     mem64_offset == mem_offset even that is not same as mem_space.start,
     otherwise would make pci_mmap_resource() path too complicated.

-v3: put back mem64_offset, as we found T4 has mem_offset != mem64_offset
     check overlapping between mem64_space and mem_space.

-v5: use pcibios_bus_to_region() requested by Bjorn.
     use pci_find_root_bus_resource().

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
---
 arch/sparc/kernel/pci.c        | 50 +++++++++++++++++++-----------------------
 arch/sparc/kernel/pci_common.c | 32 ++++++++++++++++++++-------
 arch/sparc/kernel/pci_impl.h   |  4 ++++
 3 files changed, 50 insertions(+), 36 deletions(-)

diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index badf095..269630a 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -654,12 +654,12 @@ struct pci_bus *pci_scan_one_pbm(struct pci_pbm_info *pbm,
 	printk("PCI: Scanning PBM %s\n", node->full_name);
 
 	pci_add_resource_offset(&resources, &pbm->io_space,
-				pbm->io_space.start);
+				pbm->io_offset);
 	pci_add_resource_offset(&resources, &pbm->mem_space,
-				pbm->mem_space.start);
+				pbm->mem_offset);
 	if (pbm->mem64_space.flags)
 		pci_add_resource_offset(&resources, &pbm->mem64_space,
-					pbm->mem_space.start);
+					pbm->mem64_offset);
 	pbm->busn.start = pbm->pci_first_busno;
 	pbm->busn.end	= pbm->pci_last_busno;
 	pbm->busn.flags	= IORESOURCE_BUS;
@@ -733,30 +733,28 @@ int pcibios_enable_device(struct pci_dev *dev, int mask)
 static int __pci_mmap_make_offset_bus(struct pci_dev *pdev, struct vm_area_struct *vma,
 				      enum pci_mmap_state mmap_state)
 {
-	struct pci_pbm_info *pbm = pdev->dev.archdata.host_controller;
-	unsigned long space_size, user_offset, user_size;
-
-	if (mmap_state == pci_mmap_io) {
-		space_size = resource_size(&pbm->io_space);
-	} else {
-		space_size = resource_size(&pbm->mem_space);
-	}
+	unsigned long user_offset, user_size;
+	struct resource res, *root_bus_res;
+	struct pci_bus_region region;
 
 	/* Make sure the request is in range. */
 	user_offset = vma->vm_pgoff << PAGE_SHIFT;
 	user_size = vma->vm_end - vma->vm_start;
 
-	if (user_offset >= space_size ||
-	    (user_offset + user_size) > space_size)
+	region.start = user_offset;
+	region.end = user_offset + user_size - 1;
+	memset(&res, 0, sizeof(res));
+	if (mmap_state == pci_mmap_io)
+		res.flags = IORESOURCE_IO;
+	else
+		res.flags = IORESOURCE_MEM;
+
+	pcibios_bus_to_resource(pdev->bus, &res, &region);
+	root_bus_res = pci_find_root_bus_resource(pdev->bus, &res);
+	if (!root_bus_res)
 		return -EINVAL;
 
-	if (mmap_state == pci_mmap_io) {
-		vma->vm_pgoff = (pbm->io_space.start +
-				 user_offset) >> PAGE_SHIFT;
-	} else {
-		vma->vm_pgoff = (pbm->mem_space.start +
-				 user_offset) >> PAGE_SHIFT;
-	}
+	vma->vm_pgoff = res.start >> PAGE_SHIFT;
 
 	return 0;
 }
@@ -977,16 +975,12 @@ void pci_resource_to_user(const struct pci_dev *pdev, int bar,
 			  const struct resource *rp, resource_size_t *start,
 			  resource_size_t *end)
 {
-	struct pci_pbm_info *pbm = pdev->dev.archdata.host_controller;
-	unsigned long offset;
+	struct pci_bus_region region;
 
-	if (rp->flags & IORESOURCE_IO)
-		offset = pbm->io_space.start;
-	else
-		offset = pbm->mem_space.start;
+	pcibios_resource_to_bus(pdev->bus, &region, (struct resource *)rp);
 
-	*start = rp->start - offset;
-	*end = rp->end - offset;
+	*start = region.start;
+	*end = region.end;
 }
 
 void pcibios_set_master(struct pci_dev *dev)
diff --git a/arch/sparc/kernel/pci_common.c b/arch/sparc/kernel/pci_common.c
index 33524c1..76998f8 100644
--- a/arch/sparc/kernel/pci_common.c
+++ b/arch/sparc/kernel/pci_common.c
@@ -410,13 +410,16 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 
 	for (i = 0; i < num_pbm_ranges; i++) {
 		const struct linux_prom_pci_ranges *pr = &pbm_ranges[i];
-		unsigned long a, size;
+		unsigned long a, size, region_a;
 		u32 parent_phys_hi, parent_phys_lo;
+		u32 child_phys_mid, child_phys_lo;
 		u32 size_hi, size_lo;
 		int type;
 
 		parent_phys_hi = pr->parent_phys_hi;
 		parent_phys_lo = pr->parent_phys_lo;
+		child_phys_mid = pr->child_phys_mid;
+		child_phys_lo = pr->child_phys_lo;
 		if (tlb_type == hypervisor)
 			parent_phys_hi &= 0x0fffffff;
 
@@ -426,6 +429,8 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 		type = (pr->child_phys_hi >> 24) & 0x3;
 		a = (((unsigned long)parent_phys_hi << 32UL) |
 		     ((unsigned long)parent_phys_lo  <<  0UL));
+		region_a = (((unsigned long)child_phys_mid << 32UL) |
+		     ((unsigned long)child_phys_lo  <<  0UL));
 		size = (((unsigned long)size_hi << 32UL) |
 			((unsigned long)size_lo  <<  0UL));
 
@@ -440,6 +445,7 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 			pbm->io_space.start = a;
 			pbm->io_space.end = a + size - 1UL;
 			pbm->io_space.flags = IORESOURCE_IO;
+			pbm->io_offset = a - region_a;
 			saw_io = 1;
 			break;
 
@@ -448,6 +454,7 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 			pbm->mem_space.start = a;
 			pbm->mem_space.end = a + size - 1UL;
 			pbm->mem_space.flags = IORESOURCE_MEM;
+			pbm->mem_offset = a - region_a;
 			saw_mem = 1;
 			break;
 
@@ -456,6 +463,7 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 			pbm->mem64_space.start = a;
 			pbm->mem64_space.end = a + size - 1UL;
 			pbm->mem64_space.flags = IORESOURCE_MEM;
+			pbm->mem64_offset = a - region_a;
 			saw_mem = 1;
 			break;
 
@@ -471,14 +479,22 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 		prom_halt();
 	}
 
-	printk("%s: PCI IO[%llx] MEM[%llx]",
-	       pbm->name,
-	       pbm->io_space.start,
-	       pbm->mem_space.start);
+	if (pbm->io_space.flags)
+		printk("%s: PCI IO %pR offset %llx\n",
+		       pbm->name, &pbm->io_space, pbm->io_offset);
+	if (pbm->mem_space.flags)
+		printk("%s: PCI MEM %pR offset %llx\n",
+		       pbm->name, &pbm->mem_space, pbm->mem_offset);
+	if (pbm->mem64_space.flags && pbm->mem_space.flags) {
+		if (pbm->mem64_space.start <= pbm->mem_space.end)
+			pbm->mem64_space.start = pbm->mem_space.end + 1;
+		if (pbm->mem64_space.start > pbm->mem64_space.end)
+			pbm->mem64_space.flags = 0;
+	}
+
 	if (pbm->mem64_space.flags)
-		printk(" MEM64[%llx]",
-		       pbm->mem64_space.start);
-	printk("\n");
+		printk("%s: PCI MEM64 %pR offset %llx\n",
+		       pbm->name, &pbm->mem64_space, pbm->mem64_offset);
 
 	pbm->io_space.name = pbm->mem_space.name = pbm->name;
 	pbm->mem64_space.name = pbm->name;
diff --git a/arch/sparc/kernel/pci_impl.h b/arch/sparc/kernel/pci_impl.h
index 37222ca..2853af7 100644
--- a/arch/sparc/kernel/pci_impl.h
+++ b/arch/sparc/kernel/pci_impl.h
@@ -99,6 +99,10 @@ struct pci_pbm_info {
 	struct resource			mem_space;
 	struct resource			mem64_space;
 	struct resource			busn;
+	/* offset */
+	resource_size_t			io_offset;
+	resource_size_t			mem_offset;
+	resource_size_t			mem64_offset;
 
 	/* Base of PCI Config space, can be per-PBM or shared. */
 	unsigned long			config_space;
-- 
1.8.4.5

[PATCH v8 04/61] sparc/PCI: Reserve legacy mmio after PCI mmio

[Yinghai Lu]

On one system found bunch of claim resource fail from pci device.
pci_sun4v f02b894c: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x2007e00000000-0x2007e0fffffff] (bus address [0x0000-0xfffffff])
pci_bus 0000:00: root bus resource [mem 0x2000000000000-0x200007effffff] (bus address [0x00000000-0x7effffff])
pci_bus 0000:00: root bus resource [mem 0x2000100000000-0x20007ffffffff] (bus address [0x100000000-0x7ffffffff])
...
PCI: Claiming 0000:00:02.0: Resource 14: 0002000000000000..00020000004fffff [200]
pci 0000:00:02.0: can't claim BAR 14 [mem 0x2000000000000-0x20000004fffff]: address conflict with Video RAM area [??? 0x20000000a0000-0x20000000bffff flags 0x80000000]
pci 0000:02:00.0: can't claim BAR 0 [mem 0x2000000000000-0x20000000fffff]: no compatible bridge window
PCI: Claiming 0000:02:00.0: Resource 3: 0002000000100000..0002000000103fff [200]
pci 0000:02:00.0: can't claim BAR 3 [mem 0x2000000100000-0x2000000103fff]: no compatible bridge window
PCI: Claiming 0000:02:00.1: Resource 0: 0002000000200000..00020000002fffff [200]
pci 0000:02:00.1: can't claim BAR 0 [mem 0x2000000200000-0x20000002fffff]: no compatible bridge window
PCI: Claiming 0000:02:00.1: Resource 3: 0002000000104000..0002000000107fff [200]
pci 0000:02:00.1: can't claim BAR 3 [mem 0x2000000104000-0x2000000107fff]: no compatible bridge window
PCI: Claiming 0000:02:00.2: Resource 0: 0002000000300000..00020000003fffff [200]
pci 0000:02:00.2: can't claim BAR 0 [mem 0x2000000300000-0x20000003fffff]: no compatible bridge window
PCI: Claiming 0000:02:00.2: Resource 3: 0002000000108000..000200000010bfff [200]
pci 0000:02:00.2: can't claim BAR 3 [mem 0x2000000108000-0x200000010bfff]: no compatible bridge window
PCI: Claiming 0000:02:00.3: Resource 0: 0002000000400000..00020000004fffff [200]
pci 0000:02:00.3: can't claim BAR 0 [mem 0x2000000400000-0x20000004fffff]: no compatible bridge window
PCI: Claiming 0000:02:00.3: Resource 3: 000200000010c000..000200000010ffff [200]
pci 0000:02:00.3: can't claim BAR 3 [mem 0x200000010c000-0x200000010ffff]: no compatible bridge window

The bridge 00:02.0 resource does not get reserved as Video RAM take the position early,
and following children all failed.

Move down Video RAM area reservation after pci mmio get reserved,
so we leave pci driver to use those regions.

-v5: merge simplify one and use pcibios_bus_to_resource()
     and pci_find_root_bus_resource()

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
---
 arch/sparc/kernel/pci.c        |  1 +
 arch/sparc/kernel/pci_common.c | 59 ++++++++++++++++++++++--------------------
 arch/sparc/kernel/pci_impl.h   |  1 +
 3 files changed, 33 insertions(+), 28 deletions(-)

diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index 269630a..0e82584 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -677,6 +677,7 @@ struct pci_bus *pci_scan_one_pbm(struct pci_pbm_info *pbm,
 	pci_bus_register_of_sysfs(bus);
 
 	pci_claim_bus_resources(bus);
+	pci_register_legacy_regions(bus);
 	pci_bus_add_devices(bus);
 	return bus;
 }
diff --git a/arch/sparc/kernel/pci_common.c b/arch/sparc/kernel/pci_common.c
index 76998f8..09380a5 100644
--- a/arch/sparc/kernel/pci_common.c
+++ b/arch/sparc/kernel/pci_common.c
@@ -328,41 +328,46 @@ void pci_get_pbm_props(struct pci_pbm_info *pbm)
 	}
 }
 
-static void pci_register_legacy_regions(struct resource *io_res,
-					struct resource *mem_res)
+static void pci_register_region(struct pci_bus *bus, const char *name,
+				resource_size_t rstart, resource_size_t size)
 {
-	struct resource *p;
+	struct resource *res, *conflict, *root_bus_res;
+	struct pci_bus_region region;
 
-	/* VGA Video RAM. */
-	p = kzalloc(sizeof(*p), GFP_KERNEL);
-	if (!p)
+	res = kzalloc(sizeof(*res), GFP_KERNEL);
+	if (!res)
 		return;
 
-	p->name = "Video RAM area";
-	p->start = mem_res->start + 0xa0000UL;
-	p->end = p->start + 0x1ffffUL;
-	p->flags = IORESOURCE_BUSY;
-	request_resource(mem_res, p);
+	res->flags = IORESOURCE_MEM;
 
-	p = kzalloc(sizeof(*p), GFP_KERNEL);
-	if (!p)
+	region.start = rstart;
+	region.end = rstart + size - 1UL;
+	pcibios_bus_to_resource(bus, res, &region);
+	root_bus_res = pci_find_root_bus_resource(bus, res);
+	if (!root_bus_res) {
+		kfree(res);
 		return;
+	}
+
+	res->name = name;
+	res->flags |= IORESOURCE_BUSY;
+	conflict = request_resource_conflict(root_bus_res, res);
+	if (conflict) {
+		dev_printk(KERN_DEBUG, &bus->dev,
+			" can't claim %s %pR: address conflict with %s %pR\n",
+			res->name, res, conflict->name, conflict);
+		kfree(res);
+	}
+}
 
-	p->name = "System ROM";
-	p->start = mem_res->start + 0xf0000UL;
-	p->end = p->start + 0xffffUL;
-	p->flags = IORESOURCE_BUSY;
-	request_resource(mem_res, p);
+void pci_register_legacy_regions(struct pci_bus *bus)
+{
+	/* VGA Video RAM. */
+	pci_register_region(bus, "Video RAM area", 0xa0000UL, 0x20000UL);
 
-	p = kzalloc(sizeof(*p), GFP_KERNEL);
-	if (!p)
-		return;
+	pci_register_region(bus, "System ROM",     0xf0000UL, 0x10000UL);
 
-	p->name = "Video ROM";
-	p->start = mem_res->start + 0xc0000UL;
-	p->end = p->start + 0x7fffUL;
-	p->flags = IORESOURCE_BUSY;
-	request_resource(mem_res, p);
+	pci_register_region(bus, "Video ROM",      0xc0000UL,  0x8000UL);
 }
 
 static void pci_register_iommu_region(struct pci_pbm_info *pbm)
@@ -504,8 +509,6 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 	if (pbm->mem64_space.flags)
 		request_resource(&iomem_resource, &pbm->mem64_space);
 
-	pci_register_legacy_regions(&pbm->io_space,
-				    &pbm->mem_space);
 	pci_register_iommu_region(pbm);
 }
 
diff --git a/arch/sparc/kernel/pci_impl.h b/arch/sparc/kernel/pci_impl.h
index 2853af7..ff8f5e1 100644
--- a/arch/sparc/kernel/pci_impl.h
+++ b/arch/sparc/kernel/pci_impl.h
@@ -167,6 +167,7 @@ void pci_get_pbm_props(struct pci_pbm_info *pbm);
 struct pci_bus *pci_scan_one_pbm(struct pci_pbm_info *pbm,
 				 struct device *parent);
 void pci_determine_mem_io_space(struct pci_pbm_info *pbm);
+void pci_register_legacy_regions(struct pci_bus *bus);
 
 /* Error reporting support. */
 void pci_scan_for_target_abort(struct pci_pbm_info *, struct pci_bus *);
-- 
1.8.4.5

[PATCH v8 05/61] sparc/PCI: Add IORESOURCE_MEM_64 for 64-bit resource in OF parsing

[Yinghai Lu]

For device resource PREF bit setting under bridge 64-bit pref resource,
we need to make sure only set PREF for 64bit resource, so set
IORESOUCE_MEM_64 for 64bit resource during of device resource flags
parsing.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=96261
Link: https://bugzilla.kernel.org/show_bug.cgi?id=96241
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: sparclinux@vger.kernel.org
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
---
 arch/sparc/kernel/of_device_32.c | 5 +++--
 arch/sparc/kernel/of_device_64.c | 5 +++--
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/arch/sparc/kernel/of_device_32.c b/arch/sparc/kernel/of_device_32.c
index 185aa96..3e9f273 100644
--- a/arch/sparc/kernel/of_device_32.c
+++ b/arch/sparc/kernel/of_device_32.c
@@ -83,11 +83,12 @@ static unsigned long of_bus_pci_get_flags(const u32 *addr, unsigned long flags)
 	case 0x01:
 		flags |= IORESOURCE_IO;
 		break;
-
 	case 0x02: /* 32 bits */
-	case 0x03: /* 64 bits */
 		flags |= IORESOURCE_MEM;
 		break;
+	case 0x03: /* 64 bits */
+		flags |= IORESOURCE_MEM | IORESOURCE_MEM_64;
+		break;
 	}
 	if (w & 0x40000000)
 		flags |= IORESOURCE_PREFETCH;
diff --git a/arch/sparc/kernel/of_device_64.c b/arch/sparc/kernel/of_device_64.c
index 7bbdc26..defee61 100644
--- a/arch/sparc/kernel/of_device_64.c
+++ b/arch/sparc/kernel/of_device_64.c
@@ -146,11 +146,12 @@ static unsigned long of_bus_pci_get_flags(const u32 *addr, unsigned long flags)
 	case 0x01:
 		flags |= IORESOURCE_IO;
 		break;
-
 	case 0x02: /* 32 bits */
-	case 0x03: /* 64 bits */
 		flags |= IORESOURCE_MEM;
 		break;
+	case 0x03: /* 64 bits */
+		flags |= IORESOURCE_MEM | IORESOURCE_MEM_64;
+		break;
 	}
 	if (w & 0x40000000)
 		flags |= IORESOURCE_PREFETCH;
-- 
1.8.4.5

[PATCH v8 05/61] sparc/PCI: Add IORESOURCE_MEM_64 for 64-bit resource in OF parsing

[Yinghai Lu]

For device resource PREF bit setting under bridge 64-bit pref resource,
we need to make sure only set PREF for 64bit resource, so set
IORESOUCE_MEM_64 for 64bit resource during of device resource flags
parsing.

Link: https://bugzilla.kernel.org/show_bug.cgi?id–261
Link: https://bugzilla.kernel.org/show_bug.cgi?id–241
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: sparclinux@vger.kernel.org
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
---
 arch/sparc/kernel/of_device_32.c | 5 +++--
 arch/sparc/kernel/of_device_64.c | 5 +++--
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/arch/sparc/kernel/of_device_32.c b/arch/sparc/kernel/of_device_32.c
index 185aa96..3e9f273 100644
--- a/arch/sparc/kernel/of_device_32.c
+++ b/arch/sparc/kernel/of_device_32.c
@@ -83,11 +83,12 @@ static unsigned long of_bus_pci_get_flags(const u32 *addr, unsigned long flags)
 	case 0x01:
 		flags |= IORESOURCE_IO;
 		break;
-
 	case 0x02: /* 32 bits */
-	case 0x03: /* 64 bits */
 		flags |= IORESOURCE_MEM;
 		break;
+	case 0x03: /* 64 bits */
+		flags |= IORESOURCE_MEM | IORESOURCE_MEM_64;
+		break;
 	}
 	if (w & 0x40000000)
 		flags |= IORESOURCE_PREFETCH;
diff --git a/arch/sparc/kernel/of_device_64.c b/arch/sparc/kernel/of_device_64.c
index 7bbdc26..defee61 100644
--- a/arch/sparc/kernel/of_device_64.c
+++ b/arch/sparc/kernel/of_device_64.c
@@ -146,11 +146,12 @@ static unsigned long of_bus_pci_get_flags(const u32 *addr, unsigned long flags)
 	case 0x01:
 		flags |= IORESOURCE_IO;
 		break;
-
 	case 0x02: /* 32 bits */
-	case 0x03: /* 64 bits */
 		flags |= IORESOURCE_MEM;
 		break;
+	case 0x03: /* 64 bits */
+		flags |= IORESOURCE_MEM | IORESOURCE_MEM_64;
+		break;
 	}
 	if (w & 0x40000000)
 		flags |= IORESOURCE_PREFETCH;
-- 
1.8.4.5

[PATCH v8 06/61] sparc/PCI: Keep resource idx order with bridge register number

[Yinghai Lu]

On one system found strang "no compatible bridge window" warning

PCI: Claiming 0000:00:01.0: Resource 14: 0002000100000000..000200010fffffff [10220c]
PCI: Claiming 0000:01:00.0: Resource 1: 0002000100000000..000200010000ffff [100214]
pci 0000:01:00.0: can't claim BAR 1 [mem 0x2000100000000-0x200010000ffff 64bit]: no compatible bridge window

and we already had pref_compat support that add extra pref bit for device
resource.

It turns out that pci_resource_compatible()/pci_up_path_over_pref_mem64()
just check resource with bridge pref mmio register idx 15, and we have put
resource to use mmio register idx 14 during of_scan_pci_bridge()
as the bridge does not mmio resource.

We already fix pci_up_path_over_pref_mem64() to check all bus resources.

And at the same time, this patch will make resource to consistent sequence
like other arch or directly from pci_read_bridge_bases(),
even non-pref mmio is missing, or out of ordering in firmware reporting.

So hold i = 1 for non pref mmio, and i = 2 for pref mmio.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
---
 arch/sparc/kernel/pci.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index 0e82584..36a0dfb 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -472,7 +472,7 @@ static void of_scan_pci_bridge(struct pci_pbm_info *pbm,
 		pci_read_bridge_bases(bus);
 		goto after_ranges;
 	}
-	i = 1;
+	i = 3;
 	for (; len >= 32; len -= 32, ranges += 8) {
 		u64 start;
 
@@ -504,6 +504,12 @@ static void of_scan_pci_bridge(struct pci_pbm_info *pbm,
 				       " for bridge %s\n", node->full_name);
 				continue;
 			}
+		} else if ((flags & IORESOURCE_PREFETCH) &&
+			   !bus->resource[2]->flags) {
+			res = bus->resource[2];
+		} else if (((flags & (IORESOURCE_MEM | IORESOURCE_PREFETCH)) ==
+			    IORESOURCE_MEM) && !bus->resource[1]->flags) {
+			res = bus->resource[1];
 		} else {
 			if (i >= PCI_NUM_RESOURCES - PCI_BRIDGE_RESOURCES) {
 				printk(KERN_ERR "PCI: too many memory ranges"
-- 
1.8.4.5

[PATCH v8 07/61] PCI: Kill wrong quirk about M7101

[Yinghai Lu]

Meelis reported that qla2000 driver does not get loaded on one sparc system.

schizo f00732d0: PCI host bridge to bus 0001:00
pci_bus 0001:00: root bus resource [io  0x7fe01000000-0x7fe01ffffff] (bus address [0x0000-0xffffff])
pci 0001:00:06.0: quirk: [io  0x7fe01000800-0x7fe0100083f] claimed by ali7101 ACPI
pci 0001:00:06.0: quirk: [io  0x7fe01000600-0x7fe0100061f] claimed by ali7101 SMB
pci 0001:00:07.0: can't claim BAR 0 [io  0x7fe01000000-0x7fe0100ffff]: address conflict with 0001:00:06.0 [io  0x7fe01000600-0x7fe0100061f]

So the quirk for M7101 claim the io range early.

According to spec with M7101 in M1543 page 103/104,
	http://www.versalogic.com/Support/Downloads/pdf/ali1543.pdf
0xe0, and 0xe2 do not include address info for acpi/smb.

Kill wrong quirk about them.

Link: http://kodu.ut.ee/~mroos/dm/dm.v240
Link: http://kodu.ut.ee/~mroos/dm/dm.sb100
Reported-by: Meelis Roos <mroos@linux.ee>
Cc: Meelis Roos <mroos@linux.ee>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
---
 drivers/pci/quirks.c | 18 ------------------
 1 file changed, 18 deletions(-)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index b03373f..55bb661 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -433,24 +433,6 @@ static void quirk_amd_nl_class(struct pci_dev *pdev)
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_NL_USB,
 		quirk_amd_nl_class);
 
-/*
- * Let's make the southbridge information explicit instead
- * of having to worry about people probing the ACPI areas,
- * for example.. (Yes, it happens, and if you read the wrong
- * ACPI register it will put the machine to sleep with no
- * way of waking it up again. Bummer).
- *
- * ALI M7101: Two IO regions pointed to by words at
- *	0xE0 (64 bytes of ACPI registers)
- *	0xE2 (32 bytes of SMB registers)
- */
-static void quirk_ali7101_acpi(struct pci_dev *dev)
-{
-	quirk_io_region(dev, 0xE0, 64, PCI_BRIDGE_RESOURCES, "ali7101 ACPI");
-	quirk_io_region(dev, 0xE2, 32, PCI_BRIDGE_RESOURCES+1, "ali7101 SMB");
-}
-DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AL,	PCI_DEVICE_ID_AL_M7101,		quirk_ali7101_acpi);
-
 static void piix4_io_quirk(struct pci_dev *dev, const char *name, unsigned int port, unsigned int enable)
 {
 	u32 devres;
-- 
1.8.4.5

[PATCH v8 08/61] PCI: Ignore BAR for ALi M1533 PCI-ISA bridge

[Yinghai Lu]

Meelis reported strange conflicts on sparc v210:
pci 0001:00:07.0: can't claim BAR 2 [mem 0x7ff00000000-0x7ff000fffff]: address conflict with 0001:00:07.0 [mem 0x7ff00000000-0x7ff000fffff]
pci 0001:00:06.0: can't claim BAR 0 [io 0x7fe01000000-0x7fe0100000f]: address conflict with 0001:00:07.0 [io 0x7fe01000000-0x7fe0100ffff]
pci 0001:00:0d.0: can't claim BAR 0 [io 0x7fe01000900-0x7fe01000907]: address conflict with 0001:00:07.0 [io 0x7fe01000000-0x7fe0100ffff]
pci 0001:00:0d.0: can't claim BAR 1 [io 0x7fe01000918-0x7fe0100091f]: address conflict with 0001:00:07.0 [io 0x7fe01000000-0x7fe0100ffff]
pci 0001:00:0d.0: can't claim BAR 2 [io 0x7fe01000910-0x7fe01000917]: address conflict with 0001:00:07.0 [io 0x7fe01000000-0x7fe0100ffff]
pci 0001:00:0d.0: can't claim BAR 3 [io 0x7fe01000908-0x7fe0100090f]: address conflict with 0001:00:07.0 [io 0x7fe01000000-0x7fe0100ffff]
pci 0001:00:0d.0: can't claim BAR 4 [io 0x7fe01000920-0x7fe0100092f]: address conflict with 0001:00:07.0 [io 0x7fe01000000-0x7fe0100ffff]
PCI: /pci@1e,600000 can't claim Video RAM area [mem 0x7ff000a0000-0x7ff000bffff]: address conflict with 0001:00:07.0 [mem 0x7ff00000000-0x7ff000fffff]
PCI: /pci@1e,600000 can't claim System ROM [mem 0x7ff000f0000-0x7ff000fffff]: address conflict with 0001:00:07.0 [mem 0x7ff00000000-0x7ff000fffff]
PCI: /pci@1e,600000 can't claim Video ROM [mem 0x7ff000c0000-0x7ff000c7fff]: address conflict with 0001:00:07.0 [mem 0x7ff00000000-0x7ff000fffff]

we have
pci_bus 0001:00: root bus resource [io  0x7fe01000000-0x7fe01ffffff] (bus address [0x0000-0xffffff])
pci_bus 0001:00: root bus resource [mem 0x7ff00000000-0x7ffffffffff] (bus address [0x00000000-0xffffffff])

so pci bus address in the bars for 0001:00:07.0 has all 0s.

0001:00:07.0 ISA bridge: ULi Electronics Inc. M1533/M1535/M1543 PCI to ISA Bridge [Aladdin IV/V/V+]
        Region 0: [virtual] I/O ports at 0000 [size=64K]
        Region 1: Memory at <unassigned> (32-bit, non-prefetchable) [size=1M]
        Region 2: Memory at <unassigned> (32-bit, non-prefetchable) [size=1M]
00: b9 10 33 15 0f 00 10 02 00 00 01 06 00 00 00 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 a0 00 00 00 00 00 00 00 00 00 00 00

According to http://www.versalogic.com/Support/Downloads/pdf/ali1543.pdf
page 28: The indices before 40h are read-only.
and we have all 0 from 0x10-0x2f, according to lspci.
So those BAR do not work as regular BAR, just clean flags, and ignore them all
the way include claim and sizing and alloc etc.

Reported-by: Meelis Roos <mroos@linux.ee>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/quirks.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 55bb661..af75580 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -433,6 +433,21 @@ static void quirk_amd_nl_class(struct pci_dev *pdev)
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_NL_USB,
 		quirk_amd_nl_class);
 
+
+/*
+ * ALi m1533 pci to isa bridge does not have BAR according to datasheet,
+ * clear the flags, so we will try to claim them or reallocate res.
+ */
+static void quirk_ali1533(struct pci_dev *dev)
+{
+	int i;
+
+	for (i = 0; i < PCI_ROM_RESOURCE; i++)
+		dev->resource[i].flags = 0;
+}
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_AL, PCI_DEVICE_ID_AL_M1533,
+			 quirk_ali1533);
+
 static void piix4_io_quirk(struct pci_dev *dev, const char *name, unsigned int port, unsigned int enable)
 {
 	u32 devres;
-- 
1.8.4.5

[PATCH v8 09/61] powerpc/PCI: Keep resource idx order with bridge register number

[Yinghai Lu]

Same as sparc version.

Make resource with consistent sequence
like other arch or directly from pci_read_bridge_bases(),
even non-pref mmio is missing, or out of ordering in firmware reporting.

So hold i = 1 for non pref mmio, and i = 2 for pref mmio.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 arch/powerpc/kernel/pci_of_scan.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/pci_of_scan.c b/arch/powerpc/kernel/pci_of_scan.c
index 2e710c1..349ed11 100644
--- a/arch/powerpc/kernel/pci_of_scan.c
+++ b/arch/powerpc/kernel/pci_of_scan.c
@@ -255,7 +255,7 @@ void of_scan_pci_bridge(struct pci_dev *dev)
 		bus->resource[i] = res;
 		++res;
 	}
-	i = 1;
+	i = 3;
 	for (; len >= 32; len -= 32, ranges += 8) {
 		flags = pci_parse_of_flags(of_read_number(ranges, 1), 1);
 		size = of_read_number(&ranges[6], 2);
@@ -268,6 +268,12 @@ void of_scan_pci_bridge(struct pci_dev *dev)
 				       " for bridge %s\n", node->full_name);
 				continue;
 			}
+		} else if ((flags & IORESOURCE_PREFETCH) &&
+			   !bus->resource[2]->flags) {
+			res = bus->resource[2];
+		} else if (((flags & (IORESOURCE_MEM | IORESOURCE_PREFETCH)) ==
+			    IORESOURCE_MEM) && !bus->resource[1]->flags) {
+			res = bus->resource[1];
 		} else {
 			if (i >= PCI_NUM_RESOURCES - PCI_BRIDGE_RESOURCES) {
 				printk(KERN_ERR "PCI: too many memory ranges"
-- 
1.8.4.5

[PATCH v8 10/61] powerpc/PCI: Add IORESOURCE_MEM_64 for 64-bit resource in OF parsing

[Yinghai Lu]

For device resource PREF bit setting under bridge 64-bit pref resource,
we need to make sure only set PREF for 64bit resource, so set
IORESOUCE_MEM_64 for 64bit resource during of device resource flags
parsing.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=96261
Link: https://bugzilla.kernel.org/show_bug.cgi?id=96241
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Gavin Shan <gwshan@linux.vnet.ibm.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Cc: Anton Blanchard <anton@samba.org>
Cc: linuxppc-dev@lists.ozlabs.org
---
 arch/powerpc/kernel/pci_of_scan.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/pci_of_scan.c b/arch/powerpc/kernel/pci_of_scan.c
index 349ed11..d7305d7 100644
--- a/arch/powerpc/kernel/pci_of_scan.c
+++ b/arch/powerpc/kernel/pci_of_scan.c
@@ -44,8 +44,10 @@ static unsigned int pci_parse_of_flags(u32 addr0, int bridge)
 
 	if (addr0 & 0x02000000) {
 		flags = IORESOURCE_MEM | PCI_BASE_ADDRESS_SPACE_MEMORY;
-		flags |= (addr0 >> 22) & PCI_BASE_ADDRESS_MEM_TYPE_64;
 		flags |= (addr0 >> 28) & PCI_BASE_ADDRESS_MEM_TYPE_1M;
+		if (addr0 & 0x01000000)
+			flags |= IORESOURCE_MEM_64
+				 | PCI_BASE_ADDRESS_MEM_TYPE_64;
 		if (addr0 & 0x40000000)
 			flags |= IORESOURCE_PREFETCH
 				 | PCI_BASE_ADDRESS_MEM_PREFETCH;
-- 
1.8.4.5

[PATCH v8 11/61] OF/PCI: Add IORESOURCE_MEM_64 for 64-bit resource

[Yinghai Lu]

For device resource PREF bit setting under bridge 64-bit pref resource,
we need to make sure only set PREF for 64bit resource, so set
IORESOUCE_MEM_64 for 64bit resource during OF device resource flags
parsing.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=96261
Link: https://bugzilla.kernel.org/show_bug.cgi?id=96241
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Rob Herring <robh+dt@kernel.org>
Cc: devicetree@vger.kernel.org
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
---
 drivers/of/address.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/of/address.c b/drivers/of/address.c
index 384574c..3399602 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -128,9 +128,11 @@ static unsigned int of_bus_pci_get_flags(const __be32 *addr)
 		flags |= IORESOURCE_IO;
 		break;
 	case 0x02: /* 32 bits */
-	case 0x03: /* 64 bits */
 		flags |= IORESOURCE_MEM;
 		break;
+	case 0x03: /* 64 bits */
+		flags |= IORESOURCE_MEM | IORESOURCE_MEM_64;
+		break;
 	}
 	if (w & 0x40000000)
 		flags |= IORESOURCE_PREFETCH;
-- 
1.8.4.5

[PATCH v8 12/61] PCI: Check pref compatible bit for mem64 resource of PCIe device

[Yinghai Lu]

We still get "no compatible bridge window" warning on sparc T5-8
after we add support for 64bit resource parsing for root bus.

 PCI: scan_bus[/pci@300/pci@1/pci@0/pci@6] bus no 8
 PCI: Claiming 0000:00:01.0: Resource 15: 0000800100000000..00008004afffffff [220c]
 PCI: Claiming 0000:01:00.0: Resource 15: 0000800100000000..00008004afffffff [220c]
 PCI: Claiming 0000:02:04.0: Resource 15: 0000800100000000..000080012fffffff [220c]
 PCI: Claiming 0000:03:00.0: Resource 15: 0000800100000000..000080012fffffff [220c]
 PCI: Claiming 0000:04:06.0: Resource 14: 0000800100000000..000080010fffffff [220c]
 PCI: Claiming 0000:05:00.0: Resource 0: 0000800100000000..0000800100001fff [204]
 pci 0000:05:00.0: can't claim BAR 0 [mem 0x800100000000-0x800100001fff]: no compatible bridge window

All the bridges 64-bit resource have pref bit, but the device resource does not
have pref set, then we can not find parent for the device resource,
as we can not put non-pref mem under pref mem.

According to pcie spec errta
https://www.pcisig.com/specifications/pciexpress/base2/PCIe_Base_r2.1_Errata_08Jun10.pdf
page 13, in some case it is ok to mark some as pref.

Mark if the entire path from the host to the adapter is over PCI Express.
Then set pref compatible bit for claim/sizing/assign for 64bit mem resource
on that pcie device.

-v2: set pref for mmio 64 when whole path is PCI Express, according to David Miller.
-v3: don't set pref directly, change to UNDER_PREF, and set PREF before
     sizing and assign resource, and cleart PREF afterwards. requested by BenH.
-v4: use on_all_pcie_path device flag instead.
-v5: update after pci_find_root_bus_resource() change

Fixes: commit d63e2e1f3df9 ("sparc/PCI: Clip bridge windows to fit in upstream windows")
Link: http://lkml.kernel.org/r/CAE9FiQU1gJY1LYrxs+ma5LCTEEe4xmtjRG0aXJ9K_Tsu+m9Wuw@mail.gmail.com
Reported-by: David Ahern <david.ahern@oracle.com>
Tested-by: David Ahern <david.ahern@oracle.com>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
Tested-by: TJ <linux@iam.tj>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
---
 drivers/pci/pci.c       | 10 ++++++----
 drivers/pci/pci.h       |  2 ++
 drivers/pci/probe.c     | 33 +++++++++++++++++++++++++++++++++
 drivers/pci/setup-bus.c | 21 ++++++++++++++++++---
 drivers/pci/setup-res.c |  4 ++++
 include/linux/pci.h     |  1 +
 6 files changed, 64 insertions(+), 7 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 613eef1..e18176b 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -415,7 +415,7 @@ int pci_find_ht_capability(struct pci_dev *dev, int ht_cap)
 EXPORT_SYMBOL_GPL(pci_find_ht_capability);
 
 static struct resource *pci_find_bus_resource(const struct pci_bus *bus,
-					      struct resource *res)
+					      struct resource *res, int flags)
 {
 	struct resource *r;
 	int i;
@@ -430,7 +430,7 @@ static struct resource *pci_find_bus_resource(const struct pci_bus *bus,
 			 * not, the allocator made a mistake.
 			 */
 			if (r->flags & IORESOURCE_PREFETCH &&
-			    !(res->flags & IORESOURCE_PREFETCH))
+			    !(flags & IORESOURCE_PREFETCH))
 				return NULL;
 
 			/*
@@ -458,7 +458,9 @@ static struct resource *pci_find_bus_resource(const struct pci_bus *bus,
 struct resource *pci_find_parent_resource(const struct pci_dev *dev,
 					  struct resource *res)
 {
-	return pci_find_bus_resource(dev->bus, res);
+	int flags = pci_resource_pref_compatible(dev, res);
+
+	return pci_find_bus_resource(dev->bus, res, flags);
 }
 EXPORT_SYMBOL(pci_find_parent_resource);
 
@@ -468,7 +470,7 @@ struct resource *pci_find_root_bus_resource(struct pci_bus *bus,
 	while (bus->parent)
 		bus = bus->parent;
 
-	return pci_find_bus_resource(bus, res);
+	return pci_find_bus_resource(bus, res, res->flags);
 }
 
 /**
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 24ba9dc..6ad0f05 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -337,4 +337,6 @@ static inline int pci_dev_specific_reset(struct pci_dev *dev, int probe)
 
 struct pci_host_bridge *pci_find_host_bridge(struct pci_bus *bus);
 
+int pci_resource_pref_compatible(const struct pci_dev *dev,
+				 struct resource *res);
 #endif /* DRIVERS_PCI_H */
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 8361d27..2c69441 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1633,6 +1633,36 @@ static void pci_set_msi_domain(struct pci_dev *dev)
 				   dev_get_msi_domain(&dev->bus->dev));
 }
 
+static bool pci_up_path_over_pcie(struct pci_bus *bus)
+{
+	if (pci_is_root_bus(bus))
+		return true;
+
+	if (bus->self && !pci_is_pcie(bus->self))
+		return false;
+
+	return pci_up_path_over_pcie(bus->parent);
+}
+
+/*
+ * According to
+ * https://www.pcisig.com/specifications/pciexpress/base2/PCIe_Base_r2.1_Errata_08Jun10.pdf
+ * page 13, system firmware could put some 64bit non-pref under 64bit pref,
+ * on some cases.
+ * Let's mark if entire path from the host to the adapter is over PCI
+ * Express. later will use that compute pref compaitable bit.
+ */
+static void pci_set_on_all_pcie_path(struct pci_dev *dev)
+{
+	if (!pci_is_pcie(dev))
+		return;
+
+	if (!pci_up_path_over_pcie(dev->bus))
+		return;
+
+	dev->on_all_pcie_path = 1;
+}
+
 void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
 {
 	int ret;
@@ -1663,6 +1693,9 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
 	/* Initialize various capabilities */
 	pci_init_capabilities(dev);
 
+	/* After pcie_cap is assigned */
+	pci_set_on_all_pcie_path(dev);
+
 	/*
 	 * Add the device to our list of discovered devices
 	 * and the bus list for fixup functions, etc.
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 508cc56..6c43c43 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -739,6 +739,20 @@ int pci_claim_bridge_resource(struct pci_dev *bridge, int i)
 	return -EINVAL;
 }
 
+int pci_resource_pref_compatible(const struct pci_dev *dev,
+				 struct resource *res)
+{
+	if (res->flags & IORESOURCE_PREFETCH)
+		return res->flags;
+
+	if ((res->flags & IORESOURCE_MEM) &&
+	    (res->flags & IORESOURCE_MEM_64) &&
+	    dev->on_all_pcie_path)
+		return res->flags | IORESOURCE_PREFETCH;
+
+	return res->flags;
+}
+
 /* Check whether the bridge supports optional I/O and
    prefetchable memory ranges. If not, the respective
    base/limit registers must be read-only and read as 0. */
@@ -1036,10 +1050,11 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 			resource_size_t r_size;
+			int flags = pci_resource_pref_compatible(dev, r);
 
-			if (r->parent || ((r->flags & mask) != type &&
-					  (r->flags & mask) != type2 &&
-					  (r->flags & mask) != type3))
+			if (r->parent || ((flags & mask) != type &&
+					  (flags & mask) != type2 &&
+					  (flags & mask) != type3))
 				continue;
 			r_size = resource_size(r);
 #ifdef CONFIG_PCI_IOV
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 232f925..b19aa5b 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -250,15 +250,19 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 static int _pci_assign_resource(struct pci_dev *dev, int resno,
 				resource_size_t size, resource_size_t min_align)
 {
+	struct resource *res = dev->resource + resno;
+	int old_flags = res->flags;
 	struct pci_bus *bus;
 	int ret;
 
+	res->flags = pci_resource_pref_compatible(dev, res);
 	bus = dev->bus;
 	while ((ret = __pci_assign_resource(bus, dev, resno, size, min_align))) {
 		if (!bus->parent || !bus->self->transparent)
 			break;
 		bus = bus->parent;
 	}
+	res->flags = old_flags;
 
 	return ret;
 }
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 05f5eac..f7468f7 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -311,6 +311,7 @@ struct pci_dev {
 						   powered on/off by the
 						   corresponding bridge */
 	unsigned int	ignore_hotplug:1;	/* Ignore hotplug events */
+	unsigned int	on_all_pcie_path:1;	/* up to host-bridge all pcie */
 	unsigned int	d3_delay;	/* D3->D0 transition time in ms */
 	unsigned int	d3cold_delay;	/* D3cold->D0 transition time in ms */
 
-- 
1.8.4.5

[PATCH v8 13/61] PCI: Only treat non-pref mmio64 as pref if all bridges have MEM_64

[Yinghai Lu]

If any bridge up to root only have 32bit pref mmio, We don't need to
treat device non-pref mmio64 as as pref mmio64.

We need to move pci_bridge_check_ranges calling early.
for parent bridges pref mmio BAR may not allocated by BIOS, res flags
is still 0, we need to have it correct set before we check them for
child device resources.

-v2: check all bus resources instead of just res[15].

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
---
 drivers/pci/setup-bus.c | 31 +++++++++++++++++++++++++++++--
 1 file changed, 29 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 6c43c43..b1e4595 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -739,6 +739,29 @@ int pci_claim_bridge_resource(struct pci_dev *bridge, int i)
 	return -EINVAL;
 }
 
+static bool pci_up_path_over_pref_mem64(struct pci_bus *bus)
+{
+	if (pci_is_root_bus(bus))
+		return true;
+
+	if (bus->self) {
+		int i;
+		bool found = false;
+		struct resource *res;
+
+		pci_bus_for_each_resource(bus, res, i)
+			if (res->flags & IORESOURCE_MEM_64) {
+				found = true;
+				break;
+			}
+
+		if (!found)
+			return false;
+	}
+
+	return pci_up_path_over_pref_mem64(bus->parent);
+}
+
 int pci_resource_pref_compatible(const struct pci_dev *dev,
 				 struct resource *res)
 {
@@ -747,7 +770,8 @@ int pci_resource_pref_compatible(const struct pci_dev *dev,
 
 	if ((res->flags & IORESOURCE_MEM) &&
 	    (res->flags & IORESOURCE_MEM_64) &&
-	    dev->on_all_pcie_path)
+	    dev->on_all_pcie_path &&
+	    pci_up_path_over_pref_mem64(dev->bus))
 		return res->flags | IORESOURCE_PREFETCH;
 
 	return res->flags;
@@ -1239,6 +1263,10 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 	struct resource *b_res;
 	int ret;
 
+	if (!pci_is_root_bus(bus) &&
+	    (bus->self->class >> 8) == PCI_CLASS_BRIDGE_PCI)
+		pci_bridge_check_ranges(bus);
+
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		struct pci_bus *b = dev->subordinate;
 		if (!b)
@@ -1266,7 +1294,6 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 		break;
 
 	case PCI_CLASS_BRIDGE_PCI:
-		pci_bridge_check_ranges(bus);
 		if (bus->self->is_hotplug_bridge) {
 			additional_io_size  = pci_hotplug_io_size;
 			additional_mem_size = pci_hotplug_mem_size;
-- 
1.8.4.5

[PATCH v8 14/61] PCI: Add has_mem64 for struct host_bridge

[Yinghai Lu]

Add has_mem64 for struct host_bridge, on root bus that does not support
mmio64 above 4g, will not set that.

We will use that info next two following patches:
1. Don't treat non-pref mmio64 as pref mmio, so will not put
   it under bridge's pref range when rescan the devices
2. will keep pref mmio64 and pref mmio32 under bridge pref bar.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
---
 drivers/pci/probe.c | 7 +++++++
 include/linux/pci.h | 1 +
 2 files changed, 8 insertions(+)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 2c69441..edc8d08 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2125,6 +2125,13 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
 		} else
 			bus_addr[0] = '\0';
 		dev_info(&b->dev, "root bus resource %pR%s\n", res, bus_addr);
+
+		if (resource_type(res) == IORESOURCE_MEM) {
+			if ((res->end - offset) > 0xffffffff)
+				bridge->has_mem64 = 1;
+			if ((res->start - offset) > 0xffffffff)
+				res->flags |= IORESOURCE_MEM_64;
+		}
 	}
 
 	down_write(&pci_bus_sem);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index f7468f7..cffaed4 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -413,6 +413,7 @@ struct pci_host_bridge {
 	void (*release_fn)(struct pci_host_bridge *);
 	void *release_data;
 	unsigned int ignore_reset_delay:1;	/* for entire hierarchy */
+	unsigned int has_mem64:1;
 };
 
 #define	to_pci_host_bridge(n) container_of(n, struct pci_host_bridge, dev)
-- 
1.8.4.5

[PATCH v8 15/61] PCI: Only treat non-pref mmio64 as pref if host bridge has mmio64

[Yinghai Lu]

If host bridge does not have mmio64 above 4G, We don't need to
treat device non-pref mmio64 as as pref mmio64.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
---
 drivers/pci/setup-bus.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index b1e4595..95d18ef 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -742,7 +742,7 @@ int pci_claim_bridge_resource(struct pci_dev *bridge, int i)
 static bool pci_up_path_over_pref_mem64(struct pci_bus *bus)
 {
 	if (pci_is_root_bus(bus))
-		return true;
+		return to_pci_host_bridge(bus->bridge)->has_mem64;
 
 	if (bus->self) {
 		int i;
-- 
1.8.4.5

[PATCH v8 16/61] PCI: Restore pref MMIO allocation logic for host bridge without mmio64

[Yinghai Lu]

>From 5b2854155 (PCI: Restrict 64-bit prefetchable bridge windows to 64-bit
resources), we change the logic for pref mmio allocation:
When bridge pref support mmio64, we will only put children pref
that support mmio64 into it, and will put children pref mmio32
into bridge's non-pref mmio32.

That could leave bridge pref bar not used when that pref bar is mmio64,
and children res only has mmio32.
Also could have allocation failure when non-pref mmio32 is not big
enough space for those children pref mmio32.

That is not rational when the host bridge does not 64bit mmio above 4g
at all.

The patch restore to old logic:
when host bridge does not have has_mem64, put children pref mmio64 and
pref mmio32 all under bridges pref bars.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Tested-by: Khalid Aziz <khalid.aziz@oracle.com>
---
 drivers/pci/bus.c       |  4 +++-
 drivers/pci/setup-bus.c | 13 +++++++++----
 drivers/pci/setup-res.c |  9 ++++++---
 3 files changed, 18 insertions(+), 8 deletions(-)

diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index d3346d2..0859e77 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -202,8 +202,10 @@ int pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res,
 {
 #ifdef CONFIG_PCI_BUS_ADDR_T_64BIT
 	int rc;
+	unsigned long mmio64 = pci_find_host_bridge(bus)->has_mem64 ?
+				IORESOURCE_MEM_64 : 0;
 
-	if (res->flags & IORESOURCE_MEM_64) {
+	if (res->flags & mmio64) {
 		rc = pci_bus_alloc_from_region(bus, res, size, align, min,
 					       type_mask, alignf, alignf_data,
 					       &pci_high);
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 95d18ef..0d64677 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1311,7 +1311,8 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 		b_res = &bus->self->resource[PCI_BRIDGE_RESOURCES];
 		mask = IORESOURCE_MEM;
 		prefmask = IORESOURCE_MEM | IORESOURCE_PREFETCH;
-		if (b_res[2].flags & IORESOURCE_MEM_64) {
+		if ((b_res[2].flags & IORESOURCE_MEM_64) &&
+		    pci_find_host_bridge(bus)->has_mem64) {
 			prefmask |= IORESOURCE_MEM_64;
 			ret = pbus_size_mem(bus, prefmask, prefmask,
 				  prefmask, prefmask,
@@ -1470,17 +1471,21 @@ static void pci_bridge_release_resources(struct pci_bus *bus,
 	 *	  io port.
 	 *     2. if there is non pref mmio assign fail, release bridge
 	 *	  nonpref mmio.
-	 *     3. if there is 64bit pref mmio assign fail, and bridge pref
+	 *     3. if there is pref mmio assign fail, and host bridge does
+	 *	  have 64bit mmio, release bridge pref mmio.
+	 *     4. if there is 64bit pref mmio assign fail, and bridge pref
 	 *	  is 64bit, release bridge pref mmio.
-	 *     4. if there is pref mmio assign fail, and bridge pref is
+	 *     5. if there is pref mmio assign fail, and bridge pref is
 	 *	  32bit mmio, release bridge pref mmio
-	 *     5. if there is pref mmio assign fail, and bridge pref is not
+	 *     6. if there is pref mmio assign fail, and bridge pref is not
 	 *	  assigned, release bridge nonpref mmio.
 	 */
 	if (type & IORESOURCE_IO)
 		idx = 0;
 	else if (!(type & IORESOURCE_PREFETCH))
 		idx = 1;
+	else if (!pci_find_host_bridge(bus)->has_mem64)
+		idx = 2;
 	else if ((type & IORESOURCE_MEM_64) &&
 		 (b_res[2].flags & IORESOURCE_MEM_64))
 		idx = 2;
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index b19aa5b..26aedde 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -205,6 +205,8 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 	struct resource *res = dev->resource + resno;
 	resource_size_t min;
 	int ret;
+	unsigned long mmio64 = pci_find_host_bridge(bus)->has_mem64 ?
+				IORESOURCE_MEM_64 : 0;
 
 	min = (res->flags & IORESOURCE_IO) ? PCIBIOS_MIN_IO : PCIBIOS_MIN_MEM;
 
@@ -216,7 +218,7 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 	 * things differently than they were sized, not everything will fit.
 	 */
 	ret = pci_bus_alloc_resource(bus, res, size, align, min,
-				     IORESOURCE_PREFETCH | IORESOURCE_MEM_64,
+				     IORESOURCE_PREFETCH | mmio64,
 				     pcibios_align_resource, dev);
 	if (ret == 0)
 		return 0;
@@ -225,7 +227,8 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 	 * If the prefetchable window is only 32 bits wide, we can put
 	 * 64-bit prefetchable resources in it.
 	 */
-	if ((res->flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64)) ==
+	if (mmio64 &&
+	    (res->flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64)) ==
 	     (IORESOURCE_PREFETCH | IORESOURCE_MEM_64)) {
 		ret = pci_bus_alloc_resource(bus, res, size, align, min,
 					     IORESOURCE_PREFETCH,
@@ -240,7 +243,7 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 	 * non-prefetchable, the first call already tried the only possibility
 	 * so we don't need to try again.
 	 */
-	if (res->flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64))
+	if (res->flags & (IORESOURCE_PREFETCH | mmio64))
 		ret = pci_bus_alloc_resource(bus, res, size, align, min, 0,
 					     pcibios_align_resource, dev);
 
-- 
1.8.4.5

[PATCH v8 17/61] PCI: Don't release fixed resource for realloc

[Yinghai Lu]

We should not release bridge resource if there is fixed resources
under it, otherwise the children firmware would stop working.

Reported-by: Paul Johnson <pjay@nwtrail.com>
Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=92351
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: stable@vger.kernel.org
---
 drivers/pci/setup-bus.c |  6 ++++--
 include/linux/ioport.h  |  2 +-
 kernel/resource.c       | 28 ++++++++++++++++++++++++++--
 3 files changed, 31 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 0d64677..8cd6c6b 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1497,14 +1497,16 @@ static void pci_bridge_release_resources(struct pci_bus *bus,
 
 	r = &b_res[idx];
 
-	if (!r->parent)
+	if (!r->parent || r->flags & IORESOURCE_PCI_FIXED)
 		return;
 
 	/*
 	 * if there are children under that, we should release them
 	 *  all
 	 */
-	release_child_resources(r);
+	if (!release_child_resources(r))
+		return;
+
 	if (!release_resource(r)) {
 		type = old_flags = r->flags & type_mask;
 		dev_printk(KERN_DEBUG, &dev->dev, "resource %d %pR released\n",
diff --git a/include/linux/ioport.h b/include/linux/ioport.h
index 388e3ae..27dbb18 100644
--- a/include/linux/ioport.h
+++ b/include/linux/ioport.h
@@ -141,7 +141,7 @@ extern struct resource iomem_resource;
 extern struct resource *request_resource_conflict(struct resource *root, struct resource *new);
 extern int request_resource(struct resource *root, struct resource *new);
 extern int release_resource(struct resource *new);
-void release_child_resources(struct resource *new);
+bool release_child_resources(struct resource *new);
 extern void reserve_region_with_split(struct resource *root,
 			     resource_size_t start, resource_size_t end,
 			     const char *name);
diff --git a/kernel/resource.c b/kernel/resource.c
index f150dbb..6927298 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -275,11 +275,35 @@ static void __release_child_resources(struct resource *r)
 	}
 }
 
-void release_child_resources(struct resource *r)
+static bool __has_fixed_child_resources(struct resource *r)
 {
+	struct resource *p;
+
+	p = r->child;
+	while (p) {
+		if (p->flags & IORESOURCE_PCI_FIXED)
+			return true;
+
+		if (__has_fixed_child_resources(p))
+			return true;
+
+		p = p->sibling;
+	}
+
+	return false;
+}
+
+bool release_child_resources(struct resource *r)
+{
+	bool fixed;
+
 	write_lock(&resource_lock);
-	__release_child_resources(r);
+	fixed = __has_fixed_child_resources(r);
+	if (!fixed)
+		__release_child_resources(r);
 	write_unlock(&resource_lock);
+
+	return !fixed;
 }
 
 /**
-- 
1.8.4.5

[PATCH v8 18/61] PCI: Claim fixed resource during remove/rescan path

[Yinghai Lu]

During driver loading kernel will check if resources get reserved.
so we need to make sure resources get reserved before pci_bus_add().

On remove/rescan path, we will leave those fixed resource
not reserved. In that path, We don't call pcibios_resource_survery()
before pci_assign_unassigned_bus_resources(), and that is intentional
for us to get new resources for rescan. We do need to use rescan to make
device get resource allocated while ignoring BIOS allocate resource.

But fixed resources are not allocated via
pci_assign_unassigned_bus_resources(), so we need to reserve them
explicitly.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/quirks.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index af75580..1d85f37 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -323,6 +323,23 @@ static void quirk_s3_64M(struct pci_dev *dev)
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_S3,	PCI_DEVICE_ID_S3_868,		quirk_s3_64M);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_S3,	PCI_DEVICE_ID_S3_968,		quirk_s3_64M);
 
+/* for pci remove and rescan */
+static void quirk_allocate_fixed(struct pci_dev *dev)
+{
+	int i;
+	for (i = 0; i < PCI_NUM_RESOURCES; i++) {
+		struct resource *r = &dev->resource[i];
+
+		if (r->parent ||
+		    !(r->flags & IORESOURCE_PCI_FIXED) ||
+		    !(r->flags & (IORESOURCE_IO | IORESOURCE_MEM)))
+			continue;
+
+		pci_claim_resource(dev, i);
+	}
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_ANY_ID,	PCI_ANY_ID,	quirk_allocate_fixed);
+
 static void quirk_io(struct pci_dev *dev, int pos, unsigned size,
 		     const char *name)
 {
-- 
1.8.4.5

[PATCH v8 19/61] PCI: Set resource to FIXED for LSI devices

[Yinghai Lu]

LSI HBA firmware stop responding pci read from host if pci core ever change
pci device BAR values.

Set their resources to FIXED, so will allow realloc to skip them.

Reported-by: Paul Johnson <pjay@nwtrail.com>
Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=92351
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: stable@vger.kernel.org
---
 drivers/pci/pci.h       |  1 +
 drivers/pci/quirks.c    | 20 ++++++++++++++++++++
 drivers/pci/setup-bus.c |  4 ++++
 3 files changed, 25 insertions(+)

diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 6ad0f05..39d2c50 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -170,6 +170,7 @@ static inline void pci_msix_clear_and_set_ctrl(struct pci_dev *dev, u16 clear, u
 }
 
 void pci_realloc_get_opt(char *);
+bool pci_realloc_user_enabled(void);
 
 static inline int pci_no_d1d2(struct pci_dev *dev)
 {
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 1d85f37..be0bdfb 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -323,6 +323,26 @@ static void quirk_s3_64M(struct pci_dev *dev)
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_S3,	PCI_DEVICE_ID_S3_868,		quirk_s3_64M);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_S3,	PCI_DEVICE_ID_S3_968,		quirk_s3_64M);
 
+/*
+ * LSI devices firmware does not like BAR get changed
+ */
+static void quirk_bar_fixed(struct pci_dev *dev)
+{
+	int i;
+
+	if (pci_realloc_user_enabled())
+		return;
+
+	for (i = 0; i < PCI_STD_RESOURCE_END; i++) {
+		struct resource *r = &dev->resource[i];
+
+		if (!r->start || !r->flags)
+			continue;
+		r->flags |= IORESOURCE_PCI_FIXED;
+	}
+}
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_LSI_LOGIC,	PCI_ANY_ID,	quirk_bar_fixed);
+
 /* for pci remove and rescan */
 static void quirk_allocate_fixed(struct pci_dev *dev)
 {
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 8cd6c6b..d49c6e5 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1634,6 +1634,10 @@ void __init pci_realloc_get_opt(char *str)
 	else if (!strncmp(str, "on", 2))
 		pci_realloc_enable = user_enabled;
 }
+bool pci_realloc_user_enabled(void)
+{
+	return pci_realloc_enable == user_enabled;
+}
 static bool pci_realloc_enabled(enum enable_type enable)
 {
 	return enable >= user_enabled;
-- 
1.8.4.5

[PATCH v8 20/61] PCI: Separate realloc list checking after allocation

[Yinghai Lu]

We check the realloc list, as list must be empty after allocation.

Separate the realloc list checking to another function.

Add checking that is missed in acpiphp driver.

-v2: change to WARN_ON according to Rafael.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
Cc: Len Brown <lenb@kernel.org>
Cc: linux-acpi@vger.kernel.org
---
 drivers/pci/hotplug/acpiphp_glue.c |  1 +
 drivers/pci/pci.h                  |  1 +
 drivers/pci/setup-bus.c            | 12 +++++++++---
 3 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/hotplug/acpiphp_glue.c b/drivers/pci/hotplug/acpiphp_glue.c
index ff53856..ce9fb21 100644
--- a/drivers/pci/hotplug/acpiphp_glue.c
+++ b/drivers/pci/hotplug/acpiphp_glue.c
@@ -507,6 +507,7 @@ static void enable_slot(struct acpiphp_slot *slot)
 		}
 	}
 	__pci_bus_assign_resources(bus, &add_list, NULL);
+	pci_bus_check_realloc(&add_list);
 
 	acpiphp_sanitize_bus(bus);
 	pcie_bus_configure_settings(bus);
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 39d2c50..c34c2aa 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -238,6 +238,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus,
 void __pci_bus_assign_resources(const struct pci_bus *bus,
 				struct list_head *realloc_head,
 				struct list_head *fail_head);
+void pci_bus_check_realloc(struct list_head *realloc_head);
 bool pci_bus_clip_resource(struct pci_dev *dev, int idx);
 
 void pci_reassigndev_resource_alignment(struct pci_dev *dev);
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index d49c6e5..53ab81f 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -280,6 +280,12 @@ out:
 	}
 }
 
+void pci_bus_check_realloc(struct list_head *realloc_head)
+{
+	if (WARN_ON(!list_empty(realloc_head)))
+		free_list(realloc_head);
+}
+
 /**
  * assign_requested_resources_sorted() - satisfy resource requests
  *
@@ -1733,7 +1739,7 @@ again:
 	/* Depth last, allocate resources and update the hardware. */
 	__pci_bus_assign_resources(bus, add_list, &fail_head);
 	if (add_list)
-		BUG_ON(!list_empty(add_list));
+		pci_bus_check_realloc(add_list);
 	tried_times++;
 
 	/* any device complain? */
@@ -1808,7 +1814,7 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
 again:
 	__pci_bus_size_bridges(parent, &add_list);
 	__pci_bridge_assign_resources(bridge, &add_list, &fail_head);
-	BUG_ON(!list_empty(&add_list));
+	pci_bus_check_realloc(&add_list);
 	tried_times++;
 
 	if (list_empty(&fail_head))
@@ -1867,6 +1873,6 @@ void pci_assign_unassigned_bus_resources(struct pci_bus *bus)
 							 &add_list);
 	up_read(&pci_bus_sem);
 	__pci_bus_assign_resources(bus, &add_list, NULL);
-	BUG_ON(!list_empty(&add_list));
+	pci_bus_check_realloc(&add_list);
 }
 EXPORT_SYMBOL_GPL(pci_assign_unassigned_bus_resources);
-- 
1.8.4.5

[PATCH v8 21/61] PCI: Treat optional as required in first try for bridge rescan

[Yinghai Lu]

For rescan bridge/bus that children are removed before, we should treat
optional as required just like root bus the boot time in 19aa7ee432ce
(PCI: make re-allocation try harder by reassigning ranges higher in
the heirarchy).

The reason: allocate required and expand to optional path do not
put failed resource to fail list, so will lose required info before
next try.

So we are using following way:
1. First and following try before last try:
   We don't keep realloc list so treat every optional as required.
   allocate for required+optional and put failed in the fail list.
   then size info (include must and optonal separatedly) will be kept
   for next try.
2. last try:
   a: try to allocate required+optional to see if all get allocated.
   b: try to allocate required then expand to optional.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 53ab81f..5505942 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1802,25 +1802,34 @@ void __init pci_assign_unassigned_resources(void)
 void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
 {
 	struct pci_bus *parent = bridge->subordinate;
-	LIST_HEAD(add_list); /* list of resources that
+	LIST_HEAD(realloc_head); /* list of resources that
 					want additional resources */
+	struct list_head *add_list = NULL;
 	int tried_times = 0;
 	LIST_HEAD(fail_head);
 	struct pci_dev_resource *fail_res;
 	int retval;
 	unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
 				  IORESOURCE_PREFETCH | IORESOURCE_MEM_64;
+	int pci_try_num = 2;
 
 again:
-	__pci_bus_size_bridges(parent, &add_list);
-	__pci_bridge_assign_resources(bridge, &add_list, &fail_head);
-	pci_bus_check_realloc(&add_list);
+	/*
+	 * last try will use add_list, otherwise will try good to have as
+	 * must have, so can realloc parent bridge resource
+	 */
+	if (tried_times + 1 == pci_try_num)
+		add_list = &realloc_head;
+	__pci_bus_size_bridges(parent, add_list);
+	__pci_bridge_assign_resources(bridge, add_list, &fail_head);
+	if (add_list)
+		pci_bus_check_realloc(add_list);
 	tried_times++;
 
 	if (list_empty(&fail_head))
 		goto enable_all;
 
-	if (tried_times >= 2) {
+	if (tried_times >= pci_try_num) {
 		/* still fail, don't need to try more */
 		free_list(&fail_head);
 		goto enable_all;
-- 
1.8.4.5

[PATCH v8 22/61] PCI: Get new realloc size for bridge for last try

[Yinghai Lu]

Current realloc path would not shrink bridge resource through
pbus_size_mem() checking with the old size.

That cause problem: when "required+optional" resource allocation fails,
the cached bridge resource size will prevent "required" resource to get
allocated smaller resource.

Clear the old resource size for last try or third and later try.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
Tested-by: TJ <linux@iam.tj>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 5505942..278cb6d 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1695,6 +1695,17 @@ static enum enable_type pci_realloc_detect(struct pci_bus *bus,
 }
 #endif
 
+static void reset_bridge_resource_size(struct pci_dev *dev,
+				       struct resource *res)
+{
+	int idx = res - &dev->resource[0];
+
+	if (idx >= PCI_BRIDGE_RESOURCES && idx <= PCI_BRIDGE_RESOURCE_END) {
+		res->start = 0;
+		res->end = res->start - 1;
+	}
+}
+
 /*
  * first try will not touch pci bridge res
  * second and later try will clear small leaf bridge res
@@ -1779,8 +1790,13 @@ again:
 		res->start = fail_res->start;
 		res->end = fail_res->end;
 		res->flags = fail_res->flags;
-		if (fail_res->dev->subordinate)
+		if (fail_res->dev->subordinate) {
 			res->flags = 0;
+			/* last or third times and later */
+			if (tried_times + 1 == pci_try_num ||
+			    tried_times + 1 > 2)
+				reset_bridge_resource_size(fail_res->dev, res);
+		}
 	}
 	free_list(&fail_head);
 
@@ -1854,8 +1870,11 @@ again:
 		res->start = fail_res->start;
 		res->end = fail_res->end;
 		res->flags = fail_res->flags;
-		if (fail_res->dev->subordinate)
+		if (fail_res->dev->subordinate) {
 			res->flags = 0;
+			/* last time */
+			reset_bridge_resource_size(fail_res->dev, res);
+		}
 	}
 	free_list(&fail_head);
 
-- 
1.8.4.5

[PATCH v8 23/61] PCI: Don't release sibling bridge resources during hotplug

[Yinghai Lu]

On hotplug path, we can not touch sibling bridges that is outside
of the slot.

That could happen when BIOS does not assign some bridge BARs and
later kernel can not assign resource to them in first try.

Check if fail dev is the parent bridge, then just use subordinate
bus instead use parent bus.

Reported-by: Andreas Noever <andreas.noever@gmail.com>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 278cb6d..10270e4 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1858,10 +1858,16 @@ again:
 	 * Try to release leaf bridge's resources that doesn't fit resource of
 	 * child device under that bridge
 	 */
-	list_for_each_entry(fail_res, &fail_head, list)
-		pci_bus_release_bridge_resources(fail_res->dev->bus,
+	list_for_each_entry(fail_res, &fail_head, list) {
+		struct pci_bus *bus = fail_res->dev->bus;
+
+		if (fail_res->dev == bridge)
+			bus = bridge->subordinate;
+
+		pci_bus_release_bridge_resources(bus,
 						 fail_res->flags & type_mask,
 						 whole_subtree);
+	}
 
 	/* restore size and flags */
 	list_for_each_entry(fail_res, &fail_head, list) {
-- 
1.8.4.5

[PATCH v8 24/61] PCI: Cleanup res_to_dev_res() printout

[Yinghai Lu]

Now get_res_add_size() and get_res_add_align() all have same printout
from res_to_dev_res(), and it is confusing.

Move out debug messages printout from res_to_dev_res(),
so later we will reuse res_to_dev_res() in other functions.

-v2: does not print out when add_size or min_align is 0
-v3: change to %#llx according to Bjorn.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 10270e4..f905c81 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -104,19 +104,9 @@ static struct pci_dev_resource *res_to_dev_res(struct list_head *head,
 {
 	struct pci_dev_resource *dev_res;
 
-	list_for_each_entry(dev_res, head, list) {
-		if (dev_res->res == res) {
-			int idx = res - &dev_res->dev->resource[0];
-
-			dev_printk(KERN_DEBUG, &dev_res->dev->dev,
-				 "res[%d]=%pR res_to_dev_res add_size %llx min_align %llx\n",
-				 idx, dev_res->res,
-				 (unsigned long long)dev_res->add_size,
-				 (unsigned long long)dev_res->min_align);
-
+	list_for_each_entry(dev_res, head, list)
+		if (dev_res->res == res)
 			return dev_res;
-		}
-	}
 
 	return NULL;
 }
@@ -127,7 +117,15 @@ static resource_size_t get_res_add_size(struct list_head *head,
 	struct pci_dev_resource *dev_res;
 
 	dev_res = res_to_dev_res(head, res);
-	return dev_res ? dev_res->add_size : 0;
+	if (!dev_res || !dev_res->add_size)
+		return 0;
+
+	dev_printk(KERN_DEBUG, &dev_res->dev->dev,
+		   "BAR %d: %pR get_res_add_size add_size   %#llx\n",
+		   (int)(res - &dev_res->dev->resource[0]),
+		   res, (unsigned long long)dev_res->add_size);
+
+	return dev_res->add_size;
 }
 
 static resource_size_t get_res_add_align(struct list_head *head,
@@ -136,7 +134,15 @@ static resource_size_t get_res_add_align(struct list_head *head,
 	struct pci_dev_resource *dev_res;
 
 	dev_res = res_to_dev_res(head, res);
-	return dev_res ? dev_res->min_align : 0;
+	if (!dev_res || !dev_res->min_align)
+		return 0;
+
+	dev_printk(KERN_DEBUG, &dev_res->dev->dev,
+		   "BAR %d: %pR get_res_add_align min_align %#llx\n",
+		   (int)(res - &dev_res->dev->resource[0]),
+		   res, (unsigned long long)dev_res->min_align);
+
+	return dev_res->min_align;
 }
 
 
-- 
1.8.4.5

[PATCH v8 25/61] PCI: Reuse res_to_dev_res() in reassign_resources_sorted()

[Yinghai Lu]

Now res_to_dev_res() does not print out debug message anymore, so
we can reuse it in reassign_resource_sorted() without confusing printout.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 11 +----------
 1 file changed, 1 insertion(+), 10 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index f905c81..e075495 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -240,26 +240,17 @@ static void reassign_resources_sorted(struct list_head *realloc_head,
 {
 	struct resource *res;
 	struct pci_dev_resource *add_res, *tmp;
-	struct pci_dev_resource *dev_res;
 	resource_size_t add_size, align;
 	int idx;
 
 	list_for_each_entry_safe(add_res, tmp, realloc_head, list) {
-		bool found_match = false;
-
 		res = add_res->res;
 		/* skip resource that has been reset */
 		if (!res->flags)
 			goto out;
 
 		/* skip this resource if not found in head list */
-		list_for_each_entry(dev_res, head, list) {
-			if (dev_res->res == res) {
-				found_match = true;
-				break;
-			}
-		}
-		if (!found_match)/* just skip */
+		if (!res_to_dev_res(head, res))
 			continue;
 
 		idx = res - &add_res->dev->resource[0];
-- 
1.8.4.5

[PATCH v8 26/61] PCI: Use correct align for optional only resources during sorting

[Yinghai Lu]

During sorting before assign, we only put resource with non-zero align
in the sorted list, so for optional resources that required size is 0 and
only have optional parts, we need to have correct align.

While treating SRIOV as optional resources, we always read alignment for
SRIOV bars every time, so they are ok.
Hotplug bridge resources are using STARTALIGN so it is ok when size is 0
if we have correct start for them.

Later we want to treat the ROM BAR as optional resource, and it has
SIZEALIGN, so align=size will be 0. We need to find a way to get align
for them.

We can use optional resource align instead in that case, and it
is ok for SRIOV path and hotplug bridge resource path.

We need to pass realloc list from sizing stage to sorting stage, and
get entry from realloc list and calculate align from the entry.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
Reported-by: TJ <linux@iam.tj>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 49 ++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 42 insertions(+), 7 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index e075495..ab67cdb 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -145,9 +145,42 @@ static resource_size_t get_res_add_align(struct list_head *head,
 	return dev_res->min_align;
 }
 
+static resource_size_t __pci_resource_alignment(
+				struct pci_dev *dev,
+				struct resource *r,
+				struct list_head *realloc_head)
+{
+	resource_size_t r_align = pci_resource_alignment(dev, r);
+	resource_size_t orig_start, orig_end;
+	struct pci_dev_resource *dev_res;
+
+	if (r_align || !realloc_head)
+		return r_align;
+
+	dev_res = res_to_dev_res(realloc_head, r);
+	if (!dev_res || !dev_res->add_size)
+		return r_align;
+
+	orig_start = r->start;
+	orig_end = r->end;
+	r->end += dev_res->add_size;
+	if ((r->flags & IORESOURCE_STARTALIGN)) {
+		resource_size_t r_size = resource_size(r);
+
+		r->start = dev_res->min_align;
+		r->end = r->start + r_size - 1;
+	}
+	r_align = pci_resource_alignment(dev, r);
+	r->start = orig_start;
+	r->end = orig_end;
+
+	return r_align;
+}
 
 /* Sort resources by alignment */
-static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head)
+static void pdev_sort_resources(struct pci_dev *dev,
+				 struct list_head *realloc_head,
+				 struct list_head *head)
 {
 	int i;
 
@@ -165,7 +198,7 @@ static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head)
 		if (!(r->flags) || r->parent)
 			continue;
 
-		r_align = pci_resource_alignment(dev, r);
+		r_align = __pci_resource_alignment(dev, r, realloc_head);
 		if (!r_align) {
 			dev_warn(&dev->dev, "BAR %d: %pR has bogus alignment\n",
 				 i, r);
@@ -183,8 +216,9 @@ static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head)
 		list_for_each_entry(dev_res, head, list) {
 			resource_size_t align;
 
-			align = pci_resource_alignment(dev_res->dev,
-							 dev_res->res);
+			align = __pci_resource_alignment(dev_res->dev,
+							 dev_res->res,
+							 realloc_head);
 
 			if (r_align > align) {
 				n = &dev_res->list;
@@ -197,6 +231,7 @@ static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head)
 }
 
 static void __dev_sort_resources(struct pci_dev *dev,
+				 struct list_head *realloc_head,
 				 struct list_head *head)
 {
 	u16 class = dev->class >> 8;
@@ -213,7 +248,7 @@ static void __dev_sort_resources(struct pci_dev *dev,
 			return;
 	}
 
-	pdev_sort_resources(dev, head);
+	pdev_sort_resources(dev, realloc_head, head);
 }
 
 static inline void reset_resource(struct resource *res)
@@ -507,7 +542,7 @@ static void pdev_assign_resources_sorted(struct pci_dev *dev,
 {
 	LIST_HEAD(head);
 
-	__dev_sort_resources(dev, &head);
+	__dev_sort_resources(dev, add_head, &head);
 	__assign_resources_sorted(&head, add_head, fail_head);
 
 }
@@ -520,7 +555,7 @@ static void pbus_assign_resources_sorted(const struct pci_bus *bus,
 	LIST_HEAD(head);
 
 	list_for_each_entry(dev, &bus->devices, bus_list)
-		__dev_sort_resources(dev, &head);
+		__dev_sort_resources(dev, realloc_head, &head);
 
 	__assign_resources_sorted(&head, realloc_head, fail_head);
 }
-- 
1.8.4.5

[PATCH v8 27/61] PCI: Optimize bus min_align/size calculation during sizing

[Yinghai Lu]

During bus mmio resource sizing stage, current code try to get alignment as
small as possible and use that to align size to get final size. But it does
not handle resource that size is bigger than alignment in optimal way, kernel
only use max alignment for them.

For example:
 When we have resources with align/size: 1M/2M, 512M/512M,
 current code will have bus resource min_align/size: 512M/1024M,
 but optimal value should be 256M/768M, as we can fit them into
 [256M,768M) or [512M,1280M) instead of [512M,1536M).

 0M        256M        512M       768M       1024M      1280M
 |----------|-----------|----------|----------|----------|----------|
when we have [256M,1024M)
            |---------------------------------|
            |-2M-|      |---512M--------------|
when we have [512M,1280M)
                        |--------------------------------|
                        |---512M--------------|-2M-|

For following cases that we have resource size that is bigger
than resource alignment:
1. SRIOV bar.
2. PCI bridges with children that need several MMIOs that are more than 1M.

We can keep on trying to allocate children devices resources from range
[offset, offset + aligned_size) and offset is aligned with half min_align.
If it sucesses, we can use that half min_align as new min_align.

After this patch, we get:
 align/size: 1M/2M, 2M/4M, 4M/8M, 8M/16M
 new min_align/min_size: 4M/32M, and original is 8M/32M

 align/size: 1M/2M, 2M/4M, 4M/8M
 new min_align/min_size: 2M/14M, and original is 4M/16M

 align/size: 1M/2M, 512M/512M
 new min_align/min_size: 256M/768M, and original is 512M/1024M

The real result from one system with one pcie card that has
four functions that support sriov:
 children resources with align/size:
   00800000/00800000, 00800000/00800000, 00800000/00800000,
   00800000/00800000, 00010000/00200000, 00010000/00200000,
   00010000/00200000, 00010000/00200000, 00008000/00008000,
   00008000/00008000, 00008000/00008000, 00008000/00008000,
   00004000/00080000, 00004000/00080000, 00004000/00080000,
   00004000/00080000
for the bridge:
With original code we have min_align/min_size: 00400000/02c00000,
and with this patch we have min_align/min_size: 00100000/02b00000
So min_align will be 1M instead of 4M and we even have smaller size.

-v2: Need to check more offset with every min_alignment.
-v3: skip r_size <= 1 for optional only bridge resources.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
Reported-by: TJ <linux@iam.tj>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 195 ++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 157 insertions(+), 38 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index ab67cdb..8f0ad46 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -30,6 +30,34 @@
 
 unsigned int pci_flags;
 
+static inline bool is_before(resource_size_t align1, resource_size_t size1,
+			     resource_size_t align2, resource_size_t size2)
+{
+	resource_size_t size1_left, size2_left;
+
+	/* big align is before small align */
+	if (align1 > align2)
+		return true;
+
+	/*
+	 * for same align:
+	 *   aligned is before not aligned
+	 *   for not aligned, big remainder is before small remainder
+	 */
+	if (align1 == align2) {
+		size1_left = size1 & (align1 - 1);
+		if (!size1_left)
+			size1_left = align1;
+		size2_left = size2 & (align2 - 1);
+		if (!size2_left)
+			size2_left = align2;
+		if (size1_left > size2_left)
+			return true;
+	}
+
+	return false;
+}
+
 struct pci_dev_resource {
 	struct list_head list;
 	struct resource *res;
@@ -1042,26 +1070,125 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 	}
 }
 
-static inline resource_size_t calculate_mem_align(resource_size_t *aligns,
-						  int max_order)
+struct align_test_res {
+	struct list_head list;
+	struct resource res;
+	resource_size_t size;
+	resource_size_t align;
+};
+
+static void free_align_test_list(struct list_head *head)
 {
-	resource_size_t align = 0;
-	resource_size_t min_align = 0;
-	int order;
+	struct align_test_res *p, *tmp;
 
-	for (order = 0; order <= max_order; order++) {
-		resource_size_t align1 = 1;
+	list_for_each_entry_safe(p, tmp, head, list) {
+		list_del(&p->list);
+		kfree(p);
+	}
+}
 
-		align1 <<= (order + 20);
+static int add_to_align_test_list(struct list_head *head,
+				  resource_size_t align, resource_size_t size)
+{
+	struct align_test_res *tmp;
+
+	tmp = kzalloc(sizeof(*tmp), GFP_KERNEL);
+	if (!tmp)
+		return -ENOMEM;
+
+	tmp->align = align;
+	tmp->size = size;
+
+	list_add_tail(&tmp->list, head);
+
+	return 0;
+}
+
+static void sort_align_test(struct list_head *head)
+{
+	struct align_test_res *res1, *tmp_res, *res2;
 
-		if (!align)
-			min_align = align1;
-		else if (ALIGN(align + min_align, min_align) < align1)
-			min_align = align1 >> 1;
-		align += aligns[order];
+	list_for_each_entry_safe(res1, tmp_res, head, list) {
+		/* reorder it */
+		list_for_each_entry(res2, head, list) {
+			if (res2 == res1)
+				break;
+
+			if (is_before(res1->align, res1->size,
+				      res2->align, res2->size)) {
+				list_move_tail(&res1->list, &res2->list);
+				break;
+			}
+		}
+	}
+}
+
+static bool is_align_size_good(struct list_head *head,
+			resource_size_t min_align, resource_size_t size,
+			resource_size_t start)
+{
+	struct align_test_res *p;
+	struct resource root;
+
+	memset(&root, 0, sizeof(root));
+	root.start = start;
+	root.end = start + size - 1;
+
+	list_for_each_entry(p, head, list)
+		memset(&p->res, 0, sizeof(p->res));
+
+	list_for_each_entry(p, head, list)
+		if (allocate_resource(&root, &p->res, p->size,
+				0, (resource_size_t)-1ULL,
+				p->align, NULL, NULL))
+			return false;
+
+	return true;
+}
+
+static resource_size_t calculate_mem_align(struct list_head *head,
+				resource_size_t max_align, resource_size_t size,
+				resource_size_t align_low)
+{
+	struct align_test_res *p;
+	resource_size_t min_align, good_align, aligned_size, start;
+	int count = 0;
+
+	if (max_align <= align_low) {
+		good_align = align_low;
+		goto out;
 	}
 
-	return min_align;
+	good_align = max_align;
+
+	list_for_each_entry(p, head, list)
+		count++;
+
+	if (count <= 1)
+		goto out;
+
+	sort_align_test(head);
+
+	do {
+		/* check if we can use smaller align */
+		min_align = good_align >> 1;
+		aligned_size = ALIGN(size, min_align);
+
+		/* need to make sure every offset work */
+		for (start = min_align; start < max_align; start += min_align) {
+			/* checked already with last align ? */
+			if (!(start & (good_align - 1)))
+				continue;
+
+			if (!is_align_size_good(head, min_align, aligned_size,
+					       start))
+				goto out;
+		}
+		good_align = min_align;
+	} while (min_align > align_low);
+
+out:
+	return good_align;
 }
 
 /**
@@ -1091,19 +1218,17 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 {
 	struct pci_dev *dev;
 	resource_size_t min_align, align, size, size0, size1;
-	resource_size_t aligns[18];	/* Alignments from 1Mb to 128Gb */
-	int order, max_order;
+	resource_size_t max_align = 0;
 	struct resource *b_res = find_free_bus_resource(bus,
 					mask | IORESOURCE_PREFETCH, type);
 	resource_size_t children_add_size = 0;
 	resource_size_t children_add_align = 0;
 	resource_size_t add_align = 0;
+	LIST_HEAD(align_test_list);
 
 	if (!b_res)
 		return -ENOSPC;
 
-	memset(aligns, 0, sizeof(aligns));
-	max_order = 0;
 	size = 0;
 
 	list_for_each_entry(dev, &bus->devices, bus_list) {
@@ -1130,29 +1255,20 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 				continue;
 			}
 #endif
-			/*
-			 * aligns[0] is for 1MB (since bridge memory
-			 * windows are always at least 1MB aligned), so
-			 * keep "order" from being negative for smaller
-			 * resources.
-			 */
 			align = pci_resource_alignment(dev, r);
-			order = __ffs(align) - 20;
-			if (order < 0)
-				order = 0;
-			if (order >= ARRAY_SIZE(aligns)) {
+			if (align > (1ULL<<37)) { /*128 Gb*/
 				dev_warn(&dev->dev, "disabling BAR %d: %pR (bad alignment %#llx)\n",
-					 i, r, (unsigned long long) align);
+					i, r, (unsigned long long) align);
 				r->flags = 0;
 				continue;
 			}
+
+			if (r_size > 1)
+				add_to_align_test_list(&align_test_list,
+							align, r_size);
 			size += r_size;
-			/* Exclude ranges with size > align from
-			   calculation of the alignment. */
-			if (r_size == align)
-				aligns[order] += align;
-			if (order > max_order)
-				max_order = order;
+			if (align > max_align)
+				max_align = align;
 
 			if (realloc_head) {
 				children_add_size += get_res_add_size(realloc_head, r);
@@ -1162,9 +1278,12 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		}
 	}
 
-	min_align = calculate_mem_align(aligns, max_order);
-	min_align = max(min_align, window_alignment(bus, b_res->flags));
-	size0 = calculate_memsize(size, min_size, 0, resource_size(b_res), min_align);
+	max_align = max(max_align, window_alignment(bus, b_res->flags));
+	min_align = calculate_mem_align(&align_test_list, max_align, size,
+					window_alignment(bus, b_res->flags));
+	size0 = calculate_memsize(size, min_size, 0,
+				  resource_size(b_res), min_align);
+	free_align_test_list(&align_test_list);
 	add_align = max(min_align, add_align);
 	if (children_add_size > add_size)
 		add_size = children_add_size;
-- 
1.8.4.5

[PATCH v8 28/61] PCI: Optimize bus align/size calculation for optional during sizing

[Yinghai Lu]

Current add_align always use max align, that make required+optional
to get allocated more than needed in some cases.

Now we have new calculate_mem_align(), so we could use it for
add_align calculation.

Need to create separated list for required+optional align/size info.

After that we can get smaller add_align/size, and we have more chance
to make required+optional to get allocated sucessfully.

The result for bridge that have Intel 4x10g card installed:

 pci 0000:20:03.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
	to [bus 2a-31] calculate_mem for required
 align/size:
   00800000/00800000, 00800000/00800000, 00800000/00800000,
   00800000/00800000, 00008000/00008000, 00008000/00008000,
   00008000/00008000, 00008000/00008000
 original min_align/min_size: 00400000/02400000
 new min_align/min_size: 00400000/02400000

 pci 0000:20:03.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
	to [bus 2a-31] calculate_mem for required+optional
 align/size:
   00800000/00800000, 00800000/00800000, 00800000/00800000,
   00800000/00800000, 00010000/00200000, 00010000/00200000,
   00010000/00200000, 00010000/00200000, 00008000/00008000,
   00008000/00008000, 00008000/00008000, 00008000/00008000,
   00004000/00080000, 00004000/00080000, 00004000/00080000,
   00004000/00080000
 original code min_align/min_size: 00800000/03000000
 new min_align/min_size: 00100000/02b00000

so required align/size: 0x400000/0x2400000, and
new required+optional align/size: 0x100000/0x2b00000, and it is much better
than original required+optional align/size: 0x800000/0x3000000
and even have smaller min_align than required.

-v2: remove not used size1 in calculate_memsize

Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
Reported-by: TJ <linux@iam.tj>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>


 drivers/pci/setup-bus.c |   82 +++++++++++++++++++++++++++++-------------------
 1 file changed, 51 insertions(+), 31 deletions(-)
---
 drivers/pci/setup-bus.c | 82 ++++++++++++++++++++++++++++++-------------------
 1 file changed, 51 insertions(+), 31 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 8f0ad46..0c6977c9 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -944,7 +944,6 @@ static resource_size_t calculate_iosize(resource_size_t size,
 
 static resource_size_t calculate_memsize(resource_size_t size,
 		resource_size_t min_size,
-		resource_size_t size1,
 		resource_size_t old_size,
 		resource_size_t align)
 {
@@ -954,7 +953,7 @@ static resource_size_t calculate_memsize(resource_size_t size,
 		old_size = 0;
 	if (size < old_size)
 		size = old_size;
-	size = ALIGN(size + size1, align);
+	size = ALIGN(size, align);
 	return size;
 }
 
@@ -1217,45 +1216,46 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 			 struct list_head *realloc_head)
 {
 	struct pci_dev *dev;
-	resource_size_t min_align, align, size, size0, size1;
-	resource_size_t max_align = 0;
+	resource_size_t min_align = 0, min_add_align = 0;
+	resource_size_t max_align = 0, max_add_align = 0;
+	resource_size_t size = 0, size0 = 0, size1 = 0, sum_add_size = 0;
 	struct resource *b_res = find_free_bus_resource(bus,
 					mask | IORESOURCE_PREFETCH, type);
-	resource_size_t children_add_size = 0;
-	resource_size_t children_add_align = 0;
-	resource_size_t add_align = 0;
 	LIST_HEAD(align_test_list);
+	LIST_HEAD(align_test_add_list);
 
 	if (!b_res)
 		return -ENOSPC;
 
-	size = 0;
-
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		int i;
 
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
-			resource_size_t r_size;
+			resource_size_t r_size, align;
 			int flags = pci_resource_pref_compatible(dev, r);
 
 			if (r->parent || ((flags & mask) != type &&
 					  (flags & mask) != type2 &&
 					  (flags & mask) != type3))
 				continue;
+
 			r_size = resource_size(r);
+			align = pci_resource_alignment(dev, r);
 #ifdef CONFIG_PCI_IOV
 			/* put SRIOV requested res to the optional list */
 			if (realloc_head && i >= PCI_IOV_RESOURCES &&
 					i <= PCI_IOV_RESOURCE_END) {
-				add_align = max(pci_resource_alignment(dev, r), add_align);
+				add_to_align_test_list(&align_test_add_list,
+							align, r_size);
 				r->end = r->start - 1;
 				add_to_list(realloc_head, dev, r, r_size, 0/* don't care */);
-				children_add_size += r_size;
+				sum_add_size += r_size;
+				if (align > max_add_align)
+					max_add_align = align;
 				continue;
 			}
 #endif
-			align = pci_resource_alignment(dev, r);
 			if (align > (1ULL<<37)) { /*128 Gb*/
 				dev_warn(&dev->dev, "disabling BAR %d: %pR (bad alignment %#llx)\n",
 					i, r, (unsigned long long) align);
@@ -1263,33 +1263,52 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 				continue;
 			}
 
-			if (r_size > 1)
+			if (r_size > 1) {
 				add_to_align_test_list(&align_test_list,
 							align, r_size);
-			size += r_size;
-			if (align > max_align)
-				max_align = align;
+				size += r_size;
+				if (align > max_align)
+					max_align = align;
+			}
 
 			if (realloc_head) {
-				children_add_size += get_res_add_size(realloc_head, r);
-				children_add_align = get_res_add_align(realloc_head, r);
-				add_align = max(add_align, children_add_align);
+				resource_size_t add_r_size, add_align;
+
+				add_r_size = get_res_add_size(realloc_head, r);
+				add_align = get_res_add_align(realloc_head, r);
+				/* no add on ? */
+				if (add_align < align)
+					add_align = align;
+				add_to_align_test_list(&align_test_add_list,
+							add_align,
+							r_size + add_r_size);
+				sum_add_size += r_size + add_r_size;
+				if (add_align > max_add_align)
+					max_add_align = add_align;
 			}
 		}
 	}
 
 	max_align = max(max_align, window_alignment(bus, b_res->flags));
-	min_align = calculate_mem_align(&align_test_list, max_align, size,
-					window_alignment(bus, b_res->flags));
-	size0 = calculate_memsize(size, min_size, 0,
+	if (size || min_size) {
+		min_align = calculate_mem_align(&align_test_list, max_align,
+				 size, window_alignment(bus, b_res->flags));
+		size0 = calculate_memsize(size, min_size,
 				  resource_size(b_res), min_align);
+	}
 	free_align_test_list(&align_test_list);
-	add_align = max(min_align, add_align);
-	if (children_add_size > add_size)
-		add_size = children_add_size;
-	size1 = (!realloc_head || (realloc_head && !add_size)) ? size0 :
-		calculate_memsize(size, min_size, add_size,
-				resource_size(b_res), add_align);
+
+	if ((sum_add_size - size) < add_size)
+		sum_add_size = size + add_size;
+	if (sum_add_size > size && realloc_head) {
+		min_add_align = calculate_mem_align(&align_test_add_list,
+					max_add_align, sum_add_size,
+					window_alignment(bus, b_res->flags));
+		size1 = calculate_memsize(sum_add_size, min_size,
+				 resource_size(b_res), min_add_align);
+	}
+	free_align_test_list(&align_test_add_list);
+
 	if (!size0 && !size1) {
 		if (b_res->start || b_res->end)
 			dev_info(&bus->self->dev, "disabling bridge window %pR to %pR (unused)\n",
@@ -1301,11 +1320,12 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	b_res->end = size0 + min_align - 1;
 	b_res->flags |= IORESOURCE_STARTALIGN;
 	if (size1 > size0 && realloc_head) {
-		add_to_list(realloc_head, bus->self, b_res, size1-size0, add_align);
+		add_to_list(realloc_head, bus->self, b_res, size1 - size0,
+				min_add_align);
 		dev_printk(KERN_DEBUG, &bus->self->dev, "bridge window %pR to %pR add_size %llx add_align %llx\n",
 			   b_res, &bus->busn_res,
 			   (unsigned long long) (size1 - size0),
-			   (unsigned long long) add_align);
+			   (unsigned long long) min_add_align);
 	}
 	return 0;
 }
-- 
1.8.4.5

[PATCH v8 29/61] PCI: Don't add too much optional size for hotplug bridge MMIO

[Yinghai Lu]

Current code will always add 2M for hotplug bridge MMIO even
there is child device under it already.

For example:
	40:03.0 --- 43:00.0 --- 44:02.0 -+- 45:00.0
					 \- 45:00.1

44:02.0 will need 1M as must for 45:00.0 and 45:00.1
When we calculate add_size for 44:02.0, we pass 2M as additional
size for hotplug bridge, total will be 3M.

That is different from code before changes for optional support,
or even current code that treat optional as required directly by
not passing realloc list. We only need 2M as total.

The optional size should be 1M, and total size should be 2M.

This patch change to comparing required+optional with min_sum_size to
get smaller optional size.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 0c6977c9..5e1cc7e 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1199,7 +1199,6 @@ out:
  * @type2: second match type
  * @type3: third match type
  * @min_size : the minimum memory window that must to be allocated
- * @add_size : additional optional memory window
  * @realloc_head : track the additional memory window on this list
  *
  * Calculate the size of the bus and minimal alignment which
@@ -1212,10 +1211,11 @@ out:
 static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 			 unsigned long type, unsigned long type2,
 			 unsigned long type3,
-			 resource_size_t min_size, resource_size_t add_size,
+			 resource_size_t min_size,
 			 struct list_head *realloc_head)
 {
 	struct pci_dev *dev;
+	resource_size_t min_sum_size = 0;
 	resource_size_t min_align = 0, min_add_align = 0;
 	resource_size_t max_align = 0, max_add_align = 0;
 	resource_size_t size = 0, size0 = 0, size1 = 0, sum_add_size = 0;
@@ -1227,6 +1227,11 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	if (!b_res)
 		return -ENOSPC;
 
+	if (realloc_head) {
+		min_sum_size = min_size;
+		min_size = 0;
+	}
+
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		int i;
 
@@ -1298,8 +1303,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	}
 	free_align_test_list(&align_test_list);
 
-	if ((sum_add_size - size) < add_size)
-		sum_add_size = size + add_size;
+	if (sum_add_size < min_sum_size)
+		sum_add_size = min_sum_size;
 	if (sum_add_size > size && realloc_head) {
 		min_add_align = calculate_mem_align(&align_test_add_list,
 					max_add_align, sum_add_size,
@@ -1436,7 +1441,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 {
 	struct pci_dev *dev;
 	unsigned long mask, prefmask, type2 = 0, type3 = 0;
-	resource_size_t additional_mem_size = 0, additional_io_size = 0;
+	resource_size_t min_mem_size = 0, additional_io_size = 0;
 	struct resource *b_res;
 	int ret;
 
@@ -1473,7 +1478,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 	case PCI_CLASS_BRIDGE_PCI:
 		if (bus->self->is_hotplug_bridge) {
 			additional_io_size  = pci_hotplug_io_size;
-			additional_mem_size = pci_hotplug_mem_size;
+			min_mem_size = pci_hotplug_mem_size;
 		}
 		/* Fall through */
 	default:
@@ -1493,8 +1498,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 			prefmask |= IORESOURCE_MEM_64;
 			ret = pbus_size_mem(bus, prefmask, prefmask,
 				  prefmask, prefmask,
-				  realloc_head ? 0 : additional_mem_size,
-				  additional_mem_size, realloc_head);
+				  min_mem_size, realloc_head);
 
 			/*
 			 * If successful, all non-prefetchable resources
@@ -1517,8 +1521,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 			prefmask &= ~IORESOURCE_MEM_64;
 			ret = pbus_size_mem(bus, prefmask, prefmask,
 					 prefmask, prefmask,
-					 realloc_head ? 0 : additional_mem_size,
-					 additional_mem_size, realloc_head);
+					 min_mem_size, realloc_head);
 
 			/*
 			 * If successful, only non-prefetchable resources
@@ -1527,7 +1530,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 			if (ret == 0)
 				mask = prefmask;
 			else
-				additional_mem_size += additional_mem_size;
+				min_mem_size += min_mem_size;
 
 			type2 = type3 = IORESOURCE_MEM;
 		}
@@ -1548,8 +1551,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 		 * window.
 		 */
 		pbus_size_mem(bus, mask, IORESOURCE_MEM, type2, type3,
-				realloc_head ? 0 : additional_mem_size,
-				additional_mem_size, realloc_head);
+				min_mem_size, realloc_head);
 		break;
 	}
 }
-- 
1.8.4.5

[PATCH v8 30/61] PCI: Reorder resources list for required/optional resources

[Yinghai Lu]

We try to allocate required+optional before allocate required only and
expand with optional.

At first we update size and alignment for required+optional resource.
And after that we reorder them with new alignment, but current we only
do that STARTALIGN ones.

For SIZEALIGN type resource, after add back add_size, the alignment
get changed, so need to do sorting like STARTALIGN type resources.

Also we need to reorder the sorting back after we restore
resource to required only when required+optional fail to allocate for all.

So move out the reordering code from the loop to separated function,
and call it two times accordingly.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 62 +++++++++++++++++++++++++++++--------------------
 1 file changed, 37 insertions(+), 25 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 5e1cc7e..5ea06c6 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -286,6 +286,31 @@ static inline void reset_resource(struct resource *res)
 	res->flags = 0;
 }
 
+static void sort_resources(struct list_head *head)
+{
+	struct pci_dev_resource *res1, *tmp_res, *res2;
+
+	list_for_each_entry_safe(res1, tmp_res, head, list) {
+		resource_size_t align1, size1, align2, size2;
+
+		align1 = pci_resource_alignment(res1->dev, res1->res);
+		size1 = resource_size(res1->res);
+
+		/* reorder it */
+		list_for_each_entry(res2, head, list) {
+			if (res2 == res1)
+				break;
+
+			align2 = pci_resource_alignment(res2->dev, res2->res);
+			size2 = resource_size(res2->res);
+			if (is_before(align1, size1, align2, size2)) {
+				list_move_tail(&res1->list, &res2->list);
+				break;
+			}
+		}
+	}
+}
+
 /**
  * reassign_resources_sorted() - satisfy any additional resource requests
  *
@@ -454,9 +479,9 @@ static void __assign_resources_sorted(struct list_head *head,
 	LIST_HEAD(save_head);
 	LIST_HEAD(local_fail_head);
 	struct pci_dev_resource *save_res;
-	struct pci_dev_resource *dev_res, *tmp_res, *dev_res2;
+	struct pci_dev_resource *dev_res, *tmp_res;
 	unsigned long fail_type;
-	resource_size_t add_align, align;
+	resource_size_t add_align;
 
 	/* Check if optional add_size is there */
 	if (!realloc_head || list_empty(realloc_head))
@@ -471,47 +496,32 @@ static void __assign_resources_sorted(struct list_head *head,
 	}
 
 	/* Update res in head list with add_size in realloc_head list */
-	list_for_each_entry_safe(dev_res, tmp_res, head, list) {
+	list_for_each_entry(dev_res, head, list) {
 		dev_res->res->end += get_res_add_size(realloc_head,
 							dev_res->res);
 
 		/*
 		 * There are two kinds of additional resources in the list:
-		 * 1. bridge resource  -- IORESOURCE_STARTALIGN
-		 * 2. SR-IOV resource   -- IORESOURCE_SIZEALIGN
-		 * Here just fix the additional alignment for bridge
+		 * 1. bridge resource with IORESOURCE_STARTALIGN
+		 *    need to update start to change alignment
+		 * 2. resource with IORESOURCE_SIZEALIGN
+		 *    update size above already change alignment.
 		 */
 		if (!(dev_res->res->flags & IORESOURCE_STARTALIGN))
 			continue;
 
 		add_align = get_res_add_align(realloc_head, dev_res->res);
 
-		/*
-		 * The "head" list is sorted by the alignment to make sure
-		 * resources with bigger alignment will be assigned first.
-		 * After we change the alignment of a dev_res in "head" list,
-		 * we need to reorder the list by alignment to make it
-		 * consistent.
-		 */
-		if (add_align > dev_res->res->start) {
+		if (add_align) {
 			resource_size_t r_size = resource_size(dev_res->res);
 
 			dev_res->res->start = add_align;
 			dev_res->res->end = add_align + r_size - 1;
-
-			list_for_each_entry(dev_res2, head, list) {
-				align = pci_resource_alignment(dev_res2->dev,
-							       dev_res2->res);
-				if (add_align > align) {
-					list_move_tail(&dev_res->list,
-						       &dev_res2->list);
-					break;
-				}
-			}
                }
-
 	}
 
+	sort_resources(head);
+
 	/* Try updated head list with add_size added */
 	assign_requested_resources_sorted(head, &local_fail_head);
 
@@ -553,6 +563,8 @@ static void __assign_resources_sorted(struct list_head *head,
 	}
 	free_list(&save_head);
 
+	sort_resources(head);
+
 requested_and_reassign:
 	/* Satisfy the must-have resource requests */
 	assign_requested_resources_sorted(head, fail_head);
-- 
1.8.4.5

[PATCH v8 31/61] PCI: Remove duplicated code for resource sorting

[Yinghai Lu]

Now sort_resources() and pdev_sort_resources() all have sorting
code.

As we are going to call sort_resources() several places later for
alt_size support, so choose to remove related code in
pdev_sort_resources().

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 22 +++-------------------
 1 file changed, 3 insertions(+), 19 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 5ea06c6..bcf7cba 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -214,9 +214,8 @@ static void pdev_sort_resources(struct pci_dev *dev,
 
 	for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 		struct resource *r;
-		struct pci_dev_resource *dev_res, *tmp;
+		struct pci_dev_resource *tmp;
 		resource_size_t r_align;
-		struct list_head *n;
 
 		r = &dev->resource[i];
 
@@ -239,22 +238,7 @@ static void pdev_sort_resources(struct pci_dev *dev,
 		tmp->res = r;
 		tmp->dev = dev;
 
-		/* fallback is smallest one or list is empty*/
-		n = head;
-		list_for_each_entry(dev_res, head, list) {
-			resource_size_t align;
-
-			align = __pci_resource_alignment(dev_res->dev,
-							 dev_res->res,
-							 realloc_head);
-
-			if (r_align > align) {
-				n = &dev_res->list;
-				break;
-			}
-		}
-		/* Insert it just before n*/
-		list_add_tail(&tmp->list, n);
+		list_add_tail(&tmp->list, head);
 	}
 }
 
@@ -563,9 +547,9 @@ static void __assign_resources_sorted(struct list_head *head,
 	}
 	free_list(&save_head);
 
+requested_and_reassign:
 	sort_resources(head);
 
-requested_and_reassign:
 	/* Satisfy the must-have resource requests */
 	assign_requested_resources_sorted(head, fail_head);
 
-- 
1.8.4.5

[PATCH v8 32/61] PCI: Rename pdev_sort_resources() to pdev_assign_resources_prepare()

[Yinghai Lu]

pdev_sort_resources() etc was checking devices resources and putting
resources that need to assign to one list in sorted order.

Now we don't do sorting in those functions anymore, so change to
pdev_assign_resources_prepare() instead.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index bcf7cba..74ad4ce 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -205,8 +205,8 @@ static resource_size_t __pci_resource_alignment(
 	return r_align;
 }
 
-/* Sort resources by alignment */
-static void pdev_sort_resources(struct pci_dev *dev,
+/* check resources and save to the list */
+static void pdev_assign_resources_prepare(struct pci_dev *dev,
 				 struct list_head *realloc_head,
 				 struct list_head *head)
 {
@@ -242,7 +242,7 @@ static void pdev_sort_resources(struct pci_dev *dev,
 	}
 }
 
-static void __dev_sort_resources(struct pci_dev *dev,
+static void __dev_assign_resources_prepare(struct pci_dev *dev,
 				 struct list_head *realloc_head,
 				 struct list_head *head)
 {
@@ -260,7 +260,7 @@ static void __dev_sort_resources(struct pci_dev *dev,
 			return;
 	}
 
-	pdev_sort_resources(dev, realloc_head, head);
+	pdev_assign_resources_prepare(dev, realloc_head, head);
 }
 
 static inline void reset_resource(struct resource *res)
@@ -566,7 +566,7 @@ static void pdev_assign_resources_sorted(struct pci_dev *dev,
 {
 	LIST_HEAD(head);
 
-	__dev_sort_resources(dev, add_head, &head);
+	__dev_assign_resources_prepare(dev, add_head, &head);
 	__assign_resources_sorted(&head, add_head, fail_head);
 
 }
@@ -579,7 +579,7 @@ static void pbus_assign_resources_sorted(const struct pci_bus *bus,
 	LIST_HEAD(head);
 
 	list_for_each_entry(dev, &bus->devices, bus_list)
-		__dev_sort_resources(dev, realloc_head, &head);
+		__dev_assign_resources_prepare(dev, realloc_head, &head);
 
 	__assign_resources_sorted(&head, realloc_head, fail_head);
 }
-- 
1.8.4.5

[PATCH v8 33/61] PCI: Treat ROM resource as optional during realloc

[Yinghai Lu]

Current on realloc path, we just ignore ROM resource if we can not assign
them in first try.

Treat ROM resources as optional resources,so try to allocate them together
with required ones, if can not assign them, could go with other required
resources only, and try to allocate them second time in expand path.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 37 ++++++++++++++++++++-----------------
 1 file changed, 20 insertions(+), 17 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 74ad4ce..d2d38bc 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -377,18 +377,10 @@ static void assign_requested_resources_sorted(struct list_head *head,
 		idx = res - &dev_res->dev->resource[0];
 		if (resource_size(res) &&
 		    pci_assign_resource(dev_res->dev, idx)) {
-			if (fail_head) {
-				/*
-				 * if the failed res is for ROM BAR, and it will
-				 * be enabled later, don't add it to the list
-				 */
-				if (!((idx == PCI_ROM_RESOURCE) &&
-				      (!(res->flags & IORESOURCE_ROM_ENABLE))))
-					add_to_list(fail_head,
-						    dev_res->dev, res,
-						    0 /* don't care */,
-						    0 /* don't care */);
-			}
+			if (fail_head)
+				add_to_list(fail_head, dev_res->dev, res,
+					    0 /* don't care */,
+					    0 /* don't care */);
 			reset_resource(res);
 		}
 	}
@@ -1186,6 +1178,19 @@ out:
 	return good_align;
 }
 
+static inline bool is_optional(int i)
+{
+
+	if (i == PCI_ROM_RESOURCE)
+		return true;
+
+#ifdef CONFIG_PCI_IOV
+	if (i >= PCI_IOV_RESOURCES && i <= PCI_IOV_RESOURCE_END)
+		return true;
+#endif
+
+	return false;
+}
 /**
  * pbus_size_mem() - size the memory window of a given bus
  *
@@ -1243,10 +1248,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 			r_size = resource_size(r);
 			align = pci_resource_alignment(dev, r);
-#ifdef CONFIG_PCI_IOV
-			/* put SRIOV requested res to the optional list */
-			if (realloc_head && i >= PCI_IOV_RESOURCES &&
-					i <= PCI_IOV_RESOURCE_END) {
+			/* put SRIOV/ROM res to realloc list */
+			if (realloc_head && is_optional(i)) {
 				add_to_align_test_list(&align_test_add_list,
 							align, r_size);
 				r->end = r->start - 1;
@@ -1256,7 +1259,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 					max_add_align = align;
 				continue;
 			}
-#endif
+
 			if (align > (1ULL<<37)) { /*128 Gb*/
 				dev_warn(&dev->dev, "disabling BAR %d: %pR (bad alignment %#llx)\n",
 					i, r, (unsigned long long) align);
-- 
1.8.4.5

[PATCH v8 34/61] PCI: Add debug printout during releasing partial assigned resources

[Yinghai Lu]

We try to assign required+optional at first, and we only accept the result
if all resources get allocated. Otherwise will release assigned in the
list, and try to assign required and expand to optional.

We have to do that to make sure any required has priority over any optional.

When that happens, we only print out "assigned" info, that is confusing
as it looks like same range is assigned to two peer resources at the same
time.

Add printout for releasing so we have whole picture in debug messages.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index d2d38bc..c9dfd1c 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -526,9 +526,17 @@ static void __assign_resources_sorted(struct list_head *head,
 
 	free_list(&local_fail_head);
 	/* Release assigned resource */
-	list_for_each_entry(dev_res, head, list)
-		if (dev_res->res->parent)
-			release_resource(dev_res->res);
+	list_for_each_entry(dev_res, head, list) {
+		struct resource *res = dev_res->res;
+
+		if (res->parent) {
+			dev_printk(KERN_DEBUG, &dev_res->dev->dev,
+				   "BAR %d: released %pR\n",
+				   (int)(res - &dev_res->dev->resource[0]),
+				   res);
+			release_resource(res);
+		}
+	}
 	/* Restore start/end/flags from saved list */
 	list_for_each_entry(save_res, &save_head, list) {
 		struct resource *res = save_res->res;
-- 
1.8.4.5

[PATCH v8 35/61] PCI: Simplify res reference using in __assign_resources_sorted()

[Yinghai Lu]

There are couples of dev_res->res reference, to make code more readable
use res instead of dev_res->res directly.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index c9dfd1c..d575c56 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -458,6 +458,7 @@ static void __assign_resources_sorted(struct list_head *head,
 	struct pci_dev_resource *dev_res, *tmp_res;
 	unsigned long fail_type;
 	resource_size_t add_align;
+	struct resource *res;
 
 	/* Check if optional add_size is there */
 	if (!realloc_head || list_empty(realloc_head))
@@ -473,8 +474,8 @@ static void __assign_resources_sorted(struct list_head *head,
 
 	/* Update res in head list with add_size in realloc_head list */
 	list_for_each_entry(dev_res, head, list) {
-		dev_res->res->end += get_res_add_size(realloc_head,
-							dev_res->res);
+		res = dev_res->res;
+		res->end += get_res_add_size(realloc_head, res);
 
 		/*
 		 * There are two kinds of additional resources in the list:
@@ -483,16 +484,16 @@ static void __assign_resources_sorted(struct list_head *head,
 		 * 2. resource with IORESOURCE_SIZEALIGN
 		 *    update size above already change alignment.
 		 */
-		if (!(dev_res->res->flags & IORESOURCE_STARTALIGN))
+		if (!(res->flags & IORESOURCE_STARTALIGN))
 			continue;
 
-		add_align = get_res_add_align(realloc_head, dev_res->res);
+		add_align = get_res_add_align(realloc_head, res);
 
 		if (add_align) {
-			resource_size_t r_size = resource_size(dev_res->res);
+			resource_size_t r_size = resource_size(res);
 
-			dev_res->res->start = add_align;
-			dev_res->res->end = add_align + r_size - 1;
+			res->start = add_align;
+			res->end = add_align + r_size - 1;
                }
 	}
 
@@ -514,21 +515,21 @@ static void __assign_resources_sorted(struct list_head *head,
 	/* check failed type */
 	fail_type = pci_fail_res_type_mask(&local_fail_head);
 	/* remove not need to be released assigned res from head list etc */
-	list_for_each_entry_safe(dev_res, tmp_res, head, list)
-		if (dev_res->res->parent &&
-		    !pci_need_to_release(fail_type, dev_res->res)) {
+	list_for_each_entry_safe(dev_res, tmp_res, head, list) {
+		res = dev_res->res;
+		if (res->parent && !pci_need_to_release(fail_type, res)) {
 			/* remove it from realloc_head list */
-			remove_from_list(realloc_head, dev_res->res);
-			remove_from_list(&save_head, dev_res->res);
+			remove_from_list(realloc_head, res);
+			remove_from_list(&save_head, res);
 			list_del(&dev_res->list);
 			kfree(dev_res);
 		}
+	}
 
 	free_list(&local_fail_head);
 	/* Release assigned resource */
 	list_for_each_entry(dev_res, head, list) {
-		struct resource *res = dev_res->res;
-
+		res = dev_res->res;
 		if (res->parent) {
 			dev_printk(KERN_DEBUG, &dev_res->dev->dev,
 				   "BAR %d: released %pR\n",
@@ -539,8 +540,7 @@ static void __assign_resources_sorted(struct list_head *head,
 	}
 	/* Restore start/end/flags from saved list */
 	list_for_each_entry(save_res, &save_head, list) {
-		struct resource *res = save_res->res;
-
+		res = save_res->res;
 		res->start = save_res->start;
 		res->end = save_res->end;
 		res->flags = save_res->flags;
-- 
1.8.4.5

[PATCH v8 36/61] PCI: Add __add_to_list()

[Yinghai Lu]

For alt_size support, we will add more entries to realloc list.

Add new __add_to_list() to take alt_size, alt_align.

And simplify add_to_list() not to take add/alt input.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 51 ++++++++++++++++++++++++++++++-------------------
 1 file changed, 31 insertions(+), 20 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index d575c56..b86417d 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -66,6 +66,8 @@ struct pci_dev_resource {
 	resource_size_t end;
 	resource_size_t add_size;
 	resource_size_t min_align;
+	resource_size_t alt_size;
+	resource_size_t alt_align;
 	unsigned long flags;
 };
 
@@ -88,15 +90,16 @@ static void free_list(struct list_head *head)
  * @add_size:	additional size to be optionally added
  *              to the resource
  */
-static int add_to_list(struct list_head *head,
+static int __add_to_list(struct list_head *head,
 		 struct pci_dev *dev, struct resource *res,
-		 resource_size_t add_size, resource_size_t min_align)
+		 resource_size_t add_size, resource_size_t min_align,
+		 resource_size_t alt_size, resource_size_t alt_align)
 {
 	struct pci_dev_resource *tmp;
 
 	tmp = kzalloc(sizeof(*tmp), GFP_KERNEL);
 	if (!tmp) {
-		pr_warn("add_to_list: kmalloc() failed!\n");
+		pr_warn("__add_to_list: kmalloc() failed!\n");
 		return -ENOMEM;
 	}
 
@@ -107,12 +110,20 @@ static int add_to_list(struct list_head *head,
 	tmp->flags = res->flags;
 	tmp->add_size = add_size;
 	tmp->min_align = min_align;
+	tmp->alt_size = alt_size;
+	tmp->alt_align = alt_align;
 
 	list_add(&tmp->list, head);
 
 	return 0;
 }
 
+static int add_to_list(struct list_head *head,
+		 struct pci_dev *dev, struct resource *res)
+{
+	return __add_to_list(head, dev, res, 0, 0, 0, 0);
+}
+
 static void remove_from_list(struct list_head *head,
 				 struct resource *res)
 {
@@ -378,9 +389,7 @@ static void assign_requested_resources_sorted(struct list_head *head,
 		if (resource_size(res) &&
 		    pci_assign_resource(dev_res->dev, idx)) {
 			if (fail_head)
-				add_to_list(fail_head, dev_res->dev, res,
-					    0 /* don't care */,
-					    0 /* don't care */);
+				add_to_list(fail_head, dev_res->dev, res);
 			reset_resource(res);
 		}
 	}
@@ -466,7 +475,7 @@ static void __assign_resources_sorted(struct list_head *head,
 
 	/* Save original start, end, flags etc at first */
 	list_for_each_entry(dev_res, head, list) {
-		if (add_to_list(&save_head, dev_res->dev, dev_res->res, 0, 0)) {
+		if (add_to_list(&save_head, dev_res->dev, dev_res->res)) {
 			free_list(&save_head);
 			goto requested_and_reassign;
 		}
@@ -1057,8 +1066,8 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 	b_res->end = b_res->start + size0 - 1;
 	b_res->flags |= IORESOURCE_STARTALIGN;
 	if (size1 > size0 && realloc_head) {
-		add_to_list(realloc_head, bus->self, b_res, size1-size0,
-			    min_align);
+		__add_to_list(realloc_head, bus->self, b_res,
+			      size1 - size0, min_align, 0, 0);
 		dev_printk(KERN_DEBUG, &bus->self->dev, "bridge window %pR to %pR add_size %llx\n",
 			   b_res, &bus->busn_res,
 			   (unsigned long long)size1-size0);
@@ -1261,7 +1270,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 				add_to_align_test_list(&align_test_add_list,
 							align, r_size);
 				r->end = r->start - 1;
-				add_to_list(realloc_head, dev, r, r_size, 0/* don't care */);
+				__add_to_list(realloc_head, dev, r,
+					      r_size, align, 0, 0);
 				sum_add_size += r_size;
 				if (align > max_add_align)
 					max_add_align = align;
@@ -1332,8 +1342,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	b_res->end = size0 + min_align - 1;
 	b_res->flags |= IORESOURCE_STARTALIGN;
 	if (size1 > size0 && realloc_head) {
-		add_to_list(realloc_head, bus->self, b_res, size1 - size0,
-				min_add_align);
+		__add_to_list(realloc_head, bus->self, b_res, size1 - size0,
+				min_add_align, 0, 0);
 		dev_printk(KERN_DEBUG, &bus->self->dev, "bridge window %pR to %pR add_size %llx add_align %llx\n",
 			   b_res, &bus->busn_res,
 			   (unsigned long long) (size1 - size0),
@@ -1370,8 +1380,8 @@ static void pci_bus_size_cardbus(struct pci_bus *bus,
 	b_res[0].flags |= IORESOURCE_IO | IORESOURCE_STARTALIGN;
 	if (realloc_head) {
 		b_res[0].end -= pci_cardbus_io_size;
-		add_to_list(realloc_head, bridge, b_res, pci_cardbus_io_size,
-				pci_cardbus_io_size);
+		__add_to_list(realloc_head, bridge, b_res,
+			      pci_cardbus_io_size, pci_cardbus_io_size, 0, 0);
 	}
 
 handle_b_res_1:
@@ -1382,8 +1392,8 @@ handle_b_res_1:
 	b_res[1].flags |= IORESOURCE_IO | IORESOURCE_STARTALIGN;
 	if (realloc_head) {
 		b_res[1].end -= pci_cardbus_io_size;
-		add_to_list(realloc_head, bridge, b_res+1, pci_cardbus_io_size,
-				 pci_cardbus_io_size);
+		__add_to_list(realloc_head, bridge, b_res + 1,
+			      pci_cardbus_io_size, pci_cardbus_io_size, 0, 0);
 	}
 
 handle_b_res_2:
@@ -1420,8 +1430,9 @@ handle_b_res_2:
 				  IORESOURCE_STARTALIGN;
 		if (realloc_head) {
 			b_res[2].end -= pci_cardbus_mem_size;
-			add_to_list(realloc_head, bridge, b_res+2,
-				 pci_cardbus_mem_size, pci_cardbus_mem_size);
+			__add_to_list(realloc_head, bridge, b_res + 2,
+				pci_cardbus_mem_size, pci_cardbus_mem_size,
+				0, 0);
 		}
 
 		/* reduce that to half */
@@ -1436,8 +1447,8 @@ handle_b_res_3:
 	b_res[3].flags |= IORESOURCE_MEM | IORESOURCE_STARTALIGN;
 	if (realloc_head) {
 		b_res[3].end -= b_res_3_size;
-		add_to_list(realloc_head, bridge, b_res+3, b_res_3_size,
-				 pci_cardbus_mem_size);
+		__add_to_list(realloc_head, bridge, b_res + 3,
+				b_res_3_size, pci_cardbus_mem_size, 0, 0);
 	}
 
 handle_done:
-- 
1.8.4.5

[PATCH v8 37/61] PCI: Cache window alignment value during bus sizing

[Yinghai Lu]

There are several calling to window_alignment(), and we will have more
for alt_size support, cache the value instead of keeping on getting it.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index b86417d..deed016 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1241,6 +1241,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 					mask | IORESOURCE_PREFETCH, type);
 	LIST_HEAD(align_test_list);
 	LIST_HEAD(align_test_add_list);
+	resource_size_t window_align;
 
 	if (!b_res)
 		return -ENOSPC;
@@ -1250,6 +1251,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		min_size = 0;
 	}
 
+	window_align = window_alignment(bus, b_res->flags);
+
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		int i;
 
@@ -1311,10 +1314,10 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		}
 	}
 
-	max_align = max(max_align, window_alignment(bus, b_res->flags));
+	max_align = max(max_align, window_align);
 	if (size || min_size) {
 		min_align = calculate_mem_align(&align_test_list, max_align,
-				 size, window_alignment(bus, b_res->flags));
+						size, window_align);
 		size0 = calculate_memsize(size, min_size,
 				  resource_size(b_res), min_align);
 	}
@@ -1325,7 +1328,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	if (sum_add_size > size && realloc_head) {
 		min_add_align = calculate_mem_align(&align_test_add_list,
 					max_add_align, sum_add_size,
-					window_alignment(bus, b_res->flags));
+					window_align);
 		size1 = calculate_memsize(sum_add_size, min_size,
 				 resource_size(b_res), min_add_align);
 	}
-- 
1.8.4.5

[PATCH v8 38/61] PCI: Check if resource is allocated before trying to assign one

[Yinghai Lu]

In following alt_size support, we will call pci_assign_resource() several
times on one resource list, and some resources could have been assigned
already.

Skip allocated resource in the list, as pci_assign_resource()
only can handle not assigned resource.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index deed016..ca34346 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -386,7 +386,7 @@ static void assign_requested_resources_sorted(struct list_head *head,
 	list_for_each_entry(dev_res, head, list) {
 		res = dev_res->res;
 		idx = res - &dev_res->dev->resource[0];
-		if (resource_size(res) &&
+		if (!res->parent && resource_size(res) &&
 		    pci_assign_resource(dev_res->dev, idx)) {
 			if (fail_head)
 				add_to_list(fail_head, dev_res->dev, res);
-- 
1.8.4.5

[PATCH v8 39/61] PCI: Separate out save_resources()/restore_resources()

[Yinghai Lu]

We need to save/restore resources several times for alt_size support,
separate the save_resources()/resources() to save some lines later.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 49 ++++++++++++++++++++++++++++++-------------------
 1 file changed, 30 insertions(+), 19 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index ca34346..dc83c2c 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -436,6 +436,29 @@ static bool pci_need_to_release(unsigned long mask, struct resource *res)
 	return false;	/* should not get here */
 }
 
+static bool save_resources(struct list_head *head,
+			   struct list_head *save_head)
+{
+	struct pci_dev_resource *dev_res;
+
+	/* Save original start, end, flags etc at first */
+	list_for_each_entry(dev_res, head, list)
+		if (add_to_list(save_head, dev_res->dev, dev_res->res)) {
+			free_list(save_head);
+			return false;
+		}
+
+	return true;
+}
+
+static void restore_resource(struct pci_dev_resource *save_res,
+			     struct resource *res)
+{
+	res->start = save_res->start;
+	res->end = save_res->end;
+	res->flags = save_res->flags;
+}
+
 static void __assign_resources_sorted(struct list_head *head,
 				 struct list_head *realloc_head,
 				 struct list_head *fail_head)
@@ -473,13 +496,8 @@ static void __assign_resources_sorted(struct list_head *head,
 	if (!realloc_head || list_empty(realloc_head))
 		goto requested_and_reassign;
 
-	/* Save original start, end, flags etc at first */
-	list_for_each_entry(dev_res, head, list) {
-		if (add_to_list(&save_head, dev_res->dev, dev_res->res)) {
-			free_list(&save_head);
-			goto requested_and_reassign;
-		}
-	}
+	if (!save_resources(head, &save_head))
+		goto requested_and_reassign;
 
 	/* Update res in head list with add_size in realloc_head list */
 	list_for_each_entry(dev_res, head, list) {
@@ -548,12 +566,9 @@ static void __assign_resources_sorted(struct list_head *head,
 		}
 	}
 	/* Restore start/end/flags from saved list */
-	list_for_each_entry(save_res, &save_head, list) {
-		res = save_res->res;
-		res->start = save_res->start;
-		res->end = save_res->end;
-		res->flags = save_res->flags;
-	}
+	list_for_each_entry(save_res, &save_head, list)
+		restore_resource(save_res, save_res->res);
+
 	free_list(&save_head);
 
 requested_and_reassign:
@@ -1981,9 +1996,7 @@ again:
 	list_for_each_entry(fail_res, &fail_head, list) {
 		struct resource *res = fail_res->res;
 
-		res->start = fail_res->start;
-		res->end = fail_res->end;
-		res->flags = fail_res->flags;
+		restore_resource(fail_res, res);
 		if (fail_res->dev->subordinate) {
 			res->flags = 0;
 			/* last or third times and later */
@@ -2067,9 +2080,7 @@ again:
 	list_for_each_entry(fail_res, &fail_head, list) {
 		struct resource *res = fail_res->res;
 
-		res->start = fail_res->start;
-		res->end = fail_res->end;
-		res->flags = fail_res->flags;
+		restore_resource(fail_res, res);
 		if (fail_res->dev->subordinate) {
 			res->flags = 0;
 			/* last time */
-- 
1.8.4.5

[PATCH v8 40/61] PCI: Move comment to pci_need_to_release()

[Yinghai Lu]

Move comment from caller to pci_need_to_release(), as we will have one new
caller for alt_size support.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 27 +++++++++++++++------------
 1 file changed, 15 insertions(+), 12 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index dc83c2c..09844d9 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -415,6 +415,20 @@ static unsigned long pci_fail_res_type_mask(struct list_head *fail_head)
 
 static bool pci_need_to_release(unsigned long mask, struct resource *res)
 {
+	/*
+	 * Separate three resource type checking if we need to release
+	 * assigned resource.
+	 *	1. if there is io port assign fail, will release assigned
+	 *	   io port.
+	 *	2. if there is pref mmio assign fail, release assigned
+	 *	   pref mmio.
+	 *	   if assigned pref mmio's parent is non-pref mmio and there
+	 *	   is non-pref mmio assign fail, will release that assigned
+	 *	   pref mmio.
+	 *	3. if there is non-pref mmio assign fail or pref mmio
+	 *	   assigned fail, will release assigned non-pref mmio.
+	 */
+
 	if (res->flags & IORESOURCE_IO)
 		return !!(mask & IORESOURCE_IO);
 
@@ -471,19 +485,8 @@ static void __assign_resources_sorted(struct list_head *head,
 	 *  if could do that, could get out early.
 	 *  if could not do that, we still try to assign requested at first,
 	 *    then try to reassign add_size for some resources.
-	 *
-	 * Separate three resource type checking if we need to release
-	 * assigned resource after requested + add_size try.
-	 *	1. if there is io port assign fail, will release assigned
-	 *	   io port.
-	 *	2. if there is pref mmio assign fail, release assigned
-	 *	   pref mmio.
-	 *	   if assigned pref mmio's parent is non-pref mmio and there
-	 *	   is non-pref mmio assign fail, will release that assigned
-	 *	   pref mmio.
-	 *	3. if there is non-pref mmio assign fail or pref mmio
-	 *	   assigned fail, will release assigned non-pref mmio.
 	 */
+
 	LIST_HEAD(save_head);
 	LIST_HEAD(local_fail_head);
 	struct pci_dev_resource *save_res;
-- 
1.8.4.5

[PATCH v8 41/61] PCI: Separate required+optional assigning to another function

[Yinghai Lu]

__assign_resources_sorted() is getting too big if we put alt_size support
into it.  Split out required+optional assigning code to another function.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 47 +++++++++++++++++++++++++++--------------------
 1 file changed, 27 insertions(+), 20 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 09844d9..c49bfde 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -473,20 +473,9 @@ static void restore_resource(struct pci_dev_resource *save_res,
 	res->flags = save_res->flags;
 }
 
-static void __assign_resources_sorted(struct list_head *head,
-				 struct list_head *realloc_head,
-				 struct list_head *fail_head)
+static bool __assign_resources_required_optional_sorted(struct list_head *head,
+				 struct list_head *realloc_head)
 {
-	/*
-	 * Should not assign requested resources at first.
-	 *   they could be adjacent, so later reassign can not reallocate
-	 *   them one by one in parent resource window.
-	 * Try to assign requested + add_size at beginning
-	 *  if could do that, could get out early.
-	 *  if could not do that, we still try to assign requested at first,
-	 *    then try to reassign add_size for some resources.
-	 */
-
 	LIST_HEAD(save_head);
 	LIST_HEAD(local_fail_head);
 	struct pci_dev_resource *save_res;
@@ -495,12 +484,8 @@ static void __assign_resources_sorted(struct list_head *head,
 	resource_size_t add_align;
 	struct resource *res;
 
-	/* Check if optional add_size is there */
-	if (!realloc_head || list_empty(realloc_head))
-		goto requested_and_reassign;
-
 	if (!save_resources(head, &save_head))
-		goto requested_and_reassign;
+		return false;
 
 	/* Update res in head list with add_size in realloc_head list */
 	list_for_each_entry(dev_res, head, list) {
@@ -539,7 +524,8 @@ static void __assign_resources_sorted(struct list_head *head,
 			remove_from_list(realloc_head, dev_res->res);
 		free_list(&save_head);
 		free_list(head);
-		return;
+
+		return true;
 	}
 
 	/* check failed type */
@@ -574,7 +560,28 @@ static void __assign_resources_sorted(struct list_head *head,
 
 	free_list(&save_head);
 
-requested_and_reassign:
+	return false;
+}
+
+static void __assign_resources_sorted(struct list_head *head,
+				 struct list_head *realloc_head,
+				 struct list_head *fail_head)
+{
+	/*
+	 * Should not assign required resources at first.
+	 *   they could be adjacent, so later reassign can not reallocate
+	 *   them one by one in parent resource window.
+	 * Try to assign required + optional at beginning
+	 *  if could do that, could get out early.
+	 *  if could not do that, we still try to assign required at first,
+	 *    then try to reassign add_size for some resources.
+	 */
+
+	/* Check required+optional add */
+	if (realloc_head && !list_empty(realloc_head) &&
+	    __assign_resources_required_optional_sorted(head, realloc_head))
+		return;
+
 	sort_resources(head);
 
 	/* Satisfy the must-have resource requests */
-- 
1.8.4.5

[PATCH v8 42/61] PCI: Skip required+optional if there is no optional

[Yinghai Lu]

If the bridge does not support hotplug and has no child with sriov support,
We will not have optional resources. We could get out early and
don't try required+optional allocation.

Also in the loop that update res with optional add info, skip resource
that add_size is 0.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 32 +++++++++++++++++++++++++++++---
 1 file changed, 29 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index c49bfde..5143ea2 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -450,6 +450,24 @@ static bool pci_need_to_release(unsigned long mask, struct resource *res)
 	return false;	/* should not get here */
 }
 
+static bool has_addon(struct list_head *head,
+			struct list_head *realloc_head)
+{
+	int add_count = 0;
+	struct pci_dev_resource *dev_res, *tmp_res;
+
+	/* check if we have add really */
+	list_for_each_entry(dev_res, head, list) {
+		tmp_res = res_to_dev_res(realloc_head, dev_res->res);
+		if (!tmp_res || !tmp_res->add_size)
+			continue;
+
+		add_count++;
+	}
+
+	return add_count != 0;
+}
+
 static bool save_resources(struct list_head *head,
 			   struct list_head *save_head)
 {
@@ -481,16 +499,24 @@ static bool __assign_resources_required_optional_sorted(struct list_head *head,
 	struct pci_dev_resource *save_res;
 	struct pci_dev_resource *dev_res, *tmp_res;
 	unsigned long fail_type;
-	resource_size_t add_align;
+	resource_size_t add_align, add_size;
 	struct resource *res;
 
+	if (!has_addon(head, realloc_head))
+		return false;
+
 	if (!save_resources(head, &save_head))
 		return false;
 
 	/* Update res in head list with add_size in realloc_head list */
 	list_for_each_entry(dev_res, head, list) {
 		res = dev_res->res;
-		res->end += get_res_add_size(realloc_head, res);
+		add_size = get_res_add_size(realloc_head, res);
+
+		if (!add_size)
+			continue;
+
+		res->end += add_size;
 
 		/*
 		 * There are two kinds of additional resources in the list:
@@ -578,7 +604,7 @@ static void __assign_resources_sorted(struct list_head *head,
 	 */
 
 	/* Check required+optional add */
-	if (realloc_head && !list_empty(realloc_head) &&
+	if (realloc_head &&
 	    __assign_resources_required_optional_sorted(head, realloc_head))
 		return;
 
-- 
1.8.4.5

[PATCH v8 43/61] PCI: Move saved required resource list out of required+optional assigning

[Yinghai Lu]

We will need to share saved required list for alt_size support, so move
it out from required+optional assigning.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 30 ++++++++++++++++--------------
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 5143ea2..ede619c 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -456,6 +456,9 @@ static bool has_addon(struct list_head *head,
 	int add_count = 0;
 	struct pci_dev_resource *dev_res, *tmp_res;
 
+	if (!realloc_head)
+		return false;
+
 	/* check if we have add really */
 	list_for_each_entry(dev_res, head, list) {
 		tmp_res = res_to_dev_res(realloc_head, dev_res->res);
@@ -492,9 +495,9 @@ static void restore_resource(struct pci_dev_resource *save_res,
 }
 
 static bool __assign_resources_required_optional_sorted(struct list_head *head,
+				 struct list_head *save_head,
 				 struct list_head *realloc_head)
 {
-	LIST_HEAD(save_head);
 	LIST_HEAD(local_fail_head);
 	struct pci_dev_resource *save_res;
 	struct pci_dev_resource *dev_res, *tmp_res;
@@ -502,12 +505,6 @@ static bool __assign_resources_required_optional_sorted(struct list_head *head,
 	resource_size_t add_align, add_size;
 	struct resource *res;
 
-	if (!has_addon(head, realloc_head))
-		return false;
-
-	if (!save_resources(head, &save_head))
-		return false;
-
 	/* Update res in head list with add_size in realloc_head list */
 	list_for_each_entry(dev_res, head, list) {
 		res = dev_res->res;
@@ -548,7 +545,6 @@ static bool __assign_resources_required_optional_sorted(struct list_head *head,
 		/* Remove head list from realloc_head list */
 		list_for_each_entry(dev_res, head, list)
 			remove_from_list(realloc_head, dev_res->res);
-		free_list(&save_head);
 		free_list(head);
 
 		return true;
@@ -562,7 +558,7 @@ static bool __assign_resources_required_optional_sorted(struct list_head *head,
 		if (res->parent && !pci_need_to_release(fail_type, res)) {
 			/* remove it from realloc_head list */
 			remove_from_list(realloc_head, res);
-			remove_from_list(&save_head, res);
+			remove_from_list(save_head, res);
 			list_del(&dev_res->list);
 			kfree(dev_res);
 		}
@@ -581,11 +577,9 @@ static bool __assign_resources_required_optional_sorted(struct list_head *head,
 		}
 	}
 	/* Restore start/end/flags from saved list */
-	list_for_each_entry(save_res, &save_head, list)
+	list_for_each_entry(save_res, save_head, list)
 		restore_resource(save_res, save_res->res);
 
-	free_list(&save_head);
-
 	return false;
 }
 
@@ -603,16 +597,24 @@ static void __assign_resources_sorted(struct list_head *head,
 	 *    then try to reassign add_size for some resources.
 	 */
 
+	LIST_HEAD(save_head);
+
 	/* Check required+optional add */
-	if (realloc_head &&
-	    __assign_resources_required_optional_sorted(head, realloc_head))
+	if (has_addon(head, realloc_head) &&
+	    save_resources(head, &save_head) &&
+	    __assign_resources_required_optional_sorted(head, &save_head,
+					       realloc_head)) {
+		free_list(&save_head);
 		return;
+	}
 
 	sort_resources(head);
 
 	/* Satisfy the must-have resource requests */
 	assign_requested_resources_sorted(head, fail_head);
 
+	free_list(&save_head);
+
 	/* Try to satisfy any additional optional resource
 		requests */
 	if (realloc_head)
-- 
1.8.4.5

[PATCH v8 44/61] PCI: Add alt_size ressource allocation support

[Yinghai Lu]

On system with several pcie switches, BIOS allocate very tight resources
to the bridge bar, and it is not aligned to min_align as kernel allocation
code.

For example:
  02:03.0---0c:00.0---0d:04.0---18:00.0

  18:00.0 need 0x10000000, and 0x00010000.
BIOS only allocate 0x10100000 to 0d:04.0 and above bridges.
Later after using /sys/bus/pci/devices/0000:0c:00.0/remove to remove 0c:00.0,
rescan with /sys/bus/pci/rescan can not allocate 0x18000000 to 0c:00.0.
as current min_align solution will need 0x18000000.

Another example:
  00:1c.0---02:00.0---03:01.0---04:00.0---05:19.0---06:00.0

  06:00.0 need 0x4000000 and 0x800000.
BIOS only allocate 0x4800000 to 05:19.0 and 04:00.0.
when 05:19.0 get removed via /sys/bus/pci/devices/0000:05:19.0/remove,
rescan with /sys/bus/pci/rescan will fail.
 pci 0000:05:19.0: BAR 14: no space for [mem size 0x06000000]
 pci 0000:05:19.0: BAR 14: failed to assign [mem size 0x06000000]
 pci 0000:06:00.0: BAR 2: no space for [mem size 0x04000000 64bit]
 pci 0000:06:00.0: BAR 2: failed to assign [mem size 0x04000000 64bit]
 pci 0000:06:00.0: BAR 0: no space for [mem size 0x00800000]
 pci 0000:06:00.0: BAR 0: failed to assign [mem size 0x00800000]
current code try to use align 0x2000000 and size 0x6000000, but parent
bridge only have 0x4800000.

Introduce alt_align/alt_size and store them in realloc list in addition
to addon info, and will try it after min_align/min_size allocation fails.

The alt_align is max_align, and alt_size is aligned size with bridge
minimum window alignment.

On my test setup:
  00:1c.7---61:00.0---62:00.0

  62:00.0 needs 0x800000 and 0x20000, and 00:1c.7 only have 9M allocated
for mmio, with this patch we have

 pci 0000:61:00.0: bridge window [mem 0x00400000-0x00ffffff] to [bus 62]
   add_size 0 add_align 0 alt_size 900000 alt_align 800000
   req_size c00000 req_align 400000
 pci 0000:61:00.0: BAR 14: no space for [mem size 0x00c00000]
 pci 0000:61:00.0: BAR 14: failed to assign [mem size 0x00c00000]
 pci 0000:61:00.0: BAR 14: assigned [mem 0xdf000000-0xdf8fffff]
 pci 0000:62:00.0: BAR 0: assigned [mem 0xdf000000-0xdf7fffff pref]
 pci 0000:62:00.0: BAR 1: assigned [mem 0xdf800000-0xdf81ffff]
 pci 0000:61:00.0: PCI bridge to [bus 62]
 pci 0000:61:00.0:   bridge window [io  0x6000-0x6fff]
 pci 0000:61:00.0:   bridge window [mem 0xdf000000-0xdf8fffff]
 pci 0000:00:1c.7: PCI bridge to [bus 61-68]
 pci 0000:00:1c.7:   bridge window [io  0x6000-0x6fff]
 pci 0000:00:1c.7:   bridge window [mem 0xdf000000-0xdf8fffff]

So for 61:00.0 first try with 12M fails, and second try with 9M the
alt_size works. Later 62:00.0 get correct resource allocated too.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=100451
Reported-by: Yijing Wang <wangyijing@huawei.com>
Tested-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 203 +++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 191 insertions(+), 12 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index ede619c..dea0344 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -323,7 +323,7 @@ static void reassign_resources_sorted(struct list_head *realloc_head,
 {
 	struct resource *res;
 	struct pci_dev_resource *add_res, *tmp;
-	resource_size_t add_size, align;
+	resource_size_t add_size, align, r_size;
 	int idx;
 
 	list_for_each_entry_safe(add_res, tmp, realloc_head, list) {
@@ -339,12 +339,23 @@ static void reassign_resources_sorted(struct list_head *realloc_head,
 		idx = res - &add_res->dev->resource[0];
 		add_size = add_res->add_size;
 		align = add_res->min_align;
-		if (!resource_size(res)) {
+		if (!add_size || !align) /* alt_size only */
+			goto out;
+
+		r_size = resource_size(res);
+		if (!r_size) {
 			res->start = align;
 			res->end = res->start + add_size - 1;
 			if (pci_assign_resource(add_res->dev, idx))
 				reset_resource(res);
 		} else {
+			/* could just assigned with alt, add difference ? */
+			resource_size_t size;
+
+			size = add_res->end - add_res->start + 1;
+			if (r_size < size)
+				add_size += size - r_size;
+
 			res->flags |= add_res->flags &
 				 (IORESOURCE_STARTALIGN|IORESOURCE_SIZEALIGN);
 			if (pci_reassign_resource(add_res->dev, idx,
@@ -583,6 +594,104 @@ static bool __assign_resources_required_optional_sorted(struct list_head *head,
 	return false;
 }
 
+static bool __has_alt(struct list_head *head,
+		    struct list_head *realloc_head)
+{
+	int alt_count = 0;
+	struct pci_dev_resource *dev_res, *alt_res;
+
+	if (!realloc_head)
+		return false;
+
+	/* check if we have alt really */
+	list_for_each_entry(dev_res, head, list) {
+		alt_res = res_to_dev_res(realloc_head, dev_res->res);
+		if (!alt_res || !alt_res->alt_size)
+			continue;
+
+		alt_count++;
+	}
+
+	if (!alt_count)
+		return false;
+
+	return true;
+}
+
+static void __assign_resources_alt_sorted(struct list_head *head,
+				 struct list_head *save_head,
+				 struct list_head *realloc_head,
+				 struct list_head *local_fail_head)
+{
+	LIST_HEAD(local_alt_fail_head);
+	struct pci_dev_resource *dev_res;
+	struct pci_dev_resource *alt_res, *fail_res, *save_res;
+	unsigned long fail_type;
+	struct resource *res;
+
+	/* check failed type */
+	fail_type = pci_fail_res_type_mask(local_fail_head);
+	/* release resource with same type that failes */
+	list_for_each_entry(dev_res, head, list) {
+		res = dev_res->res;
+		if (res->parent) {
+			if (!pci_need_to_release(fail_type, res))
+				continue;
+
+			/*
+			 * have to use saved info, as resource that does not
+			 * have addon/alt is not in realloc list.
+			 */
+			save_res = res_to_dev_res(save_head, res);
+			if (!save_res)
+				continue;
+
+			dev_printk(KERN_DEBUG, &dev_res->dev->dev,
+				   "BAR %d: released %pR\n",
+				   (int)(res - &dev_res->dev->resource[0]),
+				   res);
+			release_resource(dev_res->res);
+			restore_resource(save_res, res);
+		} else {
+			/* restore fail one */
+			fail_res = res_to_dev_res(local_fail_head, res);
+			if (fail_res) {
+				restore_resource(fail_res, res);
+				remove_from_list(local_fail_head, res);
+			}
+		}
+
+		alt_res = res_to_dev_res(realloc_head, res);
+		if (!alt_res || !alt_res->alt_size)
+			continue;
+
+		/* change res to alt */
+		if (res->flags & IORESOURCE_STARTALIGN)
+			res->start = alt_res->alt_align;
+		else
+			res->start = 0;
+		res->end = res->start + alt_res->alt_size - 1;
+	}
+
+	sort_resources(head);
+	/* Satisfy the alt resource requests */
+	assign_requested_resources_sorted(head, &local_alt_fail_head);
+
+	/* update local fail list */
+	list_for_each_entry(fail_res, &local_alt_fail_head, list) {
+		res = fail_res->res;
+		dev_res = res_to_dev_res(realloc_head, res);
+		/* change res back to required */
+		if (dev_res && dev_res->alt_size)
+			restore_resource(dev_res, res);
+
+		if (!res_to_dev_res(local_fail_head, res))
+			add_to_list(local_fail_head, fail_res->dev, res);
+		reset_resource(res);
+	}
+	free_list(&local_alt_fail_head);
+}
+
 static void __assign_resources_sorted(struct list_head *head,
 				 struct list_head *realloc_head,
 				 struct list_head *fail_head)
@@ -598,6 +707,8 @@ static void __assign_resources_sorted(struct list_head *head,
 	 */
 
 	LIST_HEAD(save_head);
+	LIST_HEAD(local_fail_head);
+	bool has_alt;
 
 	/* Check required+optional add */
 	if (has_addon(head, realloc_head) &&
@@ -610,15 +721,29 @@ static void __assign_resources_sorted(struct list_head *head,
 
 	sort_resources(head);
 
+	has_alt = __has_alt(head, realloc_head);
+	if (has_alt && list_empty(&save_head))
+		save_resources(head, &save_head);
+
 	/* Satisfy the must-have resource requests */
-	assign_requested_resources_sorted(head, fail_head);
+	assign_requested_resources_sorted(head, &local_fail_head);
+
+	if (has_alt && !list_empty(&local_fail_head) && !list_empty(&save_head))
+		__assign_resources_alt_sorted(head, &save_head,
+					      realloc_head,
+					      &local_fail_head);
 
 	free_list(&save_head);
 
-	/* Try to satisfy any additional optional resource
-		requests */
+	/* Try to satisfy any additional optional resource requests */
 	if (realloc_head)
 		reassign_resources_sorted(realloc_head, head);
+
+	if (fail_head)
+		list_splice_tail(&local_fail_head, fail_head);
+	else
+		free_list(&local_fail_head);
+
 	free_list(head);
 }
 
@@ -1294,6 +1419,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 					mask | IORESOURCE_PREFETCH, type);
 	LIST_HEAD(align_test_list);
 	LIST_HEAD(align_test_add_list);
+	resource_size_t alt_size = 0, alt_align = 0;
 	resource_size_t window_align;
 
 	if (!b_res)
@@ -1351,6 +1477,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 			if (realloc_head) {
 				resource_size_t add_r_size, add_align;
+				struct pci_dev_resource *dev_res;
 
 				add_r_size = get_res_add_size(realloc_head, r);
 				add_align = get_res_add_align(realloc_head, r);
@@ -1363,6 +1490,17 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 				sum_add_size += r_size + add_r_size;
 				if (add_align > max_add_align)
 					max_add_align = add_align;
+
+				dev_res = res_to_dev_res(realloc_head, r);
+				if (dev_res && dev_res->alt_size) {
+					alt_size += dev_res->alt_size;
+					if (alt_align < dev_res->alt_align)
+						alt_align = dev_res->alt_align;
+				} else if (r_size > 1) {
+					alt_size += r_size;
+					if (alt_align < align)
+						alt_align = align;
+				}
 			}
 		}
 	}
@@ -1376,6 +1514,17 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	}
 	free_align_test_list(&align_test_list);
 
+	if (size0 && realloc_head) {
+		alt_align = max(alt_align, window_align);
+		alt_size = calculate_memsize(alt_size, min_size,
+					     0, window_align);
+		/* required is better ? */
+		if (alt_size >= size0) {
+			alt_align = 0;
+			alt_size = 0;
+		}
+	}
+
 	if (sum_add_size < min_sum_size)
 		sum_add_size = min_sum_size;
 	if (sum_add_size > size && realloc_head) {
@@ -1397,13 +1546,43 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	b_res->start = min_align;
 	b_res->end = size0 + min_align - 1;
 	b_res->flags |= IORESOURCE_STARTALIGN;
-	if (size1 > size0 && realloc_head) {
-		__add_to_list(realloc_head, bus->self, b_res, size1 - size0,
-				min_add_align, 0, 0);
-		dev_printk(KERN_DEBUG, &bus->self->dev, "bridge window %pR to %pR add_size %llx add_align %llx\n",
-			   b_res, &bus->busn_res,
-			   (unsigned long long) (size1 - size0),
-			   (unsigned long long) min_add_align);
+	if (realloc_head) {
+		resource_size_t final_add_size = 0;
+
+		if (size1 > size0)
+			final_add_size = size1 - size0;
+		else
+			min_add_align = 0;
+
+		/*
+		 * realloc list include three type entries
+		 * 1. optional only:
+		 *    add_size != 0, alt_size == 0, req_size == 0
+		 * 2. required only with smaller alt_size.
+		 *    add_size == 0, alt_size != 0, req_size > alt_size
+		 * 3. required + optional:
+		 *    add_size != 0, alt_size < req_size, req_size != 0
+		 *
+		 * So there is no req_size != 0, and alt_size == req_size.
+		 * in that case, we already set alt_size = 0.
+		 *
+		 * req_align/req_size is not stored directly, and we
+		 * have dev_res start/end/flags instead.
+		 */
+		if (final_add_size || alt_size) {
+			__add_to_list(realloc_head, bus->self, b_res,
+				      final_add_size, min_add_align,
+				      alt_size, alt_align);
+			dev_printk(KERN_DEBUG, &bus->self->dev,
+				   "bridge window %pR to %pR add_size %llx add_align %llx alt_size %llx alt_align %llx req_size %llx req_align %llx\n",
+				   b_res, &bus->busn_res,
+				   (unsigned long long)final_add_size,
+				   (unsigned long long)min_add_align,
+				   (unsigned long long)alt_size,
+				   (unsigned long long)alt_align,
+				   (unsigned long long)size0,
+				   (unsigned long long)min_align);
+		}
 	}
 	return 0;
 }
-- 
1.8.4.5

[PATCH v8 45/61] PCI: Add support for more than two alt_size entries under same bridge

[Yinghai Lu]

When we have two bridges under parent bridge, and each child
bridge has alt_size, we need to increase parent alt_size to make
sure it could fit all alt entries.

In the patch, we first select one big size, and then keep reducing
the size and retrying to get the minimum value for alt_size.

For example, two bridges:
  one have 8M/8M, and 1M/1M children res.
  one have 4M/4M, and 1M/1M children res.

Then we have child pridges alt_align/alt_size: 8M/9M, 4M/5M.
Before this patch, parent bridge alt_align/alt_size is 8M/14M
that is wrong, as it can not fit two alt entries at all.
With this patch	parent bridge alt_align/alt_size: 8M/17M.
               8M            16M   20M   24M
  |------------|-------------|-----|-----|
	       8M			   25M
	       |---------------------------|
			       17M
               |---9M----------|   |-5M----|


At same time, child bridges required align/size: 4M/12M, 2M/6M.
and prarent bridge required align/size: 4M/20M.

So at last, we use 8M/17M as parent bridge alt_align/alt_size.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=100451
Reported-by: Yijing Wang <wangyijing@huawei.com>
Tested-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 56 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 54 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index dea0344..7e7663a 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1373,6 +1373,47 @@ out:
 	return good_align;
 }
 
+static resource_size_t calculate_mem_alt_size(struct list_head *head,
+				resource_size_t max_align, resource_size_t size,
+				resource_size_t align_low)
+{
+	struct align_test_res *p;
+	resource_size_t tmp;
+	resource_size_t good_size, bad_size;
+	int count = 0, order;
+
+	good_size = ALIGN(size, align_low);
+
+	list_for_each_entry(p, head, list)
+		count++;
+
+	if (count <= 1)
+		goto out;
+
+	sort_align_test(head);
+
+	tmp = max(size, max_align);
+	order = __fls(count);
+	if ((1ULL << order) < count)
+		order++;
+	good_size = ALIGN((tmp << order), align_low);
+	bad_size = ALIGN(size, align_low) - align_low;
+	size = good_size;
+	while (size > bad_size) {
+		/* check if align/size fit all entries */
+		if (is_align_size_good(head, max_align, size, 0))
+			good_size = size;
+		else
+			bad_size = size;
+
+		size = bad_size + ((good_size - bad_size) >> 1);
+		size = round_down(size, align_low);
+	}
+
+out:
+	return good_size;
+}
+
 static inline bool is_optional(int i)
 {
 
@@ -1419,6 +1460,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 					mask | IORESOURCE_PREFETCH, type);
 	LIST_HEAD(align_test_list);
 	LIST_HEAD(align_test_add_list);
+	LIST_HEAD(align_test_alt_list);
 	resource_size_t alt_size = 0, alt_align = 0;
 	resource_size_t window_align;
 
@@ -1493,10 +1535,17 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 				dev_res = res_to_dev_res(realloc_head, r);
 				if (dev_res && dev_res->alt_size) {
+					add_to_align_test_list(
+						&align_test_alt_list,
+						dev_res->alt_align,
+						dev_res->alt_size);
 					alt_size += dev_res->alt_size;
 					if (alt_align < dev_res->alt_align)
 						alt_align = dev_res->alt_align;
 				} else if (r_size > 1) {
+					add_to_align_test_list(
+						&align_test_alt_list,
+						align, r_size);
 					alt_size += r_size;
 					if (alt_align < align)
 						alt_align = align;
@@ -1516,14 +1565,17 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 	if (size0 && realloc_head) {
 		alt_align = max(alt_align, window_align);
-		alt_size = calculate_memsize(alt_size, min_size,
-					     0, window_align);
+		/* need to increase size to fit more alt */
+		alt_size = calculate_mem_alt_size(&align_test_alt_list,
+						  alt_align, alt_size,
+						  window_align);
 		/* required is better ? */
 		if (alt_size >= size0) {
 			alt_align = 0;
 			alt_size = 0;
 		}
 	}
+	free_align_test_list(&align_test_alt_list);
 
 	if (sum_add_size < min_sum_size)
 		sum_add_size = min_sum_size;
-- 
1.8.4.5

[PATCH v8 46/61] PCI: Fix size calculation with old_size on rescan path

[Yinghai Lu]

On booting path, we don't pass realloc at first, and treat all optional
just as required, in some case we can have smaller size/align with optional
than required only.

  04:00.0 has children bridges: 05:03.0, 05:04.0
pref layout after booting path like followings:

pci 0000:04:00.0: BAR 9: assigned [mem 0x84000000-0x9fffffff 64bit pref]
pci 0000:05:04.0: BAR 9: assigned [mem 0x88000000-0x9fffffff 64bit pref]
pci 0000:05:03.0: BAR 9: assigned [mem 0x84000000-0x841fffff 64bit pref]
pci 0000:05:03.0: PCI bridge to [bus 08-0f]
pci 0000:05:03.0:   bridge window [mem 0x84000000-0x841fffff 64bit pref]
pci 0000:05:04.0: PCI bridge to [bus 10]
pci 0000:05:04.0:   bridge window [mem 0x88000000-0x9fffffff 64bit pref]
pci 0000:04:00.0: PCI bridge to [bus 05-10]
pci 0000:04:00.0:   bridge window [mem 0x84000000-0x9fffffff 64bit pref]

so the old size in rescan for 04:00.0 would be 0x1c000000, and align is 0x4000000

during remove and rescan:

pci 0000:05:03.0: bridge window [mem 0x00000000-0xffffffffffffffff 64bit pref] to [bus 08-0f] add_size 200000 add_align 100000 alt_size 0 alt_align 0 must_size 0 must_align 0
pci 0000:05:03.0: bridge window [mem 0x00000000-0xffffffffffffffff] to [bus 08-0f] add_size 200000 add_align 100000 alt_size 0 alt_align 0 must_size 0 must_align 0
pci 0000:05:04.0: bridge window [mem 0x08000000-0x1fffffff 64bit pref] to [bus 10] add_size 0 add_align 0 alt_size 10100000 alt_align 10000000 must_size 18000000 must_align 8000000
pci 0000:05:03.0: BAR 9: [mem 0x00000000-0xffffffffffffffff 64bit pref] get_res_add_size add_size   200000
pci 0000:05:03.0: BAR 9: [mem 0x00000000-0xffffffffffffffff 64bit pref] get_res_add_align min_align 100000
pci 0000:04:00.0: bridge window [mem 0x08000000-0x27ffffff 64bit pref] to [bus 05-10] add_size 0 add_align 0 alt_size 10100000 alt_align 10000000 must_size 20000000 must_align 8000000

align old size 0x1c000000 to 0x2000000 as size0, 0x1c000000 as size1.
so for 04:00.0 will have big must and no optional size anymore.

So don't align old size, then we will have same size0 and size1,
and use smaller add_align as must align.

After the patch, rescan works properly.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 7e7663a..00a39be 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1134,9 +1134,9 @@ static resource_size_t calculate_memsize(resource_size_t size,
 		size = min_size;
 	if (old_size == 1)
 		old_size = 0;
+	size = ALIGN(size, align);
 	if (size < old_size)
 		size = old_size;
-	size = ALIGN(size, align);
 	return size;
 }
 
@@ -1595,6 +1595,17 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		b_res->flags = 0;
 		return 0;
 	}
+
+	/*
+	 * It happens when boot path is not passing realloc
+	 * and later rescan is passing realloc.
+	 * The old value from boot path is bigger, and calculate_size will
+	 * use old value as size0 and size1, and also have
+	 * chance optional align is smaller than must only align.
+	 */
+	if(size0 == size1 && min_align > min_add_align)
+		min_align = min_add_align;
+
 	b_res->start = min_align;
 	b_res->end = size0 + min_align - 1;
 	b_res->flags |= IORESOURCE_STARTALIGN;
-- 
1.8.4.5

[PATCH v8 47/61] PCI: Don't add too much optional size for hotplug bridge io

[Yinghai Lu]

Same as patch for MMIO (PCI: Don't add too much optional size for hotplug
bridge MMIO), and this one is for io port.

It will compare required+optional with min_sum_size to get smaller
optional size.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 00a39be..d4aeedc 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1177,7 +1177,6 @@ static resource_size_t window_alignment(struct pci_bus *bus,
  *
  * @bus : the bus
  * @min_size : the minimum io window that must to be allocated
- * @add_size : additional optional io window
  * @realloc_head : track the additional io window on this list
  *
  * Sizing the IO windows of the PCI-PCI bridge is trivial,
@@ -1186,9 +1185,11 @@ static resource_size_t window_alignment(struct pci_bus *bus,
  * We must be careful with the ISA aliasing though.
  */
 static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
-		resource_size_t add_size, struct list_head *realloc_head)
+			 struct list_head *realloc_head)
 {
 	struct pci_dev *dev;
+	resource_size_t min_sum_size = 0;
+	resource_size_t sum_add_size;
 	struct resource *b_res = find_free_bus_resource(bus, IORESOURCE_IO,
 							IORESOURCE_IO);
 	resource_size_t size = 0, size0 = 0, size1 = 0;
@@ -1198,6 +1199,11 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 	if (!b_res)
 		return;
 
+	if (realloc_head) {
+		min_sum_size = min_size;
+		min_size = 0;
+	}
+
 	min_align = window_alignment(bus, IORESOURCE_IO);
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		int i;
@@ -1227,10 +1233,11 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 
 	size0 = calculate_iosize(size, min_size, size1,
 			resource_size(b_res), min_align);
-	if (children_add_size > add_size)
-		add_size = children_add_size;
-	size1 = (!realloc_head || (realloc_head && !add_size)) ? size0 :
-		calculate_iosize(size, min_size, add_size + size1,
+	sum_add_size = children_add_size + size + size1;
+	if (sum_add_size < min_sum_size)
+		sum_add_size = min_sum_size;
+	size1 = !realloc_head ? size0 :
+		calculate_iosize(size, min_size, sum_add_size - size,
 			resource_size(b_res), min_align);
 	if (!size0 && !size1) {
 		if (b_res->start || b_res->end)
@@ -1757,7 +1764,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 {
 	struct pci_dev *dev;
 	unsigned long mask, prefmask, type2 = 0, type3 = 0;
-	resource_size_t min_mem_size = 0, additional_io_size = 0;
+	resource_size_t min_mem_size = 0, min_io_size = 0;
 	struct resource *b_res;
 	int ret;
 
@@ -1793,13 +1800,12 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 
 	case PCI_CLASS_BRIDGE_PCI:
 		if (bus->self->is_hotplug_bridge) {
-			additional_io_size  = pci_hotplug_io_size;
+			min_io_size  = pci_hotplug_io_size;
 			min_mem_size = pci_hotplug_mem_size;
 		}
 		/* Fall through */
 	default:
-		pbus_size_io(bus, realloc_head ? 0 : additional_io_size,
-			     additional_io_size, realloc_head);
+		pbus_size_io(bus, min_io_size, realloc_head);
 
 		/*
 		 * If there's a 64-bit prefetchable MMIO window, compute
-- 
1.8.4.5

[PATCH v8 48/61] PCI: Move ISA io port align out of calculate_iosize()

[Yinghai Lu]

We need to move ISA io port align out of calculate_iosize(),
so we could unify calculate_iosize and calculate_memsize later.

That extra aligning or offset is to work around ISA devices:
When one bridge have several children devices, and every device
has several io port resources and resource size < 0x400.
We need to check size, and add extra size to make sure bit8/9
to be zero.

Also need to apply same checking for optional size path.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 39 +++++++++++++++++++++++++++------------
 1 file changed, 27 insertions(+), 12 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index d4aeedc..4e7f0aa 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1114,11 +1114,6 @@ static resource_size_t calculate_iosize(resource_size_t size,
 		size = min_size;
 	if (old_size == 1)
 		old_size = 0;
-	/* To be fixed in 2.5: we should have sort of HAVE_ISA
-	   flag in the struct pci_bus. */
-#if defined(CONFIG_ISA) || defined(CONFIG_EISA)
-	size = (size & 0xff) + ((size & ~0xffUL) << 2);
-#endif
 	size = ALIGN(size + size1, align);
 	if (size < old_size)
 		size = old_size;
@@ -1172,6 +1167,18 @@ static resource_size_t window_alignment(struct pci_bus *bus,
 	return max(align, arch_align);
 }
 
+static resource_size_t size_aligned_for_isa(resource_size_t size)
+{
+	/*
+	 * To be fixed in 2.5: we should have sort of HAVE_ISA
+	 *  flag in the struct pci_bus.
+	 */
+#if defined(CONFIG_ISA) || defined(CONFIG_EISA)
+	size = (size & 0xff) + ((size & ~0xffUL) << 2);
+#endif
+	return size;
+}
+
 /**
  * pbus_size_io() - size the io window of a given bus
  *
@@ -1189,11 +1196,10 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 {
 	struct pci_dev *dev;
 	resource_size_t min_sum_size = 0;
-	resource_size_t sum_add_size;
 	struct resource *b_res = find_free_bus_resource(bus, IORESOURCE_IO,
 							IORESOURCE_IO);
 	resource_size_t size = 0, size0 = 0, size1 = 0;
-	resource_size_t children_add_size = 0;
+	resource_size_t sum_add_size = 0, sum_add_size1 = 0;
 	resource_size_t min_align, align;
 
 	if (!b_res)
@@ -1210,7 +1216,7 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
-			unsigned long r_size;
+			unsigned long r_size, r_add_size;
 
 			if (r->parent || !(r->flags & IORESOURCE_IO))
 				continue;
@@ -1226,18 +1232,27 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 			if (align > min_align)
 				min_align = align;
 
-			if (realloc_head)
-				children_add_size += get_res_add_size(realloc_head, r);
+			if (realloc_head) {
+				r_add_size = get_res_add_size(realloc_head, r);
+				r_add_size += r_size;
+				if (r_add_size < 0x400)
+					/* Might be re-aligned for ISA */
+					sum_add_size += r_add_size;
+				else
+					sum_add_size1 += r_add_size;
+			}
 		}
 	}
 
+	size = size_aligned_for_isa(size);
 	size0 = calculate_iosize(size, min_size, size1,
 			resource_size(b_res), min_align);
-	sum_add_size = children_add_size + size + size1;
+	sum_add_size = size_aligned_for_isa(sum_add_size);
+	sum_add_size += sum_add_size1;
 	if (sum_add_size < min_sum_size)
 		sum_add_size = min_sum_size;
 	size1 = !realloc_head ? size0 :
-		calculate_iosize(size, min_size, sum_add_size - size,
+		calculate_iosize(sum_add_size, min_size, 0,
 			resource_size(b_res), min_align);
 	if (!size0 && !size1) {
 		if (b_res->start || b_res->end)
-- 
1.8.4.5

[PATCH v8 49/61] PCI: Don't add too much io port for hotplug bridge with old size

[Yinghai Lu]

Now we add too much for hotplug bridge io port.
For example, when hotplug bridge has two children bridges,
every child bridge will need 0x1000, so size1 will be 0x2000
and size is 0. The min_size for the hotplug bridge is 0x100.
with old version calculate_iosize, we get 0x3000 for final
size as we are using size to compare with min_size at first.
That is not right, we should have 0x2000.

We can check size+size1 with min_size for io port, and just add size1
to size without passing extra size1 into calculate_iosize().

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 4e7f0aa..d4e8da1 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1106,7 +1106,6 @@ static struct resource *find_free_bus_resource(struct pci_bus *bus,
 
 static resource_size_t calculate_iosize(resource_size_t size,
 		resource_size_t min_size,
-		resource_size_t size1,
 		resource_size_t old_size,
 		resource_size_t align)
 {
@@ -1114,7 +1113,7 @@ static resource_size_t calculate_iosize(resource_size_t size,
 		size = min_size;
 	if (old_size == 1)
 		old_size = 0;
-	size = ALIGN(size + size1, align);
+	size = ALIGN(size, align);
 	if (size < old_size)
 		size = old_size;
 	return size;
@@ -1245,14 +1244,15 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 	}
 
 	size = size_aligned_for_isa(size);
-	size0 = calculate_iosize(size, min_size, size1,
+	size += size1;
+	size0 = calculate_iosize(size, min_size,
 			resource_size(b_res), min_align);
 	sum_add_size = size_aligned_for_isa(sum_add_size);
 	sum_add_size += sum_add_size1;
 	if (sum_add_size < min_sum_size)
 		sum_add_size = min_sum_size;
 	size1 = !realloc_head ? size0 :
-		calculate_iosize(sum_add_size, min_size, 0,
+		calculate_iosize(sum_add_size, min_size,
 			resource_size(b_res), min_align);
 	if (!size0 && !size1) {
 		if (b_res->start || b_res->end)
-- 
1.8.4.5

[PATCH v8 50/61] PCI: Unify calculate_size() for io port and MMIO

[Yinghai Lu]

Now calculate_memsize() and calculate_iosize() is the same.

Change them to calculate_size().

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 25 +++++--------------------
 1 file changed, 5 insertions(+), 20 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index d4e8da1..a6eea60 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1104,22 +1104,7 @@ static struct resource *find_free_bus_resource(struct pci_bus *bus,
 	return NULL;
 }
 
-static resource_size_t calculate_iosize(resource_size_t size,
-		resource_size_t min_size,
-		resource_size_t old_size,
-		resource_size_t align)
-{
-	if (size < min_size)
-		size = min_size;
-	if (old_size == 1)
-		old_size = 0;
-	size = ALIGN(size, align);
-	if (size < old_size)
-		size = old_size;
-	return size;
-}
-
-static resource_size_t calculate_memsize(resource_size_t size,
+static resource_size_t calculate_size(resource_size_t size,
 		resource_size_t min_size,
 		resource_size_t old_size,
 		resource_size_t align)
@@ -1245,14 +1230,14 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 
 	size = size_aligned_for_isa(size);
 	size += size1;
-	size0 = calculate_iosize(size, min_size,
+	size0 = calculate_size(size, min_size,
 			resource_size(b_res), min_align);
 	sum_add_size = size_aligned_for_isa(sum_add_size);
 	sum_add_size += sum_add_size1;
 	if (sum_add_size < min_sum_size)
 		sum_add_size = min_sum_size;
 	size1 = !realloc_head ? size0 :
-		calculate_iosize(sum_add_size, min_size,
+		calculate_size(sum_add_size, min_size,
 			resource_size(b_res), min_align);
 	if (!size0 && !size1) {
 		if (b_res->start || b_res->end)
@@ -1580,7 +1565,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	if (size || min_size) {
 		min_align = calculate_mem_align(&align_test_list, max_align,
 						size, window_align);
-		size0 = calculate_memsize(size, min_size,
+		size0 = calculate_size(size, min_size,
 				  resource_size(b_res), min_align);
 	}
 	free_align_test_list(&align_test_list);
@@ -1605,7 +1590,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		min_add_align = calculate_mem_align(&align_test_add_list,
 					max_add_align, sum_add_size,
 					window_align);
-		size1 = calculate_memsize(sum_add_size, min_size,
+		size1 = calculate_size(sum_add_size, min_size,
 				 resource_size(b_res), min_add_align);
 	}
 	free_align_test_list(&align_test_add_list);
-- 
1.8.4.5

[PATCH v8 51/61] PCI: Allow bridge optional only io port resource required size to be 0

[Yinghai Lu]

When there is no child device under the non hotplug bridge,
We can use 0 for required size, and do not use old size as required size.

That will save some io port range for other bridges, as BIOS could do
some partial assign, and we want to use those not used io port range.

When there is child device, size will not be 0.
when the bridge supports hotplug, min_size will not be 0.
So they will still honor the old size as required size.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index a6eea60..bbc10d8 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1230,8 +1230,9 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 
 	size = size_aligned_for_isa(size);
 	size += size1;
-	size0 = calculate_size(size, min_size,
-			resource_size(b_res), min_align);
+	if (size || min_size)
+		size0 = calculate_size(size, min_size,
+					resource_size(b_res), min_align);
 	sum_add_size = size_aligned_for_isa(sum_add_size);
 	sum_add_size += sum_add_size1;
 	if (sum_add_size < min_sum_size)
@@ -1247,7 +1248,7 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 		return;
 	}
 
-	b_res->start = min_align;
+	b_res->start = size0 ? min_align : 0;
 	b_res->end = b_res->start + size0 - 1;
 	b_res->flags |= IORESOURCE_STARTALIGN;
 	if (size1 > size0 && realloc_head) {
-- 
1.8.4.5

[PATCH v8 52/61] PCI: Unify skip_ioresource_align()

[Yinghai Lu]

There are powerpc generic version and x86 local version for
skip_ioresource_align().

Move the powerpc version to setup-bus.c, and kill x86 local version.

Also kill dummy version in microblaze.

Cc: Michal Simek <monstr@monstr.eu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-arch@vger.kernel.org
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Michael Ellerman <mpe@ellerman.id.au>
---
 arch/microblaze/pci/pci-common.c |  8 --------
 arch/powerpc/kernel/pci-common.c | 11 +----------
 arch/x86/include/asm/pci_x86.h   |  1 -
 arch/x86/pci/common.c            |  4 ++--
 arch/x86/pci/i386.c              | 12 ++----------
 drivers/pci/setup-bus.c          |  9 +++++++++
 include/asm-generic/pci-bridge.h |  2 ++
 7 files changed, 16 insertions(+), 31 deletions(-)

diff --git a/arch/microblaze/pci/pci-common.c b/arch/microblaze/pci/pci-common.c
index ae838ed..09b1af6 100644
--- a/arch/microblaze/pci/pci-common.c
+++ b/arch/microblaze/pci/pci-common.c
@@ -878,11 +878,6 @@ void pcibios_fixup_bus(struct pci_bus *bus)
 }
 EXPORT_SYMBOL(pcibios_fixup_bus);
 
-static int skip_isa_ioresource_align(struct pci_dev *dev)
-{
-	return 0;
-}
-
 /*
  * We need to avoid collisions with `mirrored' VGA ports
  * and other strange ISA hardware, so we always want the
@@ -899,12 +894,9 @@ static int skip_isa_ioresource_align(struct pci_dev *dev)
 resource_size_t pcibios_align_resource(void *data, const struct resource *res,
 				resource_size_t size, resource_size_t align)
 {
-	struct pci_dev *dev = data;
 	resource_size_t start = res->start;
 
 	if (res->flags & IORESOURCE_IO) {
-		if (skip_isa_ioresource_align(dev))
-			return start;
 		if (start & 0x300)
 			start = (start + 0x3ff) & ~0x3ff;
 	}
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 7587b2a..8853667 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -1052,15 +1052,6 @@ void pci_fixup_cardbus(struct pci_bus *bus)
 	pcibios_setup_bus_devices(bus);
 }
 
-
-static int skip_isa_ioresource_align(struct pci_dev *dev)
-{
-	if (pci_has_flag(PCI_CAN_SKIP_ISA_ALIGN) &&
-	    !(dev->bus->bridge_ctl & PCI_BRIDGE_CTL_ISA))
-		return 1;
-	return 0;
-}
-
 /*
  * We need to avoid collisions with `mirrored' VGA ports
  * and other strange ISA hardware, so we always want the
@@ -1081,7 +1072,7 @@ resource_size_t pcibios_align_resource(void *data, const struct resource *res,
 	resource_size_t start = res->start;
 
 	if (res->flags & IORESOURCE_IO) {
-		if (skip_isa_ioresource_align(dev))
+		if (skip_isa_ioresource_align(dev->bus))
 			return start;
 		if (start & 0x300)
 			start = (start + 0x3ff) & ~0x3ff;
diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index fa1195d..81a7abf 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -28,7 +28,6 @@ do {						\
 #define PCI_ASSIGN_ROMS		0x1000
 #define PCI_BIOS_IRQ_SCAN	0x2000
 #define PCI_ASSIGN_ALL_BUSSES	0x4000
-#define PCI_CAN_SKIP_ISA_ALIGN	0x8000
 #define PCI_USE__CRS		0x10000
 #define PCI_CHECK_ENABLE_AMD_MMCONF	0x20000
 #define PCI_HAS_IO_ECS		0x40000
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index dc78a4a..7c018df 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -83,7 +83,7 @@ DEFINE_RAW_SPINLOCK(pci_config_lock);
 
 static int __init can_skip_ioresource_align(const struct dmi_system_id *d)
 {
-	pci_probe |= PCI_CAN_SKIP_ISA_ALIGN;
+	pci_add_flags(PCI_CAN_SKIP_ISA_ALIGN);
 	printk(KERN_INFO "PCI: %s detected, can skip ISA alignment\n", d->ident);
 	return 0;
 }
@@ -619,7 +619,7 @@ char *__init pcibios_setup(char *str)
 		pci_routeirq = 1;
 		return NULL;
 	} else if (!strcmp(str, "skip_isa_align")) {
-		pci_probe |= PCI_CAN_SKIP_ISA_ALIGN;
+		pci_add_flags(PCI_CAN_SKIP_ISA_ALIGN);
 		return NULL;
 	} else if (!strcmp(str, "noioapicquirk")) {
 		noioapicquirk = 1;
diff --git a/arch/x86/pci/i386.c b/arch/x86/pci/i386.c
index 0a9f2ca..3f17726 100644
--- a/arch/x86/pci/i386.c
+++ b/arch/x86/pci/i386.c
@@ -33,6 +33,7 @@
 #include <linux/errno.h>
 #include <linux/bootmem.h>
 
+#include <asm-generic/pci-bridge.h>
 #include <asm/pat.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
@@ -128,15 +129,6 @@ static void __init pcibios_fw_addr_list_del(void)
 	pcibios_fw_addr_done = true;
 }
 
-static int
-skip_isa_ioresource_align(struct pci_dev *dev) {
-
-	if ((pci_probe & PCI_CAN_SKIP_ISA_ALIGN) &&
-	    !(dev->bus->bridge_ctl & PCI_BRIDGE_CTL_ISA))
-		return 1;
-	return 0;
-}
-
 /*
  * We need to avoid collisions with `mirrored' VGA ports
  * and other strange ISA hardware, so we always want the
@@ -158,7 +150,7 @@ pcibios_align_resource(void *data, const struct resource *res,
 	resource_size_t start = res->start;
 
 	if (res->flags & IORESOURCE_IO) {
-		if (skip_isa_ioresource_align(dev))
+		if (skip_isa_ioresource_align(dev->bus))
 			return start;
 		if (start & 0x300)
 			start = (start + 0x3ff) & ~0x3ff;
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index bbc10d8..f1c7b46 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1151,6 +1151,15 @@ static resource_size_t window_alignment(struct pci_bus *bus,
 	return max(align, arch_align);
 }
 
+int skip_isa_ioresource_align(struct pci_bus *bus)
+{
+	if (pci_has_flag(PCI_CAN_SKIP_ISA_ALIGN) &&
+	    !(bus->bridge_ctl & PCI_BRIDGE_CTL_ISA))
+		return 1;
+
+	return 0;
+}
+
 static resource_size_t size_aligned_for_isa(resource_size_t size)
 {
 	/*
diff --git a/include/asm-generic/pci-bridge.h b/include/asm-generic/pci-bridge.h
index 20db2e5..fab4bd0 100644
--- a/include/asm-generic/pci-bridge.h
+++ b/include/asm-generic/pci-bridge.h
@@ -60,6 +60,8 @@ static inline int pci_has_flag(int flag)
 {
 	return pci_flags & flag;
 }
+
+int skip_isa_ioresource_align(struct pci_bus *bus);
 #else
 static inline void pci_set_flags(int flags) { }
 static inline void pci_add_flags(int flags) { }
-- 
1.8.4.5

[PATCH v8 53/61] PCI: Kill macro checking for bus io port sizing

[Yinghai Lu]

We can use new generic version skip_isa_ioresource_align() instead
of macro, and then kill the marco.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index f1c7b46..419eaaf 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1160,15 +1160,12 @@ int skip_isa_ioresource_align(struct pci_bus *bus)
 	return 0;
 }
 
-static resource_size_t size_aligned_for_isa(resource_size_t size)
+static resource_size_t size_aligned_for_isa(resource_size_t size,
+					    struct pci_bus *bus)
 {
-	/*
-	 * To be fixed in 2.5: we should have sort of HAVE_ISA
-	 *  flag in the struct pci_bus.
-	 */
-#if defined(CONFIG_ISA) || defined(CONFIG_EISA)
-	size = (size & 0xff) + ((size & ~0xffUL) << 2);
-#endif
+	if (!skip_isa_ioresource_align(bus))
+		size = (size & 0xff) + ((size & ~0xffUL) << 2);
+
 	return size;
 }
 
@@ -1237,12 +1234,12 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 		}
 	}
 
-	size = size_aligned_for_isa(size);
+	size = size_aligned_for_isa(size, bus);
 	size += size1;
 	if (size || min_size)
 		size0 = calculate_size(size, min_size,
 					resource_size(b_res), min_align);
-	sum_add_size = size_aligned_for_isa(sum_add_size);
+	sum_add_size = size_aligned_for_isa(sum_add_size, bus);
 	sum_add_size += sum_add_size1;
 	if (sum_add_size < min_sum_size)
 		sum_add_size = min_sum_size;
-- 
1.8.4.5

[PATCH v8 54/61] resources: Split out __allocate_resource()

[Yinghai Lu]

This one is separated from next patch for best fit allocation.

Split out __allocate_resource(), and it will not hold lock, so we could use it
in other functions that hold the resource lock already.

-v2: according to Linus, using "bool lock" as parameter
     aka "conditionally take lock" is *wrong*.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
---
 kernel/resource.c | 70 +++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 50 insertions(+), 20 deletions(-)

diff --git a/kernel/resource.c b/kernel/resource.c
index 6927298..62321b0 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -654,7 +654,7 @@ static int find_resource(struct resource *root, struct resource *new,
 }
 
 /**
- * reallocate_resource - allocate a slot in the resource tree given range & alignment.
+ * __reallocate_resource - allocate a slot in the resource tree given range & alignment.
  *	The resource will be relocated if the new size cannot be reallocated in the
  *	current location.
  *
@@ -663,7 +663,7 @@ static int find_resource(struct resource *root, struct resource *new,
  * @newsize: new size of the resource descriptor
  * @constraint: the size and alignment constraints to be met.
  */
-static int reallocate_resource(struct resource *root, struct resource *old,
+static int __reallocate_resource(struct resource *root, struct resource *old,
 			resource_size_t newsize,
 			struct resource_constraint  *constraint)
 {
@@ -671,8 +671,6 @@ static int reallocate_resource(struct resource *root, struct resource *old,
 	struct resource new = *old;
 	struct resource *conflict;
 
-	write_lock(&resource_lock);
-
 	if ((err = __find_resource(root, old, &new, newsize, constraint)))
 		goto out;
 
@@ -697,14 +695,13 @@ static int reallocate_resource(struct resource *root, struct resource *old,
 		BUG_ON(conflict);
 	}
 out:
-	write_unlock(&resource_lock);
 	return err;
 }
 
-
 /**
- * allocate_resource - allocate empty slot in the resource tree given range & alignment.
- * 	The resource will be reallocated with a new size if it was already allocated
+ * __allocate_resource - allocate empty slot in the resource tree given range & alignment.
+ *	The resource will be reallocated with a new size if it was already
+ *	allocated
  * @root: root resource descriptor
  * @new: resource descriptor desired by caller
  * @size: requested resource region size
@@ -713,15 +710,17 @@ out:
  * @align: alignment requested, in bytes
  * @alignf: alignment function, optional, called if not NULL
  * @alignf_data: arbitrary data to pass to the @alignf function
+ *
+ * Caller need to hold resource_lock if needed.
  */
-int allocate_resource(struct resource *root, struct resource *new,
-		      resource_size_t size, resource_size_t min,
-		      resource_size_t max, resource_size_t align,
-		      resource_size_t (*alignf)(void *,
-						const struct resource *,
-						resource_size_t,
-						resource_size_t),
-		      void *alignf_data)
+static int __allocate_resource(struct resource *root, struct resource *new,
+				resource_size_t size, resource_size_t min,
+				resource_size_t max, resource_size_t align,
+				resource_size_t (*alignf)(void *,
+						  const struct resource *,
+						  resource_size_t,
+						  resource_size_t),
+				void *alignf_data)
 {
 	int err;
 	struct resource_constraint constraint;
@@ -735,20 +734,51 @@ int allocate_resource(struct resource *root, struct resource *new,
 	constraint.alignf = alignf;
 	constraint.alignf_data = alignf_data;
 
-	if ( new->parent ) {
+	if (new->parent) {
 		/* resource is already allocated, try reallocating with
 		   the new constraints */
-		return reallocate_resource(root, new, size, &constraint);
+		return __reallocate_resource(root, new, size, &constraint);
 	}
 
-	write_lock(&resource_lock);
 	err = find_resource(root, new, size, &constraint);
 	if (err >= 0 && __request_resource(root, new))
 		err = -EBUSY;
-	write_unlock(&resource_lock);
+
 	return err;
 }
 
+/**
+ * allocate_resource - allocate empty slot in the resource tree given range & alignment.
+ *	The resource will be reallocated with a new size if it was already
+ *	allocated
+ * @root: root resource descriptor
+ * @new: resource descriptor desired by caller
+ * @size: requested resource region size
+ * @min: minimum boundary to allocate
+ * @max: maximum boundary to allocate
+ * @align: alignment requested, in bytes
+ * @alignf: alignment function, optional, called if not NULL
+ * @alignf_data: arbitrary data to pass to the @alignf function
+ */
+int allocate_resource(struct resource *root, struct resource *new,
+		      resource_size_t size, resource_size_t min,
+		      resource_size_t max, resource_size_t align,
+		      resource_size_t (*alignf)(void *,
+						const struct resource *,
+						resource_size_t,
+						resource_size_t),
+		      void *alignf_data)
+{
+	int ret;
+
+	write_lock(&resource_lock);
+	ret = __allocate_resource(root, new, size, min, max, align,
+				   alignf, alignf_data);
+	write_unlock(&resource_lock);
+
+	return ret;
+}
+
 EXPORT_SYMBOL(allocate_resource);
 
 /**
-- 
1.8.4.5

[PATCH v8 55/61] resources: Make allocate_resource() return best fit resource

[Yinghai Lu]

Current code just allocate from first avail window.

We can find all suitable empty slots and pick one with smallest size, so
we could save the big slot for needed ones later when we have several pci
bridges under parent bridge and some bridges get assigned from bios and we
need to assign others in kernel.

For examples: we have window
[0xc0000000, 0xd0000000), and [0xe0000000,0xe1000000)

and we try allocate 0x200000 size resource.

in this patch will reserve [0xc0000000, 0xd0000000) and
[0xe0000000,0xe1000000) at first, then pick [0xe0000000,0xe1000000)
to allocate 0x200000 size.

-v2: updated after __allocate_resource change, and add field in constraint
	instead of passing it directly.
-v3: Use best fit instead of just fit according to Bjorn.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 kernel/resource.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 68 insertions(+), 13 deletions(-)

diff --git a/kernel/resource.c b/kernel/resource.c
index 62321b0..c468875 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -48,6 +48,7 @@ struct resource_constraint {
 	resource_size_t (*alignf)(void *, const struct resource *,
 			resource_size_t, resource_size_t);
 	void *alignf_data;
+	bool fit;
 };
 
 static DEFINE_RWLOCK(resource_lock);
@@ -589,12 +590,15 @@ static void resource_clip(struct resource *res, resource_size_t min,
  * alignment constraints
  */
 static int __find_resource(struct resource *root, struct resource *old,
-			 struct resource *new,
+			 struct resource *new, struct resource *avail,
 			 resource_size_t  size,
 			 struct resource_constraint *constraint)
 {
 	struct resource *this = root->child;
-	struct resource tmp = *new, avail, alloc;
+	struct resource tmp = *new, availx, alloc;
+
+	if (!avail || avail == new)
+		avail = &availx;
 
 	tmp.start = root->start;
 	/*
@@ -618,15 +622,16 @@ static int __find_resource(struct resource *root, struct resource *old,
 		arch_remove_reservations(&tmp);
 
 		/* Check for overflow after ALIGN() */
-		avail.start = ALIGN(tmp.start, constraint->align);
-		avail.end = tmp.end;
-		avail.flags = new->flags & ~IORESOURCE_UNSET;
-		if (avail.start >= tmp.start) {
-			alloc.flags = avail.flags;
-			alloc.start = constraint->alignf(constraint->alignf_data, &avail,
+		avail->start = ALIGN(tmp.start, constraint->align);
+		avail->end = tmp.end;
+		avail->flags = new->flags & ~IORESOURCE_UNSET;
+		if (avail->start >= tmp.start) {
+			alloc.flags = avail->flags;
+			alloc.start = constraint->alignf(
+					constraint->alignf_data, avail,
 					size, constraint->align);
 			alloc.end = alloc.start + size - 1;
-			if (resource_contains(&avail, &alloc)) {
+			if (resource_contains(avail, &alloc)) {
 				new->start = alloc.start;
 				new->end = alloc.end;
 				return 0;
@@ -643,6 +648,11 @@ next:		if (!this || this->end == root->end)
 	return -EBUSY;
 }
 
+struct good_resource {
+	struct list_head list;
+	struct resource avail;
+	struct resource new;
+};
 /*
  * Find empty slot in the resource tree given range and alignment.
  */
@@ -650,7 +660,49 @@ static int find_resource(struct resource *root, struct resource *new,
 			resource_size_t size,
 			struct resource_constraint  *constraint)
 {
-	return  __find_resource(root, NULL, new, size, constraint);
+	int ret = -1;
+	LIST_HEAD(head);
+	struct good_resource *good, *tmp;
+	resource_size_t avail_size = (resource_size_t)-1ULL;
+
+	if (!constraint->fit)
+		return __find_resource(root, NULL, new, NULL, size,
+					constraint);
+
+	/* find all suitable ones and add to the list */
+	for (;;) {
+		good = kzalloc(sizeof(*good), GFP_KERNEL);
+		if (!good)
+			break;
+
+		good->new.start = new->start;
+		good->new.end = new->end;
+		good->new.flags = new->flags;
+		ret = __find_resource(root, NULL, &good->new, &good->avail,
+					size, constraint);
+		if (ret || __request_resource(root, &good->avail)) {
+			ret = -EBUSY;
+			kfree(good);
+			break;
+		}
+
+		list_add(&good->list, &head);
+	}
+
+	/* pick up the smallest one and delete the list */
+	list_for_each_entry_safe(good, tmp, &head, list) {
+		if (resource_size(&good->avail) < avail_size) {
+			avail_size = resource_size(&good->avail);
+			new->start = good->new.start;
+			new->end = good->new.end;
+			ret = 0;
+		}
+		list_del(&good->list);
+		__release_resource(&good->avail);
+		kfree(good);
+	}
+
+	return ret;
 }
 
 /**
@@ -671,7 +723,8 @@ static int __reallocate_resource(struct resource *root, struct resource *old,
 	struct resource new = *old;
 	struct resource *conflict;
 
-	if ((err = __find_resource(root, old, &new, newsize, constraint)))
+	err = __find_resource(root, old, &new, NULL, newsize, constraint);
+	if (err)
 		goto out;
 
 	if (resource_contains(&new, old)) {
@@ -710,6 +763,7 @@ out:
  * @align: alignment requested, in bytes
  * @alignf: alignment function, optional, called if not NULL
  * @alignf_data: arbitrary data to pass to the @alignf function
+ * @fit: only allocate fit range.
  *
  * Caller need to hold resource_lock if needed.
  */
@@ -720,7 +774,7 @@ static int __allocate_resource(struct resource *root, struct resource *new,
 						  const struct resource *,
 						  resource_size_t,
 						  resource_size_t),
-				void *alignf_data)
+				void *alignf_data, bool fit)
 {
 	int err;
 	struct resource_constraint constraint;
@@ -733,6 +787,7 @@ static int __allocate_resource(struct resource *root, struct resource *new,
 	constraint.align = align;
 	constraint.alignf = alignf;
 	constraint.alignf_data = alignf_data;
+	constraint.fit = fit;
 
 	if (new->parent) {
 		/* resource is already allocated, try reallocating with
@@ -773,7 +828,7 @@ int allocate_resource(struct resource *root, struct resource *new,
 
 	write_lock(&resource_lock);
 	ret = __allocate_resource(root, new, size, min, max, align,
-				   alignf, alignf_data);
+				   alignf, alignf_data, true);
 	write_unlock(&resource_lock);
 
 	return ret;
-- 
1.8.4.5

[PATCH v8 56/61] PCI, x86: Allocate from high in available window for MMIO

[Yinghai Lu]

Current code just use aligned start from avialable window, that could waste
big alignment from start.

We can align to the end from avialable window, so will save
start with big align to others: like second try for pref mmio
after first try already have non-pref assigned.

pci tree:
-[0000:00]-+-00.0
           +-1c.0-[01-10]--+-00.0-[02-10]--+-01.0-[03]----00.0  PLX Technology, Inc. Device 87b1
           |               |               +-02.0-[04-09]--+-00.0-[05-09]--+-01.0-[06]----00.0  PLX Technology, Inc. Device 87b1
           |               |               |               |               +-02.0-[07]----00.0  Broadcom Corporation Device 8650
           |               |               |               |               +-03.0-[08]--
           |               |               |               |               \-04.0-[09]----00.0  Altera Corporation Device 0201
           |               |               |               +-00.1  PLX Technology, Inc. Device 87d0
           |               |               |               +-00.2  PLX Technology, Inc. Device 87d0
           |               |               |               +-00.3  PLX Technology, Inc. Device 87d0
           |               |               |               \-00.4  PLX Technology, Inc. Device 87d0
           |               |               +-03.0-[0a-0f]--+-00.0-[0b-0f]--+-01.0-[0c]----00.0  PLX Technology, Inc. Device 87b1
           |               |               |               |               +-02.0-[0d]----00.0  Broadcom Corporation Device 8650
           |               |               |               |               +-03.0-[0e]--
           |               |               |               |               \-04.0-[0f]----00.0  Altera Corporation Device 0201
           |               |               |               +-00.1  PLX Technology, Inc. Device 87d0
           |               |               |               +-00.2  PLX Technology, Inc. Device 87d0
           |               |               |               +-00.3  PLX Technology, Inc. Device 87d0
           |               |               |               \-00.4  PLX Technology, Inc. Device 87d0
           |               |               \-04.0-[10]--
           |               +-00.1  PLX Technology, Inc. Device 87d0
           |               +-00.2  PLX Technology, Inc. Device 87d0
           |               +-00.3  PLX Technology, Inc. Device 87d0
           |               \-00.4  PLX Technology, Inc. Device 87d0
           +-1c.3-[11]----00.0

hotplug device under 0000:02:03.0

before the patch:

pci 0000:0a:00.0: BAR 9: no space for [mem size 0x03000000 64bit pref]
pci 0000:0a:00.0: BAR 9: failed to assign [mem size 0x03000000 64bit pref]
pci 0000:0a:00.0: BAR 8: assigned [mem 0xb0000000-0xb01fffff]  **************
pci 0000:0a:00.0: BAR 0: assigned [mem 0xb0200000-0xb023ffff]
pci 0000:0a:00.1: BAR 0: assigned [mem 0xb0240000-0xb0241fff]
pci 0000:0a:00.2: BAR 0: assigned [mem 0xb0242000-0xb0243fff]
pci 0000:0a:00.3: BAR 0: assigned [mem 0xb0244000-0xb0245fff]
pci 0000:0a:00.4: BAR 0: assigned [mem 0xb0246000-0xb0247fff]
pci 0000:0b:04.0: BAR 9: no space for [mem size 0x03000000 64bit pref]
pci 0000:0b:04.0: BAR 9: failed to assign [mem size 0x03000000 64bit pref]
pci 0000:0b:01.0: BAR 8: assigned [mem 0xb0000000-0xb00fffff]
pci 0000:0b:02.0: BAR 8: assigned [mem 0xb0100000-0xb01fffff]
pci 0000:0c:00.0: BAR 0: assigned [mem 0xb0000000-0xb003ffff]
pci 0000:0b:01.0: PCI bridge to [bus 0c]
pci 0000:0b:01.0:   bridge window [mem 0xb0000000-0xb00fffff]
pci 0000:0d:00.0: BAR 0: assigned [mem 0xb0100000-0xb013ffff 64bit]
pci 0000:0b:02.0: PCI bridge to [bus 0d]
pci 0000:0b:02.0:   bridge window [mem 0xb0100000-0xb01fffff]
pci 0000:0b:03.0: PCI bridge to [bus 0e]
pci 0000:0f:00.0: BAR 0: no space for [mem size 0x02000000 64bit pref]
pci 0000:0f:00.0: BAR 0: failed to assign [mem size 0x02000000 64bit pref]
pci 0000:0f:00.0: BAR 2: no space for [mem size 0x00010000 64bit pref]
pci 0000:0f:00.0: BAR 2: failed to assign [mem size 0x00010000 64bit pref]
pci 0000:0b:04.0: PCI bridge to [bus 0f]
pci 0000:0a:00.0: PCI bridge to [bus 0b-0f]
pci 0000:0a:00.0:   bridge window [mem 0xb0000000-0xb01fffff]
pcieport 0000:02:03.0: PCI bridge to [bus 0a-0f]
pcieport 0000:02:03.0:   bridge window [io  0x2000-0x2fff]
pcieport 0000:02:03.0:   bridge window [mem 0xb0000000-0xb24fffff]
pcieport 0000:02:03.0:   bridge window [mem 0x80200000-0x803fffff 64bit pref]
PCI: No. 2 try to assign unassigned res
pcieport 0000:02:03.0: resource 9 [mem 0x80200000-0x803fffff 64bit pref] released
pcieport 0000:02:03.0: PCI bridge to [bus 0a-0f]
pcieport 0000:02:03.0: BAR 9: no space for [mem size 0x03000000 64bit pref]
pcieport 0000:02:03.0: BAR 9: failed to assign [mem size 0x03000000 64bit pref]
pcieport 0000:02:03.0: BAR 9: no space for [mem size 0x02100000 64bit pref]
pcieport 0000:02:03.0: BAR 9: failed to assign [mem size 0x02100000 64bit pref]
pci 0000:0a:00.0: BAR 9: no space for [mem size 0x03000000 64bit pref]
pci 0000:0a:00.0: BAR 9: failed to assign [mem size 0x03000000 64bit pref]
pci 0000:0a:00.0: BAR 9: no space for [mem size 0x02100000 64bit pref]   **************
pci 0000:0a:00.0: BAR 9: failed to assign [mem size 0x02100000 64bit pref]
pci 0000:0b:04.0: BAR 9: no space for [mem size 0x03000000 64bit pref]
pci 0000:0b:04.0: BAR 9: failed to assign [mem size 0x03000000 64bit pref]
pci 0000:0b:04.0: BAR 9: no space for [mem size 0x02100000 64bit pref]   **************
pci 0000:0b:04.0: BAR 9: failed to assign [mem size 0x02100000 64bit pref]
pci 0000:0b:01.0: PCI bridge to [bus 0c]
pci 0000:0b:01.0:   bridge window [mem 0xb0000000-0xb00fffff]
pci 0000:0b:02.0: PCI bridge to [bus 0d]
pci 0000:0b:02.0:   bridge window [mem 0xb0100000-0xb01fffff]
pci 0000:0b:03.0: PCI bridge to [bus 0e]
pci 0000:0f:00.0: BAR 0: no space for [mem size 0x02000000 64bit pref]
pci 0000:0f:00.0: BAR 0: failed to assign [mem size 0x02000000 64bit pref]
pci 0000:0f:00.0: BAR 2: no space for [mem size 0x00010000 64bit pref]
pci 0000:0f:00.0: BAR 2: failed to assign [mem size 0x00010000 64bit pref]
pci 0000:0b:04.0: PCI bridge to [bus 0f]
pci 0000:0a:00.0: PCI bridge to [bus 0b-0f]
pci 0000:0a:00.0:   bridge window [mem 0xb0000000-0xb01fffff]
pcieport 0000:02:03.0: PCI bridge to [bus 0a-0f]
pcieport 0000:02:03.0:   bridge window [io  0x2000-0x2fff]
pcieport 0000:02:03.0:   bridge window [mem 0xb0000000-0xb24fffff]


after the patch:

pci 0000:0a:00.0: BAR 9: no space for [mem size 0x03000000 64bit pref]
pci 0000:0a:00.0: BAR 9: failed to assign [mem size 0x03000000 64bit pref]
pci 0000:0a:00.0: BAR 8: assigned [mem 0xb2300000-0xb24fffff]   *************
pci 0000:0a:00.0: BAR 0: assigned [mem 0xb22c0000-0xb22fffff]
pci 0000:0a:00.1: BAR 0: assigned [mem 0xb22be000-0xb22bffff]
pci 0000:0a:00.2: BAR 0: assigned [mem 0xb22bc000-0xb22bdfff]
pci 0000:0a:00.3: BAR 0: assigned [mem 0xb22ba000-0xb22bbfff]
pci 0000:0a:00.4: BAR 0: assigned [mem 0xb22b8000-0xb22b9fff]
pci 0000:0b:04.0: BAR 9: no space for [mem size 0x03000000 64bit pref]
pci 0000:0b:04.0: BAR 9: failed to assign [mem size 0x03000000 64bit pref]
pci 0000:0b:01.0: BAR 8: assigned [mem 0xb2400000-0xb24fffff]
pci 0000:0b:02.0: BAR 8: assigned [mem 0xb2300000-0xb23fffff]
pci 0000:0c:00.0: BAR 0: assigned [mem 0xb24c0000-0xb24fffff]
pci 0000:0b:01.0: PCI bridge to [bus 0c]
pci 0000:0b:01.0:   bridge window [mem 0xb2400000-0xb24fffff]
pci 0000:0d:00.0: BAR 0: assigned [mem 0xb23c0000-0xb23fffff 64bit]
pci 0000:0b:02.0: PCI bridge to [bus 0d]
pci 0000:0b:02.0:   bridge window [mem 0xb2300000-0xb23fffff]
pci 0000:0b:03.0: PCI bridge to [bus 0e]
pci 0000:0f:00.0: BAR 0: no space for [mem size 0x02000000 64bit pref]
pci 0000:0f:00.0: BAR 0: failed to assign [mem size 0x02000000 64bit pref]
pci 0000:0f:00.0: BAR 2: no space for [mem size 0x00010000 64bit pref]
pci 0000:0f:00.0: BAR 2: failed to assign [mem size 0x00010000 64bit pref]
pci 0000:0b:04.0: PCI bridge to [bus 0f]
pci 0000:0a:00.0: PCI bridge to [bus 0b-0f]
pci 0000:0a:00.0:   bridge window [mem 0xb2300000-0xb24fffff]
pcieport 0000:02:03.0: PCI bridge to [bus 0a-0f]
pcieport 0000:02:03.0:   bridge window [io  0x2000-0x2fff]
pcieport 0000:02:03.0:   bridge window [mem 0xb0000000-0xb24fffff]
pcieport 0000:02:03.0:   bridge window [mem 0x9fc00000-0x9fdfffff 64bit pref]
PCI: No. 2 try to assign unassigned res
pcieport 0000:02:03.0: resource 9 [mem 0x9fc00000-0x9fdfffff 64bit pref] released
pcieport 0000:02:03.0: PCI bridge to [bus 0a-0f]
pcieport 0000:02:03.0: BAR 9: no space for [mem size 0x03000000 64bit pref]
pcieport 0000:02:03.0: BAR 9: failed to assign [mem size 0x03000000 64bit pref]
pcieport 0000:02:03.0: BAR 9: no space for [mem size 0x02100000 64bit pref]
pcieport 0000:02:03.0: BAR 9: failed to assign [mem size 0x02100000 64bit pref]
pci 0000:0a:00.0: BAR 9: no space for [mem size 0x03000000 64bit pref]
pci 0000:0a:00.0: BAR 9: failed to assign [mem size 0x03000000 64bit pref]
pci 0000:0a:00.0: BAR 9: assigned [mem 0xb0000000-0xb20fffff 64bit pref]  *********
pci 0000:0b:04.0: BAR 9: no space for [mem size 0x03000000 64bit pref]
pci 0000:0b:04.0: BAR 9: failed to assign [mem size 0x03000000 64bit pref]
pci 0000:0b:04.0: BAR 9: assigned [mem 0xb0000000-0xb20fffff 64bit pref]  *********
pci 0000:0b:01.0: PCI bridge to [bus 0c]
pci 0000:0b:01.0:   bridge window [mem 0xb2400000-0xb24fffff]
pci 0000:0b:02.0: PCI bridge to [bus 0d]
pci 0000:0b:02.0:   bridge window [mem 0xb2300000-0xb23fffff]
pci 0000:0b:03.0: PCI bridge to [bus 0e]
pci 0000:0f:00.0: BAR 0: assigned [mem 0xb0000000-0xb1ffffff 64bit pref]   ********
pci 0000:0f:00.0: BAR 2: assigned [mem 0xb20f0000-0xb20fffff 64bit pref]   ********
pci 0000:0b:04.0: PCI bridge to [bus 0f]
pci 0000:0b:04.0:   bridge window [mem 0xb0000000-0xb20fffff 64bit pref]
pci 0000:0a:00.0: PCI bridge to [bus 0b-0f]
pci 0000:0a:00.0:   bridge window [mem 0xb2300000-0xb24fffff]
pci 0000:0a:00.0:   bridge window [mem 0xb0000000-0xb20fffff 64bit pref]
pcieport 0000:02:03.0: PCI bridge to [bus 0a-0f]
pcieport 0000:02:03.0:   bridge window [io  0x2000-0x2fff]
pcieport 0000:02:03.0:   bridge window [mem 0xb0000000-0xb24fffff]

So we allocate high for 0a:00.0 and etc, and leave low range like 0xb0000000 to
0b:04.0 and 0f:00.0

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 arch/x86/pci/i386.c     | 20 ++++++++++++++++++++
 drivers/pci/setup-bus.c | 11 ++++++++++-
 include/linux/pci.h     |  3 +++
 3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/arch/x86/pci/i386.c b/arch/x86/pci/i386.c
index 3f17726..21f3e3e 100644
--- a/arch/x86/pci/i386.c
+++ b/arch/x86/pci/i386.c
@@ -129,6 +129,24 @@ static void __init pcibios_fw_addr_list_del(void)
 	pcibios_fw_addr_done = true;
 }
 
+resource_size_t
+pcibios_align_end_resource(void *data, const struct resource *res,
+			resource_size_t size, resource_size_t align)
+{
+	resource_size_t start = res->start;
+
+	/* Take near end */
+	if (res->end + 1 > size) {
+		resource_size_t new_start;
+
+		new_start = round_down(res->end + 1 - size, align);
+		if (new_start > start)
+			start = new_start;
+	}
+
+	return start;
+}
+
 /*
  * We need to avoid collisions with `mirrored' VGA ports
  * and other strange ISA hardware, so we always want the
@@ -155,6 +173,8 @@ pcibios_align_resource(void *data, const struct resource *res,
 		if (start & 0x300)
 			start = (start + 0x3ff) & ~0x3ff;
 	} else if (res->flags & IORESOURCE_MEM) {
+		start = pcibios_align_end_resource(data, res, size, align);
+
 		/* The low 1MB range is reserved for ISA cards */
 		if (start < BIOS_END)
 			start = BIOS_END;
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 419eaaf..1889351 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1319,6 +1319,15 @@ static void sort_align_test(struct list_head *head)
 	}
 }
 
+resource_size_t __weak pcibios_align_end_resource(void *data,
+					  const struct resource *res,
+					  resource_size_t size,
+					  resource_size_t align)
+{
+	/* default is not aligned to end */
+	return res->start;
+}
+
 static bool is_align_size_good(struct list_head *head,
 			resource_size_t min_align, resource_size_t size,
 			resource_size_t start)
@@ -1336,7 +1345,7 @@ static bool is_align_size_good(struct list_head *head,
 	list_for_each_entry(p, head, list)
 		if (allocate_resource(&root, &p->res, p->size,
 				0, (resource_size_t)-1ULL,
-				p->align, NULL, NULL))
+				p->align, pcibios_align_end_resource, NULL))
 			return false;
 
 	return true;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index cffaed4..6bc56f1 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -774,6 +774,9 @@ char *pcibios_setup(char *str);
 resource_size_t pcibios_align_resource(void *, const struct resource *,
 				resource_size_t,
 				resource_size_t);
+resource_size_t pcibios_align_end_resource(void *, const struct resource *,
+				resource_size_t,
+				resource_size_t);
 void pcibios_update_irq(struct pci_dev *, int irq);
 
 /* Weak but can be overriden by arch */
-- 
1.8.4.5

[PATCH v8 57/61] PCI: Add debug print out for min_align and alt_size

[Yinghai Lu]

Put all print out for all children align/size and result align/size
together.

We can print out device name at same time with min_align/alt_size
calculation.

So we can shut off debug print out from get_res_add_size() and
get_res_add_align().

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 76 ++++++++++++++++++++++++++++++++++++-------------
 1 file changed, 56 insertions(+), 20 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 1889351..673778c 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -159,11 +159,6 @@ static resource_size_t get_res_add_size(struct list_head *head,
 	if (!dev_res || !dev_res->add_size)
 		return 0;
 
-	dev_printk(KERN_DEBUG, &dev_res->dev->dev,
-		   "BAR %d: %pR get_res_add_size add_size   %#llx\n",
-		   (int)(res - &dev_res->dev->resource[0]),
-		   res, (unsigned long long)dev_res->add_size);
-
 	return dev_res->add_size;
 }
 
@@ -176,11 +171,6 @@ static resource_size_t get_res_add_align(struct list_head *head,
 	if (!dev_res || !dev_res->min_align)
 		return 0;
 
-	dev_printk(KERN_DEBUG, &dev_res->dev->dev,
-		   "BAR %d: %pR get_res_add_align min_align %#llx\n",
-		   (int)(res - &dev_res->dev->resource[0]),
-		   res, (unsigned long long)dev_res->min_align);
-
 	return dev_res->min_align;
 }
 
@@ -1271,6 +1261,8 @@ struct align_test_res {
 	struct resource res;
 	resource_size_t size;
 	resource_size_t align;
+	struct device *dev;
+	int idx;
 };
 
 static void free_align_test_list(struct list_head *head)
@@ -1284,7 +1276,8 @@ static void free_align_test_list(struct list_head *head)
 }
 
 static int add_to_align_test_list(struct list_head *head,
-				  resource_size_t align, resource_size_t size)
+				  resource_size_t align, resource_size_t size,
+				  struct device *dev, int idx)
 {
 	struct align_test_res *tmp;
 
@@ -1294,6 +1287,8 @@ static int add_to_align_test_list(struct list_head *head,
 
 	tmp->align = align;
 	tmp->size = size;
+	tmp->dev = dev;
+	tmp->idx = idx;
 
 	list_add_tail(&tmp->list, head);
 
@@ -1359,6 +1354,19 @@ static resource_size_t calculate_mem_align(struct list_head *head,
 	resource_size_t min_align, good_align, aligned_size, start;
 	int count = 0;
 
+	list_for_each_entry(p, head, list)
+		count++;
+
+	printk(KERN_DEBUG "  ===========BEGIN===calculate_mem_align========\n");
+	if (count) {
+		printk(KERN_DEBUG "  align/size:\n");
+		list_for_each_entry(p, head, list)
+			dev_printk(KERN_DEBUG, p->dev,
+				   "BAR %d:     %08llx/%08llx\n", p->idx,
+				   (unsigned long long)p->align,
+				   (unsigned long long)p->size);
+	}
+
 	if (max_align <= align_low) {
 		good_align = align_low;
 		goto out;
@@ -1366,9 +1374,6 @@ static resource_size_t calculate_mem_align(struct list_head *head,
 
 	good_align = max_align;
 
-	list_for_each_entry(p, head, list)
-		count++;
-
 	if (count <= 1)
 		goto out;
 
@@ -1393,6 +1398,11 @@ static resource_size_t calculate_mem_align(struct list_head *head,
 	} while (min_align > align_low);
 
 out:
+	printk(KERN_DEBUG "      min_align/aligned_size: %08llx/%08llx\n",
+			(unsigned long long)good_align,
+			(unsigned long long)ALIGN(size, good_align));
+	printk(KERN_DEBUG "  ===========END===calculate_mem_align==========\n");
+
 	return good_align;
 }
 
@@ -1410,6 +1420,16 @@ static resource_size_t calculate_mem_alt_size(struct list_head *head,
 	list_for_each_entry(p, head, list)
 		count++;
 
+	printk(KERN_DEBUG "  ===========BEGIN===calculate_mem_alt_size=====\n");
+	if (count) {
+		printk(KERN_DEBUG "  align/size:\n");
+		list_for_each_entry(p, head, list)
+			dev_printk(KERN_DEBUG, p->dev,
+				   "BAR %d:     %08llx/%08llx\n", p->idx,
+				   (unsigned long long)p->align,
+				   (unsigned long long)p->size);
+	}
+
 	if (count <= 1)
 		goto out;
 
@@ -1434,6 +1454,11 @@ static resource_size_t calculate_mem_alt_size(struct list_head *head,
 	}
 
 out:
+	printk(KERN_DEBUG "   alt_align/alt_size: %08llx/%08llx\n",
+			(unsigned long long)max_align,
+			(unsigned long long)good_size);
+	printk(KERN_DEBUG "  ===========END===calculate_mem_alt_size=======\n");
+
 	return good_size;
 }
 
@@ -1515,7 +1540,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 			/* put SRIOV/ROM res to realloc list */
 			if (realloc_head && is_optional(i)) {
 				add_to_align_test_list(&align_test_add_list,
-							align, r_size);
+						align, r_size, &dev->dev, i);
 				r->end = r->start - 1;
 				__add_to_list(realloc_head, dev, r,
 					      r_size, align, 0, 0);
@@ -1534,7 +1559,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 			if (r_size > 1) {
 				add_to_align_test_list(&align_test_list,
-							align, r_size);
+						align, r_size, &dev->dev, i);
 				size += r_size;
 				if (align > max_align)
 					max_align = align;
@@ -1551,7 +1576,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 					add_align = align;
 				add_to_align_test_list(&align_test_add_list,
 							add_align,
-							r_size + add_r_size);
+							r_size + add_r_size,
+							&dev->dev, i);
 				sum_add_size += r_size + add_r_size;
 				if (add_align > max_add_align)
 					max_add_align = add_align;
@@ -1561,14 +1587,14 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 					add_to_align_test_list(
 						&align_test_alt_list,
 						dev_res->alt_align,
-						dev_res->alt_size);
+						dev_res->alt_size, &dev->dev, i);
 					alt_size += dev_res->alt_size;
 					if (alt_align < dev_res->alt_align)
 						alt_align = dev_res->alt_align;
 				} else if (r_size > 1) {
 					add_to_align_test_list(
 						&align_test_alt_list,
-						align, r_size);
+						align, r_size, &dev->dev, i);
 					alt_size += r_size;
 					if (alt_align < align)
 						alt_align = align;
@@ -1579,6 +1605,9 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 	max_align = max(max_align, window_align);
 	if (size || min_size) {
+		dev_printk(KERN_DEBUG, &bus->self->dev,
+			   "BAR %d: bridge window %pR to %pR calculate_mem for MUST\n",
+			   (int)(b_res - &bus->self->resource[0]), b_res, &bus->busn_res);
 		min_align = calculate_mem_align(&align_test_list, max_align,
 						size, window_align);
 		size0 = calculate_size(size, min_size,
@@ -1588,6 +1617,9 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 	if (size0 && realloc_head) {
 		alt_align = max(alt_align, window_align);
+		dev_printk(KERN_DEBUG, &bus->self->dev,
+			   "BAR %d: bridge window %pR to %pR calculate_mem for ALT\n",
+			   (int)(b_res - &bus->self->resource[0]), b_res, &bus->busn_res);
 		/* need to increase size to fit more alt */
 		alt_size = calculate_mem_alt_size(&align_test_alt_list,
 						  alt_align, alt_size,
@@ -1603,6 +1635,9 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	if (sum_add_size < min_sum_size)
 		sum_add_size = min_sum_size;
 	if (sum_add_size > size && realloc_head) {
+		dev_printk(KERN_DEBUG, &bus->self->dev,
+			   "BAR %d: bridge window %pR to %pR calculate_mem for ADD\n",
+			   (int)(b_res - &bus->self->resource[0]), b_res, &bus->busn_res);
 		min_add_align = calculate_mem_align(&align_test_add_list,
 					max_add_align, sum_add_size,
 					window_align);
@@ -1660,7 +1695,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 				      final_add_size, min_add_align,
 				      alt_size, alt_align);
 			dev_printk(KERN_DEBUG, &bus->self->dev,
-				   "bridge window %pR to %pR add_size %llx add_align %llx alt_size %llx alt_align %llx req_size %llx req_align %llx\n",
+				   "BAR %d: bridge window %pR to %pR add_size %llx add_align %llx alt_size %llx alt_align %llx req_size %llx req_align %llx\n",
+				   (int)(b_res - &bus->self->resource[0]),
 				   b_res, &bus->busn_res,
 				   (unsigned long long)final_add_size,
 				   (unsigned long long)min_add_align,
-- 
1.8.4.5

[PATCH v8 58/61] PCI, x86: Add pci=assign_pref_bars to reallocate pref BARs

[Yinghai Lu]

Now some BIOS tend to allocate pref MMIO under non-pref MMIO, or allocate
64bit pref MMIO under 4G.

Add pci=assign_pref_bars to clear and allocate resource to pref BARS.
So could reallocate pref mmio64 above 4G and pref under bridges pref BARs.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 arch/x86/include/asm/pci_x86.h |  1 +
 arch/x86/pci/common.c          |  3 +++
 arch/x86/pci/i386.c            | 56 ++++++++++++++++++++++++++----------------
 3 files changed, 39 insertions(+), 21 deletions(-)

diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index 81a7abf..23bc5d6 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -34,6 +34,7 @@ do {						\
 #define PCI_NOASSIGN_ROMS	0x80000
 #define PCI_ROOT_NO_CRS		0x100000
 #define PCI_NOASSIGN_BARS	0x200000
+#define PCI_ASSIGN_PREF_BARS	0x400000
 
 extern unsigned int pci_probe;
 extern unsigned long pirq_table_addr;
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index 7c018df..7dc28c2 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -606,6 +606,9 @@ char *__init pcibios_setup(char *str)
 	} else if (!strcmp(str, "assign-busses")) {
 		pci_probe |= PCI_ASSIGN_ALL_BUSSES;
 		return NULL;
+	} else if (!strcmp(str, "assign_pref_bars")) {
+		pci_probe |= PCI_ASSIGN_PREF_BARS;
+		return NULL;
 	} else if (!strcmp(str, "use_crs")) {
 		pci_probe |= PCI_USE__CRS;
 		return NULL;
diff --git a/arch/x86/pci/i386.c b/arch/x86/pci/i386.c
index 21f3e3e..6f27780 100644
--- a/arch/x86/pci/i386.c
+++ b/arch/x86/pci/i386.c
@@ -228,16 +228,25 @@ static void pcibios_allocate_bridge_resources(struct pci_dev *dev)
 			continue;
 		if (r->parent)	/* Already allocated */
 			continue;
-		if (!r->start || pci_claim_bridge_resource(dev, idx) < 0) {
-			/*
-			 * Something is wrong with the region.
-			 * Invalidate the resource to prevent
-			 * child resource allocations in this
-			 * range.
-			 */
-			r->start = r->end = 0;
-			r->flags = 0;
-		}
+
+		if ((r->flags & IORESOURCE_PREFETCH) &&
+		    (pci_probe & PCI_ASSIGN_PREF_BARS))
+			goto clear;
+
+		if (!r->start)
+			goto clear;
+
+		if (pci_claim_bridge_resource(dev, idx) == 0)
+			continue;
+
+clear:
+		/*
+		 * Something is wrong with the region.
+		 * Invalidate the resource to prevent
+		 * child resource allocations in this range.
+		 */
+		r->start = r->end = 0;
+		r->flags = 0;
 	}
 }
 
@@ -283,21 +292,26 @@ static void pcibios_allocate_dev_resources(struct pci_dev *dev, int pass)
 			else
 				disabled = !(command & PCI_COMMAND_MEMORY);
 			if (pass == disabled) {
+				if ((r->flags & IORESOURCE_PREFETCH) &&
+				    (pci_probe & PCI_ASSIGN_PREF_BARS))
+					goto clear;
+
 				dev_dbg(&dev->dev,
 					"BAR %d: reserving %pr (d=%d, p=%d)\n",
 					idx, r, disabled, pass);
-				if (pci_claim_resource(dev, idx) < 0) {
-					if (r->flags & IORESOURCE_PCI_FIXED) {
-						dev_info(&dev->dev, "BAR %d %pR is immovable\n",
-							 idx, r);
-					} else {
-						/* We'll assign a new address later */
-						pcibios_save_fw_addr(dev,
-								idx, r->start);
-						r->end -= r->start;
-						r->start = 0;
-					}
+				if (pci_claim_resource(dev, idx) == 0)
+					continue;
+				if (r->flags & IORESOURCE_PCI_FIXED) {
+					dev_info(&dev->dev, "BAR %d %pR is immovable\n",
+						 idx, r);
+					continue;
 				}
+
+clear:
+				/* We'll assign a new address later */
+				pcibios_save_fw_addr(dev, idx, r->start);
+				r->end -= r->start;
+				r->start = 0;
 			}
 		}
 	if (!pass) {
-- 
1.8.4.5

[PATCH v8 59/61] PCI: Introduce resource_disabled()

[Yinghai Lu]

Current is using !flags, and we are going to use
IORESOURCE_DISABLED instead of clearing resource flags.

Let's convert all !flags to helper function resource_disabled().
resource_disabled will check !flags and IORESOURCE_DISABLED both.

Cc: linux-alpha@vger.kernel.org
Cc: linux-ia64@vger.kernel.org
Cc: linux-am33-list@redhat.com
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-s390@vger.kernel.org
Cc: sparclinux@vger.kernel.org
Cc: linux-pci@vger.kernel.org
Cc: linux-xtensa@linux-xtensa.org
Cc: iommu@lists.linux-foundation.org
Cc: linux-sh@vger.kernel.org
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Acked-by: Michael Ellerman <mpe@ellerman.id.au>
---
 arch/alpha/kernel/pci.c                   |  2 +-
 arch/ia64/pci/pci.c                       |  4 ++--
 arch/microblaze/pci/pci-common.c          | 15 ++++++++-------
 arch/mn10300/unit-asb2305/pci-asb2305.c   |  4 ++--
 arch/mn10300/unit-asb2305/pci.c           |  4 ++--
 arch/powerpc/kernel/pci-common.c          | 16 +++++++++-------
 arch/powerpc/platforms/powernv/pci-ioda.c | 12 ++++++------
 arch/s390/pci/pci.c                       |  2 +-
 arch/sparc/kernel/pci.c                   |  2 +-
 arch/x86/pci/i386.c                       |  4 ++--
 arch/xtensa/kernel/pci.c                  |  4 ++--
 drivers/iommu/intel-iommu.c               |  3 ++-
 drivers/pci/host/pcie-rcar.c              |  2 +-
 drivers/pci/iov.c                         |  2 +-
 drivers/pci/probe.c                       |  2 +-
 drivers/pci/quirks.c                      |  4 ++--
 drivers/pci/rom.c                         |  2 +-
 drivers/pci/setup-bus.c                   |  8 ++++----
 drivers/pci/setup-res.c                   |  2 +-
 include/linux/ioport.h                    |  4 ++++
 20 files changed, 53 insertions(+), 45 deletions(-)

diff --git a/arch/alpha/kernel/pci.c b/arch/alpha/kernel/pci.c
index 5f387ee..c89c8ef 100644
--- a/arch/alpha/kernel/pci.c
+++ b/arch/alpha/kernel/pci.c
@@ -282,7 +282,7 @@ pcibios_claim_one_bus(struct pci_bus *b)
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 
-			if (r->parent || !r->start || !r->flags)
+			if (r->parent || !r->start || resource_disabled(r))
 				continue;
 			if (pci_has_flag(PCI_PROBE_ONLY) ||
 			    (r->flags & IORESOURCE_PCI_FIXED)) {
diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
index 7cc3be9..cc293ea 100644
--- a/arch/ia64/pci/pci.c
+++ b/arch/ia64/pci/pci.c
@@ -501,7 +501,7 @@ void pcibios_fixup_device_resources(struct pci_dev *dev)
 	for (idx = 0; idx < PCI_BRIDGE_RESOURCES; idx++) {
 		struct resource *r = &dev->resource[idx];
 
-		if (!r->flags || r->parent || !r->start)
+		if (resource_disabled(r) || r->parent || !r->start)
 			continue;
 
 		pci_claim_resource(dev, idx);
@@ -519,7 +519,7 @@ static void pcibios_fixup_bridge_resources(struct pci_dev *dev)
 	for (idx = PCI_BRIDGE_RESOURCES; idx < PCI_NUM_RESOURCES; idx++) {
 		struct resource *r = &dev->resource[idx];
 
-		if (!r->flags || r->parent || !r->start)
+		if (resource_disabled(r) || r->parent || !r->start)
 			continue;
 
 		pci_claim_bridge_resource(dev, idx);
diff --git a/arch/microblaze/pci/pci-common.c b/arch/microblaze/pci/pci-common.c
index 09b1af6..c123d3c 100644
--- a/arch/microblaze/pci/pci-common.c
+++ b/arch/microblaze/pci/pci-common.c
@@ -705,7 +705,7 @@ static void pcibios_fixup_resources(struct pci_dev *dev)
 	}
 	for (i = 0; i < DEVICE_COUNT_RESOURCE; i++) {
 		struct resource *res = dev->resource + i;
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 		if (res->start = 0) {
 			pr_debug("PCI:%s Resource %d %016llx-%016llx [%x]",
@@ -806,7 +806,7 @@ static void pcibios_fixup_bridge(struct pci_bus *bus)
 	pci_bus_for_each_resource(bus, res, i) {
 		if (!res)
 			continue;
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 		if (i >= 3 && bus->self->transparent)
 			continue;
@@ -985,7 +985,7 @@ static void pcibios_allocate_bus_resources(struct pci_bus *bus)
 		 pci_domain_nr(bus), bus->number);
 
 	pci_bus_for_each_resource(bus, res, i) {
-		if (!res || !res->flags
+		if (!res || resource_disabled(res)
 		    || res->start > res->end || res->parent)
 			continue;
 		if (bus->parent = NULL)
@@ -1087,7 +1087,8 @@ static void __init pcibios_allocate_resources(int pass)
 			r = &dev->resource[idx];
 			if (r->parent)		/* Already allocated */
 				continue;
-			if (!r->flags || (r->flags & IORESOURCE_UNSET))
+			if (resource_disabled(r) ||
+			    (r->flags & IORESOURCE_UNSET))
 				continue;	/* Not assigned at all */
 			/* We only allocate ROMs on pass 1 just in case they
 			 * have been screwed up by firmware
@@ -1218,7 +1219,7 @@ void pcibios_claim_one_bus(struct pci_bus *bus)
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 
-			if (r->parent || !r->start || !r->flags)
+			if (r->parent || !r->start || resource_disabled(r))
 				continue;
 
 			pr_debug("PCI: Claiming %s: ", pci_name(dev));
@@ -1278,7 +1279,7 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose,
 	res->start = (res->start + io_offset) & 0xffffffffu;
 	res->end = (res->end + io_offset) & 0xffffffffu;
 
-	if (!res->flags) {
+	if (resource_disabled(res)) {
 		pr_warn("PCI: I/O resource not set for host ");
 		pr_cont("bridge %s (domain %d)\n",
 			hose->dn->full_name, hose->global_number);
@@ -1298,7 +1299,7 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose,
 	/* Hookup PHB Memory resources */
 	for (i = 0; i < 3; ++i) {
 		res = &hose->mem_resources[i];
-		if (!res->flags) {
+		if (resource_disabled(res)) {
 			if (i > 0)
 				continue;
 			pr_err("PCI: Memory resource 0 not set for ");
diff --git a/arch/mn10300/unit-asb2305/pci-asb2305.c b/arch/mn10300/unit-asb2305/pci-asb2305.c
index b7ab837..7e70e51 100644
--- a/arch/mn10300/unit-asb2305/pci-asb2305.c
+++ b/arch/mn10300/unit-asb2305/pci-asb2305.c
@@ -103,7 +103,7 @@ static void __init pcibios_allocate_bus_resources(struct list_head *bus_list)
 			     idx < PCI_NUM_RESOURCES;
 			     idx++) {
 				r = &dev->resource[idx];
-				if (!r->flags)
+				if (resource_disabled(r))
 					continue;
 				if (!r->start ||
 				    pci_claim_bridge_resource(dev, idx) < 0) {
@@ -188,7 +188,7 @@ static int __init pcibios_assign_resources(void)
 	   addresses. */
 	for_each_pci_dev(dev) {
 		r = &dev->resource[PCI_ROM_RESOURCE];
-		if (!r->flags || !r->start)
+		if (resource_disabled(r) || !r->start)
 			continue;
 		if (pci_claim_resource(dev, PCI_ROM_RESOURCE) < 0) {
 			r->end -= r->start;
diff --git a/arch/mn10300/unit-asb2305/pci.c b/arch/mn10300/unit-asb2305/pci.c
index 3dfe2d3..ad77b18 100644
--- a/arch/mn10300/unit-asb2305/pci.c
+++ b/arch/mn10300/unit-asb2305/pci.c
@@ -291,7 +291,7 @@ static void pcibios_fixup_device_resources(struct pci_dev *dev)
 	for (idx = 0; idx < PCI_BRIDGE_RESOURCES; idx++) {
 		struct resource *r = &dev->resource[idx];
 
-		if (!r->flags || r->parent || !r->start)
+		if (resource_disabled(r) || r->parent || !r->start)
 			continue;
 
 		pci_claim_resource(dev, idx);
@@ -308,7 +308,7 @@ static void pcibios_fixup_bridge_resources(struct pci_dev *dev)
 	for (idx = PCI_BRIDGE_RESOURCES; idx < PCI_NUM_RESOURCES; idx++) {
 		struct resource *r = &dev->resource[idx];
 
-		if (!r->flags || r->parent || !r->start)
+		if (resource_disabled(r) || r->parent || !r->start)
 			continue;
 
 		pci_claim_bridge_resource(dev, idx);
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 8853667..a830e0c 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -810,7 +810,7 @@ static void pcibios_fixup_resources(struct pci_dev *dev)
 	for (i = 0; i < DEVICE_COUNT_RESOURCE; i++) {
 		struct resource *res = dev->resource + i;
 		struct pci_bus_region reg;
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 
 		/* If we're going to re-assign everything, we mark all resources
@@ -919,7 +919,7 @@ static void pcibios_fixup_bridge(struct pci_bus *bus)
 	struct pci_dev *dev = bus->self;
 
 	pci_bus_for_each_resource(bus, res, i) {
-		if (!res || !res->flags)
+		if (!res || resource_disabled(res))
 			continue;
 		if (i >= 3 && bus->self->transparent)
 			continue;
@@ -1160,7 +1160,8 @@ static void pcibios_allocate_bus_resources(struct pci_bus *bus)
 		 pci_domain_nr(bus), bus->number);
 
 	pci_bus_for_each_resource(bus, res, i) {
-		if (!res || !res->flags || res->start > res->end || res->parent)
+		if (!res || resource_disabled(res) ||
+		    res->start > res->end || res->parent)
 			continue;
 
 		/* If the resource was left unset at this point, we clear it */
@@ -1255,7 +1256,8 @@ static void __init pcibios_allocate_resources(int pass)
 			r = &dev->resource[idx];
 			if (r->parent)		/* Already allocated */
 				continue;
-			if (!r->flags || (r->flags & IORESOURCE_UNSET))
+			if (resource_disabled(r) ||
+			    (r->flags & IORESOURCE_UNSET))
 				continue;	/* Not assigned at all */
 			/* We only allocate ROMs on pass 1 just in case they
 			 * have been screwed up by firmware
@@ -1393,7 +1395,7 @@ void pcibios_claim_one_bus(struct pci_bus *bus)
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 
-			if (r->parent || !r->start || !r->flags)
+			if (r->parent || !r->start || resource_disabled(r))
 				continue;
 
 			pr_debug("PCI: Claiming %s: Resource %d: %pR\n",
@@ -1474,7 +1476,7 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose,
 	/* Hookup PHB IO resource */
 	res = &hose->io_resource;
 
-	if (!res->flags) {
+	if (resource_disabled(res)) {
 		pr_info("PCI: I/O resource not set for host"
 		       " bridge %s (domain %d)\n",
 		       hose->dn->full_name, hose->global_number);
@@ -1489,7 +1491,7 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose,
 	/* Hookup PHB Memory resources */
 	for (i = 0; i < 3; ++i) {
 		res = &hose->mem_resources[i];
-		if (!res->flags) {
+		if (resource_disabled(res)) {
 			if (i = 0)
 				printk(KERN_ERR "PCI: Memory resource 0 not set for "
 				       "host bridge %s (domain %d)\n",
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 414fd1a..81484c1 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -874,7 +874,7 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)
 	num_vfs = pdn->num_vfs;
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &dev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || !res->parent)
+		if (resource_disabled(res) || !res->parent)
 			continue;
 
 		if (!pnv_pci_is_mem_pref_64(res->flags))
@@ -905,7 +905,7 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)
 	 */
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &dev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || !res->parent)
+		if (resource_disabled(res) || !res->parent)
 			continue;
 
 		if (!pnv_pci_is_mem_pref_64(res->flags))
@@ -1188,7 +1188,7 @@ static int pnv_pci_vf_assign_m64(struct pci_dev *pdev, u16 num_vfs)
 
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &pdev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || !res->parent)
+		if (resource_disabled(res) || !res->parent)
 			continue;
 
 		if (!pnv_pci_is_mem_pref_64(res->flags))
@@ -2757,7 +2757,7 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev)
 
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &pdev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || res->parent)
+		if (resource_disabled(res) || res->parent)
 			continue;
 		if (!pnv_pci_is_mem_pref_64(res->flags)) {
 			dev_warn(&pdev->dev, " non M64 VF BAR%d: %pR\n",
@@ -2779,7 +2779,7 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev)
 
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &pdev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || res->parent)
+		if (resource_disabled(res) || res->parent)
 			continue;
 		if (!pnv_pci_is_mem_pref_64(res->flags)) {
 			dev_warn(&pdev->dev, "Skipping expanding VF BAR%d: %pR\n",
@@ -2820,7 +2820,7 @@ static void pnv_ioda_setup_pe_seg(struct pci_controller *hose,
 	BUG_ON(!(pe->flags & (PNV_IODA_PE_BUS | PNV_IODA_PE_BUS_ALL)));
 
 	pci_bus_for_each_resource(pe->pbus, res, i) {
-		if (!res || !res->flags ||
+		if (!res || resource_disabled(res) ||
 		    res->start > res->end)
 			continue;
 
diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c
index 7ef12a3..72c4676 100644
--- a/arch/s390/pci/pci.c
+++ b/arch/s390/pci/pci.c
@@ -653,7 +653,7 @@ int pcibios_add_device(struct pci_dev *pdev)
 
 	for (i = 0; i < PCI_BAR_COUNT; i++) {
 		res = &pdev->resource[i];
-		if (res->parent || !res->flags)
+		if (res->parent || resource_disabled(res))
 			continue;
 		pci_claim_resource(pdev, i);
 	}
diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index 36a0dfb..afc4720 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -631,7 +631,7 @@ static void pci_claim_bus_resources(struct pci_bus *bus)
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 
-			if (r->parent || !r->start || !r->flags)
+			if (r->parent || !r->start || resource_disabled(r))
 				continue;
 
 			if (ofpci_verbose)
diff --git a/arch/x86/pci/i386.c b/arch/x86/pci/i386.c
index 6f27780..48c367c 100644
--- a/arch/x86/pci/i386.c
+++ b/arch/x86/pci/i386.c
@@ -224,7 +224,7 @@ static void pcibios_allocate_bridge_resources(struct pci_dev *dev)
 
 	for (idx = PCI_BRIDGE_RESOURCES; idx < PCI_NUM_RESOURCES; idx++) {
 		r = &dev->resource[idx];
-		if (!r->flags)
+		if (resource_disabled(r))
 			continue;
 		if (r->parent)	/* Already allocated */
 			continue;
@@ -353,7 +353,7 @@ static void pcibios_allocate_dev_rom_resource(struct pci_dev *dev)
 	 * addresses.
 	 */
 	r = &dev->resource[PCI_ROM_RESOURCE];
-	if (!r->flags || !r->start)
+	if (resource_disabled(r) || !r->start)
 		return;
 	if (r->parent) /* Already allocated */
 		return;
diff --git a/arch/xtensa/kernel/pci.c b/arch/xtensa/kernel/pci.c
index b848cc3..f34d061 100644
--- a/arch/xtensa/kernel/pci.c
+++ b/arch/xtensa/kernel/pci.c
@@ -142,7 +142,7 @@ static void __init pci_controller_apertures(struct pci_controller *pci_ctrl,
 
 	io_offset = (unsigned long)pci_ctrl->io_space.base;
 	res = &pci_ctrl->io_resource;
-	if (!res->flags) {
+	if (resource_disabled(res)) {
 		if (io_offset)
 			printk (KERN_ERR "I/O resource not set for host"
 				" bridge %d\n", pci_ctrl->index);
@@ -156,7 +156,7 @@ static void __init pci_controller_apertures(struct pci_controller *pci_ctrl,
 
 	for (i = 0; i < 3; i++) {
 		res = &pci_ctrl->mem_resources[i];
-		if (!res->flags) {
+		if (resource_disabled(res)) {
 			if (i > 0)
 				continue;
 			printk(KERN_ERR "Memory resource not set for "
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 041bc18..d4a06d2 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1771,7 +1771,8 @@ static int dmar_init_reserved_ranges(void)
 
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			r = &pdev->resource[i];
-			if (!r->flags || !(r->flags & IORESOURCE_MEM))
+			if (resource_disabled(r) ||
+			    !(r->flags & IORESOURCE_MEM))
 				continue;
 			iova = reserve_iova(&reserved_iova_list,
 					    IOVA_PFN(r->start),
diff --git a/drivers/pci/host/pcie-rcar.c b/drivers/pci/host/pcie-rcar.c
index 7678fe0..3f307a3 100644
--- a/drivers/pci/host/pcie-rcar.c
+++ b/drivers/pci/host/pcie-rcar.c
@@ -365,7 +365,7 @@ static int rcar_pcie_setup(int nr, struct pci_sys_data *sys)
 	for (i = 0; i < RCAR_PCI_MAX_RESOURCES; i++) {
 
 		res = &pcie->res[i];
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 
 		rcar_pcie_setup_window(i, pcie);
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index ee0ebff..808d08b 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -438,7 +438,7 @@ found:
 		res = &dev->resource[i + PCI_IOV_RESOURCES];
 		bar64 = __pci_read_base(dev, pci_bar_unknown, res,
 					pos + PCI_SRIOV_BAR + i * 4);
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 		if (resource_size(res) & (PAGE_SIZE - 1)) {
 			rc = -EIO;
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index edc8d08..f9589d9 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2203,7 +2203,7 @@ void pci_bus_release_busn_res(struct pci_bus *b)
 	struct resource *res = &b->busn_res;
 	int ret;
 
-	if (!res->flags || !res->parent)
+	if (resource_disabled(res) || !res->parent)
 		return;
 
 	ret = release_resource(res);
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index be0bdfb..c51c0e6 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -336,7 +336,7 @@ static void quirk_bar_fixed(struct pci_dev *dev)
 	for (i = 0; i < PCI_STD_RESOURCE_END; i++) {
 		struct resource *r = &dev->resource[i];
 
-		if (!r->start || !r->flags)
+		if (!r->start || resource_disabled(r))
 			continue;
 		r->flags |= IORESOURCE_PCI_FIXED;
 	}
@@ -350,7 +350,7 @@ static void quirk_allocate_fixed(struct pci_dev *dev)
 	for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 		struct resource *r = &dev->resource[i];
 
-		if (r->parent ||
+		if (r->parent || resource_disabled(r) ||
 		    !(r->flags & IORESOURCE_PCI_FIXED) ||
 		    !(r->flags & (IORESOURCE_IO | IORESOURCE_MEM)))
 			continue;
diff --git a/drivers/pci/rom.c b/drivers/pci/rom.c
index eb0ad53..585a9d3 100644
--- a/drivers/pci/rom.c
+++ b/drivers/pci/rom.c
@@ -28,7 +28,7 @@ int pci_enable_rom(struct pci_dev *pdev)
 	struct pci_bus_region region;
 	u32 rom_addr;
 
-	if (!res->flags)
+	if (resource_disabled(res))
 		return -1;
 
 	pcibios_resource_to_bus(pdev->bus, &region, res);
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 673778c..23318bd 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -223,7 +223,7 @@ static void pdev_assign_resources_prepare(struct pci_dev *dev,
 		if (r->flags & IORESOURCE_PCI_FIXED)
 			continue;
 
-		if (!(r->flags) || r->parent)
+		if (resource_disabled(r) || r->parent)
 			continue;
 
 		r_align = __pci_resource_alignment(dev, r, realloc_head);
@@ -319,7 +319,7 @@ static void reassign_resources_sorted(struct list_head *realloc_head,
 	list_for_each_entry_safe(add_res, tmp, realloc_head, list) {
 		res = add_res->res;
 		/* skip resource that has been reset */
-		if (!res->flags)
+		if (resource_disabled(res))
 			goto out;
 
 		/* skip this resource if not found in head list */
@@ -2124,7 +2124,7 @@ static void pci_bus_dump_res(struct pci_bus *bus)
 	int i;
 
 	pci_bus_for_each_resource(bus, res, i) {
-		if (!res || !res->end || !res->flags)
+		if (!res || !res->end || resource_disabled(res))
 			continue;
 
 		dev_printk(KERN_DEBUG, &bus->dev, "resource %d %pR\n", i, res);
@@ -2207,7 +2207,7 @@ static int iov_resources_unassigned(struct pci_dev *dev, void *data)
 		struct pci_bus_region region;
 
 		/* Not assigned or rejected by kernel? */
-		if (!r->flags)
+		if (resource_disabled(r))
 			continue;
 
 		pcibios_resource_to_bus(dev->bus, &region, r);
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 26aedde..55caf7a 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -40,7 +40,7 @@ void pci_update_resource(struct pci_dev *dev, int resno)
 	 * Ignore resources for unimplemented BARs and unused resource slots
 	 * for 64 bit BARs.
 	 */
-	if (!res->flags)
+	if (resource_disabled(res))
 		return;
 
 	if (res->flags & IORESOURCE_UNSET)
diff --git a/include/linux/ioport.h b/include/linux/ioport.h
index 27dbb18..fb21179 100644
--- a/include/linux/ioport.h
+++ b/include/linux/ioport.h
@@ -179,6 +179,10 @@ static inline bool resource_contains(struct resource *r1, struct resource *r2)
 	return r1->start <= r2->start && r1->end >= r2->end;
 }
 
+static inline bool resource_disabled(struct resource *r)
+{
+	return !r->flags || (r->flags & IORESOURCE_DISABLED);
+}
 
 /* Convenience shorthand with allocation */
 #define request_region(start,n,name)		__request_region(&ioport_resource, (start), (n), (name), 0)
-- 
1.8.4.5

[PATCH v8 59/61] PCI: Introduce resource_disabled()

[Yinghai Lu]

Current is using !flags, and we are going to use
IORESOURCE_DISABLED instead of clearing resource flags.

Let's convert all !flags to helper function resource_disabled().
resource_disabled will check !flags and IORESOURCE_DISABLED both.

Cc: linux-alpha@vger.kernel.org
Cc: linux-ia64@vger.kernel.org
Cc: linux-am33-list@redhat.com
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-s390@vger.kernel.org
Cc: sparclinux@vger.kernel.org
Cc: linux-pci@vger.kernel.org
Cc: linux-xtensa@linux-xtensa.org
Cc: iommu@lists.linux-foundation.org
Cc: linux-sh@vger.kernel.org
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Acked-by: Michael Ellerman <mpe@ellerman.id.au>
---
 arch/alpha/kernel/pci.c                   |  2 +-
 arch/ia64/pci/pci.c                       |  4 ++--
 arch/microblaze/pci/pci-common.c          | 15 ++++++++-------
 arch/mn10300/unit-asb2305/pci-asb2305.c   |  4 ++--
 arch/mn10300/unit-asb2305/pci.c           |  4 ++--
 arch/powerpc/kernel/pci-common.c          | 16 +++++++++-------
 arch/powerpc/platforms/powernv/pci-ioda.c | 12 ++++++------
 arch/s390/pci/pci.c                       |  2 +-
 arch/sparc/kernel/pci.c                   |  2 +-
 arch/x86/pci/i386.c                       |  4 ++--
 arch/xtensa/kernel/pci.c                  |  4 ++--
 drivers/iommu/intel-iommu.c               |  3 ++-
 drivers/pci/host/pcie-rcar.c              |  2 +-
 drivers/pci/iov.c                         |  2 +-
 drivers/pci/probe.c                       |  2 +-
 drivers/pci/quirks.c                      |  4 ++--
 drivers/pci/rom.c                         |  2 +-
 drivers/pci/setup-bus.c                   |  8 ++++----
 drivers/pci/setup-res.c                   |  2 +-
 include/linux/ioport.h                    |  4 ++++
 20 files changed, 53 insertions(+), 45 deletions(-)

diff --git a/arch/alpha/kernel/pci.c b/arch/alpha/kernel/pci.c
index 5f387ee..c89c8ef 100644
--- a/arch/alpha/kernel/pci.c
+++ b/arch/alpha/kernel/pci.c
@@ -282,7 +282,7 @@ pcibios_claim_one_bus(struct pci_bus *b)
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 
-			if (r->parent || !r->start || !r->flags)
+			if (r->parent || !r->start || resource_disabled(r))
 				continue;
 			if (pci_has_flag(PCI_PROBE_ONLY) ||
 			    (r->flags & IORESOURCE_PCI_FIXED)) {
diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
index 7cc3be9..cc293ea 100644
--- a/arch/ia64/pci/pci.c
+++ b/arch/ia64/pci/pci.c
@@ -501,7 +501,7 @@ void pcibios_fixup_device_resources(struct pci_dev *dev)
 	for (idx = 0; idx < PCI_BRIDGE_RESOURCES; idx++) {
 		struct resource *r = &dev->resource[idx];
 
-		if (!r->flags || r->parent || !r->start)
+		if (resource_disabled(r) || r->parent || !r->start)
 			continue;
 
 		pci_claim_resource(dev, idx);
@@ -519,7 +519,7 @@ static void pcibios_fixup_bridge_resources(struct pci_dev *dev)
 	for (idx = PCI_BRIDGE_RESOURCES; idx < PCI_NUM_RESOURCES; idx++) {
 		struct resource *r = &dev->resource[idx];
 
-		if (!r->flags || r->parent || !r->start)
+		if (resource_disabled(r) || r->parent || !r->start)
 			continue;
 
 		pci_claim_bridge_resource(dev, idx);
diff --git a/arch/microblaze/pci/pci-common.c b/arch/microblaze/pci/pci-common.c
index 09b1af6..c123d3c 100644
--- a/arch/microblaze/pci/pci-common.c
+++ b/arch/microblaze/pci/pci-common.c
@@ -705,7 +705,7 @@ static void pcibios_fixup_resources(struct pci_dev *dev)
 	}
 	for (i = 0; i < DEVICE_COUNT_RESOURCE; i++) {
 		struct resource *res = dev->resource + i;
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 		if (res->start == 0) {
 			pr_debug("PCI:%s Resource %d %016llx-%016llx [%x]",
@@ -806,7 +806,7 @@ static void pcibios_fixup_bridge(struct pci_bus *bus)
 	pci_bus_for_each_resource(bus, res, i) {
 		if (!res)
 			continue;
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 		if (i >= 3 && bus->self->transparent)
 			continue;
@@ -985,7 +985,7 @@ static void pcibios_allocate_bus_resources(struct pci_bus *bus)
 		 pci_domain_nr(bus), bus->number);
 
 	pci_bus_for_each_resource(bus, res, i) {
-		if (!res || !res->flags
+		if (!res || resource_disabled(res)
 		    || res->start > res->end || res->parent)
 			continue;
 		if (bus->parent == NULL)
@@ -1087,7 +1087,8 @@ static void __init pcibios_allocate_resources(int pass)
 			r = &dev->resource[idx];
 			if (r->parent)		/* Already allocated */
 				continue;
-			if (!r->flags || (r->flags & IORESOURCE_UNSET))
+			if (resource_disabled(r) ||
+			    (r->flags & IORESOURCE_UNSET))
 				continue;	/* Not assigned at all */
 			/* We only allocate ROMs on pass 1 just in case they
 			 * have been screwed up by firmware
@@ -1218,7 +1219,7 @@ void pcibios_claim_one_bus(struct pci_bus *bus)
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 
-			if (r->parent || !r->start || !r->flags)
+			if (r->parent || !r->start || resource_disabled(r))
 				continue;
 
 			pr_debug("PCI: Claiming %s: ", pci_name(dev));
@@ -1278,7 +1279,7 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose,
 	res->start = (res->start + io_offset) & 0xffffffffu;
 	res->end = (res->end + io_offset) & 0xffffffffu;
 
-	if (!res->flags) {
+	if (resource_disabled(res)) {
 		pr_warn("PCI: I/O resource not set for host ");
 		pr_cont("bridge %s (domain %d)\n",
 			hose->dn->full_name, hose->global_number);
@@ -1298,7 +1299,7 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose,
 	/* Hookup PHB Memory resources */
 	for (i = 0; i < 3; ++i) {
 		res = &hose->mem_resources[i];
-		if (!res->flags) {
+		if (resource_disabled(res)) {
 			if (i > 0)
 				continue;
 			pr_err("PCI: Memory resource 0 not set for ");
diff --git a/arch/mn10300/unit-asb2305/pci-asb2305.c b/arch/mn10300/unit-asb2305/pci-asb2305.c
index b7ab837..7e70e51 100644
--- a/arch/mn10300/unit-asb2305/pci-asb2305.c
+++ b/arch/mn10300/unit-asb2305/pci-asb2305.c
@@ -103,7 +103,7 @@ static void __init pcibios_allocate_bus_resources(struct list_head *bus_list)
 			     idx < PCI_NUM_RESOURCES;
 			     idx++) {
 				r = &dev->resource[idx];
-				if (!r->flags)
+				if (resource_disabled(r))
 					continue;
 				if (!r->start ||
 				    pci_claim_bridge_resource(dev, idx) < 0) {
@@ -188,7 +188,7 @@ static int __init pcibios_assign_resources(void)
 	   addresses. */
 	for_each_pci_dev(dev) {
 		r = &dev->resource[PCI_ROM_RESOURCE];
-		if (!r->flags || !r->start)
+		if (resource_disabled(r) || !r->start)
 			continue;
 		if (pci_claim_resource(dev, PCI_ROM_RESOURCE) < 0) {
 			r->end -= r->start;
diff --git a/arch/mn10300/unit-asb2305/pci.c b/arch/mn10300/unit-asb2305/pci.c
index 3dfe2d3..ad77b18 100644
--- a/arch/mn10300/unit-asb2305/pci.c
+++ b/arch/mn10300/unit-asb2305/pci.c
@@ -291,7 +291,7 @@ static void pcibios_fixup_device_resources(struct pci_dev *dev)
 	for (idx = 0; idx < PCI_BRIDGE_RESOURCES; idx++) {
 		struct resource *r = &dev->resource[idx];
 
-		if (!r->flags || r->parent || !r->start)
+		if (resource_disabled(r) || r->parent || !r->start)
 			continue;
 
 		pci_claim_resource(dev, idx);
@@ -308,7 +308,7 @@ static void pcibios_fixup_bridge_resources(struct pci_dev *dev)
 	for (idx = PCI_BRIDGE_RESOURCES; idx < PCI_NUM_RESOURCES; idx++) {
 		struct resource *r = &dev->resource[idx];
 
-		if (!r->flags || r->parent || !r->start)
+		if (resource_disabled(r) || r->parent || !r->start)
 			continue;
 
 		pci_claim_bridge_resource(dev, idx);
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 8853667..a830e0c 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -810,7 +810,7 @@ static void pcibios_fixup_resources(struct pci_dev *dev)
 	for (i = 0; i < DEVICE_COUNT_RESOURCE; i++) {
 		struct resource *res = dev->resource + i;
 		struct pci_bus_region reg;
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 
 		/* If we're going to re-assign everything, we mark all resources
@@ -919,7 +919,7 @@ static void pcibios_fixup_bridge(struct pci_bus *bus)
 	struct pci_dev *dev = bus->self;
 
 	pci_bus_for_each_resource(bus, res, i) {
-		if (!res || !res->flags)
+		if (!res || resource_disabled(res))
 			continue;
 		if (i >= 3 && bus->self->transparent)
 			continue;
@@ -1160,7 +1160,8 @@ static void pcibios_allocate_bus_resources(struct pci_bus *bus)
 		 pci_domain_nr(bus), bus->number);
 
 	pci_bus_for_each_resource(bus, res, i) {
-		if (!res || !res->flags || res->start > res->end || res->parent)
+		if (!res || resource_disabled(res) ||
+		    res->start > res->end || res->parent)
 			continue;
 
 		/* If the resource was left unset at this point, we clear it */
@@ -1255,7 +1256,8 @@ static void __init pcibios_allocate_resources(int pass)
 			r = &dev->resource[idx];
 			if (r->parent)		/* Already allocated */
 				continue;
-			if (!r->flags || (r->flags & IORESOURCE_UNSET))
+			if (resource_disabled(r) ||
+			    (r->flags & IORESOURCE_UNSET))
 				continue;	/* Not assigned at all */
 			/* We only allocate ROMs on pass 1 just in case they
 			 * have been screwed up by firmware
@@ -1393,7 +1395,7 @@ void pcibios_claim_one_bus(struct pci_bus *bus)
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 
-			if (r->parent || !r->start || !r->flags)
+			if (r->parent || !r->start || resource_disabled(r))
 				continue;
 
 			pr_debug("PCI: Claiming %s: Resource %d: %pR\n",
@@ -1474,7 +1476,7 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose,
 	/* Hookup PHB IO resource */
 	res = &hose->io_resource;
 
-	if (!res->flags) {
+	if (resource_disabled(res)) {
 		pr_info("PCI: I/O resource not set for host"
 		       " bridge %s (domain %d)\n",
 		       hose->dn->full_name, hose->global_number);
@@ -1489,7 +1491,7 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose,
 	/* Hookup PHB Memory resources */
 	for (i = 0; i < 3; ++i) {
 		res = &hose->mem_resources[i];
-		if (!res->flags) {
+		if (resource_disabled(res)) {
 			if (i == 0)
 				printk(KERN_ERR "PCI: Memory resource 0 not set for "
 				       "host bridge %s (domain %d)\n",
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 414fd1a..81484c1 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -874,7 +874,7 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)
 	num_vfs = pdn->num_vfs;
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &dev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || !res->parent)
+		if (resource_disabled(res) || !res->parent)
 			continue;
 
 		if (!pnv_pci_is_mem_pref_64(res->flags))
@@ -905,7 +905,7 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)
 	 */
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &dev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || !res->parent)
+		if (resource_disabled(res) || !res->parent)
 			continue;
 
 		if (!pnv_pci_is_mem_pref_64(res->flags))
@@ -1188,7 +1188,7 @@ static int pnv_pci_vf_assign_m64(struct pci_dev *pdev, u16 num_vfs)
 
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &pdev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || !res->parent)
+		if (resource_disabled(res) || !res->parent)
 			continue;
 
 		if (!pnv_pci_is_mem_pref_64(res->flags))
@@ -2757,7 +2757,7 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev)
 
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &pdev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || res->parent)
+		if (resource_disabled(res) || res->parent)
 			continue;
 		if (!pnv_pci_is_mem_pref_64(res->flags)) {
 			dev_warn(&pdev->dev, " non M64 VF BAR%d: %pR\n",
@@ -2779,7 +2779,7 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev)
 
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &pdev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || res->parent)
+		if (resource_disabled(res) || res->parent)
 			continue;
 		if (!pnv_pci_is_mem_pref_64(res->flags)) {
 			dev_warn(&pdev->dev, "Skipping expanding VF BAR%d: %pR\n",
@@ -2820,7 +2820,7 @@ static void pnv_ioda_setup_pe_seg(struct pci_controller *hose,
 	BUG_ON(!(pe->flags & (PNV_IODA_PE_BUS | PNV_IODA_PE_BUS_ALL)));
 
 	pci_bus_for_each_resource(pe->pbus, res, i) {
-		if (!res || !res->flags ||
+		if (!res || resource_disabled(res) ||
 		    res->start > res->end)
 			continue;
 
diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c
index 7ef12a3..72c4676 100644
--- a/arch/s390/pci/pci.c
+++ b/arch/s390/pci/pci.c
@@ -653,7 +653,7 @@ int pcibios_add_device(struct pci_dev *pdev)
 
 	for (i = 0; i < PCI_BAR_COUNT; i++) {
 		res = &pdev->resource[i];
-		if (res->parent || !res->flags)
+		if (res->parent || resource_disabled(res))
 			continue;
 		pci_claim_resource(pdev, i);
 	}
diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index 36a0dfb..afc4720 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -631,7 +631,7 @@ static void pci_claim_bus_resources(struct pci_bus *bus)
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 
-			if (r->parent || !r->start || !r->flags)
+			if (r->parent || !r->start || resource_disabled(r))
 				continue;
 
 			if (ofpci_verbose)
diff --git a/arch/x86/pci/i386.c b/arch/x86/pci/i386.c
index 6f27780..48c367c 100644
--- a/arch/x86/pci/i386.c
+++ b/arch/x86/pci/i386.c
@@ -224,7 +224,7 @@ static void pcibios_allocate_bridge_resources(struct pci_dev *dev)
 
 	for (idx = PCI_BRIDGE_RESOURCES; idx < PCI_NUM_RESOURCES; idx++) {
 		r = &dev->resource[idx];
-		if (!r->flags)
+		if (resource_disabled(r))
 			continue;
 		if (r->parent)	/* Already allocated */
 			continue;
@@ -353,7 +353,7 @@ static void pcibios_allocate_dev_rom_resource(struct pci_dev *dev)
 	 * addresses.
 	 */
 	r = &dev->resource[PCI_ROM_RESOURCE];
-	if (!r->flags || !r->start)
+	if (resource_disabled(r) || !r->start)
 		return;
 	if (r->parent) /* Already allocated */
 		return;
diff --git a/arch/xtensa/kernel/pci.c b/arch/xtensa/kernel/pci.c
index b848cc3..f34d061 100644
--- a/arch/xtensa/kernel/pci.c
+++ b/arch/xtensa/kernel/pci.c
@@ -142,7 +142,7 @@ static void __init pci_controller_apertures(struct pci_controller *pci_ctrl,
 
 	io_offset = (unsigned long)pci_ctrl->io_space.base;
 	res = &pci_ctrl->io_resource;
-	if (!res->flags) {
+	if (resource_disabled(res)) {
 		if (io_offset)
 			printk (KERN_ERR "I/O resource not set for host"
 				" bridge %d\n", pci_ctrl->index);
@@ -156,7 +156,7 @@ static void __init pci_controller_apertures(struct pci_controller *pci_ctrl,
 
 	for (i = 0; i < 3; i++) {
 		res = &pci_ctrl->mem_resources[i];
-		if (!res->flags) {
+		if (resource_disabled(res)) {
 			if (i > 0)
 				continue;
 			printk(KERN_ERR "Memory resource not set for "
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 041bc18..d4a06d2 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1771,7 +1771,8 @@ static int dmar_init_reserved_ranges(void)
 
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			r = &pdev->resource[i];
-			if (!r->flags || !(r->flags & IORESOURCE_MEM))
+			if (resource_disabled(r) ||
+			    !(r->flags & IORESOURCE_MEM))
 				continue;
 			iova = reserve_iova(&reserved_iova_list,
 					    IOVA_PFN(r->start),
diff --git a/drivers/pci/host/pcie-rcar.c b/drivers/pci/host/pcie-rcar.c
index 7678fe0..3f307a3 100644
--- a/drivers/pci/host/pcie-rcar.c
+++ b/drivers/pci/host/pcie-rcar.c
@@ -365,7 +365,7 @@ static int rcar_pcie_setup(int nr, struct pci_sys_data *sys)
 	for (i = 0; i < RCAR_PCI_MAX_RESOURCES; i++) {
 
 		res = &pcie->res[i];
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 
 		rcar_pcie_setup_window(i, pcie);
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index ee0ebff..808d08b 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -438,7 +438,7 @@ found:
 		res = &dev->resource[i + PCI_IOV_RESOURCES];
 		bar64 = __pci_read_base(dev, pci_bar_unknown, res,
 					pos + PCI_SRIOV_BAR + i * 4);
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 		if (resource_size(res) & (PAGE_SIZE - 1)) {
 			rc = -EIO;
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index edc8d08..f9589d9 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2203,7 +2203,7 @@ void pci_bus_release_busn_res(struct pci_bus *b)
 	struct resource *res = &b->busn_res;
 	int ret;
 
-	if (!res->flags || !res->parent)
+	if (resource_disabled(res) || !res->parent)
 		return;
 
 	ret = release_resource(res);
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index be0bdfb..c51c0e6 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -336,7 +336,7 @@ static void quirk_bar_fixed(struct pci_dev *dev)
 	for (i = 0; i < PCI_STD_RESOURCE_END; i++) {
 		struct resource *r = &dev->resource[i];
 
-		if (!r->start || !r->flags)
+		if (!r->start || resource_disabled(r))
 			continue;
 		r->flags |= IORESOURCE_PCI_FIXED;
 	}
@@ -350,7 +350,7 @@ static void quirk_allocate_fixed(struct pci_dev *dev)
 	for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 		struct resource *r = &dev->resource[i];
 
-		if (r->parent ||
+		if (r->parent || resource_disabled(r) ||
 		    !(r->flags & IORESOURCE_PCI_FIXED) ||
 		    !(r->flags & (IORESOURCE_IO | IORESOURCE_MEM)))
 			continue;
diff --git a/drivers/pci/rom.c b/drivers/pci/rom.c
index eb0ad53..585a9d3 100644
--- a/drivers/pci/rom.c
+++ b/drivers/pci/rom.c
@@ -28,7 +28,7 @@ int pci_enable_rom(struct pci_dev *pdev)
 	struct pci_bus_region region;
 	u32 rom_addr;
 
-	if (!res->flags)
+	if (resource_disabled(res))
 		return -1;
 
 	pcibios_resource_to_bus(pdev->bus, &region, res);
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 673778c..23318bd 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -223,7 +223,7 @@ static void pdev_assign_resources_prepare(struct pci_dev *dev,
 		if (r->flags & IORESOURCE_PCI_FIXED)
 			continue;
 
-		if (!(r->flags) || r->parent)
+		if (resource_disabled(r) || r->parent)
 			continue;
 
 		r_align = __pci_resource_alignment(dev, r, realloc_head);
@@ -319,7 +319,7 @@ static void reassign_resources_sorted(struct list_head *realloc_head,
 	list_for_each_entry_safe(add_res, tmp, realloc_head, list) {
 		res = add_res->res;
 		/* skip resource that has been reset */
-		if (!res->flags)
+		if (resource_disabled(res))
 			goto out;
 
 		/* skip this resource if not found in head list */
@@ -2124,7 +2124,7 @@ static void pci_bus_dump_res(struct pci_bus *bus)
 	int i;
 
 	pci_bus_for_each_resource(bus, res, i) {
-		if (!res || !res->end || !res->flags)
+		if (!res || !res->end || resource_disabled(res))
 			continue;
 
 		dev_printk(KERN_DEBUG, &bus->dev, "resource %d %pR\n", i, res);
@@ -2207,7 +2207,7 @@ static int iov_resources_unassigned(struct pci_dev *dev, void *data)
 		struct pci_bus_region region;
 
 		/* Not assigned or rejected by kernel? */
-		if (!r->flags)
+		if (resource_disabled(r))
 			continue;
 
 		pcibios_resource_to_bus(dev->bus, &region, r);
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 26aedde..55caf7a 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -40,7 +40,7 @@ void pci_update_resource(struct pci_dev *dev, int resno)
 	 * Ignore resources for unimplemented BARs and unused resource slots
 	 * for 64 bit BARs.
 	 */
-	if (!res->flags)
+	if (resource_disabled(res))
 		return;
 
 	if (res->flags & IORESOURCE_UNSET)
diff --git a/include/linux/ioport.h b/include/linux/ioport.h
index 27dbb18..fb21179 100644
--- a/include/linux/ioport.h
+++ b/include/linux/ioport.h
@@ -179,6 +179,10 @@ static inline bool resource_contains(struct resource *r1, struct resource *r2)
 	return r1->start <= r2->start && r1->end >= r2->end;
 }
 
+static inline bool resource_disabled(struct resource *r)
+{
+	return !r->flags || (r->flags & IORESOURCE_DISABLED);
+}
 
 /* Convenience shorthand with allocation */
 #define request_region(start,n,name)		__request_region(&ioport_resource, (start), (n), (name), 0)
-- 
1.8.4.5

[PATCH v8 60/61] PCI: Don't set flags to 0 when assign resource fail

[Yinghai Lu]

We want to keep resource flags instead of clearing it after resource
allocation fails.

Make flags take IORESOURCE_UNSET | IORESOURCE_DISABLED instead.

Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/bus.c       |  2 +-
 drivers/pci/setup-bus.c | 45 +++++++++++++++++++++++----------------------
 drivers/pci/setup-res.c |  3 ++-
 3 files changed, 26 insertions(+), 24 deletions(-)

diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index 0859e77..69f1b5e 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -140,7 +140,7 @@ static int pci_bus_alloc_from_region(struct pci_bus *bus, struct resource *res,
 	type_mask |= IORESOURCE_TYPE_BITS;
 
 	pci_bus_for_each_resource(bus, r, i) {
-		if (!r)
+		if (!r || resource_disabled(r))
 			continue;
 
 		/* type_mask must match */
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 23318bd..22ee1a1 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -264,13 +264,6 @@ static void __dev_assign_resources_prepare(struct pci_dev *dev,
 	pdev_assign_resources_prepare(dev, realloc_head, head);
 }
 
-static inline void reset_resource(struct resource *res)
-{
-	res->start = 0;
-	res->end = 0;
-	res->flags = 0;
-}
-
 static void sort_resources(struct list_head *head)
 {
 	struct pci_dev_resource *res1, *tmp_res, *res2;
@@ -337,7 +330,7 @@ static void reassign_resources_sorted(struct list_head *realloc_head,
 			res->start = align;
 			res->end = res->start + add_size - 1;
 			if (pci_assign_resource(add_res->dev, idx))
-				reset_resource(res);
+				res->flags |= IORESOURCE_DISABLED;
 		} else {
 			/* could just assigned with alt, add difference ? */
 			resource_size_t size;
@@ -391,7 +384,7 @@ static void assign_requested_resources_sorted(struct list_head *head,
 		    pci_assign_resource(dev_res->dev, idx)) {
 			if (fail_head)
 				add_to_list(fail_head, dev_res->dev, res);
-			reset_resource(res);
+			res->flags |= IORESOURCE_DISABLED;
 		}
 	}
 }
@@ -677,7 +670,7 @@ static void __assign_resources_alt_sorted(struct list_head *head,
 
 		if (!res_to_dev_res(local_fail_head, res))
 			add_to_list(local_fail_head, fail_res->dev, res);
-		reset_resource(res);
+		res->flags |= IORESOURCE_UNSET | IORESOURCE_DISABLED;
 	}
 	free_list(&local_alt_fail_head);
 }
@@ -843,7 +836,7 @@ static void pci_setup_bridge_io(struct pci_dev *bridge)
 	/* Set up the top and bottom of the PCI I/O segment for this bus. */
 	res = &bridge->resource[PCI_BRIDGE_RESOURCES + 0];
 	pcibios_resource_to_bus(bridge->bus, &region, res);
-	if (res->flags & IORESOURCE_IO) {
+	if ((res->flags & IORESOURCE_IO) && !(res->flags & IORESOURCE_UNSET)) {
 		pci_read_config_word(bridge, PCI_IO_BASE, &l);
 		io_base_lo = (region.start >> 8) & io_mask;
 		io_limit_lo = (region.end >> 8) & io_mask;
@@ -873,7 +866,8 @@ static void pci_setup_bridge_mmio(struct pci_dev *bridge)
 	/* Set up the top and bottom of the PCI Memory segment for this bus. */
 	res = &bridge->resource[PCI_BRIDGE_RESOURCES + 1];
 	pcibios_resource_to_bus(bridge->bus, &region, res);
-	if (res->flags & IORESOURCE_MEM) {
+	if ((res->flags & IORESOURCE_MEM) &&
+	    !(res->flags & IORESOURCE_UNSET)) {
 		l = (region.start >> 16) & 0xfff0;
 		l |= region.end & 0xfff00000;
 		dev_info(&bridge->dev, "  bridge window %pR\n", res);
@@ -898,7 +892,8 @@ static void pci_setup_bridge_mmio_pref(struct pci_dev *bridge)
 	bu = lu = 0;
 	res = &bridge->resource[PCI_BRIDGE_RESOURCES + 2];
 	pcibios_resource_to_bus(bridge->bus, &region, res);
-	if (res->flags & IORESOURCE_PREFETCH) {
+	if ((res->flags & IORESOURCE_PREFETCH) &&
+	    !(res->flags & IORESOURCE_UNSET)) {
 		l = (region.start >> 16) & 0xfff0;
 		l |= region.end & 0xfff00000;
 		if (res->flags & IORESOURCE_MEM_64) {
@@ -1028,6 +1023,7 @@ static void pci_bridge_check_ranges(struct pci_bus *bus)
 
 	b_res = &bridge->resource[PCI_BRIDGE_RESOURCES];
 	b_res[1].flags |= IORESOURCE_MEM;
+	b_res[1].flags &= ~IORESOURCE_DISABLED;
 
 	pci_read_config_word(bridge, PCI_IO_BASE, &io);
 	if (!io) {
@@ -1035,8 +1031,10 @@ static void pci_bridge_check_ranges(struct pci_bus *bus)
 		pci_read_config_word(bridge, PCI_IO_BASE, &io);
 		pci_write_config_word(bridge, PCI_IO_BASE, 0x0);
 	}
-	if (io)
+	if (io) {
 		b_res[0].flags |= IORESOURCE_IO;
+		b_res[0].flags &= ~IORESOURCE_DISABLED;
+	}
 
 	/*  DECchip 21050 pass 2 errata: the bridge may miss an address
 	    disconnect boundary by one PCI data phase.
@@ -1053,6 +1051,7 @@ static void pci_bridge_check_ranges(struct pci_bus *bus)
 	}
 	if (pmem) {
 		b_res[2].flags |= IORESOURCE_MEM | IORESOURCE_PREFETCH;
+		b_res[2].flags &= ~IORESOURCE_DISABLED;
 		if ((pmem & PCI_PREF_RANGE_TYPE_MASK) ==
 		    PCI_PREF_RANGE_TYPE_64) {
 			b_res[2].flags |= IORESOURCE_MEM_64;
@@ -1198,8 +1197,10 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 			struct resource *r = &dev->resource[i];
 			unsigned long r_size, r_add_size;
 
-			if (r->parent || !(r->flags & IORESOURCE_IO))
+			if (r->parent || !(r->flags & IORESOURCE_IO) ||
+			    resource_disabled(r))
 				continue;
+
 			r_size = resource_size(r);
 
 			if (r_size < 0x400)
@@ -1240,7 +1241,7 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 		if (b_res->start || b_res->end)
 			dev_info(&bus->self->dev, "disabling bridge window %pR to %pR (unused)\n",
 				 b_res, &bus->busn_res);
-		b_res->flags = 0;
+		b_res->flags |= IORESOURCE_UNSET | IORESOURCE_DISABLED;
 		return;
 	}
 
@@ -1532,7 +1533,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 			if (r->parent || ((flags & mask) != type &&
 					  (flags & mask) != type2 &&
-					  (flags & mask) != type3))
+					  (flags & mask) != type3) ||
+			    resource_disabled(r))
 				continue;
 
 			r_size = resource_size(r);
@@ -1553,7 +1555,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 			if (align > (1ULL<<37)) { /*128 Gb*/
 				dev_warn(&dev->dev, "disabling BAR %d: %pR (bad alignment %#llx)\n",
 					i, r, (unsigned long long) align);
-				r->flags = 0;
+				r->flags |= IORESOURCE_UNSET |
+					    IORESOURCE_DISABLED;
 				continue;
 			}
 
@@ -1650,7 +1653,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		if (b_res->start || b_res->end)
 			dev_info(&bus->self->dev, "disabling bridge window %pR to %pR (unused)\n",
 				 b_res, &bus->busn_res);
-		b_res->flags = 0;
+		b_res->flags |= IORESOURCE_UNSET | IORESOURCE_DISABLED;
 		return 0;
 	}
 
@@ -2067,7 +2070,7 @@ static void pci_bridge_release_resources(struct pci_bus *bus,
 		/* keep the old size */
 		r->end = resource_size(r) - 1;
 		r->start = 0;
-		r->flags = 0;
+		r->flags |= IORESOURCE_UNSET | IORESOURCE_DISABLED;
 
 		/* avoiding touch the one without PREF */
 		if (type & IORESOURCE_PREFETCH)
@@ -2336,7 +2339,6 @@ again:
 
 		restore_resource(fail_res, res);
 		if (fail_res->dev->subordinate) {
-			res->flags = 0;
 			/* last or third times and later */
 			if (tried_times + 1 == pci_try_num ||
 			    tried_times + 1 > 2)
@@ -2420,7 +2422,6 @@ again:
 
 		restore_resource(fail_res, res);
 		if (fail_res->dev->subordinate) {
-			res->flags = 0;
 			/* last time */
 			reset_bridge_resource_size(fail_res->dev, res);
 		}
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 55caf7a..d17f23c 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -364,7 +364,8 @@ int pci_enable_resources(struct pci_dev *dev, int mask)
 
 		r = &dev->resource[i];
 
-		if (!(r->flags & (IORESOURCE_IO | IORESOURCE_MEM)))
+		if (!(r->flags & (IORESOURCE_IO | IORESOURCE_MEM)) ||
+		    resource_disabled(r))
 			continue;
 		if ((i == PCI_ROM_RESOURCE) &&
 				(!(r->flags & IORESOURCE_ROM_ENABLE)))
-- 
1.8.4.5

[PATCH v8 61/61] PCI: Only try to assign io port only for root bus that support it

[Yinghai Lu]

The PCI subsystem always assumes that I/O is supported on root bus and
tries to assign an I/O window to each child bus even if that is not the
case.

The use cases is on Intel 8 socket system that have 8 root buses,
last two root buses would not have io port resources from _CRS.

Check if root bus supports I/O, and later during sizing and
assigning, check that flags and skip those resources.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/probe.c     | 6 ++++++
 drivers/pci/setup-bus.c | 9 +++++++++
 include/linux/pci.h     | 1 +
 3 files changed, 16 insertions(+)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index f9589d9..d5fca94 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -339,6 +339,9 @@ static void pci_read_bridge_io(struct pci_bus *child)
 	struct pci_bus_region region;
 	struct resource *res;
 
+	if (!pci_find_host_bridge(child)->has_ioport)
+		return;
+
 	io_mask = PCI_IO_RANGE_MASK;
 	io_granularity = 0x1000;
 	if (dev->io_window_1k) {
@@ -2126,6 +2129,9 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
 			bus_addr[0] = '\0';
 		dev_info(&b->dev, "root bus resource %pR%s\n", res, bus_addr);
 
+		if (resource_type(res) == IORESOURCE_IO)
+			bridge->has_ioport = 1;
+
 		if (resource_type(res) == IORESOURCE_MEM) {
 			if ((res->end - offset) > 0xffffffff)
 				bridge->has_mem64 = 1;
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 22ee1a1..f4918d7 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -226,6 +226,10 @@ static void pdev_assign_resources_prepare(struct pci_dev *dev,
 		if (resource_disabled(r) || r->parent)
 			continue;
 
+		if ((r->flags & IORESOURCE_IO) &&
+		    !pci_find_host_bridge(dev->bus)->has_ioport)
+			continue;
+
 		r_align = __pci_resource_alignment(dev, r, realloc_head);
 		if (!r_align) {
 			dev_warn(&dev->dev, "BAR %d: %pR has bogus alignment\n",
@@ -1189,6 +1193,11 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 		min_size = 0;
 	}
 
+	if (!pci_find_host_bridge(bus)->has_ioport) {
+		b_res->flags |= IORESOURCE_UNSET | IORESOURCE_DISABLED;
+		return;
+	}
+
 	min_align = window_alignment(bus, IORESOURCE_IO);
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		int i;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 6bc56f1..9ffe03c 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -414,6 +414,7 @@ struct pci_host_bridge {
 	void *release_data;
 	unsigned int ignore_reset_delay:1;	/* for entire hierarchy */
 	unsigned int has_mem64:1;
+	unsigned int has_ioport:1;
 };
 
 #define	to_pci_host_bridge(n) container_of(n, struct pci_host_bridge, dev)
-- 
1.8.4.5

Re: [PATCH v8 00/61] PCI: Resource allocation cleanup for v4.4

[Khalid Aziz]

[-- Attachment #1: Type: text/plain, Size: 2166 bytes --]

On 10/27/2015 02:54 PM, Yinghai Lu wrote:
> Hi,
>
> After 5b28541552ef (PCI: Restrict 64-bit prefetchable bridge windows
> to 64-bit resources), we have several reports on resource allocation
> failure, and we try to fix the problem with resource clip, and find
> more problems.
>
> One is realloc fail with two graphics cards above 4G.
> One is from sparc that have problem with clip as we don't parse
> mem64 for it.
>
> Other report is about pci remove/rescan does not work on some setup
> when BIOS tend to allocate small bus size.
>
> This patchset enhance resource allocation to address those problems.
>
> patch 1-11: parse MEM64 for sparc and other system with OF
> patch 12-16: MMIO64 allocation enhancement
> 	treat non-pref mmio64 if parent bridges are all pcie.
> 	restore old pref allocation logic if hostbridge does not support mmio64.
> patch 17-19: FIXED resource handling during realloc
> 	don't realloc resource if device firmware does not support bar change.
> patch 20-23: bridge MMIO allocation with hotplug and last try.
> 	treat optional as required on first try when hotplug.
> 	MMIO size set to 0 for last try during realloc
> patch 24-57: enhancement for mmio resource allocation:
> 	optimize bus mmio alignment calculation.
> 	optimize bus mmio optional alignment calculation.
> 	add support for alt size to prefer small bus size to small bus alignment.
> 	treat ROM bar as optional resource.
> 	during allocation, will pick up best fit resource, and allocate near end.
> patch 58: add pci=assign_pref_bars to clear and assign pref bars.
> patch 59-60: don't clear resource when allocation fails
> patch 61: don't try io port allocation if root bus does not have io port.
>

Hi Yinghai,

I applied patches 1-8 and 11-16 to 4.3.0-rc7 to test on sparc platforms. 
I am seeing a "can't claim BAR" message on a T4:

pci 0000:04:00.0: can't claim BAR 6 [mem 0x84000000000-0x8400000ffff]: 
address conflict with 0000:04:00.0 [mem 0x84000000000-0x840007fffff]

I have attached boot up log, /proc/iomem, and output from "lspci -tv" 
and "lsspci -vvxxx".

These patches worked fine on a T7 and there were no PCI issues in the log.

Thanks,
Khalid


[-- Attachment #2: dmesg.4.3.0-rc7-pci.t4 --]
[-- Type: text/plain, Size: 63521 bytes --]

PROMLIB: Sun IEEE Boot Prom 'OBP 4.38.1 2015/08/21 14:24'
PROMLIB: Root node compatible: sun4v
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Initializing cgroup subsys cpuacct
Linux version 4.3.0-rc7-kh+ (khalid@ca-qasparc20.us.oracle.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4.0.13) (GCC) ) #24 SMP Fri Oct 30 15:41:41 EDT 2015
debug: ignoring loglevel setting.
bootconsole [earlyprom0] enabled
ARCH: SUN4V
Ethernet address: 00:21:28:f9:f6:4e
MM: PAGE_OFFSET is 0xfff8000000000000 (max_phys_bits == 47)
MM: VMALLOC [0x0000000100000000 --> 0x0006000000000000]
MM: VMEMMAP [0x0006000000000000 --> 0x000c000000000000]
Kernel: Using 3 locked TLB entries for main kernel image.
Remapping the kernel... done.
OF stdout device is: /virtual-devices@100/console@1
PROM: Built device tree with 371493 bytes of memory.
MDESC: Size is 107936 bytes.
PLATFORM: banner-name [SPARC T4-2]
PLATFORM: name [ORCL,SPARC-T4-2]
PLATFORM: hostid [85f9f64e]
PLATFORM: serial# [00f9f64e]
PLATFORM: stick-frequency [3b8f5850]
PLATFORM: mac-address [2128f9f64e]
PLATFORM: watchdog-resolution [1000 ms]
PLATFORM: watchdog-max-timeout [31536000000 ms]
PLATFORM: max-cpus [128]
Allocated 16384 bytes for kernel page tables.
Zone ranges:
  DMA      [mem 0x0000000020400000-0xffffffffffffffff]
  Normal   [mem 0x0000000000000000-0x0000001ffff3ffff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000020400000-0x000000005fdb7fff]
  node   0: [mem 0x000000005fdba000-0x000000005fdbbfff]
  node   0: [mem 0x0000000060000000-0x00000001ffffffff]
  node   1: [mem 0x0000000200000000-0x00000003ffffffff]
  node   0: [mem 0x0000000400000000-0x00000005ffffffff]
  node   1: [mem 0x0000000600000000-0x00000007ffffffff]
  node   0: [mem 0x0000000800000000-0x00000009ffffffff]
  node   1: [mem 0x0000000a00000000-0x0000000bffffffff]
  node   0: [mem 0x0000000c00000000-0x0000000dffffffff]
  node   1: [mem 0x0000000e00000000-0x0000000fffffffff]
  node   0: [mem 0x0000001000000000-0x00000011ffffffff]
  node   1: [mem 0x0000001200000000-0x00000013ffffffff]
  node   0: [mem 0x0000001400000000-0x00000015ffffffff]
  node   1: [mem 0x0000001600000000-0x00000017ffffffff]
  node   0: [mem 0x0000001800000000-0x00000019ffffffff]
  node   1: [mem 0x0000001a00000000-0x0000001bffffffff]
  node   0: [mem 0x0000001c00000000-0x0000001dffffffff]
  node   1: [mem 0x0000001e00000000-0x0000001fffe8dfff]
  node   1: [mem 0x0000001fffe9e000-0x0000001fffeb1fff]
  node   1: [mem 0x0000001ffff30000-0x0000001ffff3ffff]
Initmem setup node 0 [mem 0x0000000020400000-0x0000001dffffffff]
On node 0 totalpages: 8322269
  Normal zone: 73145 pages used for memmap
  Normal zone: 8322269 pages, LIFO batch:15
Initmem setup node 1 [mem 0x0000000200000000-0x0000001ffff3ffff]
On node 1 totalpages: 8388441
  Normal zone: 73727 pages used for memmap
  Normal zone: 8388441 pages, LIFO batch:15
Booting Linux...
CPU CAPS: [flush,stbar,swap,muldiv,v9,blkinit,n2,mul32]
CPU CAPS: [div32,v8plus,popc,vis,vis2,ASIBlkInit,fmaf,vis3]
CPU CAPS: [hpc,ima,pause,cbcond,aes,des,kasumi,camellia]
CPU CAPS: [md5,sha1,sha256,sha512,mpmul,montmul,montsqr,crc32c]
PERCPU: Embedded 9 pages/cpu @fff8001ddb000000 s32384 r8192 d33152 u131072
pcpu-alloc: s32384 r8192 d33152 u131072 alloc=1*4194304
pcpu-alloc: [0] 000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 
pcpu-alloc: [0] 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 
pcpu-alloc: [1] 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 
pcpu-alloc: [1] 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 
SUN4V: Mondo queue sizes [cpu(16384) dev(16384) r(8192) nr(256)]
Built 2 zonelists in Node order, mobility grouping on.  Total pages: 16563838
Policy zone: Normal
Kernel command line: root=/dev/mapper/VolGroup-lv_root ro rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD rd_LVM_LV=VolGroup/lv_swap SYSFONT=latarcyrheb-sun16 rd_LVM_LV=VolGroup/lv_root KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM debug ignore_loglevel ofpci_debug=1
PID hash table entries: 4096 (order: 2, 32768 bytes)
Sorting __ex_table...
Memory: 132447856K/133685680K available (5932K kernel code, 1014K rwdata, 2472K rodata, 544K init, 2004K bss, 1237824K reserved, 0K cma-reserved)
Hierarchical RCU implementation.
	Build-time adjustment of leaf fanout to 64.
	RCU restricting CPUs from NR_CPUS=1024 to nr_cpu_ids=128.
RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=128
NR_IRQS:2048 nr_irqs:2048 1
SUN4V: Using IRQ API major 3, cookie only virqs enabled
clocksource: stick: mask: 0xffffffffffffffff max_cycles: 0x1ccea542bf9, max_idle_ns: 881590497797 ns
clocksource: mult[801898] shift[23]
clockevent: mult[ffced917] shift[32]
Console: colour dummy device 80x25
console [tty0] enabled
bootconsole [earlyprom0] disabled
Calibrating delay using timer specific routine.. 2000.39 BogoMIPS (lpj=4000782)
pid_max: default: 131072 minimum: 1024
Security Framework initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
Dentry cache hash table entries: 16777216 (order: 14, 134217728 bytes)
Inode-cache hash table entries: 8388608 (order: 13, 67108864 bytes)
Mount-cache hash table entries: 262144 (order: 8, 2097152 bytes)
Mountpoint-cache hash table entries: 262144 (order: 8, 2097152 bytes)
Initializing cgroup subsys io
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys perf_event
Initializing cgroup subsys hugetlb
ftrace: allocating 20976 entries in 41 pages
Brought up 128 CPUs
devtmpfs: initialized
Performance events: 
Testing NMI watchdog ... OK.
Supported PMU type is 'niagara4'
ldc.c:v1.1 (July 22, 2008)
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
NET: Registered protocol family 16
VIO: Adding device channel-devices
VIO: Adding device vldc-port-0-0
VIO: Adding device vldc-port-0-1
VIO: Adding device vldc-port-0-2
VIO: Adding device vldc-port-1-0
VIO: Adding device vldc-port-2-0
VIO: Adding device vldc-port-3-0
VIO: Adding device vldc-port-3-1
VIO: Adding device vldc-port-3-2
VIO: Adding device vldc-port-3-3
VIO: Adding device vldc-port-3-4
VIO: Adding device vldc-port-3-6
VIO: Adding device vlds-port-0-0
VIO: Adding device ds-0
VIO: Adding device ds-1
pci_sun4v: Registered hvapi major[1] minor[0]
/pci@400: SUN4V PCI Bus Module
/pci@400: On NUMA node 0
/pci@400: PCI IO [io  0x85100000000-0x8510fffffff] offset 85100000000
/pci@400: PCI MEM [mem 0x84000000000-0x8407f7fffff] offset 84000000000
/pci@400: PCI MEM64 [mem 0x84100000000-0x847ffffffff] offset 80000000000
/pci@400: Unable to request IOMMU resource.
/pci@400: Imported 3 TSB entries from OBP
/pci@400: MSI Queue first[0] num[56] count[512] devino[0x6]
/pci@400: MSI first[0] num[512] mask[0x3ff] width[32]
/pci@400: MSI addr32[0x7f800000:0x800000] addr64[0xe00000003f800000:0x800000]
/pci@400: MSI queues at RA [0000001dd2200000]
PCI: Scanning PBM /pci@400
pci_sun4v f02a5770: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x85100000000-0x8510fffffff] (bus address [0x0000-0xfffffff])
pci_bus 0000:00: root bus resource [mem 0x84000000000-0x8407f7fffff] (bus address [0x00000000-0x7f7fffff])
pci_bus 0000:00: root bus resource [mem 0x84100000000-0x847ffffffff] (bus address [0x4100000000-0x47ffffffff])
pci_bus 0000:00: root bus resource [bus 00-0f]
PCI: scan_bus[/pci@400] bus no 0
  * /pci@400/pci@1
    create device, devfn: 8, type: pciex
    class: 0x60400 device name: 0000:00:01.0
    adding to system ...
pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@1)
    Bridge bus range [1 --> 9]
    Bridge ranges[fff8001fffe39940] simba[0]
    RAW Range[81000000:00000000:00000000:81000000:00000000:00000000:00000000:00002000]
      Using flags[00000101] start[0000000000000000] size[0000000000002000]
    RAW Range[82000000:00000000:00000000:82000000:00000000:00000000:00000000:00c00000]
      Using flags[00000200] start[0000000000000000] size[0000000000c00000]
    bus name: PCI Bus 0000:01
PCI: scan_bus[/pci@400/pci@1] bus no 1
  * /pci@400/pci@1/pci@0
    create device, devfn: 0, type: pciex
    class: 0x60400 device name: 0000:01:00.0
    adding to system ...
pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@1/pci@0)
    Bridge bus range [2 --> 9]
    Bridge ranges[fff8001fffe38800] simba[0]
    RAW Range[81000000:00000000:00000000:81000000:00000000:00000000:00000000:00002000]
      Using flags[00000101] start[0000000000000000] size[0000000000002000]
    RAW Range[82000000:00000000:00000000:82000000:00000000:00000000:00000000:00c00000]
      Using flags[00000200] start[0000000000000000] size[0000000000c00000]
    bus name: PCI Bus 0000:02
PCI: scan_bus[/pci@400/pci@1/pci@0] bus no 2
  * /pci@400/pci@1/pci@0/pci@0
    create device, devfn: 0, type: pciex
    class: 0x60400 device name: 0000:02:00.0
    adding to system ...
pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@1/pci@0/pci@0)
    Bridge bus range [3 --> 4]
    Bridge ranges[fff8001fffe376c0] simba[0]
    RAW Range[81000000:00000000:00000000:81000000:00000000:00000000:00000000:00001000]
      Using flags[00000101] start[0000000000000000] size[0000000000001000]
    RAW Range[82000000:00000000:00000000:82000000:00000000:00000000:00000000:00900000]
      Using flags[00000200] start[0000000000000000] size[0000000000900000]
    bus name: PCI Bus 0000:03
PCI: scan_bus[/pci@400/pci@1/pci@0/pci@0] bus no 3
  * /pci@400/pci@1/pci@0/pci@0/pci@0
    create device, devfn: 0, type: pciex
    class: 0x60400 device name: 0000:03:00.0
    adding to system ...
pci 0000:03:00.0: supports D1 D2
pci 0000:03:00.0: PME# supported from D0 D1 D2 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@1/pci@0/pci@0/pci@0)
    Bridge bus range [4 --> 4]
    Bridge ranges[fff8001fffe36380] simba[0]
    RAW Range[81000000:00000000:00000000:81000000:00000000:00000000:00000000:00001000]
      Using flags[00000101] start[0000000000000000] size[0000000000001000]
    RAW Range[82000000:00000000:00000000:82000000:00000000:00000000:00000000:00900000]
      Using flags[00000200] start[0000000000000000] size[0000000000900000]
    bus name: PCI Bus 0000:04
PCI: scan_bus[/pci@400/pci@1/pci@0/pci@0/pci@0] bus no 4
  * /pci@400/pci@1/pci@0/pci@0/pci@0/display@0
    create device, devfn: 0, type: display
    class: 0x30000 device name: 0000:04:00.0
    parse addresses (80 bytes) @ fff8001fffe34f40
  start: 84000000000, end: 840007fffff, i: 10
  start: 84000800000, end: 8400081ffff, i: 14
  start: 85100000000, end: 8510000007f, i: 18
  start: 84000000000, end: 8400000ffff, i: 30
    adding to system ...
pci 0000:04:00.0: supports D1 D2
pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
PCI: dev header type: 0
  * /pci@400/pci@1/pci@0/pci@4
    create device, devfn: 20, type: pciex
    class: 0x60400 device name: 0000:02:04.0
    adding to system ...
pci 0000:02:04.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@1/pci@0/pci@4)
    Bridge bus range [5 --> 5]
    Bridge ranges[fff8001fffe339c0] simba[0]
    RAW Range[81000000:00000000:00001000:81000000:00000000:00001000:00000000:00001000]
      Using flags[00000101] start[0000000000001000] size[0000000000001000]
    RAW Range[82000000:00000000:00900000:82000000:00000000:00900000:00000000:00200000]
      Using flags[00000200] start[0000000000900000] size[0000000000200000]
    bus name: PCI Bus 0000:05
PCI: scan_bus[/pci@400/pci@1/pci@0/pci@4] bus no 5
  * /pci@400/pci@1/pci@0/pci@4/network@0
    create device, devfn: 0, type: network
    class: 0x20000 device name: 0000:05:00.0
    parse addresses (100 bytes) @ fff8001fffe324c0
  start: 84000900000, end: 8400091ffff, i: 10
  start: 84000920000, end: 8400093ffff, i: 14
  start: 85100001000, end: 8510000101f, i: 18
  start: 84000940000, end: 84000943fff, i: 1c
  start: 84000960000, end: 8400097ffff, i: 30
    adding to system ...
pci 0000:05:00.0: PME# supported from D0 D3hot D3cold
pci 0000:05:00.0: reg 0x184: [mem 0x84000980000-0x84000983fff]
pci 0000:05:00.0: VF(n) BAR0 space: [mem 0x84000980000-0x8400099ffff] (contains BAR0 for 8 VFs)
pci 0000:05:00.0: reg 0x190: [mem 0x840009a0000-0x840009a3fff]
pci 0000:05:00.0: VF(n) BAR3 space: [mem 0x840009a0000-0x840009bffff] (contains BAR3 for 8 VFs)
PCI: dev header type: 0
  * /pci@400/pci@1/pci@0/pci@4/network@0,1
    create device, devfn: 1, type: network
    class: 0x20000 device name: 0000:05:00.1
    parse addresses (100 bytes) @ fff8001fffe30840
  start: 840009c0000, end: 840009dffff, i: 10
  start: 840009e0000, end: 840009fffff, i: 14
  start: 85100001020, end: 8510000103f, i: 18
  start: 84000944000, end: 84000947fff, i: 1c
  start: 84000a00000, end: 84000a1ffff, i: 30
    adding to system ...
pci 0000:05:00.1: PME# supported from D0 D3hot D3cold
pci 0000:05:00.1: reg 0x184: [mem 0x84000a20000-0x84000a23fff]
pci 0000:05:00.1: VF(n) BAR0 space: [mem 0x84000a20000-0x84000a3ffff] (contains BAR0 for 8 VFs)
pci 0000:05:00.1: reg 0x190: [mem 0x84000a40000-0x84000a43fff]
pci 0000:05:00.1: VF(n) BAR3 space: [mem 0x84000a40000-0x84000a5ffff] (contains BAR3 for 8 VFs)
PCI: dev header type: 0
  * /pci@400/pci@1/pci@0/pci@8
    create device, devfn: 40, type: pciex
    class: 0x60400 device name: 0000:02:08.0
    adding to system ...
pci 0000:02:08.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@1/pci@0/pci@8)
    Bridge bus range [6 --> 6]
    Bridge ranges[          (null)] simba[0]
pci 0000:02:08.0: PCI bridge to [bus 06]
    bus name: PCI Bus 0000:06
PCI: scan_bus[/pci@400/pci@1/pci@0/pci@8] bus no 6
  * /pci@400/pci@1/pci@0/pci@b
    create device, devfn: 58, type: pciex
    class: 0x60400 device name: 0000:02:0b.0
    adding to system ...
pci 0000:02:0b.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@1/pci@0/pci@b)
    Bridge bus range [7 --> 8]
    Bridge ranges[fff8001fffe2dd00] simba[0]
    RAW Range[82000000:00000000:00b00000:82000000:00000000:00b00000:00000000:00100000]
      Using flags[00000200] start[0000000000b00000] size[0000000000100000]
    bus name: PCI Bus 0000:07
PCI: scan_bus[/pci@400/pci@1/pci@0/pci@b] bus no 7
  * /pci@400/pci@1/pci@0/pci@b/pci@0
    create device, devfn: 0, type: pciex
    class: 0x60400 device name: 0000:07:00.0
    adding to system ...
pci 0000:07:00.0: supports D1
pci 0000:07:00.0: PME# supported from D0 D1 D3hot
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@1/pci@0/pci@b/pci@0)
    Bridge bus range [8 --> 8]
    Bridge ranges[fff8001fffe2ca00] simba[0]
    RAW Range[82000000:00000000:00b00000:82000000:00000000:00b00000:00000000:00100000]
      Using flags[00000200] start[0000000000b00000] size[0000000000100000]
    bus name: PCI Bus 0000:08
PCI: scan_bus[/pci@400/pci@1/pci@0/pci@b/pci@0] bus no 8
  * /pci@400/pci@1/pci@0/pci@b/pci@0/usb@0
    create device, devfn: 0, type: ohci
    class: 0xc0310 device name: 0000:08:00.0
    parse addresses (20 bytes) @ fff8001fffe2b800
  start: 84000b00000, end: 84000b01fff, i: 10
    adding to system ...
pci 0000:08:00.0: supports D1 D2
pci 0000:08:00.0: PME# supported from D0 D1 D2 D3hot
PCI: dev header type: 0
  * /pci@400/pci@1/pci@0/pci@b/pci@0/usb@0,1
    create device, devfn: 1, type: ohci
    class: 0xc0310 device name: 0000:08:00.1
    parse addresses (20 bytes) @ fff8001fffe2a200
  start: 84000b02000, end: 84000b03fff, i: 10
    adding to system ...
pci 0000:08:00.1: supports D1 D2
pci 0000:08:00.1: PME# supported from D0 D1 D2 D3hot
PCI: dev header type: 0
  * /pci@400/pci@1/pci@0/pci@b/pci@0/usb@0,2
    create device, devfn: 2, type: ehci
    class: 0xc0320 device name: 0000:08:00.2
    parse addresses (20 bytes) @ fff8001fffe28c00
  start: 84000b04000, end: 84000b05fff, i: 10
    adding to system ...
pci 0000:08:00.2: supports D1 D2
pci 0000:08:00.2: PME# supported from D0 D1 D2 D3hot
PCI: dev header type: 0
  * /pci@400/pci@1/pci@0/pci@c
    create device, devfn: 60, type: pciex
    class: 0x60400 device name: 0000:02:0c.0
    adding to system ...
pci 0000:02:0c.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@1/pci@0/pci@c)
    Bridge bus range [9 --> 9]
    Bridge ranges[          (null)] simba[0]
pci 0000:02:0c.0: PCI bridge to [bus 09]
    bus name: PCI Bus 0000:09
PCI: scan_bus[/pci@400/pci@1/pci@0/pci@c] bus no 9
  * /pci@400/pci@2
    create device, devfn: 10, type: pciex
    class: 0x60400 device name: 0000:00:02.0
    adding to system ...
pci 0000:00:02.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@2)
    Bridge bus range [10 --> 15]
    Bridge ranges[fff8001fffe1aa40] simba[0]
    RAW Range[81000000:00000000:00002000:81000000:00000000:00002000:00000000:00002000]
      Using flags[00000101] start[0000000000002000] size[0000000000002000]
    RAW Range[82000000:00000000:00c00000:82000000:00000000:00c00000:00000000:00200000]
      Using flags[00000200] start[0000000000c00000] size[0000000000200000]
    RAW Range[c3000000:00000041:00000000:c3000000:00000041:00000000:00000000:20000000]
      Using flags[0010220c] start[0000004100000000] size[0000000020000000]
    bus name: PCI Bus 0000:0a
PCI: scan_bus[/pci@400/pci@2] bus no 10
  * /pci@400/pci@2/pci@0
    create device, devfn: 0, type: pciex
    class: 0x60400 device name: 0000:0a:00.0
    adding to system ...
pci 0000:0a:00.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@2/pci@0)
    Bridge bus range [11 --> 15]
    Bridge ranges[fff8001fffe19900] simba[0]
    RAW Range[81000000:00000000:00002000:81000000:00000000:00002000:00000000:00002000]
      Using flags[00000101] start[0000000000002000] size[0000000000002000]
    RAW Range[82000000:00000000:00c00000:82000000:00000000:00c00000:00000000:00200000]
      Using flags[00000200] start[0000000000c00000] size[0000000000200000]
    RAW Range[c3000000:00000041:00000000:c3000000:00000041:00000000:00000000:20000000]
      Using flags[0010220c] start[0000004100000000] size[0000000020000000]
    bus name: PCI Bus 0000:0b
PCI: scan_bus[/pci@400/pci@2/pci@0] bus no 11
  * /pci@400/pci@2/pci@0/pci@0
    create device, devfn: 0, type: pciex
    class: 0x60400 device name: 0000:0b:00.0
    adding to system ...
pci 0000:0b:00.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@2/pci@0/pci@0)
    Bridge bus range [12 --> 12]
    Bridge ranges[          (null)] simba[0]
pci 0000:0b:00.0: PCI bridge to [bus 0c]
    bus name: PCI Bus 0000:0c
PCI: scan_bus[/pci@400/pci@2/pci@0/pci@0] bus no 12
  * /pci@400/pci@2/pci@0/pci@4
    create device, devfn: 20, type: pciex
    class: 0x60400 device name: 0000:0b:04.0
    adding to system ...
pci 0000:0b:04.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@2/pci@0/pci@4)
    Bridge bus range [13 --> 13]
    Bridge ranges[          (null)] simba[0]
pci 0000:0b:04.0: PCI bridge to [bus 0d]
    bus name: PCI Bus 0000:0d
PCI: scan_bus[/pci@400/pci@2/pci@0/pci@4] bus no 13
  * /pci@400/pci@2/pci@0/pci@8
    create device, devfn: 40, type: pciex
    class: 0x60400 device name: 0000:0b:08.0
    adding to system ...
pci 0000:0b:08.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@2/pci@0/pci@8)
    Bridge bus range [14 --> 14]
    Bridge ranges[fff8001fffe161c0] simba[0]
    RAW Range[81000000:00000000:00002000:81000000:00000000:00002000:00000000:00001000]
      Using flags[00000101] start[0000000000002000] size[0000000000001000]
    RAW Range[82000000:00000000:00c00000:82000000:00000000:00c00000:00000000:00100000]
      Using flags[00000200] start[0000000000c00000] size[0000000000100000]
    RAW Range[c3000000:00000041:00000000:c3000000:00000041:00000000:00000000:10000000]
      Using flags[0010220c] start[0000004100000000] size[0000000010000000]
    bus name: PCI Bus 0000:0e
PCI: scan_bus[/pci@400/pci@2/pci@0/pci@8] bus no 14
  * /pci@400/pci@2/pci@0/pci@8/SUNW,emlxs@0
    create device, devfn: 0, type: scsi-fcp
    class: 0xc0400 device name: 0000:0e:00.0
    parse addresses (80 bytes) @ fff8001fffe14f80
  start: 84100000000, end: 84100001fff, i: 10
  start: 84100004000, end: 84100007fff, i: 18
  start: 85100002000, end: 851000020ff, i: 20
  start: 84000c00000, end: 84000c3ffff, i: 30
    adding to system ...
PCI: dev header type: 0
  * /pci@400/pci@2/pci@0/pci@8/SUNW,emlxs@0,1
    create device, devfn: 1, type: scsi-fcp
    class: 0xc0400 device name: 0000:0e:00.1
    parse addresses (80 bytes) @ fff8001fffe61280
  start: 84100002000, end: 84100003fff, i: 10
  start: 84100008000, end: 8410000bfff, i: 18
  start: 85100002100, end: 851000021ff, i: 20
  start: 84000c40000, end: 84000c7ffff, i: 30
    adding to system ...
PCI: dev header type: 0
  * /pci@400/pci@2/pci@0/pci@e
    create device, devfn: 70, type: pciex
    class: 0x60400 device name: 0000:0b:0e.0
    adding to system ...
pci 0000:0b:0e.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@400/pci@2/pci@0/pci@e)
    Bridge bus range [15 --> 15]
    Bridge ranges[fff8001fffe5e140] simba[0]
    RAW Range[81000000:00000000:00003000:81000000:00000000:00003000:00000000:00001000]
      Using flags[00000101] start[0000000000003000] size[0000000000001000]
    RAW Range[82000000:00000000:00d00000:82000000:00000000:00d00000:00000000:00100000]
      Using flags[00000200] start[0000000000d00000] size[0000000000100000]
    RAW Range[c3000000:00000041:10000000:c3000000:00000041:10000000:00000000:10000000]
      Using flags[0010220c] start[0000004110000000] size[0000000010000000]
    bus name: PCI Bus 0000:0f
PCI: scan_bus[/pci@400/pci@2/pci@0/pci@e] bus no 15
  * /pci@400/pci@2/pci@0/pci@e/scsi@0
    create device, devfn: 0, type: scsi-sas
    class: 0x10700 device name: 0000:0f:00.0
    parse addresses (80 bytes) @ fff8001fffe5cb40
  start: 85100003000, end: 851000030ff, i: 10
  start: 84110000000, end: 84110003fff, i: 14
  start: 84110040000, end: 8411007ffff, i: 1c
  start: 84000d00000, end: 84000d7ffff, i: 30
    adding to system ...
pci 0000:0f:00.0: supports D1 D2
pci 0000:0f:00.0: reg 0x174: [mem 0x84110080000-0x84110083fff 64bit]
pci 0000:0f:00.0: VF(n) BAR0 space: [mem 0x84110080000-0x841100bffff 64bit] (contains BAR0 for 16 VFs)
pci 0000:0f:00.0: reg 0x17c: [mem 0x841100c0000-0x841100fffff 64bit]
pci 0000:0f:00.0: VF(n) BAR2 space: [mem 0x841100c0000-0x841104bffff 64bit] (contains BAR2 for 16 VFs)
PCI: dev header type: 0
PCI: Claiming 0000:00:01.0: Resource 13: 0000085100000000..0000085100001fff [101]
PCI: Claiming 0000:00:01.0: Resource 14: 0000084000000000..0000084000bfffff [200]
PCI: Claiming 0000:00:02.0: Resource 13: 0000085100002000..0000085100003fff [101]
PCI: Claiming 0000:00:02.0: Resource 14: 0000084000c00000..0000084000dfffff [200]
PCI: Claiming 0000:00:02.0: Resource 15: 0000084100000000..000008411fffffff [10220c]
PCI: Claiming 0000:01:00.0: Resource 13: 0000085100000000..0000085100001fff [101]
PCI: Claiming 0000:01:00.0: Resource 14: 0000084000000000..0000084000bfffff [200]
PCI: Claiming 0000:02:00.0: Resource 13: 0000085100000000..0000085100000fff [101]
PCI: Claiming 0000:02:00.0: Resource 14: 0000084000000000..00000840008fffff [200]
PCI: Claiming 0000:02:04.0: Resource 13: 0000085100001000..0000085100001fff [101]
PCI: Claiming 0000:02:04.0: Resource 14: 0000084000900000..0000084000afffff [200]
PCI: Claiming 0000:02:0b.0: Resource 14: 0000084000b00000..0000084000bfffff [200]
PCI: Claiming 0000:03:00.0: Resource 13: 0000085100000000..0000085100000fff [101]
PCI: Claiming 0000:03:00.0: Resource 14: 0000084000000000..00000840008fffff [200]
PCI: Claiming 0000:04:00.0: Resource 0: 0000084000000000..00000840007fffff [200]
PCI: Claiming 0000:04:00.0: Resource 1: 0000084000800000..000008400081ffff [200]
PCI: Claiming 0000:04:00.0: Resource 2: 0000085100000000..000008510000007f [101]
PCI: Claiming 0000:04:00.0: Resource 6: 0000084000000000..000008400000ffff [44200]
pci 0000:04:00.0: can't claim BAR 6 [mem 0x84000000000-0x8400000ffff]: address conflict with 0000:04:00.0 [mem 0x84000000000-0x840007fffff]
PCI: Claiming 0000:05:00.0: Resource 0: 0000084000900000..000008400091ffff [200]
PCI: Claiming 0000:05:00.0: Resource 1: 0000084000920000..000008400093ffff [200]
PCI: Claiming 0000:05:00.0: Resource 2: 0000085100001000..000008510000101f [101]
PCI: Claiming 0000:05:00.0: Resource 3: 0000084000940000..0000084000943fff [200]
PCI: Claiming 0000:05:00.0: Resource 6: 0000084000960000..000008400097ffff [44200]
PCI: Claiming 0000:05:00.0: Resource 7: 0000084000980000..000008400099ffff [40200]
PCI: Claiming 0000:05:00.0: Resource 10: 00000840009a0000..00000840009bffff [40200]
PCI: Claiming 0000:05:00.1: Resource 0: 00000840009c0000..00000840009dffff [200]
PCI: Claiming 0000:05:00.1: Resource 1: 00000840009e0000..00000840009fffff [200]
PCI: Claiming 0000:05:00.1: Resource 2: 0000085100001020..000008510000103f [101]
PCI: Claiming 0000:05:00.1: Resource 3: 0000084000944000..0000084000947fff [200]
PCI: Claiming 0000:05:00.1: Resource 6: 0000084000a00000..0000084000a1ffff [44200]
PCI: Claiming 0000:05:00.1: Resource 7: 0000084000a20000..0000084000a3ffff [40200]
PCI: Claiming 0000:05:00.1: Resource 10: 0000084000a40000..0000084000a5ffff [40200]
PCI: Claiming 0000:07:00.0: Resource 14: 0000084000b00000..0000084000bfffff [200]
PCI: Claiming 0000:08:00.0: Resource 0: 0000084000b00000..0000084000b01fff [200]
PCI: Claiming 0000:08:00.1: Resource 0: 0000084000b02000..0000084000b03fff [200]
PCI: Claiming 0000:08:00.2: Resource 0: 0000084000b04000..0000084000b05fff [200]
PCI: Claiming 0000:0a:00.0: Resource 13: 0000085100002000..0000085100003fff [101]
PCI: Claiming 0000:0a:00.0: Resource 14: 0000084000c00000..0000084000dfffff [200]
PCI: Claiming 0000:0a:00.0: Resource 15: 0000084100000000..000008411fffffff [10220c]
PCI: Claiming 0000:0b:08.0: Resource 13: 0000085100002000..0000085100002fff [101]
PCI: Claiming 0000:0b:08.0: Resource 14: 0000084000c00000..0000084000cfffff [200]
PCI: Claiming 0000:0b:08.0: Resource 15: 0000084100000000..000008410fffffff [10220c]
PCI: Claiming 0000:0b:0e.0: Resource 13: 0000085100003000..0000085100003fff [101]
PCI: Claiming 0000:0b:0e.0: Resource 14: 0000084000d00000..0000084000dfffff [200]
PCI: Claiming 0000:0b:0e.0: Resource 15: 0000084110000000..000008411fffffff [10220c]
PCI: Claiming 0000:0e:00.0: Resource 0: 0000084100000000..0000084100001fff [100204]
PCI: Claiming 0000:0e:00.0: Resource 2: 0000084100004000..0000084100007fff [100204]
PCI: Claiming 0000:0e:00.0: Resource 4: 0000085100002000..00000851000020ff [101]
PCI: Claiming 0000:0e:00.0: Resource 6: 0000084000c00000..0000084000c3ffff [44200]
PCI: Claiming 0000:0e:00.1: Resource 0: 0000084100002000..0000084100003fff [100204]
PCI: Claiming 0000:0e:00.1: Resource 2: 0000084100008000..000008410000bfff [100204]
PCI: Claiming 0000:0e:00.1: Resource 4: 0000085100002100..00000851000021ff [101]
PCI: Claiming 0000:0e:00.1: Resource 6: 0000084000c40000..0000084000c7ffff [44200]
PCI: Claiming 0000:0f:00.0: Resource 0: 0000085100003000..00000851000030ff [101]
PCI: Claiming 0000:0f:00.0: Resource 1: 0000084110000000..0000084110003fff [100204]
PCI: Claiming 0000:0f:00.0: Resource 3: 0000084110040000..000008411007ffff [100204]
PCI: Claiming 0000:0f:00.0: Resource 6: 0000084000d00000..0000084000d7ffff [44200]
PCI: Claiming 0000:0f:00.0: Resource 7: 0000084110080000..00000841100bffff [140204]
PCI: Claiming 0000:0f:00.0: Resource 9: 00000841100c0000..00000841104bffff [140204]
pci_bus 0000:00:  can't claim Video RAM area [mem 0x840000a0000-0x840000bffff]: address conflict with PCI Bus 0000:01 [mem 0x84000000000-0x84000bfffff]
pci_bus 0000:00:  can't claim System ROM [mem 0x840000f0000-0x840000fffff]: address conflict with PCI Bus 0000:01 [mem 0x84000000000-0x84000bfffff]
pci_bus 0000:00:  can't claim Video ROM [mem 0x840000c0000-0x840000c7fff]: address conflict with PCI Bus 0000:01 [mem 0x84000000000-0x84000bfffff]
/pci@500: SUN4V PCI Bus Module
/pci@500: On NUMA node 1
/pci@500: PCI IO [io  0x8d100000000-0x8d10fffffff] offset 8d100000000
/pci@500: PCI MEM [mem 0x8c000000000-0x8c07f7fffff] offset 8c000000000
/pci@500: PCI MEM64 [mem 0x8c100000000-0x8c7ffffffff] offset 88000000000
/pci@500: Unable to request IOMMU resource.
/pci@500: MSI Queue first[0] num[56] count[512] devino[0x6]
/pci@500: MSI first[0] num[512] mask[0x3ff] width[32]
/pci@500: MSI addr32[0x7f800000:0x800000] addr64[0xe00000003f800000:0x800000]
/pci@500: MSI queues at RA [0000001fd3600000]
PCI: Scanning PBM /pci@500
pci_sun4v f02b4a94: PCI host bridge to bus 0001:00
pci_bus 0001:00: root bus resource [io  0x8d100000000-0x8d10fffffff] (bus address [0x0000-0xfffffff])
pci_bus 0001:00: root bus resource [mem 0x8c000000000-0x8c07f7fffff] (bus address [0x00000000-0x7f7fffff])
pci_bus 0001:00: root bus resource [mem 0x8c100000000-0x8c7ffffffff] (bus address [0x4100000000-0x47ffffffff])
pci_bus 0001:00: root bus resource [bus 00-0a]
PCI: scan_bus[/pci@500] bus no 0
  * /pci@500/pci@1
    create device, devfn: 8, type: pciex
    class: 0x60400 device name: 0001:00:01.0
    adding to system ...
pci 0001:00:01.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@500/pci@1)
    Bridge bus range [1 --> 5]
    Bridge ranges[fff8001fffe585c0] simba[0]
    RAW Range[81000000:00000000:00000000:81000000:00000000:00000000:00000000:00001000]
      Using flags[00000101] start[0000000000000000] size[0000000000001000]
    RAW Range[82000000:00000000:00000000:82000000:00000000:00000000:00000000:00200000]
      Using flags[00000200] start[0000000000000000] size[0000000000200000]
    bus name: PCI Bus 0001:01
PCI: scan_bus[/pci@500/pci@1] bus no 1
  * /pci@500/pci@1/pci@0
    create device, devfn: 0, type: pciex
    class: 0x60400 device name: 0001:01:00.0
    adding to system ...
pci 0001:01:00.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@500/pci@1/pci@0)
    Bridge bus range [2 --> 5]
    Bridge ranges[fff8001fffe57480] simba[0]
    RAW Range[81000000:00000000:00000000:81000000:00000000:00000000:00000000:00001000]
      Using flags[00000101] start[0000000000000000] size[0000000000001000]
    RAW Range[82000000:00000000:00000000:82000000:00000000:00000000:00000000:00200000]
      Using flags[00000200] start[0000000000000000] size[0000000000200000]
    bus name: PCI Bus 0001:02
PCI: scan_bus[/pci@500/pci@1/pci@0] bus no 2
  * /pci@500/pci@1/pci@0/pci@0
    create device, devfn: 0, type: pciex
    class: 0x60400 device name: 0001:02:00.0
    adding to system ...
pci 0001:02:00.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@500/pci@1/pci@0/pci@0)
    Bridge bus range [3 --> 3]
    Bridge ranges[          (null)] simba[0]
pci 0001:02:00.0: PCI bridge to [bus 03]
    bus name: PCI Bus 0001:03
PCI: scan_bus[/pci@500/pci@1/pci@0/pci@0] bus no 3
  * /pci@500/pci@1/pci@0/pci@5
    create device, devfn: 28, type: pciex
    class: 0x60400 device name: 0001:02:05.0
    adding to system ...
pci 0001:02:05.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@500/pci@1/pci@0/pci@5)
    Bridge bus range [4 --> 4]
    Bridge ranges[fff8001fffe55040] simba[0]
    RAW Range[81000000:00000000:00000000:81000000:00000000:00000000:00000000:00001000]
      Using flags[00000101] start[0000000000000000] size[0000000000001000]
    RAW Range[82000000:00000000:00000000:82000000:00000000:00000000:00000000:00200000]
      Using flags[00000200] start[0000000000000000] size[0000000000200000]
    bus name: PCI Bus 0001:04
PCI: scan_bus[/pci@500/pci@1/pci@0/pci@5] bus no 4
  * /pci@500/pci@1/pci@0/pci@5/network@0
    create device, devfn: 0, type: network
    class: 0x20000 device name: 0001:04:00.0
    parse addresses (100 bytes) @ fff8001fffe53b40
  start: 8c000000000, end: 8c00001ffff, i: 10
  start: 8c000020000, end: 8c00003ffff, i: 14
  start: 8d100000000, end: 8d10000001f, i: 18
  start: 8c000040000, end: 8c000043fff, i: 1c
  start: 8c000060000, end: 8c00007ffff, i: 30
    adding to system ...
pci 0001:04:00.0: PME# supported from D0 D3hot D3cold
pci 0001:04:00.0: reg 0x184: [mem 0x8c000080000-0x8c000083fff]
pci 0001:04:00.0: VF(n) BAR0 space: [mem 0x8c000080000-0x8c00009ffff] (contains BAR0 for 8 VFs)
pci 0001:04:00.0: reg 0x190: [mem 0x8c0000a0000-0x8c0000a3fff]
pci 0001:04:00.0: VF(n) BAR3 space: [mem 0x8c0000a0000-0x8c0000bffff] (contains BAR3 for 8 VFs)
PCI: dev header type: 0
  * /pci@500/pci@1/pci@0/pci@5/network@0,1
    create device, devfn: 1, type: network
    class: 0x20000 device name: 0001:04:00.1
    parse addresses (100 bytes) @ fff8001fffe51ec0
  start: 8c0000c0000, end: 8c0000dffff, i: 10
  start: 8c0000e0000, end: 8c0000fffff, i: 14
  start: 8d100000020, end: 8d10000003f, i: 18
  start: 8c000044000, end: 8c000047fff, i: 1c
  start: 8c000100000, end: 8c00011ffff, i: 30
    adding to system ...
pci 0001:04:00.1: PME# supported from D0 D3hot D3cold
pci 0001:04:00.1: reg 0x184: [mem 0x8c000120000-0x8c000123fff]
pci 0001:04:00.1: VF(n) BAR0 space: [mem 0x8c000120000-0x8c00013ffff] (contains BAR0 for 8 VFs)
pci 0001:04:00.1: reg 0x190: [mem 0x8c000140000-0x8c000143fff]
pci 0001:04:00.1: VF(n) BAR3 space: [mem 0x8c000140000-0x8c00015ffff] (contains BAR3 for 8 VFs)
PCI: dev header type: 0
  * /pci@500/pci@1/pci@0/pci@6
    create device, devfn: 30, type: pciex
    class: 0x60400 device name: 0001:02:06.0
    adding to system ...
pci 0001:02:06.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@500/pci@1/pci@0/pci@6)
    Bridge bus range [5 --> 5]
    Bridge ranges[          (null)] simba[0]
pci 0001:02:06.0: PCI bridge to [bus 05]
    bus name: PCI Bus 0001:05
PCI: scan_bus[/pci@500/pci@1/pci@0/pci@6] bus no 5
  * /pci@500/pci@2
    create device, devfn: 10, type: pciex
    class: 0x60400 device name: 0001:00:02.0
    adding to system ...
pci 0001:00:02.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@500/pci@2)
    Bridge bus range [6 --> 10]
    Bridge ranges[fff8001fffe4f340] simba[0]
    RAW Range[81000000:00000000:00001000:81000000:00000000:00001000:00000000:00001000]
      Using flags[00000101] start[0000000000001000] size[0000000000001000]
    RAW Range[82000000:00000000:00200000:82000000:00000000:00200000:00000000:00100000]
      Using flags[00000200] start[0000000000200000] size[0000000000100000]
    RAW Range[c3000000:00000041:00000000:c3000000:00000041:00000000:00000000:10000000]
      Using flags[0010220c] start[0000004100000000] size[0000000010000000]
    bus name: PCI Bus 0001:06
PCI: scan_bus[/pci@500/pci@2] bus no 6
  * /pci@500/pci@2/pci@0
    create device, devfn: 0, type: pciex
    class: 0x60400 device name: 0001:06:00.0
    adding to system ...
pci 0001:06:00.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@500/pci@2/pci@0)
    Bridge bus range [7 --> 10]
    Bridge ranges[fff8001fffe4e200] simba[0]
    RAW Range[81000000:00000000:00001000:81000000:00000000:00001000:00000000:00001000]
      Using flags[00000101] start[0000000000001000] size[0000000000001000]
    RAW Range[82000000:00000000:00200000:82000000:00000000:00200000:00000000:00100000]
      Using flags[00000200] start[0000000000200000] size[0000000000100000]
    RAW Range[c3000000:00000041:00000000:c3000000:00000041:00000000:00000000:10000000]
      Using flags[0010220c] start[0000004100000000] size[0000000010000000]
    bus name: PCI Bus 0001:07
PCI: scan_bus[/pci@500/pci@2/pci@0] bus no 7
  * /pci@500/pci@2/pci@0/pci@0
    create device, devfn: 0, type: pciex
    class: 0x60400 device name: 0001:07:00.0
    adding to system ...
pci 0001:07:00.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@500/pci@2/pci@0/pci@0)
    Bridge bus range [8 --> 8]
    Bridge ranges[          (null)] simba[0]
pci 0001:07:00.0: PCI bridge to [bus 08]
    bus name: PCI Bus 0001:08
PCI: scan_bus[/pci@500/pci@2/pci@0/pci@0] bus no 8
  * /pci@500/pci@2/pci@0/pci@6
    create device, devfn: 30, type: pciex
    class: 0x60400 device name: 0001:07:06.0
    adding to system ...
pci 0001:07:06.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@500/pci@2/pci@0/pci@6)
    Bridge bus range [9 --> 9]
    Bridge ranges[          (null)] simba[0]
pci 0001:07:06.0: PCI bridge to [bus 09]
    bus name: PCI Bus 0001:09
PCI: scan_bus[/pci@500/pci@2/pci@0/pci@6] bus no 9
  * /pci@500/pci@2/pci@0/pci@a
    create device, devfn: 50, type: pciex
    class: 0x60400 device name: 0001:07:0a.0
    adding to system ...
pci 0001:07:0a.0: PME# supported from D0 D3hot D3cold
PCI: dev header type: 1
of_scan_pci_bridge(/pci@500/pci@2/pci@0/pci@a)
    Bridge bus range [10 --> 10]
    Bridge ranges[fff8001fffe4aac0] simba[0]
    RAW Range[81000000:00000000:00001000:81000000:00000000:00001000:00000000:00001000]
      Using flags[00000101] start[0000000000001000] size[0000000000001000]
    RAW Range[82000000:00000000:00200000:82000000:00000000:00200000:00000000:00100000]
      Using flags[00000200] start[0000000000200000] size[0000000000100000]
    RAW Range[c3000000:00000041:00000000:c3000000:00000041:00000000:00000000:10000000]
      Using flags[0010220c] start[0000004100000000] size[0000000010000000]
    bus name: PCI Bus 0001:0a
PCI: scan_bus[/pci@500/pci@2/pci@0/pci@a] bus no 10
  * /pci@500/pci@2/pci@0/pci@a/network@0
    create device, devfn: 0, type: network
    class: 0x20000 device name: 0001:0a:00.0
    parse addresses (80 bytes) @ fff8001fffe496c0
  start: 8c100000000, end: 8c10007ffff, i: 10
  start: 8d100001000, end: 8d10000101f, i: 18
  start: 8c100080000, end: 8c100083fff, i: 20
  start: 8c000200000, end: 8c00027ffff, i: 30
    adding to system ...
pci 0001:0a:00.0: PME# supported from D0 D3hot
pci 0001:0a:00.0: reg 0x184: [mem 0x8c100084000-0x8c100087fff 64bit]
pci 0001:0a:00.0: VF(n) BAR0 space: [mem 0x8c100084000-0x8c100183fff 64bit] (contains BAR0 for 64 VFs)
pci 0001:0a:00.0: reg 0x190: [mem 0x8c100184000-0x8c100187fff 64bit]
pci 0001:0a:00.0: VF(n) BAR3 space: [mem 0x8c100184000-0x8c100283fff 64bit] (contains BAR3 for 64 VFs)
PCI: dev header type: 0
  * /pci@500/pci@2/pci@0/pci@a/network@0,1
    create device, devfn: 1, type: network
    class: 0x20000 device name: 0001:0a:00.1
    parse addresses (80 bytes) @ fff8001fffe47540
  start: 8c100300000, end: 8c10037ffff, i: 10
  start: 8d100001020, end: 8d10000103f, i: 18
  start: 8c100284000, end: 8c100287fff, i: 20
  start: 8c000280000, end: 8c0002fffff, i: 30
    adding to system ...
pci 0001:0a:00.1: PME# supported from D0 D3hot
pci 0001:0a:00.1: reg 0x184: [mem 0x8c100380000-0x8c100383fff 64bit]
pci 0001:0a:00.1: VF(n) BAR0 space: [mem 0x8c100380000-0x8c10047ffff 64bit] (contains BAR0 for 64 VFs)
pci 0001:0a:00.1: reg 0x190: [mem 0x8c100480000-0x8c100483fff 64bit]
pci 0001:0a:00.1: VF(n) BAR3 space: [mem 0x8c100480000-0x8c10057ffff 64bit] (contains BAR3 for 64 VFs)
PCI: dev header type: 0
PCI: Claiming 0001:00:01.0: Resource 13: 000008d100000000..000008d100000fff [101]
PCI: Claiming 0001:00:01.0: Resource 14: 000008c000000000..000008c0001fffff [200]
PCI: Claiming 0001:00:02.0: Resource 13: 000008d100001000..000008d100001fff [101]
PCI: Claiming 0001:00:02.0: Resource 14: 000008c000200000..000008c0002fffff [200]
PCI: Claiming 0001:00:02.0: Resource 15: 000008c100000000..000008c10fffffff [10220c]
PCI: Claiming 0001:01:00.0: Resource 13: 000008d100000000..000008d100000fff [101]
PCI: Claiming 0001:01:00.0: Resource 14: 000008c000000000..000008c0001fffff [200]
PCI: Claiming 0001:02:05.0: Resource 13: 000008d100000000..000008d100000fff [101]
PCI: Claiming 0001:02:05.0: Resource 14: 000008c000000000..000008c0001fffff [200]
PCI: Claiming 0001:04:00.0: Resource 0: 000008c000000000..000008c00001ffff [200]
PCI: Claiming 0001:04:00.0: Resource 1: 000008c000020000..000008c00003ffff [200]
PCI: Claiming 0001:04:00.0: Resource 2: 000008d100000000..000008d10000001f [101]
PCI: Claiming 0001:04:00.0: Resource 3: 000008c000040000..000008c000043fff [200]
PCI: Claiming 0001:04:00.0: Resource 6: 000008c000060000..000008c00007ffff [44200]
PCI: Claiming 0001:04:00.0: Resource 7: 000008c000080000..000008c00009ffff [40200]
PCI: Claiming 0001:04:00.0: Resource 10: 000008c0000a0000..000008c0000bffff [40200]
PCI: Claiming 0001:04:00.1: Resource 0: 000008c0000c0000..000008c0000dffff [200]
PCI: Claiming 0001:04:00.1: Resource 1: 000008c0000e0000..000008c0000fffff [200]
PCI: Claiming 0001:04:00.1: Resource 2: 000008d100000020..000008d10000003f [101]
PCI: Claiming 0001:04:00.1: Resource 3: 000008c000044000..000008c000047fff [200]
PCI: Claiming 0001:04:00.1: Resource 6: 000008c000100000..000008c00011ffff [44200]
PCI: Claiming 0001:04:00.1: Resource 7: 000008c000120000..000008c00013ffff [40200]
PCI: Claiming 0001:04:00.1: Resource 10: 000008c000140000..000008c00015ffff [40200]
PCI: Claiming 0001:06:00.0: Resource 13: 000008d100001000..000008d100001fff [101]
PCI: Claiming 0001:06:00.0: Resource 14: 000008c000200000..000008c0002fffff [200]
PCI: Claiming 0001:06:00.0: Resource 15: 000008c100000000..000008c10fffffff [10220c]
PCI: Claiming 0001:07:0a.0: Resource 13: 000008d100001000..000008d100001fff [101]
PCI: Claiming 0001:07:0a.0: Resource 14: 000008c000200000..000008c0002fffff [200]
PCI: Claiming 0001:07:0a.0: Resource 15: 000008c100000000..000008c10fffffff [10220c]
PCI: Claiming 0001:0a:00.0: Resource 0: 000008c100000000..000008c10007ffff [100204]
PCI: Claiming 0001:0a:00.0: Resource 2: 000008d100001000..000008d10000101f [101]
PCI: Claiming 0001:0a:00.0: Resource 4: 000008c100080000..000008c100083fff [100204]
PCI: Claiming 0001:0a:00.0: Resource 6: 000008c000200000..000008c00027ffff [44200]
PCI: Claiming 0001:0a:00.0: Resource 7: 000008c100084000..000008c100183fff [140204]
PCI: Claiming 0001:0a:00.0: Resource 10: 000008c100184000..000008c100283fff [140204]
PCI: Claiming 0001:0a:00.1: Resource 0: 000008c100300000..000008c10037ffff [100204]
PCI: Claiming 0001:0a:00.1: Resource 2: 000008d100001020..000008d10000103f [101]
PCI: Claiming 0001:0a:00.1: Resource 4: 000008c100284000..000008c100287fff [100204]
PCI: Claiming 0001:0a:00.1: Resource 6: 000008c000280000..000008c0002fffff [44200]
PCI: Claiming 0001:0a:00.1: Resource 7: 000008c100380000..000008c10047ffff [140204]
PCI: Claiming 0001:0a:00.1: Resource 10: 000008c100480000..000008c10057ffff [140204]
pci_bus 0001:00:  can't claim Video RAM area [mem 0x8c0000a0000-0x8c0000bffff]: address conflict with PCI Bus 0001:01 [mem 0x8c000000000-0x8c0001fffff]
pci_bus 0001:00:  can't claim System ROM [mem 0x8c0000f0000-0x8c0000fffff]: address conflict with PCI Bus 0001:01 [mem 0x8c000000000-0x8c0001fffff]
pci_bus 0001:00:  can't claim Video ROM [mem 0x8c0000c0000-0x8c0000c7fff]: address conflict with PCI Bus 0001:01 [mem 0x8c000000000-0x8c0001fffff]
vgaarb: device added: PCI:0000:04:00.0,decodes=io+mem,owns=none,locks=none
vgaarb: loaded
vgaarb: bridge control possible 0000:04:00.0
SCSI subsystem initialized
libata version 3.00 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
SUN4V: Reboot data supported (maj=1,min=0).
ds.c:v1.0 (Jul 11, 2007)
clocksource: Switched to clocksource stick
ds-1: Registered pri service.
ds-1: Registered var-config-backup service.
NET: Registered protocol family 2
TCP established hash table entries: 524288 (order: 9, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 7, 1048576 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
UDP hash table entries: 65536 (order: 8, 2097152 bytes)
UDP-Lite hash table entries: 65536 (order: 8, 2097152 bytes)
NET: Registered protocol family 1
PCI: Enabling device: (0000:08:00.2), cmd 2
PCI: CLS mismatch (64 != 512), using 64 bytes
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 31520K (fff8000064000000 - fff8000065ec8000)
futex hash table entries: 32768 (order: 8, 2097152 bytes)
audit: initializing netlink subsys (disabled)
audit: type=2000 audit(2.451:1): initialized
Initialise system trusted keyring
HugeTLB registered 8 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 1024 (order 0, 8192 bytes)
SELinux:  Registering netfilter hooks
Key type asymmetric registered
Asymmetric key parser 'x509' registered
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
io scheduler noop registered
io scheduler deadline registered (default)
io scheduler cfq registered
f02993d8: ttyS0 at I/O 0x0 (irq = 1, base_baud = 115200) is a SUN4V HCONS
console [ttyHV0] enabled
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
n2rng.c:v0.2 (July 27, 2011)
n2rng f029907c: Registered RNG HVAPI major 2 minor 0
n2rng f029907c: Found multi-unit-capable RNG, units: 2
n2rng f029907c: Selftest passed on unit 0
n2rng f029907c: Selftest passed on unit 1
n2rng f029907c: RNG ready
brd: module loaded
loop: module loaded
zram: Added device: zram0
libphy: Fixed MDIO Bus: probed
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
ehci-pci 0000:08:00.2: EHCI Host Controller
ehci-pci 0000:08:00.2: new USB bus registered, assigned bus number 1
ehci-pci 0000:08:00.2: irq 5, io mem 0x84000b04000
ehci-pci 0000:08:00.2: USB 2.0 started, EHCI 1.00
usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb1: Product: EHCI Host Controller
usb usb1: Manufacturer: Linux 4.3.0-rc7-kh+ ehci_hcd
usb usb1: SerialNumber: 0000:08:00.2
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 5 ports detected
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci-pci: OHCI PCI platform driver
ohci-pci 0000:08:00.0: OHCI PCI host controller
ohci-pci 0000:08:00.0: new USB bus registered, assigned bus number 2
ohci-pci 0000:08:00.0: irq 6, io mem 0x84000b00000
usb usb2: New USB device found, idVendor=1d6b, idProduct=0001
usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb2: Product: OHCI PCI host controller
usb usb2: Manufacturer: Linux 4.3.0-rc7-kh+ ohci_hcd
usb usb2: SerialNumber: 0000:08:00.0
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 3 ports detected
ohci-pci 0000:08:00.1: OHCI PCI host controller
ohci-pci 0000:08:00.1: new USB bus registered, assigned bus number 3
ohci-pci 0000:08:00.1: irq 4, io mem 0x84000b02000
usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
usb usb3: Product: OHCI PCI host controller
usb usb3: Manufacturer: Linux 4.3.0-rc7-kh+ ohci_hcd
usb usb3: SerialNumber: 0000:08:00.1
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
uhci_hcd: USB Universal Host Controller Interface driver
usb 1-2: new high-speed USB device number 2 using ehci-pci
mousedev: PS/2 mouse device common for all mice
rtc-sun4v rtc-sun4v: rtc core: registered sun4v as rtc0
hidraw: raw HID events driver (C) Jiri Kosina
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
drop_monitor: Initializing network drop monitor service
Initializing XFRM netlink socket
NET: Registered protocol family 17
Key type dns_resolver registered
registered taskstats version 1
Loading compiled-in X.509 certificates
Problem loading in-kernel X.509 certificate (-76)
ima: No TPM chip found, activating TPM-bypass!
rtc-sun4v rtc-sun4v: setting system clock to 2015-10-30 21:31:07 UTC (1446240667)
usb 1-2: New USB device found, idVendor=04b4, idProduct=6560
usb 1-2: New USB device strings: Mfr=0, Product=0, SerialNumber=0
hub 1-2:1.0: USB hub found
hub 1-2:1.0: 4 ports detected
dracut: dracut-004-388.0.2.el6
dracut: rd_NO_LUKS: removing cryptoluks activation
dm_mod: module verification failed: signature and/or required key missing - tainting kernel
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.33.0-ioctl (2015-8-18) initialised: dm-devel@redhat.com
usb 1-2.3: new high-speed USB device number 3 using ehci-pci
udev: starting version 147
usb 1-2.3: New USB device found, idVendor=046b, idProduct=ff01
usb 1-2.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 1-2.3: Product: Generic Hub
usb 1-2.3: Manufacturer: American Megatrends Inc.
hub 1-2.3:1.0: USB hub found
hub 1-2.3:1.0: 3 ports detected
dracut: Starting plymouth daemon
dracut: rd_NO_DM: removing DM RAID activation
usb 1-2.4: new low-speed USB device number 4 using ehci-pci
dracut: rd_NO_MD: removing MD RAID activation
sha256_sparc64: Using sparc64 sha256 opcode optimized SHA-256/SHA-224 implementation
Loading iSCSI transport class v2.0-870.
iscsi: registered transport (qla4xxx)
QLogic iSCSI HBA Driver
md5_sparc64: Using sparc64 md5 opcode optimized MD5 implementation
NET: Registered protocol family 10
sha1_sparc64: Using sparc64 sha1 opcode optimized SHA-1 implementation
crc32c_sparc64: Using sparc64 crc32c opcode optimized CRC32C implementation
aes_sparc64: Using sparc64 aes opcodes optimized AES implementation
camellia_sparc64: Using sparc64 camellia opcodes optimized CAMELLIA implementation
sha512_sparc64: Using sparc64 sha512 opcode optimized SHA-512/SHA-384 implementation
des_sparc64: Using sparc64 des opcodes optimized DES implementation
usb 1-2.4: New USB device found, idVendor=046b, idProduct=ff10
usb 1-2.4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 1-2.4: Product: Virtual Keyboard and Mouse
usb 1-2.4: Manufacturer: American Megatrends Inc.
input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/root/f02a5770/pci0000:00/0000:00:01.0/0000:01:00.0/0000:02:0b.0/0000:07:00.0/0000:08:00.2/usb1/1-2/1-2.4/1-2.4:1.0/0003:046B:FF10.0001/input/input0
hid-generic 0003:046B:FF10.0001: input,hidraw0: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:08:00.2-2.4/input0
input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/root/f02a5770/pci0000:00/0000:00:01.0/0000:01:00.0/0000:02:0b.0/0000:07:00.0/0000:08:00.2/usb1/1-2/1-2.4/1-2.4:1.1/0003:046B:FF10.0002/input/input1
hid-generic 0003:046B:FF10.0002: input,hidraw1: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0000:08:00.2-2.4/input1
usb 1-2.3.2: new high-speed USB device number 5 using ehci-pci
usb 1-2.3.2: New USB device found, idVendor=046b, idProduct=ff20
usb 1-2.3.2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 1-2.3.2: Product: Virtual Cdrom Device
usb 1-2.3.2: Manufacturer: American Megatrends Inc.
usb 1-2.3.3: new high-speed USB device number 6 using ehci-pci
libcxgbi:libcxgbi_init_module: Chelsio iSCSI driver library libcxgbi v0.9.1-ko (Apr. 2015)
libcxgbi:libcxgbi_init_module: tag itt 0x1fff, 13 bits, age 0xf, 4 bits.
usb 1-2.3.3: New USB device found, idVendor=0430, idProduct=a4a2
usb 1-2.3.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 1-2.3.3: Product: Virtual Eth Device
usb 1-2.3.3: Manufacturer: SunMicro
libcxgbi:ddp_setup_host_page_size: system PAGE 8192, ddp idx 1.
Chelsio T3 iSCSI Driver cxgb3i v2.0.1-ko (Apr. 2015)
iscsi: registered transport (cxgb3i)
Chelsio T4/T5 iSCSI Driver cxgb4i v0.9.5-ko (Apr. 2015)
iscsi: registered transport (cxgb4i)
cnic: QLogic cnicDriver v2.5.22 (July 20, 2015)
QLogic NetXtreme II iSCSI Driver bnx2i v2.7.10.1 (Jul 16, 2014)
iscsi: registered transport (bnx2i)
iscsi: registered transport (be2iscsi)
In beiscsi_module_init, tt=00000000102e40e0
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
PTP clock support registered
igb: Intel(R) Gigabit Ethernet Network Driver - version 5.3.0-k
igb: Copyright (c) 2007-2014 Intel Corporation.
PCI: Enabling device: (0000:05:00.0), cmd 146
igb 0000:05:00.0: added PHC on eth0
igb 0000:05:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0000:05:00.0: eth0: (PCIe:2.5Gb/s:Width x4) 00:21:28:f9:f6:4e
igb 0000:05:00.0: eth0: PBA No: Unknown
igb 0000:05:00.0: Using legacy interrupts. 1 rx queue(s), 1 tx queue(s)
PCI: Enabling device: (0000:05:00.1), cmd 146
igb 0000:05:00.1: added PHC on eth1
igb 0000:05:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0000:05:00.1: eth1: (PCIe:2.5Gb/s:Width x4) 00:21:28:f9:f6:4f
igb 0000:05:00.1: eth1: PBA No: Unknown
igb 0000:05:00.1: Using legacy interrupts. 1 rx queue(s), 1 tx queue(s)
PCI: Enabling device: (0001:04:00.0), cmd 146
igb 0001:04:00.0: added PHC on eth2
igb 0001:04:00.0: Intel(R) Gigabit Ethernet Network Connection
igb 0001:04:00.0: eth2: (PCIe:2.5Gb/s:Width x4) 00:21:28:f9:f6:50
igb 0001:04:00.0: eth2: PBA No: Unknown
igb 0001:04:00.0: Using legacy interrupts. 1 rx queue(s), 1 tx queue(s)
PCI: Enabling device: (0001:04:00.1), cmd 146
igb 0001:04:00.1: added PHC on eth3
igb 0001:04:00.1: Intel(R) Gigabit Ethernet Network Connection
igb 0001:04:00.1: eth3: (PCIe:2.5Gb/s:Width x4) 00:21:28:f9:f6:51
igb 0001:04:00.1: eth3: PBA No: Unknown
igb 0001:04:00.1: Using legacy interrupts. 1 rx queue(s), 1 tx queue(s)
Emulex LightPulse Fibre Channel SCSI driver 10.7.0.0.
Copyright(c) 2004-2015 Emulex.  All rights reserved.
scsi host0: Emulex LPe12000-S PCIe Fibre Channel Adapter  on PCI bus 0e device 00 irq 5
scsi host1: Emulex LPe12000-S PCIe Fibre Channel Adapter  on PCI bus 0e device 01 irq 7
mpt2sas version 20.100.00.00 loaded
mpt2sas0: 32 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (132479920 kB)
mpt2sas0: MSI-X vectors supported: 1, no of cores: 128, max_msix_vectors: 8
mpt2sas0: IO-APIC enabled: IRQ 6
mpt2sas0: iomem(0x0000084110000000), mapped(0x0000084110000000), size(16384)
mpt2sas0: ioport(0x0000085100003000), size(256)
mpt2sas0: Allocated physical memory: size(2231 kB)
mpt2sas0: Current Controller Queue Depth(1469), Max Controller Queue Depth(1600)
mpt2sas0: Scatter Gather Elements per IO(128)
mpt2sas0: LSISAS2008: FWVersion(09.00.00.00), ChipRevision(0x03), BiosVersion(00.00.00.00)
mpt2sas0: Protocol=(Initiator,Target), Capabilities=(Raid,TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)
scsi host2: Fusion MPT SAS Host
mpt2sas0: sending port enable !!
mpt2sas0: host_add: handle(0x0001), sas_addr(0x50800200011410e8), phys(8)
mpt2sas0: port enable: SUCCESS
scsi 2:0:0:0: Direct-Access     HITACHI  H106060SDSUN600G A2B0 PQ: 0 ANSI: 6
scsi 2:0:0:0: SSP: handle(0x0009), sas_addr(0x5000cca02593bc51), phy(0), device_name(0xa0cc005053bc9325)
scsi 2:0:0:0: SSP: enclosure_logical_id(0x50800200011410e8), slot(0)
scsi 2:0:1:0: CD-ROM            TEAC     DV-W28SS-V       1.0B PQ: 0 ANSI: 0
scsi 2:0:1:0: SATA: handle(0x000a), sas_addr(0x4433221107000000), phy(7), device_name(0x0000000000000000)
scsi 2:0:1:0: SATA: enclosure_logical_id(0x50800200011410e8), slot(7)
scsi 2:0:1:0: atapi(y), ncq(n), asyn_notify(n), smart(n), fua(n), sw_preserve(n)
scsi 2:0:2:0: Direct-Access     ATA      SEAGATE ST95000N SF03 PQ: 0 ANSI: 5
scsi 2:0:2:0: SATA: handle(0x000b), sas_addr(0x4433221102000000), phy(2), device_name(0xc50050002c5b2703)
scsi 2:0:2:0: SATA: enclosure_logical_id(0x50800200011410e8), slot(2)
scsi 2:0:2:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y)
scsi 2:0:3:0: Direct-Access     HITACHI  H106060SDSUN600G A2B0 PQ: 0 ANSI: 6
scsi 2:0:3:0: SSP: handle(0x000c), sas_addr(0x5000cca02593ca7d), phy(1), device_name(0xa0cc00507fca9325)
scsi 2:0:3:0: SSP: enclosure_logical_id(0x50800200011410e8), slot(1)
scsi 2:0:4:0: Direct-Access     HITACHI  H106060SDSUN600G A2B0 PQ: 0 ANSI: 6
scsi 2:0:4:0: SSP: handle(0x000d), sas_addr(0x5000cca025934549), phy(3), device_name(0xa0cc00504b459325)
scsi 2:0:4:0: SSP: enclosure_logical_id(0x50800200011410e8), slot(3)
ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 4.0.1-k
ixgbe: Copyright (c) 1999-2015 Intel Corporation.
PCI: Enabling device: (0001:0a:00.0), cmd 146
ixgbe 0001:0a:00.0: Failed to allocate MSI-X interrupts. Err: -22
ixgbe 0001:0a:00.0: Disabling SR-IOV support
ixgbe 0001:0a:00.0: Disabling RSS support
ixgbe 0001:0a:00.0: Failed to allocate MSI interrupt, falling back to legacy. Error: -22
ixgbe 0001:0a:00.0: Multiqueue Disabled: Rx Queue count = 1, Tx Queue count = 1
ixgbe 0001:0a:00.0: PCI Express bandwidth of 32GT/s available
ixgbe 0001:0a:00.0: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
ixgbe 0001:0a:00.0: MAC: 2, PHY: 17, SFP+: 5, PBA No: E70856-008
ixgbe 0001:0a:00.0: 90:e2:ba:14:31:88
ixgbe 0001:0a:00.0: Intel(R) 10 Gigabit Network Connection
PCI: Enabling device: (0001:0a:00.1), cmd 146
ixgbe 0001:0a:00.1: Failed to allocate MSI-X interrupts. Err: -22
ixgbe 0001:0a:00.1: Disabling SR-IOV support
ixgbe 0001:0a:00.1: Disabling RSS support
ixgbe 0001:0a:00.1: Failed to allocate MSI interrupt, falling back to legacy. Error: -22
ixgbe 0001:0a:00.1: Multiqueue Disabled: Rx Queue count = 1, Tx Queue count = 1
ixgbe 0001:0a:00.1: PCI Express bandwidth of 32GT/s available
ixgbe 0001:0a:00.1: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%)
ixgbe 0001:0a:00.1: MAC: 2, PHY: 17, SFP+: 6, PBA No: E70856-008
ixgbe 0001:0a:00.1: 90:e2:ba:14:31:89
ixgbe 0001:0a:00.1: Intel(R) 10 Gigabit Network Connection
usb-storage 1-2.3.2:1.0: USB Mass Storage device detected
scsi host3: usb-storage 1-2.3.2:1.0
usbcore: registered new interface driver usb-storage
sd 2:0:0:0: [sda] 1172123568 512-byte logical blocks: (600 GB/558 GiB)
sd 2:0:4:0: [sdd] 1172123568 512-byte logical blocks: (600 GB/558 GiB)
sd 2:0:3:0: [sdc] 1172123568 512-byte logical blocks: (600 GB/558 GiB)
sd 2:0:2:0: [sdb] 976773168 512-byte logical blocks: (500 GB/465 GiB)
sd 2:0:0:0: [sda] Write Protect is off
sd 2:0:0:0: [sda] Mode Sense: e7 00 10 08
sd 2:0:3:0: [sdc] Write Protect is off
sd 2:0:3:0: [sdc] Mode Sense: e7 00 10 08
sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
sd 2:0:3:0: [sdc] Write cache: disabled, read cache: enabled, supports DPO and FUA
 sda: sda1 sda3
 sdc: sdc3 sdc4 sdc5 sdc6 sdc7
sd 2:0:0:0: [sda] Attached SCSI disk
sd 2:0:3:0: [sdc] Attached SCSI disk
sd 2:0:2:0: [sdb] Write Protect is off
sd 2:0:2:0: [sdb] Mode Sense: 7f 00 00 08
sd 2:0:2:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 sdb: sdb1 sdb2 sdb3
sd 2:0:4:0: [sdd] Write Protect is off
sd 2:0:4:0: [sdd] Mode Sense: e7 00 10 08
sd 2:0:4:0: [sdd] Write cache: disabled, read cache: enabled, supports DPO and FUA
 sdd:
sd 2:0:4:0: [sdd] Attached SCSI disk
sd 2:0:2:0: [sdb] Attached SCSI disk
sr 2:0:1:0: [sr0] out of memory.
cdrom: Uniform CD-ROM driver Revision: 3.20
sr 2:0:1:0: Attached scsi CD-ROM sr0
dracut: Scanning devices sdb2  for LVM logical volumes VolGroup/lv_swap VolGroup/lv_root 
random: lvm urandom read with 112 bits of entropy available
random: nonblocking pool is initialized
scsi 3:0:0:0: CD-ROM            AMI      Virtual CDROM    1.00 PQ: 0 ANSI: 0 CCS
sr 3:0:0:0: [sr1] out of memory.
sr 3:0:0:0: Attached scsi CD-ROM sr1
dracut: inactive '/dev/VolGroup/lv_root' [454.85 GiB] inherit
dracut: inactive '/dev/VolGroup/lv_swap' [9.91 GiB] inherit
EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null)
dracut: Mounted root filesystem /dev/mapper/VolGroup-lv_root
dracut: Loading SELinux policy
SELinux: 65536 avtab hash slots, 305186 rules.
SELinux: 65536 avtab hash slots, 305186 rules.
SELinux:  9 users, 12 roles, 4200 types, 234 bools, 1 sens, 1024 cats
SELinux:  81 classes, 305186 rules
SELinux:  Permission execmod in class dir not defined in policy.
SELinux:  Permission execmod in class lnk_file not defined in policy.
SELinux:  Permission execmod in class blk_file not defined in policy.
SELinux:  Permission execmod in class sock_file not defined in policy.
SELinux:  Permission execmod in class fifo_file not defined in policy.
SELinux:  Class netlink_iscsi_socket not defined in policy.
SELinux:  Class netlink_fib_lookup_socket not defined in policy.
SELinux:  Class netlink_connector_socket not defined in policy.
SELinux:  Class netlink_netfilter_socket not defined in policy.
SELinux:  Class netlink_generic_socket not defined in policy.
SELinux:  Class netlink_scsitransport_socket not defined in policy.
SELinux:  Class netlink_rdma_socket not defined in policy.
SELinux:  Class netlink_crypto_socket not defined in policy.
SELinux:  Permission syslog in class capability2 not defined in policy.
SELinux:  Permission wake_alarm in class capability2 not defined in policy.
SELinux:  Permission block_suspend in class capability2 not defined in policy.
SELinux:  Permission audit_read in class capability2 not defined in policy.
SELinux:  Permission attach_queue in class tun_socket not defined in policy.
SELinux:  Class binder not defined in policy.
SELinux: the above unknown classes and permissions will be allowed
SELinux:  Completing initialization.
SELinux:  Setting up existing superblocks.
audit: type=1403 audit(1446240688.159:2): policy loaded auid=4294967295 ses=4294967295
dracut: 
dracut: Switching root
udev: starting version 147
sd 2:0:0:0: Attached scsi generic sg0 type 0
sr 2:0:1:0: Attached scsi generic sg1 type 5
sd 2:0:2:0: Attached scsi generic sg2 type 0
sd 2:0:3:0: Attached scsi generic sg3 type 0
sd 2:0:4:0: Attached scsi generic sg4 type 0
sr 3:0:0:0: Attached scsi generic sg5 type 5
cdc_ether 1-2.3.3:2.0 usb0: register 'cdc_ether' at usb-0000:08:00.2-2.3.3, CDC Ethernet Device, 02:21:28:57:47:17
usbcore: registered new interface driver cdc_ether
device-mapper: ioctl: device doesn't appear to be in the dev hash table.
EXT4-fs (dm-0): re-mounted. Opts: (null)
EXT4-fs (sdb1): mounting ext3 file system using the ext4 subsystem
EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
Adding 10387448k swap on /dev/mapper/VolGroup-lv_swap.  Priority:-1 extents:1 across:10387448k FS
IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
igb 0000:05:00.0 eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
audit: type=1305 audit(1446240711.047:3): audit_pid=3147 old=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=1
8021q: 802.1Q VLAN Support v1.8
8021q: adding VLAN 0 to HW filter on device eth0
bnx2fc: QLogic NetXtreme II FCoE Driver bnx2fc v2.4.2 (Dec 11, 2013)
cdc_ether 1-2.3.3:2.0 usb0: kevent 12 may have been dropped
IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
8021q: adding VLAN 0 to HW filter on device eth1
IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready
8021q: adding VLAN 0 to HW filter on device eth2
IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready
8021q: adding VLAN 0 to HW filter on device eth3
ip_tables: (C) 2000-2006 Netfilter Core Team
nf_conntrack version 0.5.0 (65536 buckets, 262144 max)
igb 0000:05:00.1 eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
igb 0001:04:00.1 eth3: igb: eth3 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
IPv6: ADDRCONF(NETDEV_CHANGE): eth3: link becomes ready
Process accounting resumed

[-- Attachment #3: iomem.4.3.0-rc7-pci.t4 --]
[-- Type: text/plain, Size: 4921 bytes --]

20400000-5fdb7fff : System RAM
5fdba000-5fdbbfff : System RAM
60000000-1fffe8dfff : System RAM
  60004000-605cf0bf : Kernel code
  605cf0c0-6093790f : Kernel data
  609ca000-60bbf040 : Kernel bss
1fffe9e000-1fffeb1fff : System RAM
1ffff30000-1ffff3ffff : System RAM
84000000000-8407f7fffff : /pci@400
  84000000000-84000bfffff : PCI Bus 0000:01
    84000000000-84000bfffff : PCI Bus 0000:02
      84000000000-840008fffff : PCI Bus 0000:03
        84000000000-840008fffff : PCI Bus 0000:04
          84000000000-840007fffff : 0000:04:00.0
          84000800000-8400081ffff : 0000:04:00.0
      84000900000-84000afffff : PCI Bus 0000:05
        84000900000-8400091ffff : 0000:05:00.0
          84000900000-8400091ffff : igb
        84000920000-8400093ffff : 0000:05:00.0
          84000920000-8400093ffff : igb
        84000940000-84000943fff : 0000:05:00.0
          84000940000-84000943fff : igb
        84000944000-84000947fff : 0000:05:00.1
          84000944000-84000947fff : igb
        84000960000-8400097ffff : 0000:05:00.0
        84000980000-8400099ffff : 0000:05:00.0
        840009a0000-840009bffff : 0000:05:00.0
        840009c0000-840009dffff : 0000:05:00.1
          840009c0000-840009dffff : igb
        840009e0000-840009fffff : 0000:05:00.1
          840009e0000-840009fffff : igb
        84000a00000-84000a1ffff : 0000:05:00.1
        84000a20000-84000a3ffff : 0000:05:00.1
        84000a40000-84000a5ffff : 0000:05:00.1
      84000b00000-84000bfffff : PCI Bus 0000:07
        84000b00000-84000bfffff : PCI Bus 0000:08
          84000b00000-84000b01fff : 0000:08:00.0
          84000b00000-84000b01fff : ohci_hcd
          84000b02000-84000b03fff : 0000:08:00.1
          84000b02000-84000b03fff : ohci_hcd
          84000b04000-84000b05fff : 0000:08:00.2
          84000b04000-84000b05fff : ehci_hcd
  84000c00000-84000dfffff : PCI Bus 0000:0a
    84000c00000-84000dfffff : PCI Bus 0000:0b
      84000c00000-84000cfffff : PCI Bus 0000:0e
        84000c00000-84000c3ffff : 0000:0e:00.0
        84000c40000-84000c7ffff : 0000:0e:00.1
      84000d00000-84000dfffff : PCI Bus 0000:0f
        84000d00000-84000d7ffff : 0000:0f:00.0
84100000000-847ffffffff : /pci@400
  84100000000-8411fffffff : PCI Bus 0000:0a
    84100000000-8411fffffff : PCI Bus 0000:0b
      84100000000-8410fffffff : PCI Bus 0000:0e
        84100000000-84100001fff : 0000:0e:00.0
          84100000000-84100001fff : lpfc
        84100002000-84100003fff : 0000:0e:00.1
          84100002000-84100003fff : lpfc
        84100004000-84100007fff : 0000:0e:00.0
          84100004000-84100007fff : lpfc
        84100008000-8410000bfff : 0000:0e:00.1
          84100008000-8410000bfff : lpfc
      84110000000-8411fffffff : PCI Bus 0000:0f
        84110000000-84110003fff : 0000:0f:00.0
          84110000000-84110003fff : mpt2sas
        84110040000-8411007ffff : 0000:0f:00.0
          84110040000-8411007ffff : mpt2sas
        84110080000-841100bffff : 0000:0f:00.0
        841100c0000-841104bffff : 0000:0f:00.0
8c000000000-8c07f7fffff : /pci@500
  8c000000000-8c0001fffff : PCI Bus 0001:01
    8c000000000-8c0001fffff : PCI Bus 0001:02
      8c000000000-8c0001fffff : PCI Bus 0001:04
        8c000000000-8c00001ffff : 0001:04:00.0
          8c000000000-8c00001ffff : igb
        8c000020000-8c00003ffff : 0001:04:00.0
          8c000020000-8c00003ffff : igb
        8c000040000-8c000043fff : 0001:04:00.0
          8c000040000-8c000043fff : igb
        8c000044000-8c000047fff : 0001:04:00.1
          8c000044000-8c000047fff : igb
        8c000060000-8c00007ffff : 0001:04:00.0
        8c000080000-8c00009ffff : 0001:04:00.0
        8c0000a0000-8c0000bffff : 0001:04:00.0
        8c0000c0000-8c0000dffff : 0001:04:00.1
          8c0000c0000-8c0000dffff : igb
        8c0000e0000-8c0000fffff : 0001:04:00.1
          8c0000e0000-8c0000fffff : igb
        8c000100000-8c00011ffff : 0001:04:00.1
        8c000120000-8c00013ffff : 0001:04:00.1
        8c000140000-8c00015ffff : 0001:04:00.1
  8c000200000-8c0002fffff : PCI Bus 0001:06
    8c000200000-8c0002fffff : PCI Bus 0001:07
      8c000200000-8c0002fffff : PCI Bus 0001:0a
        8c000200000-8c00027ffff : 0001:0a:00.0
        8c000280000-8c0002fffff : 0001:0a:00.1
8c100000000-8c7ffffffff : /pci@500
  8c100000000-8c10fffffff : PCI Bus 0001:06
    8c100000000-8c10fffffff : PCI Bus 0001:07
      8c100000000-8c10fffffff : PCI Bus 0001:0a
        8c100000000-8c10007ffff : 0001:0a:00.0
          8c100000000-8c10007ffff : ixgbe
        8c100080000-8c100083fff : 0001:0a:00.0
          8c100080000-8c100083fff : ixgbe
        8c100084000-8c100183fff : 0001:0a:00.0
        8c100184000-8c100283fff : 0001:0a:00.0
        8c100284000-8c100287fff : 0001:0a:00.1
          8c100284000-8c100287fff : ixgbe
        8c100300000-8c10037ffff : 0001:0a:00.1
          8c100300000-8c10037ffff : ixgbe
        8c100380000-8c10047ffff : 0001:0a:00.1
        8c100480000-8c10057ffff : 0001:0a:00.1

[-- Attachment #4: lspci.t4-1 --]
[-- Type: text/plain, Size: 2098 bytes --]

-+-[0001:00]-+-01.0-[01-05]----00.0-[02-05]--+-00.0-[03]--
 |           |                               +-05.0-[04]--+-00.0  Intel Corporation 82576 Gigabit Network Connection
 |           |                               |            \-00.1  Intel Corporation 82576 Gigabit Network Connection
 |           |                               \-06.0-[05]--
 |           \-02.0-[06-0a]----00.0-[07-0a]--+-00.0-[08]--
 |                                           +-06.0-[09]--
 |                                           \-0a.0-[0a]--+-00.0  Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection
 |                                                        \-00.1  Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection
 \-[0000:00]-+-01.0-[01-09]----00.0-[02-09]--+-00.0-[03-04]----00.0-[04]----00.0  ASPEED Technology, Inc. ASPEED Graphics Family
             |                               +-04.0-[05]--+-00.0  Intel Corporation 82576 Gigabit Network Connection
             |                               |            \-00.1  Intel Corporation 82576 Gigabit Network Connection
             |                               +-08.0-[06]--
             |                               +-0b.0-[07-08]----00.0-[08]--+-00.0  NEC Corporation OHCI USB Controller
             |                               |                            +-00.1  NEC Corporation OHCI USB Controller
             |                               |                            \-00.2  NEC Corporation uPD72010x USB 2.0 Controller
             |                               \-0c.0-[09]--
             \-02.0-[0a-0f]----00.0-[0b-0f]--+-00.0-[0c]--
                                             +-04.0-[0d]--
                                             +-08.0-[0e]--+-00.0  Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter
                                             |            \-00.1  Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter
                                             \-0e.0-[0f]----00.0  LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]

[-- Attachment #5: lspci.t4-2 --]
[-- Type: text/plain, Size: 147384 bytes --]

0000:00:01.0 PCI bridge: Oracle/SUN Device 8186 (rev 01) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Bus: primary=00, secondary=01, subordinate=09, sec-latency=0
	I/O behind bridge: 00000000-00001fff
	Memory behind bridge: 00000000-00bfffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [60] Express (v2) Root Port (Slot-), MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x8, ASPM unknown, Latency L0 <2us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 05, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [138 v1] Vendor Specific Information: ID=1112 Rev=1 Len=014 <?>
	Capabilities: [14c v1] #00
00: 8e 10 86 81 07 00 10 00 01 00 04 06 00 00 01 00
10: 00 00 00 00 00 00 00 00 00 01 09 00 01 11 00 00
20: 00 00 b0 00 f1 ff 01 00 4d 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 01 60 03 c8 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 10 00 42 00 01 80 00 00 00 00 00 00 82 50 39 00
70: 00 00 82 70 00 00 00 00 00 00 00 01 00 00 00 00
80: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
90: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:00:02.0 PCI bridge: Oracle/SUN Device 8186 (rev 01) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Bus: primary=00, secondary=0a, subordinate=0f, sec-latency=0
	I/O behind bridge: 00002000-00003fff
	Memory behind bridge: 00c00000-00dfffff
	Prefetchable memory behind bridge: 0000004100000000-000000411fffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [60] Express (v2) Root Port (Slot-), MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #1, Speed 5GT/s, Width x8, ASPM unknown, Latency L0 <2us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 05, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [138 v1] Vendor Specific Information: ID=1112 Rev=1 Len=014 <?>
	Capabilities: [14c v1] #00
00: 8e 10 86 81 07 00 10 00 01 00 04 06 00 00 01 00
10: 00 00 00 00 00 00 00 00 00 0a 0f 00 21 31 00 00
20: c0 00 d0 00 01 00 f1 1f 41 00 00 00 41 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 01 60 03 c8 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 10 00 42 00 01 80 00 00 00 00 00 00 82 50 39 01
70: 00 00 82 70 00 00 00 00 00 00 00 01 00 00 00 00
80: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
90: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:01:00.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=01, secondary=02, subordinate=09, sec-latency=0
	I/O behind bridge: 00000000-00001fff
	Memory behind bridge: 00000000-00bfffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Upstream Port, MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] #00 [0000]
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 01 02 09 00 01 11 00 00
20: 00 00 b0 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 52 00 24 80 00 00 00 00 00 00 82 6c 01 00
50: 00 00 82 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0000:02:00.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=02, secondary=03, subordinate=04, sec-latency=0
	I/O behind bridge: 00000000-00000fff
	Memory behind bridge: 00000000-008fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot-), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #10, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive+ BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 02 03 04 00 01 01 00 00
20: 00 00 80 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 00 24 80 00 00 00 00 00 00 42 6c 39 0a
50: 00 00 11 20 00 00 00 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0000:02:04.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=02, secondary=05, subordinate=05, sec-latency=0
	I/O behind bridge: 00001000-00001fff
	Memory behind bridge: 00900000-00afffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot-), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #4, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk- DLActive+ BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd+
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 02 05 05 00 11 11 00 00
20: 90 00 a0 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 00 24 80 00 00 00 00 00 00 42 6c 39 04
50: 00 00 41 20 00 00 00 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 20 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0000:02:08.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Physical Slot: 8
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=02, secondary=06, subordinate=06, sec-latency=0
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot+), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #8, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #6, PowerLimit 0.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 02 06 06 00 f1 01 00 00
20: f0 ff 00 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: ff ff 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 01 24 80 00 00 00 00 00 00 82 6c 39 08
50: 00 00 01 00 00 00 30 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0000:02:0b.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=02, secondary=07, subordinate=08, sec-latency=0
	Memory behind bridge: 00b00000-00bfffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot-), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #11, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive+ BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 02 07 08 00 f1 01 00 00
20: b0 00 b0 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: ff ff 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 00 24 80 00 00 00 00 00 00 42 6c 39 0b
50: 00 00 11 20 00 00 00 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0000:02:0c.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=02, secondary=09, subordinate=09, sec-latency=0
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot+), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #12, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #8, PowerLimit 0.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 02 09 09 00 f1 01 00 00
20: f0 ff 00 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: ff ff 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 01 24 80 00 00 00 00 00 00 82 6c 39 0c
50: 00 00 01 00 00 00 40 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0000:03:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=03, secondary=04, subordinate=04, sec-latency=0
	I/O behind bridge: 00000000-00000fff
	Memory behind bridge: 00000000-008fffff
	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [78] Power Management version 3
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [80] Express (v1) PCI/PCI-X Bridge, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ BrConfRtry-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #1, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <2us, L1 <2us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
	Capabilities: [a4] Subsystem: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
00: 03 1a 50 11 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 03 04 04 00 01 01 20 02
20: 00 00 80 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 00 01 00 00
40: 00 00 00 00 60 61 82 01 00 00 00 00 00 00 00 00
50: 05 78 80 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 11 78 00 00 00 00 00 00
70: 01 00 00 00 00 00 00 00 01 80 c3 ff 08 00 00 00
80: 10 a4 71 00 00 80 00 00 10 28 10 00 11 dc 00 01
90: 00 00 11 10 00 00 00 00 c0 03 00 00 00 00 00 00
a0: 00 00 00 00 0d 00 00 00 03 1a 50 11 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:04:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 10) (prog-if 00 [VGA controller])
	Subsystem: ASPEED Technology, Inc. ASPEED Graphics Family
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 00000004
	Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [size=8M]
	Region 1: Memory at 00800000 (32-bit, non-prefetchable) [size=128K]
	Region 2: I/O ports at 00000000 [size=128]
	[virtual] Expansion ROM at 84000000000 [disabled] [size=64K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=375mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
00: 03 1a 00 20 03 00 10 02 10 00 00 03 00 00 00 00
10: 00 00 00 00 00 00 80 00 01 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 03 1a 00 20
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 01 00 00
40: 01 00 c3 ff 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:05:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
	Subsystem: Oracle/SUN Device 4848
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 00000004
	Region 0: Memory at 00900000 (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at 00920000 (32-bit, non-prefetchable) [size=128K]
	Region 2: I/O ports at 00001000 [disabled] [size=32]
	Region 3: Memory at 00940000 (32-bit, non-prefetchable) [size=16K]
	Expansion ROM at 00960000 [disabled] [size=128K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable- Count=10 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00002000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
		LnkCap:	Port #4, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 14, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number 00-21-28-ff-ff-f9-f6-4e
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 1
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy+
		IOVSta:	Migration-
		Initial VFs: 8, Total VFs: 8, Number of VFs: 8, Function Dependency Link: 00
		VF offset: 128, stride: 2, Device ID: 10ca
		Supported Page Size: 00000553, System Page Size: 00000002
		Region 0: Memory at 00980000 (32-bit, non-prefetchable)
		Region 3: Memory at 009a0000 (32-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Kernel driver in use: igb
00: 86 80 c9 10 46 01 10 00 01 00 00 02 10 00 80 00
10: 00 00 90 00 00 00 92 00 01 10 00 00 00 00 94 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 8e 10 48 48
30: 00 00 96 00 40 00 00 00 00 00 00 00 00 01 00 00
40: 01 50 23 c8 00 20 00 1a 00 00 00 00 00 00 00 00
50: 05 70 80 01 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 11 a0 09 00 03 00 00 00 03 20 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 10 00 02 00 c2 8c 00 10 10 28 19 00 41 6c 03 04
b0: 00 00 41 10 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 1f 00 00 00 05 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:05:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
	Subsystem: Oracle/SUN Device 4848
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 00000005
	Region 0: Memory at 009c0000 (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at 009e0000 (32-bit, non-prefetchable) [size=128K]
	Region 2: I/O ports at 00001020 [disabled] [size=32]
	Region 3: Memory at 00944000 (32-bit, non-prefetchable) [size=16K]
	Expansion ROM at 00a00000 [disabled] [size=128K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable- Count=10 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00002000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
		LnkCap:	Port #4, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 14, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number 00-21-28-ff-ff-f9-f6-4e
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy-
		IOVSta:	Migration-
		Initial VFs: 8, Total VFs: 8, Number of VFs: 8, Function Dependency Link: 01
		VF offset: 128, stride: 2, Device ID: 10ca
		Supported Page Size: 00000553, System Page Size: 00000002
		Region 0: Memory at 00a20000 (32-bit, non-prefetchable)
		Region 3: Memory at 00a40000 (32-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Kernel driver in use: igb
00: 86 80 c9 10 46 01 10 00 01 00 00 02 10 00 80 00
10: 00 00 9c 00 00 00 9e 00 21 10 00 00 00 40 94 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 8e 10 48 48
30: 00 00 a0 00 40 00 00 00 00 00 00 00 00 02 00 00
40: 01 50 23 c8 00 20 00 1a 00 00 00 00 00 00 00 00
50: 05 70 80 01 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 11 a0 09 00 03 00 00 00 03 20 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 10 00 02 00 c2 8c 00 10 10 28 19 00 41 6c 03 04
b0: 00 00 41 10 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 1f 00 00 00 05 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:07:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=07, secondary=08, subordinate=08, sec-latency=0
	Memory behind bridge: 00b00000-00bfffff
	Prefetchable memory behind bridge: 00000000-ffffffff
	Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [60] Express (v1) PCI/PCI-X Bridge, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset-
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- BrConfRtry-
			MaxPayload 128 bytes, MaxReadReq 4096 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Latency L0 <1us, L1 <16us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
	Capabilities: [100 v1] Power Budgeting <?>
00: b5 10 12 81 07 00 10 00 aa 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 07 08 08 00 f0 00 20 02
20: b0 00 b0 00 00 00 f0 ff 00 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 01 00 00
40: 01 50 02 5a 00 00 00 00 11 20 0e 00 00 00 00 00
50: 05 60 80 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 10 00 71 00 20 00 00 00 07 50 00 00 11 4c 02 00
70: 00 00 11 00 80 0c 00 00 00 00 40 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 33 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:08:00.0 USB controller: NEC Corporation OHCI USB Controller (rev 43) (prog-if 10 [OHCI])
	Subsystem: NEC Corporation USB Controller
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 64 (250ns min, 10500ns max), Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 00000006
	Region 0: Memory at 00b00000 (32-bit, non-prefetchable) [size=8K]
	Capabilities: [40] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: ohci-pci
00: 33 10 35 00 06 00 10 02 43 10 03 0c 10 40 80 00
10: 00 00 b0 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 33 10 35 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 01 01 2a
40: 01 00 02 7e 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 05 33 b0 6c 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:08:00.1 USB controller: NEC Corporation OHCI USB Controller (rev 43) (prog-if 10 [OHCI])
	Subsystem: NEC Corporation USB Controller
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 64 (250ns min, 10500ns max), Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 00000004
	Region 0: Memory at 00b02000 (32-bit, non-prefetchable) [size=8K]
	Capabilities: [40] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: ohci-pci
00: 33 10 35 00 06 00 10 02 43 10 03 0c 10 40 00 00
10: 00 20 b0 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 33 10 35 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 02 01 2a
40: 01 00 02 7e 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:08:00.2 USB controller: NEC Corporation uPD72010x USB 2.0 Controller (rev 04) (prog-if 20 [EHCI])
	Subsystem: NEC Corporation uPD72010x USB 2.0 Controller
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 64 (4000ns min, 8500ns max), Cache Line Size: 64 bytes
	Interrupt: pin C routed to IRQ 00000005
	Region 0: Memory at 00b04000 (32-bit, non-prefetchable) [size=8K]
	Capabilities: [40] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: ehci-pci
00: 33 10 e0 00 16 00 10 02 04 20 03 0c 10 40 00 00
10: 00 40 b0 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 33 10 e0 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 03 10 22
40: 01 00 02 7e 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 20 20 3f 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 05 33 b0 6c 00 00 00 00 01 00 00 00 00 00 00 c0
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:0a:00.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=0a, secondary=0b, subordinate=0f, sec-latency=0
	I/O behind bridge: 00002000-00003fff
	Memory behind bridge: 00c00000-00dfffff
	Prefetchable memory behind bridge: 0000004100000000-000000411fffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Upstream Port, MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] #00 [0000]
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 0a 0b 0f 00 21 31 00 00
20: c0 00 d0 00 01 00 f1 1f 41 00 00 00 41 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 52 00 24 80 00 00 00 00 00 00 82 6c 01 00
50: 00 00 82 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0000:0b:00.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Physical Slot: 0
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=0b, secondary=0c, subordinate=0c, sec-latency=0
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot+), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #12, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #4, PowerLimit 0.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 0b 0c 0c 00 f1 01 00 00
20: f0 ff 00 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: ff ff 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 01 24 80 00 00 00 00 00 00 42 6c 39 0c
50: 00 00 01 00 00 00 20 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0000:0b:04.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Physical Slot: 4
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=0b, secondary=0d, subordinate=0d, sec-latency=0
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot+), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #4, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #2, PowerLimit 0.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 0b 0d 0d 00 f1 01 00 00
20: f0 ff 00 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: ff ff 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 01 24 80 00 00 00 00 00 00 82 6c 39 04
50: 00 00 01 00 00 00 10 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0000:0b:08.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=0b, secondary=0e, subordinate=0e, sec-latency=0
	I/O behind bridge: 00002000-00002fff
	Memory behind bridge: 00c00000-00cfffff
	Prefetchable memory behind bridge: 0000004100000000-000000410fffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot+), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #8, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk- DLActive+ BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #0, PowerLimit 0.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState+
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 0b 0e 0e 00 21 21 00 00
20: c0 00 c0 00 01 00 f1 0f 41 00 00 00 41 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 01 24 80 00 00 00 00 00 00 82 6c 39 08
50: 00 00 81 20 00 00 00 00 00 00 40 01 00 00 00 00
60: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0000:0b:0e.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=0b, secondary=0f, subordinate=0f, sec-latency=0
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: 00d00000-00dfffff
	Prefetchable memory behind bridge: 0000004110000000-000000411fffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot-), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #14, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk- DLActive+ BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd+
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 0b 0f 0f 00 31 31 00 00
20: d0 00 d0 00 01 10 f1 1f 41 00 00 00 41 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 00 24 80 00 00 00 00 00 00 82 6c 39 0e
50: 00 00 82 20 00 00 00 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 20 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0000:0e:00.0 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter (rev 03)
	Subsystem: Emulex Corporation Device fc42
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 00000005
	Region 0: Memory at 4100000000 (64-bit, non-prefetchable) [size=8K]
	Region 2: Memory at 4100004000 (64-bit, non-prefetchable) [size=16K]
	Region 4: I/O ports at 00002000 [disabled] [size=256]
	Expansion ROM at 00c00000 [disabled] [size=256K]
	Capabilities: [58] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [60] MSI: Enable- Count=1/16 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [78] MSI-X: Enable- Count=32 Masked-
		Vector table: BAR=2 offset=00002000
		PBA: BAR=2 offset=00003000
	Capabilities: [84] Vital Product Data
		Product Name: Sun StorageTek 8Gb FC PCIe HBA, dual port, Emulex
		Read-only fields:
			[PN] Part number: 371-4306-01
			[SN] Serial number: 0999VM0-12120020F1
			[V0] Vendor specific: VM21180036
			[EC] Engineering changes: R50
			[MN] Manufacture ID: 53 55 4e 20 4d 49 43 52 4f 53 59 53 54 45 4d 53 2c 20 49 4e 43 2e
			[V1] Vendor specific: Sun StorageTek LPe12002-S 8Gb 2-port PCIe FC HBA
			[V2] Vendor specific: LPe12002-S
			[V3] Vendor specific: T2:78,T3:7C,T7:73,TB:73,TFF:78
			[V4] Vendor specific: 0
			[RV] Reserved: checksum good, 16 byte(s) reserved
		End
	Capabilities: [94] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <1us, L1 unlimited
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x8, ASPM L0s, Latency L0 <1us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [12c v1] Power Budgeting <?>
	Kernel driver in use: lpfc
00: df 10 40 fc 46 00 10 00 03 00 04 0c 10 00 80 00
10: 04 00 00 00 41 00 00 00 04 40 00 00 41 00 00 00
20: 01 20 00 00 00 00 00 00 00 00 00 00 df 10 42 fc
30: 00 00 c0 00 58 00 00 00 00 00 00 00 ff 01 00 00
40: 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 01 60 03 00 00 00 00 00
60: 05 78 88 01 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 11 84 1f 00 02 20 00 00
80: 02 30 00 00 03 94 fc 80 00 00 00 78 00 00 00 00
90: 00 00 00 00 10 00 02 00 24 8f 00 00 10 28 09 00
a0: 82 44 03 00 00 00 81 10 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 1f 00 00 00 00 00 00 00
c0: 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:0e:00.1 Fibre Channel: Emulex Corporation Saturn-X: LightPulse Fibre Channel Host Adapter (rev 03)
	Subsystem: Emulex Corporation Device fc42
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 00000007
	Region 0: Memory at 4100002000 (64-bit, non-prefetchable) [size=8K]
	Region 2: Memory at 4100008000 (64-bit, non-prefetchable) [size=16K]
	Region 4: I/O ports at 00002100 [disabled] [size=256]
	Expansion ROM at 00c40000 [disabled] [size=256K]
	Capabilities: [58] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [60] MSI: Enable- Count=1/16 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [78] MSI-X: Enable- Count=32 Masked-
		Vector table: BAR=2 offset=00002000
		PBA: BAR=2 offset=00003000
	Capabilities: [84] Vital Product Data
		Product Name: Sun StorageTek 8Gb FC PCIe HBA, dual port, Emulex
		Read-only fields:
			[PN] Part number: 371-4306-01
			[SN] Serial number: 0999VM0-12120020F1
			[V0] Vendor specific: VM21180036
			[EC] Engineering changes: R50
			[MN] Manufacture ID: 53 55 4e 20 4d 49 43 52 4f 53 59 53 54 45 4d 53 2c 20 49 4e 43 2e
			[V1] Vendor specific: Sun StorageTek LPe12002-S 8Gb 2-port PCIe FC HBA
			[V2] Vendor specific: LPe12002-S
			[V3] Vendor specific: T2:78,T3:7C,T7:73,TB:73,TFF:78
			[V4] Vendor specific: 1
			[RV] Reserved: checksum good, 16 byte(s) reserved
		End
	Capabilities: [94] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <1us, L1 unlimited
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x8, ASPM L0s, Latency L0 <1us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [12c v1] Power Budgeting <?>
	Kernel driver in use: lpfc
00: df 10 40 fc 46 00 10 00 03 00 04 0c 10 00 80 00
10: 04 20 00 00 41 00 00 00 04 80 00 00 41 00 00 00
20: 01 21 00 00 00 00 00 00 00 00 00 00 df 10 42 fc
30: 00 00 c4 00 58 00 00 00 00 00 00 00 ff 02 00 00
40: 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 01 60 03 00 00 00 00 00
60: 05 78 88 01 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 11 84 1f 00 02 20 00 00
80: 02 30 00 00 03 94 fc 80 00 00 00 78 00 00 00 00
90: 00 00 00 00 10 00 02 00 24 8f 00 00 10 28 09 00
a0: 82 44 03 00 00 00 81 10 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 1f 00 00 00 00 00 00 00
c0: 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0000:0f:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
	Subsystem: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 512 bytes
	Interrupt: pin A routed to IRQ 00000006
	Region 0: I/O ports at 00003000 [disabled] [size=256]
	Region 1: Memory at 4110000000 (64-bit, non-prefetchable) [size=16K]
	Region 3: Memory at 4110040000 (64-bit, non-prefetchable) [size=256K]
	Expansion ROM at 00d00000 [disabled] [size=512K]
	Capabilities: [50] Power Management version 3
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [68] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 4096 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x8, ASPM L0s, Latency L0 <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range BC, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [d0] Vital Product Data
		Unknown small resource type 00, will not decode more.
	Capabilities: [a8] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [c0] MSI-X: Enable- Count=15 Masked-
		Vector table: BAR=1 offset=00002000
		PBA: BAR=1 offset=00003800
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr+ BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [138 v1] Power Budgeting <?>
	Capabilities: [150 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy+
		IOVSta:	Migration-
		Initial VFs: 16, Total VFs: 16, Number of VFs: 16, Function Dependency Link: 00
		VF offset: 1, stride: 1, Device ID: 0072
		Supported Page Size: 00000553, System Page Size: 00000002
		Region 0: Memory at 0000004110080000 (64-bit, non-prefetchable)
		Region 2: Memory at 00000041100c0000 (64-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Capabilities: [190 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Kernel driver in use: mpt2sas
00: 00 10 72 00 06 00 10 00 03 00 07 01 80 00 00 00
10: 01 30 00 00 04 00 00 10 41 00 00 00 04 00 04 10
20: 41 00 00 00 00 00 00 00 00 00 00 00 00 10 72 00
30: 00 00 d0 00 50 00 00 00 00 00 00 00 00 01 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 01 68 03 06 08 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 0f 00 00 10 d0 02 00 25 80 00 10
70: 10 28 09 00 82 04 00 00 00 00 82 10 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 16 00 00 00
90: 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 05 c0 80 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 11 00 0e 00 01 20 00 00 01 38 00 00 00 00 00 00
d0: 03 a8 00 80 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0001:00:01.0 PCI bridge: Oracle/SUN Device 8186 (rev 01) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Bus: primary=00, secondary=01, subordinate=05, sec-latency=0
	I/O behind bridge: 00000000-00000fff
	Memory behind bridge: 00000000-001fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [60] Express (v2) Root Port (Slot-), MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x8, ASPM unknown, Latency L0 <2us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 05, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [138 v1] Vendor Specific Information: ID=1112 Rev=1 Len=014 <?>
	Capabilities: [14c v1] #00
00: 8e 10 86 81 07 00 10 00 01 00 04 06 00 00 01 00
10: 00 00 00 00 00 00 00 00 00 01 05 00 01 01 00 00
20: 00 00 10 00 f1 ff 01 00 4d 00 00 00 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 01 60 03 c8 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 10 00 42 00 01 80 00 00 00 00 00 00 82 50 39 00
70: 00 00 82 70 00 00 00 00 00 00 00 01 00 00 00 00
80: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
90: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0001:00:02.0 PCI bridge: Oracle/SUN Device 8186 (rev 01) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Bus: primary=00, secondary=06, subordinate=0a, sec-latency=0
	I/O behind bridge: 00001000-00001fff
	Memory behind bridge: 00200000-002fffff
	Prefetchable memory behind bridge: 0000004100000000-000000410fffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [60] Express (v2) Root Port (Slot-), MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #1, Speed 5GT/s, Width x8, ASPM unknown, Latency L0 <2us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 05, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [138 v1] Vendor Specific Information: ID=1112 Rev=1 Len=014 <?>
	Capabilities: [14c v1] #00
00: 8e 10 86 81 07 00 10 00 01 00 04 06 00 00 01 00
10: 00 00 00 00 00 00 00 00 00 06 0a 00 11 11 00 00
20: 20 00 20 00 01 00 f1 0f 41 00 00 00 41 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 01 60 03 c8 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 10 00 42 00 01 80 00 00 00 00 00 00 82 50 39 01
70: 00 00 82 70 00 00 00 00 00 00 00 01 00 00 00 00
80: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
90: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0001:01:00.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=01, secondary=02, subordinate=05, sec-latency=0
	I/O behind bridge: 00000000-00000fff
	Memory behind bridge: 00000000-001fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Upstream Port, MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #2, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] #00 [0000]
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 01 02 05 00 01 01 00 00
20: 00 00 10 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 52 00 24 80 00 00 00 00 00 00 82 6c 01 02
50: 00 00 82 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0001:02:00.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=02, secondary=03, subordinate=03, sec-latency=0
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot+), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #14, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #9, PowerLimit 0.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 02 03 03 00 f1 01 00 00
20: f0 ff 00 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: ff ff 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 01 24 80 00 00 00 00 00 00 82 6c 39 0e
50: 00 00 01 00 00 00 48 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0001:02:05.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=02, secondary=04, subordinate=04, sec-latency=0
	I/O behind bridge: 00000000-00000fff
	Memory behind bridge: 00000000-001fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot-), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #5, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk- DLActive+ BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd+
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 02 04 04 00 01 01 00 00
20: 00 00 10 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 00 24 80 00 00 00 00 00 00 42 6c 39 05
50: 00 00 41 20 00 00 00 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 20 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0001:02:06.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=02, secondary=05, subordinate=05, sec-latency=0
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot+), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #6, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #7, PowerLimit 0.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 02 05 05 00 f1 01 00 00
20: f0 ff 00 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: ff ff 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 01 24 80 00 00 00 00 00 00 82 6c 39 06
50: 00 00 01 00 00 00 38 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0001:04:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
	Subsystem: Oracle/SUN Device 4848
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 0000000a
	Region 0: Memory at <unassigned> (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at 00020000 (32-bit, non-prefetchable) [size=128K]
	Region 2: I/O ports at <unassigned> [disabled] [size=32]
	Region 3: Memory at 00040000 (32-bit, non-prefetchable) [size=16K]
	Expansion ROM at 00060000 [disabled] [size=128K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable- Count=10 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00002000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
		LnkCap:	Port #5, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 14, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number 00-21-28-ff-ff-f9-f6-50
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 1
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy+
		IOVSta:	Migration-
		Initial VFs: 8, Total VFs: 8, Number of VFs: 8, Function Dependency Link: 00
		VF offset: 128, stride: 2, Device ID: 10ca
		Supported Page Size: 00000553, System Page Size: 00000002
		Region 0: Memory at 00080000 (32-bit, non-prefetchable)
		Region 3: Memory at 000a0000 (32-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Kernel driver in use: igb
00: 86 80 c9 10 46 01 10 00 01 00 00 02 10 00 80 00
10: 00 00 00 00 00 00 02 00 01 00 00 00 00 00 04 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 8e 10 48 48
30: 00 00 06 00 40 00 00 00 00 00 00 00 00 01 00 00
40: 01 50 23 c8 00 20 00 1a 00 00 00 00 00 00 00 00
50: 05 70 80 01 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 11 a0 09 00 03 00 00 00 03 20 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 10 00 02 00 c2 8c 00 10 10 28 19 00 41 6c 03 05
b0: 00 00 41 10 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 1f 00 00 00 05 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0001:04:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
	Subsystem: Oracle/SUN Device 4848
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 0000000b
	Region 0: Memory at 000c0000 (32-bit, non-prefetchable) [size=128K]
	Region 1: Memory at 000e0000 (32-bit, non-prefetchable) [size=128K]
	Region 2: I/O ports at 00000020 [disabled] [size=32]
	Region 3: Memory at 00044000 (32-bit, non-prefetchable) [size=16K]
	Expansion ROM at 00100000 [disabled] [size=128K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable- Count=10 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00002000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
		LnkCap:	Port #5, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <64us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 16ms to 55ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 14, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [140 v1] Device Serial Number 00-21-28-ff-ff-f9-f6-50
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy-
		IOVSta:	Migration-
		Initial VFs: 8, Total VFs: 8, Number of VFs: 8, Function Dependency Link: 01
		VF offset: 128, stride: 2, Device ID: 10ca
		Supported Page Size: 00000553, System Page Size: 00000002
		Region 0: Memory at 00120000 (32-bit, non-prefetchable)
		Region 3: Memory at 00140000 (32-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Kernel driver in use: igb
00: 86 80 c9 10 46 01 10 00 01 00 00 02 10 00 80 00
10: 00 00 0c 00 00 00 0e 00 21 00 00 00 00 40 04 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 8e 10 48 48
30: 00 00 10 00 40 00 00 00 00 00 00 00 00 02 00 00
40: 01 50 23 c8 00 20 00 1a 00 00 00 00 00 00 00 00
50: 05 70 80 01 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 11 a0 09 00 03 00 00 00 03 20 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 10 00 02 00 c2 8c 00 10 10 28 19 00 41 6c 03 05
b0: 00 00 41 10 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 1f 00 00 00 05 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0001:06:00.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=06, secondary=07, subordinate=0a, sec-latency=0
	I/O behind bridge: 00001000-00001fff
	Memory behind bridge: 00200000-002fffff
	Prefetchable memory behind bridge: 0000004100000000-000000410fffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Upstream Port, MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #2, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] #00 [0000]
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 06 07 0a 00 11 11 00 00
20: 20 00 20 00 01 00 f1 0f 41 00 00 00 41 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 52 00 24 80 00 00 00 00 00 00 82 6c 01 02
50: 00 00 82 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0001:07:00.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=07, secondary=08, subordinate=08, sec-latency=0
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot+), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #13, Speed 5GT/s, Width x4, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #5, PowerLimit 0.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 07 08 08 00 f1 01 00 00
20: f0 ff 00 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: ff ff 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 01 24 80 00 00 00 00 00 00 42 6c 39 0d
50: 00 00 01 00 00 00 28 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0001:07:06.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=07, secondary=09, subordinate=09, sec-latency=0
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot+), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #6, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #3, PowerLimit 0.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 07 09 09 00 f1 01 00 00
20: f0 ff 00 00 f1 ff 01 00 ff ff ff ff 00 00 00 00
30: ff ff 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 01 24 80 00 00 00 00 00 00 82 6c 39 06
50: 00 00 01 00 00 00 18 00 00 00 40 00 00 00 00 00
60: 00 00 00 00 20 00 00 00 00 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0001:07:0a.0 PCI bridge: Integrated Device Technology, Inc. [IDT] Device 8077 (rev 02) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=07, secondary=0a, subordinate=0a, sec-latency=0
	I/O behind bridge: 00001000-00001fff
	Memory behind bridge: 00200000-002fffff
	Prefetchable memory behind bridge: 0000004100000000-000000410fffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Downstream Port (Slot+), MSI 00
		DevCap:	MaxPayload 2048 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #10, Speed 5GT/s, Width x8, ASPM L0s L1, Latency L0 <4us, L1 <4us
			ClockPM- Surprise+ LLActRep+ BwNot+
		LnkCtl:	ASPM Disabled; Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk- DLActive+ BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #1, PowerLimit 0.000W; Interlock- NoCompl-
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState+
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd+
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [c0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [200 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=4
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=04 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
			Port Arbitration Table <?>
	Capabilities: [320 v1] #00
	Capabilities: [330 v1] #12
00: 1d 11 77 80 07 00 10 00 02 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 07 0a 0a 00 11 11 00 00
20: 20 00 20 00 01 00 f1 0f 41 00 00 00 41 00 00 00
30: 00 00 00 00 40 00 00 00 00 00 00 00 00 00 00 00
40: 10 c0 62 01 24 80 00 00 00 00 00 00 82 6c 39 0a
50: 00 00 82 20 00 00 08 00 00 00 40 01 00 00 00 00
60: 00 00 00 00 20 00 00 00 20 00 00 00 00 00 00 00
70: 42 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 01 d0 03 c8 08 00 00 00 00 00 00 00 00 00 00 00
d0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 0d 00 00 00 00 00 00 00 00 00 00 00 1d 11 77 80

0001:0a:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
	Subsystem: Oracle/SUN Ethernet Server Adapter X520-2
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 0000000c
	Region 0: Memory at 4100000000 (64-bit, non-prefetchable) [size=512K]
	Region 2: I/O ports at 00001000 [disabled] [size=32]
	Region 4: Memory at 4100080000 (64-bit, non-prefetchable) [size=16K]
	Expansion ROM at 00200000 [disabled] [size=512K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable- Count=64 Masked-
		Vector table: BAR=4 offset=00000000
		PBA: BAR=4 offset=00002000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #10, Speed 5GT/s, Width x8, ASPM L0s, Latency L0 unlimited, L1 <32us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [140 v1] Device Serial Number 90-e2-ba-ff-ff-14-31-88
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 1
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy+
		IOVSta:	Migration-
		Initial VFs: 64, Total VFs: 64, Number of VFs: 64, Function Dependency Link: 00
		VF offset: 128, stride: 2, Device ID: 10ed
		Supported Page Size: 00000553, System Page Size: 00000002
		Region 0: Memory at 0000004100084000 (64-bit, non-prefetchable)
		Region 3: Memory at 0000004100184000 (64-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Kernel driver in use: ixgbe
00: 86 80 fb 10 46 01 10 00 01 00 00 02 10 00 80 00
10: 04 00 00 00 41 00 00 00 01 10 00 00 00 00 00 00
20: 04 00 08 00 41 00 00 00 00 00 00 00 8e 10 11 7b
30: 00 00 20 00 40 00 00 00 00 00 00 00 00 01 00 00
40: 01 50 23 48 00 20 00 36 00 00 00 00 00 00 00 00
50: 05 70 80 01 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 11 a0 3f 00 04 00 00 00 04 20 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 10 00 02 00 c2 8c 00 10 10 28 09 00 82 f4 02 0a
b0: 00 00 82 10 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 1f 00 00 00 00 00 00 00 00 00 00 00
d0: 02 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

0001:0a:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
	Subsystem: Oracle/SUN Ethernet Server Adapter X520-2
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 0000000d
	Region 0: Memory at 4100300000 (64-bit, non-prefetchable) [size=512K]
	Region 2: I/O ports at 00001020 [disabled] [size=32]
	Region 4: Memory at 4100284000 (64-bit, non-prefetchable) [size=16K]
	Expansion ROM at 00280000 [disabled] [size=512K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable- Count=64 Masked-
		Vector table: BAR=4 offset=00000000
		PBA: BAR=4 offset=00002000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #10, Speed 5GT/s, Width x8, ASPM L0s, Latency L0 unlimited, L1 <32us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [140 v1] Device Serial Number 90-e2-ba-ff-ff-14-31-88
	Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy-
		IOVSta:	Migration-
		Initial VFs: 64, Total VFs: 64, Number of VFs: 64, Function Dependency Link: 01
		VF offset: 128, stride: 2, Device ID: 10ed
		Supported Page Size: 00000553, System Page Size: 00000002
		Region 0: Memory at 0000004100380000 (64-bit, non-prefetchable)
		Region 3: Memory at 0000004100480000 (64-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Kernel driver in use: ixgbe
00: 86 80 fb 10 46 01 10 00 01 00 00 02 10 00 80 00
10: 04 00 30 00 41 00 00 00 21 10 00 00 00 00 00 00
20: 04 40 28 00 41 00 00 00 00 00 00 00 8e 10 11 7b
30: 00 00 28 00 40 00 00 00 00 00 00 00 00 02 00 00
40: 01 50 23 48 00 20 00 36 00 00 00 00 00 00 00 00
50: 05 70 80 01 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 11 a0 3f 00 04 00 00 00 04 20 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 10 00 02 00 c2 8c 00 10 10 28 09 00 82 f4 02 0a
b0: 00 00 82 10 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 1f 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Re: [PATCH v8 00/61] PCI: Resource allocation cleanup for v4.4

[Yinghai Lu]

On Fri, Oct 30, 2015 at 2:47 PM, Khalid Aziz <khalid.aziz@oracle.com> wrote:
>
> I applied patches 1-8 and 11-16 to 4.3.0-rc7 to test on sparc platforms. I
> am seeing a "can't claim BAR" message on a T4:
>
> pci 0000:04:00.0: can't claim BAR 6 [mem 0x84000000000-0x8400000ffff]:
> address conflict with 0000:04:00.0 [mem 0x84000000000-0x840007fffff]

PCI: scan_bus[/pci@400/pci@1/pci@0/pci@0/pci@0] bus no 4
  * /pci@400/pci@1/pci@0/pci@0/pci@0/display@0
    create device, devfn: 0, type: display
    class: 0x30000 device name: 0000:04:00.0
    parse addresses (80 bytes) @ fff8001fffe34f40
  start: 84000000000, end: 840007fffff, i: 10
  start: 84000800000, end: 8400081ffff, i: 14
  start: 85100000000, end: 8510000007f, i: 18
  start: 84000000000, end: 8400000ffff, i: 30

so the BAR 6 the ROM bar is overlapping with BAR0.
and the ROM bar get rejected.

Looks like only the device BAR6 is not assigned form OF?

Thanks

Yinghai

Re: [PATCH v8 00/61] PCI: Resource allocation cleanup for v4.4

[David Miller]

From: Yinghai Lu <yinghai@kernel.org>
Date: Fri, 30 Oct 2015 18:53:23 -0700

> On Fri, Oct 30, 2015 at 2:47 PM, Khalid Aziz <khalid.aziz@oracle.com> wrote:
>>
>> I applied patches 1-8 and 11-16 to 4.3.0-rc7 to test on sparc platforms. I
>> am seeing a "can't claim BAR" message on a T4:
>>
>> pci 0000:04:00.0: can't claim BAR 6 [mem 0x84000000000-0x8400000ffff]:
>> address conflict with 0000:04:00.0 [mem 0x84000000000-0x840007fffff]
> 
> PCI: scan_bus[/pci@400/pci@1/pci@0/pci@0/pci@0] bus no 4
>   * /pci@400/pci@1/pci@0/pci@0/pci@0/display@0
>     create device, devfn: 0, type: display
>     class: 0x30000 device name: 0000:04:00.0
>     parse addresses (80 bytes) @ fff8001fffe34f40
>   start: 84000000000, end: 840007fffff, i: 10
>   start: 84000800000, end: 8400081ffff, i: 14
>   start: 85100000000, end: 8510000007f, i: 18
>   start: 84000000000, end: 8400000ffff, i: 30
> 
> so the BAR 6 the ROM bar is overlapping with BAR0.
> and the ROM bar get rejected.

This is the way OF seems to work.

It maps all of the ROMs essentially to the same address range, but
only enables one at a time as it inspects the ROMs and builds the
device tree during power-on.

Then it makes sure all of them are disabled, and can therefore use
some of that address range for mapping other BARs.

So if ROMs are disabled, you cannot put the address of such ROMs BAR
in the kernel's assigned PCI address resource list.

I hope that is what you are doing?

Re: [PATCH v8 00/61] PCI: Resource allocation cleanup for v4.4

[Benjamin Herrenschmidt]

On Sat, 2015-10-31 at 14:51 -0400, David Miller wrote:
> This is the way OF seems to work.
> 
> It maps all of the ROMs essentially to the same address range, but
> only enables one at a time as it inspects the ROMs and builds the
> device tree during power-on.
> 
> Then it makes sure all of them are disabled, and can therefore use
> some of that address range for mapping other BARs.
> 
> So if ROMs are disabled, you cannot put the address of such ROMs BAR
> in the kernel's assigned PCI address resource list.
> 
> I hope that is what you are doing?

I've seen that sort of stuff on x86 as well. Possibly on POWER, I don't
remember for sure.

I've also seen the VBIOS  of some cards manually remap the ROM BAR to
cover BAR 0, extract stuff from it, then disable it.

I think the ROM BAR must be treated as "special". It should probably
done in a separate pass from all the other BARs of all the other
adapters to be honest, and remapped if there's any conflict.

Cheers,
Ben

Re: [PATCH v8 00/61] PCI: Resource allocation cleanup for v4.4

[Khalid Aziz]

On Fri, 2015-10-30 at 18:53 -0700, Yinghai Lu wrote:
> On Fri, Oct 30, 2015 at 2:47 PM, Khalid Aziz <khalid.aziz@oracle.com> wrote:
> >
> > I applied patches 1-8 and 11-16 to 4.3.0-rc7 to test on sparc platforms. I
> > am seeing a "can't claim BAR" message on a T4:
> >
> > pci 0000:04:00.0: can't claim BAR 6 [mem 0x84000000000-0x8400000ffff]:
> > address conflict with 0000:04:00.0 [mem 0x84000000000-0x840007fffff]
> 
> PCI: scan_bus[/pci@400/pci@1/pci@0/pci@0/pci@0] bus no 4
>   * /pci@400/pci@1/pci@0/pci@0/pci@0/display@0
>     create device, devfn: 0, type: display
>     class: 0x30000 device name: 0000:04:00.0
>     parse addresses (80 bytes) @ fff8001fffe34f40
>   start: 84000000000, end: 840007fffff, i: 10
>   start: 84000800000, end: 8400081ffff, i: 14
>   start: 85100000000, end: 8510000007f, i: 18
>   start: 84000000000, end: 8400000ffff, i: 30
> 
> so the BAR 6 the ROM bar is overlapping with BAR0.
> and the ROM bar get rejected.
> 
> Looks like only the device BAR6 is not assigned form OF?
> 

Hi Yinghai,

Any progress on this? Do you need any more information from me?

Thanks,
Khalid

Re: [PATCH v8 00/61] PCI: Resource allocation cleanup for v4.4

[Yinghai Lu]

On Mon, Nov 23, 2015 at 1:31 PM, Khalid Aziz <khalid.aziz@oracle.com> wrote:
>> Looks like only the device BAR6 is not assigned form OF?
>
> Any progress on this? Do you need any more information from me?

Should be one warning about this device in that system.

We don't need to do anything about it if the device is working with
this patch set.

Thanks

Yinghai

Re: [PATCH v8 00/61] PCI: Resource allocation cleanup for v4.4

[Khalid Aziz]

On 11/23/2015 07:39 PM, Yinghai Lu wrote:
> On Mon, Nov 23, 2015 at 1:31 PM, Khalid Aziz <khalid.aziz@oracle.com> wrote:
>>> Looks like only the device BAR6 is not assigned form OF?
>>
>> Any progress on this? Do you need any more information from me?
>
> Should be one warning about this device in that system.
>
> We don't need to do anything about it if the device is working with
> this patch set.

Hi Yinghai,

Picking this up again after thanksgiving.......

The device does work with this patch. Doesn't this warning mean BAR6 
resources could not be reserved which means the device driver may not be 
able to get the BAR6 resource at some point if another device grabs the 
unreserved resource?

David M had pointed out an issue with how OF driver uses the same 
address range for all ROMs enabling them only one at a time. That seems 
to be in play here in the issue I am seeing. Does this patch series need 
to address this?

Thanks,
Khalid

Re: [PATCH v8 00/61] PCI: Resource allocation cleanup for v4.4

[Yinghai Lu]

On Tue, Dec 1, 2015 at 1:23 PM, Khalid Aziz <khalid.aziz@oracle.com> wrote:
> The device does work with this patch.

Good.

> Doesn't this warning mean BAR6
> resources could not be reserved which means the device driver may not be
> able to get the BAR6 resource at some point if another device grabs the
> unreserved resource?

Current sparc pci only have checking, does not have assign unassigned
bar from firmware.

because firmware does not assign resource to that BAR6, the driver may
not need that bar at all.

>
> David M had pointed out an issue with how OF driver uses the same address
> range for all ROMs enabling them only one at a time. That seems to be in
> play here in the issue I am seeing. Does this patch series need to address
> this?

That trick may have problem when we have devices under bridge. as bridge
window need to be setup at first.

If the driver really need to use that trick, it should borrow from BAR0 (MEM)
instead.

Thanks

Yinghai