[PATCH v3 00/51] PCI: Resource allocation cleanup for v4.3

[PATCH v3 00/51] PCI: Resource allocation cleanup for v4.3

[Yinghai Lu]

Hi,

After 5b28541552ef (PCI: Restrict 64-bit prefetchable bridge windows
to 64-bit resources), we have several reports on resource allocation
failure, and we try to fix the problem with resource clip, and get
more problems.

One is realloc fail with two graphics cards above 4G.
One is from sparc that have problem with clip as we don't parse
mem64 for it.

Other report is about pci remove/rescan does not work on some setup
when BIOS tend to allocate small bus size.

This patchset enhance resource allocation to address those problems.
1. optimize bus mmio alignment calculation.
2. optimize bus mmio optional alignment calculation.
3. add support for alt size to prefer small bus size to small bus alignment.
   when we have small resource window on parent bridges.
4. treat ROM bar as optional resource.
5. during allocation, will pick up just fit resource.
6. parse MEM64 for sparc and other system with OF.
7. treat non-pref mmio64 if parent bridges are all pcie.
8. restore old pref allocation logic if hostbridge does not support mmio64 really.
9. don't realloc resource if device firmware does not support bar change.
10. add pci=assign_pref_bars to clear and assign pref bars.
11. don't clear resource when allocation fails.

I put latest copy at:
  git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git for-pci-v4.3-next

That is based on kind of v4.2-rc4.

Yijing and Wei Yang both test the branch, and the patchset fixs rescan problem,
and is working well on powerpc setup.

Thanks

Yinghai

v2:
    - fix resource_disabled allocation from powerpc
    - other warnings that were found by Fengguang's build robot.
    - io port alignment clean up.
    - rebased to v4.2-rc2
    - better for two alt_size support.

v3:
    - fix reset bridge resource on last try: only with bridge resources
    - remove/rescan path checking with old_size.


Yinghai Lu (51):
  PCI: Cleanup res_to_dev_res() printout for addon resources
  PCI: Reuse res_to_dev_res in reassign_resources_sorted
  PCI: Use correct align for optional only resources during sorting
  PCI: Optimize bus align/size calculation during sizing
  PCI: Optimize bus align/size calculation for optional during sizing
  PCI: Don't add too much optional size for hotplug bridge mmio
  PCI: Reorder resources list for must/optional resources
  PCI: Remove duplicated code for resource sorting
  PCI: Rename pdev_sort_resources to pdev_check_resources
  PCI: Treat ROM resource as optional during realloc
  PCI: Add debug printout during releasing partial assigned resources
  PCI: Simplify res reference using in __assign_resourcs_sorted
  PCI: Separate realloc list checking after allocation
  PCI: Add __add_to_list()
  PCI: Cache window alignment value
  PCI: Check if resource is allocated before pci_assign
  PCI: Separate out save_resources/restore_resource
  PCI: Move comment to pci_need_to_release()
  PCI: Separate must+optional assigning to another function
  PCI: Skip must+optional if there is no optional addon
  PCI: Move saved required resource list out of must+optional assigning
  PCI: Add alt_size allocation support
  PCI: Add support for more than two alt_size under same bridge
  PCI: Better support for two alt_size
  PCI: Fix size calculation with old_size on rescan path
  PCI: Don't add too much optional size for hotplug bridge io
  PCI: Move ISA ioport align out of calculate_iosize
  PCI: Unifiy calculate_size for io port and mmio
  PCI: Allow optional only io resource must size to be 0
  PCI: Unify skip_ioresource_align()
  PCI: Kill macro checking for bus io port sizing
  resources: Split out __allocate_resource()
  resources: Make allocate_resource return just fit resource
  PCI: Check pref compatible bit for mem64 resource of pcie device
  PCI: Only treat non-pef mmio64 as pref if all bridges has MEM_64
  PCI: Add has_mem64 for host_bridge
  PCI: Only treat non-pef mmio64 as pref if host-bridge has_mem64
  PCI: Restore pref mmio allocation logic for hostbridge without mmio64
  sparc/PCI: Add mem64 resource parsing for root bus
  sparc/PCI: Add IORESOURCE_MEM_64 for 64-bit resource in of parsing
  powerpc/PCI: Add IORESOURCE_MEM_64 for 64-bit resource in of parsing
  of/PCI: Add IORESOURCE_MEM_64 for 64-bit resource
  PCI: Treat optional as must in first try for bridge rescan
  PCI: Get new realloc size for bridge for last try
  PCI: Don't release sibiling bridge resources during hotplug
  PCI: Don't release fixed resource for realloc
  PCI: Claim fixed resource during remove/rescan path
  PCI: Set resource to FIXED for lsi devices
  PCI, x86: Add pci=assign_pref_bars to re-allocate pref bars
  PCI: Introduce resource_disabled()
  PCI: Don't set flags to 0 when assign resource fail

 arch/alpha/kernel/pci.c                   |    2 +-
 arch/ia64/pci/pci.c                       |    4 +-
 arch/microblaze/pci/pci-common.c          |   23 +-
 arch/mn10300/unit-asb2305/pci-asb2305.c   |    4 +-
 arch/mn10300/unit-asb2305/pci.c           |    4 +-
 arch/powerpc/kernel/pci-common.c          |   27 +-
 arch/powerpc/kernel/pci_of_scan.c         |    4 +-
 arch/powerpc/platforms/powernv/pci-ioda.c |   12 +-
 arch/s390/pci/pci.c                       |    2 +-
 arch/sparc/kernel/of_device_32.c          |    5 +-
 arch/sparc/kernel/of_device_64.c          |    5 +-
 arch/sparc/kernel/pci.c                   |    9 +-
 arch/sparc/kernel/pci_common.c            |   15 +-
 arch/sparc/kernel/pci_impl.h              |    1 +
 arch/x86/include/asm/pci_x86.h            |    2 +-
 arch/x86/pci/common.c                     |    7 +-
 arch/x86/pci/i386.c                       |   72 +-
 arch/xtensa/kernel/pci.c                  |    4 +-
 drivers/iommu/intel-iommu.c               |    3 +-
 drivers/of/address.c                      |    4 +-
 drivers/pci/bus.c                         |    6 +-
 drivers/pci/host/pcie-rcar.c              |    2 +-
 drivers/pci/hotplug/acpiphp_glue.c        |    1 +
 drivers/pci/iov.c                         |    2 +-
 drivers/pci/pci.c                         |    3 +-
 drivers/pci/pci.h                         |    4 +
 drivers/pci/probe.c                       |   44 +-
 drivers/pci/quirks.c                      |   37 +
 drivers/pci/rom.c                         |    2 +-
 drivers/pci/setup-bus.c                   | 1321 +++++++++++++++++++++--------
 drivers/pci/setup-res.c                   |   18 +-
 include/asm-generic/pci-bridge.h          |    2 +
 include/linux/ioport.h                    |    6 +-
 include/linux/pci.h                       |    2 +
 kernel/resource.c                         |  175 +++-
 35 files changed, 1361 insertions(+), 473 deletions(-)

-- 
1.8.4.5

[PATCH v3 01/51] PCI: Cleanup res_to_dev_res() printout for addon resources

[Yinghai Lu]

Now get_res_add_size and get_res_add_align all have same printout
from res_to_dev_res(), and it is confusing.

Move out debug messages printout from res_to_dev_res(),
and later we will reuse res_to_dev_res() in other functions.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 508cc56..f0fa705 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -104,19 +104,9 @@ static struct pci_dev_resource *res_to_dev_res(struct list_head *head,
 {
 	struct pci_dev_resource *dev_res;
 
-	list_for_each_entry(dev_res, head, list) {
-		if (dev_res->res == res) {
-			int idx = res - &dev_res->dev->resource[0];
-
-			dev_printk(KERN_DEBUG, &dev_res->dev->dev,
-				 "res[%d]=%pR res_to_dev_res add_size %llx min_align %llx\n",
-				 idx, dev_res->res,
-				 (unsigned long long)dev_res->add_size,
-				 (unsigned long long)dev_res->min_align);
-
+	list_for_each_entry(dev_res, head, list)
+		if (dev_res->res == res)
 			return dev_res;
-		}
-	}
 
 	return NULL;
 }
@@ -127,7 +117,15 @@ static resource_size_t get_res_add_size(struct list_head *head,
 	struct pci_dev_resource *dev_res;
 
 	dev_res = res_to_dev_res(head, res);
-	return dev_res ? dev_res->add_size : 0;
+	if (!dev_res || !dev_res->add_size)
+		return 0;
+
+	dev_printk(KERN_DEBUG, &dev_res->dev->dev,
+		   "BAR %d: %pR get_res_add_size add_size   %llx\n",
+		   (int)(res - &dev_res->dev->resource[0]),
+		   res, (unsigned long long)dev_res->add_size);
+
+	return dev_res->add_size;
 }
 
 static resource_size_t get_res_add_align(struct list_head *head,
@@ -136,7 +134,15 @@ static resource_size_t get_res_add_align(struct list_head *head,
 	struct pci_dev_resource *dev_res;
 
 	dev_res = res_to_dev_res(head, res);
-	return dev_res ? dev_res->min_align : 0;
+	if (!dev_res || !dev_res->min_align)
+		return 0;
+
+	dev_printk(KERN_DEBUG, &dev_res->dev->dev,
+		   "BAR %d: %pR get_res_add_align min_align %llx\n",
+		   (int)(res - &dev_res->dev->resource[0]),
+		   res, (unsigned long long)dev_res->min_align);
+
+	return dev_res->min_align;
 }
 
 
-- 
1.8.4.5

[PATCH v3 02/51] PCI: Reuse res_to_dev_res in reassign_resources_sorted

[Yinghai Lu]

Now res_to_dev_res() does not print out debug message anymore, so
reuse it in reassign_resource_sorted().

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 11 +----------
 1 file changed, 1 insertion(+), 10 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index f0fa705..247d8fe 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -240,26 +240,17 @@ static void reassign_resources_sorted(struct list_head *realloc_head,
 {
 	struct resource *res;
 	struct pci_dev_resource *add_res, *tmp;
-	struct pci_dev_resource *dev_res;
 	resource_size_t add_size, align;
 	int idx;
 
 	list_for_each_entry_safe(add_res, tmp, realloc_head, list) {
-		bool found_match = false;
-
 		res = add_res->res;
 		/* skip resource that has been reset */
 		if (!res->flags)
 			goto out;
 
 		/* skip this resource if not found in head list */
-		list_for_each_entry(dev_res, head, list) {
-			if (dev_res->res == res) {
-				found_match = true;
-				break;
-			}
-		}
-		if (!found_match)/* just skip */
+		if (!res_to_dev_res(head, res))
 			continue;
 
 		idx = res - &add_res->dev->resource[0];
-- 
1.8.4.5

[PATCH v3 03/51] PCI: Use correct align for optional only resources during sorting

[Yinghai Lu]

During sorting before assign, we only put resource with non-zero align
in the sorted list, so for optional resources that must size is 0 and only
have addon parts, we need to have correct align.

While treating SRIOV as optional resources, we always read alignment for
SRIOV bars, so they are ok.
Hotplug bridge resources are using STARTALIGN so it is ok when size is 0
if we have correct start for them.

Later we want to treat the ROM BAR as optional resource, and it has
have SIZEALIGN, we need to find a way to get align for them.

We can use addon resource align instead in that case, and it will
be ok for SRIOV path and hotplug bridge resource path.

Sorted list will contain must resource align/size to 0/0 to hold spot for
optional resources.

We need to pass realloc_head from sizing stage to sorting stage, and
get entry from realloc list and calculate align from the entry.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
Reported-by: TJ <linux@iam.tj>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 50 ++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 43 insertions(+), 7 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 247d8fe..27cb0f0 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -145,9 +145,43 @@ static resource_size_t get_res_add_align(struct list_head *head,
 	return dev_res->min_align;
 }
 
+static resource_size_t __pci_resource_alignment(
+				struct pci_dev *dev,
+				struct resource *r,
+				struct list_head *realloc_head)
+{
+	resource_size_t r_align = pci_resource_alignment(dev, r);
+	resource_size_t orig_start, orig_end;
+	struct pci_dev_resource *dev_res;
+
+	if (r_align || !realloc_head)
+		return r_align;
+
+	dev_res = res_to_dev_res(realloc_head, r);
+	if (!dev_res || !dev_res->add_size)
+		return r_align;
+
+	orig_start = r->start;
+	orig_end = r->end;
+	r->end += dev_res->add_size;
+	if ((r->flags & IORESOURCE_STARTALIGN)) {
+		resource_size_t r_size = resource_size(r);
+		resource_size_t add_align = dev_res->min_align;
+
+		r->start = add_align;
+		r->end = add_align + r_size - 1;
+	}
+	r_align = pci_resource_alignment(dev, r);
+	r->start = orig_start;
+	r->end = orig_end;
+
+	return r_align;
+}
 
 /* Sort resources by alignment */
-static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head)
+static void pdev_sort_resources(struct pci_dev *dev,
+				 struct list_head *realloc_head,
+				 struct list_head *head)
 {
 	int i;
 
@@ -165,7 +199,7 @@ static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head)
 		if (!(r->flags) || r->parent)
 			continue;
 
-		r_align = pci_resource_alignment(dev, r);
+		r_align = __pci_resource_alignment(dev, r, realloc_head);
 		if (!r_align) {
 			dev_warn(&dev->dev, "BAR %d: %pR has bogus alignment\n",
 				 i, r);
@@ -183,8 +217,9 @@ static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head)
 		list_for_each_entry(dev_res, head, list) {
 			resource_size_t align;
 
-			align = pci_resource_alignment(dev_res->dev,
-							 dev_res->res);
+			align = __pci_resource_alignment(dev_res->dev,
+							 dev_res->res,
+							 realloc_head);
 
 			if (r_align > align) {
 				n = &dev_res->list;
@@ -197,6 +232,7 @@ static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head)
 }
 
 static void __dev_sort_resources(struct pci_dev *dev,
+				 struct list_head *realloc_head,
 				 struct list_head *head)
 {
 	u16 class = dev->class >> 8;
@@ -213,7 +249,7 @@ static void __dev_sort_resources(struct pci_dev *dev,
 			return;
 	}
 
-	pdev_sort_resources(dev, head);
+	pdev_sort_resources(dev, realloc_head, head);
 }
 
 static inline void reset_resource(struct resource *res)
@@ -501,7 +537,7 @@ static void pdev_assign_resources_sorted(struct pci_dev *dev,
 {
 	LIST_HEAD(head);
 
-	__dev_sort_resources(dev, &head);
+	__dev_sort_resources(dev, add_head, &head);
 	__assign_resources_sorted(&head, add_head, fail_head);
 
 }
@@ -514,7 +550,7 @@ static void pbus_assign_resources_sorted(const struct pci_bus *bus,
 	LIST_HEAD(head);
 
 	list_for_each_entry(dev, &bus->devices, bus_list)
-		__dev_sort_resources(dev, &head);
+		__dev_sort_resources(dev, realloc_head, &head);
 
 	__assign_resources_sorted(&head, realloc_head, fail_head);
 }
-- 
1.8.4.5

[PATCH v3 04/51] PCI: Optimize bus align/size calculation during sizing

[Yinghai Lu]

Current code try to get align as small as possible and use that to
align final size. But it does not handle resource that size is bigger
than align in optimal way, kernel only use max align for them.

For example:
 when we have resources with align/size: 1M/2M, 512M/512M,
   bus resource min_align/size0 will be 512M/1024M,
   but optimal value should be 256M/768M.

For following cases that we have resource size that is bigger
than resource alignment:
1. SRIOV bar.
2. PCI bridges with several bridges or devices as children.

We can keep on trying to allocate children devices resources under range
[half_align, half_align + aligned_size).
If sucesses, we can use that half_align as new min_align.

After this patch, we get:
 align/size: 1M/2M, 2M/4M, 4M/8M, 8M/16M
 new min_align/min_size: 4M/32M, and old is 8M/32M

 align/size: 1M/2M, 2M/4M, 4M/8M
 new min_align/min_size: 2M/14M, and old is 4M/16M

 align/size: 1M/2M, 512M/512M
 new min_align/min_size: 256M/768M, and old is 512M/1024M

The real result from one system with one pcie card that has
four functions that support sriov:
 align/size:
   00800000/00800000
   00800000/00800000
   00800000/00800000
   00800000/00800000
   00010000/00200000
   00010000/00200000
   00010000/00200000
   00010000/00200000
   00008000/00008000
   00008000/00008000
   00008000/00008000
   00008000/00008000
   00004000/00080000
   00004000/00080000
   00004000/00080000
   00004000/00080000
 old min_align/min_size: 00400000/02c00000
     min_align/min_size: 00100000/02b00000

So align will be 1M instead of 4M.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
Reported-by: TJ <linux@iam.tj>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 195 ++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 157 insertions(+), 38 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 27cb0f0..ecdf011 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -30,6 +30,34 @@
 
 unsigned int pci_flags;
 
+static inline bool is_before(resource_size_t align1, resource_size_t size1,
+			     resource_size_t align2, resource_size_t size2)
+{
+	resource_size_t size1_left, size2_left;
+
+	/* big align is before small align */
+	if (align1 > align2)
+		return true;
+
+	/*
+	 * for same align:
+	 *   aligned is before not aligned
+	 *   for not aligned, big remainder is before small remainder
+	 */
+	if (align1 == align2) {
+		size1_left = size1 & (align1 - 1);
+		if (!size1_left)
+			size1_left = align1;
+		size2_left = size2 & (align2 - 1);
+		if (!size2_left)
+			size2_left = align2;
+		if (size1_left > size2_left)
+			return true;
+	}
+
+	return false;
+}
+
 struct pci_dev_resource {
 	struct list_head list;
 	struct resource *res;
@@ -999,26 +1027,125 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 	}
 }
 
-static inline resource_size_t calculate_mem_align(resource_size_t *aligns,
-						  int max_order)
+struct align_test_res {
+	struct list_head list;
+	struct resource res;
+	resource_size_t size;
+	resource_size_t align;
+};
+
+static void free_align_test_list(struct list_head *head)
 {
-	resource_size_t align = 0;
-	resource_size_t min_align = 0;
-	int order;
+	struct align_test_res *p, *tmp;
 
-	for (order = 0; order <= max_order; order++) {
-		resource_size_t align1 = 1;
+	list_for_each_entry_safe(p, tmp, head, list) {
+		list_del(&p->list);
+		kfree(p);
+	}
+}
 
-		align1 <<= (order + 20);
+static int add_to_align_test_list(struct list_head *head,
+				  resource_size_t align, resource_size_t size)
+{
+	struct align_test_res *tmp;
+
+	tmp = kzalloc(sizeof(*tmp), GFP_KERNEL);
+	if (!tmp)
+		return -ENOMEM;
+
+	tmp->align = align;
+	tmp->size = size;
+
+	list_add_tail(&tmp->list, head);
+
+	return 0;
+}
+
+static void __sort_align_test(struct list_head *head)
+{
+	struct align_test_res *res1, *tmp_res, *res2;
 
-		if (!align)
-			min_align = align1;
-		else if (ALIGN(align + min_align, min_align) < align1)
-			min_align = align1 >> 1;
-		align += aligns[order];
+	list_for_each_entry_safe(res1, tmp_res, head, list) {
+		/* reorder it */
+		list_for_each_entry(res2, head, list) {
+			if (res2 == res1)
+				break;
+
+			if (is_before(res1->align, res1->size,
+				      res2->align, res2->size)) {
+				list_move_tail(&res1->list, &res2->list);
+				break;
+			}
+		}
+	}
+}
+
+static bool is_align_size_good(struct list_head *head,
+			resource_size_t min_align, resource_size_t size,
+			resource_size_t start)
+{
+	struct align_test_res *p;
+	struct resource root;
+
+	memset(&root, 0, sizeof(root));
+	root.start = start;
+	root.end = start + size - 1;
+
+	list_for_each_entry(p, head, list)
+		memset(&p->res, 0, sizeof(p->res));
+
+	list_for_each_entry(p, head, list)
+		if (allocate_resource(&root, &p->res, p->size,
+				0, (resource_size_t)-1ULL,
+				p->align, NULL, NULL))
+			return false;
+
+	return true;
+}
+
+static resource_size_t calculate_mem_align(struct list_head *head,
+				resource_size_t max_align, resource_size_t size,
+				resource_size_t align_low)
+{
+	struct align_test_res *p;
+	resource_size_t min_align, good_align, aligned_size, start;
+	int count = 0;
+
+	if (max_align <= align_low) {
+		good_align = align_low;
+		goto out;
 	}
 
-	return min_align;
+	good_align = max_align;
+
+	list_for_each_entry(p, head, list)
+		count++;
+
+	if (count <= 1)
+		goto out;
+
+	__sort_align_test(head);
+
+	do {
+		/* check if we can use smaller align */
+		min_align = good_align >> 1;
+		aligned_size = ALIGN(size, min_align);
+
+		/* need to make sure every offset work */
+		for (start = min_align; start < max_align; start += min_align) {
+			/* checked already with last align ? */
+			if (!(start & (good_align - 1)))
+				continue;
+
+			if (!is_align_size_good(head, min_align, aligned_size,
+					       start))
+				goto out;
+		}
+		good_align = min_align;
+	} while (min_align > align_low);
+
+out:
+	return good_align;
 }
 
 /**
@@ -1048,19 +1175,17 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 {
 	struct pci_dev *dev;
 	resource_size_t min_align, align, size, size0, size1;
-	resource_size_t aligns[18];	/* Alignments from 1Mb to 128Gb */
-	int order, max_order;
+	resource_size_t max_align = 0;
 	struct resource *b_res = find_free_bus_resource(bus,
 					mask | IORESOURCE_PREFETCH, type);
 	resource_size_t children_add_size = 0;
 	resource_size_t children_add_align = 0;
 	resource_size_t add_align = 0;
+	LIST_HEAD(align_test_list);
 
 	if (!b_res)
 		return -ENOSPC;
 
-	memset(aligns, 0, sizeof(aligns));
-	max_order = 0;
 	size = 0;
 
 	list_for_each_entry(dev, &bus->devices, bus_list) {
@@ -1086,29 +1211,20 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 				continue;
 			}
 #endif
-			/*
-			 * aligns[0] is for 1MB (since bridge memory
-			 * windows are always at least 1MB aligned), so
-			 * keep "order" from being negative for smaller
-			 * resources.
-			 */
 			align = pci_resource_alignment(dev, r);
-			order = __ffs(align) - 20;
-			if (order < 0)
-				order = 0;
-			if (order >= ARRAY_SIZE(aligns)) {
+			if (align > (1ULL<<37)) { /*128 Gb*/
 				dev_warn(&dev->dev, "disabling BAR %d: %pR (bad alignment %#llx)\n",
-					 i, r, (unsigned long long) align);
+					i, r, (unsigned long long) align);
 				r->flags = 0;
 				continue;
 			}
+
+			if (r_size > 1)
+				add_to_align_test_list(&align_test_list,
+							align, r_size);
 			size += r_size;
-			/* Exclude ranges with size > align from
-			   calculation of the alignment. */
-			if (r_size == align)
-				aligns[order] += align;
-			if (order > max_order)
-				max_order = order;
+			if (align > max_align)
+				max_align = align;
 
 			if (realloc_head) {
 				children_add_size += get_res_add_size(realloc_head, r);
@@ -1118,9 +1234,12 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		}
 	}
 
-	min_align = calculate_mem_align(aligns, max_order);
-	min_align = max(min_align, window_alignment(bus, b_res->flags));
-	size0 = calculate_memsize(size, min_size, 0, resource_size(b_res), min_align);
+	max_align = max(max_align, window_alignment(bus, b_res->flags));
+	min_align = calculate_mem_align(&align_test_list, max_align, size,
+					window_alignment(bus, b_res->flags));
+	size0 = calculate_memsize(size, min_size, 0,
+				  resource_size(b_res), min_align);
+	free_align_test_list(&align_test_list);
 	add_align = max(min_align, add_align);
 	if (children_add_size > add_size)
 		add_size = children_add_size;
-- 
1.8.4.5

[PATCH v3 05/51] PCI: Optimize bus align/size calculation for optional during sizing

[Yinghai Lu]

Current add_align always use max align, that make must+optional
to get allocated more than needed in some cases.

Now we have new calculate_mem_align, we could use it for add_align
calculation.

Need to create separated list for must+optional align/size info.

After that we can get smaller add_align/size, we have more chance
to make must+optional to get allocated.

The result for bridge that have Intel 4x10g card installed.

 pci 0000:20:03.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
	to [bus 2a-31] calculate_mem for must
 ===========BEGIN========================
 align/size:
   00800000/00800000
   00800000/00800000
   00800000/00800000
   00800000/00800000
   00008000/00008000
   00008000/00008000
   00008000/00008000
   00008000/00008000
 old min_align/min_size: 00400000/02400000
     min_align/min_size: 00400000/02400000
 ===========END========================

 pci 0000:20:03.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
	to [bus 2a-31] calculate_mem for add
 ===========BEGIN========================
 align/size:
   00800000/00800000
   00800000/00800000
   00800000/00800000
   00800000/00800000
   00010000/00200000
   00010000/00200000
   00010000/00200000
   00010000/00200000
   00008000/00008000
   00008000/00008000
   00008000/00008000
   00008000/00008000
   00004000/00080000
   00004000/00080000
   00004000/00080000
   00004000/00080000
 old min_align/min_size: 00800000/03000000
     min_align/min_size: 00100000/02b00000
 ===========END========================

so must align/size: 0x400000/0x2400000, and
 new must+optional align/size: 0x100000/0x2b00000, and it is better
than old must+optional align/size: 0x800000/0x3000000

Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
Reported-by: TJ <linux@iam.tj>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 82 ++++++++++++++++++++++++++++++-------------------
 1 file changed, 51 insertions(+), 31 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index ecdf011..4c7f25f 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -901,7 +901,6 @@ static resource_size_t calculate_iosize(resource_size_t size,
 
 static resource_size_t calculate_memsize(resource_size_t size,
 		resource_size_t min_size,
-		resource_size_t size1,
 		resource_size_t old_size,
 		resource_size_t align)
 {
@@ -911,7 +910,7 @@ static resource_size_t calculate_memsize(resource_size_t size,
 		old_size = 0;
 	if (size < old_size)
 		size = old_size;
-	size = ALIGN(size + size1, align);
+	size = ALIGN(size, align);
 	return size;
 }
 
@@ -1174,44 +1173,45 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 			 struct list_head *realloc_head)
 {
 	struct pci_dev *dev;
-	resource_size_t min_align, align, size, size0, size1;
-	resource_size_t max_align = 0;
+	resource_size_t min_align = 0, min_add_align = 0;
+	resource_size_t max_align = 0, max_add_align = 0;
+	resource_size_t size = 0, size0 = 0, size1 = 0, sum_add_size = 0;
 	struct resource *b_res = find_free_bus_resource(bus,
 					mask | IORESOURCE_PREFETCH, type);
-	resource_size_t children_add_size = 0;
-	resource_size_t children_add_align = 0;
-	resource_size_t add_align = 0;
 	LIST_HEAD(align_test_list);
+	LIST_HEAD(align_test_add_list);
 
 	if (!b_res)
 		return -ENOSPC;
 
-	size = 0;
-
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		int i;
 
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
-			resource_size_t r_size;
+			resource_size_t r_size, align;
 
 			if (r->parent || ((r->flags & mask) != type &&
 					  (r->flags & mask) != type2 &&
 					  (r->flags & mask) != type3))
 				continue;
+
 			r_size = resource_size(r);
+			align = pci_resource_alignment(dev, r);
 #ifdef CONFIG_PCI_IOV
 			/* put SRIOV requested res to the optional list */
 			if (realloc_head && i >= PCI_IOV_RESOURCES &&
 					i <= PCI_IOV_RESOURCE_END) {
-				add_align = max(pci_resource_alignment(dev, r), add_align);
+				add_to_align_test_list(&align_test_add_list,
+							align, r_size);
 				r->end = r->start - 1;
 				add_to_list(realloc_head, dev, r, r_size, 0/* don't care */);
-				children_add_size += r_size;
+				sum_add_size += r_size;
+				if (align > max_add_align)
+					max_add_align = align;
 				continue;
 			}
 #endif
-			align = pci_resource_alignment(dev, r);
 			if (align > (1ULL<<37)) { /*128 Gb*/
 				dev_warn(&dev->dev, "disabling BAR %d: %pR (bad alignment %#llx)\n",
 					i, r, (unsigned long long) align);
@@ -1219,33 +1219,52 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 				continue;
 			}
 
-			if (r_size > 1)
+			if (r_size > 1) {
 				add_to_align_test_list(&align_test_list,
 							align, r_size);
-			size += r_size;
-			if (align > max_align)
-				max_align = align;
+				size += r_size;
+				if (align > max_align)
+					max_align = align;
+			}
 
 			if (realloc_head) {
-				children_add_size += get_res_add_size(realloc_head, r);
-				children_add_align = get_res_add_align(realloc_head, r);
-				add_align = max(add_align, children_add_align);
+				resource_size_t add_r_size, add_align;
+
+				add_r_size = get_res_add_size(realloc_head, r);
+				add_align = get_res_add_align(realloc_head, r);
+				/* no add on ? */
+				if (add_align < align)
+					add_align = align;
+				add_to_align_test_list(&align_test_add_list,
+							add_align,
+							r_size + add_r_size);
+				sum_add_size += r_size + add_r_size;
+				if (add_align > max_add_align)
+					max_add_align = add_align;
 			}
 		}
 	}
 
 	max_align = max(max_align, window_alignment(bus, b_res->flags));
-	min_align = calculate_mem_align(&align_test_list, max_align, size,
-					window_alignment(bus, b_res->flags));
-	size0 = calculate_memsize(size, min_size, 0,
+	if (size || min_size) {
+		min_align = calculate_mem_align(&align_test_list, max_align,
+				 size, window_alignment(bus, b_res->flags));
+		size0 = calculate_memsize(size, min_size,
 				  resource_size(b_res), min_align);
+	}
 	free_align_test_list(&align_test_list);
-	add_align = max(min_align, add_align);
-	if (children_add_size > add_size)
-		add_size = children_add_size;
-	size1 = (!realloc_head || (realloc_head && !add_size)) ? size0 :
-		calculate_memsize(size, min_size, add_size,
-				resource_size(b_res), add_align);
+
+	if ((sum_add_size - size) < add_size)
+		sum_add_size = size + add_size;
+	if (sum_add_size > size && realloc_head) {
+		min_add_align = calculate_mem_align(&align_test_add_list,
+					max_add_align, sum_add_size,
+					window_alignment(bus, b_res->flags));
+		size1 = calculate_memsize(sum_add_size, min_size,
+				 resource_size(b_res), min_add_align);
+	}
+	free_align_test_list(&align_test_add_list);
+
 	if (!size0 && !size1) {
 		if (b_res->start || b_res->end)
 			dev_info(&bus->self->dev, "disabling bridge window %pR to %pR (unused)\n",
@@ -1257,11 +1276,12 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	b_res->end = size0 + min_align - 1;
 	b_res->flags |= IORESOURCE_STARTALIGN;
 	if (size1 > size0 && realloc_head) {
-		add_to_list(realloc_head, bus->self, b_res, size1-size0, add_align);
+		add_to_list(realloc_head, bus->self, b_res, size1 - size0,
+				min_add_align);
 		dev_printk(KERN_DEBUG, &bus->self->dev, "bridge window %pR to %pR add_size %llx add_align %llx\n",
 			   b_res, &bus->busn_res,
 			   (unsigned long long) (size1 - size0),
-			   (unsigned long long) add_align);
+			   (unsigned long long) min_add_align);
 	}
 	return 0;
 }
-- 
1.8.4.5

[PATCH v3 06/51] PCI: Don't add too much optional size for hotplug bridge mmio

[Yinghai Lu]

Current code will always add 2M for hotplug bridge mmio even
there is child device under it.

For example:
	40:03.0 --- 43:00.0 --- 44:02.0 -+- 45:00.0
					 \- 45:00.1

44:02.0 will need 1M as must for 45:00.0 and 45:00.1
When we calculate add_size for 44:02.0, we pass 2M as additional
size for hotplug bridge, total will be 3M.

That is different from code before changes for optional support,
or even current code that treat optional as must directly by
not passing realloc head. We only need 2M as total.

The optional size should be 1M, and total size should be 2M.

This patch change to comparing must+optional with min_sum_size to
get smaller optional size.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 4c7f25f..7346bbf 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1156,7 +1156,6 @@ out:
  * @type2: second match type
  * @type3: third match type
  * @min_size : the minimum memory window that must to be allocated
- * @add_size : additional optional memory window
  * @realloc_head : track the additional memory window on this list
  *
  * Calculate the size of the bus and minimal alignment which
@@ -1169,10 +1168,11 @@ out:
 static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 			 unsigned long type, unsigned long type2,
 			 unsigned long type3,
-			 resource_size_t min_size, resource_size_t add_size,
+			 resource_size_t min_size,
 			 struct list_head *realloc_head)
 {
 	struct pci_dev *dev;
+	resource_size_t min_sum_size = 0;
 	resource_size_t min_align = 0, min_add_align = 0;
 	resource_size_t max_align = 0, max_add_align = 0;
 	resource_size_t size = 0, size0 = 0, size1 = 0, sum_add_size = 0;
@@ -1184,6 +1184,11 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	if (!b_res)
 		return -ENOSPC;
 
+	if (realloc_head) {
+		min_sum_size = min_size;
+		min_size = 0;
+	}
+
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		int i;
 
@@ -1254,8 +1259,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	}
 	free_align_test_list(&align_test_list);
 
-	if ((sum_add_size - size) < add_size)
-		sum_add_size = size + add_size;
+	if (sum_add_size < min_sum_size)
+		sum_add_size = min_sum_size;
 	if (sum_add_size > size && realloc_head) {
 		min_add_align = calculate_mem_align(&align_test_add_list,
 					max_add_align, sum_add_size,
@@ -1392,7 +1397,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 {
 	struct pci_dev *dev;
 	unsigned long mask, prefmask, type2 = 0, type3 = 0;
-	resource_size_t additional_mem_size = 0, additional_io_size = 0;
+	resource_size_t min_mem_size = 0, additional_io_size = 0;
 	struct resource *b_res;
 	int ret;
 
@@ -1426,7 +1431,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 		pci_bridge_check_ranges(bus);
 		if (bus->self->is_hotplug_bridge) {
 			additional_io_size  = pci_hotplug_io_size;
-			additional_mem_size = pci_hotplug_mem_size;
+			min_mem_size = pci_hotplug_mem_size;
 		}
 		/* Fall through */
 	default:
@@ -1445,8 +1450,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 			prefmask |= IORESOURCE_MEM_64;
 			ret = pbus_size_mem(bus, prefmask, prefmask,
 				  prefmask, prefmask,
-				  realloc_head ? 0 : additional_mem_size,
-				  additional_mem_size, realloc_head);
+				  min_mem_size, realloc_head);
 
 			/*
 			 * If successful, all non-prefetchable resources
@@ -1469,8 +1473,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 			prefmask &= ~IORESOURCE_MEM_64;
 			ret = pbus_size_mem(bus, prefmask, prefmask,
 					 prefmask, prefmask,
-					 realloc_head ? 0 : additional_mem_size,
-					 additional_mem_size, realloc_head);
+					 min_mem_size, realloc_head);
 
 			/*
 			 * If successful, only non-prefetchable resources
@@ -1479,7 +1482,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 			if (ret == 0)
 				mask = prefmask;
 			else
-				additional_mem_size += additional_mem_size;
+				min_mem_size += min_mem_size;
 
 			type2 = type3 = IORESOURCE_MEM;
 		}
@@ -1500,8 +1503,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 		 * window.
 		 */
 		pbus_size_mem(bus, mask, IORESOURCE_MEM, type2, type3,
-				realloc_head ? 0 : additional_mem_size,
-				additional_mem_size, realloc_head);
+				min_mem_size, realloc_head);
 		break;
 	}
 }
-- 
1.8.4.5

[PATCH v3 07/51] PCI: Reorder resources list for must/optional resources

[Yinghai Lu]

After we update size and alignment for must+optional resource, we
reorder them with new alignment, but this is only for STARTALIGN.

For SIZEALIGN type resource, after add back add_size, the alignment
get changed, so need to do the sorting like STARTALIGN type resources.

Also we need to reorder the sorting back after we restore
resource to must only when must+optional fail to allocate for all.

So move out the reordering code from the loop to separated function,
and call it two times accordingly.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 62 +++++++++++++++++++++++++++++--------------------
 1 file changed, 37 insertions(+), 25 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 7346bbf..6f2d508 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -287,6 +287,31 @@ static inline void reset_resource(struct resource *res)
 	res->flags = 0;
 }
 
+static void __sort_resources(struct list_head *head)
+{
+	struct pci_dev_resource *res1, *tmp_res, *res2;
+
+	list_for_each_entry_safe(res1, tmp_res, head, list) {
+		resource_size_t align1, size1, align2, size2;
+
+		align1 = pci_resource_alignment(res1->dev, res1->res);
+		size1 = resource_size(res1->res);
+
+		/* reorder it */
+		list_for_each_entry(res2, head, list) {
+			if (res2 == res1)
+				break;
+
+			align2 = pci_resource_alignment(res2->dev, res2->res);
+			size2 = resource_size(res2->res);
+			if (is_before(align1, size1, align2, size2)) {
+				list_move_tail(&res1->list, &res2->list);
+				break;
+			}
+		}
+	}
+}
+
 /**
  * reassign_resources_sorted() - satisfy any additional resource requests
  *
@@ -449,9 +474,9 @@ static void __assign_resources_sorted(struct list_head *head,
 	LIST_HEAD(save_head);
 	LIST_HEAD(local_fail_head);
 	struct pci_dev_resource *save_res;
-	struct pci_dev_resource *dev_res, *tmp_res, *dev_res2;
+	struct pci_dev_resource *dev_res, *tmp_res;
 	unsigned long fail_type;
-	resource_size_t add_align, align;
+	resource_size_t add_align;
 
 	/* Check if optional add_size is there */
 	if (!realloc_head || list_empty(realloc_head))
@@ -466,47 +491,32 @@ static void __assign_resources_sorted(struct list_head *head,
 	}
 
 	/* Update res in head list with add_size in realloc_head list */
-	list_for_each_entry_safe(dev_res, tmp_res, head, list) {
+	list_for_each_entry(dev_res, head, list) {
 		dev_res->res->end += get_res_add_size(realloc_head,
 							dev_res->res);
 
 		/*
 		 * There are two kinds of additional resources in the list:
-		 * 1. bridge resource  -- IORESOURCE_STARTALIGN
-		 * 2. SR-IOV resource   -- IORESOURCE_SIZEALIGN
-		 * Here just fix the additional alignment for bridge
+		 * 1. bridge resource with IORESOURCE_STARTALIGN
+		 *    need to update start to change alignment
+		 * 2. resource with IORESOURCE_SIZEALIGN
+		 *    update size above already change alignment.
 		 */
 		if (!(dev_res->res->flags & IORESOURCE_STARTALIGN))
 			continue;
 
 		add_align = get_res_add_align(realloc_head, dev_res->res);
 
-		/*
-		 * The "head" list is sorted by the alignment to make sure
-		 * resources with bigger alignment will be assigned first.
-		 * After we change the alignment of a dev_res in "head" list,
-		 * we need to reorder the list by alignment to make it
-		 * consistent.
-		 */
-		if (add_align > dev_res->res->start) {
+		if (add_align) {
 			resource_size_t r_size = resource_size(dev_res->res);
 
 			dev_res->res->start = add_align;
 			dev_res->res->end = add_align + r_size - 1;
-
-			list_for_each_entry(dev_res2, head, list) {
-				align = pci_resource_alignment(dev_res2->dev,
-							       dev_res2->res);
-				if (add_align > align) {
-					list_move_tail(&dev_res->list,
-						       &dev_res2->list);
-					break;
-				}
-			}
                }
-
 	}
 
+	__sort_resources(head);
+
 	/* Try updated head list with add_size added */
 	assign_requested_resources_sorted(head, &local_fail_head);
 
@@ -548,6 +558,8 @@ static void __assign_resources_sorted(struct list_head *head,
 	}
 	free_list(&save_head);
 
+	__sort_resources(head);
+
 requested_and_reassign:
 	/* Satisfy the must-have resource requests */
 	assign_requested_resources_sorted(head, fail_head);
-- 
1.8.4.5

[PATCH v3 08/51] PCI: Remove duplicated code for resource sorting

[Yinghai Lu]

Now __sort_resources, and pdev_sort_resources all have sorting
code.

As we are going to call __sort_resources several places later, so
choose to keep __sort_resources, and remove related code in
pdev_sort_resources.


Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 22 +++-------------------
 1 file changed, 3 insertions(+), 19 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 6f2d508..6642a60 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -215,9 +215,8 @@ static void pdev_sort_resources(struct pci_dev *dev,
 
 	for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 		struct resource *r;
-		struct pci_dev_resource *dev_res, *tmp;
+		struct pci_dev_resource *tmp;
 		resource_size_t r_align;
-		struct list_head *n;
 
 		r = &dev->resource[i];
 
@@ -240,22 +239,7 @@ static void pdev_sort_resources(struct pci_dev *dev,
 		tmp->res = r;
 		tmp->dev = dev;
 
-		/* fallback is smallest one or list is empty*/
-		n = head;
-		list_for_each_entry(dev_res, head, list) {
-			resource_size_t align;
-
-			align = __pci_resource_alignment(dev_res->dev,
-							 dev_res->res,
-							 realloc_head);
-
-			if (r_align > align) {
-				n = &dev_res->list;
-				break;
-			}
-		}
-		/* Insert it just before n*/
-		list_add_tail(&tmp->list, n);
+		list_add_tail(&tmp->list, head);
 	}
 }
 
@@ -558,9 +542,9 @@ static void __assign_resources_sorted(struct list_head *head,
 	}
 	free_list(&save_head);
 
+requested_and_reassign:
 	__sort_resources(head);
 
-requested_and_reassign:
 	/* Satisfy the must-have resource requests */
 	assign_requested_resources_sorted(head, fail_head);
 
-- 
1.8.4.5

[PATCH v3 09/51] PCI: Rename pdev_sort_resources to pdev_check_resources

[Yinghai Lu]

We don't do sorting in those functions anymore, so change "sort" to "check"
instead.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 6642a60..292f2a5 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -206,8 +206,8 @@ static resource_size_t __pci_resource_alignment(
 	return r_align;
 }
 
-/* Sort resources by alignment */
-static void pdev_sort_resources(struct pci_dev *dev,
+/* check resources and save to the list */
+static void pdev_check_resources(struct pci_dev *dev,
 				 struct list_head *realloc_head,
 				 struct list_head *head)
 {
@@ -243,7 +243,7 @@ static void pdev_sort_resources(struct pci_dev *dev,
 	}
 }
 
-static void __dev_sort_resources(struct pci_dev *dev,
+static void __dev_check_resources(struct pci_dev *dev,
 				 struct list_head *realloc_head,
 				 struct list_head *head)
 {
@@ -261,7 +261,7 @@ static void __dev_sort_resources(struct pci_dev *dev,
 			return;
 	}
 
-	pdev_sort_resources(dev, realloc_head, head);
+	pdev_check_resources(dev, realloc_head, head);
 }
 
 static inline void reset_resource(struct resource *res)
@@ -561,7 +561,7 @@ static void pdev_assign_resources_sorted(struct pci_dev *dev,
 {
 	LIST_HEAD(head);
 
-	__dev_sort_resources(dev, add_head, &head);
+	__dev_check_resources(dev, add_head, &head);
 	__assign_resources_sorted(&head, add_head, fail_head);
 
 }
@@ -574,7 +574,7 @@ static void pbus_assign_resources_sorted(const struct pci_bus *bus,
 	LIST_HEAD(head);
 
 	list_for_each_entry(dev, &bus->devices, bus_list)
-		__dev_sort_resources(dev, realloc_head, &head);
+		__dev_check_resources(dev, realloc_head, &head);
 
 	__assign_resources_sorted(&head, realloc_head, fail_head);
 }
-- 
1.8.4.5

[PATCH v3 10/51] PCI: Treat ROM resource as optional during realloc

[Yinghai Lu]

So will try to allocate them together with must-have ones, if can
not assign them, could go with must-have one only, and just skip
ROM resources.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 37 ++++++++++++++++++++-----------------
 1 file changed, 20 insertions(+), 17 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 292f2a5..3abf249 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -372,18 +372,10 @@ static void assign_requested_resources_sorted(struct list_head *head,
 		idx = res - &dev_res->dev->resource[0];
 		if (resource_size(res) &&
 		    pci_assign_resource(dev_res->dev, idx)) {
-			if (fail_head) {
-				/*
-				 * if the failed res is for ROM BAR, and it will
-				 * be enabled later, don't add it to the list
-				 */
-				if (!((idx == PCI_ROM_RESOURCE) &&
-				      (!(res->flags & IORESOURCE_ROM_ENABLE))))
-					add_to_list(fail_head,
-						    dev_res->dev, res,
-						    0 /* don't care */,
-						    0 /* don't care */);
-			}
+			if (fail_head)
+				add_to_list(fail_head, dev_res->dev, res,
+					    0 /* don't care */,
+					    0 /* don't care */);
 			reset_resource(res);
 		}
 	}
@@ -1143,6 +1135,19 @@ out:
 	return good_align;
 }
 
+static inline bool is_optional(int i)
+{
+
+	if (i == PCI_ROM_RESOURCE)
+		return true;
+
+#ifdef CONFIG_PCI_IOV
+	if (i >= PCI_IOV_RESOURCES && i <= PCI_IOV_RESOURCE_END)
+		return true;
+#endif
+
+	return false;
+}
 /**
  * pbus_size_mem() - size the memory window of a given bus
  *
@@ -1199,10 +1204,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 			r_size = resource_size(r);
 			align = pci_resource_alignment(dev, r);
-#ifdef CONFIG_PCI_IOV
-			/* put SRIOV requested res to the optional list */
-			if (realloc_head && i >= PCI_IOV_RESOURCES &&
-					i <= PCI_IOV_RESOURCE_END) {
+			/* put SRIOV/ROM res to realloc list */
+			if (realloc_head && is_optional(i)) {
 				add_to_align_test_list(&align_test_add_list,
 							align, r_size);
 				r->end = r->start - 1;
@@ -1212,7 +1215,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 					max_add_align = align;
 				continue;
 			}
-#endif
+
 			if (align > (1ULL<<37)) { /*128 Gb*/
 				dev_warn(&dev->dev, "disabling BAR %d: %pR (bad alignment %#llx)\n",
 					i, r, (unsigned long long) align);
-- 
1.8.4.5

[PATCH v3 11/51] PCI: Add debug printout during releasing partial assigned resources

[Yinghai Lu]

We try to assign must+optional at first, and we only accept the result if
all resources get allocated. Otherwise will release assigned in the list,
and try to assign must and expand to optional.

We have to do that to make sure any must has priority than any optional
addon.

When that happens, we only print out "assigned" info, that is confusing
as it looks like same range is assigned to two peer resources at the same
time.

Add printout for releasing so we have whole picture in debug messages.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 3abf249..6dff258 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -521,9 +521,17 @@ static void __assign_resources_sorted(struct list_head *head,
 
 	free_list(&local_fail_head);
 	/* Release assigned resource */
-	list_for_each_entry(dev_res, head, list)
-		if (dev_res->res->parent)
-			release_resource(dev_res->res);
+	list_for_each_entry(dev_res, head, list) {
+		struct resource *res = dev_res->res;
+
+		if (res->parent) {
+			dev_printk(KERN_DEBUG, &dev_res->dev->dev,
+				   "BAR %d: released %pR\n",
+				   (int)(res - &dev_res->dev->resource[0]),
+				   res);
+			release_resource(res);
+		}
+	}
 	/* Restore start/end/flags from saved list */
 	list_for_each_entry(save_res, &save_head, list) {
 		struct resource *res = save_res->res;
-- 
1.8.4.5

[PATCH v3 12/51] PCI: Simplify res reference using in __assign_resourcs_sorted

[Yinghai Lu]

Use res instead of dev_res->res.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 6dff258..06664db 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -453,6 +453,7 @@ static void __assign_resources_sorted(struct list_head *head,
 	struct pci_dev_resource *dev_res, *tmp_res;
 	unsigned long fail_type;
 	resource_size_t add_align;
+	struct resource *res;
 
 	/* Check if optional add_size is there */
 	if (!realloc_head || list_empty(realloc_head))
@@ -468,8 +469,8 @@ static void __assign_resources_sorted(struct list_head *head,
 
 	/* Update res in head list with add_size in realloc_head list */
 	list_for_each_entry(dev_res, head, list) {
-		dev_res->res->end += get_res_add_size(realloc_head,
-							dev_res->res);
+		res = dev_res->res;
+		res->end += get_res_add_size(realloc_head, res);
 
 		/*
 		 * There are two kinds of additional resources in the list:
@@ -478,16 +479,16 @@ static void __assign_resources_sorted(struct list_head *head,
 		 * 2. resource with IORESOURCE_SIZEALIGN
 		 *    update size above already change alignment.
 		 */
-		if (!(dev_res->res->flags & IORESOURCE_STARTALIGN))
+		if (!(res->flags & IORESOURCE_STARTALIGN))
 			continue;
 
-		add_align = get_res_add_align(realloc_head, dev_res->res);
+		add_align = get_res_add_align(realloc_head, res);
 
 		if (add_align) {
-			resource_size_t r_size = resource_size(dev_res->res);
+			resource_size_t r_size = resource_size(res);
 
-			dev_res->res->start = add_align;
-			dev_res->res->end = add_align + r_size - 1;
+			res->start = add_align;
+			res->end = add_align + r_size - 1;
                }
 	}
 
@@ -509,21 +510,21 @@ static void __assign_resources_sorted(struct list_head *head,
 	/* check failed type */
 	fail_type = pci_fail_res_type_mask(&local_fail_head);
 	/* remove not need to be released assigned res from head list etc */
-	list_for_each_entry_safe(dev_res, tmp_res, head, list)
-		if (dev_res->res->parent &&
-		    !pci_need_to_release(fail_type, dev_res->res)) {
+	list_for_each_entry_safe(dev_res, tmp_res, head, list) {
+		res = dev_res->res;
+		if (res->parent && !pci_need_to_release(fail_type, res)) {
 			/* remove it from realloc_head list */
-			remove_from_list(realloc_head, dev_res->res);
-			remove_from_list(&save_head, dev_res->res);
+			remove_from_list(realloc_head, res);
+			remove_from_list(&save_head, res);
 			list_del(&dev_res->list);
 			kfree(dev_res);
 		}
+	}
 
 	free_list(&local_fail_head);
 	/* Release assigned resource */
 	list_for_each_entry(dev_res, head, list) {
-		struct resource *res = dev_res->res;
-
+		res = dev_res->res;
 		if (res->parent) {
 			dev_printk(KERN_DEBUG, &dev_res->dev->dev,
 				   "BAR %d: released %pR\n",
@@ -534,8 +535,7 @@ static void __assign_resources_sorted(struct list_head *head,
 	}
 	/* Restore start/end/flags from saved list */
 	list_for_each_entry(save_res, &save_head, list) {
-		struct resource *res = save_res->res;
-
+		res = save_res->res;
 		res->start = save_res->start;
 		res->end = save_res->end;
 		res->flags = save_res->flags;
-- 
1.8.4.5

[PATCH v3 13/51] PCI: Separate realloc list checking after allocation

[Yinghai Lu]

We check the realloc list, as list must be empty after allocation.

Separate the realloc list checking to another function.

Add checking that is missed in acpiphp driver.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
Cc: Len Brown <lenb@kernel.org>
Cc: linux-acpi@vger.kernel.org
---
 drivers/pci/hotplug/acpiphp_glue.c |  1 +
 drivers/pci/pci.h                  |  1 +
 drivers/pci/setup-bus.c            | 11 ++++++++---
 3 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/hotplug/acpiphp_glue.c b/drivers/pci/hotplug/acpiphp_glue.c
index ff53856..134caee 100644
--- a/drivers/pci/hotplug/acpiphp_glue.c
+++ b/drivers/pci/hotplug/acpiphp_glue.c
@@ -507,6 +507,7 @@ static void enable_slot(struct acpiphp_slot *slot)
 		}
 	}
 	__pci_bus_assign_resources(bus, &add_list, NULL);
+	__pci_bus_check_realloc(&add_list);
 
 	acpiphp_sanitize_bus(bus);
 	pcie_bus_configure_settings(bus);
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 4ff0ff1..2b83977 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -235,6 +235,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus,
 void __pci_bus_assign_resources(const struct pci_bus *bus,
 				struct list_head *realloc_head,
 				struct list_head *fail_head);
+void __pci_bus_check_realloc(struct list_head *realloc_head);
 bool pci_bus_clip_resource(struct pci_dev *dev, int idx);
 
 void pci_reassigndev_resource_alignment(struct pci_dev *dev);
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 06664db..f30225c 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -350,6 +350,11 @@ out:
 	}
 }
 
+void __pci_bus_check_realloc(struct list_head *realloc_head)
+{
+	BUG_ON(!list_empty(realloc_head));
+}
+
 /**
  * assign_requested_resources_sorted() - satisfy resource requests
  *
@@ -1861,7 +1866,7 @@ again:
 	/* Depth last, allocate resources and update the hardware. */
 	__pci_bus_assign_resources(bus, add_list, &fail_head);
 	if (add_list)
-		BUG_ON(!list_empty(add_list));
+		__pci_bus_check_realloc(add_list);
 	tried_times++;
 
 	/* any device complain? */
@@ -1936,7 +1941,7 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
 again:
 	__pci_bus_size_bridges(parent, &add_list);
 	__pci_bridge_assign_resources(bridge, &add_list, &fail_head);
-	BUG_ON(!list_empty(&add_list));
+	__pci_bus_check_realloc(&add_list);
 	tried_times++;
 
 	if (list_empty(&fail_head))
@@ -1995,6 +2000,6 @@ void pci_assign_unassigned_bus_resources(struct pci_bus *bus)
 							 &add_list);
 	up_read(&pci_bus_sem);
 	__pci_bus_assign_resources(bus, &add_list, NULL);
-	BUG_ON(!list_empty(&add_list));
+	__pci_bus_check_realloc(&add_list);
 }
 EXPORT_SYMBOL_GPL(pci_assign_unassigned_bus_resources);
-- 
1.8.4.5

[PATCH v3 14/51] PCI: Add __add_to_list()

[Yinghai Lu]

to take alt_size, alt_align.

preparation patch for alt_size support.


Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 51 ++++++++++++++++++++++++++++++-------------------
 1 file changed, 31 insertions(+), 20 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index f30225c..57b5c09 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -66,6 +66,8 @@ struct pci_dev_resource {
 	resource_size_t end;
 	resource_size_t add_size;
 	resource_size_t min_align;
+	resource_size_t alt_size;
+	resource_size_t alt_align;
 	unsigned long flags;
 };
 
@@ -88,15 +90,16 @@ static void free_list(struct list_head *head)
  * @add_size:	additional size to be optionally added
  *              to the resource
  */
-static int add_to_list(struct list_head *head,
+static int __add_to_list(struct list_head *head,
 		 struct pci_dev *dev, struct resource *res,
-		 resource_size_t add_size, resource_size_t min_align)
+		 resource_size_t add_size, resource_size_t min_align,
+		 resource_size_t alt_size, resource_size_t alt_align)
 {
 	struct pci_dev_resource *tmp;
 
 	tmp = kzalloc(sizeof(*tmp), GFP_KERNEL);
 	if (!tmp) {
-		pr_warn("add_to_list: kmalloc() failed!\n");
+		pr_warn("__add_to_list: kmalloc() failed!\n");
 		return -ENOMEM;
 	}
 
@@ -107,12 +110,20 @@ static int add_to_list(struct list_head *head,
 	tmp->flags = res->flags;
 	tmp->add_size = add_size;
 	tmp->min_align = min_align;
+	tmp->alt_size = alt_size;
+	tmp->alt_align = alt_align;
 
 	list_add(&tmp->list, head);
 
 	return 0;
 }
 
+static int add_to_list(struct list_head *head,
+		 struct pci_dev *dev, struct resource *res)
+{
+	return __add_to_list(head, dev, res, 0, 0, 0, 0);
+}
+
 static void remove_from_list(struct list_head *head,
 				 struct resource *res)
 {
@@ -378,9 +389,7 @@ static void assign_requested_resources_sorted(struct list_head *head,
 		if (resource_size(res) &&
 		    pci_assign_resource(dev_res->dev, idx)) {
 			if (fail_head)
-				add_to_list(fail_head, dev_res->dev, res,
-					    0 /* don't care */,
-					    0 /* don't care */);
+				add_to_list(fail_head, dev_res->dev, res);
 			reset_resource(res);
 		}
 	}
@@ -466,7 +475,7 @@ static void __assign_resources_sorted(struct list_head *head,
 
 	/* Save original start, end, flags etc at first */
 	list_for_each_entry(dev_res, head, list) {
-		if (add_to_list(&save_head, dev_res->dev, dev_res->res, 0, 0)) {
+		if (add_to_list(&save_head, dev_res->dev, dev_res->res)) {
 			free_list(&save_head);
 			goto requested_and_reassign;
 		}
@@ -1019,8 +1028,8 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 	b_res->end = b_res->start + size0 - 1;
 	b_res->flags |= IORESOURCE_STARTALIGN;
 	if (size1 > size0 && realloc_head) {
-		add_to_list(realloc_head, bus->self, b_res, size1-size0,
-			    min_align);
+		__add_to_list(realloc_head, bus->self, b_res,
+			      size1 - size0, min_align, 0, 0);
 		dev_printk(KERN_DEBUG, &bus->self->dev, "bridge window %pR to %pR add_size %llx\n",
 			   b_res, &bus->busn_res,
 			   (unsigned long long)size1-size0);
@@ -1222,7 +1231,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 				add_to_align_test_list(&align_test_add_list,
 							align, r_size);
 				r->end = r->start - 1;
-				add_to_list(realloc_head, dev, r, r_size, 0/* don't care */);
+				__add_to_list(realloc_head, dev, r,
+					      r_size, align, 0, 0);
 				sum_add_size += r_size;
 				if (align > max_add_align)
 					max_add_align = align;
@@ -1293,8 +1303,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	b_res->end = size0 + min_align - 1;
 	b_res->flags |= IORESOURCE_STARTALIGN;
 	if (size1 > size0 && realloc_head) {
-		add_to_list(realloc_head, bus->self, b_res, size1 - size0,
-				min_add_align);
+		__add_to_list(realloc_head, bus->self, b_res, size1 - size0,
+				min_add_align, 0, 0);
 		dev_printk(KERN_DEBUG, &bus->self->dev, "bridge window %pR to %pR add_size %llx add_align %llx\n",
 			   b_res, &bus->busn_res,
 			   (unsigned long long) (size1 - size0),
@@ -1331,8 +1341,8 @@ static void pci_bus_size_cardbus(struct pci_bus *bus,
 	b_res[0].flags |= IORESOURCE_IO | IORESOURCE_STARTALIGN;
 	if (realloc_head) {
 		b_res[0].end -= pci_cardbus_io_size;
-		add_to_list(realloc_head, bridge, b_res, pci_cardbus_io_size,
-				pci_cardbus_io_size);
+		__add_to_list(realloc_head, bridge, b_res,
+			      pci_cardbus_io_size, pci_cardbus_io_size, 0, 0);
 	}
 
 handle_b_res_1:
@@ -1343,8 +1353,8 @@ handle_b_res_1:
 	b_res[1].flags |= IORESOURCE_IO | IORESOURCE_STARTALIGN;
 	if (realloc_head) {
 		b_res[1].end -= pci_cardbus_io_size;
-		add_to_list(realloc_head, bridge, b_res+1, pci_cardbus_io_size,
-				 pci_cardbus_io_size);
+		__add_to_list(realloc_head, bridge, b_res + 1,
+			      pci_cardbus_io_size, pci_cardbus_io_size, 0, 0);
 	}
 
 handle_b_res_2:
@@ -1381,8 +1391,9 @@ handle_b_res_2:
 				  IORESOURCE_STARTALIGN;
 		if (realloc_head) {
 			b_res[2].end -= pci_cardbus_mem_size;
-			add_to_list(realloc_head, bridge, b_res+2,
-				 pci_cardbus_mem_size, pci_cardbus_mem_size);
+			__add_to_list(realloc_head, bridge, b_res + 2,
+				pci_cardbus_mem_size, pci_cardbus_mem_size,
+				0, 0);
 		}
 
 		/* reduce that to half */
@@ -1397,8 +1408,8 @@ handle_b_res_3:
 	b_res[3].flags |= IORESOURCE_MEM | IORESOURCE_STARTALIGN;
 	if (realloc_head) {
 		b_res[3].end -= b_res_3_size;
-		add_to_list(realloc_head, bridge, b_res+3, b_res_3_size,
-				 pci_cardbus_mem_size);
+		__add_to_list(realloc_head, bridge, b_res + 3,
+				b_res_3_size, pci_cardbus_mem_size, 0, 0);
 	}
 
 handle_done:
-- 
1.8.4.5

[PATCH v3 15/51] PCI: Cache window alignment value

[Yinghai Lu]

There are several calling to window_alignment().
And we will have more for alt_size support.

Cache the value instead of keeping on getting it.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 57b5c09..1b5fbca 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1203,6 +1203,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 					mask | IORESOURCE_PREFETCH, type);
 	LIST_HEAD(align_test_list);
 	LIST_HEAD(align_test_add_list);
+	resource_size_t window_align;
 
 	if (!b_res)
 		return -ENOSPC;
@@ -1212,6 +1213,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		min_size = 0;
 	}
 
+	window_align = window_alignment(bus, b_res->flags);
+
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		int i;
 
@@ -1272,10 +1275,10 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		}
 	}
 
-	max_align = max(max_align, window_alignment(bus, b_res->flags));
+	max_align = max(max_align, window_align);
 	if (size || min_size) {
 		min_align = calculate_mem_align(&align_test_list, max_align,
-				 size, window_alignment(bus, b_res->flags));
+						size, window_align);
 		size0 = calculate_memsize(size, min_size,
 				  resource_size(b_res), min_align);
 	}
@@ -1286,7 +1289,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	if (sum_add_size > size && realloc_head) {
 		min_add_align = calculate_mem_align(&align_test_add_list,
 					max_add_align, sum_add_size,
-					window_alignment(bus, b_res->flags));
+					window_align);
 		size1 = calculate_memsize(sum_add_size, min_size,
 				 resource_size(b_res), min_add_align);
 	}
-- 
1.8.4.5

[PATCH v3 16/51] PCI: Check if resource is allocated before pci_assign

[Yinghai Lu]

Skip allocated resource in the list, as pci_assign_resource()
only can handle not assigned resource. And we could have
assigned resource already in the list before alt_size trying.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 1b5fbca..1622ad2 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -386,7 +386,7 @@ static void assign_requested_resources_sorted(struct list_head *head,
 	list_for_each_entry(dev_res, head, list) {
 		res = dev_res->res;
 		idx = res - &dev_res->dev->resource[0];
-		if (resource_size(res) &&
+		if (!res->parent && resource_size(res) &&
 		    pci_assign_resource(dev_res->dev, idx)) {
 			if (fail_head)
 				add_to_list(fail_head, dev_res->dev, res);
-- 
1.8.4.5

[PATCH v3 17/51] PCI: Separate out save_resources/restore_resource

[Yinghai Lu]

will reuse it in alt_size support.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 49 ++++++++++++++++++++++++++++++-------------------
 1 file changed, 30 insertions(+), 19 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 1622ad2..2e3d00b 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -436,6 +436,29 @@ static bool pci_need_to_release(unsigned long mask, struct resource *res)
 	return false;	/* should not get here */
 }
 
+static bool save_resources(struct list_head *head,
+			   struct list_head *save_head)
+{
+	struct pci_dev_resource *dev_res;
+
+	/* Save original start, end, flags etc at first */
+	list_for_each_entry(dev_res, head, list)
+		if (add_to_list(save_head, dev_res->dev, dev_res->res)) {
+			free_list(save_head);
+			return false;
+		}
+
+	return true;
+}
+
+static void restore_resource(struct pci_dev_resource *save_res,
+			     struct resource *res)
+{
+	res->start = save_res->start;
+	res->end = save_res->end;
+	res->flags = save_res->flags;
+}
+
 static void __assign_resources_sorted(struct list_head *head,
 				 struct list_head *realloc_head,
 				 struct list_head *fail_head)
@@ -473,13 +496,8 @@ static void __assign_resources_sorted(struct list_head *head,
 	if (!realloc_head || list_empty(realloc_head))
 		goto requested_and_reassign;
 
-	/* Save original start, end, flags etc at first */
-	list_for_each_entry(dev_res, head, list) {
-		if (add_to_list(&save_head, dev_res->dev, dev_res->res)) {
-			free_list(&save_head);
-			goto requested_and_reassign;
-		}
-	}
+	if (!save_resources(head, &save_head))
+		goto requested_and_reassign;
 
 	/* Update res in head list with add_size in realloc_head list */
 	list_for_each_entry(dev_res, head, list) {
@@ -548,12 +566,9 @@ static void __assign_resources_sorted(struct list_head *head,
 		}
 	}
 	/* Restore start/end/flags from saved list */
-	list_for_each_entry(save_res, &save_head, list) {
-		res = save_res->res;
-		res->start = save_res->start;
-		res->end = save_res->end;
-		res->flags = save_res->flags;
-	}
+	list_for_each_entry(save_res, &save_head, list)
+		restore_resource(save_res, save_res->res);
+
 	free_list(&save_head);
 
 requested_and_reassign:
@@ -1917,9 +1932,7 @@ again:
 	list_for_each_entry(fail_res, &fail_head, list) {
 		struct resource *res = fail_res->res;
 
-		res->start = fail_res->start;
-		res->end = fail_res->end;
-		res->flags = fail_res->flags;
+		restore_resource(fail_res, res);
 		if (fail_res->dev->subordinate)
 			res->flags = 0;
 	}
@@ -1983,9 +1996,7 @@ again:
 	list_for_each_entry(fail_res, &fail_head, list) {
 		struct resource *res = fail_res->res;
 
-		res->start = fail_res->start;
-		res->end = fail_res->end;
-		res->flags = fail_res->flags;
+		restore_resource(fail_res, res);
 		if (fail_res->dev->subordinate)
 			res->flags = 0;
 	}
-- 
1.8.4.5

[PATCH v3 18/51] PCI: Move comment to pci_need_to_release()

[Yinghai Lu]

Move comment from caller to callee, as we will have one new caller
for alt_size support later.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 27 +++++++++++++++------------
 1 file changed, 15 insertions(+), 12 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 2e3d00b..f8b9a24 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -415,6 +415,20 @@ static unsigned long pci_fail_res_type_mask(struct list_head *fail_head)
 
 static bool pci_need_to_release(unsigned long mask, struct resource *res)
 {
+	/*
+	 * Separate three resource type checking if we need to release
+	 * assigned resource.
+	 *	1. if there is io port assign fail, will release assigned
+	 *	   io port.
+	 *	2. if there is pref mmio assign fail, release assigned
+	 *	   pref mmio.
+	 *	   if assigned pref mmio's parent is non-pref mmio and there
+	 *	   is non-pref mmio assign fail, will release that assigned
+	 *	   pref mmio.
+	 *	3. if there is non-pref mmio assign fail or pref mmio
+	 *	   assigned fail, will release assigned non-pref mmio.
+	 */
+
 	if (res->flags & IORESOURCE_IO)
 		return !!(mask & IORESOURCE_IO);
 
@@ -471,19 +485,8 @@ static void __assign_resources_sorted(struct list_head *head,
 	 *  if could do that, could get out early.
 	 *  if could not do that, we still try to assign requested at first,
 	 *    then try to reassign add_size for some resources.
-	 *
-	 * Separate three resource type checking if we need to release
-	 * assigned resource after requested + add_size try.
-	 *	1. if there is io port assign fail, will release assigned
-	 *	   io port.
-	 *	2. if there is pref mmio assign fail, release assigned
-	 *	   pref mmio.
-	 *	   if assigned pref mmio's parent is non-pref mmio and there
-	 *	   is non-pref mmio assign fail, will release that assigned
-	 *	   pref mmio.
-	 *	3. if there is non-pref mmio assign fail or pref mmio
-	 *	   assigned fail, will release assigned non-pref mmio.
 	 */
+
 	LIST_HEAD(save_head);
 	LIST_HEAD(local_fail_head);
 	struct pci_dev_resource *save_res;
-- 
1.8.4.5

[PATCH v3 19/51] PCI: Separate must+optional assigning to another function

[Yinghai Lu]

__assign_resources_sorted() is getting too big if we put alt_size support
into it.  Split must_add assigning code out to another function.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 47 +++++++++++++++++++++++++++--------------------
 1 file changed, 27 insertions(+), 20 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index f8b9a24..d1f9e19 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -473,20 +473,9 @@ static void restore_resource(struct pci_dev_resource *save_res,
 	res->flags = save_res->flags;
 }
 
-static void __assign_resources_sorted(struct list_head *head,
-				 struct list_head *realloc_head,
-				 struct list_head *fail_head)
+static bool __assign_resources_must_add_sorted(struct list_head *head,
+				 struct list_head *realloc_head)
 {
-	/*
-	 * Should not assign requested resources at first.
-	 *   they could be adjacent, so later reassign can not reallocate
-	 *   them one by one in parent resource window.
-	 * Try to assign requested + add_size at beginning
-	 *  if could do that, could get out early.
-	 *  if could not do that, we still try to assign requested at first,
-	 *    then try to reassign add_size for some resources.
-	 */
-
 	LIST_HEAD(save_head);
 	LIST_HEAD(local_fail_head);
 	struct pci_dev_resource *save_res;
@@ -495,12 +484,8 @@ static void __assign_resources_sorted(struct list_head *head,
 	resource_size_t add_align;
 	struct resource *res;
 
-	/* Check if optional add_size is there */
-	if (!realloc_head || list_empty(realloc_head))
-		goto requested_and_reassign;
-
 	if (!save_resources(head, &save_head))
-		goto requested_and_reassign;
+		return false;
 
 	/* Update res in head list with add_size in realloc_head list */
 	list_for_each_entry(dev_res, head, list) {
@@ -539,7 +524,8 @@ static void __assign_resources_sorted(struct list_head *head,
 			remove_from_list(realloc_head, dev_res->res);
 		free_list(&save_head);
 		free_list(head);
-		return;
+
+		return true;
 	}
 
 	/* check failed type */
@@ -574,7 +560,28 @@ static void __assign_resources_sorted(struct list_head *head,
 
 	free_list(&save_head);
 
-requested_and_reassign:
+	return false;
+}
+
+static void __assign_resources_sorted(struct list_head *head,
+				 struct list_head *realloc_head,
+				 struct list_head *fail_head)
+{
+	/*
+	 * Should not assign requested resources at first.
+	 *   they could be adjacent, so later reassign can not reallocate
+	 *   them one by one in parent resource window.
+	 * Try to assign requested + add_size at beginning
+	 *  if could do that, could get out early.
+	 *  if could not do that, we still try to assign requested at first,
+	 *    then try to reassign add_size for some resources.
+	 */
+
+	/* Check must+optional add */
+	if (realloc_head && !list_empty(realloc_head) &&
+	    __assign_resources_must_add_sorted(head, realloc_head))
+		return;
+
 	__sort_resources(head);
 
 	/* Satisfy the must-have resource requests */
-- 
1.8.4.5

[PATCH v3 20/51] PCI: Skip must+optional if there is no optional addon

[Yinghai Lu]

If the bridge does not support hotplug or no child with sriov support
we could get out early and don't try must+optional allocation.

Also in the loop that update res with optional add info, skip resource
that add_size is 0.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 32 +++++++++++++++++++++++++++++---
 1 file changed, 29 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index d1f9e19..64ef516 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -450,6 +450,24 @@ static bool pci_need_to_release(unsigned long mask, struct resource *res)
 	return false;	/* should not get here */
 }
 
+static bool __has_addon(struct list_head *head,
+			struct list_head *realloc_head)
+{
+	int add_count = 0;
+	struct pci_dev_resource *dev_res, *tmp_res;
+
+	/* check if we have add really */
+	list_for_each_entry(dev_res, head, list) {
+		tmp_res = res_to_dev_res(realloc_head, dev_res->res);
+		if (!tmp_res || !tmp_res->add_size)
+			continue;
+
+		add_count++;
+	}
+
+	return add_count != 0;
+}
+
 static bool save_resources(struct list_head *head,
 			   struct list_head *save_head)
 {
@@ -481,16 +499,24 @@ static bool __assign_resources_must_add_sorted(struct list_head *head,
 	struct pci_dev_resource *save_res;
 	struct pci_dev_resource *dev_res, *tmp_res;
 	unsigned long fail_type;
-	resource_size_t add_align;
+	resource_size_t add_align, add_size;
 	struct resource *res;
 
+	if (!__has_addon(head, realloc_head))
+		return false;
+
 	if (!save_resources(head, &save_head))
 		return false;
 
 	/* Update res in head list with add_size in realloc_head list */
 	list_for_each_entry(dev_res, head, list) {
 		res = dev_res->res;
-		res->end += get_res_add_size(realloc_head, res);
+		add_size = get_res_add_size(realloc_head, res);
+
+		if (!add_size)
+			continue;
+
+		res->end += add_size;
 
 		/*
 		 * There are two kinds of additional resources in the list:
@@ -578,7 +604,7 @@ static void __assign_resources_sorted(struct list_head *head,
 	 */
 
 	/* Check must+optional add */
-	if (realloc_head && !list_empty(realloc_head) &&
+	if (realloc_head &&
 	    __assign_resources_must_add_sorted(head, realloc_head))
 		return;
 
-- 
1.8.4.5

[PATCH v3 21/51] PCI: Move saved required resource list out of must+optional assigning

[Yinghai Lu]

We will need to share that saved list for alt_size support.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 30 ++++++++++++++++--------------
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 64ef516..1c0b4c5 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -456,6 +456,9 @@ static bool __has_addon(struct list_head *head,
 	int add_count = 0;
 	struct pci_dev_resource *dev_res, *tmp_res;
 
+	if (!realloc_head)
+		return false;
+
 	/* check if we have add really */
 	list_for_each_entry(dev_res, head, list) {
 		tmp_res = res_to_dev_res(realloc_head, dev_res->res);
@@ -492,9 +495,9 @@ static void restore_resource(struct pci_dev_resource *save_res,
 }
 
 static bool __assign_resources_must_add_sorted(struct list_head *head,
+				 struct list_head *save_head,
 				 struct list_head *realloc_head)
 {
-	LIST_HEAD(save_head);
 	LIST_HEAD(local_fail_head);
 	struct pci_dev_resource *save_res;
 	struct pci_dev_resource *dev_res, *tmp_res;
@@ -502,12 +505,6 @@ static bool __assign_resources_must_add_sorted(struct list_head *head,
 	resource_size_t add_align, add_size;
 	struct resource *res;
 
-	if (!__has_addon(head, realloc_head))
-		return false;
-
-	if (!save_resources(head, &save_head))
-		return false;
-
 	/* Update res in head list with add_size in realloc_head list */
 	list_for_each_entry(dev_res, head, list) {
 		res = dev_res->res;
@@ -548,7 +545,6 @@ static bool __assign_resources_must_add_sorted(struct list_head *head,
 		/* Remove head list from realloc_head list */
 		list_for_each_entry(dev_res, head, list)
 			remove_from_list(realloc_head, dev_res->res);
-		free_list(&save_head);
 		free_list(head);
 
 		return true;
@@ -562,7 +558,7 @@ static bool __assign_resources_must_add_sorted(struct list_head *head,
 		if (res->parent && !pci_need_to_release(fail_type, res)) {
 			/* remove it from realloc_head list */
 			remove_from_list(realloc_head, res);
-			remove_from_list(&save_head, res);
+			remove_from_list(save_head, res);
 			list_del(&dev_res->list);
 			kfree(dev_res);
 		}
@@ -581,11 +577,9 @@ static bool __assign_resources_must_add_sorted(struct list_head *head,
 		}
 	}
 	/* Restore start/end/flags from saved list */
-	list_for_each_entry(save_res, &save_head, list)
+	list_for_each_entry(save_res, save_head, list)
 		restore_resource(save_res, save_res->res);
 
-	free_list(&save_head);
-
 	return false;
 }
 
@@ -603,16 +597,24 @@ static void __assign_resources_sorted(struct list_head *head,
 	 *    then try to reassign add_size for some resources.
 	 */
 
+	LIST_HEAD(save_head);
+
 	/* Check must+optional add */
-	if (realloc_head &&
-	    __assign_resources_must_add_sorted(head, realloc_head))
+	if (__has_addon(head, realloc_head) &&
+	    save_resources(head, &save_head) &&
+	    __assign_resources_must_add_sorted(head, &save_head,
+					       realloc_head)) {
+		free_list(&save_head);
 		return;
+	}
 
 	__sort_resources(head);
 
 	/* Satisfy the must-have resource requests */
 	assign_requested_resources_sorted(head, fail_head);
 
+	free_list(&save_head);
+
 	/* Try to satisfy any additional optional resource
 		requests */
 	if (realloc_head)
-- 
1.8.4.5

[PATCH v3 22/51] PCI: Add alt_size allocation support

[Yinghai Lu]

On system with several pcie switches, BIOS allocate very tight resources
to the bridge bar, and it is not aligned to min_align as kernel allocation
code.

For example:
	02:03.0---0c:00.0---0d:04.0---18:00.0
18:00.0 need 0x10000000, and 0x00010000.
BIOS only allocate 0x10100000 to 0d:04.0 and above bridges.
Later after using /sys/bus/pci/devices/0000:0c:00.0/remove to remove 0c:00.0,
rescan with /sys/bus/pci/rescan can not allocate 0x18000000 to 0c:00.0.

another example:
00:1c.0-[02-21]----00.0-[03-21]--+-01.0-[04-12]----00.0-[05-12]----19.0-[06-12]----00.0
                                 +-05.0-[13]--
                                 +-07.0-[14-20]----00.0-[15-20]--+-08.0-[16]--+-00.0
                                 |                               |            \-00.1
                                 |                               +-14.0-[17]----00.0
                                 |                               \-19.0-[18-20]----00.0
                                 \-09.0-[21]--
06:00.0 need 0x4000000 and 0x800000.
BIOS only allocate 0x4800000 to 05:19.0 and 04:00.0.
when 05:19.0 get removed via /sys/bus/pci/devices/0000:05:19.0/remove,
rescan with /sys/bus/pci/rescan will fail.
 pci 0000:05:19.0: BAR 14: no space for [mem size 0x06000000]
 pci 0000:05:19.0: BAR 14: failed to assign [mem size 0x06000000]
 pci 0000:06:00.0: BAR 2: no space for [mem size 0x04000000 64bit]
 pci 0000:06:00.0: BAR 2: failed to assign [mem size 0x04000000 64bit]
 pci 0000:06:00.0: BAR 0: no space for [mem size 0x00800000]
 pci 0000:06:00.0: BAR 0: failed to assign [mem size 0x00800000]
current code try to use align 0x2000000 and size 0x6000000, but parent
bridge only have 0x4800000.

Introduce alt_align/alt_size and store them in realloc list in addition
to addon info, and will try it after min_align/min_size allocation fails.

The alt_align is max_align, and alt_size is aligned size with bridge
minimum window alignment.

on my test setup:
	00:1c.7---61:00.0---62:00.0
62:00.0 needs 0x800000 and 0x20000.
and 00:1c.7 only have 9M allocated for mmio, with this patch we have

 pci 0000:61:00.0: bridge window [mem 0x00400000-0x00ffffff] to [bus 62] add_size 0 add_align 0 alt_size 900000 alt_align 800000 must_size c00000 must_align 400000
 pci 0000:61:00.0: BAR 14: no space for [mem size 0x00c00000]
 pci 0000:61:00.0: BAR 14: failed to assign [mem size 0x00c00000]
 pci 0000:61:00.0: BAR 14: assigned [mem 0xdf000000-0xdf8fffff]
 pci 0000:62:00.0: BAR 0: assigned [mem 0xdf000000-0xdf7fffff pref]
 pci 0000:62:00.0: BAR 1: assigned [mem 0xdf800000-0xdf81ffff]
 pci 0000:61:00.0: PCI bridge to [bus 62]
 pci 0000:61:00.0:   bridge window [io  0x6000-0x6fff]
 pci 0000:61:00.0:   bridge window [mem 0xdf000000-0xdf8fffff]
 pci 0000:00:1c.7: PCI bridge to [bus 61-68]
 pci 0000:00:1c.7:   bridge window [io  0x6000-0x6fff]
 pci 0000:00:1c.7:   bridge window [mem 0xdf000000-0xdf8fffff]

so for 61:00.0 first try with 12M fails, and second try with 9M the
alt_size works. Later 62:00.0 get correct resource allocated too.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=100451
Reported-by: Yijing Wang <wangyijing@huawei.com>
Tested-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 203 +++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 191 insertions(+), 12 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 1c0b4c5..9da8b23 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -324,7 +324,7 @@ static void reassign_resources_sorted(struct list_head *realloc_head,
 {
 	struct resource *res;
 	struct pci_dev_resource *add_res, *tmp;
-	resource_size_t add_size, align;
+	resource_size_t add_size, align, r_size;
 	int idx;
 
 	list_for_each_entry_safe(add_res, tmp, realloc_head, list) {
@@ -340,12 +340,23 @@ static void reassign_resources_sorted(struct list_head *realloc_head,
 		idx = res - &add_res->dev->resource[0];
 		add_size = add_res->add_size;
 		align = add_res->min_align;
-		if (!resource_size(res)) {
+		if (!add_size || !align) /* alt_size only */
+			goto out;
+
+		r_size = resource_size(res);
+		if (!r_size) {
 			res->start = align;
 			res->end = res->start + add_size - 1;
 			if (pci_assign_resource(add_res->dev, idx))
 				reset_resource(res);
 		} else {
+			/* could just assigned with alt, add difference ? */
+			resource_size_t must_size;
+
+			must_size = add_res->end - add_res->start + 1;
+			if (r_size < must_size)
+				add_size += must_size - r_size;
+
 			res->flags |= add_res->flags &
 				 (IORESOURCE_STARTALIGN|IORESOURCE_SIZEALIGN);
 			if (pci_reassign_resource(add_res->dev, idx,
@@ -583,6 +594,104 @@ static bool __assign_resources_must_add_sorted(struct list_head *head,
 	return false;
 }
 
+static bool __has_alt(struct list_head *head,
+		    struct list_head *realloc_head)
+{
+	int alt_count = 0;
+	struct pci_dev_resource *dev_res, *alt_res;
+
+	if (!realloc_head)
+		return false;
+
+	/* check if we have alt really */
+	list_for_each_entry(dev_res, head, list) {
+		alt_res = res_to_dev_res(realloc_head, dev_res->res);
+		if (!alt_res || !alt_res->alt_size)
+			continue;
+
+		alt_count++;
+	}
+
+	if (!alt_count)
+		return false;
+
+	return true;
+}
+
+static void __assign_resources_alt_sorted(struct list_head *head,
+				 struct list_head *save_head,
+				 struct list_head *realloc_head,
+				 struct list_head *local_fail_head)
+{
+	LIST_HEAD(local_alt_fail_head);
+	struct pci_dev_resource *dev_res;
+	struct pci_dev_resource *alt_res, *fail_res, *save_res;
+	unsigned long fail_type;
+	struct resource *res;
+
+	/* check failed type */
+	fail_type = pci_fail_res_type_mask(local_fail_head);
+	/* release resource with same type that failes */
+	list_for_each_entry(dev_res, head, list) {
+		res = dev_res->res;
+		if (res->parent) {
+			if (!pci_need_to_release(fail_type, res))
+				continue;
+
+			/*
+			 * have to use saved info, as resource that does not
+			 * have addon/alt is not in realloc list.
+			 */
+			save_res = res_to_dev_res(save_head, res);
+			if (!save_res)
+				continue;
+
+			dev_printk(KERN_DEBUG, &dev_res->dev->dev,
+				   "BAR %d: released %pR\n",
+				   (int)(res - &dev_res->dev->resource[0]),
+				   res);
+			release_resource(dev_res->res);
+			restore_resource(save_res, res);
+		} else {
+			/* restore fail one */
+			fail_res = res_to_dev_res(local_fail_head, res);
+			if (fail_res) {
+				restore_resource(fail_res, res);
+				remove_from_list(local_fail_head, res);
+			}
+		}
+
+		alt_res = res_to_dev_res(realloc_head, res);
+		if (!alt_res || !alt_res->alt_size)
+			continue;
+
+		/* change res to alt */
+		if (res->flags & IORESOURCE_STARTALIGN)
+			res->start = alt_res->alt_align;
+		else
+			res->start = 0;
+		res->end = res->start + alt_res->alt_size - 1;
+	}
+
+	__sort_resources(head);
+	/* Satisfy the alt resource requests */
+	assign_requested_resources_sorted(head, &local_alt_fail_head);
+
+	/* update local fail list */
+	list_for_each_entry(fail_res, &local_alt_fail_head, list) {
+		res = fail_res->res;
+		dev_res = res_to_dev_res(realloc_head, res);
+		/* change res back to must */
+		if (dev_res && dev_res->alt_size)
+			restore_resource(dev_res, res);
+
+		if (!res_to_dev_res(local_fail_head, res))
+			add_to_list(local_fail_head, fail_res->dev, res);
+		reset_resource(res);
+	}
+	free_list(&local_alt_fail_head);
+}
+
 static void __assign_resources_sorted(struct list_head *head,
 				 struct list_head *realloc_head,
 				 struct list_head *fail_head)
@@ -598,6 +707,8 @@ static void __assign_resources_sorted(struct list_head *head,
 	 */
 
 	LIST_HEAD(save_head);
+	LIST_HEAD(local_fail_head);
+	bool has_alt;
 
 	/* Check must+optional add */
 	if (__has_addon(head, realloc_head) &&
@@ -610,15 +721,29 @@ static void __assign_resources_sorted(struct list_head *head,
 
 	__sort_resources(head);
 
+	has_alt = __has_alt(head, realloc_head);
+	if (has_alt && list_empty(&save_head))
+		save_resources(head, &save_head);
+
 	/* Satisfy the must-have resource requests */
-	assign_requested_resources_sorted(head, fail_head);
+	assign_requested_resources_sorted(head, &local_fail_head);
+
+	if (has_alt && !list_empty(&local_fail_head) && !list_empty(&save_head))
+		__assign_resources_alt_sorted(head, &save_head,
+					      realloc_head,
+					      &local_fail_head);
 
 	free_list(&save_head);
 
-	/* Try to satisfy any additional optional resource
-		requests */
+	/* Try to satisfy any additional optional resource requests */
 	if (realloc_head)
 		reassign_resources_sorted(realloc_head, head);
+
+	if (fail_head)
+		list_splice_tail(&local_fail_head, fail_head);
+	else
+		free_list(&local_fail_head);
+
 	free_list(head);
 }
 
@@ -1256,6 +1381,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 					mask | IORESOURCE_PREFETCH, type);
 	LIST_HEAD(align_test_list);
 	LIST_HEAD(align_test_add_list);
+	resource_size_t alt_size = 0, alt_align = 0;
 	resource_size_t window_align;
 
 	if (!b_res)
@@ -1312,6 +1438,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 			if (realloc_head) {
 				resource_size_t add_r_size, add_align;
+				struct pci_dev_resource *dev_res;
 
 				add_r_size = get_res_add_size(realloc_head, r);
 				add_align = get_res_add_align(realloc_head, r);
@@ -1324,6 +1451,17 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 				sum_add_size += r_size + add_r_size;
 				if (add_align > max_add_align)
 					max_add_align = add_align;
+
+				dev_res = res_to_dev_res(realloc_head, r);
+				if (dev_res && dev_res->alt_size) {
+					alt_size += dev_res->alt_size;
+					if (alt_align < dev_res->alt_align)
+						alt_align = dev_res->alt_align;
+				} else if (r_size > 1) {
+					alt_size += r_size;
+					if (alt_align < align)
+						alt_align = align;
+				}
 			}
 		}
 	}
@@ -1337,6 +1475,17 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	}
 	free_align_test_list(&align_test_list);
 
+	if (size0 && realloc_head) {
+		alt_align = max(alt_align, window_align);
+		alt_size = calculate_memsize(alt_size, min_size,
+					     0, window_align);
+		/* must is better ? */
+		if (alt_size >= size0) {
+			alt_align = 0;
+			alt_size = 0;
+		}
+	}
+
 	if (sum_add_size < min_sum_size)
 		sum_add_size = min_sum_size;
 	if (sum_add_size > size && realloc_head) {
@@ -1358,13 +1507,43 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	b_res->start = min_align;
 	b_res->end = size0 + min_align - 1;
 	b_res->flags |= IORESOURCE_STARTALIGN;
-	if (size1 > size0 && realloc_head) {
-		__add_to_list(realloc_head, bus->self, b_res, size1 - size0,
-				min_add_align, 0, 0);
-		dev_printk(KERN_DEBUG, &bus->self->dev, "bridge window %pR to %pR add_size %llx add_align %llx\n",
-			   b_res, &bus->busn_res,
-			   (unsigned long long) (size1 - size0),
-			   (unsigned long long) min_add_align);
+	if (realloc_head) {
+		resource_size_t final_add_size = 0;
+
+		if (size1 > size0)
+			final_add_size = size1 - size0;
+		else
+			min_add_align = 0;
+
+		/*
+		 * realloc list include three type entries
+		 * 1. optional only:
+		 *      add_size != 0, alt_size == 0, must_size == 0
+		 * 2. must only with smaller alt_size.
+		 *      add_size == 0, alt_size != 0, must_size > alt_size
+		 * 3. must + optional:
+		 *      add_size != 0, alt_size < must_size, must_size != 0
+		 *
+		 * So there is no must_size != 0, and alt_size == must_size.
+		 *   in that case, we already set alt_size = 0.
+		 *
+		 * must_align/must_size is not stored directly, and we have
+		 *   dev_res start/end/flags instead.
+		 */
+		if (final_add_size || alt_size) {
+			__add_to_list(realloc_head, bus->self, b_res,
+				      final_add_size, min_add_align,
+				      alt_size, alt_align);
+			dev_printk(KERN_DEBUG, &bus->self->dev,
+				   "bridge window %pR to %pR add_size %llx add_align %llx alt_size %llx alt_align %llx must_size %llx must_align %llx\n",
+				   b_res, &bus->busn_res,
+				   (unsigned long long)final_add_size,
+				   (unsigned long long)min_add_align,
+				   (unsigned long long)alt_size,
+				   (unsigned long long)alt_align,
+				   (unsigned long long)size0,
+				   (unsigned long long)min_align);
+		}
 	}
 	return 0;
 }
-- 
1.8.4.5

[PATCH v3 23/51] PCI: Add support for more than two alt_size under same bridge

[Yinghai Lu]

Need to increase size to make sure it could fit all alt entries.

In the patch, we first select one big size, and then keep reducing
the size and retrying to get the minimum value for alt_size.

Example:
two bridges: one have 8M/8M, and 1M/1M children res.
	     one have 4M/4M, and 1M/1M children res.

Then we have child pridges alt_align/alt_size: 8M/9M, 4M/5M.
Before this patch, parent bridge alt_align/alt_size is 8M/14M
that is wrong.
With this patch	parent bridge alt_align/alt_size: 8M/17M.

At same time, child bridges must align/size: 4M/12M, 2M/6M.
and prarent bridge must align/size: 4M/20M.

So at last, we use 8M/17M as parent bridge alt_align/alt_size.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=100451
Reported-by: Yijing Wang <wangyijing@huawei.com>
Tested-by: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 56 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 54 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 9da8b23..bf28f32 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1335,6 +1335,47 @@ out:
 	return good_align;
 }
 
+static resource_size_t calculate_mem_alt_size(struct list_head *head,
+				resource_size_t max_align, resource_size_t size,
+				resource_size_t align_low)
+{
+	struct align_test_res *p;
+	resource_size_t tmp;
+	resource_size_t good_size, bad_size;
+	int count = 0, order;
+
+	good_size = ALIGN(size, align_low);
+
+	list_for_each_entry(p, head, list)
+		count++;
+
+	if (count <= 1)
+		goto out;
+
+	__sort_align_test(head);
+
+	tmp = max(size, max_align);
+	order = __fls(count);
+	if ((1ULL << order) < count)
+		order++;
+	good_size = ALIGN((tmp << order), align_low);
+	bad_size = ALIGN(size, align_low) - align_low;
+	size = good_size;
+	while (size > bad_size) {
+		/* check if align/size fit all entries */
+		if (is_align_size_good(head, max_align, size, 0))
+			good_size = size;
+		else
+			bad_size = size;
+
+		size = bad_size + ((good_size - bad_size) >> 1);
+		size = round_down(size, align_low);
+	}
+
+out:
+	return good_size;
+}
+
 static inline bool is_optional(int i)
 {
 
@@ -1381,6 +1422,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 					mask | IORESOURCE_PREFETCH, type);
 	LIST_HEAD(align_test_list);
 	LIST_HEAD(align_test_add_list);
+	LIST_HEAD(align_test_alt_list);
 	resource_size_t alt_size = 0, alt_align = 0;
 	resource_size_t window_align;
 
@@ -1454,10 +1496,17 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 				dev_res = res_to_dev_res(realloc_head, r);
 				if (dev_res && dev_res->alt_size) {
+					add_to_align_test_list(
+						&align_test_alt_list,
+						dev_res->alt_align,
+						dev_res->alt_size);
 					alt_size += dev_res->alt_size;
 					if (alt_align < dev_res->alt_align)
 						alt_align = dev_res->alt_align;
 				} else if (r_size > 1) {
+					add_to_align_test_list(
+						&align_test_alt_list,
+						align, r_size);
 					alt_size += r_size;
 					if (alt_align < align)
 						alt_align = align;
@@ -1477,14 +1526,17 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 	if (size0 && realloc_head) {
 		alt_align = max(alt_align, window_align);
-		alt_size = calculate_memsize(alt_size, min_size,
-					     0, window_align);
+		/* need to increase size to fit more alt */
+		alt_size = calculate_mem_alt_size(&align_test_alt_list,
+						  alt_align, alt_size,
+						  window_align);
 		/* must is better ? */
 		if (alt_size >= size0) {
 			alt_align = 0;
 			alt_size = 0;
 		}
 	}
+	free_align_test_list(&align_test_alt_list);
 
 	if (sum_add_size < min_sum_size)
 		sum_add_size = min_sum_size;
-- 
1.8.4.5

[PATCH v3 24/51] PCI: Better support for two alt_size

[Yinghai Lu]

Need to put aligned with max align before not aligned.

For example:
alt align/size: 8M/9M, 4M/8M
before this patch we have 8M/20M.
After this patch we will have 8M/17M.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 78 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index bf28f32..8999ead 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -58,6 +58,20 @@ static inline bool is_before(resource_size_t align1, resource_size_t size1,
 	return false;
 }
 
+static inline bool is_before_alt(resource_size_t align, resource_size_t size1,
+			     resource_size_t size2)
+{
+	resource_size_t size1_left, size2_left;
+
+	/*  aligned is before not aligned */
+	size1_left = size1 & (align - 1);
+	size2_left = size2 & (align - 1);
+	if (!size1_left && size2_left)
+		return true;
+
+	return false;
+}
+
 struct pci_dev_resource {
 	struct list_head list;
 	struct resource *res;
@@ -307,6 +321,42 @@ static void __sort_resources(struct list_head *head)
 	}
 }
 
+static void __sort_resources_alt(struct list_head *head)
+{
+	struct pci_dev_resource *res1, *tmp_res, *res2;
+	resource_size_t align = 0;
+
+
+	__sort_resources(head);
+
+	/* get max align at first */
+	list_for_each_entry(res1, head, list) {
+		resource_size_t align1;
+
+		align1 = pci_resource_alignment(res1->dev, res1->res);
+		if (align1 > align)
+			align = align1;
+	}
+
+	list_for_each_entry_safe(res1, tmp_res, head, list) {
+		resource_size_t size1, size2;
+
+		size1 = resource_size(res1->res);
+
+		/* reorder it */
+		list_for_each_entry(res2, head, list) {
+			if (res2 == res1)
+				break;
+
+			size2 = resource_size(res2->res);
+			if (is_before_alt(align, size1, size2)) {
+				list_move_tail(&res1->list, &res2->list);
+				break;
+			}
+		}
+	}
+}
+
 /**
  * reassign_resources_sorted() - satisfy any additional resource requests
  *
@@ -673,7 +723,7 @@ static void __assign_resources_alt_sorted(struct list_head *head,
 		res->end = res->start + alt_res->alt_size - 1;
 	}
 
-	__sort_resources(head);
+	__sort_resources_alt(head);
 	/* Satisfy the alt resource requests */
 	assign_requested_resources_sorted(head, &local_alt_fail_head);
 
@@ -1267,6 +1317,32 @@ static void __sort_align_test(struct list_head *head)
 	}
 }
 
+static void __sort_align_test_alt(struct list_head *head)
+{
+	struct align_test_res *res1, *tmp_res, *res2;
+	resource_size_t align = 0;
+
+	__sort_align_test(head);
+
+	/* get max align at first */
+	list_for_each_entry(res1, head, list)
+		if (res1->align > align)
+			align = res1->align;
+
+	list_for_each_entry_safe(res1, tmp_res, head, list) {
+		/* reorder it */
+		list_for_each_entry(res2, head, list) {
+			if (res2 == res1)
+				break;
+
+			if (is_before_alt(align, res1->size, res2->size)) {
+				list_move_tail(&res1->list, &res2->list);
+				break;
+			}
+		}
+	}
+}
+
 static bool is_align_size_good(struct list_head *head,
 			resource_size_t min_align, resource_size_t size,
 			resource_size_t start)
@@ -1352,7 +1428,7 @@ static resource_size_t calculate_mem_alt_size(struct list_head *head,
 	if (count <= 1)
 		goto out;
 
-	__sort_align_test(head);
+	__sort_align_test_alt(head);
 
 	tmp = max(size, max_align);
 	order = __fls(count);
-- 
1.8.4.5

[PATCH v3 25/51] PCI: Fix size calculation with old_size on rescan path

[Yinghai Lu]

On booting path, we don't pass realloc at first, and treat all optional just
as must, in some case we can smaller size/align with optional than must only.

04:00.0 has four child bridges: 05:01.0, 05:02.0, 05:03.0, 05:04.0
pref layout after booting path like followings:

pci 0000:04:00.0: BAR 9: assigned [mem 0x84000000-0x9fffffff 64bit pref]
pci 0000:05:04.0: BAR 9: assigned [mem 0x88000000-0x9fffffff 64bit pref]
pci 0000:05:03.0: BAR 9: assigned [mem 0x84000000-0x841fffff 64bit pref]
pci 0000:07:00.0: BAR 0: assigned [mem 0x81200000-0x8123ffff 64bit]
pci 0000:05:03.0: PCI bridge to [bus 08-0f]
pci 0000:05:03.0:   bridge window [mem 0x84000000-0x841fffff 64bit pref]
pci 0000:10:00.0: BAR 0: assigned [mem 0x90000000-0x9fffffff 64bit pref]
pci 0000:10:00.0: BAR 2: assigned [mem 0x88000000-0x8800ffff 64bit pref]
pci 0000:05:04.0: PCI bridge to [bus 10]
pci 0000:05:04.0:   bridge window [mem 0x88000000-0x9fffffff 64bit pref]
pci 0000:04:00.0: PCI bridge to [bus 05-10]
pci 0000:04:00.0:   bridge window [mem 0x84000000-0x9fffffff 64bit pref]

so the old size in rescan for 04:00.0 would be 0x1c000000, and align is 0x4000000

during remove and rescan:

pci 0000:05:03.0: bridge window [mem 0x00000000-0xffffffffffffffff 64bit pref] to [bus 08-0f] add_size 200000 add_align 100000 alt_size 0 alt_align 0 must_size 0 must_align 0
pci 0000:05:03.0: bridge window [mem 0x00000000-0xffffffffffffffff] to [bus 08-0f] add_size 200000 add_align 100000 alt_size 0 alt_align 0 must_size 0 must_align 0
pci 0000:05:04.0: bridge window [mem 0x08000000-0x1fffffff 64bit pref] to [bus 10] add_size 0 add_align 0 alt_size 10100000 alt_align 10000000 must_size 18000000 must_align 8000000
pci 0000:05:03.0: BAR 9: [mem 0x00000000-0xffffffffffffffff 64bit pref] get_res_add_size add_size   200000
pci 0000:05:03.0: BAR 9: [mem 0x00000000-0xffffffffffffffff 64bit pref] get_res_add_align min_align 100000
pci 0000:04:00.0: bridge window [mem 0x08000000-0x27ffffff 64bit pref] to [bus 05-10] add_size 0 add_align 0 alt_size 10100000 alt_align 10000000 must_size 20000000 must_align 8000000

align old size 0x1c000000 to 0x2000000 as size0, 0x1c000000 as size1.
so for 04:00.0 will have big must and no optional size anymore.

So don't align old size, then we will have same size0 and size1.
Also use smaller add_align as must align.

After the patch, rescan works properly.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 8999ead..2679ccc 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1146,9 +1146,9 @@ static resource_size_t calculate_memsize(resource_size_t size,
 		size = min_size;
 	if (old_size == 1)
 		old_size = 0;
+	size = ALIGN(size, align);
 	if (size < old_size)
 		size = old_size;
-	size = ALIGN(size, align);
 	return size;
 }
 
@@ -1632,6 +1632,17 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		b_res->flags = 0;
 		return 0;
 	}
+
+	/*
+	 * It happens when boot path is not passing realloc
+	 * and later rescan is passing realloc.
+	 * The old value from boot path is bigger, and calculate_size will
+	 * use old value as size0 and size1, and also have
+	 * chance optional align is smaller than must only align.
+	 */
+	if(size0 == size1 && min_align > min_add_align)
+		min_align = min_add_align;
+
 	b_res->start = min_align;
 	b_res->end = size0 + min_align - 1;
 	b_res->flags |= IORESOURCE_STARTALIGN;
-- 
1.8.4.5

[PATCH v3 26/51] PCI: Don't add too much optional size for hotplug bridge io

[Yinghai Lu]

Same as patch for mmio (PCI: Don't add too much optional size for hotplug
bridge mmio), and this one is addressing io port.

It will compare must+optional with min_sum_size to get smaller
optional size.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 2679ccc..b44921e 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1189,7 +1189,6 @@ static resource_size_t window_alignment(struct pci_bus *bus,
  *
  * @bus : the bus
  * @min_size : the minimum io window that must to be allocated
- * @add_size : additional optional io window
  * @realloc_head : track the additional io window on this list
  *
  * Sizing the IO windows of the PCI-PCI bridge is trivial,
@@ -1198,9 +1197,11 @@ static resource_size_t window_alignment(struct pci_bus *bus,
  * We must be careful with the ISA aliasing though.
  */
 static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
-		resource_size_t add_size, struct list_head *realloc_head)
+			 struct list_head *realloc_head)
 {
 	struct pci_dev *dev;
+	resource_size_t min_sum_size = 0;
+	resource_size_t sum_add_size;
 	struct resource *b_res = find_free_bus_resource(bus, IORESOURCE_IO,
 							IORESOURCE_IO);
 	resource_size_t size = 0, size0 = 0, size1 = 0;
@@ -1210,6 +1211,11 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 	if (!b_res)
 		return;
 
+	if (realloc_head) {
+		min_sum_size = min_size;
+		min_size = 0;
+	}
+
 	min_align = window_alignment(bus, IORESOURCE_IO);
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		int i;
@@ -1239,10 +1245,11 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 
 	size0 = calculate_iosize(size, min_size, size1,
 			resource_size(b_res), min_align);
-	if (children_add_size > add_size)
-		add_size = children_add_size;
-	size1 = (!realloc_head || (realloc_head && !add_size)) ? size0 :
-		calculate_iosize(size, min_size, add_size + size1,
+	sum_add_size = children_add_size + size + size1;
+	if (sum_add_size < min_sum_size)
+		sum_add_size = min_sum_size;
+	size1 = !realloc_head ? size0 :
+		calculate_iosize(size, min_size, sum_add_size - size,
 			resource_size(b_res), min_align);
 	if (!size0 && !size1) {
 		if (b_res->start || b_res->end)
@@ -1794,7 +1801,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 {
 	struct pci_dev *dev;
 	unsigned long mask, prefmask, type2 = 0, type3 = 0;
-	resource_size_t min_mem_size = 0, additional_io_size = 0;
+	resource_size_t min_mem_size = 0, min_io_size = 0;
 	struct resource *b_res;
 	int ret;
 
@@ -1827,13 +1834,12 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 	case PCI_CLASS_BRIDGE_PCI:
 		pci_bridge_check_ranges(bus);
 		if (bus->self->is_hotplug_bridge) {
-			additional_io_size  = pci_hotplug_io_size;
+			min_io_size  = pci_hotplug_io_size;
 			min_mem_size = pci_hotplug_mem_size;
 		}
 		/* Fall through */
 	default:
-		pbus_size_io(bus, realloc_head ? 0 : additional_io_size,
-			     additional_io_size, realloc_head);
+		pbus_size_io(bus, min_io_size, realloc_head);
 
 		/*
 		 * If there's a 64-bit prefetchable MMIO window, compute
-- 
1.8.4.5

[PATCH v3 27/51] PCI: Move ISA ioport align out of calculate_iosize

[Yinghai Lu]

So we could unify calculate_iosize and calculate_memsize later.

when one bridge have several children devices, and every devices
have several io port resources and resource size < 0x400.

We need to check size, and add extra size to make sure bit8/9
to be zero.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 39 +++++++++++++++++++++++++++------------
 1 file changed, 27 insertions(+), 12 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index b44921e..3db2838 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1126,11 +1126,6 @@ static resource_size_t calculate_iosize(resource_size_t size,
 		size = min_size;
 	if (old_size == 1)
 		old_size = 0;
-	/* To be fixed in 2.5: we should have sort of HAVE_ISA
-	   flag in the struct pci_bus. */
-#if defined(CONFIG_ISA) || defined(CONFIG_EISA)
-	size = (size & 0xff) + ((size & ~0xffUL) << 2);
-#endif
 	size = ALIGN(size + size1, align);
 	if (size < old_size)
 		size = old_size;
@@ -1184,6 +1179,18 @@ static resource_size_t window_alignment(struct pci_bus *bus,
 	return max(align, arch_align);
 }
 
+static resource_size_t size_aligned_for_isa(resource_size_t size)
+{
+	/*
+	 * To be fixed in 2.5: we should have sort of HAVE_ISA
+	 *  flag in the struct pci_bus.
+	 */
+#if defined(CONFIG_ISA) || defined(CONFIG_EISA)
+	size = (size & 0xff) + ((size & ~0xffUL) << 2);
+#endif
+	return size;
+}
+
 /**
  * pbus_size_io() - size the io window of a given bus
  *
@@ -1201,11 +1208,10 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 {
 	struct pci_dev *dev;
 	resource_size_t min_sum_size = 0;
-	resource_size_t sum_add_size;
 	struct resource *b_res = find_free_bus_resource(bus, IORESOURCE_IO,
 							IORESOURCE_IO);
 	resource_size_t size = 0, size0 = 0, size1 = 0;
-	resource_size_t children_add_size = 0;
+	resource_size_t sum_add_size = 0, sum_add_size1 = 0;
 	resource_size_t min_align, align;
 
 	if (!b_res)
@@ -1222,7 +1228,7 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
-			unsigned long r_size;
+			unsigned long r_size, r_add_size;
 
 			if (r->parent || !(r->flags & IORESOURCE_IO))
 				continue;
@@ -1238,18 +1244,27 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 			if (align > min_align)
 				min_align = align;
 
-			if (realloc_head)
-				children_add_size += get_res_add_size(realloc_head, r);
+			if (realloc_head) {
+				r_add_size = get_res_add_size(realloc_head, r);
+				r_add_size += r_size;
+				if (r_add_size < 0x400)
+					/* Might be re-aligned for ISA */
+					sum_add_size += r_add_size;
+				else
+					sum_add_size1 += r_add_size;
+			}
 		}
 	}
 
+	size = size_aligned_for_isa(size);
 	size0 = calculate_iosize(size, min_size, size1,
 			resource_size(b_res), min_align);
-	sum_add_size = children_add_size + size + size1;
+	sum_add_size = size_aligned_for_isa(sum_add_size);
+	sum_add_size += sum_add_size1;
 	if (sum_add_size < min_sum_size)
 		sum_add_size = min_sum_size;
 	size1 = !realloc_head ? size0 :
-		calculate_iosize(size, min_size, sum_add_size - size,
+		calculate_iosize(sum_add_size, min_size, 0,
 			resource_size(b_res), min_align);
 	if (!size0 && !size1) {
 		if (b_res->start || b_res->end)
-- 
1.8.4.5

[PATCH v3 28/51] PCI: Unifiy calculate_size for io port and mmio

[Yinghai Lu]

We should check size+size1 with min_size for io port.
For example, when hotplug bridge has two children bridges,
every child bridge will need 0x1000, so size1 will be 0x2000
and size is 0. The min_size for the hotplug bridge is 0x100.
with old version calculate_iosize, we get 0x3000 for final
size because we are using size to compare with min_size. That is
not right, we should use 0x2000 instead.

After this change, calculate_memsize and calculate_iosize
is the same.

Change them to calculate_size.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 27 ++++++---------------------
 1 file changed, 6 insertions(+), 21 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 3db2838..aeed716 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1116,23 +1116,7 @@ static struct resource *find_free_bus_resource(struct pci_bus *bus,
 	return NULL;
 }
 
-static resource_size_t calculate_iosize(resource_size_t size,
-		resource_size_t min_size,
-		resource_size_t size1,
-		resource_size_t old_size,
-		resource_size_t align)
-{
-	if (size < min_size)
-		size = min_size;
-	if (old_size == 1)
-		old_size = 0;
-	size = ALIGN(size + size1, align);
-	if (size < old_size)
-		size = old_size;
-	return size;
-}
-
-static resource_size_t calculate_memsize(resource_size_t size,
+static resource_size_t calculate_size(resource_size_t size,
 		resource_size_t min_size,
 		resource_size_t old_size,
 		resource_size_t align)
@@ -1257,14 +1241,15 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 	}
 
 	size = size_aligned_for_isa(size);
-	size0 = calculate_iosize(size, min_size, size1,
+	size += size1;
+	size0 = calculate_size(size, min_size,
 			resource_size(b_res), min_align);
 	sum_add_size = size_aligned_for_isa(sum_add_size);
 	sum_add_size += sum_add_size1;
 	if (sum_add_size < min_sum_size)
 		sum_add_size = min_sum_size;
 	size1 = !realloc_head ? size0 :
-		calculate_iosize(sum_add_size, min_size, 0,
+		calculate_size(sum_add_size, min_size,
 			resource_size(b_res), min_align);
 	if (!size0 && !size1) {
 		if (b_res->start || b_res->end)
@@ -1617,7 +1602,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 	if (size || min_size) {
 		min_align = calculate_mem_align(&align_test_list, max_align,
 						size, window_align);
-		size0 = calculate_memsize(size, min_size,
+		size0 = calculate_size(size, min_size,
 				  resource_size(b_res), min_align);
 	}
 	free_align_test_list(&align_test_list);
@@ -1642,7 +1627,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		min_add_align = calculate_mem_align(&align_test_add_list,
 					max_add_align, sum_add_size,
 					window_align);
-		size1 = calculate_memsize(sum_add_size, min_size,
+		size1 = calculate_size(sum_add_size, min_size,
 				 resource_size(b_res), min_add_align);
 	}
 	free_align_test_list(&align_test_add_list);
-- 
1.8.4.5

[PATCH v3 29/51] PCI: Allow optional only io resource must size to be 0

[Yinghai Lu]

When there is no child device under the non hotplug bridge,
We can use 0 for must size, and do not use old size as must size.

When there is child device, size will not be 0.
when the bridge is not hotplug, min_size will not be 0.
So they will still honor the old size as must size.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index aeed716..3a94617 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1242,8 +1242,9 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 
 	size = size_aligned_for_isa(size);
 	size += size1;
-	size0 = calculate_size(size, min_size,
-			resource_size(b_res), min_align);
+	if (size || min_size)
+		size0 = calculate_size(size, min_size,
+					resource_size(b_res), min_align);
 	sum_add_size = size_aligned_for_isa(sum_add_size);
 	sum_add_size += sum_add_size1;
 	if (sum_add_size < min_sum_size)
@@ -1259,7 +1260,7 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 		return;
 	}
 
-	b_res->start = min_align;
+	b_res->start = size0 ? min_align : 0;
 	b_res->end = b_res->start + size0 - 1;
 	b_res->flags |= IORESOURCE_STARTALIGN;
 	if (size1 > size0 && realloc_head) {
-- 
1.8.4.5

[PATCH v3 30/51] PCI: Unify skip_ioresource_align()

[Yinghai Lu]

There are powerpc generic version and x86 local version.

Move the powerpc version to setup-bus.c, and kill x86 local version.

Also kill dummy version in microblaze.

Cc: Michal Simek <monstr@monstr.eu>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-arch@vger.kernel.org
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 arch/microblaze/pci/pci-common.c |  8 --------
 arch/powerpc/kernel/pci-common.c | 11 +----------
 arch/x86/include/asm/pci_x86.h   |  1 -
 arch/x86/pci/common.c            |  4 ++--
 arch/x86/pci/i386.c              | 12 ++----------
 drivers/pci/setup-bus.c          |  9 +++++++++
 include/asm-generic/pci-bridge.h |  2 ++
 7 files changed, 16 insertions(+), 31 deletions(-)

diff --git a/arch/microblaze/pci/pci-common.c b/arch/microblaze/pci/pci-common.c
index ae838ed..09b1af6 100644
--- a/arch/microblaze/pci/pci-common.c
+++ b/arch/microblaze/pci/pci-common.c
@@ -878,11 +878,6 @@ void pcibios_fixup_bus(struct pci_bus *bus)
 }
 EXPORT_SYMBOL(pcibios_fixup_bus);
 
-static int skip_isa_ioresource_align(struct pci_dev *dev)
-{
-	return 0;
-}
-
 /*
  * We need to avoid collisions with `mirrored' VGA ports
  * and other strange ISA hardware, so we always want the
@@ -899,12 +894,9 @@ static int skip_isa_ioresource_align(struct pci_dev *dev)
 resource_size_t pcibios_align_resource(void *data, const struct resource *res,
 				resource_size_t size, resource_size_t align)
 {
-	struct pci_dev *dev = data;
 	resource_size_t start = res->start;
 
 	if (res->flags & IORESOURCE_IO) {
-		if (skip_isa_ioresource_align(dev))
-			return start;
 		if (start & 0x300)
 			start = (start + 0x3ff) & ~0x3ff;
 	}
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index b9de34d..2d8d654 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -1064,15 +1064,6 @@ void pci_fixup_cardbus(struct pci_bus *bus)
 	pcibios_setup_bus_devices(bus);
 }
 
-
-static int skip_isa_ioresource_align(struct pci_dev *dev)
-{
-	if (pci_has_flag(PCI_CAN_SKIP_ISA_ALIGN) &&
-	    !(dev->bus->bridge_ctl & PCI_BRIDGE_CTL_ISA))
-		return 1;
-	return 0;
-}
-
 /*
  * We need to avoid collisions with `mirrored' VGA ports
  * and other strange ISA hardware, so we always want the
@@ -1093,7 +1084,7 @@ resource_size_t pcibios_align_resource(void *data, const struct resource *res,
 	resource_size_t start = res->start;
 
 	if (res->flags & IORESOURCE_IO) {
-		if (skip_isa_ioresource_align(dev))
+		if (skip_isa_ioresource_align(dev->bus))
 			return start;
 		if (start & 0x300)
 			start = (start + 0x3ff) & ~0x3ff;
diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index 164e3f8..ddac225 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -28,7 +28,6 @@ do {						\
 #define PCI_ASSIGN_ROMS		0x1000
 #define PCI_BIOS_IRQ_SCAN	0x2000
 #define PCI_ASSIGN_ALL_BUSSES	0x4000
-#define PCI_CAN_SKIP_ISA_ALIGN	0x8000
 #define PCI_USE__CRS		0x10000
 #define PCI_CHECK_ENABLE_AMD_MMCONF	0x20000
 #define PCI_HAS_IO_ECS		0x40000
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index 8fd6f44..e8df922 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -83,7 +83,7 @@ DEFINE_RAW_SPINLOCK(pci_config_lock);
 
 static int __init can_skip_ioresource_align(const struct dmi_system_id *d)
 {
-	pci_probe |= PCI_CAN_SKIP_ISA_ALIGN;
+	pci_add_flags(PCI_CAN_SKIP_ISA_ALIGN);
 	printk(KERN_INFO "PCI: %s detected, can skip ISA alignment\n", d->ident);
 	return 0;
 }
@@ -619,7 +619,7 @@ char *__init pcibios_setup(char *str)
 		pci_routeirq = 1;
 		return NULL;
 	} else if (!strcmp(str, "skip_isa_align")) {
-		pci_probe |= PCI_CAN_SKIP_ISA_ALIGN;
+		pci_add_flags(PCI_CAN_SKIP_ISA_ALIGN);
 		return NULL;
 	} else if (!strcmp(str, "noioapicquirk")) {
 		noioapicquirk = 1;
diff --git a/arch/x86/pci/i386.c b/arch/x86/pci/i386.c
index 0a9f2ca..3f17726 100644
--- a/arch/x86/pci/i386.c
+++ b/arch/x86/pci/i386.c
@@ -33,6 +33,7 @@
 #include <linux/errno.h>
 #include <linux/bootmem.h>
 
+#include <asm-generic/pci-bridge.h>
 #include <asm/pat.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
@@ -128,15 +129,6 @@ static void __init pcibios_fw_addr_list_del(void)
 	pcibios_fw_addr_done = true;
 }
 
-static int
-skip_isa_ioresource_align(struct pci_dev *dev) {
-
-	if ((pci_probe & PCI_CAN_SKIP_ISA_ALIGN) &&
-	    !(dev->bus->bridge_ctl & PCI_BRIDGE_CTL_ISA))
-		return 1;
-	return 0;
-}
-
 /*
  * We need to avoid collisions with `mirrored' VGA ports
  * and other strange ISA hardware, so we always want the
@@ -158,7 +150,7 @@ pcibios_align_resource(void *data, const struct resource *res,
 	resource_size_t start = res->start;
 
 	if (res->flags & IORESOURCE_IO) {
-		if (skip_isa_ioresource_align(dev))
+		if (skip_isa_ioresource_align(dev->bus))
 			return start;
 		if (start & 0x300)
 			start = (start + 0x3ff) & ~0x3ff;
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 3a94617..d4610a8 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1163,6 +1163,15 @@ static resource_size_t window_alignment(struct pci_bus *bus,
 	return max(align, arch_align);
 }
 
+int skip_isa_ioresource_align(struct pci_bus *bus)
+{
+	if (pci_has_flag(PCI_CAN_SKIP_ISA_ALIGN) &&
+	    !(bus->bridge_ctl & PCI_BRIDGE_CTL_ISA))
+		return 1;
+
+	return 0;
+}
+
 static resource_size_t size_aligned_for_isa(resource_size_t size)
 {
 	/*
diff --git a/include/asm-generic/pci-bridge.h b/include/asm-generic/pci-bridge.h
index 20db2e5..fab4bd0 100644
--- a/include/asm-generic/pci-bridge.h
+++ b/include/asm-generic/pci-bridge.h
@@ -60,6 +60,8 @@ static inline int pci_has_flag(int flag)
 {
 	return pci_flags & flag;
 }
+
+int skip_isa_ioresource_align(struct pci_bus *bus);
 #else
 static inline void pci_set_flags(int flags) { }
 static inline void pci_add_flags(int flags) { }
-- 
1.8.4.5

[PATCH v3 31/51] PCI: Kill macro checking for bus io port sizing

[Yinghai Lu]

Use new generic version skip_isa_ioresource_align() instead.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index d4610a8..f8f89cd 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1172,15 +1172,12 @@ int skip_isa_ioresource_align(struct pci_bus *bus)
 	return 0;
 }
 
-static resource_size_t size_aligned_for_isa(resource_size_t size)
+static resource_size_t size_aligned_for_isa(resource_size_t size,
+					    struct pci_bus *bus)
 {
-	/*
-	 * To be fixed in 2.5: we should have sort of HAVE_ISA
-	 *  flag in the struct pci_bus.
-	 */
-#if defined(CONFIG_ISA) || defined(CONFIG_EISA)
-	size = (size & 0xff) + ((size & ~0xffUL) << 2);
-#endif
+	if (!skip_isa_ioresource_align(bus))
+		size = (size & 0xff) + ((size & ~0xffUL) << 2);
+
 	return size;
 }
 
@@ -1249,12 +1246,12 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 		}
 	}
 
-	size = size_aligned_for_isa(size);
+	size = size_aligned_for_isa(size, bus);
 	size += size1;
 	if (size || min_size)
 		size0 = calculate_size(size, min_size,
 					resource_size(b_res), min_align);
-	sum_add_size = size_aligned_for_isa(sum_add_size);
+	sum_add_size = size_aligned_for_isa(sum_add_size, bus);
 	sum_add_size += sum_add_size1;
 	if (sum_add_size < min_sum_size)
 		sum_add_size = min_sum_size;
-- 
1.8.4.5

[PATCH v3 32/51] resources: Split out __allocate_resource()

[Yinghai Lu]

It will not hold lock, so we could use it in other functions that
hold the resource lock already.

-v2: according to Linus, using "bool lock" as parameter
     aka "conditionally take lock" is *wrong*.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
---
 kernel/resource.c | 70 +++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 50 insertions(+), 20 deletions(-)

diff --git a/kernel/resource.c b/kernel/resource.c
index fed052a..67b58a5 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -619,7 +619,7 @@ static int find_resource(struct resource *root, struct resource *new,
 }
 
 /**
- * reallocate_resource - allocate a slot in the resource tree given range & alignment.
+ * __reallocate_resource - allocate a slot in the resource tree given range & alignment.
  *	The resource will be relocated if the new size cannot be reallocated in the
  *	current location.
  *
@@ -628,7 +628,7 @@ static int find_resource(struct resource *root, struct resource *new,
  * @newsize: new size of the resource descriptor
  * @constraint: the size and alignment constraints to be met.
  */
-static int reallocate_resource(struct resource *root, struct resource *old,
+static int __reallocate_resource(struct resource *root, struct resource *old,
 			resource_size_t newsize,
 			struct resource_constraint  *constraint)
 {
@@ -636,8 +636,6 @@ static int reallocate_resource(struct resource *root, struct resource *old,
 	struct resource new = *old;
 	struct resource *conflict;
 
-	write_lock(&resource_lock);
-
 	if ((err = __find_resource(root, old, &new, newsize, constraint)))
 		goto out;
 
@@ -662,14 +660,13 @@ static int reallocate_resource(struct resource *root, struct resource *old,
 		BUG_ON(conflict);
 	}
 out:
-	write_unlock(&resource_lock);
 	return err;
 }
 
-
 /**
- * allocate_resource - allocate empty slot in the resource tree given range & alignment.
- * 	The resource will be reallocated with a new size if it was already allocated
+ * __allocate_resource - allocate empty slot in the resource tree given range & alignment.
+ *	The resource will be reallocated with a new size if it was already
+ *	allocated
  * @root: root resource descriptor
  * @new: resource descriptor desired by caller
  * @size: requested resource region size
@@ -678,15 +675,17 @@ out:
  * @align: alignment requested, in bytes
  * @alignf: alignment function, optional, called if not NULL
  * @alignf_data: arbitrary data to pass to the @alignf function
+ *
+ * Caller need to hold resource_lock if needed.
  */
-int allocate_resource(struct resource *root, struct resource *new,
-		      resource_size_t size, resource_size_t min,
-		      resource_size_t max, resource_size_t align,
-		      resource_size_t (*alignf)(void *,
-						const struct resource *,
-						resource_size_t,
-						resource_size_t),
-		      void *alignf_data)
+static int __allocate_resource(struct resource *root, struct resource *new,
+				resource_size_t size, resource_size_t min,
+				resource_size_t max, resource_size_t align,
+				resource_size_t (*alignf)(void *,
+						  const struct resource *,
+						  resource_size_t,
+						  resource_size_t),
+				void *alignf_data)
 {
 	int err;
 	struct resource_constraint constraint;
@@ -700,20 +699,51 @@ int allocate_resource(struct resource *root, struct resource *new,
 	constraint.alignf = alignf;
 	constraint.alignf_data = alignf_data;
 
-	if ( new->parent ) {
+	if (new->parent) {
 		/* resource is already allocated, try reallocating with
 		   the new constraints */
-		return reallocate_resource(root, new, size, &constraint);
+		return __reallocate_resource(root, new, size, &constraint);
 	}
 
-	write_lock(&resource_lock);
 	err = find_resource(root, new, size, &constraint);
 	if (err >= 0 && __request_resource(root, new))
 		err = -EBUSY;
-	write_unlock(&resource_lock);
+
 	return err;
 }
 
+/**
+ * allocate_resource - allocate empty slot in the resource tree given range & alignment.
+ *	The resource will be reallocated with a new size if it was already
+ *	allocated
+ * @root: root resource descriptor
+ * @new: resource descriptor desired by caller
+ * @size: requested resource region size
+ * @min: minimum boundary to allocate
+ * @max: maximum boundary to allocate
+ * @align: alignment requested, in bytes
+ * @alignf: alignment function, optional, called if not NULL
+ * @alignf_data: arbitrary data to pass to the @alignf function
+ */
+int allocate_resource(struct resource *root, struct resource *new,
+		      resource_size_t size, resource_size_t min,
+		      resource_size_t max, resource_size_t align,
+		      resource_size_t (*alignf)(void *,
+						const struct resource *,
+						resource_size_t,
+						resource_size_t),
+		      void *alignf_data)
+{
+	int ret;
+
+	write_lock(&resource_lock);
+	ret = __allocate_resource(root, new, size, min, max, align,
+				   alignf, alignf_data);
+	write_unlock(&resource_lock);
+
+	return ret;
+}
+
 EXPORT_SYMBOL(allocate_resource);
 
 /**
-- 
1.8.4.5

[PATCH v3 33/51] resources: Make allocate_resource return just fit resource

[Yinghai Lu]

Find all suitable empty slots and pick one just fit, so we could save
the big slot for needed ones later when we have several pcie switches
and some bridges get assigned bios and we need to assign others in kernel.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 kernel/resource.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 68 insertions(+), 13 deletions(-)

diff --git a/kernel/resource.c b/kernel/resource.c
index 67b58a5..697b8ca 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -48,6 +48,7 @@ struct resource_constraint {
 	resource_size_t (*alignf)(void *, const struct resource *,
 			resource_size_t, resource_size_t);
 	void *alignf_data;
+	bool fit;
 };
 
 static DEFINE_RWLOCK(resource_lock);
@@ -554,12 +555,15 @@ static void resource_clip(struct resource *res, resource_size_t min,
  * alignment constraints
  */
 static int __find_resource(struct resource *root, struct resource *old,
-			 struct resource *new,
+			 struct resource *new, struct resource *avail,
 			 resource_size_t  size,
 			 struct resource_constraint *constraint)
 {
 	struct resource *this = root->child;
-	struct resource tmp = *new, avail, alloc;
+	struct resource tmp = *new, availx, alloc;
+
+	if (!avail || avail == new)
+		avail = &availx;
 
 	tmp.start = root->start;
 	/*
@@ -583,15 +587,16 @@ static int __find_resource(struct resource *root, struct resource *old,
 		arch_remove_reservations(&tmp);
 
 		/* Check for overflow after ALIGN() */
-		avail.start = ALIGN(tmp.start, constraint->align);
-		avail.end = tmp.end;
-		avail.flags = new->flags & ~IORESOURCE_UNSET;
-		if (avail.start >= tmp.start) {
-			alloc.flags = avail.flags;
-			alloc.start = constraint->alignf(constraint->alignf_data, &avail,
+		avail->start = ALIGN(tmp.start, constraint->align);
+		avail->end = tmp.end;
+		avail->flags = new->flags & ~IORESOURCE_UNSET;
+		if (avail->start >= tmp.start) {
+			alloc.flags = avail->flags;
+			alloc.start = constraint->alignf(
+					constraint->alignf_data, avail,
 					size, constraint->align);
 			alloc.end = alloc.start + size - 1;
-			if (resource_contains(&avail, &alloc)) {
+			if (resource_contains(avail, &alloc)) {
 				new->start = alloc.start;
 				new->end = alloc.end;
 				return 0;
@@ -608,6 +613,11 @@ next:		if (!this || this->end == root->end)
 	return -EBUSY;
 }
 
+struct good_resource {
+	struct list_head list;
+	struct resource avail;
+	struct resource new;
+};
 /*
  * Find empty slot in the resource tree given range and alignment.
  */
@@ -615,7 +625,49 @@ static int find_resource(struct resource *root, struct resource *new,
 			resource_size_t size,
 			struct resource_constraint  *constraint)
 {
-	return  __find_resource(root, NULL, new, size, constraint);
+	int ret = -1;
+	LIST_HEAD(head);
+	struct good_resource *good, *tmp;
+	resource_size_t avail_size = (resource_size_t)-1ULL;
+
+	if (!constraint->fit)
+		return __find_resource(root, NULL, new, NULL, size,
+					constraint);
+
+	/* find all suitable ones and add to the list */
+	for (;;) {
+		good = kzalloc(sizeof(*good), GFP_KERNEL);
+		if (!good)
+			break;
+
+		good->new.start = new->start;
+		good->new.end = new->end;
+		good->new.flags = new->flags;
+		ret = __find_resource(root, NULL, &good->new, &good->avail,
+					size, constraint);
+		if (ret || __request_resource(root, &good->avail)) {
+			ret = -EBUSY;
+			kfree(good);
+			break;
+		}
+
+		list_add(&good->list, &head);
+	}
+
+	/* pick up the smallest one and delete the list */
+	list_for_each_entry_safe(good, tmp, &head, list) {
+		if (resource_size(&good->avail) < avail_size) {
+			avail_size = resource_size(&good->avail);
+			new->start = good->new.start;
+			new->end = good->new.end;
+			ret = 0;
+		}
+		list_del(&good->list);
+		__release_resource(&good->avail);
+		kfree(good);
+	}
+
+	return ret;
 }
 
 /**
@@ -636,7 +688,8 @@ static int __reallocate_resource(struct resource *root, struct resource *old,
 	struct resource new = *old;
 	struct resource *conflict;
 
-	if ((err = __find_resource(root, old, &new, newsize, constraint)))
+	err = __find_resource(root, old, &new, NULL, newsize, constraint);
+	if (err)
 		goto out;
 
 	if (resource_contains(&new, old)) {
@@ -675,6 +728,7 @@ out:
  * @align: alignment requested, in bytes
  * @alignf: alignment function, optional, called if not NULL
  * @alignf_data: arbitrary data to pass to the @alignf function
+ * @fit: only allocate fit range.
  *
  * Caller need to hold resource_lock if needed.
  */
@@ -685,7 +739,7 @@ static int __allocate_resource(struct resource *root, struct resource *new,
 						  const struct resource *,
 						  resource_size_t,
 						  resource_size_t),
-				void *alignf_data)
+				void *alignf_data, bool fit)
 {
 	int err;
 	struct resource_constraint constraint;
@@ -698,6 +752,7 @@ static int __allocate_resource(struct resource *root, struct resource *new,
 	constraint.align = align;
 	constraint.alignf = alignf;
 	constraint.alignf_data = alignf_data;
+	constraint.fit = fit;
 
 	if (new->parent) {
 		/* resource is already allocated, try reallocating with
@@ -738,7 +793,7 @@ int allocate_resource(struct resource *root, struct resource *new,
 
 	write_lock(&resource_lock);
 	ret = __allocate_resource(root, new, size, min, max, align,
-				   alignf, alignf_data);
+				   alignf, alignf_data, true);
 	write_unlock(&resource_lock);
 
 	return ret;
-- 
1.8.4.5

[PATCH v3 34/51] PCI: Check pref compatible bit for mem64 resource of pcie device

[Yinghai Lu]

We still get "no compatible bridge window" warning on sparc T5-8
after we add support for 64bit resource parsing for root bus.

 PCI: scan_bus[/pci@300/pci@1/pci@0/pci@6] bus no 8
 PCI: Claiming 0000:00:01.0: Resource 15: 0000800100000000..00008004afffffff [220c]
 PCI: Claiming 0000:01:00.0: Resource 15: 0000800100000000..00008004afffffff [220c]
 PCI: Claiming 0000:02:04.0: Resource 15: 0000800100000000..000080012fffffff [220c]
 PCI: Claiming 0000:03:00.0: Resource 15: 0000800100000000..000080012fffffff [220c]
 PCI: Claiming 0000:04:06.0: Resource 14: 0000800100000000..000080010fffffff [220c]
 PCI: Claiming 0000:05:00.0: Resource 0: 0000800100000000..0000800100001fff [204]
 pci 0000:05:00.0: can't claim BAR 0 [mem 0x800100000000-0x800100001fff]: no compatible bridge window

All the bridges 64-bit resource have pref bit, but the device resource does not
have pref set, then we can not find parent for the device resource,
as we can not put non-pref mem under pref mem.

According to pcie spec errta
https://www.pcisig.com/specifications/pciexpress/base2/PCIe_Base_r2.1_Errata_08Jun10.pdf
page 13, in some case it is ok to mark some as pref.

Mark if the entire path from the host to the adapter is over PCI Express.
Then set pref compatible bit for claim/sizing/assign for 64bit mem resource
on that pcie device.

Fixes: commit d63e2e1f3df9 ("sparc/PCI: Clip bridge windows to fit in upstream windows")
Link: http://lkml.kernel.org/r/CAE9FiQU1gJY1LYrxs+ma5LCTEEe4xmtjRG0aXJ9K_Tsu+m9Wuw@mail.gmail.com
Reported-by: David Ahern <david.ahern@oracle.com>
Tested-by: David Ahern <david.ahern@oracle.com>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
Tested-by: TJ <linux@iam.tj>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: <stable@vger.kernel.org> #3.19
---
 drivers/pci/pci.c       |  3 ++-
 drivers/pci/pci.h       |  2 ++
 drivers/pci/probe.c     | 33 +++++++++++++++++++++++++++++++++
 drivers/pci/setup-bus.c | 21 ++++++++++++++++++---
 drivers/pci/setup-res.c |  4 ++++
 include/linux/pci.h     |  1 +
 6 files changed, 60 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 0008c95..ff1192a 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -414,6 +414,7 @@ EXPORT_SYMBOL_GPL(pci_find_ht_capability);
 struct resource *pci_find_parent_resource(const struct pci_dev *dev,
 					  struct resource *res)
 {
+	int flags = pci_resource_pref_compatible(dev, res);
 	const struct pci_bus *bus = dev->bus;
 	struct resource *r;
 	int i;
@@ -428,7 +429,7 @@ struct resource *pci_find_parent_resource(const struct pci_dev *dev,
 			 * not, the allocator made a mistake.
 			 */
 			if (r->flags & IORESOURCE_PREFETCH &&
-			    !(res->flags & IORESOURCE_PREFETCH))
+			    !(flags & IORESOURCE_PREFETCH))
 				return NULL;
 
 			/*
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 2b83977..1804d44 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -336,4 +336,6 @@ static inline int pci_dev_specific_reset(struct pci_dev *dev, int probe)
 
 struct pci_host_bridge *pci_find_host_bridge(struct pci_bus *bus);
 
+int pci_resource_pref_compatible(const struct pci_dev *dev,
+				 struct resource *res);
 #endif /* DRIVERS_PCI_H */
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index cefd636..010d8d9 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1544,6 +1544,36 @@ static void pci_init_capabilities(struct pci_dev *dev)
 	pci_enable_acs(dev);
 }
 
+static bool pci_up_path_over_pcie(struct pci_bus *bus)
+{
+	if (pci_is_root_bus(bus))
+		return true;
+
+	if (bus->self && !pci_is_pcie(bus->self))
+		return false;
+
+	return pci_up_path_over_pcie(bus->parent);
+}
+
+/*
+ * According to
+ * https://www.pcisig.com/specifications/pciexpress/base2/PCIe_Base_r2.1_Errata_08Jun10.pdf
+ * page 13, system firmware could put some 64bit non-pref under 64bit pref,
+ * on some cases.
+ * Let's mark if entire path from the host to the adapter is over PCI
+ * Express. later will use that compute pref compaitable bit.
+ */
+static void pci_set_on_all_pcie_path(struct pci_dev *dev)
+{
+	if (!pci_is_pcie(dev))
+		return;
+
+	if (!pci_up_path_over_pcie(dev->bus))
+		return;
+
+	dev->on_all_pcie_path = 1;
+}
+
 void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
 {
 	int ret;
@@ -1574,6 +1604,9 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
 	/* Initialize various capabilities */
 	pci_init_capabilities(dev);
 
+	/* After pcie_cap is assigned */
+	pci_set_on_all_pcie_path(dev);
+
 	/*
 	 * Add the device to our list of discovered devices
 	 * and the bus list for fixup functions, etc.
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index f8f89cd..78b0c36 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1038,6 +1038,20 @@ int pci_claim_bridge_resource(struct pci_dev *bridge, int i)
 	return -EINVAL;
 }
 
+int pci_resource_pref_compatible(const struct pci_dev *dev,
+				 struct resource *res)
+{
+	if (res->flags & IORESOURCE_PREFETCH)
+		return res->flags;
+
+	if ((res->flags & IORESOURCE_MEM) &&
+	    (res->flags & IORESOURCE_MEM_64) &&
+	    dev->on_all_pcie_path)
+		return res->flags | IORESOURCE_PREFETCH;
+
+	return res->flags;
+}
+
 /* Check whether the bridge supports optional I/O and
    prefetchable memory ranges. If not, the respective
    base/limit registers must be read-only and read as 0. */
@@ -1532,10 +1546,11 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 			resource_size_t r_size, align;
+			int flags = pci_resource_pref_compatible(dev, r);
 
-			if (r->parent || ((r->flags & mask) != type &&
-					  (r->flags & mask) != type2 &&
-					  (r->flags & mask) != type3))
+			if (r->parent || ((flags & mask) != type &&
+					  (flags & mask) != type2 &&
+					  (flags & mask) != type3))
 				continue;
 
 			r_size = resource_size(r);
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 232f925..b19aa5b 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -250,15 +250,19 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 static int _pci_assign_resource(struct pci_dev *dev, int resno,
 				resource_size_t size, resource_size_t min_align)
 {
+	struct resource *res = dev->resource + resno;
+	int old_flags = res->flags;
 	struct pci_bus *bus;
 	int ret;
 
+	res->flags = pci_resource_pref_compatible(dev, res);
 	bus = dev->bus;
 	while ((ret = __pci_assign_resource(bus, dev, resno, size, min_align))) {
 		if (!bus->parent || !bus->self->transparent)
 			break;
 		bus = bus->parent;
 	}
+	res->flags = old_flags;
 
 	return ret;
 }
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 8a0321a..33ef25f 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -309,6 +309,7 @@ struct pci_dev {
 						   powered on/off by the
 						   corresponding bridge */
 	unsigned int	ignore_hotplug:1;	/* Ignore hotplug events */
+	unsigned int	on_all_pcie_path:1;	/* up to host-bridge all pcie */
 	unsigned int	d3_delay;	/* D3->D0 transition time in ms */
 	unsigned int	d3cold_delay;	/* D3cold->D0 transition time in ms */
 
-- 
1.8.4.5

[PATCH v3 35/51] PCI: Only treat non-pef mmio64 as pref if all bridges has MEM_64

[Yinghai Lu]

If any bridge up to root only have 32bit pref mmio, We don't need to
treat device non-pref mmio64 as as pref mmio64.

We need to move pci_bridge_check_ranges calling early.
for parent bridges pref mmio BAR may not allocated by BIOS, res flags
is still 0, we need to have it correct set before we check them for
child device resources.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 78b0c36..c6c9d56 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1038,6 +1038,18 @@ int pci_claim_bridge_resource(struct pci_dev *bridge, int i)
 	return -EINVAL;
 }
 
+static bool pci_up_path_over_pref_mem64(struct pci_bus *bus)
+{
+	if (pci_is_root_bus(bus))
+		return true;
+
+	if (bus->self && !(bus->self->resource[PCI_BRIDGE_RESOURCES + 2].flags &
+			   IORESOURCE_MEM_64))
+		return false;
+
+	return pci_up_path_over_pref_mem64(bus->parent);
+}
+
 int pci_resource_pref_compatible(const struct pci_dev *dev,
 				 struct resource *res)
 {
@@ -1046,7 +1058,8 @@ int pci_resource_pref_compatible(const struct pci_dev *dev,
 
 	if ((res->flags & IORESOURCE_MEM) &&
 	    (res->flags & IORESOURCE_MEM_64) &&
-	    dev->on_all_pcie_path)
+	    dev->on_all_pcie_path &&
+	    pci_up_path_over_pref_mem64(dev->bus))
 		return res->flags | IORESOURCE_PREFETCH;
 
 	return res->flags;
@@ -1827,6 +1840,10 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 	struct resource *b_res;
 	int ret;
 
+	if (!pci_is_root_bus(bus) &&
+	    (bus->self->class >> 8) == PCI_CLASS_BRIDGE_PCI)
+		pci_bridge_check_ranges(bus);
+
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		struct pci_bus *b = dev->subordinate;
 		if (!b)
@@ -1854,7 +1871,6 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 		break;
 
 	case PCI_CLASS_BRIDGE_PCI:
-		pci_bridge_check_ranges(bus);
 		if (bus->self->is_hotplug_bridge) {
 			min_io_size  = pci_hotplug_io_size;
 			min_mem_size = pci_hotplug_mem_size;
-- 
1.8.4.5

[PATCH v3 36/51] PCI: Add has_mem64 for host_bridge

[Yinghai Lu]

On system that does not support mmio64 above 4g, will not set that.
We will use that info next two following patches:
1. Don't treat non-pref mmio64 as pref mmio, so will not put
   it under bridge's pref range when rescan the devices
2. will keep pref mmio64 and pref mmio32 under bridge pref bar.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/probe.c | 9 +++++++++
 include/linux/pci.h | 1 +
 2 files changed, 10 insertions(+)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 010d8d9..14bdbca 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2051,6 +2051,15 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
 		dev_info(&b->dev, "root bus resource %pR%s\n", res, bus_addr);
 	}
 
+	resource_list_for_each_entry(window, &bridge->windows) {
+		res = window->res;
+		if (resource_type(res) == IORESOURCE_MEM &&
+		    (res->end - window->offset) > 0xffffffff) {
+			bridge->has_mem64 = 1;
+			break;
+		}
+	}
+
 	down_write(&pci_bus_sem);
 	list_add_tail(&b->node, &pci_root_buses);
 	up_write(&pci_bus_sem);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 33ef25f..0771b37 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -409,6 +409,7 @@ struct pci_host_bridge {
 	void (*release_fn)(struct pci_host_bridge *);
 	void *release_data;
 	unsigned int ignore_reset_delay:1;	/* for entire hierarchy */
+	unsigned int has_mem64:1;
 };
 
 #define	to_pci_host_bridge(n) container_of(n, struct pci_host_bridge, dev)
-- 
1.8.4.5

[PATCH v3 37/51] PCI: Only treat non-pef mmio64 as pref if host-bridge has_mem64

[Yinghai Lu]

If host bridge does not have mmio64 above 4G, We don't need to
treat device non-pref mmio64 as as pref mmio64.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index c6c9d56..0269868 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1041,7 +1041,7 @@ int pci_claim_bridge_resource(struct pci_dev *bridge, int i)
 static bool pci_up_path_over_pref_mem64(struct pci_bus *bus)
 {
 	if (pci_is_root_bus(bus))
-		return true;
+		return to_pci_host_bridge(bus->bridge)->has_mem64;
 
 	if (bus->self && !(bus->self->resource[PCI_BRIDGE_RESOURCES + 2].flags &
 			   IORESOURCE_MEM_64))
-- 
1.8.4.5

[PATCH v3 38/51] PCI: Restore pref mmio allocation logic for hostbridge without mmio64

[Yinghai Lu]

>From 5b2854155 (PCI: Restrict 64-bit prefetchable bridge windows to 64-bit
resources), we change the logic for pref mmio allocation:
When bridge pref support mmio64, we will only put children pref
that support mmio64 into it, and will put children pref mmio32
into bridge's non-pref mmio32.

That could leave bridge pref bar not used when that pref bar is mmio64,
and children res only has mmio32.
Also could have allocation failure when non-pref mmio32 is not big
enough space for those children pref mmio32.

That is not rational when the host bridge does not 64bit mmio above 4g
at all.

The patch restore to old logic:
when hostbridge does not have has_mem64 so put children pref mmio64 and
pref mmio32 all under bridges pref bars.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/bus.c       |  4 +++-
 drivers/pci/setup-bus.c | 13 +++++++++----
 drivers/pci/setup-res.c |  9 ++++++---
 3 files changed, 18 insertions(+), 8 deletions(-)

diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index 6fbd3f2..b043bdf 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -202,8 +202,10 @@ int pci_bus_alloc_resource(struct pci_bus *bus, struct resource *res,
 {
 #ifdef CONFIG_PCI_BUS_ADDR_T_64BIT
 	int rc;
+	unsigned long mmio64 = pci_find_host_bridge(bus)->has_mem64 ?
+				IORESOURCE_MEM_64 : 0;
 
-	if (res->flags & IORESOURCE_MEM_64) {
+	if (res->flags & mmio64) {
 		rc = pci_bus_alloc_from_region(bus, res, size, align, min,
 					       type_mask, alignf, alignf_data,
 					       &pci_high);
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 0269868..54f0261 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1887,7 +1887,8 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head)
 		b_res = &bus->self->resource[PCI_BRIDGE_RESOURCES];
 		mask = IORESOURCE_MEM;
 		prefmask = IORESOURCE_MEM | IORESOURCE_PREFETCH;
-		if (b_res[2].flags & IORESOURCE_MEM_64) {
+		if ((b_res[2].flags & IORESOURCE_MEM_64) &&
+		    pci_find_host_bridge(bus)->has_mem64) {
 			prefmask |= IORESOURCE_MEM_64;
 			ret = pbus_size_mem(bus, prefmask, prefmask,
 				  prefmask, prefmask,
@@ -2043,17 +2044,21 @@ static void pci_bridge_release_resources(struct pci_bus *bus,
 	 *	  io port.
 	 *     2. if there is non pref mmio assign fail, release bridge
 	 *	  nonpref mmio.
-	 *     3. if there is 64bit pref mmio assign fail, and bridge pref
+	 *     3. if there is pref mmio assign fail, and host bridge does
+	 *	  have 64bit mmio, release bridge pref mmio.
+	 *     4. if there is 64bit pref mmio assign fail, and bridge pref
 	 *	  is 64bit, release bridge pref mmio.
-	 *     4. if there is pref mmio assign fail, and bridge pref is
+	 *     5. if there is pref mmio assign fail, and bridge pref is
 	 *	  32bit mmio, release bridge pref mmio
-	 *     5. if there is pref mmio assign fail, and bridge pref is not
+	 *     6. if there is pref mmio assign fail, and bridge pref is not
 	 *	  assigned, release bridge nonpref mmio.
 	 */
 	if (type & IORESOURCE_IO)
 		idx = 0;
 	else if (!(type & IORESOURCE_PREFETCH))
 		idx = 1;
+	else if (!pci_find_host_bridge(bus)->has_mem64)
+		idx = 2;
 	else if ((type & IORESOURCE_MEM_64) &&
 		 (b_res[2].flags & IORESOURCE_MEM_64))
 		idx = 2;
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index b19aa5b..26aedde 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -205,6 +205,8 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 	struct resource *res = dev->resource + resno;
 	resource_size_t min;
 	int ret;
+	unsigned long mmio64 = pci_find_host_bridge(bus)->has_mem64 ?
+				IORESOURCE_MEM_64 : 0;
 
 	min = (res->flags & IORESOURCE_IO) ? PCIBIOS_MIN_IO : PCIBIOS_MIN_MEM;
 
@@ -216,7 +218,7 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 	 * things differently than they were sized, not everything will fit.
 	 */
 	ret = pci_bus_alloc_resource(bus, res, size, align, min,
-				     IORESOURCE_PREFETCH | IORESOURCE_MEM_64,
+				     IORESOURCE_PREFETCH | mmio64,
 				     pcibios_align_resource, dev);
 	if (ret == 0)
 		return 0;
@@ -225,7 +227,8 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 	 * If the prefetchable window is only 32 bits wide, we can put
 	 * 64-bit prefetchable resources in it.
 	 */
-	if ((res->flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64)) ==
+	if (mmio64 &&
+	    (res->flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64)) ==
 	     (IORESOURCE_PREFETCH | IORESOURCE_MEM_64)) {
 		ret = pci_bus_alloc_resource(bus, res, size, align, min,
 					     IORESOURCE_PREFETCH,
@@ -240,7 +243,7 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
 	 * non-prefetchable, the first call already tried the only possibility
 	 * so we don't need to try again.
 	 */
-	if (res->flags & (IORESOURCE_PREFETCH | IORESOURCE_MEM_64))
+	if (res->flags & (IORESOURCE_PREFETCH | mmio64))
 		ret = pci_bus_alloc_resource(bus, res, size, align, min, 0,
 					     pcibios_align_resource, dev);
 
-- 
1.8.4.5

[PATCH v3 39/51] sparc/PCI: Add mem64 resource parsing for root bus

[Yinghai Lu]

Found "no compatible bridge window" warning in boot log from T5-8.

pci 0000:00:01.0: can't claim BAR 15 [mem 0x100000000-0x4afffffff pref]: no compatible bridge window

That resource is above 4G, but does not get offset correctly as
root bus only report io and mem32.

pci_sun4v f02dbcfc: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x804000000000-0x80400fffffff] (bus address [0x0000-0xfffffff])
pci_bus 0000:00: root bus resource [mem 0x800000000000-0x80007effffff] (bus address [0x00000000-0x7effffff])
pci_bus 0000:00: root bus resource [bus 00-77]

Add mem64 handling in pci_common for sparc, so we can have 64bit resource
registered for root bus at first.

After patch, will have:
pci_sun4v f02dbcfc: PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x804000000000-0x80400fffffff] (bus address [0x0000-0xfffffff])
pci_bus 0000:00: root bus resource [mem 0x800000000000-0x80007effffff] (bus address [0x00000000-0x7effffff])
pci_bus 0000:00: root bus resource [mem 0x800100000000-0x8007ffffffff] (bus address [0x100000000-0x7ffffffff])
pci_bus 0000:00: root bus resource [bus 00-77]

Fixes: commit d63e2e1f3df9 ("sparc/PCI: Clip bridge windows to fit in upstream windows")
Link: http://lkml.kernel.org/r/CAE9FiQU1gJY1LYrxs+ma5LCTEEe4xmtjRG0aXJ9K_Tsu+m9Wuw@mail.gmail.com
Reported-by: David Ahern <david.ahern@oracle.com>
Tested-by: David Ahern <david.ahern@oracle.com>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: <stable@vger.kernel.org> #3.19
---
 arch/sparc/kernel/pci.c        |  7 ++++++-
 arch/sparc/kernel/pci_common.c | 15 +++++++++++++--
 arch/sparc/kernel/pci_impl.h   |  1 +
 3 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index c928bc6..bfd0b70 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -185,8 +185,10 @@ static unsigned long pci_parse_of_flags(u32 addr0)
 
 	if (addr0 & 0x02000000) {
 		flags = IORESOURCE_MEM | PCI_BASE_ADDRESS_SPACE_MEMORY;
-		flags |= (addr0 >> 22) & PCI_BASE_ADDRESS_MEM_TYPE_64;
 		flags |= (addr0 >> 28) & PCI_BASE_ADDRESS_MEM_TYPE_1M;
+		if (addr0 & 0x01000000)
+			flags |= IORESOURCE_MEM_64
+				 | PCI_BASE_ADDRESS_MEM_TYPE_64;
 		if (addr0 & 0x40000000)
 			flags |= IORESOURCE_PREFETCH
 				 | PCI_BASE_ADDRESS_MEM_PREFETCH;
@@ -660,6 +662,9 @@ struct pci_bus *pci_scan_one_pbm(struct pci_pbm_info *pbm,
 				pbm->io_space.start);
 	pci_add_resource_offset(&resources, &pbm->mem_space,
 				pbm->mem_space.start);
+	if (pbm->mem64_space.flags)
+		pci_add_resource_offset(&resources, &pbm->mem64_space,
+					pbm->mem_space.start);
 	pbm->busn.start = pbm->pci_first_busno;
 	pbm->busn.end	= pbm->pci_last_busno;
 	pbm->busn.flags	= IORESOURCE_BUS;
diff --git a/arch/sparc/kernel/pci_common.c b/arch/sparc/kernel/pci_common.c
index 944a065..a859a86 100644
--- a/arch/sparc/kernel/pci_common.c
+++ b/arch/sparc/kernel/pci_common.c
@@ -406,6 +406,7 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 	}
 
 	num_pbm_ranges = i / sizeof(*pbm_ranges);
+	memset(&pbm->mem64_space, 0, sizeof(struct resource));
 
 	for (i = 0; i < num_pbm_ranges; i++) {
 		const struct linux_prom_pci_ranges *pr = &pbm_ranges[i];
@@ -451,7 +452,11 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 			break;
 
 		case 3:
-			/* XXX 64-bit MEM handling XXX */
+			/* 64-bit MEM handling */
+			pbm->mem64_space.start = a;
+			pbm->mem64_space.end = a + size - 1UL;
+			pbm->mem64_space.flags = IORESOURCE_MEM;
+			break;
 
 		default:
 			break;
@@ -465,15 +470,21 @@ void pci_determine_mem_io_space(struct pci_pbm_info *pbm)
 		prom_halt();
 	}
 
-	printk("%s: PCI IO[%llx] MEM[%llx]\n",
+	printk("%s: PCI IO[%llx] MEM[%llx]",
 	       pbm->name,
 	       pbm->io_space.start,
 	       pbm->mem_space.start);
+	if (pbm->mem64_space.flags)
+		printk(" MEM64[%llx]",
+		       pbm->mem64_space.start);
+	printk("\n");
 
 	pbm->io_space.name = pbm->mem_space.name = pbm->name;
 
 	request_resource(&ioport_resource, &pbm->io_space);
 	request_resource(&iomem_resource, &pbm->mem_space);
+	if (pbm->mem64_space.flags)
+		request_resource(&iomem_resource, &pbm->mem64_space);
 
 	pci_register_legacy_regions(&pbm->io_space,
 				    &pbm->mem_space);
diff --git a/arch/sparc/kernel/pci_impl.h b/arch/sparc/kernel/pci_impl.h
index 75803c7..37222ca 100644
--- a/arch/sparc/kernel/pci_impl.h
+++ b/arch/sparc/kernel/pci_impl.h
@@ -97,6 +97,7 @@ struct pci_pbm_info {
 	/* PBM I/O and Memory space resources. */
 	struct resource			io_space;
 	struct resource			mem_space;
+	struct resource			mem64_space;
 	struct resource			busn;
 
 	/* Base of PCI Config space, can be per-PBM or shared. */
-- 
1.8.4.5

[PATCH v3 40/51] sparc/PCI: Add IORESOURCE_MEM_64 for 64-bit resource in of parsing

[Yinghai Lu]

For device resource PREF bit setting under bridge 64-bit pref resource,
we need to make sure only set PREF for 64bit resource, so set
IORESOUCE_MEM_64 for 64bit resource during of device resource flags
parsing.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=96261
Link: https://bugzilla.kernel.org/show_bug.cgi?id=96241
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: sparclinux@vger.kernel.org
---
 arch/sparc/kernel/of_device_32.c | 5 +++--
 arch/sparc/kernel/of_device_64.c | 5 +++--
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/arch/sparc/kernel/of_device_32.c b/arch/sparc/kernel/of_device_32.c
index 185aa96..3e9f273 100644
--- a/arch/sparc/kernel/of_device_32.c
+++ b/arch/sparc/kernel/of_device_32.c
@@ -83,11 +83,12 @@ static unsigned long of_bus_pci_get_flags(const u32 *addr, unsigned long flags)
 	case 0x01:
 		flags |= IORESOURCE_IO;
 		break;
-
 	case 0x02: /* 32 bits */
-	case 0x03: /* 64 bits */
 		flags |= IORESOURCE_MEM;
 		break;
+	case 0x03: /* 64 bits */
+		flags |= IORESOURCE_MEM | IORESOURCE_MEM_64;
+		break;
 	}
 	if (w & 0x40000000)
 		flags |= IORESOURCE_PREFETCH;
diff --git a/arch/sparc/kernel/of_device_64.c b/arch/sparc/kernel/of_device_64.c
index 7bbdc26..defee61 100644
--- a/arch/sparc/kernel/of_device_64.c
+++ b/arch/sparc/kernel/of_device_64.c
@@ -146,11 +146,12 @@ static unsigned long of_bus_pci_get_flags(const u32 *addr, unsigned long flags)
 	case 0x01:
 		flags |= IORESOURCE_IO;
 		break;
-
 	case 0x02: /* 32 bits */
-	case 0x03: /* 64 bits */
 		flags |= IORESOURCE_MEM;
 		break;
+	case 0x03: /* 64 bits */
+		flags |= IORESOURCE_MEM | IORESOURCE_MEM_64;
+		break;
 	}
 	if (w & 0x40000000)
 		flags |= IORESOURCE_PREFETCH;
-- 
1.8.4.5

[PATCH v3 41/51] powerpc/PCI: Add IORESOURCE_MEM_64 for 64-bit resource in of parsing

[Yinghai Lu]

For device resource PREF bit setting under bridge 64-bit pref resource,
we need to make sure only set PREF for 64bit resource, so set
IORESOUCE_MEM_64 for 64bit resource during of device resource flags
parsing.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=96261
Link: https://bugzilla.kernel.org/show_bug.cgi?id=96241
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Gavin Shan <gwshan@linux.vnet.ibm.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Cc: Anton Blanchard <anton@samba.org>
Cc: linuxppc-dev@lists.ozlabs.org
---
 arch/powerpc/kernel/pci_of_scan.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/kernel/pci_of_scan.c b/arch/powerpc/kernel/pci_of_scan.c
index 42e02a2..f31bfd0 100644
--- a/arch/powerpc/kernel/pci_of_scan.c
+++ b/arch/powerpc/kernel/pci_of_scan.c
@@ -44,8 +44,10 @@ static unsigned int pci_parse_of_flags(u32 addr0, int bridge)
 
 	if (addr0 & 0x02000000) {
 		flags = IORESOURCE_MEM | PCI_BASE_ADDRESS_SPACE_MEMORY;
-		flags |= (addr0 >> 22) & PCI_BASE_ADDRESS_MEM_TYPE_64;
 		flags |= (addr0 >> 28) & PCI_BASE_ADDRESS_MEM_TYPE_1M;
+		if (addr0 & 0x01000000)
+			flags |= IORESOURCE_MEM_64
+				 | PCI_BASE_ADDRESS_MEM_TYPE_64;
 		if (addr0 & 0x40000000)
 			flags |= IORESOURCE_PREFETCH
 				 | PCI_BASE_ADDRESS_MEM_PREFETCH;
-- 
1.8.4.5

[PATCH v3 42/51] of/PCI: Add IORESOURCE_MEM_64 for 64-bit resource

[Yinghai Lu]

For device resource PREF bit setting under bridge 64-bit pref resource,
we need to make sure only set PREF for 64bit resource, so set
IORESOUCE_MEM_64 for 64bit resource during of device resource flags
parsing.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=96261
Link: https://bugzilla.kernel.org/show_bug.cgi?id=96241
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Rob Herring <robh+dt@kernel.org>
Cc: devicetree@vger.kernel.org
---
 drivers/of/address.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/of/address.c b/drivers/of/address.c
index 8bfda6a..073125f 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -128,9 +128,11 @@ static unsigned int of_bus_pci_get_flags(const __be32 *addr)
 		flags |= IORESOURCE_IO;
 		break;
 	case 0x02: /* 32 bits */
-	case 0x03: /* 64 bits */
 		flags |= IORESOURCE_MEM;
 		break;
+	case 0x03: /* 64 bits */
+		flags |= IORESOURCE_MEM | IORESOURCE_MEM_64;
+		break;
 	}
 	if (w & 0x40000000)
 		flags |= IORESOURCE_PREFETCH;
-- 
1.8.4.5

[PATCH v3 43/51] PCI: Treat optional as must in first try for bridge rescan

[Yinghai Lu]

For rescan bridge/bus that children are removed before, we should treat
optional as must just like root bus the boot time in 19aa7ee432ce
(PCI: make re-allocation try harder by reassigning ranges higher in
the heirarchy).

The reason: allocate must and expand to optional path do not
put failed resource to fail list, so will lose must info before
next try.

So we are using following way:
1. First and following try before last try:
   We don't keep realloc list so treat every optional as must.
   allocate for must+optional and put failed in the fail list.
   then size info (include must and optonal separatedly) will be kept
   for next try.
2. last try:
   a: try to allocate must+optional to see if all get allocated.
   b: try to allocate must then expand to optional.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 54f0261..541cea5 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -2361,25 +2361,34 @@ void __init pci_assign_unassigned_resources(void)
 void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
 {
 	struct pci_bus *parent = bridge->subordinate;
-	LIST_HEAD(add_list); /* list of resources that
+	LIST_HEAD(realloc_head); /* list of resources that
 					want additional resources */
+	struct list_head *add_list = NULL;
 	int tried_times = 0;
 	LIST_HEAD(fail_head);
 	struct pci_dev_resource *fail_res;
 	int retval;
 	unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
 				  IORESOURCE_PREFETCH | IORESOURCE_MEM_64;
+	int pci_try_num = 2;
 
 again:
-	__pci_bus_size_bridges(parent, &add_list);
-	__pci_bridge_assign_resources(bridge, &add_list, &fail_head);
-	__pci_bus_check_realloc(&add_list);
+	/*
+	 * last try will use add_list, otherwise will try good to have as
+	 * must have, so can realloc parent bridge resource
+	 */
+	if (tried_times + 1 == pci_try_num)
+		add_list = &realloc_head;
+	__pci_bus_size_bridges(parent, add_list);
+	__pci_bridge_assign_resources(bridge, add_list, &fail_head);
+	if (add_list)
+		__pci_bus_check_realloc(add_list);
 	tried_times++;
 
 	if (list_empty(&fail_head))
 		goto enable_all;
 
-	if (tried_times >= 2) {
+	if (tried_times >= pci_try_num) {
 		/* still fail, don't need to try more */
 		free_list(&fail_head);
 		goto enable_all;
-- 
1.8.4.5

[PATCH v3 44/51] PCI: Get new realloc size for bridge for last try

[Yinghai Lu]

Current realloc path would not shrink bridge resource through
pbus_size_mem() checking with the old size.

That cause problem: when "must+optional" resource allocation fails,
the cached bridge resource size will prevent "must" resource to get
allocated smaller resource.

Clear the old resource size for last try or third and later try.

-v3: change reset_bridge_resource_size to static according to Fengguang.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
Tested-by: TJ <linux@iam.tj>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 541cea5..83e3201 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -2256,6 +2256,17 @@ static enum enable_type pci_realloc_detect(struct pci_bus *bus,
 }
 #endif
 
+static void reset_bridge_resource_size(struct pci_dev *dev,
+				       struct resource *res)
+{
+	int idx = res - &dev->resource[0];
+
+	if (idx >= PCI_BRIDGE_RESOURCES && idx <= PCI_BRIDGE_RESOURCE_END) {
+		res->start = 0;
+		res->end = res->start - 1;
+	}
+}
+
 /*
  * first try will not touch pci bridge res
  * second and later try will clear small leaf bridge res
@@ -2338,8 +2349,13 @@ again:
 		struct resource *res = fail_res->res;
 
 		restore_resource(fail_res, res);
-		if (fail_res->dev->subordinate)
+		if (fail_res->dev->subordinate) {
 			res->flags = 0;
+			/* last or third times and later */
+			if (tried_times + 1 == pci_try_num ||
+			    tried_times + 1 > 2)
+				reset_bridge_resource_size(fail_res->dev, res);
+		}
 	}
 	free_list(&fail_head);
 
@@ -2411,8 +2427,11 @@ again:
 		struct resource *res = fail_res->res;
 
 		restore_resource(fail_res, res);
-		if (fail_res->dev->subordinate)
+		if (fail_res->dev->subordinate) {
 			res->flags = 0;
+			/* last time */
+			reset_bridge_resource_size(fail_res->dev, res);
+		}
 	}
 	free_list(&fail_head);
 
-- 
1.8.4.5

[PATCH v3 45/51] PCI: Don't release sibiling bridge resources during hotplug

[Yinghai Lu]

On hotplug path, we can not touch sibling bridges that is out
side of the slot.

That could happen when BIOS does not assign some bridge BARs and
later can not assign resource to them in first try.

Check if fail dev is the parent bridge, then just use subordinate
bus instead use parent bus.

Reported-by: Andreas Noever <andreas.noever@gmail.com>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 83e3201..63a5d4cf 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -2417,10 +2417,16 @@ again:
 	 * Try to release leaf bridge's resources that doesn't fit resource of
 	 * child device under that bridge
 	 */
-	list_for_each_entry(fail_res, &fail_head, list)
-		pci_bus_release_bridge_resources(fail_res->dev->bus,
+	list_for_each_entry(fail_res, &fail_head, list) {
+		struct pci_bus *bus = fail_res->dev->bus;
+
+		if (fail_res->dev == bridge)
+			bus = bridge->subordinate;
+
+		pci_bus_release_bridge_resources(bus,
 						 fail_res->flags & type_mask,
 						 whole_subtree);
+	}
 
 	/* restore size and flags */
 	list_for_each_entry(fail_res, &fail_head, list) {
-- 
1.8.4.5

[PATCH v3 46/51] PCI: Don't release fixed resource for realloc

[Yinghai Lu]

We should not release bridge resource if there is fixed resources
under it, otherwise the children firmware would stop working.

Reported-by: Paul Johnson <pjay@nwtrail.com>
Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=92351
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: stable@vger.kernel.org
---
 drivers/pci/setup-bus.c |  6 ++++--
 include/linux/ioport.h  |  2 +-
 kernel/resource.c       | 28 ++++++++++++++++++++++++++--
 3 files changed, 31 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 63a5d4cf..dcbbe04 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -2070,14 +2070,16 @@ static void pci_bridge_release_resources(struct pci_bus *bus,
 
 	r = &b_res[idx];
 
-	if (!r->parent)
+	if (!r->parent || r->flags & IORESOURCE_PCI_FIXED)
 		return;
 
 	/*
 	 * if there are children under that, we should release them
 	 *  all
 	 */
-	release_child_resources(r);
+	if (!release_child_resources(r))
+		return;
+
 	if (!release_resource(r)) {
 		type = old_flags = r->flags & type_mask;
 		dev_printk(KERN_DEBUG, &dev->dev, "resource %d %pR released\n",
diff --git a/include/linux/ioport.h b/include/linux/ioport.h
index 388e3ae..27dbb18 100644
--- a/include/linux/ioport.h
+++ b/include/linux/ioport.h
@@ -141,7 +141,7 @@ extern struct resource iomem_resource;
 extern struct resource *request_resource_conflict(struct resource *root, struct resource *new);
 extern int request_resource(struct resource *root, struct resource *new);
 extern int release_resource(struct resource *new);
-void release_child_resources(struct resource *new);
+bool release_child_resources(struct resource *new);
 extern void reserve_region_with_split(struct resource *root,
 			     resource_size_t start, resource_size_t end,
 			     const char *name);
diff --git a/kernel/resource.c b/kernel/resource.c
index 697b8ca..4e43f9b 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -276,11 +276,35 @@ static void __release_child_resources(struct resource *r)
 	}
 }
 
-void release_child_resources(struct resource *r)
+static bool __has_fixed_child_resources(struct resource *r)
 {
+	struct resource *p;
+
+	p = r->child;
+	while (p) {
+		if (p->flags & IORESOURCE_PCI_FIXED)
+			return true;
+
+		if (__has_fixed_child_resources(p))
+			return true;
+
+		p = p->sibling;
+	}
+
+	return false;
+}
+
+bool release_child_resources(struct resource *r)
+{
+	bool fixed;
+
 	write_lock(&resource_lock);
-	__release_child_resources(r);
+	fixed = __has_fixed_child_resources(r);
+	if (!fixed)
+		__release_child_resources(r);
 	write_unlock(&resource_lock);
+
+	return !fixed;
 }
 
 /**
-- 
1.8.4.5

[PATCH v3 47/51] PCI: Claim fixed resource during remove/rescan path

[Yinghai Lu]

We need to reserve them before pci_bus_add().

During driver loading kernel will check if those reource get reserved.

On remove/rescan path, we will leave those fixed resource
not reserved, as we don't call pcibios_resource_survery() before
pci_assign_unassigned_bus_resources(), and that is intentional
for us to get new resources for rescan.
We do have example to use rescan to make device get resource
allocated.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/quirks.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index e9fd0e9..9efa983 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -324,6 +324,23 @@ static void quirk_s3_64M(struct pci_dev *dev)
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_S3,	PCI_DEVICE_ID_S3_868,		quirk_s3_64M);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_S3,	PCI_DEVICE_ID_S3_968,		quirk_s3_64M);
 
+/* for pci remove and rescan */
+static void quirk_allocate_fixed(struct pci_dev *dev)
+{
+	int i;
+	for (i = 0; i < PCI_NUM_RESOURCES; i++) {
+		struct resource *r = &dev->resource[i];
+
+		if (r->parent ||
+		    !(r->flags & IORESOURCE_PCI_FIXED) ||
+		    !(r->flags & (IORESOURCE_IO | IORESOURCE_MEM)))
+			continue;
+
+		pci_claim_resource(dev, i);
+	}
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_ANY_ID,	PCI_ANY_ID,	quirk_allocate_fixed);
+
 static void quirk_io(struct pci_dev *dev, int pos, unsigned size,
 		     const char *name)
 {
-- 
1.8.4.5

[PATCH v3 48/51] PCI: Set resource to FIXED for lsi devices

[Yinghai Lu]

LSI HBA firmware stop responding pci read from host if pci core ever change
pci device BAR values.

Set their resources to FIXED, so will allow realloc to skip them.

Reported-by: Paul Johnson <pjay@nwtrail.com>
Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=92351
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
Cc: stable@vger.kernel.org
---
 drivers/pci/pci.h       |  1 +
 drivers/pci/quirks.c    | 20 ++++++++++++++++++++
 drivers/pci/setup-bus.c |  4 ++++
 3 files changed, 25 insertions(+)

diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 1804d44..dec1c18 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -168,6 +168,7 @@ static inline void pci_msix_clear_and_set_ctrl(struct pci_dev *dev, u16 clear, u
 }
 
 void pci_realloc_get_opt(char *);
+bool pci_realloc_user_enabled(void);
 
 static inline int pci_no_d1d2(struct pci_dev *dev)
 {
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 9efa983..a2945a4 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -324,6 +324,26 @@ static void quirk_s3_64M(struct pci_dev *dev)
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_S3,	PCI_DEVICE_ID_S3_868,		quirk_s3_64M);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_S3,	PCI_DEVICE_ID_S3_968,		quirk_s3_64M);
 
+/*
+ * LSI devices firmware does not like BAR get changed
+ */
+static void quirk_bar_fixed(struct pci_dev *dev)
+{
+	int i;
+
+	if (pci_realloc_user_enabled())
+		return;
+
+	for (i = 0; i < PCI_STD_RESOURCE_END; i++) {
+		struct resource *r = &dev->resource[i];
+
+		if (!r->start || !r->flags)
+			continue;
+		r->flags |= IORESOURCE_PCI_FIXED;
+	}
+}
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_LSI_LOGIC,	PCI_ANY_ID,	quirk_bar_fixed);
+
 /* for pci remove and rescan */
 static void quirk_allocate_fixed(struct pci_dev *dev)
 {
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index dcbbe04..7b296b0 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -2207,6 +2207,10 @@ void __init pci_realloc_get_opt(char *str)
 	else if (!strncmp(str, "on", 2))
 		pci_realloc_enable = user_enabled;
 }
+bool pci_realloc_user_enabled(void)
+{
+	return pci_realloc_enable == user_enabled;
+}
 static bool pci_realloc_enabled(enum enable_type enable)
 {
 	return enable >= user_enabled;
-- 
1.8.4.5

[PATCH v3 49/51] PCI, x86: Add pci=assign_pref_bars to re-allocate pref bars

[Yinghai Lu]

So could reallocate pref mmio64 above 4G later.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 arch/x86/include/asm/pci_x86.h |  1 +
 arch/x86/pci/common.c          |  3 +++
 arch/x86/pci/i386.c            | 56 ++++++++++++++++++++++++++----------------
 3 files changed, 39 insertions(+), 21 deletions(-)

diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index ddac225..7b634b8 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -34,6 +34,7 @@ do {						\
 #define PCI_NOASSIGN_ROMS	0x80000
 #define PCI_ROOT_NO_CRS		0x100000
 #define PCI_NOASSIGN_BARS	0x200000
+#define PCI_ASSIGN_PREF_BARS	0x400000
 
 extern unsigned int pci_probe;
 extern unsigned long pirq_table_addr;
diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c
index e8df922..dcc7c48 100644
--- a/arch/x86/pci/common.c
+++ b/arch/x86/pci/common.c
@@ -606,6 +606,9 @@ char *__init pcibios_setup(char *str)
 	} else if (!strcmp(str, "assign-busses")) {
 		pci_probe |= PCI_ASSIGN_ALL_BUSSES;
 		return NULL;
+	} else if (!strcmp(str, "assign_pref_bars")) {
+		pci_probe |= PCI_ASSIGN_PREF_BARS;
+		return NULL;
 	} else if (!strcmp(str, "use_crs")) {
 		pci_probe |= PCI_USE__CRS;
 		return NULL;
diff --git a/arch/x86/pci/i386.c b/arch/x86/pci/i386.c
index 3f17726..0b74efe 100644
--- a/arch/x86/pci/i386.c
+++ b/arch/x86/pci/i386.c
@@ -208,16 +208,25 @@ static void pcibios_allocate_bridge_resources(struct pci_dev *dev)
 			continue;
 		if (r->parent)	/* Already allocated */
 			continue;
-		if (!r->start || pci_claim_bridge_resource(dev, idx) < 0) {
-			/*
-			 * Something is wrong with the region.
-			 * Invalidate the resource to prevent
-			 * child resource allocations in this
-			 * range.
-			 */
-			r->start = r->end = 0;
-			r->flags = 0;
-		}
+
+		if ((r->flags & IORESOURCE_PREFETCH) &&
+		    (pci_probe & PCI_ASSIGN_PREF_BARS))
+			goto clear;
+
+		if (!r->start)
+			goto clear;
+
+		if (pci_claim_bridge_resource(dev, idx) == 0)
+			continue;
+
+clear:
+		/*
+		 * Something is wrong with the region.
+		 * Invalidate the resource to prevent
+		 * child resource allocations in this range.
+		 */
+		r->start = r->end = 0;
+		r->flags = 0;
 	}
 }
 
@@ -263,21 +272,26 @@ static void pcibios_allocate_dev_resources(struct pci_dev *dev, int pass)
 			else
 				disabled = !(command & PCI_COMMAND_MEMORY);
 			if (pass == disabled) {
+				if ((r->flags & IORESOURCE_PREFETCH) &&
+				    (pci_probe & PCI_ASSIGN_PREF_BARS))
+					goto clear;
+
 				dev_dbg(&dev->dev,
 					"BAR %d: reserving %pr (d=%d, p=%d)\n",
 					idx, r, disabled, pass);
-				if (pci_claim_resource(dev, idx) < 0) {
-					if (r->flags & IORESOURCE_PCI_FIXED) {
-						dev_info(&dev->dev, "BAR %d %pR is immovable\n",
-							 idx, r);
-					} else {
-						/* We'll assign a new address later */
-						pcibios_save_fw_addr(dev,
-								idx, r->start);
-						r->end -= r->start;
-						r->start = 0;
-					}
+				if (pci_claim_resource(dev, idx) == 0)
+					continue;
+				if (r->flags & IORESOURCE_PCI_FIXED) {
+					dev_info(&dev->dev, "BAR %d %pR is immovable\n",
+						 idx, r);
+					continue;
 				}
+
+clear:
+				/* We'll assign a new address later */
+				pcibios_save_fw_addr(dev, idx, r->start);
+				r->end -= r->start;
+				r->start = 0;
 			}
 		}
 	if (!pass) {
-- 
1.8.4.5

[PATCH v3 50/51] PCI: Introduce resource_disabled()

[Yinghai Lu]

so we can cover !flags and IORESOURCE_DISABLED both.

Cc: linux-alpha@vger.kernel.org
Cc: linux-ia64@vger.kernel.org
Cc: linux-am33-list@redhat.com
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-s390@vger.kernel.org
Cc: sparclinux@vger.kernel.org
Cc: linux-pci@vger.kernel.org
Cc: linux-xtensa@linux-xtensa.org
Cc: iommu@lists.linux-foundation.org
Cc: linux-sh@vger.kernel.org
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 arch/alpha/kernel/pci.c                   |  2 +-
 arch/ia64/pci/pci.c                       |  4 ++--
 arch/microblaze/pci/pci-common.c          | 15 ++++++++-------
 arch/mn10300/unit-asb2305/pci-asb2305.c   |  4 ++--
 arch/mn10300/unit-asb2305/pci.c           |  4 ++--
 arch/powerpc/kernel/pci-common.c          | 16 +++++++++-------
 arch/powerpc/platforms/powernv/pci-ioda.c | 12 ++++++------
 arch/s390/pci/pci.c                       |  2 +-
 arch/sparc/kernel/pci.c                   |  2 +-
 arch/x86/pci/i386.c                       |  4 ++--
 arch/xtensa/kernel/pci.c                  |  4 ++--
 drivers/iommu/intel-iommu.c               |  3 ++-
 drivers/pci/host/pcie-rcar.c              |  2 +-
 drivers/pci/iov.c                         |  2 +-
 drivers/pci/probe.c                       |  2 +-
 drivers/pci/quirks.c                      |  4 ++--
 drivers/pci/rom.c                         |  2 +-
 drivers/pci/setup-bus.c                   |  8 ++++----
 drivers/pci/setup-res.c                   |  2 +-
 include/linux/ioport.h                    |  4 ++++
 20 files changed, 53 insertions(+), 45 deletions(-)

diff --git a/arch/alpha/kernel/pci.c b/arch/alpha/kernel/pci.c
index 82f738e..91a7153 100644
--- a/arch/alpha/kernel/pci.c
+++ b/arch/alpha/kernel/pci.c
@@ -282,7 +282,7 @@ pcibios_claim_one_bus(struct pci_bus *b)
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 
-			if (r->parent || !r->start || !r->flags)
+			if (r->parent || !r->start || resource_disabled(r))
 				continue;
 			if (pci_has_flag(PCI_PROBE_ONLY) ||
 			    (r->flags & IORESOURCE_PCI_FIXED)) {
diff --git a/arch/ia64/pci/pci.c b/arch/ia64/pci/pci.c
index 7cc3be9..cc293ea 100644
--- a/arch/ia64/pci/pci.c
+++ b/arch/ia64/pci/pci.c
@@ -501,7 +501,7 @@ void pcibios_fixup_device_resources(struct pci_dev *dev)
 	for (idx = 0; idx < PCI_BRIDGE_RESOURCES; idx++) {
 		struct resource *r = &dev->resource[idx];
 
-		if (!r->flags || r->parent || !r->start)
+		if (resource_disabled(r) || r->parent || !r->start)
 			continue;
 
 		pci_claim_resource(dev, idx);
@@ -519,7 +519,7 @@ static void pcibios_fixup_bridge_resources(struct pci_dev *dev)
 	for (idx = PCI_BRIDGE_RESOURCES; idx < PCI_NUM_RESOURCES; idx++) {
 		struct resource *r = &dev->resource[idx];
 
-		if (!r->flags || r->parent || !r->start)
+		if (resource_disabled(r) || r->parent || !r->start)
 			continue;
 
 		pci_claim_bridge_resource(dev, idx);
diff --git a/arch/microblaze/pci/pci-common.c b/arch/microblaze/pci/pci-common.c
index 09b1af6..c123d3c 100644
--- a/arch/microblaze/pci/pci-common.c
+++ b/arch/microblaze/pci/pci-common.c
@@ -705,7 +705,7 @@ static void pcibios_fixup_resources(struct pci_dev *dev)
 	}
 	for (i = 0; i < DEVICE_COUNT_RESOURCE; i++) {
 		struct resource *res = dev->resource + i;
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 		if (res->start == 0) {
 			pr_debug("PCI:%s Resource %d %016llx-%016llx [%x]",
@@ -806,7 +806,7 @@ static void pcibios_fixup_bridge(struct pci_bus *bus)
 	pci_bus_for_each_resource(bus, res, i) {
 		if (!res)
 			continue;
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 		if (i >= 3 && bus->self->transparent)
 			continue;
@@ -985,7 +985,7 @@ static void pcibios_allocate_bus_resources(struct pci_bus *bus)
 		 pci_domain_nr(bus), bus->number);
 
 	pci_bus_for_each_resource(bus, res, i) {
-		if (!res || !res->flags
+		if (!res || resource_disabled(res)
 		    || res->start > res->end || res->parent)
 			continue;
 		if (bus->parent == NULL)
@@ -1087,7 +1087,8 @@ static void __init pcibios_allocate_resources(int pass)
 			r = &dev->resource[idx];
 			if (r->parent)		/* Already allocated */
 				continue;
-			if (!r->flags || (r->flags & IORESOURCE_UNSET))
+			if (resource_disabled(r) ||
+			    (r->flags & IORESOURCE_UNSET))
 				continue;	/* Not assigned at all */
 			/* We only allocate ROMs on pass 1 just in case they
 			 * have been screwed up by firmware
@@ -1218,7 +1219,7 @@ void pcibios_claim_one_bus(struct pci_bus *bus)
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 
-			if (r->parent || !r->start || !r->flags)
+			if (r->parent || !r->start || resource_disabled(r))
 				continue;
 
 			pr_debug("PCI: Claiming %s: ", pci_name(dev));
@@ -1278,7 +1279,7 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose,
 	res->start = (res->start + io_offset) & 0xffffffffu;
 	res->end = (res->end + io_offset) & 0xffffffffu;
 
-	if (!res->flags) {
+	if (resource_disabled(res)) {
 		pr_warn("PCI: I/O resource not set for host ");
 		pr_cont("bridge %s (domain %d)\n",
 			hose->dn->full_name, hose->global_number);
@@ -1298,7 +1299,7 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose,
 	/* Hookup PHB Memory resources */
 	for (i = 0; i < 3; ++i) {
 		res = &hose->mem_resources[i];
-		if (!res->flags) {
+		if (resource_disabled(res)) {
 			if (i > 0)
 				continue;
 			pr_err("PCI: Memory resource 0 not set for ");
diff --git a/arch/mn10300/unit-asb2305/pci-asb2305.c b/arch/mn10300/unit-asb2305/pci-asb2305.c
index b5b036f..a249821 100644
--- a/arch/mn10300/unit-asb2305/pci-asb2305.c
+++ b/arch/mn10300/unit-asb2305/pci-asb2305.c
@@ -103,7 +103,7 @@ static void __init pcibios_allocate_bus_resources(struct list_head *bus_list)
 			     idx < PCI_NUM_RESOURCES;
 			     idx++) {
 				r = &dev->resource[idx];
-				if (!r->flags)
+				if (resource_disabled(r))
 					continue;
 				if (!r->start ||
 				    pci_claim_bridge_resource(dev, idx) < 0) {
@@ -189,7 +189,7 @@ static int __init pcibios_assign_resources(void)
 		   addresses. */
 		for_each_pci_dev(dev) {
 			r = &dev->resource[PCI_ROM_RESOURCE];
-			if (!r->flags || !r->start)
+			if (resource_disabled(r) || !r->start)
 				continue;
 			if (pci_claim_resource(dev, PCI_ROM_RESOURCE) < 0) {
 				r->end -= r->start;
diff --git a/arch/mn10300/unit-asb2305/pci.c b/arch/mn10300/unit-asb2305/pci.c
index 3dfe2d3..ad77b18 100644
--- a/arch/mn10300/unit-asb2305/pci.c
+++ b/arch/mn10300/unit-asb2305/pci.c
@@ -291,7 +291,7 @@ static void pcibios_fixup_device_resources(struct pci_dev *dev)
 	for (idx = 0; idx < PCI_BRIDGE_RESOURCES; idx++) {
 		struct resource *r = &dev->resource[idx];
 
-		if (!r->flags || r->parent || !r->start)
+		if (resource_disabled(r) || r->parent || !r->start)
 			continue;
 
 		pci_claim_resource(dev, idx);
@@ -308,7 +308,7 @@ static void pcibios_fixup_bridge_resources(struct pci_dev *dev)
 	for (idx = PCI_BRIDGE_RESOURCES; idx < PCI_NUM_RESOURCES; idx++) {
 		struct resource *r = &dev->resource[idx];
 
-		if (!r->flags || r->parent || !r->start)
+		if (resource_disabled(r) || r->parent || !r->start)
 			continue;
 
 		pci_claim_bridge_resource(dev, idx);
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 2d8d654..10bb5cd 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -810,7 +810,7 @@ static void pcibios_fixup_resources(struct pci_dev *dev)
 	for (i = 0; i < DEVICE_COUNT_RESOURCE; i++) {
 		struct resource *res = dev->resource + i;
 		struct pci_bus_region reg;
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 
 		/* If we're going to re-assign everything, we mark all resources
@@ -927,7 +927,7 @@ static void pcibios_fixup_bridge(struct pci_bus *bus)
 	struct pci_dev *dev = bus->self;
 
 	pci_bus_for_each_resource(bus, res, i) {
-		if (!res || !res->flags)
+		if (!res || resource_disabled(res))
 			continue;
 		if (i >= 3 && bus->self->transparent)
 			continue;
@@ -1174,7 +1174,8 @@ static void pcibios_allocate_bus_resources(struct pci_bus *bus)
 		 pci_domain_nr(bus), bus->number);
 
 	pci_bus_for_each_resource(bus, res, i) {
-		if (!res || !res->flags || res->start > res->end || res->parent)
+		if (!res || resource_disabled(res) ||
+		    res->start > res->end || res->parent)
 			continue;
 
 		/* If the resource was left unset at this point, we clear it */
@@ -1281,7 +1282,8 @@ static void __init pcibios_allocate_resources(int pass)
 			r = &dev->resource[idx];
 			if (r->parent)		/* Already allocated */
 				continue;
-			if (!r->flags || (r->flags & IORESOURCE_UNSET))
+			if (resource_disabled(r) ||
+			    (r->flags & IORESOURCE_UNSET))
 				continue;	/* Not assigned at all */
 			/* We only allocate ROMs on pass 1 just in case they
 			 * have been screwed up by firmware
@@ -1419,7 +1421,7 @@ void pcibios_claim_one_bus(struct pci_bus *bus)
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 
-			if (r->parent || !r->start || !r->flags)
+			if (r->parent || !r->start || resource_disabled(r))
 				continue;
 
 			pr_debug("PCI: Claiming %s: "
@@ -1504,7 +1506,7 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose,
 	/* Hookup PHB IO resource */
 	res = &hose->io_resource;
 
-	if (!res->flags) {
+	if (resource_disabled(res)) {
 		pr_info("PCI: I/O resource not set for host"
 		       " bridge %s (domain %d)\n",
 		       hose->dn->full_name, hose->global_number);
@@ -1522,7 +1524,7 @@ static void pcibios_setup_phb_resources(struct pci_controller *hose,
 	/* Hookup PHB Memory resources */
 	for (i = 0; i < 3; ++i) {
 		res = &hose->mem_resources[i];
-		if (!res->flags) {
+		if (resource_disabled(res)) {
 			if (i == 0)
 				printk(KERN_ERR "PCI: Memory resource 0 not set for "
 				       "host bridge %s (domain %d)\n",
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index 5738d31..9c42f52 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -905,7 +905,7 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)
 	num_vfs = pdn->num_vfs;
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &dev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || !res->parent)
+		if (resource_disabled(res) || !res->parent)
 			continue;
 
 		if (!pnv_pci_is_mem_pref_64(res->flags))
@@ -936,7 +936,7 @@ static int pnv_pci_vf_resource_shift(struct pci_dev *dev, int offset)
 	 */
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &dev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || !res->parent)
+		if (resource_disabled(res) || !res->parent)
 			continue;
 
 		if (!pnv_pci_is_mem_pref_64(res->flags))
@@ -1218,7 +1218,7 @@ static int pnv_pci_vf_assign_m64(struct pci_dev *pdev, u16 num_vfs)
 
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &pdev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || !res->parent)
+		if (resource_disabled(res) || !res->parent)
 			continue;
 
 		if (!pnv_pci_is_mem_pref_64(res->flags))
@@ -2771,7 +2771,7 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev)
 
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &pdev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || res->parent)
+		if (resource_disabled(res) || res->parent)
 			continue;
 		if (!pnv_pci_is_mem_pref_64(res->flags)) {
 			dev_warn(&pdev->dev, " non M64 VF BAR%d: %pR\n",
@@ -2793,7 +2793,7 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev)
 
 	for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
 		res = &pdev->resource[i + PCI_IOV_RESOURCES];
-		if (!res->flags || res->parent)
+		if (resource_disabled(res) || res->parent)
 			continue;
 		if (!pnv_pci_is_mem_pref_64(res->flags)) {
 			dev_warn(&pdev->dev, "Skipping expanding VF BAR%d: %pR\n",
@@ -2834,7 +2834,7 @@ static void pnv_ioda_setup_pe_seg(struct pci_controller *hose,
 	BUG_ON(!(pe->flags & (PNV_IODA_PE_BUS | PNV_IODA_PE_BUS_ALL)));
 
 	pci_bus_for_each_resource(pe->pbus, res, i) {
-		if (!res || !res->flags ||
+		if (!res || resource_disabled(res) ||
 		    res->start > res->end)
 			continue;
 
diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c
index 598f023..d850749 100644
--- a/arch/s390/pci/pci.c
+++ b/arch/s390/pci/pci.c
@@ -658,7 +658,7 @@ int pcibios_add_device(struct pci_dev *pdev)
 
 	for (i = 0; i < PCI_BAR_COUNT; i++) {
 		res = &pdev->resource[i];
-		if (res->parent || !res->flags)
+		if (res->parent || resource_disabled(res))
 			continue;
 		pci_claim_resource(pdev, i);
 	}
diff --git a/arch/sparc/kernel/pci.c b/arch/sparc/kernel/pci.c
index bfd0b70..f5e9536 100644
--- a/arch/sparc/kernel/pci.c
+++ b/arch/sparc/kernel/pci.c
@@ -630,7 +630,7 @@ static void pci_claim_bus_resources(struct pci_bus *bus)
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			struct resource *r = &dev->resource[i];
 
-			if (r->parent || !r->start || !r->flags)
+			if (r->parent || !r->start || resource_disabled(r))
 				continue;
 
 			if (ofpci_verbose)
diff --git a/arch/x86/pci/i386.c b/arch/x86/pci/i386.c
index 0b74efe..8538edb 100644
--- a/arch/x86/pci/i386.c
+++ b/arch/x86/pci/i386.c
@@ -204,7 +204,7 @@ static void pcibios_allocate_bridge_resources(struct pci_dev *dev)
 
 	for (idx = PCI_BRIDGE_RESOURCES; idx < PCI_NUM_RESOURCES; idx++) {
 		r = &dev->resource[idx];
-		if (!r->flags)
+		if (resource_disabled(r))
 			continue;
 		if (r->parent)	/* Already allocated */
 			continue;
@@ -333,7 +333,7 @@ static void pcibios_allocate_dev_rom_resource(struct pci_dev *dev)
 	 * addresses.
 	 */
 	r = &dev->resource[PCI_ROM_RESOURCE];
-	if (!r->flags || !r->start)
+	if (resource_disabled(r) || !r->start)
 		return;
 	if (r->parent) /* Already allocated */
 		return;
diff --git a/arch/xtensa/kernel/pci.c b/arch/xtensa/kernel/pci.c
index b848cc3..f34d061 100644
--- a/arch/xtensa/kernel/pci.c
+++ b/arch/xtensa/kernel/pci.c
@@ -142,7 +142,7 @@ static void __init pci_controller_apertures(struct pci_controller *pci_ctrl,
 
 	io_offset = (unsigned long)pci_ctrl->io_space.base;
 	res = &pci_ctrl->io_resource;
-	if (!res->flags) {
+	if (resource_disabled(res)) {
 		if (io_offset)
 			printk (KERN_ERR "I/O resource not set for host"
 				" bridge %d\n", pci_ctrl->index);
@@ -156,7 +156,7 @@ static void __init pci_controller_apertures(struct pci_controller *pci_ctrl,
 
 	for (i = 0; i < 3; i++) {
 		res = &pci_ctrl->mem_resources[i];
-		if (!res->flags) {
+		if (resource_disabled(res)) {
 			if (i > 0)
 				continue;
 			printk(KERN_ERR "Memory resource not set for "
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 0649b94..ee9bdbf 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1743,7 +1743,8 @@ static int dmar_init_reserved_ranges(void)
 
 		for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 			r = &pdev->resource[i];
-			if (!r->flags || !(r->flags & IORESOURCE_MEM))
+			if (resource_disabled(r) ||
+			    !(r->flags & IORESOURCE_MEM))
 				continue;
 			iova = reserve_iova(&reserved_iova_list,
 					    IOVA_PFN(r->start),
diff --git a/drivers/pci/host/pcie-rcar.c b/drivers/pci/host/pcie-rcar.c
index c086210..a6d6bd0 100644
--- a/drivers/pci/host/pcie-rcar.c
+++ b/drivers/pci/host/pcie-rcar.c
@@ -365,7 +365,7 @@ static int rcar_pcie_setup(int nr, struct pci_sys_data *sys)
 	for (i = 0; i < RCAR_PCI_MAX_RESOURCES; i++) {
 
 		res = &pcie->res[i];
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 
 		rcar_pcie_setup_window(i, pcie);
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index ee0ebff..808d08b 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -438,7 +438,7 @@ found:
 		res = &dev->resource[i + PCI_IOV_RESOURCES];
 		bar64 = __pci_read_base(dev, pci_bar_unknown, res,
 					pos + PCI_SRIOV_BAR + i * 4);
-		if (!res->flags)
+		if (resource_disabled(res))
 			continue;
 		if (resource_size(res) & (PAGE_SIZE - 1)) {
 			rc = -EIO;
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 14bdbca..5440fb1 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -2129,7 +2129,7 @@ void pci_bus_release_busn_res(struct pci_bus *b)
 	struct resource *res = &b->busn_res;
 	int ret;
 
-	if (!res->flags || !res->parent)
+	if (resource_disabled(res) || !res->parent)
 		return;
 
 	ret = release_resource(res);
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index a2945a4..abbc2ac 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -337,7 +337,7 @@ static void quirk_bar_fixed(struct pci_dev *dev)
 	for (i = 0; i < PCI_STD_RESOURCE_END; i++) {
 		struct resource *r = &dev->resource[i];
 
-		if (!r->start || !r->flags)
+		if (!r->start || resource_disabled(r))
 			continue;
 		r->flags |= IORESOURCE_PCI_FIXED;
 	}
@@ -351,7 +351,7 @@ static void quirk_allocate_fixed(struct pci_dev *dev)
 	for (i = 0; i < PCI_NUM_RESOURCES; i++) {
 		struct resource *r = &dev->resource[i];
 
-		if (r->parent ||
+		if (r->parent || resource_disabled(r) ||
 		    !(r->flags & IORESOURCE_PCI_FIXED) ||
 		    !(r->flags & (IORESOURCE_IO | IORESOURCE_MEM)))
 			continue;
diff --git a/drivers/pci/rom.c b/drivers/pci/rom.c
index eb0ad53..585a9d3 100644
--- a/drivers/pci/rom.c
+++ b/drivers/pci/rom.c
@@ -28,7 +28,7 @@ int pci_enable_rom(struct pci_dev *pdev)
 	struct pci_bus_region region;
 	u32 rom_addr;
 
-	if (!res->flags)
+	if (resource_disabled(res))
 		return -1;
 
 	pcibios_resource_to_bus(pdev->bus, &region, res);
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 7b296b0..b44a327 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -248,7 +248,7 @@ static void pdev_check_resources(struct pci_dev *dev,
 		if (r->flags & IORESOURCE_PCI_FIXED)
 			continue;
 
-		if (!(r->flags) || r->parent)
+		if (resource_disabled(r) || r->parent)
 			continue;
 
 		r_align = __pci_resource_alignment(dev, r, realloc_head);
@@ -380,7 +380,7 @@ static void reassign_resources_sorted(struct list_head *realloc_head,
 	list_for_each_entry_safe(add_res, tmp, realloc_head, list) {
 		res = add_res->res;
 		/* skip resource that has been reset */
-		if (!res->flags)
+		if (resource_disabled(res))
 			goto out;
 
 		/* skip this resource if not found in head list */
@@ -2144,7 +2144,7 @@ static void pci_bus_dump_res(struct pci_bus *bus)
 	int i;
 
 	pci_bus_for_each_resource(bus, res, i) {
-		if (!res || !res->end || !res->flags)
+		if (!res || !res->end || resource_disabled(res))
 			continue;
 
 		dev_printk(KERN_DEBUG, &bus->dev, "resource %d %pR\n", i, res);
@@ -2227,7 +2227,7 @@ static int iov_resources_unassigned(struct pci_dev *dev, void *data)
 		struct pci_bus_region region;
 
 		/* Not assigned or rejected by kernel? */
-		if (!r->flags)
+		if (resource_disabled(r))
 			continue;
 
 		pcibios_resource_to_bus(dev->bus, &region, r);
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 26aedde..55caf7a 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -40,7 +40,7 @@ void pci_update_resource(struct pci_dev *dev, int resno)
 	 * Ignore resources for unimplemented BARs and unused resource slots
 	 * for 64 bit BARs.
 	 */
-	if (!res->flags)
+	if (resource_disabled(res))
 		return;
 
 	if (res->flags & IORESOURCE_UNSET)
diff --git a/include/linux/ioport.h b/include/linux/ioport.h
index 27dbb18..fb21179 100644
--- a/include/linux/ioport.h
+++ b/include/linux/ioport.h
@@ -179,6 +179,10 @@ static inline bool resource_contains(struct resource *r1, struct resource *r2)
 	return r1->start <= r2->start && r1->end >= r2->end;
 }
 
+static inline bool resource_disabled(struct resource *r)
+{
+	return !r->flags || (r->flags & IORESOURCE_DISABLED);
+}
 
 /* Convenience shorthand with allocation */
 #define request_region(start,n,name)		__request_region(&ioport_resource, (start), (n), (name), 0)
-- 
1.8.4.5

[PATCH v3 51/51] PCI: Don't set flags to 0 when assign resource fail

[Yinghai Lu]

make flags take IORESOURCE_UNSET | IORESOURCE_DISABLED instead.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/bus.c       |  2 +-
 drivers/pci/setup-bus.c | 45 +++++++++++++++++++++++----------------------
 drivers/pci/setup-res.c |  3 ++-
 3 files changed, 26 insertions(+), 24 deletions(-)

diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index b043bdf..b68f1cd 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -140,7 +140,7 @@ static int pci_bus_alloc_from_region(struct pci_bus *bus, struct resource *res,
 	type_mask |= IORESOURCE_TYPE_BITS;
 
 	pci_bus_for_each_resource(bus, r, i) {
-		if (!r)
+		if (!r || resource_disabled(r))
 			continue;
 
 		/* type_mask must match */
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index b44a327..2966da7 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -289,13 +289,6 @@ static void __dev_check_resources(struct pci_dev *dev,
 	pdev_check_resources(dev, realloc_head, head);
 }
 
-static inline void reset_resource(struct resource *res)
-{
-	res->start = 0;
-	res->end = 0;
-	res->flags = 0;
-}
-
 static void __sort_resources(struct list_head *head)
 {
 	struct pci_dev_resource *res1, *tmp_res, *res2;
@@ -398,7 +391,7 @@ static void reassign_resources_sorted(struct list_head *realloc_head,
 			res->start = align;
 			res->end = res->start + add_size - 1;
 			if (pci_assign_resource(add_res->dev, idx))
-				reset_resource(res);
+				res->flags |= IORESOURCE_DISABLED;
 		} else {
 			/* could just assigned with alt, add difference ? */
 			resource_size_t must_size;
@@ -451,7 +444,7 @@ static void assign_requested_resources_sorted(struct list_head *head,
 		    pci_assign_resource(dev_res->dev, idx)) {
 			if (fail_head)
 				add_to_list(fail_head, dev_res->dev, res);
-			reset_resource(res);
+			res->flags |= IORESOURCE_DISABLED;
 		}
 	}
 }
@@ -737,7 +730,7 @@ static void __assign_resources_alt_sorted(struct list_head *head,
 
 		if (!res_to_dev_res(local_fail_head, res))
 			add_to_list(local_fail_head, fail_res->dev, res);
-		reset_resource(res);
+		res->flags |= IORESOURCE_DISABLED;
 	}
 	free_list(&local_alt_fail_head);
 }
@@ -903,7 +896,7 @@ static void pci_setup_bridge_io(struct pci_dev *bridge)
 	/* Set up the top and bottom of the PCI I/O segment for this bus. */
 	res = &bridge->resource[PCI_BRIDGE_RESOURCES + 0];
 	pcibios_resource_to_bus(bridge->bus, &region, res);
-	if (res->flags & IORESOURCE_IO) {
+	if ((res->flags & IORESOURCE_IO) && !(res->flags & IORESOURCE_UNSET)) {
 		pci_read_config_word(bridge, PCI_IO_BASE, &l);
 		io_base_lo = (region.start >> 8) & io_mask;
 		io_limit_lo = (region.end >> 8) & io_mask;
@@ -933,7 +926,8 @@ static void pci_setup_bridge_mmio(struct pci_dev *bridge)
 	/* Set up the top and bottom of the PCI Memory segment for this bus. */
 	res = &bridge->resource[PCI_BRIDGE_RESOURCES + 1];
 	pcibios_resource_to_bus(bridge->bus, &region, res);
-	if (res->flags & IORESOURCE_MEM) {
+	if ((res->flags & IORESOURCE_MEM) &&
+	    !(res->flags & IORESOURCE_UNSET)) {
 		l = (region.start >> 16) & 0xfff0;
 		l |= region.end & 0xfff00000;
 		dev_info(&bridge->dev, "  bridge window %pR\n", res);
@@ -958,7 +952,8 @@ static void pci_setup_bridge_mmio_pref(struct pci_dev *bridge)
 	bu = lu = 0;
 	res = &bridge->resource[PCI_BRIDGE_RESOURCES + 2];
 	pcibios_resource_to_bus(bridge->bus, &region, res);
-	if (res->flags & IORESOURCE_PREFETCH) {
+	if ((res->flags & IORESOURCE_PREFETCH) &&
+	    !(res->flags & IORESOURCE_UNSET)) {
 		l = (region.start >> 16) & 0xfff0;
 		l |= region.end & 0xfff00000;
 		if (res->flags & IORESOURCE_MEM_64) {
@@ -1077,6 +1072,7 @@ static void pci_bridge_check_ranges(struct pci_bus *bus)
 
 	b_res = &bridge->resource[PCI_BRIDGE_RESOURCES];
 	b_res[1].flags |= IORESOURCE_MEM;
+	b_res[1].flags &= ~IORESOURCE_DISABLED;
 
 	pci_read_config_word(bridge, PCI_IO_BASE, &io);
 	if (!io) {
@@ -1084,8 +1080,10 @@ static void pci_bridge_check_ranges(struct pci_bus *bus)
 		pci_read_config_word(bridge, PCI_IO_BASE, &io);
 		pci_write_config_word(bridge, PCI_IO_BASE, 0x0);
 	}
-	if (io)
+	if (io) {
 		b_res[0].flags |= IORESOURCE_IO;
+		b_res[0].flags &= ~IORESOURCE_DISABLED;
+	}
 
 	/*  DECchip 21050 pass 2 errata: the bridge may miss an address
 	    disconnect boundary by one PCI data phase.
@@ -1102,6 +1100,7 @@ static void pci_bridge_check_ranges(struct pci_bus *bus)
 	}
 	if (pmem) {
 		b_res[2].flags |= IORESOURCE_MEM | IORESOURCE_PREFETCH;
+		b_res[2].flags &= ~IORESOURCE_DISABLED;
 		if ((pmem & PCI_PREF_RANGE_TYPE_MASK) ==
 		    PCI_PREF_RANGE_TYPE_64) {
 			b_res[2].flags |= IORESOURCE_MEM_64;
@@ -1247,8 +1246,10 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 			struct resource *r = &dev->resource[i];
 			unsigned long r_size, r_add_size;
 
-			if (r->parent || !(r->flags & IORESOURCE_IO))
+			if (r->parent || !(r->flags & IORESOURCE_IO) ||
+			    resource_disabled(r))
 				continue;
+
 			r_size = resource_size(r);
 
 			if (r_size < 0x400)
@@ -1289,7 +1290,7 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
 		if (b_res->start || b_res->end)
 			dev_info(&bus->self->dev, "disabling bridge window %pR to %pR (unused)\n",
 				 b_res, &bus->busn_res);
-		b_res->flags = 0;
+		b_res->flags |= IORESOURCE_UNSET | IORESOURCE_DISABLED;
 		return;
 	}
 
@@ -1563,7 +1564,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 
 			if (r->parent || ((flags & mask) != type &&
 					  (flags & mask) != type2 &&
-					  (flags & mask) != type3))
+					  (flags & mask) != type3) ||
+			    resource_disabled(r))
 				continue;
 
 			r_size = resource_size(r);
@@ -1584,7 +1586,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 			if (align > (1ULL<<37)) { /*128 Gb*/
 				dev_warn(&dev->dev, "disabling BAR %d: %pR (bad alignment %#llx)\n",
 					i, r, (unsigned long long) align);
-				r->flags = 0;
+				r->flags |= IORESOURCE_UNSET |
+					    IORESOURCE_DISABLED;
 				continue;
 			}
 
@@ -1671,7 +1674,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
 		if (b_res->start || b_res->end)
 			dev_info(&bus->self->dev, "disabling bridge window %pR to %pR (unused)\n",
 				 b_res, &bus->busn_res);
-		b_res->flags = 0;
+		b_res->flags |= IORESOURCE_UNSET | IORESOURCE_DISABLED;
 		return 0;
 	}
 
@@ -2087,7 +2090,7 @@ static void pci_bridge_release_resources(struct pci_bus *bus,
 		/* keep the old size */
 		r->end = resource_size(r) - 1;
 		r->start = 0;
-		r->flags = 0;
+		r->flags |= IORESOURCE_UNSET | IORESOURCE_DISABLED;
 
 		/* avoiding touch the one without PREF */
 		if (type & IORESOURCE_PREFETCH)
@@ -2356,7 +2359,6 @@ again:
 
 		restore_resource(fail_res, res);
 		if (fail_res->dev->subordinate) {
-			res->flags = 0;
 			/* last or third times and later */
 			if (tried_times + 1 == pci_try_num ||
 			    tried_times + 1 > 2)
@@ -2440,7 +2442,6 @@ again:
 
 		restore_resource(fail_res, res);
 		if (fail_res->dev->subordinate) {
-			res->flags = 0;
 			/* last time */
 			reset_bridge_resource_size(fail_res->dev, res);
 		}
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 55caf7a..d17f23c 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -364,7 +364,8 @@ int pci_enable_resources(struct pci_dev *dev, int mask)
 
 		r = &dev->resource[i];
 
-		if (!(r->flags & (IORESOURCE_IO | IORESOURCE_MEM)))
+		if (!(r->flags & (IORESOURCE_IO | IORESOURCE_MEM)) ||
+		    resource_disabled(r))
 			continue;
 		if ((i == PCI_ROM_RESOURCE) &&
 				(!(r->flags & IORESOURCE_ROM_ENABLE)))
-- 
1.8.4.5

Re: [PATCH v3 00/51] PCI: Resource allocation cleanup for v4.3

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:18PM -0700, Yinghai Lu wrote:
> Hi,
> 
> After 5b28541552ef (PCI: Restrict 64-bit prefetchable bridge windows
> to 64-bit resources), we have several reports on resource allocation
> failure, and we try to fix the problem with resource clip, and get
> more problems.
> 
> One is realloc fail with two graphics cards above 4G.
> One is from sparc that have problem with clip as we don't parse
> mem64 for it.
> 
> Other report is about pci remove/rescan does not work on some setup
> when BIOS tend to allocate small bus size.
> 
> This patchset enhance resource allocation to address those problems.
> 1. optimize bus mmio alignment calculation.
> 2. optimize bus mmio optional alignment calculation.
> 3. add support for alt size to prefer small bus size to small bus alignment.
>    when we have small resource window on parent bridges.
> 4. treat ROM bar as optional resource.
> 5. during allocation, will pick up just fit resource.
> 6. parse MEM64 for sparc and other system with OF.
> 7. treat non-pref mmio64 if parent bridges are all pcie.
> 8. restore old pref allocation logic if hostbridge does not support mmio64 really.
> 9. don't realloc resource if device firmware does not support bar change.
> 10. add pci=assign_pref_bars to clear and assign pref bars.
> 11. don't clear resource when allocation fails.
> 
> I put latest copy at:
>   git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git for-pci-v4.3-next
> 
> That is based on kind of v4.2-rc4.
> 
> Yijing and Wei Yang both test the branch, and the patchset fixs rescan problem,
> and is working well on powerpc setup.

Overall comments:

Thanks for paying attention to your subject lines.  They are
capitalized and punctuated quite consistently.

The one-line subject is a summary.  The body of the changelog is not a
continuation of it.  The body should stand alone even if you don't
read the subject.  Sometimes that means you have to repeat the subject
line in the body.  That's OK.

Please use "()" when referring to functions.  You do it sometimes, but
not always.

I think we should talk about "required/optional" resources instead of
"must/optional" ones.  Is "addon" a third, separate, thing?

Some typo fixes below:

> Yinghai Lu (51):
>   PCI: Cleanup res_to_dev_res() printout for addon resources
>   PCI: Reuse res_to_dev_res in reassign_resources_sorted

s/reassign_resources_sorted/reassign_resources_sorted()/

>   PCI: Use correct align for optional only resources during sorting
>   PCI: Optimize bus align/size calculation during sizing
>   PCI: Optimize bus align/size calculation for optional during sizing
>   PCI: Don't add too much optional size for hotplug bridge mmio

s/mmio/MMIO/

>   PCI: Reorder resources list for must/optional resources
>   PCI: Remove duplicated code for resource sorting
>   PCI: Rename pdev_sort_resources to pdev_check_resources

s/pdev_sort_resources/pdev_sort_resources()/
s/pdev_check_resources/pdev_check_resources()/

>   PCI: Treat ROM resource as optional during realloc
>   PCI: Add debug printout during releasing partial assigned resources
>   PCI: Simplify res reference using in __assign_resourcs_sorted

s/__assign_resourcs_sorted/__assign_resources_sorted()/ (two fixes)

>   PCI: Separate realloc list checking after allocation
>   PCI: Add __add_to_list()
>   PCI: Cache window alignment value
>   PCI: Check if resource is allocated before pci_assign

s/pci_assign/pci_assign()/

>   PCI: Separate out save_resources/restore_resource

s/save_resources/save_resources()/
s/restore_resource/restore_resource()/

>   PCI: Move comment to pci_need_to_release()
>   PCI: Separate must+optional assigning to another function
>   PCI: Skip must+optional if there is no optional addon
>   PCI: Move saved required resource list out of must+optional assigning
>   PCI: Add alt_size allocation support
>   PCI: Add support for more than two alt_size under same bridge
>   PCI: Better support for two alt_size
>   PCI: Fix size calculation with old_size on rescan path
>   PCI: Don't add too much optional size for hotplug bridge io
>   PCI: Move ISA ioport align out of calculate_iosize

s/calculate_iosize/calculate_iosize()/

>   PCI: Unifiy calculate_size for io port and mmio

s/Unifiy/Unify/
s/calculate_size/calculate_size()/

>   PCI: Allow optional only io resource must size to be 0
>   PCI: Unify skip_ioresource_align()
>   PCI: Kill macro checking for bus io port sizing
>   resources: Split out __allocate_resource()
>   resources: Make allocate_resource return just fit resource

I think "just fit" is commonly referred to as "best fit," as in
http://courses.cs.vt.edu/~csonline/OS/Lessons/MemoryAllocation/index.html.

>   PCI: Check pref compatible bit for mem64 resource of pcie device

s/pcie/PCIe/

>   PCI: Only treat non-pef mmio64 as pref if all bridges has MEM_64

s/non-pef/non-pref/
s/bridges has/bridges have/

>   PCI: Add has_mem64 for host_bridge

s/host_bridge/struct pci_host_bridge/

>   PCI: Only treat non-pef mmio64 as pref if host-bridge has_mem64

s/non-pef/non-pref/
s/host-bridge/host bridge/

>   PCI: Restore pref mmio allocation logic for hostbridge without mmio64

s/hostbridge/host bridge/

>   sparc/PCI: Add mem64 resource parsing for root bus
>   sparc/PCI: Add IORESOURCE_MEM_64 for 64-bit resource in of parsing

s/of/OF/

>   powerpc/PCI: Add IORESOURCE_MEM_64 for 64-bit resource in of parsing

s/of/OF/

>   of/PCI: Add IORESOURCE_MEM_64 for 64-bit resource
>   PCI: Treat optional as must in first try for bridge rescan
>   PCI: Get new realloc size for bridge for last try
>   PCI: Don't release sibiling bridge resources during hotplug

s/sibiling/sibling/

>   PCI: Don't release fixed resource for realloc
>   PCI: Claim fixed resource during remove/rescan path
>   PCI: Set resource to FIXED for lsi devices

s/lsi/LSI/

>   PCI, x86: Add pci=assign_pref_bars to re-allocate pref bars

s/re-allocate/reallocate/
s/bars/BARs/

>   PCI: Introduce resource_disabled()
>   PCI: Don't set flags to 0 when assign resource fail
> 
>  arch/alpha/kernel/pci.c                   |    2 +-
>  arch/ia64/pci/pci.c                       |    4 +-
>  arch/microblaze/pci/pci-common.c          |   23 +-
>  arch/mn10300/unit-asb2305/pci-asb2305.c   |    4 +-
>  arch/mn10300/unit-asb2305/pci.c           |    4 +-
>  arch/powerpc/kernel/pci-common.c          |   27 +-
>  arch/powerpc/kernel/pci_of_scan.c         |    4 +-
>  arch/powerpc/platforms/powernv/pci-ioda.c |   12 +-
>  arch/s390/pci/pci.c                       |    2 +-
>  arch/sparc/kernel/of_device_32.c          |    5 +-
>  arch/sparc/kernel/of_device_64.c          |    5 +-
>  arch/sparc/kernel/pci.c                   |    9 +-
>  arch/sparc/kernel/pci_common.c            |   15 +-
>  arch/sparc/kernel/pci_impl.h              |    1 +
>  arch/x86/include/asm/pci_x86.h            |    2 +-
>  arch/x86/pci/common.c                     |    7 +-
>  arch/x86/pci/i386.c                       |   72 +-
>  arch/xtensa/kernel/pci.c                  |    4 +-
>  drivers/iommu/intel-iommu.c               |    3 +-
>  drivers/of/address.c                      |    4 +-
>  drivers/pci/bus.c                         |    6 +-
>  drivers/pci/host/pcie-rcar.c              |    2 +-
>  drivers/pci/hotplug/acpiphp_glue.c        |    1 +
>  drivers/pci/iov.c                         |    2 +-
>  drivers/pci/pci.c                         |    3 +-
>  drivers/pci/pci.h                         |    4 +
>  drivers/pci/probe.c                       |   44 +-
>  drivers/pci/quirks.c                      |   37 +
>  drivers/pci/rom.c                         |    2 +-
>  drivers/pci/setup-bus.c                   | 1321 +++++++++++++++++++++--------
>  drivers/pci/setup-res.c                   |   18 +-
>  include/asm-generic/pci-bridge.h          |    2 +
>  include/linux/ioport.h                    |    6 +-
>  include/linux/pci.h                       |    2 +
>  kernel/resource.c                         |  175 +++-
>  35 files changed, 1361 insertions(+), 473 deletions(-)
> 
> -- 
> 1.8.4.5
> 

Re: [PATCH v3 01/51] PCI: Cleanup res_to_dev_res() printout for addon resources

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:19PM -0700, Yinghai Lu wrote:
> Now get_res_add_size and get_res_add_align all have same printout
> from res_to_dev_res(), and it is confusing.
> 
> Move out debug messages printout from res_to_dev_res(),
> and later we will reuse res_to_dev_res() in other functions.
> 
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> ---
>  drivers/pci/setup-bus.c | 34 ++++++++++++++++++++--------------
>  1 file changed, 20 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 508cc56..f0fa705 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -104,19 +104,9 @@ static struct pci_dev_resource *res_to_dev_res(struct list_head *head,
>  {
>  	struct pci_dev_resource *dev_res;
>  
> -	list_for_each_entry(dev_res, head, list) {
> -		if (dev_res->res == res) {
> -			int idx = res - &dev_res->dev->resource[0];
> -
> -			dev_printk(KERN_DEBUG, &dev_res->dev->dev,
> -				 "res[%d]=%pR res_to_dev_res add_size %llx min_align %llx\n",
> -				 idx, dev_res->res,
> -				 (unsigned long long)dev_res->add_size,
> -				 (unsigned long long)dev_res->min_align);
> -
> +	list_for_each_entry(dev_res, head, list)
> +		if (dev_res->res == res)
>  			return dev_res;
> -		}
> -	}
>  
>  	return NULL;
>  }
> @@ -127,7 +117,15 @@ static resource_size_t get_res_add_size(struct list_head *head,
>  	struct pci_dev_resource *dev_res;
>  
>  	dev_res = res_to_dev_res(head, res);
> -	return dev_res ? dev_res->add_size : 0;
> +	if (!dev_res || !dev_res->add_size)
> +		return 0;
> +
> +	dev_printk(KERN_DEBUG, &dev_res->dev->dev,
> +		   "BAR %d: %pR get_res_add_size add_size   %llx\n",
> +		   (int)(res - &dev_res->dev->resource[0]),
> +		   res, (unsigned long long)dev_res->add_size);
> +
> +	return dev_res->add_size;
>  }
>  
>  static resource_size_t get_res_add_align(struct list_head *head,
> @@ -136,7 +134,15 @@ static resource_size_t get_res_add_align(struct list_head *head,
>  	struct pci_dev_resource *dev_res;
>  
>  	dev_res = res_to_dev_res(head, res);
> -	return dev_res ? dev_res->min_align : 0;
> +	if (!dev_res || !dev_res->min_align)
> +		return 0;
> +
> +	dev_printk(KERN_DEBUG, &dev_res->dev->dev,
> +		   "BAR %d: %pR get_res_add_align min_align %llx\n",
> +		   (int)(res - &dev_res->dev->resource[0]),
> +		   res, (unsigned long long)dev_res->min_align);

Please use %#llx so we have a "0x" prefix like other hex numbers.

> +
> +	return dev_res->min_align;
>  }
>  
>  
> -- 
> 1.8.4.5
> 

Re: [PATCH v3 03/51] PCI: Use correct align for optional only resources during sorting

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:21PM -0700, Yinghai Lu wrote:
> During sorting before assign, we only put resource with non-zero align
> in the sorted list, so for optional resources that must size is 0 and only
> have addon parts, we need to have correct align.
> 
> While treating SRIOV as optional resources, we always read alignment for
> SRIOV bars, so they are ok.
> Hotplug bridge resources are using STARTALIGN so it is ok when size is 0
> if we have correct start for them.
> 
> Later we want to treat the ROM BAR as optional resource, and it has
> have SIZEALIGN, we need to find a way to get align for them.

Are you saying we have a ROM resource where the ROM BAR is of non-zero
size, but the resource structure says it's zero size?

> We can use addon resource align instead in that case, and it will
> be ok for SRIOV path and hotplug bridge resource path.
> 
> Sorted list will contain must resource align/size to 0/0 to hold spot for
> optional resources.
> 
> We need to pass realloc_head from sizing stage to sorting stage, and
> get entry from realloc list and calculate align from the entry.
> 
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431

Apparently this fixes a problem?  What is the problem?  What is the
symptom of the problem?

> Reported-by: TJ <linux@iam.tj>
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> ---
>  drivers/pci/setup-bus.c | 50 ++++++++++++++++++++++++++++++++++++++++++-------
>  1 file changed, 43 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 247d8fe..27cb0f0 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -145,9 +145,43 @@ static resource_size_t get_res_add_align(struct list_head *head,
>  	return dev_res->min_align;
>  }
>  
> +static resource_size_t __pci_resource_alignment(

I don't like creating new function names by adding "__" to the
beginning of an existing function name.  It doesn't give enough of a
clue about what how the functions are different.  I don't really
understand what's going on here, but this might be a clue that this
could be refactored differently.

> +				struct pci_dev *dev,
> +				struct resource *r,
> +				struct list_head *realloc_head)
> +{
> +	resource_size_t r_align = pci_resource_alignment(dev, r);
> +	resource_size_t orig_start, orig_end;
> +	struct pci_dev_resource *dev_res;
> +
> +	if (r_align || !realloc_head)
> +		return r_align;
> +
> +	dev_res = res_to_dev_res(realloc_head, r);
> +	if (!dev_res || !dev_res->add_size)
> +		return r_align;
> +
> +	orig_start = r->start;
> +	orig_end = r->end;
> +	r->end += dev_res->add_size;
> +	if ((r->flags & IORESOURCE_STARTALIGN)) {
> +		resource_size_t r_size = resource_size(r);
> +		resource_size_t add_align = dev_res->min_align;
> +
> +		r->start = add_align;
> +		r->end = add_align + r_size - 1;

I think this would be slightly shorter and more obvious as:

  resource_size_t r_size = resource_size(r);

  r->start = dev_res->min_align;
  r->end = r->start + r_size - 1;

> +	}
> +	r_align = pci_resource_alignment(dev, r);
> +	r->start = orig_start;
> +	r->end = orig_end;
> +
> +	return r_align;
> +}
>  
>  /* Sort resources by alignment */
> -static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head)
> +static void pdev_sort_resources(struct pci_dev *dev,
> +				 struct list_head *realloc_head,
> +				 struct list_head *head)
>  {
>  	int i;
>  
> @@ -165,7 +199,7 @@ static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head)
>  		if (!(r->flags) || r->parent)
>  			continue;
>  
> -		r_align = pci_resource_alignment(dev, r);
> +		r_align = __pci_resource_alignment(dev, r, realloc_head);
>  		if (!r_align) {
>  			dev_warn(&dev->dev, "BAR %d: %pR has bogus alignment\n",
>  				 i, r);
> @@ -183,8 +217,9 @@ static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head)
>  		list_for_each_entry(dev_res, head, list) {
>  			resource_size_t align;
>  
> -			align = pci_resource_alignment(dev_res->dev,
> -							 dev_res->res);
> +			align = __pci_resource_alignment(dev_res->dev,
> +							 dev_res->res,
> +							 realloc_head);
>  
>  			if (r_align > align) {
>  				n = &dev_res->list;
> @@ -197,6 +232,7 @@ static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head)
>  }
>  
>  static void __dev_sort_resources(struct pci_dev *dev,
> +				 struct list_head *realloc_head,
>  				 struct list_head *head)
>  {
>  	u16 class = dev->class >> 8;
> @@ -213,7 +249,7 @@ static void __dev_sort_resources(struct pci_dev *dev,
>  			return;
>  	}
>  
> -	pdev_sort_resources(dev, head);
> +	pdev_sort_resources(dev, realloc_head, head);
>  }
>  
>  static inline void reset_resource(struct resource *res)
> @@ -501,7 +537,7 @@ static void pdev_assign_resources_sorted(struct pci_dev *dev,
>  {
>  	LIST_HEAD(head);
>  
> -	__dev_sort_resources(dev, &head);
> +	__dev_sort_resources(dev, add_head, &head);
>  	__assign_resources_sorted(&head, add_head, fail_head);
>  
>  }
> @@ -514,7 +550,7 @@ static void pbus_assign_resources_sorted(const struct pci_bus *bus,
>  	LIST_HEAD(head);
>  
>  	list_for_each_entry(dev, &bus->devices, bus_list)
> -		__dev_sort_resources(dev, &head);
> +		__dev_sort_resources(dev, realloc_head, &head);
>  
>  	__assign_resources_sorted(&head, realloc_head, fail_head);
>  }
> -- 
> 1.8.4.5
> 

Re: [PATCH v3 04/51] PCI: Optimize bus align/size calculation during sizing

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:22PM -0700, Yinghai Lu wrote:
> Current code try to get align as small as possible and use that to
> align final size. But it does not handle resource that size is bigger
> than align in optimal way, kernel only use max align for them.
> 
> For example:
>  when we have resources with align/size: 1M/2M, 512M/512M,
>    bus resource min_align/size0 will be 512M/1024M,
>    but optimal value should be 256M/768M.

I want to understand what makes 256M/768M "optimal."

This is basically a bin packing problem, and I'd like to have an
explicit statement of the constraints and the goal.  Right now the
goal is fuzzy and the code seems ad hoc.

Here are the possibilities I can see.  The placement notes are the
offsets into the allocated space:

  align  size    512M placement   2M placement    unused
  1024M  514M    [0-512]          [512-514]       [514-1024]
   512M  514M    [0-512]          [512-514]       none
   256M  768M    [256-768]        [254-256]       [0-254]
   128M  896M    [384-896]        [382-384]       [0-382]

Obviously, we would never choose 1024M/514M or any larger alignment:
it requires no more space than 512M/514M, but it fails in some cases
where 512M/514M would succeed.

Also obviously, we would never choose 128M/896M or a smaller
alignment: if we could get that, we could also get 256M/768M and it
would consume less space.

But it's not quite so obvious how to choose between 512M/514M and
256M/768M.  The former (if we can get it) consumes much less space.
The latter requires less alignment but wastes a lot of space.

> For following cases that we have resource size that is bigger
> than resource alignment:
> 1. SRIOV bar.
> 2. PCI bridges with several bridges or devices as children.

This doesn't have anything to do with how many children a bridge has,
does it?  As long as there are two or more BARs (1MB or larger) below
a bridge, we'll have the situation where the bridge window has to
contain both BARs but only needs to be aligned for (at most) the
larger one.

> We can keep on trying to allocate children devices resources under range
> [half_align, half_align + aligned_size).
> If sucesses, we can use that half_align as new min_align.
> 
> After this patch, we get:
>  align/size: 1M/2M, 2M/4M, 4M/8M, 8M/16M
>  new min_align/min_size: 4M/32M, and old is 8M/32M
> 
>  align/size: 1M/2M, 2M/4M, 4M/8M
>  new min_align/min_size: 2M/14M, and old is 4M/16M
> 
>  align/size: 1M/2M, 512M/512M
>  new min_align/min_size: 256M/768M, and old is 512M/1024M
> 
> The real result from one system with one pcie card that has
> four functions that support sriov:
>  align/size:
>    00800000/00800000
>    00800000/00800000
>    00800000/00800000
>    00800000/00800000
>    00010000/00200000
>    00010000/00200000
>    00010000/00200000
>    00010000/00200000
>    00008000/00008000
>    00008000/00008000
>    00008000/00008000
>    00008000/00008000
>    00004000/00080000
>    00004000/00080000
>    00004000/00080000
>    00004000/00080000
>  old min_align/min_size: 00400000/02c00000
>      min_align/min_size: 00100000/02b00000

I don't know if this example is essential, but if it is, it needs a
little more interpretation.  I assume the BARs above where
"align < size" are for SR-IOV, where size include multiple VFs.
In any case, please connect the dots from the raw data to the
conclusion.

> So align will be 1M instead of 4M.
> 
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
> Reported-by: TJ <linux@iam.tj>
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> ---
>  drivers/pci/setup-bus.c | 195 ++++++++++++++++++++++++++++++++++++++----------
>  1 file changed, 157 insertions(+), 38 deletions(-)
> 
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 27cb0f0..ecdf011 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -30,6 +30,34 @@
>  
>  unsigned int pci_flags;
>  
> +static inline bool is_before(resource_size_t align1, resource_size_t size1,
> +			     resource_size_t align2, resource_size_t size2)

Can this take two struct resource pointers instead of four numbers?  I
haven't looked at all the callers, so maybe there's a caller that
doesn't have a struct resource.

> +{
> +	resource_size_t size1_left, size2_left;
> +
> +	/* big align is before small align */
> +	if (align1 > align2)
> +		return true;
> +
> +	/*
> +	 * for same align:
> +	 *   aligned is before not aligned
> +	 *   for not aligned, big remainder is before small remainder
> +	 */
> +	if (align1 == align2) {
> +		size1_left = size1 & (align1 - 1);
> +		if (!size1_left)
> +			size1_left = align1;
> +		size2_left = size2 & (align2 - 1);
> +		if (!size2_left)
> +			size2_left = align2;
> +		if (size1_left > size2_left)
> +			return true;
> +	}
> +
> +	return false;
> +}

Re: [PATCH v3 07/51] PCI: Reorder resources list for must/optional resources

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:25PM -0700, Yinghai Lu wrote:
> After we update size and alignment for must+optional resource, we
> reorder them with new alignment, but this is only for STARTALIGN.
> 
> For SIZEALIGN type resource, after add back add_size, the alignment
> get changed, so need to do the sorting like STARTALIGN type resources.
> 
> Also we need to reorder the sorting back after we restore
> resource to must only when must+optional fail to allocate for all.
> 
> So move out the reordering code from the loop to separated function,
> and call it two times accordingly.
> 
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> ---
>  drivers/pci/setup-bus.c | 62 +++++++++++++++++++++++++++++--------------------
>  1 file changed, 37 insertions(+), 25 deletions(-)
> 
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 7346bbf..6f2d508 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -287,6 +287,31 @@ static inline void reset_resource(struct resource *res)
>  	res->flags = 0;
>  }
>  
> +static void __sort_resources(struct list_head *head)

Why did you add "__" before the function name?

> +{
> +	struct pci_dev_resource *res1, *tmp_res, *res2;
> +
> +	list_for_each_entry_safe(res1, tmp_res, head, list) {
> +		resource_size_t align1, size1, align2, size2;
> +
> +		align1 = pci_resource_alignment(res1->dev, res1->res);
> +		size1 = resource_size(res1->res);
> +
> +		/* reorder it */
> +		list_for_each_entry(res2, head, list) {
> +			if (res2 == res1)
> +				break;
> +
> +			align2 = pci_resource_alignment(res2->dev, res2->res);
> +			size2 = resource_size(res2->res);
> +			if (is_before(align1, size1, align2, size2)) {
> +				list_move_tail(&res1->list, &res2->list);
> +				break;
> +			}
> +		}
> +	}
> +}

Re: [PATCH v3 09/51] PCI: Rename pdev_sort_resources to pdev_check_resources

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:27PM -0700, Yinghai Lu wrote:
> We don't do sorting in those functions anymore, so change "sort" to "check"
> instead.

What *do* we do in those functions?  "Check" doesn't tell me anything.

> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> ---
>  drivers/pci/setup-bus.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 6642a60..292f2a5 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -206,8 +206,8 @@ static resource_size_t __pci_resource_alignment(
>  	return r_align;
>  }
>  
> -/* Sort resources by alignment */
> -static void pdev_sort_resources(struct pci_dev *dev,
> +/* check resources and save to the list */
> +static void pdev_check_resources(struct pci_dev *dev,
>  				 struct list_head *realloc_head,
>  				 struct list_head *head)
>  {
> @@ -243,7 +243,7 @@ static void pdev_sort_resources(struct pci_dev *dev,
>  	}
>  }
>  
> -static void __dev_sort_resources(struct pci_dev *dev,
> +static void __dev_check_resources(struct pci_dev *dev,
>  				 struct list_head *realloc_head,
>  				 struct list_head *head)
>  {
> @@ -261,7 +261,7 @@ static void __dev_sort_resources(struct pci_dev *dev,
>  			return;
>  	}
>  
> -	pdev_sort_resources(dev, realloc_head, head);
> +	pdev_check_resources(dev, realloc_head, head);
>  }
>  
>  static inline void reset_resource(struct resource *res)
> @@ -561,7 +561,7 @@ static void pdev_assign_resources_sorted(struct pci_dev *dev,
>  {
>  	LIST_HEAD(head);
>  
> -	__dev_sort_resources(dev, add_head, &head);
> +	__dev_check_resources(dev, add_head, &head);
>  	__assign_resources_sorted(&head, add_head, fail_head);
>  
>  }
> @@ -574,7 +574,7 @@ static void pbus_assign_resources_sorted(const struct pci_bus *bus,
>  	LIST_HEAD(head);
>  
>  	list_for_each_entry(dev, &bus->devices, bus_list)
> -		__dev_sort_resources(dev, realloc_head, &head);
> +		__dev_check_resources(dev, realloc_head, &head);
>  
>  	__assign_resources_sorted(&head, realloc_head, fail_head);
>  }
> -- 
> 1.8.4.5
> 

Re: [PATCH v3 13/51] PCI: Separate realloc list checking after allocation

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:31PM -0700, Yinghai Lu wrote:
> We check the realloc list, as list must be empty after allocation.
> 
> Separate the realloc list checking to another function.
> 
> Add checking that is missed in acpiphp driver.
> 
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
> Cc: Len Brown <lenb@kernel.org>
> Cc: linux-acpi@vger.kernel.org
> ---
>  drivers/pci/hotplug/acpiphp_glue.c |  1 +
>  drivers/pci/pci.h                  |  1 +
>  drivers/pci/setup-bus.c            | 11 ++++++++---
>  3 files changed, 10 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/pci/hotplug/acpiphp_glue.c b/drivers/pci/hotplug/acpiphp_glue.c
> index ff53856..134caee 100644
> --- a/drivers/pci/hotplug/acpiphp_glue.c
> +++ b/drivers/pci/hotplug/acpiphp_glue.c
> @@ -507,6 +507,7 @@ static void enable_slot(struct acpiphp_slot *slot)
>  		}
>  	}
>  	__pci_bus_assign_resources(bus, &add_list, NULL);
> +	__pci_bus_check_realloc(&add_list);
>  
>  	acpiphp_sanitize_bus(bus);
>  	pcie_bus_configure_settings(bus);
> diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
> index 4ff0ff1..2b83977 100644
> --- a/drivers/pci/pci.h
> +++ b/drivers/pci/pci.h
> @@ -235,6 +235,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus,
>  void __pci_bus_assign_resources(const struct pci_bus *bus,
>  				struct list_head *realloc_head,
>  				struct list_head *fail_head);
> +void __pci_bus_check_realloc(struct list_head *realloc_head);
>  bool pci_bus_clip_resource(struct pci_dev *dev, int idx);
>  
>  void pci_reassigndev_resource_alignment(struct pci_dev *dev);
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 06664db..f30225c 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -350,6 +350,11 @@ out:
>  	}
>  }
>  
> +void __pci_bus_check_realloc(struct list_head *realloc_head)
> +{
> +	BUG_ON(!list_empty(realloc_head));

I don't think this really improves readability.  At least before, we could
see what we were looking for and we knew it might BUG().

> +}
> +
>  /**
>   * assign_requested_resources_sorted() - satisfy resource requests
>   *
> @@ -1861,7 +1866,7 @@ again:
>  	/* Depth last, allocate resources and update the hardware. */
>  	__pci_bus_assign_resources(bus, add_list, &fail_head);
>  	if (add_list)
> -		BUG_ON(!list_empty(add_list));
> +		__pci_bus_check_realloc(add_list);
>  	tried_times++;
>  
>  	/* any device complain? */
> @@ -1936,7 +1941,7 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
>  again:
>  	__pci_bus_size_bridges(parent, &add_list);
>  	__pci_bridge_assign_resources(bridge, &add_list, &fail_head);
> -	BUG_ON(!list_empty(&add_list));
> +	__pci_bus_check_realloc(&add_list);
>  	tried_times++;
>  
>  	if (list_empty(&fail_head))
> @@ -1995,6 +2000,6 @@ void pci_assign_unassigned_bus_resources(struct pci_bus *bus)
>  							 &add_list);
>  	up_read(&pci_bus_sem);
>  	__pci_bus_assign_resources(bus, &add_list, NULL);
> -	BUG_ON(!list_empty(&add_list));
> +	__pci_bus_check_realloc(&add_list);
>  }
>  EXPORT_SYMBOL_GPL(pci_assign_unassigned_bus_resources);
> -- 
> 1.8.4.5
> 

Re: [PATCH v3 20/51] PCI: Skip must+optional if there is no optional addon

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:38PM -0700, Yinghai Lu wrote:
> If the bridge does not support hotplug or no child with sriov support
> we could get out early and don't try must+optional allocation.

You mention non-hotplug bridges and non-SR-IOV devices, but I can't figure
out how the patch itself relates to those.  How can I connect them?

> Also in the loop that update res with optional add info, skip resource
> that add_size is 0.
> 
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> ---
>  drivers/pci/setup-bus.c | 32 +++++++++++++++++++++++++++++---
>  1 file changed, 29 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index d1f9e19..64ef516 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -450,6 +450,24 @@ static bool pci_need_to_release(unsigned long mask, struct resource *res)
>  	return false;	/* should not get here */
>  }
>  
> +static bool __has_addon(struct list_head *head,
> +			struct list_head *realloc_head)

Why does this have a "__" prefix?

> +{
> +	int add_count = 0;
> +	struct pci_dev_resource *dev_res, *tmp_res;
> +
> +	/* check if we have add really */
> +	list_for_each_entry(dev_res, head, list) {
> +		tmp_res = res_to_dev_res(realloc_head, dev_res->res);
> +		if (!tmp_res || !tmp_res->add_size)
> +			continue;
> +
> +		add_count++;
> +	}
> +
> +	return add_count != 0;
> +}
> +
>  static bool save_resources(struct list_head *head,
>  			   struct list_head *save_head)
>  {
> @@ -481,16 +499,24 @@ static bool __assign_resources_must_add_sorted(struct list_head *head,
>  	struct pci_dev_resource *save_res;
>  	struct pci_dev_resource *dev_res, *tmp_res;
>  	unsigned long fail_type;
> -	resource_size_t add_align;
> +	resource_size_t add_align, add_size;
>  	struct resource *res;
>  
> +	if (!__has_addon(head, realloc_head))
> +		return false;
> +
>  	if (!save_resources(head, &save_head))
>  		return false;
>  
>  	/* Update res in head list with add_size in realloc_head list */
>  	list_for_each_entry(dev_res, head, list) {
>  		res = dev_res->res;
> -		res->end += get_res_add_size(realloc_head, res);
> +		add_size = get_res_add_size(realloc_head, res);
> +
> +		if (!add_size)
> +			continue;
> +
> +		res->end += add_size;
>  
>  		/*
>  		 * There are two kinds of additional resources in the list:
> @@ -578,7 +604,7 @@ static void __assign_resources_sorted(struct list_head *head,
>  	 */
>  
>  	/* Check must+optional add */
> -	if (realloc_head && !list_empty(realloc_head) &&
> +	if (realloc_head &&

Why are you dropping the !list_empty() test?

>  	    __assign_resources_must_add_sorted(head, realloc_head))
>  		return;
>  
> -- 
> 1.8.4.5
> 

Re: [PATCH v3 22/51] PCI: Add alt_size allocation support

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:40PM -0700, Yinghai Lu wrote:
> On system with several pcie switches, BIOS allocate very tight resources
> to the bridge bar, and it is not aligned to min_align as kernel allocation
> code.

I can't parse this.

> For example:
> 	02:03.0---0c:00.0---0d:04.0---18:00.0
> 18:00.0 need 0x10000000, and 0x00010000.
> BIOS only allocate 0x10100000 to 0d:04.0 and above bridges.

Do you mean the BIOS only allocated 0x10010000?

> Later after using /sys/bus/pci/devices/0000:0c:00.0/remove to remove 0c:00.0,
> rescan with /sys/bus/pci/rescan can not allocate 0x18000000 to 0c:00.0.
> 
> another example:
> 00:1c.0-[02-21]----00.0-[03-21]--+-01.0-[04-12]----00.0-[05-12]----19.0-[06-12]----00.0
>                                  +-05.0-[13]--
>                                  +-07.0-[14-20]----00.0-[15-20]--+-08.0-[16]--+-00.0
>                                  |                               |            \-00.1
>                                  |                               +-14.0-[17]----00.0
>                                  |                               \-19.0-[18-20]----00.0
>                                  \-09.0-[21]--
> 06:00.0 need 0x4000000 and 0x800000.
> BIOS only allocate 0x4800000 to 05:19.0 and 04:00.0.
> when 05:19.0 get removed via /sys/bus/pci/devices/0000:05:19.0/remove,
> rescan with /sys/bus/pci/rescan will fail.
>  pci 0000:05:19.0: BAR 14: no space for [mem size 0x06000000]
>  pci 0000:05:19.0: BAR 14: failed to assign [mem size 0x06000000]
>  pci 0000:06:00.0: BAR 2: no space for [mem size 0x04000000 64bit]
>  pci 0000:06:00.0: BAR 2: failed to assign [mem size 0x04000000 64bit]
>  pci 0000:06:00.0: BAR 0: no space for [mem size 0x00800000]
>  pci 0000:06:00.0: BAR 0: failed to assign [mem size 0x00800000]
> current code try to use align 0x2000000 and size 0x6000000, but parent
> bridge only have 0x4800000.

I *think* you're saying:
  - BIOS assigned space for device X
  - We remove X via sysfs
  - We rescan via sysfs and discover X
  - We try to assign space for X
  - We fail because we don't use the same algorithm as BIOS did

If there is an optimal way to assign space for an arbitrary number of
BARs, we could just adopt it.  I don't know what that is, and I don't
know whether an optimal algorithm exists even in principle.

If there is no single optimal algorithm, there will always be cases where
we fail because we use a different algorithm than the firmware did.

> Introduce alt_align/alt_size and store them in realloc list in addition
> to addon info, and will try it after min_align/min_size allocation fails.

What does "alt" mean?

> The alt_align is max_align, and alt_size is aligned size with bridge
> minimum window alignment.

Re: [PATCH v3 25/51] PCI: Fix size calculation with old_size on rescan path

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:43PM -0700, Yinghai Lu wrote:
> On booting path, we don't pass realloc at first, and treat all optional just
> as must, in some case we can smaller size/align with optional than must only.
> 
> 04:00.0 has four child bridges: 05:01.0, 05:02.0, 05:03.0, 05:04.0
> pref layout after booting path like followings:
> 
> pci 0000:04:00.0: BAR 9: assigned [mem 0x84000000-0x9fffffff 64bit pref]
> pci 0000:05:04.0: BAR 9: assigned [mem 0x88000000-0x9fffffff 64bit pref]
> pci 0000:05:03.0: BAR 9: assigned [mem 0x84000000-0x841fffff 64bit pref]
> pci 0000:07:00.0: BAR 0: assigned [mem 0x81200000-0x8123ffff 64bit]
> pci 0000:05:03.0: PCI bridge to [bus 08-0f]
> pci 0000:05:03.0:   bridge window [mem 0x84000000-0x841fffff 64bit pref]
> pci 0000:10:00.0: BAR 0: assigned [mem 0x90000000-0x9fffffff 64bit pref]
> pci 0000:10:00.0: BAR 2: assigned [mem 0x88000000-0x8800ffff 64bit pref]
> pci 0000:05:04.0: PCI bridge to [bus 10]
> pci 0000:05:04.0:   bridge window [mem 0x88000000-0x9fffffff 64bit pref]
> pci 0000:04:00.0: PCI bridge to [bus 05-10]
> pci 0000:04:00.0:   bridge window [mem 0x84000000-0x9fffffff 64bit pref]

Unless 07:00.0 and 10:00.0 are relevant, please remove them.

> so the old size in rescan for 04:00.0 would be 0x1c000000, and align is 0x4000000
> 
> during remove and rescan:
> 
> pci 0000:05:03.0: bridge window [mem 0x00000000-0xffffffffffffffff 64bit pref] to [bus 08-0f] add_size 200000 add_align 100000 alt_size 0 alt_align 0 must_size 0 must_align 0

What is "[mem 0x00000000-0xffffffffffffffff 64bit pref]" telling us?
Surely that's not an actual aperture, is it?

Re: [PATCH v3 27/51] PCI: Move ISA ioport align out of calculate_iosize

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:45PM -0700, Yinghai Lu wrote:
> So we could unify calculate_iosize and calculate_memsize later.

Please make this a complete sentence, not just a continuation of the
summary line.

> when one bridge have several children devices, and every devices
> have several io port resources and resource size < 0x400.

This isn't a sentence either, so I'm not sure what this is supposed to
mean.

> We need to check size, and add extra size to make sure bit8/9
> to be zero.

Re: [PATCH v3 28/51] PCI: Unifiy calculate_size for io port and mmio

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:46PM -0700, Yinghai Lu wrote:
> We should check size+size1 with min_size for io port.
> For example, when hotplug bridge has two children bridges,
> every child bridge will need 0x1000, so size1 will be 0x2000
> and size is 0. The min_size for the hotplug bridge is 0x100.

A min_size of 0x100?  Is that a typo?

> with old version calculate_iosize, we get 0x3000 for final
> size because we are using size to compare with min_size. That is
> not right, we should use 0x2000 instead.

If this fixes a bug, please make one patch that *only* fixes the bug,
and a separate one that unifies but doesn't change the behavior.

> After this change, calculate_memsize and calculate_iosize
> is the same.
> 
> Change them to calculate_size.
> 
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> ---
>  drivers/pci/setup-bus.c | 27 ++++++---------------------
>  1 file changed, 6 insertions(+), 21 deletions(-)
> 
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index 3db2838..aeed716 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -1116,23 +1116,7 @@ static struct resource *find_free_bus_resource(struct pci_bus *bus,
>  	return NULL;
>  }
>  
> -static resource_size_t calculate_iosize(resource_size_t size,
> -		resource_size_t min_size,
> -		resource_size_t size1,
> -		resource_size_t old_size,
> -		resource_size_t align)
> -{
> -	if (size < min_size)
> -		size = min_size;
> -	if (old_size == 1)
> -		old_size = 0;
> -	size = ALIGN(size + size1, align);
> -	if (size < old_size)
> -		size = old_size;
> -	return size;
> -}
> -
> -static resource_size_t calculate_memsize(resource_size_t size,
> +static resource_size_t calculate_size(resource_size_t size,
>  		resource_size_t min_size,
>  		resource_size_t old_size,
>  		resource_size_t align)
> @@ -1257,14 +1241,15 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
>  	}
>  
>  	size = size_aligned_for_isa(size);
> -	size0 = calculate_iosize(size, min_size, size1,
> +	size += size1;
> +	size0 = calculate_size(size, min_size,
>  			resource_size(b_res), min_align);
>  	sum_add_size = size_aligned_for_isa(sum_add_size);
>  	sum_add_size += sum_add_size1;
>  	if (sum_add_size < min_sum_size)
>  		sum_add_size = min_sum_size;
>  	size1 = !realloc_head ? size0 :
> -		calculate_iosize(sum_add_size, min_size, 0,
> +		calculate_size(sum_add_size, min_size,
>  			resource_size(b_res), min_align);
>  	if (!size0 && !size1) {
>  		if (b_res->start || b_res->end)
> @@ -1617,7 +1602,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
>  	if (size || min_size) {
>  		min_align = calculate_mem_align(&align_test_list, max_align,
>  						size, window_align);
> -		size0 = calculate_memsize(size, min_size,
> +		size0 = calculate_size(size, min_size,
>  				  resource_size(b_res), min_align);
>  	}
>  	free_align_test_list(&align_test_list);
> @@ -1642,7 +1627,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
>  		min_add_align = calculate_mem_align(&align_test_add_list,
>  					max_add_align, sum_add_size,
>  					window_align);
> -		size1 = calculate_memsize(sum_add_size, min_size,
> +		size1 = calculate_size(sum_add_size, min_size,
>  				 resource_size(b_res), min_add_align);
>  	}
>  	free_align_test_list(&align_test_add_list);
> -- 
> 1.8.4.5
> 

Re: [PATCH v3 32/51] resources: Split out __allocate_resource()

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:50PM -0700, Yinghai Lu wrote:
> It will not hold lock, so we could use it in other functions that
> hold the resource lock already.

What's the benefit of this patch?  Does it fix something?  Does it add some
functionality we need?
> 
> -v2: according to Linus, using "bool lock" as parameter
>      aka "conditionally take lock" is *wrong*.
> 
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
> ---
>  kernel/resource.c | 70 +++++++++++++++++++++++++++++++++++++++----------------
>  1 file changed, 50 insertions(+), 20 deletions(-)
> 
> diff --git a/kernel/resource.c b/kernel/resource.c
> index fed052a..67b58a5 100644
> --- a/kernel/resource.c
> +++ b/kernel/resource.c
> @@ -619,7 +619,7 @@ static int find_resource(struct resource *root, struct resource *new,
>  }
>  
>  /**
> - * reallocate_resource - allocate a slot in the resource tree given range & alignment.
> + * __reallocate_resource - allocate a slot in the resource tree given range & alignment.
>   *	The resource will be relocated if the new size cannot be reallocated in the
>   *	current location.
>   *
> @@ -628,7 +628,7 @@ static int find_resource(struct resource *root, struct resource *new,
>   * @newsize: new size of the resource descriptor
>   * @constraint: the size and alignment constraints to be met.
>   */
> -static int reallocate_resource(struct resource *root, struct resource *old,
> +static int __reallocate_resource(struct resource *root, struct resource *old,
>  			resource_size_t newsize,
>  			struct resource_constraint  *constraint)
>  {
> @@ -636,8 +636,6 @@ static int reallocate_resource(struct resource *root, struct resource *old,
>  	struct resource new = *old;
>  	struct resource *conflict;
>  
> -	write_lock(&resource_lock);
> -
>  	if ((err = __find_resource(root, old, &new, newsize, constraint)))
>  		goto out;
>  
> @@ -662,14 +660,13 @@ static int reallocate_resource(struct resource *root, struct resource *old,
>  		BUG_ON(conflict);
>  	}
>  out:
> -	write_unlock(&resource_lock);
>  	return err;
>  }
>  
> -
>  /**
> - * allocate_resource - allocate empty slot in the resource tree given range & alignment.
> - * 	The resource will be reallocated with a new size if it was already allocated
> + * __allocate_resource - allocate empty slot in the resource tree given range & alignment.
> + *	The resource will be reallocated with a new size if it was already
> + *	allocated
>   * @root: root resource descriptor
>   * @new: resource descriptor desired by caller
>   * @size: requested resource region size
> @@ -678,15 +675,17 @@ out:
>   * @align: alignment requested, in bytes
>   * @alignf: alignment function, optional, called if not NULL
>   * @alignf_data: arbitrary data to pass to the @alignf function
> + *
> + * Caller need to hold resource_lock if needed.
>   */
> -int allocate_resource(struct resource *root, struct resource *new,
> -		      resource_size_t size, resource_size_t min,
> -		      resource_size_t max, resource_size_t align,
> -		      resource_size_t (*alignf)(void *,
> -						const struct resource *,
> -						resource_size_t,
> -						resource_size_t),
> -		      void *alignf_data)
> +static int __allocate_resource(struct resource *root, struct resource *new,
> +				resource_size_t size, resource_size_t min,
> +				resource_size_t max, resource_size_t align,
> +				resource_size_t (*alignf)(void *,
> +						  const struct resource *,
> +						  resource_size_t,
> +						  resource_size_t),
> +				void *alignf_data)
>  {
>  	int err;
>  	struct resource_constraint constraint;
> @@ -700,20 +699,51 @@ int allocate_resource(struct resource *root, struct resource *new,
>  	constraint.alignf = alignf;
>  	constraint.alignf_data = alignf_data;
>  
> -	if ( new->parent ) {
> +	if (new->parent) {
>  		/* resource is already allocated, try reallocating with
>  		   the new constraints */
> -		return reallocate_resource(root, new, size, &constraint);
> +		return __reallocate_resource(root, new, size, &constraint);
>  	}
>  
> -	write_lock(&resource_lock);
>  	err = find_resource(root, new, size, &constraint);
>  	if (err >= 0 && __request_resource(root, new))
>  		err = -EBUSY;
> -	write_unlock(&resource_lock);
> +
>  	return err;
>  }
>  
> +/**
> + * allocate_resource - allocate empty slot in the resource tree given range & alignment.
> + *	The resource will be reallocated with a new size if it was already
> + *	allocated
> + * @root: root resource descriptor
> + * @new: resource descriptor desired by caller
> + * @size: requested resource region size
> + * @min: minimum boundary to allocate
> + * @max: maximum boundary to allocate
> + * @align: alignment requested, in bytes
> + * @alignf: alignment function, optional, called if not NULL
> + * @alignf_data: arbitrary data to pass to the @alignf function
> + */
> +int allocate_resource(struct resource *root, struct resource *new,
> +		      resource_size_t size, resource_size_t min,
> +		      resource_size_t max, resource_size_t align,
> +		      resource_size_t (*alignf)(void *,
> +						const struct resource *,
> +						resource_size_t,
> +						resource_size_t),
> +		      void *alignf_data)
> +{
> +	int ret;
> +
> +	write_lock(&resource_lock);
> +	ret = __allocate_resource(root, new, size, min, max, align,
> +				   alignf, alignf_data);
> +	write_unlock(&resource_lock);
> +
> +	return ret;
> +}
> +
>  EXPORT_SYMBOL(allocate_resource);
>  
>  /**
> -- 
> 1.8.4.5
> 

Re: [PATCH v3 33/51] resources: Make allocate_resource return just fit resource

[Bjorn Helgaas]

On Mon, Jul 27, 2015 at 04:29:51PM -0700, Yinghai Lu wrote:
> Find all suitable empty slots and pick one just fit, so we could save
> the big slot for needed ones later when we have several pcie switches
> and some bridges get assigned bios and we need to assign others in kernel.

By "just fit," do you mean "best fit"?  "Best fit" is a well-known term, so
if that's what you mean, let's use it.

I couldn't quite parse the PCIe switch stuff here.  How is that relevant to
this change?

> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
> ---
>  kernel/resource.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++---------
>  1 file changed, 68 insertions(+), 13 deletions(-)
> 
> diff --git a/kernel/resource.c b/kernel/resource.c
> index 67b58a5..697b8ca 100644
> --- a/kernel/resource.c
> +++ b/kernel/resource.c
> @@ -48,6 +48,7 @@ struct resource_constraint {
>  	resource_size_t (*alignf)(void *, const struct resource *,
>  			resource_size_t, resource_size_t);
>  	void *alignf_data;
> +	bool fit;
>  };
>  
>  static DEFINE_RWLOCK(resource_lock);
> @@ -554,12 +555,15 @@ static void resource_clip(struct resource *res, resource_size_t min,
>   * alignment constraints
>   */
>  static int __find_resource(struct resource *root, struct resource *old,
> -			 struct resource *new,
> +			 struct resource *new, struct resource *avail,
>  			 resource_size_t  size,
>  			 struct resource_constraint *constraint)
>  {
>  	struct resource *this = root->child;
> -	struct resource tmp = *new, avail, alloc;
> +	struct resource tmp = *new, availx, alloc;
> +
> +	if (!avail || avail == new)
> +		avail = &availx;
>  
>  	tmp.start = root->start;
>  	/*
> @@ -583,15 +587,16 @@ static int __find_resource(struct resource *root, struct resource *old,
>  		arch_remove_reservations(&tmp);
>  
>  		/* Check for overflow after ALIGN() */
> -		avail.start = ALIGN(tmp.start, constraint->align);
> -		avail.end = tmp.end;
> -		avail.flags = new->flags & ~IORESOURCE_UNSET;
> -		if (avail.start >= tmp.start) {
> -			alloc.flags = avail.flags;
> -			alloc.start = constraint->alignf(constraint->alignf_data, &avail,
> +		avail->start = ALIGN(tmp.start, constraint->align);
> +		avail->end = tmp.end;
> +		avail->flags = new->flags & ~IORESOURCE_UNSET;
> +		if (avail->start >= tmp.start) {
> +			alloc.flags = avail->flags;
> +			alloc.start = constraint->alignf(
> +					constraint->alignf_data, avail,
>  					size, constraint->align);
>  			alloc.end = alloc.start + size - 1;
> -			if (resource_contains(&avail, &alloc)) {
> +			if (resource_contains(avail, &alloc)) {
>  				new->start = alloc.start;
>  				new->end = alloc.end;
>  				return 0;
> @@ -608,6 +613,11 @@ next:		if (!this || this->end == root->end)
>  	return -EBUSY;
>  }
>  
> +struct good_resource {
> +	struct list_head list;
> +	struct resource avail;
> +	struct resource new;
> +};
>  /*
>   * Find empty slot in the resource tree given range and alignment.
>   */
> @@ -615,7 +625,49 @@ static int find_resource(struct resource *root, struct resource *new,
>  			resource_size_t size,
>  			struct resource_constraint  *constraint)
>  {
> -	return  __find_resource(root, NULL, new, size, constraint);
> +	int ret = -1;
> +	LIST_HEAD(head);
> +	struct good_resource *good, *tmp;
> +	resource_size_t avail_size = (resource_size_t)-1ULL;
> +
> +	if (!constraint->fit)
> +		return __find_resource(root, NULL, new, NULL, size,
> +					constraint);
> +
> +	/* find all suitable ones and add to the list */
> +	for (;;) {
> +		good = kzalloc(sizeof(*good), GFP_KERNEL);
> +		if (!good)
> +			break;
> +
> +		good->new.start = new->start;
> +		good->new.end = new->end;
> +		good->new.flags = new->flags;
> +		ret = __find_resource(root, NULL, &good->new, &good->avail,
> +					size, constraint);
> +		if (ret || __request_resource(root, &good->avail)) {
> +			ret = -EBUSY;
> +			kfree(good);
> +			break;
> +		}
> +
> +		list_add(&good->list, &head);
> +	}

Allocating memory and building a list in a function that allocates space
seems like a little bit of a hack.  I think we're holding resource_lock
anyway; can't we just find a candidate, reserve it, look for another one,
reserve it, release the larger one, and repeat?

> +	/* pick up the smallest one and delete the list */
> +	list_for_each_entry_safe(good, tmp, &head, list) {
> +		if (resource_size(&good->avail) < avail_size) {
> +			avail_size = resource_size(&good->avail);
> +			new->start = good->new.start;
> +			new->end = good->new.end;
> +			ret = 0;
> +		}
> +		list_del(&good->list);
> +		__release_resource(&good->avail);
> +		kfree(good);
> +	}
> +
> +	return ret;
>  }
>  
>  /**
> @@ -636,7 +688,8 @@ static int __reallocate_resource(struct resource *root, struct resource *old,
>  	struct resource new = *old;
>  	struct resource *conflict;
>  
> -	if ((err = __find_resource(root, old, &new, newsize, constraint)))
> +	err = __find_resource(root, old, &new, NULL, newsize, constraint);
> +	if (err)
>  		goto out;
>  
>  	if (resource_contains(&new, old)) {
> @@ -675,6 +728,7 @@ out:
>   * @align: alignment requested, in bytes
>   * @alignf: alignment function, optional, called if not NULL
>   * @alignf_data: arbitrary data to pass to the @alignf function
> + * @fit: only allocate fit range.
>   *
>   * Caller need to hold resource_lock if needed.
>   */
> @@ -685,7 +739,7 @@ static int __allocate_resource(struct resource *root, struct resource *new,
>  						  const struct resource *,
>  						  resource_size_t,
>  						  resource_size_t),
> -				void *alignf_data)
> +				void *alignf_data, bool fit)
>  {
>  	int err;
>  	struct resource_constraint constraint;
> @@ -698,6 +752,7 @@ static int __allocate_resource(struct resource *root, struct resource *new,
>  	constraint.align = align;
>  	constraint.alignf = alignf;
>  	constraint.alignf_data = alignf_data;
> +	constraint.fit = fit;
>  
>  	if (new->parent) {
>  		/* resource is already allocated, try reallocating with
> @@ -738,7 +793,7 @@ int allocate_resource(struct resource *root, struct resource *new,
>  
>  	write_lock(&resource_lock);
>  	ret = __allocate_resource(root, new, size, min, max, align,
> -				   alignf, alignf_data);
> +				   alignf, alignf_data, true);
>  	write_unlock(&resource_lock);
>  
>  	return ret;
> -- 
> 1.8.4.5
> 

Re: [PATCH v3 00/51] PCI: Resource allocation cleanup for v4.3

[Yinghai Lu]

On Mon, Aug 17, 2015 at 3:48 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> Overall comments:
>
> Thanks for paying attention to your subject lines.  They are
> capitalized and punctuated quite consistently.
>
> The one-line subject is a summary.  The body of the changelog is not a
> continuation of it.  The body should stand alone even if you don't
> read the subject.  Sometimes that means you have to repeat the subject
> line in the body.  That's OK.
>
> Please use "()" when referring to functions.  You do it sometimes, but
> not always.

Thanks for checking them. Will update those changelog.

>
> I think we should talk about "required/optional" resources instead of
> "must/optional" ones.  Is "addon" a third, separate, thing?

yes, addon should be same as optional.

Actually for devices resources, could safely use required/optional resources.
those resources either required only or optional only resources.
for bridge resources, that is little confusing, as those resources could
have required part only or optional part only or have required part +
optional part.

will update changelog to make them consistent.

also will check code/comment...

Thanks

Yinghai

Re: [PATCH v3 03/51] PCI: Use correct align for optional only resources during sorting

[Yinghai Lu]

On Mon, Aug 17, 2015 at 4:00 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:21PM -0700, Yinghai Lu wrote:
>> During sorting before assign, we only put resource with non-zero align
>> in the sorted list, so for optional resources that must size is 0 and only
>> have addon parts, we need to have correct align.
>>
>> While treating SRIOV as optional resources, we always read alignment for
>> SRIOV bars, so they are ok.
>> Hotplug bridge resources are using STARTALIGN so it is ok when size is 0
>> if we have correct start for them.
>>
>> Later we want to treat the ROM BAR as optional resource, and it has
>> have SIZEALIGN, we need to find a way to get align for them.
>
> Are you saying we have a ROM resource where the ROM BAR is of non-zero
> size, but the resource structure says it's zero size?

Try to treat the ROM resource as optional, so will set the size in the
resource to 0.
aka required part size to 0. We can not get alignment from size anymore.
When we try to sort the resources before assign, we will skip resources with
bogus alignment.


>
>> We can use addon resource align instead in that case, and it will
>> be ok for SRIOV path and hotplug bridge resource path.
>>
>> Sorted list will contain must resource align/size to 0/0 to hold spot for
>> optional resources.
>>
>> We need to pass realloc_head from sizing stage to sorting stage, and
>> get entry from realloc list and calculate align from the entry.
>>
>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
>
> Apparently this fixes a problem?  What is the problem?  What is the
> symptom of the problem?

resource allocation fail, so try to fix it by treat ROM BAR to be optional.



>
>> Reported-by: TJ <linux@iam.tj>
>> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
>> ---
>>  drivers/pci/setup-bus.c | 50 ++++++++++++++++++++++++++++++++++++++++++-------
>>  1 file changed, 43 insertions(+), 7 deletions(-)
>>
>> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
>> index 247d8fe..27cb0f0 100644
>> --- a/drivers/pci/setup-bus.c
>> +++ b/drivers/pci/setup-bus.c
>> @@ -145,9 +145,43 @@ static resource_size_t get_res_add_align(struct list_head *head,
>>       return dev_res->min_align;
>>  }
>>
>> +static resource_size_t __pci_resource_alignment(
>
> I don't like creating new function names by adding "__" to the
> beginning of an existing function name.  It doesn't give enough of a
> clue about what how the functions are different.  I don't really
> understand what's going on here, but this might be a clue that this
> could be refactored differently.

The function try to use alignment from optional part if alignment from
required part is 0.

>
>> +                             struct pci_dev *dev,
>> +                             struct resource *r,
>> +                             struct list_head *realloc_head)
>> +{
>> +     resource_size_t r_align = pci_resource_alignment(dev, r);
>> +     resource_size_t orig_start, orig_end;
>> +     struct pci_dev_resource *dev_res;
>> +
>> +     if (r_align || !realloc_head)
>> +             return r_align;
>> +
>> +     dev_res = res_to_dev_res(realloc_head, r);
>> +     if (!dev_res || !dev_res->add_size)
>> +             return r_align;
>> +
>> +     orig_start = r->start;
>> +     orig_end = r->end;
>> +     r->end += dev_res->add_size;
>> +     if ((r->flags & IORESOURCE_STARTALIGN)) {
>> +             resource_size_t r_size = resource_size(r);
>> +             resource_size_t add_align = dev_res->min_align;
>> +
>> +             r->start = add_align;
>> +             r->end = add_align + r_size - 1;
>
> I think this would be slightly shorter and more obvious as:
>
>   resource_size_t r_size = resource_size(r);
>
>   r->start = dev_res->min_align;
>   r->end = r->start + r_size - 1;

ok, will update that.

Re: [PATCH v3 04/51] PCI: Optimize bus align/size calculation during sizing

[Yinghai Lu]

On Mon, Aug 17, 2015 at 4:49 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:22PM -0700, Yinghai Lu wrote:
>> Current code try to get align as small as possible and use that to
>> align final size. But it does not handle resource that size is bigger
>> than align in optimal way, kernel only use max align for them.
>>
>> For example:
>>  when we have resources with align/size: 1M/2M, 512M/512M,
>>    bus resource min_align/size0 will be 512M/1024M,
>>    but optimal value should be 256M/768M.
>
> I want to understand what makes 256M/768M "optimal."
>
> This is basically a bin packing problem, and I'd like to have an
> explicit statement of the constraints and the goal.  Right now the
> goal is fuzzy and the code seems ad hoc.

that code is trying to use different offset to make sure it is safe
to reduce the alignment.

>
> Here are the possibilities I can see.  The placement notes are the
> offsets into the allocated space:
>
>   align  size    512M placement   2M placement    unused
>   1024M  514M    [0-512]          [512-514]       [514-1024]
>    512M  514M    [0-512]          [512-514]       none
>    256M  768M    [256-768]        [254-256]       [0-254]
>    128M  896M    [384-896]        [382-384]       [0-382]
>
> Obviously, we would never choose 1024M/514M or any larger alignment:
> it requires no more space than 512M/514M, but it fails in some cases
> where 512M/514M would succeed.
>
> Also obviously, we would never choose 128M/896M or a smaller
> alignment: if we could get that, we could also get 256M/768M and it
> would consume less space.
>
> But it's not quite so obvious how to choose between 512M/514M and
> 256M/768M.  The former (if we can get it) consumes much less space.
> The latter requires less alignment but wastes a lot of space.

one is smaller alignment and size is aligned to min_align.
other one is smaller size, but size is not aligned to max_align.

let's call them min_align solution and alt_size solution.

in following patches will have support for alt_size.

the problem for alt_size is that we can not keep on using them on
parent's sizing. for example two bridges have 512M/514M, 512M/514M,
for their parent bridge simple calculation will have 512M/1028M.
with trying out, we will need 512M/1536M instead.

and with min_align solution, two children will be 256M/768M, 256M/768M
and their parent bridge will be 256M/1536M.

so min_align will have smaller alignment and same size.

Final solution in the patchset: we are trying min_align at first, and
if it fails
then try alt_size. And during alt_size sizing will check if min_align even
could have smaller alignment and smaller size.

>
>> For following cases that we have resource size that is bigger
>> than resource alignment:
>> 1. SRIOV bar.
>> 2. PCI bridges with several bridges or devices as children.
>
> This doesn't have anything to do with how many children a bridge has,
> does it?  As long as there are two or more BARs (1MB or larger) below
> a bridge, we'll have the situation where the bridge window has to
> contain both BARs but only needs to be aligned for (at most) the
> larger one.

Yes. even one child could request several pref MMIO.

>
>> We can keep on trying to allocate children devices resources under range
>> [half_align, half_align + aligned_size).
>> If sucesses, we can use that half_align as new min_align.
>>
>> After this patch, we get:
>>  align/size: 1M/2M, 2M/4M, 4M/8M, 8M/16M
>>  new min_align/min_size: 4M/32M, and old is 8M/32M
>>
>>  align/size: 1M/2M, 2M/4M, 4M/8M
>>  new min_align/min_size: 2M/14M, and old is 4M/16M
>>
>>  align/size: 1M/2M, 512M/512M
>>  new min_align/min_size: 256M/768M, and old is 512M/1024M
>>
>> The real result from one system with one pcie card that has
>> four functions that support sriov:
>>  align/size:
>>    00800000/00800000
>>    00800000/00800000
>>    00800000/00800000
>>    00800000/00800000
>>    00010000/00200000
>>    00010000/00200000
>>    00010000/00200000
>>    00010000/00200000
>>    00008000/00008000
>>    00008000/00008000
>>    00008000/00008000
>>    00008000/00008000
>>    00004000/00080000
>>    00004000/00080000
>>    00004000/00080000
>>    00004000/00080000
>>  old min_align/min_size: 00400000/02c00000
>>      min_align/min_size: 00100000/02b00000
>
> I don't know if this example is essential, but if it is, it needs a
> little more interpretation.  I assume the BARs above where
> "align < size" are for SR-IOV, where size include multiple VFs.
> In any case, please connect the dots from the raw data to the
> conclusion.

ok.

>
>> So align will be 1M instead of 4M.
>>
>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=81431
>> Reported-by: TJ <linux@iam.tj>
>> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
>> ---
>>  drivers/pci/setup-bus.c | 195 ++++++++++++++++++++++++++++++++++++++----------
>>  1 file changed, 157 insertions(+), 38 deletions(-)
>>
>> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
>> index 27cb0f0..ecdf011 100644
>> --- a/drivers/pci/setup-bus.c
>> +++ b/drivers/pci/setup-bus.c
>> @@ -30,6 +30,34 @@
>>
>>  unsigned int pci_flags;
>>
>> +static inline bool is_before(resource_size_t align1, resource_size_t size1,
>> +                          resource_size_t align2, resource_size_t size2)
>
> Can this take two struct resource pointers instead of four numbers?  I
> haven't looked at all the callers, so maybe there's a caller that
> doesn't have a struct resource.
>
That is called in double loop, so i want to save some calculation.
also it would take (dev1, res1, dev2, res2).

Re: [PATCH v3 07/51] PCI: Reorder resources list for must/optional resources

[Yinghai Lu]

On Mon, Aug 17, 2015 at 4:52 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:25PM -0700, Yinghai Lu wrote:
>> After we update size and alignment for must+optional resource, we
>> reorder them with new alignment, but this is only for STARTALIGN.
>>
>> For SIZEALIGN type resource, after add back add_size, the alignment
>> get changed, so need to do the sorting like STARTALIGN type resources.
>>
>> Also we need to reorder the sorting back after we restore
>> resource to must only when must+optional fail to allocate for all.
>>
>> So move out the reordering code from the loop to separated function,
>> and call it two times accordingly.
>>
>> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
>> ---
>>  drivers/pci/setup-bus.c | 62 +++++++++++++++++++++++++++++--------------------
>>  1 file changed, 37 insertions(+), 25 deletions(-)
>>
>> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
>> index 7346bbf..6f2d508 100644
>> --- a/drivers/pci/setup-bus.c
>> +++ b/drivers/pci/setup-bus.c
>> @@ -287,6 +287,31 @@ static inline void reset_resource(struct resource *res)
>>       res->flags = 0;
>>  }
>>
>> +static void __sort_resources(struct list_head *head)
>
> Why did you add "__" before the function name?

ok, will remove that.

Re: [PATCH v3 01/51] PCI: Cleanup res_to_dev_res() printout for addon resources

[Yinghai Lu]

On Mon, Aug 17, 2015 at 3:50 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:19PM -0700, Yinghai Lu wrote:
>> Now get_res_add_size and get_res_add_align all have same printout
>> from res_to_dev_res(), and it is confusing.
>>
>> Move out debug messages printout from res_to_dev_res(),
>> and later we will reuse res_to_dev_res() in other functions.
>>
>> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
>> ---
>>  drivers/pci/setup-bus.c | 34 ++++++++++++++++++++--------------
>>  1 file changed, 20 insertions(+), 14 deletions(-)
>>
>> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
>> index 508cc56..f0fa705 100644
>> --- a/drivers/pci/setup-bus.c
>> +++ b/drivers/pci/setup-bus.c
>> @@ -104,19 +104,9 @@ static struct pci_dev_resource *res_to_dev_res(struct list_head *head,
>>  {
>>       struct pci_dev_resource *dev_res;
>>
>> -     list_for_each_entry(dev_res, head, list) {
>> -             if (dev_res->res == res) {
>> -                     int idx = res - &dev_res->dev->resource[0];
>> -
>> -                     dev_printk(KERN_DEBUG, &dev_res->dev->dev,
>> -                              "res[%d]=%pR res_to_dev_res add_size %llx min_align %llx\n",
>> -                              idx, dev_res->res,
>> -                              (unsigned long long)dev_res->add_size,
>> -                              (unsigned long long)dev_res->min_align);
>> -...

>> +
>> +     dev_printk(KERN_DEBUG, &dev_res->dev->dev,
>> +                "BAR %d: %pR get_res_add_align min_align %llx\n",
>> +                (int)(res - &dev_res->dev->resource[0]),
>> +                res, (unsigned long long)dev_res->min_align);
>
> Please use %#llx so we have a "0x" prefix like other hex numbers.
>

ok.
Later i'd like to just kill those debug printout if all get settled
after this patchset.

Re: [PATCH v3 09/51] PCI: Rename pdev_sort_resources to pdev_check_resources

[Yinghai Lu]

On Mon, Aug 17, 2015 at 4:53 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:27PM -0700, Yinghai Lu wrote:
>> We don't do sorting in those functions anymore, so change "sort" to "check"
>> instead.
>
> What *do* we do in those functions?  "Check" doesn't tell me anything.

Put all children devices' resources that need to assign in one list, and also
do some sanitary checking like if align is good or not.

Re: [PATCH v3 09/51] PCI: Rename pdev_sort_resources to pdev_check_resources

[Yinghai Lu]

On Tue, Aug 18, 2015 at 2:36 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> On Mon, Aug 17, 2015 at 4:53 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
>> On Mon, Jul 27, 2015 at 04:29:27PM -0700, Yinghai Lu wrote:
>>> We don't do sorting in those functions anymore, so change "sort" to "check"
>>> instead.
>>
>> What *do* we do in those functions?  "Check" doesn't tell me anything.
>
> Put all children devices' resources that need to assign in one list, and also
> do some sanitary checking like if align is good or not.

or change to dev_assign_resources_prepare() ?

Re: [PATCH v3 13/51] PCI: Separate realloc list checking after allocation

[Yinghai Lu]

On Mon, Aug 17, 2015 at 4:54 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:31PM -0700, Yinghai Lu wrote:
>> We check the realloc list, as list must be empty after allocation.
>>
>> Separate the realloc list checking to another function.
>>
>> Add checking that is missed in acpiphp driver.
>>
>> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
>> Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
>> Cc: Len Brown <lenb@kernel.org>
>> Cc: linux-acpi@vger.kernel.org
>> ---
>>  drivers/pci/hotplug/acpiphp_glue.c |  1 +
>>  drivers/pci/pci.h                  |  1 +
>>  drivers/pci/setup-bus.c            | 11 ++++++++---
>>  3 files changed, 10 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/pci/hotplug/acpiphp_glue.c b/drivers/pci/hotplug/acpiphp_glue.c
>> index ff53856..134caee 100644
>> --- a/drivers/pci/hotplug/acpiphp_glue.c
>> +++ b/drivers/pci/hotplug/acpiphp_glue.c
>> @@ -507,6 +507,7 @@ static void enable_slot(struct acpiphp_slot *slot)
>>               }
>>       }
>>       __pci_bus_assign_resources(bus, &add_list, NULL);
>> +     __pci_bus_check_realloc(&add_list);
>>
>>       acpiphp_sanitize_bus(bus);
>>       pcie_bus_configure_settings(bus);
>> diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
>> index 4ff0ff1..2b83977 100644
>> --- a/drivers/pci/pci.h
>> +++ b/drivers/pci/pci.h
>> @@ -235,6 +235,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus,
>>  void __pci_bus_assign_resources(const struct pci_bus *bus,
>>                               struct list_head *realloc_head,
>>                               struct list_head *fail_head);
>> +void __pci_bus_check_realloc(struct list_head *realloc_head);
>>  bool pci_bus_clip_resource(struct pci_dev *dev, int idx);
>>
>>  void pci_reassigndev_resource_alignment(struct pci_dev *dev);
>> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
>> index 06664db..f30225c 100644
>> --- a/drivers/pci/setup-bus.c
>> +++ b/drivers/pci/setup-bus.c
>> @@ -350,6 +350,11 @@ out:
>>       }
>>  }
>>
>> +void __pci_bus_check_realloc(struct list_head *realloc_head)
>> +{
>> +     BUG_ON(!list_empty(realloc_head));
>
> I don't think this really improves readability.  At least before, we could
> see what we were looking for and we knew it might BUG().

ok, will only missing BUG_ON into acpiphp_glue.c.

Re: [PATCH v3 20/51] PCI: Skip must+optional if there is no optional addon

[Yinghai Lu]

On Mon, Aug 17, 2015 at 4:56 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:38PM -0700, Yinghai Lu wrote:
>> If the bridge does not support hotplug or no child with sriov support
>> we could get out early and don't try must+optional allocation.
>
> You mention non-hotplug bridges and non-SR-IOV devices, but I can't figure
> out how the patch itself relates to those.  How can I connect them?

in setup that does not support hotplug and sriov, then we will not
have optional resources.

>
>> Also in the loop that update res with optional add info, skip resource
>> that add_size is 0.
>>
>> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
>> ---
>>  drivers/pci/setup-bus.c | 32 +++++++++++++++++++++++++++++---
>>  1 file changed, 29 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
>> index d1f9e19..64ef516 100644
>> --- a/drivers/pci/setup-bus.c
>> +++ b/drivers/pci/setup-bus.c
>> @@ -450,6 +450,24 @@ static bool pci_need_to_release(unsigned long mask, struct resource *res)
>>       return false;   /* should not get here */
>>  }
>>
>> +static bool __has_addon(struct list_head *head,
>> +                     struct list_head *realloc_head)
>
> Why does this have a "__" prefix?

Sorry, will remove that.

>
>> +{
>> +     int add_count = 0;
>> +     struct pci_dev_resource *dev_res, *tmp_res;
>> +
>> +     /* check if we have add really */
>> +     list_for_each_entry(dev_res, head, list) {
>> +             tmp_res = res_to_dev_res(realloc_head, dev_res->res);
>> +             if (!tmp_res || !tmp_res->add_size)
>> +                     continue;
>> +
>> +             add_count++;
>> +     }
>> +
>> +     return add_count != 0;
>> +}
>> +
>>  static bool save_resources(struct list_head *head,
>>                          struct list_head *save_head)
>>  {
>> @@ -481,16 +499,24 @@ static bool __assign_resources_must_add_sorted(struct list_head *head,
>>       struct pci_dev_resource *save_res;
>>       struct pci_dev_resource *dev_res, *tmp_res;
>>       unsigned long fail_type;
>> -     resource_size_t add_align;
>> +     resource_size_t add_align, add_size;
>>       struct resource *res;
>>
>> +     if (!__has_addon(head, realloc_head))
>> +             return false;
>> +
>>       if (!save_resources(head, &save_head))
>>               return false;
>>
>>       /* Update res in head list with add_size in realloc_head list */
>>       list_for_each_entry(dev_res, head, list) {
>>               res = dev_res->res;
>> -             res->end += get_res_add_size(realloc_head, res);
>> +             add_size = get_res_add_size(realloc_head, res);
>> +
>> +             if (!add_size)
>> +                     continue;
>> +
>> +             res->end += add_size;
>>
>>               /*
>>                * There are two kinds of additional resources in the list:
>> @@ -578,7 +604,7 @@ static void __assign_resources_sorted(struct list_head *head,
>>        */
>>
>>       /* Check must+optional add */
>> -     if (realloc_head && !list_empty(realloc_head) &&
>> +     if (realloc_head &&
>
> Why are you dropping the !list_empty() test?

count in __has_addon() will be 0, so we can skip checking here.

>
>>           __assign_resources_must_add_sorted(head, realloc_head))
>>               return;
>>
>> --
>> 1.8.4.5
>>

Re: [PATCH v3 22/51] PCI: Add alt_size allocation support

[Yinghai Lu]

On Mon, Aug 17, 2015 at 5:03 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:40PM -0700, Yinghai Lu wrote:
>> On system with several pcie switches, BIOS allocate very tight resources
>> to the bridge bar, and it is not aligned to min_align as kernel allocation
>> code.
>
> I can't parse this.

BIOS allocate resource in different way.

kernel is trying to find smallest align (min_align) and use it to get
aligned min_size.

>
>> For example:
>>       02:03.0---0c:00.0---0d:04.0---18:00.0
>> 18:00.0 need 0x10000000, and 0x00010000.
>> BIOS only allocate 0x10100000 to 0d:04.0 and above bridges.
>
> Do you mean the BIOS only allocated 0x10010000?

I can not find the exact bus layout on hand. only one similar ...

23 13:15:49  kernel: pci_bus 0000:10: scanning bus
Jun 23 13:15:49  kernel: pci 0000:10:00.0: [xxxx:xxxx] type 00 class 0x028000
Jun 23 13:15:49  kernel: pci 0000:10:00.0: reg 0x10: [mem
0xb0000000-0xbfffffff 64bit pref]
Jun 23 13:15:49  kernel: pci 0000:10:00.0: reg 0x18: [mem
0xc0000000-0xc000ffff 64bit pref]
Jun 23 13:15:49  kernel: pci_bus 0000:10: fixups for bus
Jun 23 13:15:49  kernel: pci 0000:05:04.0: PCI bridge to [bus 10-17]
Jun 23 13:15:49  kernel: pci 0000:05:04.0:   bridge window [mem
0xb0000000-0xc00fffff]
Jun 23 13:15:49  kernel: pci_bus 0000:10: bus scan returning with max=10

so device is using 0x10000000 and 0x00010000
and bridge is 0x10100000

As the bridge MMIO need to be aligned to 1M.

>
>> Later after using /sys/bus/pci/devices/0000:0c:00.0/remove to remove 0c:00.0,
>> rescan with /sys/bus/pci/rescan can not allocate 0x18000000 to 0c:00.0.
>>
>> another example:
>> 00:1c.0-[02-21]----00.0-[03-21]--+-01.0-[04-12]----00.0-[05-12]----19.0-[06-12]----00.0
>>                                  +-05.0-[13]--
>>                                  +-07.0-[14-20]----00.0-[15-20]--+-08.0-[16]--+-00.0
>>                                  |                               |            \-00.1
>>                                  |                               +-14.0-[17]----00.0
>>                                  |                               \-19.0-[18-20]----00.0
>>                                  \-09.0-[21]--
>> 06:00.0 need 0x4000000 and 0x800000.
>> BIOS only allocate 0x4800000 to 05:19.0 and 04:00.0.
>> when 05:19.0 get removed via /sys/bus/pci/devices/0000:05:19.0/remove,
>> rescan with /sys/bus/pci/rescan will fail.
>>  pci 0000:05:19.0: BAR 14: no space for [mem size 0x06000000]
>>  pci 0000:05:19.0: BAR 14: failed to assign [mem size 0x06000000]
>>  pci 0000:06:00.0: BAR 2: no space for [mem size 0x04000000 64bit]
>>  pci 0000:06:00.0: BAR 2: failed to assign [mem size 0x04000000 64bit]
>>  pci 0000:06:00.0: BAR 0: no space for [mem size 0x00800000]
>>  pci 0000:06:00.0: BAR 0: failed to assign [mem size 0x00800000]
>> current code try to use align 0x2000000 and size 0x6000000, but parent
>> bridge only have 0x4800000.
>
> I *think* you're saying:
>   - BIOS assigned space for device X
>   - We remove X via sysfs
>   - We rescan via sysfs and discover X
>   - We try to assign space for X
>   - We fail because we don't use the same algorithm as BIOS did
>
> If there is an optimal way to assign space for an arbitrary number of
> BARs, we could just adopt it.  I don't know what that is, and I don't
> know whether an optimal algorithm exists even in principle.
>
> If there is no single optimal algorithm, there will always be cases where
> we fail because we use a different algorithm than the firmware did.

That is what this patch try to do.

alt_size solution that is preferring smaller size and big alignment.

Use it together with min_align solution that is used in kernel.

>
>> Introduce alt_align/alt_size and store them in realloc list in addition
>> to addon info, and will try it after min_align/min_size allocation fails.
>
> What does "alt" mean?
>

alternative

Re: [PATCH v3 25/51] PCI: Fix size calculation with old_size on rescan path

[Yinghai Lu]

On Mon, Aug 17, 2015 at 9:09 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:43PM -0700, Yinghai Lu wrote:
>> On booting path, we don't pass realloc at first, and treat all optional just
>> as must, in some case we can smaller size/align with optional than must only.
>>
>> 04:00.0 has four child bridges: 05:01.0, 05:02.0, 05:03.0, 05:04.0
>> pref layout after booting path like followings:
>>
>> pci 0000:04:00.0: BAR 9: assigned [mem 0x84000000-0x9fffffff 64bit pref]
>> pci 0000:05:04.0: BAR 9: assigned [mem 0x88000000-0x9fffffff 64bit pref]
>> pci 0000:05:03.0: BAR 9: assigned [mem 0x84000000-0x841fffff 64bit pref]
>> pci 0000:07:00.0: BAR 0: assigned [mem 0x81200000-0x8123ffff 64bit]
>> pci 0000:05:03.0: PCI bridge to [bus 08-0f]
>> pci 0000:05:03.0:   bridge window [mem 0x84000000-0x841fffff 64bit pref]
>> pci 0000:10:00.0: BAR 0: assigned [mem 0x90000000-0x9fffffff 64bit pref]
>> pci 0000:10:00.0: BAR 2: assigned [mem 0x88000000-0x8800ffff 64bit pref]
>> pci 0000:05:04.0: PCI bridge to [bus 10]
>> pci 0000:05:04.0:   bridge window [mem 0x88000000-0x9fffffff 64bit pref]
>> pci 0000:04:00.0: PCI bridge to [bus 05-10]
>> pci 0000:04:00.0:   bridge window [mem 0x84000000-0x9fffffff 64bit pref]
>
> Unless 07:00.0 and 10:00.0 are relevant, please remove them.

ok

>
>> so the old size in rescan for 04:00.0 would be 0x1c000000, and align is 0x4000000
>>
>> during remove and rescan:
>>
>> pci 0000:05:03.0: bridge window [mem 0x00000000-0xffffffffffffffff 64bit pref] to [bus 08-0f] add_size 200000 add_align 100000 alt_size 0 alt_align 0 must_size 0 must_align 0
>
> What is "[mem 0x00000000-0xffffffffffffffff 64bit pref]" telling us?
> Surely that's not an actual aperture, is it?

required size is 0, the optional only resources.

Re: [PATCH v3 27/51] PCI: Move ISA ioport align out of calculate_iosize

[Yinghai Lu]

On Mon, Aug 17, 2015 at 9:11 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:45PM -0700, Yinghai Lu wrote:
>> So we could unify calculate_iosize and calculate_memsize later.
>
> Please make this a complete sentence, not just a continuation of the
> summary line.
>
>> when one bridge have several children devices, and every devices
>> have several io port resources and resource size < 0x400.
>
> This isn't a sentence either, so I'm not sure what this is supposed to
> mean.

will change to:

Subject: [PATCH] PCI: Move ISA io port align out of calculate_iosize()

We can move ISA io port align out of calculate_iosize(),
so we could unify calculate_iosize and calculate_memsize later.

That extra aligning or offset is to work around ISA devices:
When one bridge have several children devices, and every device
has several io port resources and resource size < 0x400.
We need to check size, and add extra size to make sure bit8/9
to be zero.

Re: [PATCH v3 28/51] PCI: Unifiy calculate_size for io port and mmio

[Yinghai Lu]

On Mon, Aug 17, 2015 at 9:13 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:46PM -0700, Yinghai Lu wrote:
>> We should check size+size1 with min_size for io port.
>> For example, when hotplug bridge has two children bridges,
>> every child bridge will need 0x1000, so size1 will be 0x2000
>> and size is 0. The min_size for the hotplug bridge is 0x100.
>
> A min_size of 0x100?  Is that a typo?

yes, it is 0x100.

#define DEFAULT_HOTPLUG_IO_SIZE         (256)
/* pci=hpmemsize=nnM,hpiosize=nn can override this */
unsigned long pci_hotplug_io_size  = DEFAULT_HOTPLUG_IO_SIZE;

and we have

                if (bus->self->is_hotplug_bridge) {
                        min_io_size  = pci_hotplug_io_size;

>
>> with old version calculate_iosize, we get 0x3000 for final
>> size because we are using size to compare with min_size. That is
>> not right, we should use 0x2000 instead.
>
> If this fixes a bug, please make one patch that *only* fixes the bug,
> and a separate one that unifies but doesn't change the behavior.

ok, will separate it into two.

Re: [PATCH v3 32/51] resources: Split out __allocate_resource()

[Yinghai Lu]

On Mon, Aug 17, 2015 at 9:14 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:50PM -0700, Yinghai Lu wrote:
>> It will not hold lock, so we could use it in other functions that
>> hold the resource lock already.
>
> What's the benefit of this patch?  Does it fix something?  Does it add some
> functionality we need?

will use it in following patch.

Re: [PATCH v3 33/51] resources: Make allocate_resource return just fit resource

[Yinghai Lu]

On Mon, Aug 17, 2015 at 9:21 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> On Mon, Jul 27, 2015 at 04:29:51PM -0700, Yinghai Lu wrote:
>> Find all suitable empty slots and pick one just fit, so we could save
>> the big slot for needed ones later when we have several pcie switches
>> and some bridges get assigned bios and we need to assign others in kernel.
>
> By "just fit," do you mean "best fit"?  "Best fit" is a well-known term, so
> if that's what you mean, let's use it.

Sure. Thanks for dig that URL out.

>
> I couldn't quite parse the PCIe switch stuff here.  How is that relevant to
> this change?

That is most use cases. One parent have several child bridges, and BIOS does
poor jobs and only resources to some child bridges. Kernel need to assign
for others. so before kernel start, parent resource window already have
several blank slots instead of one big chunk. then we need to make sure
first come only take best fit and leave big one for possible late comer.

>
> Allocating memory and building a list in a function that allocates space
> seems like a little bit of a hack.  I think we're holding resource_lock
> anyway; can't we just find a candidate, reserve it, look for another one,
> reserve it, release the larger one, and repeat?

We need to reserve the all avail windows before pick up one window.

for examples: we have window
[0xc0000000, 0xd0000000), and [0xe0000000,0xe1000000)

and we try allocate 0x200000 size resources.

in this patch will reserve [0xc0000000, 0xd0000000) and
[0xe0000000,0xe1000000) at first, then pick [0xe0000000,0xe1000000)