[PATCH v6 0/7] PCI: Change assign unassigned resources per root bus bassis

[PATCH v6 0/7] PCI: Change assign unassigned resources per root bus bassis

[Yinghai Lu]

BenH reported that there is some assign unassigned resource problem
in powerpc.

It turns out after
| commit 0c5be0cb0edfe3b5c4b62eac68aa2aa15ec681af
| Date:   Thu Feb 23 19:23:29 2012 -0800
|
|    PCI: Retry on IORESOURCE_IO type allocations

even the root bus does not have io port range, it will keep retrying
to realloc with mmio.

After checking the code, found that we bound io port and mmio fail
path together.
First patch fix the problem, that will not make mmio fall back to must-only
when only have io port fail with must+optional.

That is first patch.

During we found the fix for that problem, found that we can separate assign
unassigned resources to per root bus.
that will make the code simple, also could reuse it for hotadd path.

Besides that, "PCI: Enable pci bridge when it is needed" will delay pci
bridge enabling, that will kill another difference between booting path
and hotadd path.

-v4: split first patch into 4 patches per Bjorn.
-v5: drop two patches that will pass root bus resource mask after we found
     simple and less intrusive way to fix the problem.
-v6: refreshed on top of v3.11-rc2

could get from
        git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git for-pci-assign-unassigned

Thanks

Yinghai

Yinghai Lu (7):
  PCI: Don't use temp bus for pci_bus_release_bridge_resources
  PCI: Use pci_walk_bus to detect unassigned resources
  PCI: Check pci bus address for unassigned res
  PCI: Introduce enable_local to prepare per root bus handling
  PCI: Split pci_assign_unassigned_resources to per root bus
  PCI: Enable pci bridge when it is needed
  PCI: Retry assign unassigned resources for hotadd root bus

 arch/arm/kernel/bios32.c           |   5 --
 arch/m68k/platform/coldfire/pci.c  |   1 -
 arch/mips/pci/pci.c                |   1 -
 arch/sh/drivers/pci/pci.c          |   1 -
 drivers/acpi/pci_root.c            |   5 +-
 drivers/parisc/lba_pci.c           |   1 -
 drivers/pci/bus.c                  |  19 -----
 drivers/pci/hotplug/acpiphp_glue.c |   1 -
 drivers/pci/pci.c                  |  20 ++++++
 drivers/pci/probe.c                |   1 -
 drivers/pci/setup-bus.c            | 141 +++++++++++++++++++------------------
 drivers/pcmcia/cardbus.c           |   1 -
 include/linux/pci.h                |   2 +-
 13 files changed, 95 insertions(+), 104 deletions(-)

-- 
1.8.1.4

[PATCH v6 1/7] PCI: Don't use temp bus for pci_bus_release_bridge_resources

[Yinghai Lu]

as later bus can not be used as temp variable after we change to
per root bus handling with assign unassigned resources.

Per Bjorn, separate it from big patch that handing assign_unssigned per root bus.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index d254e23..7de30e3 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1458,12 +1458,11 @@ again:
 	 * Try to release leaf bridge's resources that doesn't fit resource of
 	 * child device under that bridge
 	 */
-	list_for_each_entry(fail_res, &fail_head, list) {
-		bus = fail_res->dev->bus;
-		pci_bus_release_bridge_resources(bus,
+	list_for_each_entry(fail_res, &fail_head, list)
+		pci_bus_release_bridge_resources(fail_res->dev->bus,
 						 fail_res->flags & type_mask,
 						 rel_type);
-	}
+
 	/* restore size and flags */
 	list_for_each_entry(fail_res, &fail_head, list) {
 		struct resource *res = fail_res->res;
-- 
1.8.1.4

[PATCH v6 2/7] PCI: Use pci_walk_bus to detect unassigned resources

[Yinghai Lu]

Per Bjorn, use pci_walk_bus instead of for_each_pci_dev or
calling pci_realloc_detect() recursively, that will make code more readable.

Per Bjorn, separate it from big patch that handing assign_unssigned per root bus.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 46 +++++++++++++++++++++++++++++++---------------
 1 file changed, 31 insertions(+), 15 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 7de30e3..ec93aa0 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1359,30 +1359,46 @@ static bool __init pci_realloc_enabled(void)
 	return pci_realloc_enable >= user_enabled;
 }
 
-static void __init pci_realloc_detect(void)
-{
 #if defined(CONFIG_PCI_IOV) && defined(CONFIG_PCI_REALLOC_ENABLE_AUTO)
-	struct pci_dev *dev = NULL;
+static int __init check_unassigned_resources(struct pci_dev *dev, void *data)
+{
+	int i;
+	int *unassigned = data;
 
-	if (pci_realloc_enable != undefined)
-		return;
+	for (i = PCI_IOV_RESOURCES; i <= PCI_IOV_RESOURCE_END; i++) {
+		struct resource *r = &dev->resource[i];
 
-	for_each_pci_dev(dev) {
-		int i;
+		/* Not assigned, or rejected by kernel ? */
+		if (r->flags && !r->start) {
+			(*unassigned)++;
+			return 1; /* return early from pci_walk_bus */
+		}
+	}
 
-		for (i = PCI_IOV_RESOURCES; i <= PCI_IOV_RESOURCE_END; i++) {
-			struct resource *r = &dev->resource[i];
+	return 0;
+}
 
-			/* Not assigned, or rejected by kernel ? */
-			if (r->flags && !r->start) {
-				pci_realloc_enable = auto_enabled;
+static void  __init pci_realloc_detect(void)
+{
+	int unassigned = 0;
+	struct pci_bus *bus;
 
-				return;
-			}
+	if (pci_realloc_enable != undefined)
+		return;
+
+	list_for_each_entry(bus, &pci_root_buses, node) {
+		pci_walk_bus(bus, check_unassigned_resources, &unassigned);
+		if (unassigned) {
+			pci_realloc_enable = auto_enabled;
+			return;
 		}
 	}
-#endif
 }
+#else
+static void __init pci_realloc_detect(void)
+{
+}
+#endif
 
 /*
  * first try will not touch pci bridge res
-- 
1.8.1.4

[PATCH v6 3/7] PCI: Check pci bus address for unassigned res

[Yinghai Lu]

We should compare res->start with root bus window offset.
Otherwise will have problem with arch that support hostbridge
resource offset.

BenH pointed out that during reviewing patchset that separate
assign unassigned to per root buses.

According to Bjorn, have it in separated patch.

Use pcibios_resource_to_bus to get region at first, and check
region.start instead.

Suggested-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index ec93aa0..87687a5 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1367,9 +1367,14 @@ static int __init check_unassigned_resources(struct pci_dev *dev, void *data)
 
 	for (i = PCI_IOV_RESOURCES; i <= PCI_IOV_RESOURCE_END; i++) {
 		struct resource *r = &dev->resource[i];
+		struct pci_bus_region region;
 
 		/* Not assigned, or rejected by kernel ? */
-		if (r->flags && !r->start) {
+		if (!r->flags)
+			continue;
+
+		pcibios_resource_to_bus(dev, &region, r);
+		if (!region.start) {
 			(*unassigned)++;
 			return 1; /* return early from pci_walk_bus */
 		}
-- 
1.8.1.4

[PATCH v6 4/7] PCI: Introduce enable_local to prepare per root bus handling

[Yinghai Lu]

Add enable_local to prepare assign unassigned resource
for per root bus.

Per Bjorn, separate it from big patch that handing assign_unssigned per root bus.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 40 +++++++++++++++++++++-------------------
 1 file changed, 21 insertions(+), 19 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 87687a5..eddc6e1 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1354,9 +1354,9 @@ void __init pci_realloc_get_opt(char *str)
 	else if (!strncmp(str, "on", 2))
 		pci_realloc_enable = user_enabled;
 }
-static bool __init pci_realloc_enabled(void)
+static bool __init pci_realloc_enabled(enum enable_type enable)
 {
-	return pci_realloc_enable >= user_enabled;
+	return enable >= user_enabled;
 }
 
 #if defined(CONFIG_PCI_IOV) && defined(CONFIG_PCI_REALLOC_ENABLE_AUTO)
@@ -1383,25 +1383,25 @@ static int __init check_unassigned_resources(struct pci_dev *dev, void *data)
 	return 0;
 }
 
-static void  __init pci_realloc_detect(void)
+static enum enable_type __init pci_realloc_detect(struct pci_bus *bus,
+			 enum enable_type enable_local)
 {
 	int unassigned = 0;
-	struct pci_bus *bus;
 
-	if (pci_realloc_enable != undefined)
-		return;
+	if (enable_local != undefined)
+		return enable_local;
 
-	list_for_each_entry(bus, &pci_root_buses, node) {
-		pci_walk_bus(bus, check_unassigned_resources, &unassigned);
-		if (unassigned) {
-			pci_realloc_enable = auto_enabled;
-			return;
-		}
-	}
+	pci_walk_bus(bus, check_unassigned_resources, &unassigned);
+	if (unassigned)
+		return auto_enabled;
+
+	return enable_local;
 }
 #else
-static void __init pci_realloc_detect(void)
+static enum enable_type __init pci_realloc_detect(struct pci_bus *bus,
+			 enum enable_type enable_local)
 {
+	return enable_local;
 }
 #endif
 
@@ -1424,10 +1424,12 @@ pci_assign_unassigned_resources(void)
 	unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
 				  IORESOURCE_PREFETCH;
 	int pci_try_num = 1;
+	enum enable_type enable_local = pci_realloc_enable;
+
+	list_for_each_entry(bus, &pci_root_buses, node)
+		enable_local = pci_realloc_detect(bus, enable_local);
 
-	/* don't realloc if asked to do so */
-	pci_realloc_detect();
-	if (pci_realloc_enabled()) {
+	if (pci_realloc_enabled(enable_local)) {
 		int max_depth = pci_get_max_depth();
 
 		pci_try_num = max_depth + 1;
@@ -1459,9 +1461,9 @@ again:
 		goto enable_and_dump;
 
 	if (tried_times >= pci_try_num) {
-		if (pci_realloc_enable == undefined)
+		if (enable_local == undefined)
 			printk(KERN_INFO "Some PCI device resources are unassigned, try booting with pci=realloc\n");
-		else if (pci_realloc_enable == auto_enabled)
+		else if (enable_local == auto_enabled)
 			printk(KERN_INFO "Automatically enabled pci realloc, if you have problem, try booting with pci=realloc=off\n");
 
 		free_list(&fail_head);
-- 
1.8.1.4

[PATCH v6 5/7] PCI: Split pci_assign_unassigned_resources to per root bus

[Yinghai Lu]

We need to split pci_assign_unassiged_resource to every root bus, so can
have different retry for assign_unassigned per root bus

Also we need root bus hot add and booting path use same code.

-v2: separate enable_local and pci_release_bridge_resources to
     other patches requested by Bjorn.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/pci/setup-bus.c | 62 ++++++++++++++++++++-----------------------------
 1 file changed, 25 insertions(+), 37 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index eddc6e1..19e619d 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1315,21 +1315,6 @@ static int __init pci_bus_get_depth(struct pci_bus *bus)
 
 	return depth;
 }
-static int __init pci_get_max_depth(void)
-{
-	int depth = 0;
-	struct pci_bus *bus;
-
-	list_for_each_entry(bus, &pci_root_buses, node) {
-		int ret;
-
-		ret = pci_bus_get_depth(bus);
-		if (ret > depth)
-			depth = ret;
-	}
-
-	return depth;
-}
 
 /*
  * -1: undefined, will auto detect later
@@ -1410,10 +1395,9 @@ static enum enable_type __init pci_realloc_detect(struct pci_bus *bus,
  * second  and later try will clear small leaf bridge res
  * will stop till to the max  deepth if can not find good one
  */
-void __init
-pci_assign_unassigned_resources(void)
+static void __init
+pci_assign_unassigned_root_bus_resources(struct pci_bus *bus)
 {
-	struct pci_bus *bus;
 	LIST_HEAD(realloc_head); /* list of resources that
 					want additional resources */
 	struct list_head *add_list = NULL;
@@ -1424,17 +1408,17 @@ pci_assign_unassigned_resources(void)
 	unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
 				  IORESOURCE_PREFETCH;
 	int pci_try_num = 1;
-	enum enable_type enable_local = pci_realloc_enable;
-
-	list_for_each_entry(bus, &pci_root_buses, node)
-		enable_local = pci_realloc_detect(bus, enable_local);
+	enum enable_type enable_local;
 
+	/* don't realloc if asked to do so */
+	enable_local = pci_realloc_detect(bus, pci_realloc_enable);
 	if (pci_realloc_enabled(enable_local)) {
-		int max_depth = pci_get_max_depth();
+		int max_depth = pci_bus_get_depth(bus);
 
 		pci_try_num = max_depth + 1;
-		printk(KERN_DEBUG "PCI: max bus depth: %d pci_try_num: %d\n",
-			 max_depth, pci_try_num);
+		dev_printk(KERN_DEBUG, &bus->dev,
+			   "max bus depth: %d pci_try_num: %d\n",
+			   max_depth, pci_try_num);
 	}
 
 again:
@@ -1446,12 +1430,10 @@ again:
 		add_list = &realloc_head;
 	/* Depth first, calculate sizes and alignments of all
 	   subordinate buses. */
-	list_for_each_entry(bus, &pci_root_buses, node)
-		__pci_bus_size_bridges(bus, add_list);
+	__pci_bus_size_bridges(bus, add_list);
 
 	/* Depth last, allocate resources and update the hardware. */
-	list_for_each_entry(bus, &pci_root_buses, node)
-		__pci_bus_assign_resources(bus, add_list, &fail_head);
+	__pci_bus_assign_resources(bus, add_list, &fail_head);
 	if (add_list)
 		BUG_ON(!list_empty(add_list));
 	tried_times++;
@@ -1462,16 +1444,16 @@ again:
 
 	if (tried_times >= pci_try_num) {
 		if (enable_local == undefined)
-			printk(KERN_INFO "Some PCI device resources are unassigned, try booting with pci=realloc\n");
+			dev_info(&bus->dev, "Some PCI device resources are unassigned, try booting with pci=realloc\n");
 		else if (enable_local == auto_enabled)
-			printk(KERN_INFO "Automatically enabled pci realloc, if you have problem, try booting with pci=realloc=off\n");
+			dev_info(&bus->dev, "Automatically enabled pci realloc, if you have problem, try booting with pci=realloc=off\n");
 
 		free_list(&fail_head);
 		goto enable_and_dump;
 	}
 
-	printk(KERN_DEBUG "PCI: No. %d try to assign unassigned res\n",
-			 tried_times + 1);
+	dev_printk(KERN_DEBUG, &bus->dev,
+		   "No. %d try to assign unassigned res\n", tried_times + 1);
 
 	/* third times and later will not check if it is leaf */
 	if ((tried_times + 1) > 2)
@@ -1502,12 +1484,18 @@ again:
 
 enable_and_dump:
 	/* Depth last, update the hardware. */
-	list_for_each_entry(bus, &pci_root_buses, node)
-		pci_enable_bridges(bus);
+	pci_enable_bridges(bus);
 
 	/* dump the resource on buses */
-	list_for_each_entry(bus, &pci_root_buses, node)
-		pci_bus_dump_resources(bus);
+	pci_bus_dump_resources(bus);
+}
+
+void __init pci_assign_unassigned_resources(void)
+{
+	struct pci_bus *root_bus;
+
+	list_for_each_entry(root_bus, &pci_root_buses, node)
+		pci_assign_unassigned_root_bus_resources(root_bus);
 }
 
 void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
-- 
1.8.1.4

[PATCH v6 6/7] PCI: Enable pci bridge when it is needed

[Yinghai Lu]

Current we enable bridges after bus scan and assign resources.
and it is spreaded a lot of places.

We can enable them later when their children pci device is enabled.
Need to go up to root bus and enable bridge one by one down to pci
device.

So that will delay enable bridge as needed bassis,
also kill one inconsistent between boot path and hot-add
path in acpi_pci_root_add().

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 arch/arm/kernel/bios32.c           |  5 -----
 arch/m68k/platform/coldfire/pci.c  |  1 -
 arch/mips/pci/pci.c                |  1 -
 arch/sh/drivers/pci/pci.c          |  1 -
 drivers/acpi/pci_root.c            |  3 ---
 drivers/parisc/lba_pci.c           |  1 -
 drivers/pci/bus.c                  | 19 -------------------
 drivers/pci/hotplug/acpiphp_glue.c |  1 -
 drivers/pci/pci.c                  | 20 ++++++++++++++++++++
 drivers/pci/probe.c                |  1 -
 drivers/pci/setup-bus.c            | 10 +++-------
 drivers/pcmcia/cardbus.c           |  1 -
 include/linux/pci.h                |  1 -
 13 files changed, 23 insertions(+), 42 deletions(-)

diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
index 261fcc8..88e14d7 100644
--- a/arch/arm/kernel/bios32.c
+++ b/arch/arm/kernel/bios32.c
@@ -525,11 +525,6 @@ void pci_common_init_dev(struct device *parent, struct hw_pci *hw)
 			 * Assign resources.
 			 */
 			pci_bus_assign_resources(bus);
-
-			/*
-			 * Enable bridges
-			 */
-			pci_enable_bridges(bus);
 		}
 
 		/*
diff --git a/arch/m68k/platform/coldfire/pci.c b/arch/m68k/platform/coldfire/pci.c
index b33f97a..df96792 100644
--- a/arch/m68k/platform/coldfire/pci.c
+++ b/arch/m68k/platform/coldfire/pci.c
@@ -319,7 +319,6 @@ static int __init mcf_pci_init(void)
 	pci_fixup_irqs(pci_common_swizzle, mcf_pci_map_irq);
 	pci_bus_size_bridges(rootbus);
 	pci_bus_assign_resources(rootbus);
-	pci_enable_bridges(rootbus);
 	return 0;
 }
 
diff --git a/arch/mips/pci/pci.c b/arch/mips/pci/pci.c
index 594e60d..33e7aa5 100644
--- a/arch/mips/pci/pci.c
+++ b/arch/mips/pci/pci.c
@@ -113,7 +113,6 @@ static void pcibios_scanbus(struct pci_controller *hose)
 		if (!pci_has_flag(PCI_PROBE_ONLY)) {
 			pci_bus_size_bridges(bus);
 			pci_bus_assign_resources(bus);
-			pci_enable_bridges(bus);
 		}
 	}
 }
diff --git a/arch/sh/drivers/pci/pci.c b/arch/sh/drivers/pci/pci.c
index 102f5d5..60ed3e1 100644
--- a/arch/sh/drivers/pci/pci.c
+++ b/arch/sh/drivers/pci/pci.c
@@ -69,7 +69,6 @@ static void pcibios_scanbus(struct pci_channel *hose)
 
 		pci_bus_size_bridges(bus);
 		pci_bus_assign_resources(bus);
-		pci_enable_bridges(bus);
 	} else {
 		pci_free_resource_list(&resources);
 	}
diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index 5917839..faa1d29 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -527,9 +527,6 @@ static int acpi_pci_root_add(struct acpi_device *device,
 	if (system_state != SYSTEM_BOOTING) {
 		pcibios_resource_survey_bus(root->bus);
 		pci_assign_unassigned_bus_resources(root->bus);
-
-		/* need to after hot-added ioapic is registered */
-		pci_enable_bridges(root->bus);
 	}
 
 	pci_bus_add_devices(root->bus);
diff --git a/drivers/parisc/lba_pci.c b/drivers/parisc/lba_pci.c
index 19f6f70..37e71ff 100644
--- a/drivers/parisc/lba_pci.c
+++ b/drivers/parisc/lba_pci.c
@@ -1590,7 +1590,6 @@ lba_driver_probe(struct parisc_device *dev)
 		lba_dump_res(&lba_dev->hba.lmmio_space, 2);
 #endif
 	}
-	pci_enable_bridges(lba_bus);
 
 	/*
 	** Once PCI register ops has walked the bus, access to config
diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index b1ff02a..fc1b740 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -216,24 +216,6 @@ void pci_bus_add_devices(const struct pci_bus *bus)
 	}
 }
 
-void pci_enable_bridges(struct pci_bus *bus)
-{
-	struct pci_dev *dev;
-	int retval;
-
-	list_for_each_entry(dev, &bus->devices, bus_list) {
-		if (dev->subordinate) {
-			if (!pci_is_enabled(dev)) {
-				retval = pci_enable_device(dev);
-				if (retval)
-					dev_err(&dev->dev, "Error enabling bridge (%d), continuing\n", retval);
-				pci_set_master(dev);
-			}
-			pci_enable_bridges(dev->subordinate);
-		}
-	}
-}
-
 /** pci_walk_bus - walk devices on/under bus, calling callback.
  *  @top      bus whose devices should be walked
  *  @cb       callback to be called for each device found
@@ -301,4 +283,3 @@ EXPORT_SYMBOL(pci_bus_put);
 EXPORT_SYMBOL(pci_bus_alloc_resource);
 EXPORT_SYMBOL_GPL(pci_bus_add_device);
 EXPORT_SYMBOL(pci_bus_add_devices);
-EXPORT_SYMBOL(pci_enable_bridges);
diff --git a/drivers/pci/hotplug/acpiphp_glue.c b/drivers/pci/hotplug/acpiphp_glue.c
index 59df857..52dee9d 100644
--- a/drivers/pci/hotplug/acpiphp_glue.c
+++ b/drivers/pci/hotplug/acpiphp_glue.c
@@ -723,7 +723,6 @@ static int __ref enable_device(struct acpiphp_slot *slot)
 	acpiphp_sanitize_bus(bus);
 	acpiphp_set_hpp_values(bus);
 	acpiphp_set_acpi_region(slot);
-	pci_enable_bridges(bus);
 
 	list_for_each_entry(dev, &bus->devices, bus_list) {
 		/* Assume that newly added devices are powered on already. */
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index e37fea6..44a1a8a 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -1145,6 +1145,24 @@ int pci_reenable_device(struct pci_dev *dev)
 	return 0;
 }
 
+static void pci_enable_bridge(struct pci_dev *dev)
+{
+	int retval;
+
+	if (!dev)
+		return;
+
+	pci_enable_bridge(dev->bus->self);
+
+	if (pci_is_enabled(dev))
+		return;
+	retval = pci_enable_device(dev);
+	if (retval)
+		dev_err(&dev->dev, "Error enabling bridge (%d), continuing\n",
+			retval);
+	pci_set_master(dev);
+}
+
 static int pci_enable_device_flags(struct pci_dev *dev, unsigned long flags)
 {
 	int err;
@@ -1165,6 +1183,8 @@ static int pci_enable_device_flags(struct pci_dev *dev, unsigned long flags)
 	if (atomic_inc_return(&dev->enable_cnt) > 1)
 		return 0;		/* already enabled */
 
+	pci_enable_bridge(dev->bus->self);
+
 	/* only skip sriov related */
 	for (i = 0; i <= PCI_ROM_RESOURCE; i++)
 		if (dev->resource[i].flags & flags)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 46ada5c..85c114c 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1979,7 +1979,6 @@ unsigned int __ref pci_rescan_bus(struct pci_bus *bus)
 
 	max = pci_scan_child_bus(bus);
 	pci_assign_unassigned_bus_resources(bus);
-	pci_enable_bridges(bus);
 	pci_bus_add_devices(bus);
 
 	return max;
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 19e619d..bd90cc1 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1440,7 +1440,7 @@ again:
 
 	/* any device complain? */
 	if (list_empty(&fail_head))
-		goto enable_and_dump;
+		goto dump;
 
 	if (tried_times >= pci_try_num) {
 		if (enable_local == undefined)
@@ -1449,7 +1449,7 @@ again:
 			dev_info(&bus->dev, "Automatically enabled pci realloc, if you have problem, try booting with pci=realloc=off\n");
 
 		free_list(&fail_head);
-		goto enable_and_dump;
+		goto dump;
 	}
 
 	dev_printk(KERN_DEBUG, &bus->dev,
@@ -1482,10 +1482,7 @@ again:
 
 	goto again;
 
-enable_and_dump:
-	/* Depth last, update the hardware. */
-	pci_enable_bridges(bus);
-
+dump:
 	/* dump the resource on buses */
 	pci_bus_dump_resources(bus);
 }
@@ -1558,7 +1555,6 @@ enable_all:
 	if (retval)
 		dev_err(&bridge->dev, "Error reenabling bridge (%d)\n", retval);
 	pci_set_master(bridge);
-	pci_enable_bridges(parent);
 }
 EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
 
diff --git a/drivers/pcmcia/cardbus.c b/drivers/pcmcia/cardbus.c
index 9d3ac99..b2a98cd 100644
--- a/drivers/pcmcia/cardbus.c
+++ b/drivers/pcmcia/cardbus.c
@@ -91,7 +91,6 @@ int __ref cb_alloc(struct pcmcia_socket *s)
 	if (s->tune_bridge)
 		s->tune_bridge(s, bus);
 
-	pci_enable_bridges(bus);
 	pci_bus_add_devices(bus);
 
 	return 0;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 0fd1f15..8cd1e6f 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1043,7 +1043,6 @@ int __must_check pci_bus_alloc_resource(struct pci_bus *bus,
 						  resource_size_t,
 						  resource_size_t),
 			void *alignf_data);
-void pci_enable_bridges(struct pci_bus *bus);
 
 /* Proper probing supporting hot-pluggable devices */
 int __must_check __pci_register_driver(struct pci_driver *, struct module *,
-- 
1.8.1.4

[PATCH v6 7/7] PCI: Retry assign unassigned resources for hotadd root bus

[Yinghai Lu]

Let root bus hotadd path use same code for booting path.
As driver is not loaded yet, we could retry to make sure
all pci devices get resources allocated.
We need this as during hotadd, firmware could assign some bars before
handle over.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
 drivers/acpi/pci_root.c |  2 +-
 drivers/pci/setup-bus.c | 15 +++++++--------
 include/linux/pci.h     |  1 +
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index faa1d29..ce04eb2 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -526,7 +526,7 @@ static int acpi_pci_root_add(struct acpi_device *device,
 
 	if (system_state != SYSTEM_BOOTING) {
 		pcibios_resource_survey_bus(root->bus);
-		pci_assign_unassigned_bus_resources(root->bus);
+		pci_assign_unassigned_root_bus_resources(root->bus);
 	}
 
 	pci_bus_add_devices(root->bus);
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index bd90cc1..4e60c44 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1297,7 +1297,7 @@ static void pci_bus_dump_resources(struct pci_bus *bus)
 	}
 }
 
-static int __init pci_bus_get_depth(struct pci_bus *bus)
+static int pci_bus_get_depth(struct pci_bus *bus)
 {
 	int depth = 0;
 	struct pci_dev *dev;
@@ -1331,7 +1331,7 @@ enum enable_type {
 	auto_enabled,
 };
 
-static enum enable_type pci_realloc_enable __initdata = undefined;
+static enum enable_type pci_realloc_enable = undefined;
 void __init pci_realloc_get_opt(char *str)
 {
 	if (!strncmp(str, "off", 3))
@@ -1339,13 +1339,13 @@ void __init pci_realloc_get_opt(char *str)
 	else if (!strncmp(str, "on", 2))
 		pci_realloc_enable = user_enabled;
 }
-static bool __init pci_realloc_enabled(enum enable_type enable)
+static bool pci_realloc_enabled(enum enable_type enable)
 {
 	return enable >= user_enabled;
 }
 
 #if defined(CONFIG_PCI_IOV) && defined(CONFIG_PCI_REALLOC_ENABLE_AUTO)
-static int __init check_unassigned_resources(struct pci_dev *dev, void *data)
+static int check_unassigned_resources(struct pci_dev *dev, void *data)
 {
 	int i;
 	int *unassigned = data;
@@ -1368,7 +1368,7 @@ static int __init check_unassigned_resources(struct pci_dev *dev, void *data)
 	return 0;
 }
 
-static enum enable_type __init pci_realloc_detect(struct pci_bus *bus,
+static enum enable_type pci_realloc_detect(struct pci_bus *bus,
 			 enum enable_type enable_local)
 {
 	int unassigned = 0;
@@ -1383,7 +1383,7 @@ static enum enable_type __init pci_realloc_detect(struct pci_bus *bus,
 	return enable_local;
 }
 #else
-static enum enable_type __init pci_realloc_detect(struct pci_bus *bus,
+static enum enable_type pci_realloc_detect(struct pci_bus *bus,
 			 enum enable_type enable_local)
 {
 	return enable_local;
@@ -1395,8 +1395,7 @@ static enum enable_type __init pci_realloc_detect(struct pci_bus *bus,
  * second  and later try will clear small leaf bridge res
  * will stop till to the max  deepth if can not find good one
  */
-static void __init
-pci_assign_unassigned_root_bus_resources(struct pci_bus *bus)
+void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus)
 {
 	LIST_HEAD(realloc_head); /* list of resources that
 					want additional resources */
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 8cd1e6f..e494c90 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1003,6 +1003,7 @@ int pci_claim_resource(struct pci_dev *, int);
 void pci_assign_unassigned_resources(void);
 void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge);
 void pci_assign_unassigned_bus_resources(struct pci_bus *bus);
+void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus);
 void pdev_enable_device(struct pci_dev *);
 int pci_enable_resources(struct pci_dev *, int mask);
 void pci_fixup_irqs(u8 (*)(struct pci_dev *, u8 *),
-- 
1.8.1.4

Re: [PATCH v6 0/7] PCI: Change assign unassigned resources per root bus bassis

[Bjorn Helgaas]

[+cc Rafael, linux-acpi]

On Mon, Jul 22, 2013 at 3:37 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> BenH reported that there is some assign unassigned resource problem
> in powerpc.
>
> It turns out after
> | commit 0c5be0cb0edfe3b5c4b62eac68aa2aa15ec681af
> | Date:   Thu Feb 23 19:23:29 2012 -0800
> |
> |    PCI: Retry on IORESOURCE_IO type allocations
>
> even the root bus does not have io port range, it will keep retrying
> to realloc with mmio.
>
> After checking the code, found that we bound io port and mmio fail
> path together.
> First patch fix the problem, that will not make mmio fall back to must-only
> when only have io port fail with must+optional.

Can you point out again which patch actually fixes Ben's problem?  I
don't see it mentioned in the changelogs, and I'd like to mention it
there.

I applied these to my pci/yinghai-assign-unassigned-v6 branch for
v3.12.  I expect to update these (at least the changelog), so this
branch will be rebased before being merged to -next.

Rafael, there are a couple changes to drivers/acpi/pci_root.c and
drivers/pci/hotplug/acpiphp_glue.c here, and I want to make sure
you're OK with them before going farther.

Here's the branch:
http://git.kernel.org/cgit/linux/kernel/git/helgaas/pci.git/log/?h=pci/yinghai-assign-unassigned-v6

Bjorn

> That is first patch.
>
> During we found the fix for that problem, found that we can separate assign
> unassigned resources to per root bus.
> that will make the code simple, also could reuse it for hotadd path.
>
> Besides that, "PCI: Enable pci bridge when it is needed" will delay pci
> bridge enabling, that will kill another difference between booting path
> and hotadd path.
>
> -v4: split first patch into 4 patches per Bjorn.
> -v5: drop two patches that will pass root bus resource mask after we found
>      simple and less intrusive way to fix the problem.
> -v6: refreshed on top of v3.11-rc2
>
> could get from
>         git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git for-pci-assign-unassigned
>
> Thanks
>
> Yinghai
>
> Yinghai Lu (7):
>   PCI: Don't use temp bus for pci_bus_release_bridge_resources
>   PCI: Use pci_walk_bus to detect unassigned resources
>   PCI: Check pci bus address for unassigned res
>   PCI: Introduce enable_local to prepare per root bus handling
>   PCI: Split pci_assign_unassigned_resources to per root bus
>   PCI: Enable pci bridge when it is needed
>   PCI: Retry assign unassigned resources for hotadd root bus
>
>  arch/arm/kernel/bios32.c           |   5 --
>  arch/m68k/platform/coldfire/pci.c  |   1 -
>  arch/mips/pci/pci.c                |   1 -
>  arch/sh/drivers/pci/pci.c          |   1 -
>  drivers/acpi/pci_root.c            |   5 +-
>  drivers/parisc/lba_pci.c           |   1 -
>  drivers/pci/bus.c                  |  19 -----
>  drivers/pci/hotplug/acpiphp_glue.c |   1 -
>  drivers/pci/pci.c                  |  20 ++++++
>  drivers/pci/probe.c                |   1 -
>  drivers/pci/setup-bus.c            | 141 +++++++++++++++++++------------------
>  drivers/pcmcia/cardbus.c           |   1 -
>  include/linux/pci.h                |   2 +-
>  13 files changed, 95 insertions(+), 104 deletions(-)
>
> --
> 1.8.1.4
>

Re: [PATCH v6 0/7] PCI: Change assign unassigned resources per root bus bassis

[Rafael J. Wysocki]

On Wednesday, July 24, 2013 04:03:48 PM Bjorn Helgaas wrote:
> [+cc Rafael, linux-acpi]
> 
> On Mon, Jul 22, 2013 at 3:37 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> > BenH reported that there is some assign unassigned resource problem
> > in powerpc.
> >
> > It turns out after
> > | commit 0c5be0cb0edfe3b5c4b62eac68aa2aa15ec681af
> > | Date:   Thu Feb 23 19:23:29 2012 -0800
> > |
> > |    PCI: Retry on IORESOURCE_IO type allocations
> >
> > even the root bus does not have io port range, it will keep retrying
> > to realloc with mmio.
> >
> > After checking the code, found that we bound io port and mmio fail
> > path together.
> > First patch fix the problem, that will not make mmio fall back to must-only
> > when only have io port fail with must+optional.
> 
> Can you point out again which patch actually fixes Ben's problem?  I
> don't see it mentioned in the changelogs, and I'd like to mention it
> there.
> 
> I applied these to my pci/yinghai-assign-unassigned-v6 branch for
> v3.12.  I expect to update these (at least the changelog), so this
> branch will be rebased before being merged to -next.
> 
> Rafael, there are a couple changes to drivers/acpi/pci_root.c and
> drivers/pci/hotplug/acpiphp_glue.c here, and I want to make sure
> you're OK with them before going farther.
> 
> Here's the branch:
> http://git.kernel.org/cgit/linux/kernel/git/helgaas/pci.git/log/?h=pci/yinghai-assign-unassigned-v6

I'm generally OK with them, as long as doing pci_bus_add_devices(bus) without
pci_enable_bridges() for that bus before is going to work.

Thanks,
Rafael


> > That is first patch.
> >
> > During we found the fix for that problem, found that we can separate assign
> > unassigned resources to per root bus.
> > that will make the code simple, also could reuse it for hotadd path.
> >
> > Besides that, "PCI: Enable pci bridge when it is needed" will delay pci
> > bridge enabling, that will kill another difference between booting path
> > and hotadd path.
> >
> > -v4: split first patch into 4 patches per Bjorn.
> > -v5: drop two patches that will pass root bus resource mask after we found
> >      simple and less intrusive way to fix the problem.
> > -v6: refreshed on top of v3.11-rc2
> >
> > could get from
> >         git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git for-pci-assign-unassigned
> >
> > Thanks
> >
> > Yinghai
> >
> > Yinghai Lu (7):
> >   PCI: Don't use temp bus for pci_bus_release_bridge_resources
> >   PCI: Use pci_walk_bus to detect unassigned resources
> >   PCI: Check pci bus address for unassigned res
> >   PCI: Introduce enable_local to prepare per root bus handling
> >   PCI: Split pci_assign_unassigned_resources to per root bus
> >   PCI: Enable pci bridge when it is needed
> >   PCI: Retry assign unassigned resources for hotadd root bus
> >
> >  arch/arm/kernel/bios32.c           |   5 --
> >  arch/m68k/platform/coldfire/pci.c  |   1 -
> >  arch/mips/pci/pci.c                |   1 -
> >  arch/sh/drivers/pci/pci.c          |   1 -
> >  drivers/acpi/pci_root.c            |   5 +-
> >  drivers/parisc/lba_pci.c           |   1 -
> >  drivers/pci/bus.c                  |  19 -----
> >  drivers/pci/hotplug/acpiphp_glue.c |   1 -
> >  drivers/pci/pci.c                  |  20 ++++++
> >  drivers/pci/probe.c                |   1 -
> >  drivers/pci/setup-bus.c            | 141 +++++++++++++++++++------------------
> >  drivers/pcmcia/cardbus.c           |   1 -
> >  include/linux/pci.h                |   2 +-
> >  13 files changed, 95 insertions(+), 104 deletions(-)
> >
> > --
> > 1.8.1.4
> >
-- 
I speak only for myself.
Rafael J. Wysocki, Intel Open Source Technology Center.

Re: [PATCH v6 0/7] PCI: Change assign unassigned resources per root bus bassis

[Yinghai Lu]

On Wed, Jul 24, 2013 at 3:03 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> [+cc Rafael, linux-acpi]
> Can you point out again which patch actually fixes Ben's problem?  I
> don't see it mentioned in the changelogs, and I'd like to mention it
> there.

oh, no.

I missed the to put that patch in patches/series.
that one should be really fix the problem for Ben.
others actually are clean up and enhancement when we try to find
solution for the problem.

just resent it
https://patchwork.kernel.org/patch/2833425/

Sorry for that.

You can put that patch in the branch, but we only
need that one for stable (3.10).

Thanks

Yinghai