[PATCH v7 00/10] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[PATCH v7 00/10] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Jiang Liu]

This patchset enhance pci_root driver to update MMCFG information when
hot-plugging PCI root bridges. It applies to 
git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git pci/next-3.5

-v2: split into smaller patches and skip updating MMCFG information when
     MMCFG is disabled
-v3: add mmconf_added to simply free path, also make pci_mmconfig_insert()
     to process extra exist case --- By Yinghai
-v4: tune arch_acpi_pci_root_add() to handle a corner case raised by Kenji
-v5: address review comments from Bjorn and Taku, also better handle corner
     cases in arch_acpi_pci_root_add()
-v6: get rid of arch_acpi_pci_root_xxx() by using existing hooks
     add MCFG information for host bridges on demand
     more corner cases clear up
     correctly handle condition compilation
     fix section mismatch issues
     fix a issue reported by Taku about a BIOS bug
-v7: unify log messages
     remove redundant host bridge resource related log messages
     fix a issue reported by Taku which breaks pnp resource allocation

Jiang Liu (10):
  PCI, x86: split out pci_mmcfg_check_reserved() for code reuse
  PCI, x86: split out pci_mmconfig_alloc() for code reuse
  PCI, x86: use RCU list to protect mmconfig list
  PCI, x86: introduce pci_mmcfg_arch_map()/pci_mmcfg_arch_unmap()
  PCI, x86: introduce pci_mmconfig_insert()/delete() for PCI root
    bridge hotplug
  PCI, ACPI: provide MCFG address for PCI host bridges
  PCI, x86: update MMCFG information when hot-plugging PCI host bridges
  PCI, x86: add MMCFG information on demand
  PCI, x86: simplify pci_mmcfg_late_insert_resources()
  PCI, x86: get rid of redundant log messages

 arch/x86/include/asm/pci_x86.h |   12 ++
 arch/x86/pci/acpi.c            |   76 ++++++++-
 arch/x86/pci/legacy.c          |    1 +
 arch/x86/pci/mmconfig-shared.c |  364 +++++++++++++++++++++++++++++-----------
 arch/x86/pci/mmconfig_32.c     |   30 +++-
 arch/x86/pci/mmconfig_64.c     |   37 ++++-
 drivers/acpi/pci_root.c        |   12 ++
 drivers/pci/pci-acpi.c         |   34 ++++
 include/acpi/acnames.h         |    1 +
 include/acpi/acpi_bus.h        |    3 +
 include/linux/pci-acpi.h       |    5 +
 11 files changed, 465 insertions(+), 110 deletions(-)

[PATCH v7 01/10] PCI, x86: split out pci_mmcfg_check_reserved() for code reuse

[Jiang Liu]

Split out pci_mmcfg_check_reserved() for code reuse, which will be used
when supporting PCI host bridge hotplug.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/pci/mmconfig-shared.c |   51 +++++++++++++++++++--------------------
 1 files changed, 25 insertions(+), 26 deletions(-)

diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 301e325..f799949 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -474,39 +474,38 @@ static int __init is_mmconf_reserved(check_reserved_t is_reserved,
 	return valid;
 }
 
+static int __devinit pci_mmcfg_check_reserved(struct pci_mmcfg_region *cfg,
+					      int early)
+{
+	if (!early && !acpi_disabled) {
+		if (is_mmconf_reserved(is_acpi_reserved, cfg, 0))
+			return 1;
+		else
+			printk(KERN_ERR FW_BUG PREFIX
+			       "MMCONFIG at %pR not reserved in "
+			       "ACPI motherboard resources\n",
+			       &cfg->res);
+	}
+
+	/* Don't try to do this check unless configuration
+	   type 1 is available. how about type 2 ?*/
+	if (raw_pci_ops)
+		return is_mmconf_reserved(e820_all_mapped, cfg, 1);
+
+	return 0;
+}
+
 static void __init pci_mmcfg_reject_broken(int early)
 {
 	struct pci_mmcfg_region *cfg;
 
 	list_for_each_entry(cfg, &pci_mmcfg_list, list) {
-		int valid = 0;
-
-		if (!early && !acpi_disabled) {
-			valid = is_mmconf_reserved(is_acpi_reserved, cfg, 0);
-
-			if (valid)
-				continue;
-			else
-				printk(KERN_ERR FW_BUG PREFIX
-				       "MMCONFIG at %pR not reserved in "
-				       "ACPI motherboard resources\n",
-				       &cfg->res);
+		if (pci_mmcfg_check_reserved(cfg, early) == 0) {
+			printk(KERN_INFO PREFIX "not using MMCONFIG\n");
+			free_all_mmcfg();
+			return;
 		}
-
-		/* Don't try to do this check unless configuration
-		   type 1 is available. how about type 2 ?*/
-		if (raw_pci_ops)
-			valid = is_mmconf_reserved(e820_all_mapped, cfg, 1);
-
-		if (!valid)
-			goto reject;
 	}
-
-	return;
-
-reject:
-	printk(KERN_INFO PREFIX "not using MMCONFIG\n");
-	free_all_mmcfg();
 }
 
 static int __initdata known_bridge;
-- 
1.7.1

[PATCH v7 02/10] PCI, x86: split out pci_mmconfig_alloc() for code reuse

[Jiang Liu]

Split out pci_mmconfig_alloc() for code reuse, which will be used
when supporting PCI root bridge hotplug.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/pci/mmconfig-shared.c |   19 +++++++++++++++----
 1 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index f799949..5e2cd2a 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -61,8 +61,9 @@ static __init void list_add_sorted(struct pci_mmcfg_region *new)
 	list_add_tail(&new->list, &pci_mmcfg_list);
 }
 
-static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
-							int end, u64 addr)
+static __devinit struct pci_mmcfg_region *pci_mmconfig_alloc(int segment,
+							     int start,
+							     int end, u64 addr)
 {
 	struct pci_mmcfg_region *new;
 	struct resource *res;
@@ -79,8 +80,6 @@ static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
 	new->start_bus = start;
 	new->end_bus = end;
 
-	list_add_sorted(new);
-
 	res = &new->res;
 	res->start = addr + PCI_MMCFG_BUS_OFFSET(start);
 	res->end = addr + PCI_MMCFG_BUS_OFFSET(end + 1) - 1;
@@ -96,6 +95,18 @@ static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
 	return new;
 }
 
+static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
+							int end, u64 addr)
+{
+	struct pci_mmcfg_region *new;
+
+	new = pci_mmconfig_alloc(segment, start, end, addr);
+	if (new)
+		list_add_sorted(new);
+
+	return new;
+}
+
 struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus)
 {
 	struct pci_mmcfg_region *cfg;
-- 
1.7.1

[PATCH v7 03/10] PCI, x86: use RCU list to protect mmconfig list

[Jiang Liu]

Use RCU list to protect mmconfig list from dynamic change
when supporting PCI host bridge hotplug.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/pci/mmconfig-shared.c |   18 ++++++++++++------
 arch/x86/pci/mmconfig_32.c     |   13 +++++++++++--
 arch/x86/pci/mmconfig_64.c     |   13 +++++++++++--
 3 files changed, 34 insertions(+), 10 deletions(-)

diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 5e2cd2a..0ac97d5 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -17,6 +17,8 @@
 #include <linux/bitmap.h>
 #include <linux/dmi.h>
 #include <linux/slab.h>
+#include <linux/mutex.h>
+#include <linux/rculist.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
 #include <asm/acpi.h>
@@ -25,6 +27,7 @@
 
 /* Indicate if the mmcfg resources have been placed into the resource table. */
 static int __initdata pci_mmcfg_resources_inserted;
+static DEFINE_MUTEX(pci_mmcfg_lock);
 
 LIST_HEAD(pci_mmcfg_list);
 
@@ -45,20 +48,20 @@ static __init void free_all_mmcfg(void)
 		pci_mmconfig_remove(cfg);
 }
 
-static __init void list_add_sorted(struct pci_mmcfg_region *new)
+static __devinit void list_add_sorted(struct pci_mmcfg_region *new)
 {
 	struct pci_mmcfg_region *cfg;
 
 	/* keep list sorted by segment and starting bus number */
-	list_for_each_entry(cfg, &pci_mmcfg_list, list) {
+	list_for_each_entry_rcu(cfg, &pci_mmcfg_list, list) {
 		if (cfg->segment > new->segment ||
 		    (cfg->segment == new->segment &&
 		     cfg->start_bus >= new->start_bus)) {
-			list_add_tail(&new->list, &cfg->list);
+			list_add_tail_rcu(&new->list, &cfg->list);
 			return;
 		}
 	}
-	list_add_tail(&new->list, &pci_mmcfg_list);
+	list_add_tail_rcu(&new->list, &pci_mmcfg_list);
 }
 
 static __devinit struct pci_mmcfg_region *pci_mmconfig_alloc(int segment,
@@ -101,8 +104,11 @@ static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
 	struct pci_mmcfg_region *new;
 
 	new = pci_mmconfig_alloc(segment, start, end, addr);
-	if (new)
+	if (new) {
+		mutex_lock(&pci_mmcfg_lock);
 		list_add_sorted(new);
+		mutex_unlock(&pci_mmcfg_lock);
+	}
 
 	return new;
 }
@@ -111,7 +117,7 @@ struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus)
 {
 	struct pci_mmcfg_region *cfg;
 
-	list_for_each_entry(cfg, &pci_mmcfg_list, list)
+	list_for_each_entry_rcu(cfg, &pci_mmcfg_list, list)
 		if (cfg->segment == segment &&
 		    cfg->start_bus <= bus && bus <= cfg->end_bus)
 			return cfg;
diff --git a/arch/x86/pci/mmconfig_32.c b/arch/x86/pci/mmconfig_32.c
index 5372e86..5dad04a 100644
--- a/arch/x86/pci/mmconfig_32.c
+++ b/arch/x86/pci/mmconfig_32.c
@@ -11,6 +11,7 @@
 
 #include <linux/pci.h>
 #include <linux/init.h>
+#include <linux/rcupdate.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
 #include <acpi/acpi.h>
@@ -60,9 +61,12 @@ err:		*value = -1;
 		return -EINVAL;
 	}
 
+	rcu_read_lock();
 	base = get_base_addr(seg, bus, devfn);
-	if (!base)
+	if (!base) {
+		rcu_read_unlock();
 		goto err;
+	}
 
 	raw_spin_lock_irqsave(&pci_config_lock, flags);
 
@@ -80,6 +84,7 @@ err:		*value = -1;
 		break;
 	}
 	raw_spin_unlock_irqrestore(&pci_config_lock, flags);
+	rcu_read_unlock();
 
 	return 0;
 }
@@ -93,9 +98,12 @@ static int pci_mmcfg_write(unsigned int seg, unsigned int bus,
 	if ((bus > 255) || (devfn > 255) || (reg > 4095))
 		return -EINVAL;
 
+	rcu_read_lock();
 	base = get_base_addr(seg, bus, devfn);
-	if (!base)
+	if (!base) {
+		rcu_read_unlock();
 		return -EINVAL;
+	}
 
 	raw_spin_lock_irqsave(&pci_config_lock, flags);
 
@@ -113,6 +121,7 @@ static int pci_mmcfg_write(unsigned int seg, unsigned int bus,
 		break;
 	}
 	raw_spin_unlock_irqrestore(&pci_config_lock, flags);
+	rcu_read_unlock();
 
 	return 0;
 }
diff --git a/arch/x86/pci/mmconfig_64.c b/arch/x86/pci/mmconfig_64.c
index 915a493..acc48c5 100644
--- a/arch/x86/pci/mmconfig_64.c
+++ b/arch/x86/pci/mmconfig_64.c
@@ -9,6 +9,7 @@
 #include <linux/init.h>
 #include <linux/acpi.h>
 #include <linux/bitmap.h>
+#include <linux/rcupdate.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
 
@@ -34,9 +35,12 @@ err:		*value = -1;
 		return -EINVAL;
 	}
 
+	rcu_read_lock();
 	addr = pci_dev_base(seg, bus, devfn);
-	if (!addr)
+	if (!addr) {
+		rcu_read_unlock();
 		goto err;
+	}
 
 	switch (len) {
 	case 1:
@@ -49,6 +53,7 @@ err:		*value = -1;
 		*value = mmio_config_readl(addr + reg);
 		break;
 	}
+	rcu_read_unlock();
 
 	return 0;
 }
@@ -62,9 +67,12 @@ static int pci_mmcfg_write(unsigned int seg, unsigned int bus,
 	if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095)))
 		return -EINVAL;
 
+	rcu_read_lock();
 	addr = pci_dev_base(seg, bus, devfn);
-	if (!addr)
+	if (!addr) {
+		rcu_read_unlock();
 		return -EINVAL;
+	}
 
 	switch (len) {
 	case 1:
@@ -77,6 +85,7 @@ static int pci_mmcfg_write(unsigned int seg, unsigned int bus,
 		mmio_config_writel(addr + reg, value);
 		break;
 	}
+	rcu_read_unlock();
 
 	return 0;
 }
-- 
1.7.1

[PATCH v7 04/10] PCI, x86: introduce pci_mmcfg_arch_map()/pci_mmcfg_arch_unmap()

[Jiang Liu]

Introduce pci_mmcfg_arch_map()/pci_mmcfg_arch_unmap(), which will be used
when supporting PCI root bridge hotplug.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/include/asm/pci_x86.h |    2 ++
 arch/x86/pci/mmconfig_32.c     |   15 +++++++++++++++
 arch/x86/pci/mmconfig_64.c     |   22 +++++++++++++++++++++-
 3 files changed, 38 insertions(+), 1 deletions(-)

diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index b3a5317..df898ce 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -135,6 +135,8 @@ struct pci_mmcfg_region {
 
 extern int __init pci_mmcfg_arch_init(void);
 extern void __init pci_mmcfg_arch_free(void);
+extern int __devinit pci_mmcfg_arch_map(struct pci_mmcfg_region *cfg);
+extern void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg);
 extern struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus);
 
 extern struct list_head pci_mmcfg_list;
diff --git a/arch/x86/pci/mmconfig_32.c b/arch/x86/pci/mmconfig_32.c
index 5dad04a..a22785d 100644
--- a/arch/x86/pci/mmconfig_32.c
+++ b/arch/x86/pci/mmconfig_32.c
@@ -141,3 +141,18 @@ int __init pci_mmcfg_arch_init(void)
 void __init pci_mmcfg_arch_free(void)
 {
 }
+
+int __devinit pci_mmcfg_arch_map(struct pci_mmcfg_region *cfg)
+{
+	return 0;
+}
+
+void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg)
+{
+	unsigned long flags;
+
+	/* Invalidate the cached mmcfg map entry. */
+	raw_spin_lock_irqsave(&pci_config_lock, flags);
+	mmcfg_last_accessed_device = 0;
+	raw_spin_unlock_irqrestore(&pci_config_lock, flags);
+}
diff --git a/arch/x86/pci/mmconfig_64.c b/arch/x86/pci/mmconfig_64.c
index acc48c5..4e05779 100644
--- a/arch/x86/pci/mmconfig_64.c
+++ b/arch/x86/pci/mmconfig_64.c
@@ -95,7 +95,7 @@ static const struct pci_raw_ops pci_mmcfg = {
 	.write =	pci_mmcfg_write,
 };
 
-static void __iomem * __init mcfg_ioremap(struct pci_mmcfg_region *cfg)
+static void __iomem * __devinit mcfg_ioremap(struct pci_mmcfg_region *cfg)
 {
 	void __iomem *addr;
 	u64 start, size;
@@ -138,3 +138,23 @@ void __init pci_mmcfg_arch_free(void)
 		}
 	}
 }
+
+int __devinit pci_mmcfg_arch_map(struct pci_mmcfg_region *cfg)
+{
+	cfg->virt = mcfg_ioremap(cfg);
+	if (!cfg->virt) {
+		printk(KERN_ERR PREFIX "can't map MMCONFIG at %pR\n",
+		       &cfg->res);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg)
+{
+	if (cfg && cfg->virt) {
+		iounmap(cfg->virt + PCI_MMCFG_BUS_OFFSET(cfg->start_bus));
+		cfg->virt = NULL;
+	}
+}
-- 
1.7.1

[PATCH v7 05/10] PCI, x86: introduce pci_mmconfig_insert()/delete() for PCI root bridge hotplug

[Jiang Liu]

Introduce pci_mmconfig_insert()/pci_mmconfig_delete(), which will be used to
update MMCFG information when supporting PCI root bridge hotplug.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/include/asm/pci_x86.h |    4 +
 arch/x86/pci/mmconfig-shared.c |  213 ++++++++++++++++++++++++++++++++--------
 2 files changed, 177 insertions(+), 40 deletions(-)

diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index df898ce..b2fb1d8 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -137,6 +137,10 @@ extern int __init pci_mmcfg_arch_init(void);
 extern void __init pci_mmcfg_arch_free(void);
 extern int __devinit pci_mmcfg_arch_map(struct pci_mmcfg_region *cfg);
 extern void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg);
+extern int __devinit pci_mmconfig_insert(struct device *dev,
+					 uint16_t seg, uint8_t start,
+					 uint8_t end, phys_addr_t addr);
+extern int pci_mmconfig_delete(uint16_t seg, uint8_t start, uint8_t end);
 extern struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus);
 
 extern struct list_head pci_mmcfg_list;
diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 0ac97d5..993e766 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -27,6 +27,8 @@
 
 /* Indicate if the mmcfg resources have been placed into the resource table. */
 static int __initdata pci_mmcfg_resources_inserted;
+static bool pci_mmcfg_running_state;
+static bool pci_mmcfg_arch_init_failed;
 static DEFINE_MUTEX(pci_mmcfg_lock);
 
 LIST_HEAD(pci_mmcfg_list);
@@ -91,10 +93,6 @@ static __devinit struct pci_mmcfg_region *pci_mmconfig_alloc(int segment,
 		 "PCI MMCONFIG %04x [bus %02x-%02x]", segment, start, end);
 	res->name = new->name;
 
-	printk(KERN_INFO PREFIX "MMCONFIG for domain %04x [bus %02x-%02x] at "
-	       "%pR (base %#lx)\n", segment, start, end, &new->res,
-	       (unsigned long) addr);
-
 	return new;
 }
 
@@ -108,6 +106,11 @@ static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
 		mutex_lock(&pci_mmcfg_lock);
 		list_add_sorted(new);
 		mutex_unlock(&pci_mmcfg_lock);
+
+		printk(KERN_INFO PREFIX
+		       "MMCONFIG entry for domain %04x [bus %02x-%02x] at %pR "
+		       "(base %#lx)\n",
+		       segment, start, end, &new->res, (unsigned long)addr);
 	}
 
 	return new;
@@ -375,14 +378,15 @@ static void __init pci_mmcfg_insert_resources(void)
 	struct pci_mmcfg_region *cfg;
 
 	list_for_each_entry(cfg, &pci_mmcfg_list, list)
-		insert_resource(&iomem_resource, &cfg->res);
+		if (!cfg->res.parent)
+			insert_resource(&iomem_resource, &cfg->res);
 
 	/* Mark that the resources have been inserted. */
 	pci_mmcfg_resources_inserted = 1;
 }
 
-static acpi_status __init check_mcfg_resource(struct acpi_resource *res,
-					      void *data)
+static acpi_status __devinit check_mcfg_resource(struct acpi_resource *res,
+						 void *data)
 {
 	struct resource *mcfg_res = data;
 	struct acpi_resource_address64 address;
@@ -418,8 +422,8 @@ static acpi_status __init check_mcfg_resource(struct acpi_resource *res,
 	return AE_OK;
 }
 
-static acpi_status __init find_mboard_resource(acpi_handle handle, u32 lvl,
-		void *context, void **rv)
+static acpi_status __devinit find_mboard_resource(acpi_handle handle, u32 lvl,
+						  void *context, void **rv)
 {
 	struct resource *mcfg_res = context;
 
@@ -432,7 +436,7 @@ static acpi_status __init find_mboard_resource(acpi_handle handle, u32 lvl,
 	return AE_OK;
 }
 
-static int __init is_acpi_reserved(u64 start, u64 end, unsigned not_used)
+static int __devinit is_acpi_reserved(u64 start, u64 end, unsigned not_used)
 {
 	struct resource mcfg_res;
 
@@ -451,13 +455,15 @@ static int __init is_acpi_reserved(u64 start, u64 end, unsigned not_used)
 
 typedef int (*check_reserved_t)(u64 start, u64 end, unsigned type);
 
-static int __init is_mmconf_reserved(check_reserved_t is_reserved,
-				    struct pci_mmcfg_region *cfg, int with_e820)
+static int __devinit is_mmconf_reserved(check_reserved_t is_reserved,
+					struct pci_mmcfg_region *cfg,
+					struct device *dev, int with_e820)
 {
 	u64 addr = cfg->res.start;
 	u64 size = resource_size(&cfg->res);
 	u64 old_size = size;
-	int valid = 0, num_buses;
+	int num_buses;
+	char *method = with_e820 ? "E820" : "ACPI motherboard resources";
 
 	while (!is_reserved(addr, addr + size, E820_RESERVED)) {
 		size >>= 1;
@@ -465,49 +471,75 @@ static int __init is_mmconf_reserved(check_reserved_t is_reserved,
 			break;
 	}
 
-	if (size >= (16UL<<20) || size == old_size) {
-		printk(KERN_INFO PREFIX "MMCONFIG at %pR reserved in %s\n",
-		       &cfg->res,
-		       with_e820 ? "E820" : "ACPI motherboard resources");
-		valid = 1;
-
-		if (old_size != size) {
-			/* update end_bus */
-			cfg->end_bus = cfg->start_bus + ((size>>20) - 1);
-			num_buses = cfg->end_bus - cfg->start_bus + 1;
-			cfg->res.end = cfg->res.start +
-			    PCI_MMCFG_BUS_OFFSET(num_buses) - 1;
-			snprintf(cfg->name, PCI_MMCFG_RESOURCE_NAME_LEN,
-				 "PCI MMCONFIG %04x [bus %02x-%02x]",
-				 cfg->segment, cfg->start_bus, cfg->end_bus);
+	if (size < (16UL<<20) && size != old_size)
+		return 0;
+
+	if (dev)
+		dev_info(dev, "MMCONFIG entry at %pR reserved in %s\n",
+			 &cfg->res, method);
+	else
+		printk(KERN_INFO PREFIX
+		       "MMCONFIG entry at %pR reserved in %s\n",
+		       &cfg->res, method);
+
+	if (old_size != size) {
+		/* update end_bus */
+		cfg->end_bus = cfg->start_bus + ((size>>20) - 1);
+		num_buses = cfg->end_bus - cfg->start_bus + 1;
+		cfg->res.end = cfg->res.start +
+		    PCI_MMCFG_BUS_OFFSET(num_buses) - 1;
+		snprintf(cfg->name, PCI_MMCFG_RESOURCE_NAME_LEN,
+			 "PCI MMCONFIG %04x [bus %02x-%02x]",
+			 cfg->segment, cfg->start_bus, cfg->end_bus);
+
+		if (dev)
+			dev_info(dev,
+				"MMCONFIG entry "
+				"at %pR (base %#lx) (size reduced!)\n",
+				&cfg->res, (unsigned long) cfg->address);
+		else
 			printk(KERN_INFO PREFIX
-			       "MMCONFIG for %04x [bus%02x-%02x] "
-			       "at %pR (base %#lx) (size reduced!)\n",
-			       cfg->segment, cfg->start_bus, cfg->end_bus,
-			       &cfg->res, (unsigned long) cfg->address);
-		}
+				"MMCONFIG entry for %04x [bus%02x-%02x] "
+				"at %pR (base %#lx) (size reduced!)\n",
+				cfg->segment, cfg->start_bus, cfg->end_bus,
+				&cfg->res, (unsigned long) cfg->address);
 	}
 
-	return valid;
+	return 1;
 }
 
-static int __devinit pci_mmcfg_check_reserved(struct pci_mmcfg_region *cfg,
-					      int early)
+static int __ref pci_mmcfg_check_reserved(struct device *dev,
+		  struct pci_mmcfg_region *cfg, int early)
 {
 	if (!early && !acpi_disabled) {
-		if (is_mmconf_reserved(is_acpi_reserved, cfg, 0))
+		if (is_mmconf_reserved(is_acpi_reserved, cfg, dev, 0))
 			return 1;
+
+		if (dev)
+			dev_err(dev, FW_BUG
+				"MMCONFIG entry at %pR not reserved in "
+				"ACPI motherboard resources\n",
+				&cfg->res);
 		else
 			printk(KERN_ERR FW_BUG PREFIX
-			       "MMCONFIG at %pR not reserved in "
+			       "MMCONFIG entry at %pR not reserved in "
 			       "ACPI motherboard resources\n",
 			       &cfg->res);
 	}
 
+	/*
+	 * e820_all_mapped() is marked as __init.
+	 * All entries from ACPI MCFG table have been checked at boot time.
+	 * For MCFG information constructed from hotpluggable host bridge's
+	 * _CBA method, just assume it's reserved.
+	 */
+	if (pci_mmcfg_running_state)
+		return 1;
+
 	/* Don't try to do this check unless configuration
 	   type 1 is available. how about type 2 ?*/
 	if (raw_pci_ops)
-		return is_mmconf_reserved(e820_all_mapped, cfg, 1);
+		return is_mmconf_reserved(e820_all_mapped, cfg, dev, 1);
 
 	return 0;
 }
@@ -517,7 +549,7 @@ static void __init pci_mmcfg_reject_broken(int early)
 	struct pci_mmcfg_region *cfg;
 
 	list_for_each_entry(cfg, &pci_mmcfg_list, list) {
-		if (pci_mmcfg_check_reserved(cfg, early) == 0) {
+		if (pci_mmcfg_check_reserved(NULL, cfg, early) == 0) {
 			printk(KERN_INFO PREFIX "not using MMCONFIG\n");
 			free_all_mmcfg();
 			return;
@@ -641,6 +673,7 @@ static void __init __pci_mmcfg_init(int early)
 		 * the architecture mmcfg setup could not initialize.
 		 */
 		pci_mmcfg_resources_inserted = 1;
+		pci_mmcfg_arch_init_failed = true;
 	}
 }
 
@@ -656,6 +689,8 @@ void __init pci_mmcfg_late_init(void)
 
 static int __init pci_mmcfg_late_insert_resources(void)
 {
+	pci_mmcfg_running_state = true;
+
 	/*
 	 * If resources are already inserted or we are not using MMCONFIG,
 	 * don't insert the resources.
@@ -681,3 +716,101 @@ static int __init pci_mmcfg_late_insert_resources(void)
  * with other system resources.
  */
 late_initcall(pci_mmcfg_late_insert_resources);
+
+/* Add MMCFG information for host bridges */
+int __devinit pci_mmconfig_insert(struct device *dev,
+				  uint16_t seg, uint8_t start, uint8_t end,
+				  phys_addr_t addr)
+{
+	int rc;
+	struct resource *tmp = NULL;
+	struct pci_mmcfg_region *cfg;
+
+	if (!(pci_probe & PCI_PROBE_MMCONF) || pci_mmcfg_arch_init_failed)
+		return -ENODEV;
+
+	if (start > end)
+		return -EINVAL;
+
+	mutex_lock(&pci_mmcfg_lock);
+	cfg = pci_mmconfig_lookup(seg, start);
+	if (cfg) {
+		if (cfg->end_bus < end)
+			dev_warn(dev, FW_BUG
+				 "MMCONFIG entry for "
+				 "domain %04x [bus %02x-%02x] "
+				 "only partially covers this bridge\n",
+				  cfg->segment, cfg->start_bus, cfg->end_bus);
+		mutex_unlock(&pci_mmcfg_lock);
+		return -EEXIST;
+	}
+
+	if (!addr) {
+		mutex_unlock(&pci_mmcfg_lock);
+		return -EINVAL;
+	}
+
+	rc = -EBUSY;
+	cfg = pci_mmconfig_alloc(seg, start, end, addr);
+	if (cfg == NULL) {
+		dev_warn(dev, "fail to add MMCONFIG entry (out of memory)\n");
+		rc = -ENOMEM;
+	} else if (!pci_mmcfg_check_reserved(dev, cfg, 0)) {
+		dev_warn(dev, FW_BUG "MMCONFIG resource %pR isn't reserved\n",
+			 &cfg->res);
+	} else {
+		/* Insert resource if it's not in boot stage */
+		if (pci_mmcfg_running_state)
+			tmp = insert_resource_conflict(&iomem_resource,
+						       &cfg->res);
+
+		if (tmp) {
+			dev_warn(dev,
+				 "MMCONFIG resource %pR conflicts with "
+				 "resource %s %pR\n",
+				 &cfg->res, tmp->name, tmp);
+		} else if (pci_mmcfg_arch_map(cfg)) {
+			dev_warn(dev, "fail to map MMCONFIG resource %pR.\n",
+				 &cfg->res);
+		} else {
+			list_add_sorted(cfg);
+			dev_info(dev, "MMCONFIG entry at %pR (base %#lx)\n",
+				 &cfg->res, (unsigned long)addr);
+			cfg = NULL;
+			rc = 0;
+		}
+	}
+
+	if (cfg) {
+		if (cfg->res.parent)
+			release_resource(&cfg->res);
+		kfree(cfg);
+	}
+
+	mutex_unlock(&pci_mmcfg_lock);
+
+	return rc;
+}
+
+/* Delete MMCFG information for host bridges */
+int pci_mmconfig_delete(uint16_t seg, uint8_t start, uint8_t end)
+{
+	struct pci_mmcfg_region *cfg;
+
+	mutex_lock(&pci_mmcfg_lock);
+	list_for_each_entry_rcu(cfg, &pci_mmcfg_list, list)
+		if (cfg->segment == seg && cfg->start_bus == start &&
+		    cfg->end_bus == end) {
+			list_del_rcu(&cfg->list);
+			synchronize_rcu();
+			pci_mmcfg_arch_unmap(cfg);
+			if (cfg->res.parent)
+				release_resource(&cfg->res);
+			mutex_unlock(&pci_mmcfg_lock);
+			kfree(cfg);
+			return 0;
+		}
+	mutex_unlock(&pci_mmcfg_lock);
+
+	return -ENOENT;
+}
-- 
1.7.1

[PATCH v7 06/10] PCI, ACPI: provide MCFG address for PCI host bridges

[Jiang Liu]

This patch provide MCFG address for PCI host bridges, which will
be used to support host bridge hotplug. It gets MCFG address
by evaluating _CBA method if available, or by scanning the ACPI
MCFG table.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 drivers/acpi/pci_root.c  |   12 ++++++++++++
 drivers/pci/pci-acpi.c   |   34 ++++++++++++++++++++++++++++++++++
 include/acpi/acnames.h   |    1 +
 include/acpi/acpi_bus.h  |    3 +++
 include/linux/pci-acpi.h |    5 +++++
 5 files changed, 55 insertions(+), 0 deletions(-)

diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c
index 7aff631..fc716cf 100644
--- a/drivers/acpi/pci_root.c
+++ b/drivers/acpi/pci_root.c
@@ -458,6 +458,7 @@ static int __devinit acpi_pci_root_add(struct acpi_device *device)
 	acpi_handle handle;
 	struct acpi_device *child;
 	u32 flags, base_flags;
+	int end_bus = -1;
 
 	root = kzalloc(sizeof(struct acpi_pci_root), GFP_KERNEL);
 	if (!root)
@@ -505,6 +506,17 @@ static int __devinit acpi_pci_root_add(struct acpi_device *device)
 	strcpy(acpi_device_class(device), ACPI_PCI_ROOT_CLASS);
 	device->driver_data = root;
 
+	root->mcfg_addr = acpi_pci_root_get_mcfg_addr(device->handle,
+		root->segment, (uint8_t)root->secondary.start, &end_bus);
+
+	/*
+	 * End bus number for MCFG may be less than root's subordinary
+	 * bus number with buggy BIOS implementation.
+	 */
+	if (end_bus < 0 || end_bus > root->secondary.end)
+		end_bus = root->secondary.end;
+	root->mcfg_end_bus = (uint8_t)end_bus;
+
 	/*
 	 * All supported architectures that use ACPI have support for
 	 * PCI domains, so we indicate this in _OSC support capabilities.
diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
index 0f150f2..9af71d2 100644
--- a/drivers/pci/pci-acpi.c
+++ b/drivers/pci/pci-acpi.c
@@ -162,6 +162,40 @@ acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
 	return remove_pm_notifier(dev, pci_acpi_wake_dev);
 }
 
+/* acpi_table_parse() is marked as __init, so cache MCFG info at boot time */
+int pci_acpi_mcfg_entries;
+struct acpi_mcfg_allocation *pci_acpi_mcfg_array;
+
+phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle, uint16_t seg,
+					uint8_t start, int *endp)
+{
+	int i, end_bus = -1;
+	acpi_status status = AE_NOT_EXIST;
+	unsigned long long mcfg_addr = 0;
+	struct acpi_mcfg_allocation *cfg;
+
+	if (handle)
+		status = acpi_evaluate_integer(handle, METHOD_NAME__CBA,
+					       NULL, &mcfg_addr);
+	if (ACPI_FAILURE(status) && pci_acpi_mcfg_entries &&
+	    pci_acpi_mcfg_array) {
+		mcfg_addr = 0;
+		cfg = pci_acpi_mcfg_array;
+		for (i = 0; i < pci_acpi_mcfg_entries; i++, cfg++)
+			if (seg == cfg->pci_segment &&
+			    start >= cfg->start_bus_number &&
+			    start <= cfg->end_bus_number) {
+				end_bus = cfg->end_bus_number;
+				mcfg_addr = cfg->address;
+				break;
+			}
+	}
+	if (endp)
+		*endp = end_bus;
+
+	return (phys_addr_t)mcfg_addr;
+}
+
 /*
  * _SxD returns the D-state with the highest power
  * (lowest D-state number) supported in the S-state "x".
diff --git a/include/acpi/acnames.h b/include/acpi/acnames.h
index 38f5088..99bda75 100644
--- a/include/acpi/acnames.h
+++ b/include/acpi/acnames.h
@@ -62,6 +62,7 @@
 #define METHOD_NAME__AEI        "_AEI"
 #define METHOD_NAME__PRW        "_PRW"
 #define METHOD_NAME__SRS        "_SRS"
+#define METHOD_NAME__CBA	"_CBA"
 
 /* Method names - these methods must appear at the namespace root */
 
diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h
index f1c8ca6..8bc5229 100644
--- a/include/acpi/acpi_bus.h
+++ b/include/acpi/acpi_bus.h
@@ -370,6 +370,9 @@ struct acpi_pci_root {
 
 	u32 osc_support_set;	/* _OSC state of support bits */
 	u32 osc_control_set;	/* _OSC state of control bits */
+	uint8_t mcfg_end_bus;	/* End bus for MCFG may differ from
+				 * root's subordinary bus. */
+	phys_addr_t mcfg_addr;
 };
 
 /* helper */
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 4462350..0369149 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -11,12 +11,17 @@
 #include <linux/acpi.h>
 
 #ifdef CONFIG_ACPI
+extern int pci_acpi_mcfg_entries;
+extern struct acpi_mcfg_allocation *pci_acpi_mcfg_array;
+
 extern acpi_status pci_acpi_add_bus_pm_notifier(struct acpi_device *dev,
 						 struct pci_bus *pci_bus);
 extern acpi_status pci_acpi_remove_bus_pm_notifier(struct acpi_device *dev);
 extern acpi_status pci_acpi_add_pm_notifier(struct acpi_device *dev,
 					     struct pci_dev *pci_dev);
 extern acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev);
+extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle,
+			uint16_t seg, uint8_t start, int *endp);
 
 static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
 {
-- 
1.7.1

[PATCH v7 07/10] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Jiang Liu]

This patch enhances x86 arch specific code to update MMCFG information
when PCI host bridge hotplug event happens.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/include/asm/pci_x86.h |    1 +
 arch/x86/pci/acpi.c            |   69 ++++++++++++++++++++++++++++++++++++++++
 arch/x86/pci/mmconfig-shared.c |    7 +---
 arch/x86/pci/mmconfig_32.c     |    2 +-
 arch/x86/pci/mmconfig_64.c     |    2 +-
 5 files changed, 73 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index b2fb1d8..3e5f43c 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -100,6 +100,7 @@ struct pci_raw_ops {
 extern const struct pci_raw_ops *raw_pci_ops;
 extern const struct pci_raw_ops *raw_pci_ext_ops;
 
+extern const struct pci_raw_ops pci_mmcfg;
 extern const struct pci_raw_ops pci_direct_conf1;
 extern bool port_cf9_safe;
 
diff --git a/arch/x86/pci/acpi.c b/arch/x86/pci/acpi.c
index 2bb885a..76ad029 100644
--- a/arch/x86/pci/acpi.c
+++ b/arch/x86/pci/acpi.c
@@ -4,6 +4,7 @@
 #include <linux/irq.h>
 #include <linux/dmi.h>
 #include <linux/slab.h>
+#include <linux/pci-acpi.h>
 #include <asm/numa.h>
 #include <asm/pci_x86.h>
 
@@ -13,6 +14,12 @@ struct pci_root_info {
 	unsigned int res_num;
 	struct resource *res;
 	struct pci_sysdata sd;
+#ifdef	CONFIG_PCI_MMCONFIG
+	bool mcfg_added;
+	uint16_t segment;
+	uint8_t start_bus;
+	uint8_t end_bus;
+#endif
 };
 
 static bool pci_use_crs = true;
@@ -119,6 +126,61 @@ void __init pci_acpi_crs_quirks(void)
 	       pci_use_crs ? "nocrs" : "use_crs");
 }
 
+static int __devinit setup_mcfg_map(struct pci_root_info *info,
+				    uint16_t seg, uint8_t start, uint8_t end,
+				    phys_addr_t addr)
+{
+#ifdef	CONFIG_PCI_MMCONFIG
+	int result;
+	struct device *dev = &info->bridge->dev;
+
+	info->start_bus = start;
+	info->end_bus = end;
+	info->mcfg_added = false;
+
+	/* return success if MMCFG is not in use */
+	if (raw_pci_ext_ops && raw_pci_ext_ops != &pci_mmcfg)
+		return 0;
+
+	if (!(pci_probe & PCI_PROBE_MMCONF)) {
+		/* still could use raw_pci_ops for devices on segment 0 */
+		if (seg)
+			dev_warn(dev,
+				 "MMCONFIG is disabled, can't access extended "
+				 "configuration space under this bridge.\n");
+		return 0;
+	}
+
+	result = pci_mmconfig_insert(dev, seg, start, end, addr);
+	if (result == 0) {
+		/* enable MMCFG if it hasn't been enabled yet */
+		if (raw_pci_ext_ops == NULL)
+			raw_pci_ext_ops = &pci_mmcfg;
+		info->mcfg_added = true;
+	} else if (result != -EEXIST && addr) {
+		/*
+		 * Failure in adding MMCFG information is not fatal,
+		 * just can't access [extended] configuration space of
+		 * devices under this host bridge.
+		 */
+		dev_warn(dev, "fail to add MMCONFIG information.\n");
+	}
+#endif
+
+	return 0;
+}
+
+static void teardown_mcfg_map(struct pci_root_info *info)
+{
+#ifdef	CONFIG_PCI_MMCONFIG
+	if (info->mcfg_added) {
+		pci_mmconfig_delete(info->segment, info->start_bus,
+				    info->end_bus);
+		info->mcfg_added = false;
+	}
+#endif
+}
+
 static acpi_status
 resource_to_addr(struct acpi_resource *resource,
 			struct acpi_resource_address64 *addr)
@@ -331,6 +393,8 @@ static void __release_pci_root_info(struct pci_root_info *info)
 
 	free_pci_root_info_res(info);
 
+	teardown_mcfg_map(info);
+
 	kfree(info);
 }
 static void release_pci_root_info(struct pci_host_bridge *bridge)
@@ -438,6 +502,11 @@ struct pci_bus * __devinit pci_acpi_scan_root(struct acpi_pci_root *root)
 			x86_pci_root_bus_resources(busnum, &resources);
 		}
 
+		if (root->mcfg_addr)
+			setup_mcfg_map(info, root->segment,
+				       (uint8_t)root->secondary.start,
+				       root->mcfg_end_bus, root->mcfg_addr);
+
 		bus = pci_create_root_bus(NULL, busnum, &pci_root_ops, sd,
 					  &resources);
 		if (bus) {
diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 993e766..fa0aa90 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -729,7 +729,7 @@ int __devinit pci_mmconfig_insert(struct device *dev,
 	if (!(pci_probe & PCI_PROBE_MMCONF) || pci_mmcfg_arch_init_failed)
 		return -ENODEV;
 
-	if (start > end)
+	if (start > end || !addr)
 		return -EINVAL;
 
 	mutex_lock(&pci_mmcfg_lock);
@@ -745,11 +745,6 @@ int __devinit pci_mmconfig_insert(struct device *dev,
 		return -EEXIST;
 	}
 
-	if (!addr) {
-		mutex_unlock(&pci_mmcfg_lock);
-		return -EINVAL;
-	}
-
 	rc = -EBUSY;
 	cfg = pci_mmconfig_alloc(seg, start, end, addr);
 	if (cfg == NULL) {
diff --git a/arch/x86/pci/mmconfig_32.c b/arch/x86/pci/mmconfig_32.c
index a22785d..db63ac2 100644
--- a/arch/x86/pci/mmconfig_32.c
+++ b/arch/x86/pci/mmconfig_32.c
@@ -126,7 +126,7 @@ static int pci_mmcfg_write(unsigned int seg, unsigned int bus,
 	return 0;
 }
 
-static const struct pci_raw_ops pci_mmcfg = {
+const struct pci_raw_ops pci_mmcfg = {
 	.read =		pci_mmcfg_read,
 	.write =	pci_mmcfg_write,
 };
diff --git a/arch/x86/pci/mmconfig_64.c b/arch/x86/pci/mmconfig_64.c
index 4e05779..34c08dd 100644
--- a/arch/x86/pci/mmconfig_64.c
+++ b/arch/x86/pci/mmconfig_64.c
@@ -90,7 +90,7 @@ static int pci_mmcfg_write(unsigned int seg, unsigned int bus,
 	return 0;
 }
 
-static const struct pci_raw_ops pci_mmcfg = {
+const struct pci_raw_ops pci_mmcfg = {
 	.read =		pci_mmcfg_read,
 	.write =	pci_mmcfg_write,
 };
-- 
1.7.1

[PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Jiang Liu]

This patch changes mmconfig logic on x86 platforms to add MMCFG
information on demand instead of adding all MMCFG entries from
the ACPI MCFG table at boot time. So only MMCFG address ranges
used by active PCI host bridges will be actually mapped.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/include/asm/pci_x86.h |    5 +++
 arch/x86/pci/legacy.c          |    1 +
 arch/x86/pci/mmconfig-shared.c |   54 +++++++++++++++++++++++++++++++++++++---
 3 files changed, 56 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index 3e5f43c..4a1a9aa 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -142,6 +142,11 @@ extern int __devinit pci_mmconfig_insert(struct device *dev,
 					 uint16_t seg, uint8_t start,
 					 uint8_t end, phys_addr_t addr);
 extern int pci_mmconfig_delete(uint16_t seg, uint8_t start, uint8_t end);
+#ifdef	CONFIG_ACPI
+extern void pci_mmconfig_probe(uint8_t start);
+#else
+static inline void pci_mmconfig_probe(uint8_t start) { }
+#endif
 extern struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus);
 
 extern struct list_head pci_mmcfg_list;
diff --git a/arch/x86/pci/legacy.c b/arch/x86/pci/legacy.c
index a1df191..e9a2384 100644
--- a/arch/x86/pci/legacy.c
+++ b/arch/x86/pci/legacy.c
@@ -49,6 +49,7 @@ void __devinit pcibios_scan_specific_bus(int busn)
 		    l != 0x0000 && l != 0xffff) {
 			DBG("Found device at %02x:%02x [%04x]\n", busn, devfn, l);
 			printk(KERN_INFO "PCI: Discovered peer bus %02x\n", busn);
+			pci_mmconfig_probe(busn);
 			pci_scan_bus_on_node(busn, &pci_root_ops, node);
 			return;
 		}
diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index fa0aa90..e80b5c2 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -19,6 +19,7 @@
 #include <linux/slab.h>
 #include <linux/mutex.h>
 #include <linux/rculist.h>
+#include <linux/pci-acpi.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
 #include <asm/acpi.h>
@@ -616,6 +617,16 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
 			return -ENODEV;
 		}
 
+		/*
+		 * MMCFG information for host brideges will be added on demand
+		 * by pci_root driver if ACPI is enabled. But there are special
+		 * requirements for devices on segment 0, MMCFG information may
+		 * be needed for fixing hardware quirks and probing for hidden
+		 * buses.
+		 */
+		if (!acpi_disabled && cfg->pci_segment)
+			continue;
+
 		if (pci_mmconfig_add(cfg->pci_segment, cfg->start_bus_number,
 				   cfg->end_bus_number, cfg->address) == NULL) {
 			printk(KERN_WARNING PREFIX
@@ -625,6 +636,13 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
 		}
 	}
 
+	i = entries * sizeof(*cfg_table);
+	pci_acpi_mcfg_array = kmalloc(i, GFP_KERNEL);
+	if (pci_acpi_mcfg_array) {
+		memcpy(pci_acpi_mcfg_array, cfg_table, i);
+		pci_acpi_mcfg_entries = entries;
+	}
+
 	return 0;
 }
 
@@ -634,14 +652,14 @@ static void __init __pci_mmcfg_init(int early)
 	if ((pci_probe & PCI_PROBE_MMCONF) == 0)
 		return;
 
-	/* MMCONFIG already enabled */
-	if (!early && !(pci_probe & PCI_PROBE_MASK & ~PCI_PROBE_MMCONF))
-		return;
-
 	/* for late to exit */
 	if (known_bridge)
 		return;
 
+	/* MMCONFIG already enabled */
+	if (!early && !(pci_probe & PCI_PROBE_MASK & ~PCI_PROBE_MMCONF))
+		goto out;
+
 	if (early) {
 		if (pci_mmcfg_check_hostbridge())
 			known_bridge = 1;
@@ -675,6 +693,14 @@ static void __init __pci_mmcfg_init(int early)
 		pci_mmcfg_resources_inserted = 1;
 		pci_mmcfg_arch_init_failed = true;
 	}
+
+out:
+	/*
+	 * Free all MCFG entries if ACPI is enabled. MCFG information will
+	 * be added back on demand by the pci_root driver later.
+	 */
+	if (!early && !acpi_disabled && !known_bridge && pci_acpi_mcfg_array)
+		free_all_mmcfg();
 }
 
 void __init pci_mmcfg_early_init(void)
@@ -809,3 +835,23 @@ int pci_mmconfig_delete(uint16_t seg, uint8_t start, uint8_t end)
 
 	return -ENOENT;
 }
+
+/* Probe MMCFG information for PCI bus blind probe */
+void __devinit pci_mmconfig_probe(uint8_t start)
+{
+	int end_bus, temp;
+	phys_addr_t addr;
+
+	if (!(pci_probe & PCI_PROBE_MMCONF) || pci_mmcfg_arch_init_failed)
+		return;
+
+	addr = acpi_pci_root_get_mcfg_addr(NULL, 0, start, &end_bus);
+	if (addr && end_bus >= 0 && end_bus <= 255) {
+		for (temp = start + 1; temp <= end_bus; temp++)
+			if (pci_find_bus(0, temp))
+				break;
+
+		temp--;
+		pci_mmconfig_insert(NULL, 0, start, (uint8_t)temp, addr);
+	}
+}
-- 
1.7.1

[PATCH v7 09/10] PCI, x86: simplify pci_mmcfg_late_insert_resources()

[Jiang Liu]

Reduce redundant code to simplify pci_mmcfg_late_insert_resources().

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/pci/mmconfig-shared.c |   34 ++++++++--------------------------
 1 files changed, 8 insertions(+), 26 deletions(-)

diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index e80b5c2..77f0db2 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -27,7 +27,6 @@
 #define PREFIX "PCI: "
 
 /* Indicate if the mmcfg resources have been placed into the resource table. */
-static int __initdata pci_mmcfg_resources_inserted;
 static bool pci_mmcfg_running_state;
 static bool pci_mmcfg_arch_init_failed;
 static DEFINE_MUTEX(pci_mmcfg_lock);
@@ -374,18 +373,6 @@ static int __init pci_mmcfg_check_hostbridge(void)
 	return !list_empty(&pci_mmcfg_list);
 }
 
-static void __init pci_mmcfg_insert_resources(void)
-{
-	struct pci_mmcfg_region *cfg;
-
-	list_for_each_entry(cfg, &pci_mmcfg_list, list)
-		if (!cfg->res.parent)
-			insert_resource(&iomem_resource, &cfg->res);
-
-	/* Mark that the resources have been inserted. */
-	pci_mmcfg_resources_inserted = 1;
-}
-
 static acpi_status __devinit check_mcfg_resource(struct acpi_resource *res,
 						 void *data)
 {
@@ -686,11 +673,7 @@ static void __init __pci_mmcfg_init(int early)
 	if (pci_mmcfg_arch_init())
 		pci_probe = (pci_probe & ~PCI_PROBE_MASK) | PCI_PROBE_MMCONF;
 	else {
-		/*
-		 * Signal not to attempt to insert mmcfg resources because
-		 * the architecture mmcfg setup could not initialize.
-		 */
-		pci_mmcfg_resources_inserted = 1;
+		free_all_mmcfg();
 		pci_mmcfg_arch_init_failed = true;
 	}
 
@@ -715,15 +698,12 @@ void __init pci_mmcfg_late_init(void)
 
 static int __init pci_mmcfg_late_insert_resources(void)
 {
+	struct pci_mmcfg_region *cfg;
+
 	pci_mmcfg_running_state = true;
 
-	/*
-	 * If resources are already inserted or we are not using MMCONFIG,
-	 * don't insert the resources.
-	 */
-	if ((pci_mmcfg_resources_inserted == 1) ||
-	    (pci_probe & PCI_PROBE_MMCONF) == 0 ||
-	    list_empty(&pci_mmcfg_list))
+	/* If we are not using MMCONFIG, don't insert the resources. */
+	if ((pci_probe & PCI_PROBE_MMCONF) == 0)
 		return 1;
 
 	/*
@@ -731,7 +711,9 @@ static int __init pci_mmcfg_late_insert_resources(void)
 	 * marked so it won't cause request errors when __request_region is
 	 * called.
 	 */
-	pci_mmcfg_insert_resources();
+	list_for_each_entry(cfg, &pci_mmcfg_list, list)
+		if (!cfg->res.parent)
+			insert_resource(&iomem_resource, &cfg->res);
 
 	return 0;
 }
-- 
1.7.1

[PATCH v7 10/10] PCI, x86: get rid of redundant log messages

[Jiang Liu]

For each resource of a PCI host bridge, the arch code and PCI
code log following messages:
"pci_root PNP0A08:00: host bridge window [io 0x0000-0x03af]",
"pci_bus 0000:00: root bus resource [io  0x0000-0x03af]"

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/pci/acpi.c |    7 -------
 1 files changed, 0 insertions(+), 7 deletions(-)

diff --git a/arch/x86/pci/acpi.c b/arch/x86/pci/acpi.c
index 76ad029..4368387 100644
--- a/arch/x86/pci/acpi.c
+++ b/arch/x86/pci/acpi.c
@@ -295,13 +295,6 @@ setup_resource(struct acpi_resource *acpi_res, void *data)
 	}
 
 	info->res_num++;
-	if (addr.translation_offset)
-		dev_info(&info->bridge->dev, "host bridge window %pR "
-			 "(PCI address [%#llx-%#llx])\n",
-			 res, res->start - addr.translation_offset,
-			 res->end - addr.translation_offset);
-	else
-		dev_info(&info->bridge->dev, "host bridge window %pR\n", res);
 
 	return AE_OK;
 }
-- 
1.7.1

Re: [PATCH v7 00/10] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Taku Izumi]

Hi Jiang,

 Looks good, and works as expected.

 Best regards,
 Taku Izumi

On Sat, 26 May 2012 17:53:53 +0800
Jiang Liu <jiang.liu@huawei.com> wrote:

> This patchset enhance pci_root driver to update MMCFG information when
> hot-plugging PCI root bridges. It applies to 
> git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git pci/next-3.5
> 
> -v2: split into smaller patches and skip updating MMCFG information when
>      MMCFG is disabled
> -v3: add mmconf_added to simply free path, also make pci_mmconfig_insert()
>      to process extra exist case --- By Yinghai
> -v4: tune arch_acpi_pci_root_add() to handle a corner case raised by Kenji
> -v5: address review comments from Bjorn and Taku, also better handle corner
>      cases in arch_acpi_pci_root_add()
> -v6: get rid of arch_acpi_pci_root_xxx() by using existing hooks
>      add MCFG information for host bridges on demand
>      more corner cases clear up
>      correctly handle condition compilation
>      fix section mismatch issues
>      fix a issue reported by Taku about a BIOS bug
> -v7: unify log messages
>      remove redundant host bridge resource related log messages
>      fix a issue reported by Taku which breaks pnp resource allocation
> 
> Jiang Liu (10):
>   PCI, x86: split out pci_mmcfg_check_reserved() for code reuse
>   PCI, x86: split out pci_mmconfig_alloc() for code reuse
>   PCI, x86: use RCU list to protect mmconfig list
>   PCI, x86: introduce pci_mmcfg_arch_map()/pci_mmcfg_arch_unmap()
>   PCI, x86: introduce pci_mmconfig_insert()/delete() for PCI root
>     bridge hotplug
>   PCI, ACPI: provide MCFG address for PCI host bridges
>   PCI, x86: update MMCFG information when hot-plugging PCI host bridges
>   PCI, x86: add MMCFG information on demand
>   PCI, x86: simplify pci_mmcfg_late_insert_resources()
>   PCI, x86: get rid of redundant log messages
> 
>  arch/x86/include/asm/pci_x86.h |   12 ++
>  arch/x86/pci/acpi.c            |   76 ++++++++-
>  arch/x86/pci/legacy.c          |    1 +
>  arch/x86/pci/mmconfig-shared.c |  364 +++++++++++++++++++++++++++++-----------
>  arch/x86/pci/mmconfig_32.c     |   30 +++-
>  arch/x86/pci/mmconfig_64.c     |   37 ++++-
>  drivers/acpi/pci_root.c        |   12 ++
>  drivers/pci/pci-acpi.c         |   34 ++++
>  include/acpi/acnames.h         |    1 +
>  include/acpi/acpi_bus.h        |    3 +
>  include/linux/pci-acpi.h       |    5 +
>  11 files changed, 465 insertions(+), 110 deletions(-)
> 
> 
> 

Re: [PATCH v7 00/10] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Bjorn Helgaas]

On Sat, May 26, 2012 at 3:53 AM, Jiang Liu <jiang.liu@huawei.com> wrote:
> This patchset enhance pci_root driver to update MMCFG information when
> hot-plugging PCI root bridges. It applies to
> git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git pci/next-3.5
>
> -v2: split into smaller patches and skip updating MMCFG information when
>     MMCFG is disabled
> -v3: add mmconf_added to simply free path, also make pci_mmconfig_insert()
>     to process extra exist case --- By Yinghai
> -v4: tune arch_acpi_pci_root_add() to handle a corner case raised by Kenji
> -v5: address review comments from Bjorn and Taku, also better handle corner
>     cases in arch_acpi_pci_root_add()
> -v6: get rid of arch_acpi_pci_root_xxx() by using existing hooks
>     add MCFG information for host bridges on demand
>     more corner cases clear up
>     correctly handle condition compilation
>     fix section mismatch issues
>     fix a issue reported by Taku about a BIOS bug
> -v7: unify log messages
>     remove redundant host bridge resource related log messages
>     fix a issue reported by Taku which breaks pnp resource allocation
>
> Jiang Liu (10):
>  PCI, x86: split out pci_mmcfg_check_reserved() for code reuse
>  PCI, x86: split out pci_mmconfig_alloc() for code reuse
>  PCI, x86: use RCU list to protect mmconfig list
>  PCI, x86: introduce pci_mmcfg_arch_map()/pci_mmcfg_arch_unmap()
>  PCI, x86: introduce pci_mmconfig_insert()/delete() for PCI root
>    bridge hotplug
>  PCI, ACPI: provide MCFG address for PCI host bridges
>  PCI, x86: update MMCFG information when hot-plugging PCI host bridges
>  PCI, x86: add MMCFG information on demand
>  PCI, x86: simplify pci_mmcfg_late_insert_resources()
>  PCI, x86: get rid of redundant log messages

I applied this to a topic/jiang-mmconfig-v7 branch with minor changes:

- I changed uint8_t and uint16_t to the more common u8 and u16.
- I changed the "not reserved in ACPI motherboard resources" message
from KERN_ERR to KERN_INFO.

Previously, we typically would not see "motherboard resources"
warnings at all because we didn't check for them during early init,
and late init returned before checking because MMCONFIG was already
enabled.  Now, the pci_mmconfig_insert() does the checking that we
skipped before.

Using KERN_ERR would break the graphical boot experience and users
would complain, we can't do anything to fix it, and MMCONFIG actually
works fine anyway.  So I don't think there's any point in having them
be errors.

I have to say I'm still ambivalent about this.  I really like the fact
that we track MMCONFIG space on a per-host bridge basis.  But the code
in the init path is ridiculously, embarrassingly complicated
(obviously this is not your fault at all).  It took me a couple hours
to trace through and figure out why my box emits the "motherboard
resources" warnings when it didn't before.

I'm going to apply the series because I do think it's a net win.  So
just consider this as encouragement to simplify the design when you
can, even if you're not adding functionality.

I'll merge this to my "next" branch tomorrow unless you see any issues
with the tweaks I made.

Bjorn

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Yinghai Lu]

On Sat, May 26, 2012 at 2:54 AM, Jiang Liu <jiang.liu@huawei.com> wrote:
> This patch changes mmconfig logic on x86 platforms to add MMCFG
> information on demand instead of adding all MMCFG entries from
> the ACPI MCFG table at boot time. So only MMCFG address ranges
> used by active PCI host bridges will be actually mapped.
>
> Signed-off-by: Jiang Liu <liuj97@gmail.com>
> ---
>  arch/x86/include/asm/pci_x86.h |    5 +++
>  arch/x86/pci/legacy.c          |    1 +
>  arch/x86/pci/mmconfig-shared.c |   54 +++++++++++++++++++++++++++++++++++++---
>  3 files changed, 56 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
> index 3e5f43c..4a1a9aa 100644
> --- a/arch/x86/include/asm/pci_x86.h
> +++ b/arch/x86/include/asm/pci_x86.h
> @@ -142,6 +142,11 @@ extern int __devinit pci_mmconfig_insert(struct device *dev,
>                                         uint16_t seg, uint8_t start,
>                                         uint8_t end, phys_addr_t addr);
>  extern int pci_mmconfig_delete(uint16_t seg, uint8_t start, uint8_t end);
> +#ifdef CONFIG_ACPI
> +extern void pci_mmconfig_probe(uint8_t start);
> +#else
> +static inline void pci_mmconfig_probe(uint8_t start) { }
> +#endif
>  extern struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus);
>
>  extern struct list_head pci_mmcfg_list;
> diff --git a/arch/x86/pci/legacy.c b/arch/x86/pci/legacy.c
> index a1df191..e9a2384 100644
> --- a/arch/x86/pci/legacy.c
> +++ b/arch/x86/pci/legacy.c
> @@ -49,6 +49,7 @@ void __devinit pcibios_scan_specific_bus(int busn)
>                    l != 0x0000 && l != 0xffff) {
>                        DBG("Found device at %02x:%02x [%04x]\n", busn, devfn, l);
>                        printk(KERN_INFO "PCI: Discovered peer bus %02x\n", busn);
> +                       pci_mmconfig_probe(busn);
>                        pci_scan_bus_on_node(busn, &pci_root_ops, node);
>                        return;
>                }
> diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
> index fa0aa90..e80b5c2 100644
> --- a/arch/x86/pci/mmconfig-shared.c
> +++ b/arch/x86/pci/mmconfig-shared.c
> @@ -19,6 +19,7 @@
>  #include <linux/slab.h>
>  #include <linux/mutex.h>
>  #include <linux/rculist.h>
> +#include <linux/pci-acpi.h>
>  #include <asm/e820.h>
>  #include <asm/pci_x86.h>
>  #include <asm/acpi.h>
> @@ -616,6 +617,16 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
>                        return -ENODEV;
>                }
>
> +               /*
> +                * MMCFG information for host brideges will be added on demand
> +                * by pci_root driver if ACPI is enabled. But there are special
> +                * requirements for devices on segment 0, MMCFG information may
> +                * be needed for fixing hardware quirks and probing for hidden
> +                * buses.
> +                */
> +               if (!acpi_disabled && cfg->pci_segment)
> +                       continue;
> +
>                if (pci_mmconfig_add(cfg->pci_segment, cfg->start_bus_number,
>                                   cfg->end_bus_number, cfg->address) == NULL) {
>                        printk(KERN_WARNING PREFIX
> @@ -625,6 +636,13 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
>                }
>        }
>
> +       i = entries * sizeof(*cfg_table);
> +       pci_acpi_mcfg_array = kmalloc(i, GFP_KERNEL);
> +       if (pci_acpi_mcfg_array) {
> +               memcpy(pci_acpi_mcfg_array, cfg_table, i);
> +               pci_acpi_mcfg_entries = entries;
> +       }
> +

here cache cfg too early.  should do that after

pci_mmcfg_reject_broken().

otherwise will use mcfg even try to reject that before.

>        return 0;
>  }
>
> @@ -634,14 +652,14 @@ static void __init __pci_mmcfg_init(int early)
>        if ((pci_probe & PCI_PROBE_MMCONF) == 0)
>                return;
>
> -       /* MMCONFIG already enabled */
> -       if (!early && !(pci_probe & PCI_PROBE_MASK & ~PCI_PROBE_MMCONF))
> -               return;
> -
>        /* for late to exit */
>        if (known_bridge)
>                return;
>
> +       /* MMCONFIG already enabled */
> +       if (!early && !(pci_probe & PCI_PROBE_MASK & ~PCI_PROBE_MMCONF))
> +               goto out;
> +
>        if (early) {
>                if (pci_mmcfg_check_hostbridge())
>                        known_bridge = 1;
> @@ -675,6 +693,14 @@ static void __init __pci_mmcfg_init(int early)
>                pci_mmcfg_resources_inserted = 1;
>                pci_mmcfg_arch_init_failed = true;
>        }
> +
> +out:
> +       /*
> +        * Free all MCFG entries if ACPI is enabled. MCFG information will
> +        * be added back on demand by the pci_root driver later.
> +        */
> +       if (!early && !acpi_disabled && !known_bridge && pci_acpi_mcfg_array)
> +               free_all_mmcfg();

that really change the logic.

looks like it will break mrst/sfi path.

the scan from pci_legacy_init() for mrst/sfi will not have ext_pci_ops
set for bus 0.

| int __init pci_subsys_init(void)
| {
|        /*
|         * The init function returns an non zero value when
|         * pci_legacy_init should be invoked.
|         */
|        if (x86_init.pci.init())
|                pci_legacy_init();
|
|        pcibios_fixup_peer_bridges();


Yinghai

>  }
>
>  void __init pci_mmcfg_early_init(void)
> @@ -809,3 +835,23 @@ int pci_mmconfig_delete(uint16_t seg, uint8_t start, uint8_t end)
>
>        return -ENOENT;
>  }
> +
> +/* Probe MMCFG information for PCI bus blind probe */
> +void __devinit pci_mmconfig_probe(uint8_t start)
> +{
> +       int end_bus, temp;
> +       phys_addr_t addr;
> +
> +       if (!(pci_probe & PCI_PROBE_MMCONF) || pci_mmcfg_arch_init_failed)
> +               return;
> +
> +       addr = acpi_pci_root_get_mcfg_addr(NULL, 0, start, &end_bus);
> +       if (addr && end_bus >= 0 && end_bus <= 255) {
> +               for (temp = start + 1; temp <= end_bus; temp++)
> +                       if (pci_find_bus(0, temp))
> +                               break;
> +
> +               temp--;
> +               pci_mmconfig_insert(NULL, 0, start, (uint8_t)temp, addr);
> +       }
> +}
> --
> 1.7.1
>
>

Re: [PATCH v7 00/10] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Yinghai Lu]

On Thu, Jun 14, 2012 at 8:06 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> I have to say I'm still ambivalent about this.  I really like the fact
> that we track MMCONFIG space on a per-host bridge basis.  But the code
> in the init path is ridiculously, embarrassingly complicated
> (obviously this is not your fault at all).  It took me a couple hours
> to trace through and figure out why my box emits the "motherboard
> resources" warnings when it didn't before.

[PATCH v7 08/10] PCI, x86: add MMCFG information on demand

still have some problem that need to be fixed.

Yinghai

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Jiang Liu]

Hi Yinghai,
	Thanks for your review and comments!
On 2012-6-15 15:15, Yinghai Lu wrote:
>> @@ -625,6 +636,13 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
>>                }
>>        }
>>
>> +       i = entries * sizeof(*cfg_table);
>> +       pci_acpi_mcfg_array = kmalloc(i, GFP_KERNEL);
>> +       if (pci_acpi_mcfg_array) {
>> +               memcpy(pci_acpi_mcfg_array, cfg_table, i);
>> +               pci_acpi_mcfg_entries = entries;
>> +       }
>> +
> 
> here cache cfg too early.  should do that after
> 
> pci_mmcfg_reject_broken().
> 
> otherwise will use mcfg even try to reject that before.
Good cache, how about the following patch to fix this?
I'm also preparing an V8 to fix build failure issues reported by Fengguang,
and also simplify the __pci_mmcfg_init() implementation.

---
diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 77f0db2..bcd0928 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -623,11 +623,27 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
                }
        }

-       i = entries * sizeof(*cfg_table);
-       pci_acpi_mcfg_array = kmalloc(i, GFP_KERNEL);
-       if (pci_acpi_mcfg_array) {
-               memcpy(pci_acpi_mcfg_array, cfg_table, i);
-               pci_acpi_mcfg_entries = entries;
+       return 0;
+}
+
+static int __init pci_cache_mcfg(struct acpi_table_header *header)
+{
+       unsigned long i;
+       struct acpi_table_mcfg *mcfg;
+       struct acpi_mcfg_allocation *cfg_table;
+
+       if (!header)
+               return -EINVAL;
+
+       i = (header->length - sizeof(struct acpi_table_mcfg));
+       if (i) {
+               pci_acpi_mcfg_array = kmalloc(i, GFP_KERNEL);
+               if (pci_acpi_mcfg_array) {
+                       mcfg = (struct acpi_table_mcfg *)header;
+                       cfg_table = (struct acpi_mcfg_allocation *) &mcfg[1];
+                       memcpy(pci_acpi_mcfg_array, cfg_table, i);
+                       pci_acpi_mcfg_entries = i / sizeof (*cfg_table);
+               }
        }

        return 0;
@@ -682,8 +698,11 @@ out:
         * Free all MCFG entries if ACPI is enabled. MCFG information will
         * be added back on demand by the pci_root driver later.
         */
-       if (!early && !acpi_disabled && !known_bridge && pci_acpi_mcfg_array)
-               free_all_mmcfg();
+       if (!early && !acpi_disabled && !known_bridge) {
+               acpi_table_parse(ACPI_SIG_MCFG, pci_cache_mcfg);
+               if (pci_acpi_mcfg_array)
+                       free_all_mmcfg();
+       }
 }

 void __init pci_mmcfg_early_init(void)
>> @@ -675,6 +693,14 @@ static void __init __pci_mmcfg_init(int early)
>>                pci_mmcfg_resources_inserted = 1;
>>                pci_mmcfg_arch_init_failed = true;
>>        }
>> +
>> +out:
>> +       /*
>> +        * Free all MCFG entries if ACPI is enabled. MCFG information will
>> +        * be added back on demand by the pci_root driver later.
>> +        */
>> +       if (!early && !acpi_disabled && !known_bridge && pci_acpi_mcfg_array)
>> +               free_all_mmcfg();
> 
> that really change the logic.
> 
> looks like it will break mrst/sfi path.
> 
> the scan from pci_legacy_init() for mrst/sfi will not have ext_pci_ops
> set for bus 0.
> 
> | int __init pci_subsys_init(void)
> | {
> |        /*
> |         * The init function returns an non zero value when
> |         * pci_legacy_init should be invoked.
> |         */
> |        if (x86_init.pci.init())
> |                pci_legacy_init();
> |
> |        pcibios_fixup_peer_bridges();

On platform with SFI implementation such as Moorsetown, acpi_disabled will
be set to 1 because there's no ACPI root pointer and acpi initialize will fail.
And sfi_init() implementation confirms that if acpi is enabled, then sfi will
be disabled. So I will it won't break SFI/MRST.

void __init sfi_init(void)
{
        if (!acpi_disabled)
                disable_sfi();

        if (sfi_disabled)
                return;
}

thanks!
Gerry

Re: [PATCH v7 00/10] PCI, x86: update MMCFG information when hot-plugging PCI host bridges

[Jiang Liu]

Hi Bjorn,
	I have prepared another patch to simplify the __pci_mmcfg_init()
implementation. What's your thoughts? If OK, I will include it in v8.
	Thanks!

diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 236872e..b4b07d4 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -626,32 +626,15 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
        return 0;
 }

-static void __init __pci_mmcfg_init(int early)
+static int __init __pci_mmcfg_init(int early)
 {
-       /* MMCONFIG disabled */
-       if ((pci_probe & PCI_PROBE_MMCONF) == 0)
-               return;
-
-       /* for late to exit */
-       if (known_bridge)
-               return;
-
-       /* MMCONFIG already enabled */
-       if (!early && !(pci_probe & PCI_PROBE_MASK & ~PCI_PROBE_MMCONF))
-               goto out;
-
-       if (early) {
-               if (pci_mmcfg_check_hostbridge())
-                       known_bridge = 1;
-       }
-
        if (!known_bridge)
                acpi_sfi_table_parse(ACPI_SIG_MCFG, pci_parse_mcfg);

        pci_mmcfg_reject_broken(early);

        if (list_empty(&pci_mmcfg_list))
-               return;
+               return -EINVAL;

        if (pcibios_last_bus < 0) {
                const struct pci_mmcfg_region *cfg;
@@ -668,28 +651,43 @@ static void __init __pci_mmcfg_init(int early)
        else {
                free_all_mmcfg();
                pci_mmcfg_arch_init_failed = true;
+               return -EINVAL;
        }

-out:
-       /*
-        * Free all MCFG entries if ACPI is enabled. MCFG information will
-        * be added back on demand by the pci_root driver later.
-        */
-       if (!early && !acpi_disabled && !known_bridge &&
-           !pci_mmcfg_arch_init_failed) {
-               if (!acpi_pci_cache_mcfg())
-                       free_all_mmcfg();
-       }
+       return 0;
 }

 void __init pci_mmcfg_early_init(void)
 {
-       __pci_mmcfg_init(1);
+       if (pci_probe & PCI_PROBE_MMCONF) {
+               if (pci_mmcfg_check_hostbridge())
+                       known_bridge = 1;
+               __pci_mmcfg_init(1);
+       }
 }

 void __init pci_mmcfg_late_init(void)
 {
-       __pci_mmcfg_init(0);
+       int ret = 0;
+
+       /* MMCONFIG disabled */
+       if ((pci_probe & PCI_PROBE_MMCONF) == 0)
+               return;
+
+       if (known_bridge)
+               return;
+
+       /* MMCONFIG hasn't been enabled yet */
+       if (pci_probe & PCI_PROBE_MASK & ~PCI_PROBE_MMCONF)
+               ret = __pci_mmcfg_init(0);
+
+       /*
+        * Free all MCFG entries if ACPI is enabled. MCFG information will
+        * be added back on demand by the pci_root driver later.
+        */
+       if (!acpi_disabled && !ret)
+               if (!acpi_pci_cache_mcfg())
+                       free_all_mmcfg();
 }

 static int __init pci_mmcfg_late_insert_resources(void)

On 2012-6-15 11:06, Bjorn Helgaas wrote:
> On Sat, May 26, 2012 at 3:53 AM, Jiang Liu <jiang.liu@huawei.com> wrote:
>> This patchset enhance pci_root driver to update MMCFG information when
>> hot-plugging PCI root bridges. It applies to
>> git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git pci/next-3.5
>>
>> -v2: split into smaller patches and skip updating MMCFG information when
>>     MMCFG is disabled
>> -v3: add mmconf_added to simply free path, also make pci_mmconfig_insert()
>>     to process extra exist case --- By Yinghai
>> -v4: tune arch_acpi_pci_root_add() to handle a corner case raised by Kenji
>> -v5: address review comments from Bjorn and Taku, also better handle corner
>>     cases in arch_acpi_pci_root_add()
>> -v6: get rid of arch_acpi_pci_root_xxx() by using existing hooks
>>     add MCFG information for host bridges on demand
>>     more corner cases clear up
>>     correctly handle condition compilation
>>     fix section mismatch issues
>>     fix a issue reported by Taku about a BIOS bug
>> -v7: unify log messages
>>     remove redundant host bridge resource related log messages
>>     fix a issue reported by Taku which breaks pnp resource allocation
>>
>> Jiang Liu (10):
>>  PCI, x86: split out pci_mmcfg_check_reserved() for code reuse
>>  PCI, x86: split out pci_mmconfig_alloc() for code reuse
>>  PCI, x86: use RCU list to protect mmconfig list
>>  PCI, x86: introduce pci_mmcfg_arch_map()/pci_mmcfg_arch_unmap()
>>  PCI, x86: introduce pci_mmconfig_insert()/delete() for PCI root
>>    bridge hotplug
>>  PCI, ACPI: provide MCFG address for PCI host bridges
>>  PCI, x86: update MMCFG information when hot-plugging PCI host bridges
>>  PCI, x86: add MMCFG information on demand
>>  PCI, x86: simplify pci_mmcfg_late_insert_resources()
>>  PCI, x86: get rid of redundant log messages
> 
> I applied this to a topic/jiang-mmconfig-v7 branch with minor changes:
> 
> - I changed uint8_t and uint16_t to the more common u8 and u16.
> - I changed the "not reserved in ACPI motherboard resources" message
> from KERN_ERR to KERN_INFO.
> 
> Previously, we typically would not see "motherboard resources"
> warnings at all because we didn't check for them during early init,
> and late init returned before checking because MMCONFIG was already
> enabled.  Now, the pci_mmconfig_insert() does the checking that we
> skipped before.
> 
> Using KERN_ERR would break the graphical boot experience and users
> would complain, we can't do anything to fix it, and MMCONFIG actually
> works fine anyway.  So I don't think there's any point in having them
> be errors.
> 
> I have to say I'm still ambivalent about this.  I really like the fact
> that we track MMCONFIG space on a per-host bridge basis.  But the code
> in the init path is ridiculously, embarrassingly complicated
> (obviously this is not your fault at all).  It took me a couple hours
> to trace through and figure out why my box emits the "motherboard
> resources" warnings when it didn't before.
> 
> I'm going to apply the series because I do think it's a net win.  So
> just consider this as encouragement to simplify the design when you
> can, even if you're not adding functionality.
> 
> I'll merge this to my "next" branch tomorrow unless you see any issues
> with the tweaks I made.
> 
> Bjorn
> 
> .
> 

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Jiang Liu]

Hi Yinghai,
	A formal patch to fix "here cache cfg too early.  should do 
that after pci_mmcfg_reject_broken()". This patch also improves
readability and fixes two condition compilation issues reported by
Fengguang. If you are ok with it, I will fold it into 
"[PATCH v7 08/10] PCI, x86: add MMCFG information on demand".
	Thanks!
	Gerry

---
diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
index 7eae174..5eb2ac9 100644
--- a/arch/x86/include/asm/pci_x86.h
+++ b/arch/x86/include/asm/pci_x86.h
@@ -142,7 +142,7 @@ extern int __devinit pci_mmconfig_insert(struct device *dev,
                                         u16 seg, u8 start, u8 end,
                                         phys_addr_t addr);
 extern int pci_mmconfig_delete(u16 seg, u8 start, u8 end);
-#ifdef CONFIG_ACPI
+#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_MMCONFIG)
 extern void pci_mmconfig_probe(u8 start);
 #else
 static inline void pci_mmconfig_probe(u8 start) { }
diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 20ab4f5..636de35 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -636,13 +636,6 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
                }
        }

-       i = entries * sizeof(*cfg_table);
-       pci_acpi_mcfg_array = kmalloc(i, GFP_KERNEL);
-       if (pci_acpi_mcfg_array) {
-               memcpy(pci_acpi_mcfg_array, cfg_table, i);
-               pci_acpi_mcfg_entries = entries;
-       }
-
        return 0;
 }

@@ -699,8 +692,11 @@ out:
         * Free all MCFG entries if ACPI is enabled. MCFG information will
         * be added back on demand by the pci_root driver later.
         */
-       if (!early && !acpi_disabled && !known_bridge && pci_acpi_mcfg_array)
-               free_all_mmcfg();
+       if (!early && !acpi_disabled && !known_bridge &&
+           !pci_mmcfg_arch_init_failed) {
+               if (!acpi_pci_cache_mcfg())
+                       free_all_mmcfg();
+       }
 }

 void __init pci_mmcfg_early_init(void)
diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
index f5d2157..93e0f91 100644
--- a/drivers/pci/pci-acpi.c
+++ b/drivers/pci/pci-acpi.c
@@ -163,8 +163,34 @@ acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
 }

 /* acpi_table_parse() is marked as __init, so cache MCFG info at boot time */
-int pci_acpi_mcfg_entries;
-struct acpi_mcfg_allocation *pci_acpi_mcfg_array;
+static int pci_acpi_mcfg_entries;
+static struct acpi_mcfg_allocation *pci_acpi_mcfg_array;
+
+static int __init pci_cache_mcfg(struct acpi_table_header *header)
+{
+       u32 sz;
+       void *ptr;
+
+       if (!header || (header->length <= sizeof(struct acpi_table_mcfg)))
+               return -EINVAL;
+
+       sz = (header->length - sizeof(struct acpi_table_mcfg));
+       pci_acpi_mcfg_array = kmalloc(sz, GFP_KERNEL);
+       if (!pci_acpi_mcfg_array)
+               return -ENOMEM;
+
+       ptr = (void *)header + sizeof(struct acpi_table_mcfg);
+       memcpy(pci_acpi_mcfg_array, ptr, sz);
+       pci_acpi_mcfg_entries = sz / sizeof (struct acpi_mcfg_allocation);
+
+       return 0;
+}
+
+int __init acpi_pci_cache_mcfg(void)
+{
+       acpi_table_parse(ACPI_SIG_MCFG, pci_cache_mcfg);
+       return pci_acpi_mcfg_array ? 0 : -EINVAL;
+}

 phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle, u16 seg,
                                        u8 start, int *endp)
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h
index 946789f..e03207c 100644
--- a/include/linux/pci-acpi.h
+++ b/include/linux/pci-acpi.h
@@ -11,15 +11,13 @@
 #include <linux/acpi.h>

 #ifdef CONFIG_ACPI
-extern int pci_acpi_mcfg_entries;
-extern struct acpi_mcfg_allocation *pci_acpi_mcfg_array;
-
 extern acpi_status pci_acpi_add_bus_pm_notifier(struct acpi_device *dev,
                                                 struct pci_bus *pci_bus);
 extern acpi_status pci_acpi_remove_bus_pm_notifier(struct acpi_device *dev);
 extern acpi_status pci_acpi_add_pm_notifier(struct acpi_device *dev,
                                             struct pci_dev *pci_dev);
 extern acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev);
+extern int acpi_pci_cache_mcfg(void);
 extern phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle,
                        u16 seg, u8 start, int *endp);

@@ -40,6 +38,8 @@ static inline acpi_handle acpi_pci_get_bridge_handle(struct pci_bus *pbus)
        return acpi_get_pci_rootbridge_handle(pci_domain_nr(pbus),
                                              pbus->number);
 }
+#else
+static inline int acpi_pci_cache_mcfg(void) { return -EINVAL; }
 #endif

 #ifdef CONFIG_ACPI_APEI
---

On 2012-6-15 15:15, Yinghai Lu wrote:
> On Sat, May 26, 2012 at 2:54 AM, Jiang Liu <jiang.liu@huawei.com> wrote:
>> This patch changes mmconfig logic on x86 platforms to add MMCFG
>> information on demand instead of adding all MMCFG entries from
>> the ACPI MCFG table at boot time. So only MMCFG address ranges
>> used by active PCI host bridges will be actually mapped.
>>
>> Signed-off-by: Jiang Liu <liuj97@gmail.com>
>> ---
>>  arch/x86/include/asm/pci_x86.h |    5 +++
>>  arch/x86/pci/legacy.c          |    1 +
>>  arch/x86/pci/mmconfig-shared.c |   54 +++++++++++++++++++++++++++++++++++++---
>>  3 files changed, 56 insertions(+), 4 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
>> index 3e5f43c..4a1a9aa 100644
>> --- a/arch/x86/include/asm/pci_x86.h
>> +++ b/arch/x86/include/asm/pci_x86.h
>> @@ -142,6 +142,11 @@ extern int __devinit pci_mmconfig_insert(struct device *dev,
>>                                         uint16_t seg, uint8_t start,
>>                                         uint8_t end, phys_addr_t addr);
>>  extern int pci_mmconfig_delete(uint16_t seg, uint8_t start, uint8_t end);
>> +#ifdef CONFIG_ACPI
>> +extern void pci_mmconfig_probe(uint8_t start);
>> +#else
>> +static inline void pci_mmconfig_probe(uint8_t start) { }
>> +#endif
>>  extern struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus);
>>
>>  extern struct list_head pci_mmcfg_list;
>> diff --git a/arch/x86/pci/legacy.c b/arch/x86/pci/legacy.c
>> index a1df191..e9a2384 100644
>> --- a/arch/x86/pci/legacy.c
>> +++ b/arch/x86/pci/legacy.c
>> @@ -49,6 +49,7 @@ void __devinit pcibios_scan_specific_bus(int busn)
>>                    l != 0x0000 && l != 0xffff) {
>>                        DBG("Found device at %02x:%02x [%04x]\n", busn, devfn, l);
>>                        printk(KERN_INFO "PCI: Discovered peer bus %02x\n", busn);
>> +                       pci_mmconfig_probe(busn);
>>                        pci_scan_bus_on_node(busn, &pci_root_ops, node);
>>                        return;
>>                }
>> diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
>> index fa0aa90..e80b5c2 100644
>> --- a/arch/x86/pci/mmconfig-shared.c
>> +++ b/arch/x86/pci/mmconfig-shared.c
>> @@ -19,6 +19,7 @@
>>  #include <linux/slab.h>
>>  #include <linux/mutex.h>
>>  #include <linux/rculist.h>
>> +#include <linux/pci-acpi.h>
>>  #include <asm/e820.h>
>>  #include <asm/pci_x86.h>
>>  #include <asm/acpi.h>
>> @@ -616,6 +617,16 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
>>                        return -ENODEV;
>>                }
>>
>> +               /*
>> +                * MMCFG information for host brideges will be added on demand
>> +                * by pci_root driver if ACPI is enabled. But there are special
>> +                * requirements for devices on segment 0, MMCFG information may
>> +                * be needed for fixing hardware quirks and probing for hidden
>> +                * buses.
>> +                */
>> +               if (!acpi_disabled && cfg->pci_segment)
>> +                       continue;
>> +
>>                if (pci_mmconfig_add(cfg->pci_segment, cfg->start_bus_number,
>>                                   cfg->end_bus_number, cfg->address) == NULL) {
>>                        printk(KERN_WARNING PREFIX
>> @@ -625,6 +636,13 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
>>                }
>>        }
>>
>> +       i = entries * sizeof(*cfg_table);
>> +       pci_acpi_mcfg_array = kmalloc(i, GFP_KERNEL);
>> +       if (pci_acpi_mcfg_array) {
>> +               memcpy(pci_acpi_mcfg_array, cfg_table, i);
>> +               pci_acpi_mcfg_entries = entries;
>> +       }
>> +
> 
> here cache cfg too early.  should do that after
> 
> pci_mmcfg_reject_broken().
> 
> otherwise will use mcfg even try to reject that before.
> 
>>        return 0;
>>  }
>>
>> @@ -634,14 +652,14 @@ static void __init __pci_mmcfg_init(int early)
>>        if ((pci_probe & PCI_PROBE_MMCONF) == 0)
>>                return;
>>
>> -       /* MMCONFIG already enabled */
>> -       if (!early && !(pci_probe & PCI_PROBE_MASK & ~PCI_PROBE_MMCONF))
>> -               return;
>> -
>>        /* for late to exit */
>>        if (known_bridge)
>>                return;
>>
>> +       /* MMCONFIG already enabled */
>> +       if (!early && !(pci_probe & PCI_PROBE_MASK & ~PCI_PROBE_MMCONF))
>> +               goto out;
>> +
>>        if (early) {
>>                if (pci_mmcfg_check_hostbridge())
>>                        known_bridge = 1;
>> @@ -675,6 +693,14 @@ static void __init __pci_mmcfg_init(int early)
>>                pci_mmcfg_resources_inserted = 1;
>>                pci_mmcfg_arch_init_failed = true;
>>        }
>> +
>> +out:
>> +       /*
>> +        * Free all MCFG entries if ACPI is enabled. MCFG information will
>> +        * be added back on demand by the pci_root driver later.
>> +        */
>> +       if (!early && !acpi_disabled && !known_bridge && pci_acpi_mcfg_array)
>> +               free_all_mmcfg();
> 
> that really change the logic.
> 
> looks like it will break mrst/sfi path.
> 
> the scan from pci_legacy_init() for mrst/sfi will not have ext_pci_ops
> set for bus 0.
> 
> | int __init pci_subsys_init(void)
> | {
> |        /*
> |         * The init function returns an non zero value when
> |         * pci_legacy_init should be invoked.
> |         */
> |        if (x86_init.pci.init())
> |                pci_legacy_init();
> |
> |        pcibios_fixup_peer_bridges();
> 
> 
> Yinghai
> 
>>  }
>>
>>  void __init pci_mmcfg_early_init(void)
>> @@ -809,3 +835,23 @@ int pci_mmconfig_delete(uint16_t seg, uint8_t start, uint8_t end)
>>
>>        return -ENOENT;
>>  }
>> +
>> +/* Probe MMCFG information for PCI bus blind probe */
>> +void __devinit pci_mmconfig_probe(uint8_t start)
>> +{
>> +       int end_bus, temp;
>> +       phys_addr_t addr;
>> +
>> +       if (!(pci_probe & PCI_PROBE_MMCONF) || pci_mmcfg_arch_init_failed)
>> +               return;
>> +
>> +       addr = acpi_pci_root_get_mcfg_addr(NULL, 0, start, &end_bus);
>> +       if (addr && end_bus >= 0 && end_bus <= 255) {
>> +               for (temp = start + 1; temp <= end_bus; temp++)
>> +                       if (pci_find_bus(0, temp))
>> +                               break;
>> +
>> +               temp--;
>> +               pci_mmconfig_insert(NULL, 0, start, (uint8_t)temp, addr);
>> +       }
>> +}
>> --
>> 1.7.1
>>
>>
> 
> .
> 

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Bjorn Helgaas]

I wrote:

> But the code in the init path is ridiculously, embarrassingly complicated ...

On Fri, Jun 15, 2012 at 1:15 AM, Yinghai Lu <yinghai@kernel.org> wrote:

> here cache cfg too early.  should do that after
>
> pci_mmcfg_reject_broken().
>
> otherwise will use mcfg even try to reject that before.
...
> that really change the logic.
>
> looks like it will break mrst/sfi path.
>
> the scan from pci_legacy_init() for mrst/sfi will not have ext_pci_ops
> set for bus 0.

I rest my case.  The current init path is unmaintainable.

I'll wait for a v8 (or later) with fixes for the build issues
Fengguang found and Yinghai's ack.  It will make things somewhat
easier for me if you start with my topic/jiang-mmconfig-v7 branch.

Bjorn

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Jiang Liu]

On 06/15/2012 11:46 PM, Bjorn Helgaas wrote:
> I wrote:
> 
>> But the code in the init path is ridiculously, embarrassingly complicated ...
> 
> On Fri, Jun 15, 2012 at 1:15 AM, Yinghai Lu <yinghai@kernel.org> wrote:
> 
>> here cache cfg too early.  should do that after
>>
>> pci_mmcfg_reject_broken().
>>
>> otherwise will use mcfg even try to reject that before.
> ...
This is an issue and I have worked out a patch for this, actually the patch
produces better readable code.

>> that really change the logic.
>>
>> looks like it will break mrst/sfi path.
>>
>> the scan from pci_legacy_init() for mrst/sfi will not have ext_pci_ops
>> set for bus 0.
> 
> I rest my case.  The current init path is unmaintainable.
I think it doesn't change the logic here. I have paid special attention
to the SFI staff when preparing the patch. On systems with SFI instead of
ACPI, such as Moorsetown, there's no ACPI root pointer table available,
so acpi_initialize() will fail and set acpi_disabled to true. We have
checked "!acpi_disabled" before calling free_all_mmcfg(), that means
we won't call free_all_mmcfg() on SFI platforms. 

> 
> I'll wait for a v8 (or later) with fixes for the build issues
> Fengguang found and Yinghai's ack.  It will make things somewhat
> easier for me if you start with my topic/jiang-mmconfig-v7 branch.
Sure, will rebase my v8 patches onto this branch tomorrow.

> 
> Bjorn

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Yinghai Lu]

On Fri, Jun 15, 2012 at 4:50 AM, Jiang Liu <jiang.liu@huawei.com> wrote:
> Hi Yinghai,
>        A formal patch to fix "here cache cfg too early.  should do
> that after pci_mmcfg_reject_broken()". This patch also improves
> readability and fixes two condition compilation issues reported by
> Fengguang. If you are ok with it, I will fold it into
> "[PATCH v7 08/10] PCI, x86: add MMCFG information on demand".
>        Thanks!
>        Gerry
>
> ---
> diff --git a/arch/x86/include/asm/pci_x86.h b/arch/x86/include/asm/pci_x86.h
> index 7eae174..5eb2ac9 100644
> --- a/arch/x86/include/asm/pci_x86.h
> +++ b/arch/x86/include/asm/pci_x86.h
> @@ -142,7 +142,7 @@ extern int __devinit pci_mmconfig_insert(struct device *dev,
>                                         u16 seg, u8 start, u8 end,
>                                         phys_addr_t addr);
>  extern int pci_mmconfig_delete(u16 seg, u8 start, u8 end);
> -#ifdef CONFIG_ACPI
> +#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_MMCONFIG)
>  extern void pci_mmconfig_probe(u8 start);
>  #else
>  static inline void pci_mmconfig_probe(u8 start) { }
> diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
> index 20ab4f5..636de35 100644
> --- a/arch/x86/pci/mmconfig-shared.c
> +++ b/arch/x86/pci/mmconfig-shared.c
> @@ -636,13 +636,6 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
>                }
>        }
>
> -       i = entries * sizeof(*cfg_table);
> -       pci_acpi_mcfg_array = kmalloc(i, GFP_KERNEL);
> -       if (pci_acpi_mcfg_array) {
> -               memcpy(pci_acpi_mcfg_array, cfg_table, i);
> -               pci_acpi_mcfg_entries = entries;
> -       }
> -
>        return 0;
>  }
>
> @@ -699,8 +692,11 @@ out:
>         * Free all MCFG entries if ACPI is enabled. MCFG information will
>         * be added back on demand by the pci_root driver later.
>         */
> -       if (!early && !acpi_disabled && !known_bridge && pci_acpi_mcfg_array)
> -               free_all_mmcfg();
> +       if (!early && !acpi_disabled && !known_bridge &&
> +           !pci_mmcfg_arch_init_failed) {
> +               if (!acpi_pci_cache_mcfg())
> +                       free_all_mmcfg();
> +       }
>  }
>
>  void __init pci_mmcfg_early_init(void)
> diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
> index f5d2157..93e0f91 100644
> --- a/drivers/pci/pci-acpi.c
> +++ b/drivers/pci/pci-acpi.c
> @@ -163,8 +163,34 @@ acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev)
>  }
>
>  /* acpi_table_parse() is marked as __init, so cache MCFG info at boot time */
> -int pci_acpi_mcfg_entries;
> -struct acpi_mcfg_allocation *pci_acpi_mcfg_array;
> +static int pci_acpi_mcfg_entries;
> +static struct acpi_mcfg_allocation *pci_acpi_mcfg_array;
> +
> +static int __init pci_cache_mcfg(struct acpi_table_header *header)
> +{
> +       u32 sz;
> +       void *ptr;
> +
> +       if (!header || (header->length <= sizeof(struct acpi_table_mcfg)))
> +               return -EINVAL;
> +
> +       sz = (header->length - sizeof(struct acpi_table_mcfg));
> +       pci_acpi_mcfg_array = kmalloc(sz, GFP_KERNEL);
> +       if (!pci_acpi_mcfg_array)
> +               return -ENOMEM;
> +
> +       ptr = (void *)header + sizeof(struct acpi_table_mcfg);
> +       memcpy(pci_acpi_mcfg_array, ptr, sz);
> +       pci_acpi_mcfg_entries = sz / sizeof (struct acpi_mcfg_allocation);
> +
> +       return 0;
> +}
> +
> +int __init acpi_pci_cache_mcfg(void)
> +{
> +       acpi_table_parse(ACPI_SIG_MCFG, pci_cache_mcfg);
> +       return pci_acpi_mcfg_array ? 0 : -EINVAL;
> +}

still have some problem:
pci_mmcfg_check_reserved==>is_mmconf_reserved

will update cfg->end_bus.

Thanks

Yinghai

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Yinghai Lu]

On Fri, Jun 15, 2012 at 8:46 AM, Bjorn Helgaas <bhelgaas@google.com> wrote:
>
>> But the code in the init path is ridiculously, embarrassingly complicated ...
>
> I rest my case.  The current init path is unmaintainable.
>
> I'll wait for a v8 (or later) with fixes for the build issues
> Fengguang found and Yinghai's ack.  It will make things somewhat
> easier for me if you start with my topic/jiang-mmconfig-v7 branch.

yes. probe/free then install is not good.

should make __pci_mmcfg_init() to cache needed stuff for all path, and
then consume stored
entry just before scan root bus.

Thanks

Yinghai

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Jiang Liu]

Hi Yinghai,
	How about following patch to solve the "probe/free then install" issue?
For pci_mmcfg_early_init(), we still need the "probe/free then install" dance
because MMCONFIG may be need for hardware quirks at early stage.
For pci_mmcfg_late_init(), the patch get rid of the redundant "probe/free"
steps. It also generates better readable code.
	Thanks!
	Gerry

---
The logic of __pci_mmcfg_init() is a little complex, so simplify it a little.

Signed-off-by: Jiang Liu <liuj97@gmail.com>
---
 arch/x86/pci/mmconfig-shared.c |   72 ++++++++++++++++++++++------------------
 1 files changed, 40 insertions(+), 32 deletions(-)

diff --git a/arch/x86/pci/mmconfig-shared.c b/arch/x86/pci/mmconfig-shared.c
index 06da933..e9a791c 100644
--- a/arch/x86/pci/mmconfig-shared.c
+++ b/arch/x86/pci/mmconfig-shared.c
@@ -566,8 +566,6 @@ static void __init pci_mmcfg_reject_broken(int early)
        }
 }

-static int __initdata known_bridge;
-
 static int __init acpi_mcfg_check_entry(struct acpi_table_mcfg *mcfg,
                                        struct acpi_mcfg_allocation *cfg)
 {
@@ -647,25 +645,10 @@ static int __init pci_parse_mcfg(struct acpi_table_header *header)
        return 0;
 }

-static void __init __pci_mmcfg_init(int early)
-{
-       /* MMCONFIG disabled */
-       if ((pci_probe & PCI_PROBE_MMCONF) == 0)
-               return;
-
-       /* for late to exit */
-       if (known_bridge)
-               return;
-
-       /* MMCONFIG already enabled */
-       if (!early && !(pci_probe & PCI_PROBE_MASK & ~PCI_PROBE_MMCONF))
-               goto out;
-
-       if (early) {
-               if (pci_mmcfg_check_hostbridge())
-                       known_bridge = 1;
-       }
+static int __initdata known_bridge;

+static void __init __pci_mmcfg_init(int early, bool create_on_demand)
+{
        if (!known_bridge)
                acpi_sfi_table_parse(ACPI_SIG_MCFG, pci_parse_mcfg);

@@ -684,6 +667,13 @@ static void __init __pci_mmcfg_init(int early)
                }
        }

+       /*
+        * Avoid redundant pci_mmcfg_arch_map()/pci_mmcfg_arch_unmap() calls
+        * for each MMCONFIG entry if they will be created on demand.
+        */
+       if (create_on_demand)
+               free_all_mmcfg();
+
        if (pci_mmcfg_arch_init())
                pci_probe = (pci_probe & ~PCI_PROBE_MASK) | PCI_PROBE_MMCONF;
        else {
@@ -694,26 +684,44 @@ static void __init __pci_mmcfg_init(int early)
                pci_mmcfg_resources_inserted = 1;
                pci_mmcfg_arch_init_failed = true;
        }
-
-out:
-       /*
-        * Free all MCFG entries if ACPI is enabled. MCFG information will
-        * be added back on demand by the pci_root driver later.
-        */
-       if (!early && !acpi_disabled && !known_bridge &&
-           !pci_mmcfg_arch_init_failed)
-               if (!acpi_pci_cache_mcfg())
-                       free_all_mmcfg();
 }

 void __init pci_mmcfg_early_init(void)
 {
-       __pci_mmcfg_init(1);
+       if (pci_probe & PCI_PROBE_MMCONF) {
+               if (pci_mmcfg_check_hostbridge())
+                       known_bridge = 1;
+               __pci_mmcfg_init(1, false);
+       }
 }

 void __init pci_mmcfg_late_init(void)
 {
-       __pci_mmcfg_init(0);
+       bool create_on_demand;
+
+       /* MMCONFIG disabled */
+       if ((pci_probe & PCI_PROBE_MMCONF) == 0)
+               return;
+
+       /* Don't touch hardcoded MMCONFIG information */
+       if (known_bridge)
+               return;
+
+       /*
+        * MMCONFIG information will be created on demand by pci_root driver
+        * when binding to ACPI host bridge deivces.
+        */
+       create_on_demand = (!acpi_disabled && !acpi_pci_cache_mcfg());
+
+       /* pci_mmcfg_early_init() fails to setup MMCONFIG, try again. */
+       if (pci_probe & PCI_PROBE_MASK & ~PCI_PROBE_MMCONF)
+               __pci_mmcfg_init(0, create_on_demand);
+       /*
+        * Free MMCONFIG information created by pci_mmcfg_early_init()
+        * if MMCONFIG information will be created on demand.
+        */
+       else if (create_on_demand)
+               free_all_mmcfg();
 }

 static int __init pci_mmcfg_late_insert_resources(void)
--
1.7.1


On 2012-6-16 0:55, Yinghai Lu wrote:
> On Fri, Jun 15, 2012 at 8:46 AM, Bjorn Helgaas <bhelgaas@google.com> wrote:
>>
>>> But the code in the init path is ridiculously, embarrassingly complicated ...
>>
>> I rest my case.  The current init path is unmaintainable.
>>
>> I'll wait for a v8 (or later) with fixes for the build issues
>> Fengguang found and Yinghai's ack.  It will make things somewhat
>> easier for me if you start with my topic/jiang-mmconfig-v7 branch.
> 
> yes. probe/free then install is not good.
> 
> should make __pci_mmcfg_init() to cache needed stuff for all path, and
> then consume stored
> entry just before scan root bus.
> 
> Thanks
> 
> Yinghai
> 
> 

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Jiang Liu]

On 2012-6-16 0:51, Yinghai Lu wrote:
> 
> still have some problem:
> pci_mmcfg_check_reserved==>is_mmconf_reserved
> 
> will update cfg->end_bus.
Hi Yinghai,
	How about following patch for this issue? I guess we need to
keep current behavior at boot time for backward compatibility, right?

---
@@ -472,6 +472,14 @@ static int __devinit is_mmconf_reserved(check_reserved_t is_reserved,
                        break;
        }

+       /*
+        * For backward compatibility, we will adjust the MMCONFIG region
+        * at boot time if it's only partially reserved by firmware.
+        * For PCI host bridge hotplug at runtime, just reject it.
+        */
+       if (pci_mmcfg_running_state && old_size != size)
+               return 0;
+
        if (size < (16UL<<20) && size != old_size)
                return 0;

---

> 
> Thanks
> 
> Yinghai
> 
> .
> 

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Yinghai Lu]

[-- Attachment #1: Type: text/plain, Size: 1123 bytes --]

On Sat, Jun 16, 2012 at 2:23 AM, Jiang Liu <jiang.liu@huawei.com> wrote:
> On 2012-6-16 0:51, Yinghai Lu wrote:
>>
>> still have some problem:
>> pci_mmcfg_check_reserved==>is_mmconf_reserved
>>
>> will update cfg->end_bus.
> Hi Yinghai,
>        How about following patch for this issue? I guess we need to
> keep current behavior at boot time for backward compatibility, right?

We'd better to make all path share as most code as possible.
1. hostbridge scanning during boot -- early, it will check chipset and e820
2. MCFG checking during boot -- early, it will check e820
3. MCFG checking during boot -- late, it will check acpi pnp
4. _CBA checking for hotplug-able pci root bus but it is installed during boot.
5. _CBA checking for hotplug-able pci root bus during run time.

please keep mapping for all entries in MCFG table. aka 1, 2, 3.
I have some local patches that will read ext pci conf space before scan pci bus.
please check attached one for nehalem-ioh.

for case 4, 5: only need to add one bool in acpi_pci_root to record if
mmconf list and mapping is touched.

Thanks

Yinghai

[-- Attachment #2: intel_bus_new_all.patch --]
[-- Type: application/octet-stream, Size: 9090 bytes --]

Subject: [PATCH] x86/pci: ioh new version read all at same time

Also it will add back default range to legacy IOH

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
 arch/x86/include/asm/pci_x86.h |    5 
 arch/x86/pci/Makefile          |    1 
 arch/x86/pci/init.c            |    2 
 arch/x86/pci/intel_bus.c       |  260 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 268 insertions(+)

Index: linux-2.6/arch/x86/include/asm/pci_x86.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/pci_x86.h
+++ linux-2.6/arch/x86/include/asm/pci_x86.h
@@ -111,6 +111,11 @@ extern void pci_direct_init(int type);
 extern void pci_pcbios_init(void);
 extern void __init dmi_check_pciprobe(void);
 extern void __init dmi_check_skip_isa_align(void);
+#ifdef CONFIG_PCI_MMCONFIG
+int intel_postarch_init(void);
+#else
+static inline int intel_postarch_init(void) { return 0; }
+#endif
 
 /* some common used subsys_initcalls */
 extern int __init pci_acpi_init(void);
Index: linux-2.6/arch/x86/pci/Makefile
===================================================================
--- linux-2.6.orig/arch/x86/pci/Makefile
+++ linux-2.6/arch/x86/pci/Makefile
@@ -21,6 +21,7 @@ obj-$(CONFIG_X86_INTEL_MID)	+= mrst.o
 
 obj-y				+= common.o early.o
 obj-y				+= bus_numa.o
+obj-$(CONFIG_PCI_MMCONFIG)	+= intel_bus.o
 
 obj-$(CONFIG_AMD_NB)		+= amd_bus.o
 obj-$(CONFIG_PCI_CNB20LE_QUIRK)	+= broadcom_bus.o
Index: linux-2.6/arch/x86/pci/init.c
===================================================================
--- linux-2.6.orig/arch/x86/pci/init.c
+++ linux-2.6/arch/x86/pci/init.c
@@ -39,6 +39,8 @@ static __init int pci_arch_init(void)
 
 	dmi_check_skip_isa_align();
 
+	intel_postarch_init();
+
 	return 0;
 }
 arch_initcall(pci_arch_init);
Index: linux-2.6/arch/x86/pci/intel_bus.c
===================================================================
--- /dev/null
+++ linux-2.6/arch/x86/pci/intel_bus.c
@@ -0,0 +1,260 @@
+/*
+ * to read io range from IOH pci conf, need to do it after mmconfig is there
+ */
+
+#include <linux/delay.h>
+#include <linux/dmi.h>
+#include <linux/pci.h>
+#include <linux/init.h>
+#include <linux/range.h>
+
+#include <asm/pci_x86.h>
+
+#include "bus_numa.h"
+
+static void __devinit subtract_mmconf(struct range *range, int nr)
+{
+	struct pci_mmcfg_region *cfg;
+
+	if (list_empty(&pci_mmcfg_list))
+		return;
+
+	list_for_each_entry(cfg, &pci_mmcfg_list, list)
+		subtract_range(range, nr, cfg->res.start, cfg->res.end + 1);
+}
+
+#define RANGE_NUM		16
+
+#define RANGE_IO_NUM		16
+#define RANGE_MMIO_NUM		32
+static struct range range_io[RANGE_IO_NUM] __initdata;
+static struct range range_mmio[RANGE_MMIO_NUM] __initdata;
+static struct pci_root_info *def_root_info;
+
+/* IOH related */
+#define IOH_LIO			0x108
+#define IOH_LMMIOL		0x10c
+#define IOH_LMMIOH		0x110
+#define IOH_LMMIOH_BASEU	0x114
+#define IOH_LMMIOH_LIMITU	0x118
+#define IOH_LCFGBUS		0x11c
+
+#define IOH_VTBAR		0x180
+#define IOH_VTSIZE		0x2000  /* Fixed HW size (not programmable) */
+
+static bool __init check_ioh_tom(int num, int slot, int func)
+{
+	u32 dword;
+	u64 tocm, tolm, tohm;
+	bool def_ioh = false;
+
+	raw_pci_read(0, num, (slot<<3)|func, 0x98, 4, &dword);
+	/* is Legacy IOH with ESI? */
+	if ((dword & (3<<10)) == 0)
+		def_ioh = true;
+
+	/* top of address */
+	tocm = 1ULL<<(((dword >> 3) & 0x1f) + (37 - 5));
+	subtract_range(range_mmio, RANGE_MMIO_NUM, tocm, -1ULL);
+	/* private CSR 64G */
+	subtract_range(range_mmio, RANGE_MMIO_NUM, tocm - (64ULL<<30), tocm);
+	/* top of low mem */
+	raw_pci_read(0, num, (slot<<3)|func, 0xd0, 4, &dword);
+	tolm = dword & (0x3f<<26);
+	tolm += 1<<26;
+	subtract_range(range_mmio, RANGE_MMIO_NUM, 0, tolm);
+	/* top of high mem */
+	raw_pci_read(0, num, (slot<<3)|func, 0xd8, 4, &dword);
+	tohm = dword;
+	tohm <<= 32;
+	raw_pci_read(0, num, (slot<<3)|func, 0xd4, 4, &dword);
+	tohm |= dword & (0x3f<<26);
+	tohm += 1<<26;
+	subtract_range(range_mmio, RANGE_MMIO_NUM, 1ULL<<32, tohm);
+	printk(KERN_DEBUG "IOH bus 0x%02x tolm: 0x%llxM, tohm: 0x%llxM, tocm: 0x%llxM\n",
+			 num, tolm>>20, tohm>>20, tocm>>20);
+
+	return def_ioh;
+}
+
+static void __init read_ioh_res(int num, int slot, int func)
+{
+	u32 dword;
+	struct pci_root_info *info;
+	u16 io_base, io_end;
+	u32 mmiol_base, mmiol_end;
+	u64 mmioh_base, mmioh_end;
+	int bus_base, bus_end;
+	struct range range[RANGE_NUM];
+	int i;
+
+	raw_pci_read(0, num, (slot<<3)|func, IOH_LCFGBUS, 2, &dword);
+	bus_base = (dword & 0xff);
+	bus_end = (dword & 0xff00) >> 8;
+	info = alloc_pci_root_info(bus_base, bus_end, 0, 0);
+	if (!info) {
+		printk(KERN_DEBUG
+			"intel_bus.c: can not allocate pci_root_info\n");
+		return;
+	}
+
+	if (check_ioh_tom(num, slot, func)) {
+		if (!def_root_info)
+			def_root_info = info;
+		else
+			printk(KERN_DEBUG "Multiple legacy IOHs ?\n");
+	}
+
+	raw_pci_read(0, num, (slot<<3)|func, IOH_LIO, 2, &dword);
+	io_base = (dword & 0xf0) << (12 - 4);
+	io_end = (dword & 0xf000) | 0xfff;
+	update_res(info, io_base, io_end, IORESOURCE_IO, 0);
+	subtract_range(range_io, RANGE_IO_NUM, io_base, io_end + 1);
+
+	raw_pci_read(0, num, (slot<<3)|func, IOH_LMMIOL, 4, &dword);
+	mmiol_base = (dword & 0xff00) << (24 - 8);
+	mmiol_end = (dword & 0xff000000) | 0xffffff;
+	subtract_range(range_mmio, RANGE_MMIO_NUM, mmiol_base, mmiol_end + 1);
+	memset(range, 0, sizeof(range));
+	add_range(range, RANGE_NUM, 0, mmiol_base, (u64)mmiol_end + 1);
+	raw_pci_read(0, num, (slot<<3)|func, IOH_VTBAR, 4, &dword);
+	if (dword & 0x1) {
+		u32 vt_base, vt_end;
+
+		vt_base = dword & 0xfffffffe;
+		vt_end = vt_base + IOH_VTSIZE - 1;
+
+		subtract_range(range, RANGE_NUM, vt_base, vt_end + 1);
+		subtract_range(range_mmio, RANGE_MMIO_NUM, vt_base, vt_end + 1);
+	}
+	for (i = 0; i < RANGE_NUM; i++) {
+		if (!range[i].end)
+			continue;
+
+		update_res(info, cap_resource(range[i].start),
+				cap_resource(range[i].end - 1),
+				IORESOURCE_MEM, 0);
+	}
+
+	raw_pci_read(0, num, (slot<<3)|func, IOH_LMMIOH, 4, &dword);
+	mmioh_base = ((u64)(dword & 0xfc00)) << (26 - 10);
+	mmioh_end = ((u64)(dword & 0xfc000000) | 0x3ffffff);
+	raw_pci_read(0, num, (slot<<3)|func, IOH_LMMIOH_BASEU, 4, &dword);
+	mmioh_base |= ((u64)(dword & 0x7ffff)) << 32;
+	raw_pci_read(0, num, (slot<<3)|func, IOH_LMMIOH_LIMITU, 4, &dword);
+	mmioh_end |= ((u64)(dword & 0x7ffff)) << 32;
+	update_res(info, cap_resource(mmioh_base), cap_resource(mmioh_end),
+			 IORESOURCE_MEM, 0);
+	subtract_range(range_mmio, RANGE_MMIO_NUM, mmioh_base, mmioh_end + 1);
+}
+
+struct pci_check_probe {
+	u32 vendor;
+	u32 device;
+	void (*f)(int num, int slot, int func);
+};
+
+static struct pci_check_probe early_qrk[] __initdata = {
+	{ PCI_VENDOR_ID_INTEL, 0x342e, read_ioh_res }, /* intel IOH */
+	{}
+};
+
+static void __init postarch_check_pci_dev(int num, int slot, int func)
+{
+	u32 vendor;
+	u32 device;
+	int i;
+
+	raw_pci_read(0, num, (slot<<3)|func, PCI_VENDOR_ID, 2, &vendor);
+	raw_pci_read(0, num, (slot<<3)|func, PCI_DEVICE_ID, 2, &device);
+
+	for (i = 0; early_qrk[i].f != NULL; i++) {
+		if (((early_qrk[i].vendor == PCI_ANY_ID) ||
+			(early_qrk[i].vendor == vendor)) &&
+			((early_qrk[i].device == PCI_ANY_ID) ||
+			(early_qrk[i].device == device)))
+				early_qrk[i].f(num, slot, func);
+	}
+}
+
+static void __init postarch_check_pci_devs(void)
+{
+	unsigned bus, slot, func;
+	struct pci_root_info *info;
+	int i;
+
+	memset(range_io, 0, sizeof(range_io));
+	add_range(range_io, RANGE_IO_NUM, 0, 0, 0xffff + 1);
+
+	memset(range_mmio, 0, sizeof(range_mmio));
+	add_range(range_mmio, RANGE_MMIO_NUM, 0, 0, -1ULL);
+
+	for (bus = 0; bus < 256; bus++) {
+		for (slot = 0; slot < 32; slot++) {
+			for (func = 0; func < 8; func++) {
+				u32 class;
+				u32 type;
+
+				raw_pci_read(0, bus, (slot<<3)|func,
+						PCI_CLASS_REVISION, 4, &class);
+				if (class == 0xffffffff)
+					continue;
+
+				postarch_check_pci_dev(bus, slot, func);
+
+				if (func == 0) {
+					raw_pci_read(0, bus, (slot<<3)|func,
+					       PCI_HEADER_TYPE, 1, &type);
+					if (!(type & 0x80))
+						break;
+				}
+			}
+		}
+	}
+
+	print_all_pci_root_info("root bus", false);
+
+	if (!def_root_info)
+		return;
+
+	/* add default io */
+	info = def_root_info;
+	for (i = 0; i < RANGE_IO_NUM; i++) {
+		if (!range_io[i].end)
+			continue;
+
+		update_res(info, range_io[i].start, range_io[i].end - 1,
+			   IORESOURCE_IO, 0);
+	}
+
+	subtract_mmconf(range_mmio, RANGE_MMIO_NUM);
+
+	/* add default default mmio */
+	for (i = 0; i < RANGE_MMIO_NUM; i++) {
+		if (!range_mmio[i].end)
+			continue;
+
+		update_res(info, cap_resource(range_mmio[i].start),
+			   cap_resource(range_mmio[i].end - 1),
+			   IORESOURCE_MEM, 0);
+	}
+	printk(KERN_DEBUG "Legacy final with default routing:\n");
+	print_pci_root_info(info, "Legacy bus", false);
+}
+
+/*
+ * need to call it just after pci_arch_init
+ * so we can have mmconf ready
+ */
+int __init intel_postarch_init(void)
+{
+	if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL)
+		return 0;
+
+	if (!pci_ext_cfg_avail(NULL))
+		return 0;
+
+	postarch_check_pci_devs();
+
+	return 0;
+}

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Bjorn Helgaas]

On Sat, Jun 16, 2012 at 2:08 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> On Sat, Jun 16, 2012 at 2:23 AM, Jiang Liu <jiang.liu@huawei.com> wrote:
>> On 2012-6-16 0:51, Yinghai Lu wrote:
>>>
>>> still have some problem:
>>> pci_mmcfg_check_reserved==>is_mmconf_reserved
>>>
>>> will update cfg->end_bus.
>> Hi Yinghai,
>>        How about following patch for this issue? I guess we need to
>> keep current behavior at boot time for backward compatibility, right?
>
> We'd better to make all path share as most code as possible.
> 1. hostbridge scanning during boot -- early, it will check chipset and e820
> 2. MCFG checking during boot -- early, it will check e820
> 3. MCFG checking during boot -- late, it will check acpi pnp
> 4. _CBA checking for hotplug-able pci root bus but it is installed during boot.
> 5. _CBA checking for hotplug-able pci root bus during run time.
>
> please keep mapping for all entries in MCFG table. aka 1, 2, 3.
> I have some local patches that will read ext pci conf space before scan pci bus.
> please check attached one for nehalem-ioh.

I don't think it's a requirement that Gerry keep your Nehalem patch
working.  Your intel_bus.c is not in the tree and you haven't provided
an explanation for why it should be.

The only requirement I'm aware of for PCI config access before we
discover the host bridges via ACPI is for segment group 0, bus 0, as
mentioned in ACPI spec 5.0, sec 5.2.3.1, PDF page 143, and I think
that applies only to the first 0x100 bytes of config space.  I don't
think there's a requirement for access to the extended configuration
space (bytes 0x100-0xFFF).  I do not see a requirement that this
pre-host bridge access happen via MMCONFIG; as far as I can tell, we
can use the legacy 0xCF8/0xCFC mechanism.

> for case 4, 5: only need to add one bool in acpi_pci_root to record if
> mmconf list and mapping is touched.
>
> Thanks
>
> Yinghai

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Yinghai Lu]

On Sat, Jun 16, 2012 at 2:48 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
>>
>> We'd better to make all path share as most code as possible.
>> 1. hostbridge scanning during boot -- early, it will check chipset and e820
>> 2. MCFG checking during boot -- early, it will check e820
>> 3. MCFG checking during boot -- late, it will check acpi pnp
>> 4. _CBA checking for hotplug-able pci root bus but it is installed during boot.
>> 5. _CBA checking for hotplug-able pci root bus during run time.
>>
>> please keep mapping for all entries in MCFG table. aka 1, 2, 3.
>> I have some local patches that will read ext pci conf space before scan pci bus.
>> please check attached one for nehalem-ioh.
>
> I don't think it's a requirement that Gerry keep your Nehalem patch
> working.  Your intel_bus.c is not in the tree and you haven't provided
> an explanation for why it should be.
>
> The only requirement I'm aware of for PCI config access before we
> discover the host bridges via ACPI is for segment group 0, bus 0, as
> mentioned in ACPI spec 5.0, sec 5.2.3.1, PDF page 143, and I think
> that applies only to the first 0x100 bytes of config space.  I don't
> think there's a requirement for access to the extended configuration
> space (bytes 0x100-0xFFF).  I do not see a requirement that this
> pre-host bridge access happen via MMCONFIG; as far as I can tell, we
> can use the legacy 0xCF8/0xCFC mechanism.

that one shot for intel host bridge resource discovery before root bus
scanning,
will need to access registers above 0x100.

Current MCFG handling have some sanitary checking during probing.

Now Jiang is trying to free result and cache it for two MCFG path 2/3.
and later use cached and map again for entries from cached entries.
but when use those cached entries sanitary of those entries are lost.

so choice would be
1. cache all checked MMCFG result and use that later
2. or just leave current MCFG handling alone, just add _CBA support.
like Jiang -v4 version.

Thanks

Yinghai

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Yinghai Lu]

[-- Attachment #1: Type: text/plain, Size: 598 bytes --]

On Sat, Jun 16, 2012 at 3:44 PM, Yinghai Lu <yinghai@kernel.org> wrote:

> Current MCFG handling have some sanitary checking during probing.
>
> Now Jiang is trying to free result and cache it for two MCFG path 2/3.
> and later use cached and map again for entries from cached entries.
> but when use those cached entries sanitary of those entries are lost.
>
> so choice would be
> 1. cache all checked MMCFG result and use that later
> 2. or just leave current MCFG handling alone, just add _CBA support.
> like Jiang -v4 version.
>

I have used updated jiang's -v4 version for a while.

Yinghai

[-- Attachment #2: jiang_mmconf_v4_1.patch --]
[-- Type: application/octet-stream, Size: 2626 bytes --]

From: Jiang Liu <liuj97@gmail.com>
To: Yinghai Lu <yinghai@kernel.org>,
	Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>,
	Taku Izumi <izumi.taku@jp.fujitsu.com>
Cc: Jiang Liu <jiang.liu@huawei.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Jiang Liu <liuj97@gmail.com>,
	Keping Chen <chenkeping@huawei.com>,
	linux-pci@vger.kernel.org
Subject: [PATCH V4 1/6] PCI, x86: split out pci_mmcfg_check_reserved() for code reuse
Date: Wed, 11 Apr 2012 08:10:58 +0800
Message-Id: <1334103063-2283-2-git-send-email-jiang.liu@huawei.com>

Split out pci_mmcfg_check_reserved() for code reuse, which will be used
when supporting PCI host bridge hotplug.

Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
---
 arch/x86/pci/mmconfig-shared.c |   51 ++++++++++++++++++++---------------------
 1 file changed, 25 insertions(+), 26 deletions(-)

Index: linux-2.6/arch/x86/pci/mmconfig-shared.c
===================================================================
--- linux-2.6.orig/arch/x86/pci/mmconfig-shared.c
+++ linux-2.6/arch/x86/pci/mmconfig-shared.c
@@ -474,39 +474,38 @@ static int __init is_mmconf_reserved(che
 	return valid;
 }
 
+static int __devinit pci_mmcfg_check_reserved(struct pci_mmcfg_region *cfg,
+					      int early)
+{
+	if (!early && !acpi_disabled) {
+		if (is_mmconf_reserved(is_acpi_reserved, cfg, 0))
+			return 1;
+		else
+			printk(KERN_ERR FW_BUG PREFIX
+			       "MMCONFIG at %pR not reserved in "
+			       "ACPI motherboard resources\n",
+			       &cfg->res);
+	}
+
+	/* Don't try to do this check unless configuration
+	   type 1 is available. how about type 2 ?*/
+	if (raw_pci_ops)
+		return is_mmconf_reserved(e820_all_mapped, cfg, 1);
+
+	return 0;
+}
+
 static void __init pci_mmcfg_reject_broken(int early)
 {
 	struct pci_mmcfg_region *cfg;
 
 	list_for_each_entry(cfg, &pci_mmcfg_list, list) {
-		int valid = 0;
-
-		if (!early && !acpi_disabled) {
-			valid = is_mmconf_reserved(is_acpi_reserved, cfg, 0);
-
-			if (valid)
-				continue;
-			else
-				printk(KERN_ERR FW_BUG PREFIX
-				       "MMCONFIG at %pR not reserved in "
-				       "ACPI motherboard resources\n",
-				       &cfg->res);
+		if (pci_mmcfg_check_reserved(cfg, early) == 0) {
+			printk(KERN_INFO PREFIX "not using MMCONFIG\n");
+			free_all_mmcfg();
+			return;
 		}
-
-		/* Don't try to do this check unless configuration
-		   type 1 is available. how about type 2 ?*/
-		if (raw_pci_ops)
-			valid = is_mmconf_reserved(e820_all_mapped, cfg, 1);
-
-		if (!valid)
-			goto reject;
 	}
-
-	return;
-
-reject:
-	printk(KERN_INFO PREFIX "not using MMCONFIG\n");
-	free_all_mmcfg();
 }
 
 static int __initdata known_bridge;

[-- Attachment #3: jiang_mmconf_v4_2.patch --]
[-- Type: application/octet-stream, Size: 2080 bytes --]

From: Jiang Liu <liuj97@gmail.com>
To: Yinghai Lu <yinghai@kernel.org>,
	Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>,
	Taku Izumi <izumi.taku@jp.fujitsu.com>
Cc: Jiang Liu <jiang.liu@huawei.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Jiang Liu <liuj97@gmail.com>,
	Keping Chen <chenkeping@huawei.com>,
	linux-pci@vger.kernel.org
Subject: [PATCH V4 2/6] PCI, x86: split out pci_mmconfig_alloc() for code reuse
Date: Wed, 11 Apr 2012 08:10:59 +0800
Message-Id: <1334103063-2283-3-git-send-email-jiang.liu@huawei.com>

Split out pci_mmconfig_alloc() for code reuse, which will be used
when supporting PCI root bridge hotplug.

Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
---
 arch/x86/pci/mmconfig-shared.c |   19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

Index: linux-2.6/arch/x86/pci/mmconfig-shared.c
===================================================================
--- linux-2.6.orig/arch/x86/pci/mmconfig-shared.c
+++ linux-2.6/arch/x86/pci/mmconfig-shared.c
@@ -61,8 +61,9 @@ static __init void list_add_sorted(struc
 	list_add_tail(&new->list, &pci_mmcfg_list);
 }
 
-static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
-							int end, u64 addr)
+static __devinit struct pci_mmcfg_region *pci_mmconfig_alloc(int segment,
+							     int start,
+							     int end, u64 addr)
 {
 	struct pci_mmcfg_region *new;
 	struct resource *res;
@@ -79,8 +80,6 @@ static __init struct pci_mmcfg_region *p
 	new->start_bus = start;
 	new->end_bus = end;
 
-	list_add_sorted(new);
-
 	res = &new->res;
 	res->start = addr + PCI_MMCFG_BUS_OFFSET(start);
 	res->end = addr + PCI_MMCFG_BUS_OFFSET(end + 1) - 1;
@@ -95,6 +94,18 @@ static __init struct pci_mmcfg_region *p
 
 	return new;
 }
+
+static __init struct pci_mmcfg_region *pci_mmconfig_add(int segment, int start,
+							int end, u64 addr)
+{
+	struct pci_mmcfg_region *new;
+
+	new = pci_mmconfig_alloc(segment, start, end, addr);
+	if (new)
+		list_add_sorted(new);
+
+	return new;
+}
 
 struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus)
 {

[-- Attachment #4: jiang_mmconf_v4_3.patch --]
[-- Type: application/octet-stream, Size: 4850 bytes --]

From: Jiang Liu <liuj97@gmail.com>
To: Yinghai Lu <yinghai@kernel.org>,
	Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>,
	Taku Izumi <izumi.taku@jp.fujitsu.com>
Cc: Jiang Liu <jiang.liu@huawei.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Jiang Liu <liuj97@gmail.com>,
	Keping Chen <chenkeping@huawei.com>,
	linux-pci@vger.kernel.org
Subject: [PATCH V4 3/6] PCI, x86: use RCU list to protect mmconfig list
Date: Wed, 11 Apr 2012 08:11:00 +0800
Message-Id: <1334103063-2283-4-git-send-email-jiang.liu@huawei.com>

Use RCU list to protect mmconfig list from dynamic change
when supporting PCI host bridge hotplug.

Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
---
 arch/x86/pci/mmconfig-shared.c |   11 ++++++-----
 arch/x86/pci/mmconfig_32.c     |   13 +++++++++++--
 arch/x86/pci/mmconfig_64.c     |   13 +++++++++++--
 3 files changed, 28 insertions(+), 9 deletions(-)

Index: linux-2.6/arch/x86/pci/mmconfig-shared.c
===================================================================
--- linux-2.6.orig/arch/x86/pci/mmconfig-shared.c
+++ linux-2.6/arch/x86/pci/mmconfig-shared.c
@@ -17,6 +17,7 @@
 #include <linux/bitmap.h>
 #include <linux/dmi.h>
 #include <linux/slab.h>
+#include <linux/rculist.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
 #include <asm/acpi.h>
@@ -45,20 +46,20 @@ static __init void free_all_mmcfg(void)
 		pci_mmconfig_remove(cfg);
 }
 
-static __init void list_add_sorted(struct pci_mmcfg_region *new)
+static __devinit void list_add_sorted(struct pci_mmcfg_region *new)
 {
 	struct pci_mmcfg_region *cfg;
 
 	/* keep list sorted by segment and starting bus number */
-	list_for_each_entry(cfg, &pci_mmcfg_list, list) {
+	list_for_each_entry_rcu(cfg, &pci_mmcfg_list, list) {
 		if (cfg->segment > new->segment ||
 		    (cfg->segment == new->segment &&
 		     cfg->start_bus >= new->start_bus)) {
-			list_add_tail(&new->list, &cfg->list);
+			list_add_tail_rcu(&new->list, &cfg->list);
 			return;
 		}
 	}
-	list_add_tail(&new->list, &pci_mmcfg_list);
+	list_add_tail_rcu(&new->list, &pci_mmcfg_list);
 }
 
 static __devinit struct pci_mmcfg_region *pci_mmconfig_alloc(int segment,
@@ -111,7 +112,7 @@ struct pci_mmcfg_region *pci_mmconfig_lo
 {
 	struct pci_mmcfg_region *cfg;
 
-	list_for_each_entry(cfg, &pci_mmcfg_list, list)
+	list_for_each_entry_rcu(cfg, &pci_mmcfg_list, list)
 		if (cfg->segment == segment &&
 		    cfg->start_bus <= bus && bus <= cfg->end_bus)
 			return cfg;
Index: linux-2.6/arch/x86/pci/mmconfig_32.c
===================================================================
--- linux-2.6.orig/arch/x86/pci/mmconfig_32.c
+++ linux-2.6/arch/x86/pci/mmconfig_32.c
@@ -11,6 +11,7 @@
 
 #include <linux/pci.h>
 #include <linux/init.h>
+#include <linux/rcupdate.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
 #include <acpi/acpi.h>
@@ -60,9 +61,12 @@ err:		*value = -1;
 		return -EINVAL;
 	}
 
+	rcu_read_lock();
 	base = get_base_addr(seg, bus, devfn);
-	if (!base)
+	if (!base) {
+		rcu_read_unlock();
 		goto err;
+	}
 
 	raw_spin_lock_irqsave(&pci_config_lock, flags);
 
@@ -80,6 +84,7 @@ err:		*value = -1;
 		break;
 	}
 	raw_spin_unlock_irqrestore(&pci_config_lock, flags);
+	rcu_read_unlock();
 
 	return 0;
 }
@@ -93,9 +98,12 @@ static int pci_mmcfg_write(unsigned int
 	if ((bus > 255) || (devfn > 255) || (reg > 4095))
 		return -EINVAL;
 
+	rcu_read_lock();
 	base = get_base_addr(seg, bus, devfn);
-	if (!base)
+	if (!base) {
+		rcu_read_unlock();
 		return -EINVAL;
+	}
 
 	raw_spin_lock_irqsave(&pci_config_lock, flags);
 
@@ -113,6 +121,7 @@ static int pci_mmcfg_write(unsigned int
 		break;
 	}
 	raw_spin_unlock_irqrestore(&pci_config_lock, flags);
+	rcu_read_unlock();
 
 	return 0;
 }
Index: linux-2.6/arch/x86/pci/mmconfig_64.c
===================================================================
--- linux-2.6.orig/arch/x86/pci/mmconfig_64.c
+++ linux-2.6/arch/x86/pci/mmconfig_64.c
@@ -9,6 +9,7 @@
 #include <linux/init.h>
 #include <linux/acpi.h>
 #include <linux/bitmap.h>
+#include <linux/rcupdate.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
 
@@ -34,9 +35,12 @@ err:		*value = -1;
 		return -EINVAL;
 	}
 
+	rcu_read_lock();
 	addr = pci_dev_base(seg, bus, devfn);
-	if (!addr)
+	if (!addr) {
+		rcu_read_unlock();
 		goto err;
+	}
 
 	switch (len) {
 	case 1:
@@ -49,6 +53,7 @@ err:		*value = -1;
 		*value = mmio_config_readl(addr + reg);
 		break;
 	}
+	rcu_read_unlock();
 
 	return 0;
 }
@@ -62,9 +67,12 @@ static int pci_mmcfg_write(unsigned int
 	if (unlikely((bus > 255) || (devfn > 255) || (reg > 4095)))
 		return -EINVAL;
 
+	rcu_read_lock();
 	addr = pci_dev_base(seg, bus, devfn);
-	if (!addr)
+	if (!addr) {
+		rcu_read_unlock();
 		return -EINVAL;
+	}
 
 	switch (len) {
 	case 1:
@@ -77,6 +85,7 @@ static int pci_mmcfg_write(unsigned int
 		mmio_config_writel(addr + reg, value);
 		break;
 	}
+	rcu_read_unlock();
 
 	return 0;
 }

[-- Attachment #5: jiang_mmconf_v4_4.patch --]
[-- Type: application/octet-stream, Size: 3153 bytes --]

From: Jiang Liu <liuj97@gmail.com>
To: Yinghai Lu <yinghai@kernel.org>,
	Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>,
	Taku Izumi <izumi.taku@jp.fujitsu.com>
Cc: Jiang Liu <jiang.liu@huawei.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Jiang Liu <liuj97@gmail.com>,
	Keping Chen <chenkeping@huawei.com>,
	linux-pci@vger.kernel.org
Subject: [PATCH V4 4/6] PCI, x86: introduce pci_mmcfg_arch_map()/pci_mmcfg_arch_unmap()
Date: Wed, 11 Apr 2012 08:11:01 +0800
Message-Id: <1334103063-2283-5-git-send-email-jiang.liu@huawei.com>

Introduce pci_mmcfg_arch_map()/pci_mmcfg_arch_unmap(), which will be used
when supporting PCI root bridge hotplug.

Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
---
 arch/x86/include/asm/pci_x86.h |    2 ++
 arch/x86/pci/mmconfig_32.c     |   15 +++++++++++++++
 arch/x86/pci/mmconfig_64.c     |   22 +++++++++++++++++++++-
 3 files changed, 38 insertions(+), 1 deletion(-)

Index: linux-2.6/arch/x86/include/asm/pci_x86.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/pci_x86.h
+++ linux-2.6/arch/x86/include/asm/pci_x86.h
@@ -139,6 +139,8 @@ struct pci_mmcfg_region {
 
 extern int __init pci_mmcfg_arch_init(void);
 extern void __init pci_mmcfg_arch_free(void);
+extern int __devinit pci_mmcfg_arch_map(struct pci_mmcfg_region *cfg);
+extern void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg);
 extern struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus);
 
 extern struct list_head pci_mmcfg_list;
Index: linux-2.6/arch/x86/pci/mmconfig_32.c
===================================================================
--- linux-2.6.orig/arch/x86/pci/mmconfig_32.c
+++ linux-2.6/arch/x86/pci/mmconfig_32.c
@@ -141,3 +141,18 @@ int __init pci_mmcfg_arch_init(void)
 void __init pci_mmcfg_arch_free(void)
 {
 }
+
+int __devinit pci_mmcfg_arch_map(struct pci_mmcfg_region *cfg)
+{
+	return 0;
+}
+
+void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg)
+{
+	unsigned long flags;
+
+	/* Invalidate the cached mmcfg map entry. */
+	raw_spin_lock_irqsave(&pci_config_lock, flags);
+	mmcfg_last_accessed_device = 0;
+	raw_spin_unlock_irqrestore(&pci_config_lock, flags);
+}
Index: linux-2.6/arch/x86/pci/mmconfig_64.c
===================================================================
--- linux-2.6.orig/arch/x86/pci/mmconfig_64.c
+++ linux-2.6/arch/x86/pci/mmconfig_64.c
@@ -95,7 +95,7 @@ static const struct pci_raw_ops pci_mmcf
 	.write =	pci_mmcfg_write,
 };
 
-static void __iomem * __init mcfg_ioremap(struct pci_mmcfg_region *cfg)
+static void __iomem * __devinit mcfg_ioremap(struct pci_mmcfg_region *cfg)
 {
 	void __iomem *addr;
 	u64 start, size;
@@ -138,3 +138,23 @@ void __init pci_mmcfg_arch_free(void)
 		}
 	}
 }
+
+int __devinit pci_mmcfg_arch_map(struct pci_mmcfg_region *cfg)
+{
+	cfg->virt = mcfg_ioremap(cfg);
+	if (!cfg->virt) {
+		printk(KERN_ERR PREFIX "can't map MMCONFIG at %pR\n",
+		       &cfg->res);
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg)
+{
+	if (cfg && cfg->virt) {
+		iounmap(cfg->virt + PCI_MMCFG_BUS_OFFSET(cfg->start_bus));
+		cfg->virt = NULL;
+	}
+}

[-- Attachment #6: jiang_mmconf_v4_5.patch --]
[-- Type: application/octet-stream, Size: 5959 bytes --]

From: Jiang Liu <liuj97@gmail.com>
To: Yinghai Lu <yinghai@kernel.org>,
	Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>,
	Taku Izumi <izumi.taku@jp.fujitsu.com>
Cc: Jiang Liu <jiang.liu@huawei.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Jiang Liu <liuj97@gmail.com>,
	Keping Chen <chenkeping@huawei.com>,
	linux-pci@vger.kernel.org
Subject: [PATCH V4 5/6] PCI, x86: introduce pci_mmconfig_insert()/delete() for PCI root bridge hotplug
Date: Wed, 11 Apr 2012 08:11:02 +0800
Message-Id: <1334103063-2283-6-git-send-email-jiang.liu@huawei.com>

Introduce pci_mmconfig_insert()/pci_mmconfig_delete(), which will be used to
update MMCFG information when supporting PCI root bridge hotplug.

Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
---
 arch/x86/include/asm/pci_x86.h |    2 
 arch/x86/pci/mmconfig-shared.c |  104 ++++++++++++++++++++++++++++++++++++++---
 2 files changed, 99 insertions(+), 7 deletions(-)

Index: linux-2.6/arch/x86/include/asm/pci_x86.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/pci_x86.h
+++ linux-2.6/arch/x86/include/asm/pci_x86.h
@@ -141,6 +141,8 @@ extern int __init pci_mmcfg_arch_init(vo
 extern void __init pci_mmcfg_arch_free(void);
 extern int __devinit pci_mmcfg_arch_map(struct pci_mmcfg_region *cfg);
 extern void pci_mmcfg_arch_unmap(struct pci_mmcfg_region *cfg);
+extern int __devinit pci_mmconfig_insert(int seg, int start, int end, u64 addr);
+extern int pci_mmconfig_delete(int seg, int start, int end);
 extern struct pci_mmcfg_region *pci_mmconfig_lookup(int segment, int bus);
 
 extern struct list_head pci_mmcfg_list;
Index: linux-2.6/arch/x86/pci/mmconfig-shared.c
===================================================================
--- linux-2.6.orig/arch/x86/pci/mmconfig-shared.c
+++ linux-2.6/arch/x86/pci/mmconfig-shared.c
@@ -17,6 +17,7 @@
 #include <linux/bitmap.h>
 #include <linux/dmi.h>
 #include <linux/slab.h>
+#include <linux/mutex.h>
 #include <linux/rculist.h>
 #include <asm/e820.h>
 #include <asm/pci_x86.h>
@@ -376,8 +377,8 @@ static void __init pci_mmcfg_insert_reso
 	pci_mmcfg_resources_inserted = 1;
 }
 
-static acpi_status __init check_mcfg_resource(struct acpi_resource *res,
-					      void *data)
+static acpi_status __devinit check_mcfg_resource(struct acpi_resource *res,
+						 void *data)
 {
 	struct resource *mcfg_res = data;
 	struct acpi_resource_address64 address;
@@ -413,8 +414,8 @@ static acpi_status __init check_mcfg_res
 	return AE_OK;
 }
 
-static acpi_status __init find_mboard_resource(acpi_handle handle, u32 lvl,
-		void *context, void **rv)
+static acpi_status __devinit find_mboard_resource(acpi_handle handle, u32 lvl,
+						  void *context, void **rv)
 {
 	struct resource *mcfg_res = context;
 
@@ -427,7 +428,7 @@ static acpi_status __init find_mboard_re
 	return AE_OK;
 }
 
-static int __init is_acpi_reserved(u64 start, u64 end, unsigned not_used)
+static int __devinit is_acpi_reserved(u64 start, u64 end, unsigned not_used)
 {
 	struct resource mcfg_res;
 
@@ -446,8 +447,9 @@ static int __init is_acpi_reserved(u64 s
 
 typedef int (*check_reserved_t)(u64 start, u64 end, unsigned type);
 
-static int __init is_mmconf_reserved(check_reserved_t is_reserved,
-				    struct pci_mmcfg_region *cfg, int with_e820)
+static int __devinit is_mmconf_reserved(check_reserved_t is_reserved,
+					struct pci_mmcfg_region *cfg,
+					int with_e820)
 {
 	u64 addr = cfg->res.start;
 	u64 size = resource_size(&cfg->res);
@@ -676,3 +678,91 @@ static int __init pci_mmcfg_late_insert_
  * with other system resources.
  */
 late_initcall(pci_mmcfg_late_insert_resources);
+
+static DEFINE_MUTEX(pci_mmcfg_lock);
+
+/* Add MMCFG information for hot-added host bridges at runtime */
+int __devinit pci_mmconfig_insert(int segment, int start, int end, u64 addr)
+{
+	int rc = -EINVAL;
+	struct pci_mmcfg_region *cfg = NULL;
+
+	if (segment < 0 || segment > USHRT_MAX ||
+	    start < 0 || start > 255 || end < start || end > 255)
+		return rc;
+
+	mutex_lock(&pci_mmcfg_lock);
+	cfg = pci_mmconfig_lookup(segment, start);
+	if (cfg) {
+		if (cfg->start_bus <= start && cfg->end_bus >= end) {
+			rc = -EEXIST;
+		} else {
+			printk(KERN_WARNING PREFIX
+			       "MMCONFIG for domain %04x [bus %02x-%02x] "
+			       "conflicts with domain %04x [bus %02x-%02x]\n",
+			       segment, start, end,
+			       cfg->segment, cfg->start_bus, cfg->end_bus);
+		}
+		goto out;
+	}
+	if (!addr)
+		goto out;
+
+	cfg = pci_mmconfig_alloc(segment, start, end, addr);
+	if (cfg == NULL) {
+		rc = -ENOMEM;
+	} else if (!pci_mmcfg_check_reserved(cfg, 0)) {
+		printk(KERN_WARNING PREFIX
+		       "MMCONFIG for domain %04x [bus %02x-%02x] "
+		       "isn't reserved\n", segment, start, end);
+	} else if (insert_resource(&iomem_resource, &cfg->res)) {
+		rc = -EBUSY;
+		printk(KERN_WARNING PREFIX
+		       "failed to insert resource for domain "
+		       "%04x [bus %02x-%02x]\n", segment, start, end);
+	} else if (pci_mmcfg_arch_map(cfg)) {
+		rc = -EBUSY;
+		printk(KERN_WARNING PREFIX
+		       "failed to map resource for domain "
+		       "%04x [bus %02x-%02x]\n", segment, start, end);
+	} else {
+		list_add_sorted(cfg);
+		cfg = NULL;
+		rc = 0;
+	}
+
+	if (cfg) {
+		if (cfg->res.parent)
+			release_resource(&cfg->res);
+		kfree(cfg);
+	}
+
+out:
+	mutex_unlock(&pci_mmcfg_lock);
+
+	return rc;
+}
+
+/* Delete MMCFG information at runtime */
+int pci_mmconfig_delete(int segment, int start, int end)
+{
+	struct pci_mmcfg_region *cfg;
+
+	mutex_lock(&pci_mmcfg_lock);
+	list_for_each_entry_rcu(cfg, &pci_mmcfg_list, list) {
+		if (cfg->segment == segment && cfg->start_bus == start &&
+		    cfg->end_bus == end) {
+			list_del_rcu(&cfg->list);
+			synchronize_rcu();
+			pci_mmcfg_arch_unmap(cfg);
+			if (cfg->res.parent)
+				release_resource(&cfg->res);
+			mutex_unlock(&pci_mmcfg_lock);
+			kfree(cfg);
+			return 0;
+		}
+	}
+	mutex_unlock(&pci_mmcfg_lock);
+
+	return -ENOENT;
+}

[-- Attachment #7: jiang_mmconf_v4_6.patch --]
[-- Type: application/octet-stream, Size: 7061 bytes --]

From: Jiang Liu <liuj97@gmail.com>
To: Yinghai Lu <yinghai@kernel.org>,
	Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>,
	Taku Izumi <izumi.taku@jp.fujitsu.com>
Cc: Jiang Liu <jiang.liu@huawei.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Jiang Liu <liuj97@gmail.com>,
	Keping Chen <chenkeping@huawei.com>,
	linux-pci@vger.kernel.org
Subject: [PATCH V4 6/6] PCI, ACPI, x86: update MMCFG information when hot-plugging PCI host bridges
Date: Wed, 11 Apr 2012 08:11:03 +0800
Message-Id: <1334103063-2283-7-git-send-email-jiang.liu@huawei.com>

This patch enhances pci_root driver to update MMCFG information when
hot-plugging PCI root bridges on x86 platforms.

Signed-off-by: Jiang Liu <jiang.liu@huawei.com>
---
 arch/x86/include/asm/pci_x86.h |    1 
 arch/x86/pci/acpi.c            |   58 +++++++++++++++++++++++++++++++++++++++++
 arch/x86/pci/mmconfig_32.c     |    2 -
 arch/x86/pci/mmconfig_64.c     |    2 -
 drivers/acpi/pci_root.c        |   16 +++++++++++
 include/acpi/acnames.h         |    1 
 include/acpi/acpi_bus.h        |    1 
 include/linux/pci-acpi.h       |    3 ++
 8 files changed, 82 insertions(+), 2 deletions(-)

Index: linux-2.6/arch/x86/include/asm/pci_x86.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/pci_x86.h
+++ linux-2.6/arch/x86/include/asm/pci_x86.h
@@ -104,6 +104,7 @@ struct pci_raw_ops {
 extern const struct pci_raw_ops *raw_pci_ops;
 extern const struct pci_raw_ops *raw_pci_ext_ops;
 
+extern const struct pci_raw_ops pci_mmcfg;
 extern const struct pci_raw_ops pci_direct_conf1;
 extern bool port_cf9_safe;
 
Index: linux-2.6/arch/x86/pci/acpi.c
===================================================================
--- linux-2.6.orig/arch/x86/pci/acpi.c
+++ linux-2.6/arch/x86/pci/acpi.c
@@ -4,6 +4,7 @@
 #include <linux/irq.h>
 #include <linux/dmi.h>
 #include <linux/slab.h>
+#include <linux/pci-acpi.h>
 #include <asm/numa.h>
 #include <asm/pci_x86.h>
 
@@ -478,6 +479,63 @@ struct pci_bus * __devinit pci_acpi_scan
 	return bus;
 }
 
+int __devinit arch_acpi_pci_root_add(struct acpi_pci_root *root)
+{
+	int result = 0;
+	acpi_status status;
+	unsigned long long base_addr;
+
+	/* MMCFG not in use */
+	if (raw_pci_ext_ops && raw_pci_ext_ops != &pci_mmcfg)
+		return result;
+
+	status = acpi_evaluate_integer(root->device->handle, METHOD_NAME__CBA,
+				       NULL, &base_addr);
+	if (ACPI_FAILURE(status))
+		base_addr = 0;
+
+	/*
+	 * Try to insert MMCFG information for host bridge
+	 */
+	result = pci_mmconfig_insert(root->segment, root->secondary.start,
+				     root->secondary.end, base_addr);
+	if (result == -EEXIST)
+		result = 0;
+	else if (!result) {
+		root->mmconf_added = true;
+
+		/* Try to enable MMCFG if it hasn't been enabled yet */
+		if (raw_pci_ext_ops == NULL) {
+			if (pci_probe & PCI_PROBE_MMCONF)
+				raw_pci_ext_ops = &pci_mmcfg;
+			else {
+				arch_acpi_pci_root_remove(root);
+				result = -ENODEV;
+			}
+		}
+	}
+	if (result)
+		printk(KERN_ERR
+			"can't add MMCFG information for Bus %04x:%02x\n",
+			root->segment, (unsigned int)root->secondary.start);
+
+	/* still could use conf1 with it */
+	if (!root->segment)
+		result = 0;
+
+	return result;
+}
+
+void arch_acpi_pci_root_remove(struct acpi_pci_root *root)
+{
+	if (root->mmconf_added) {
+		pci_mmconfig_delete(root->segment,
+				    root->secondary.start,
+				    root->secondary.end);
+		root->mmconf_added = false;
+	}
+}
+
 int __init pci_acpi_init(void)
 {
 	struct pci_dev *dev = NULL;
Index: linux-2.6/arch/x86/pci/mmconfig_32.c
===================================================================
--- linux-2.6.orig/arch/x86/pci/mmconfig_32.c
+++ linux-2.6/arch/x86/pci/mmconfig_32.c
@@ -126,7 +126,7 @@ static int pci_mmcfg_write(unsigned int
 	return 0;
 }
 
-static const struct pci_raw_ops pci_mmcfg = {
+const struct pci_raw_ops pci_mmcfg = {
 	.read =		pci_mmcfg_read,
 	.write =	pci_mmcfg_write,
 };
Index: linux-2.6/arch/x86/pci/mmconfig_64.c
===================================================================
--- linux-2.6.orig/arch/x86/pci/mmconfig_64.c
+++ linux-2.6/arch/x86/pci/mmconfig_64.c
@@ -90,7 +90,7 @@ static int pci_mmcfg_write(unsigned int
 	return 0;
 }
 
-static const struct pci_raw_ops pci_mmcfg = {
+const struct pci_raw_ops pci_mmcfg = {
 	.read =		pci_mmcfg_read,
 	.write =	pci_mmcfg_write,
 };
Index: linux-2.6/drivers/acpi/pci_root.c
===================================================================
--- linux-2.6.orig/drivers/acpi/pci_root.c
+++ linux-2.6/drivers/acpi/pci_root.c
@@ -448,6 +448,15 @@ out:
 }
 EXPORT_SYMBOL(acpi_pci_osc_control_set);
 
+int __weak arch_acpi_pci_root_add(struct acpi_pci_root *root)
+{
+	return 0;
+}
+
+void __weak arch_acpi_pci_root_remove(struct acpi_pci_root *root)
+{
+}
+
 static int __devinit acpi_pci_root_add(struct acpi_device *device)
 {
 	unsigned long long segment, bus;
@@ -504,6 +513,11 @@ static int __devinit acpi_pci_root_add(s
 	strcpy(acpi_device_class(device), ACPI_PCI_ROOT_CLASS);
 	device->driver_data = root;
 
+	if (arch_acpi_pci_root_add(root)) {
+		result = -ENODEV;
+		goto out_free;
+	}
+
 	/*
 	 * All supported architectures that use ACPI have support for
 	 * PCI domains, so we indicate this in _OSC support capabilities.
@@ -629,6 +643,7 @@ out_del_root:
 	list_del_rcu(&root->node);
 	mutex_unlock(&acpi_pci_root_lock);
 	synchronize_rcu();
+	arch_acpi_pci_root_remove(root);
 out_free:
 	kfree(root);
 	return result;
@@ -686,6 +701,7 @@ out:
 	list_del_rcu(&root->node);
 	mutex_unlock(&acpi_pci_root_lock);
 	synchronize_rcu();
+	arch_acpi_pci_root_remove(root);
 	kfree(root);
 
 	return 0;
Index: linux-2.6/include/acpi/acnames.h
===================================================================
--- linux-2.6.orig/include/acpi/acnames.h
+++ linux-2.6/include/acpi/acnames.h
@@ -62,6 +62,7 @@
 #define METHOD_NAME__AEI        "_AEI"
 #define METHOD_NAME__PRW        "_PRW"
 #define METHOD_NAME__SRS        "_SRS"
+#define METHOD_NAME__CBA	"_CBA"
 
 /* Method names - these methods must appear at the namespace root */
 
Index: linux-2.6/include/acpi/acpi_bus.h
===================================================================
--- linux-2.6.orig/include/acpi/acpi_bus.h
+++ linux-2.6/include/acpi/acpi_bus.h
@@ -412,6 +412,7 @@ struct acpi_pci_root {
 	u32 osc_support_set;	/* _OSC state of support bits */
 	u32 osc_control_set;	/* _OSC state of control bits */
 	bool hot_added;
+	bool mmconf_added;
 };
 
 /* helper */
Index: linux-2.6/include/linux/pci-acpi.h
===================================================================
--- linux-2.6.orig/include/linux/pci-acpi.h
+++ linux-2.6/include/linux/pci-acpi.h
@@ -18,6 +18,9 @@ extern acpi_status pci_acpi_add_pm_notif
 					     struct pci_dev *pci_dev);
 extern acpi_status pci_acpi_remove_pm_notifier(struct acpi_device *dev);
 
+int arch_acpi_pci_root_add(struct acpi_pci_root *root);
+void arch_acpi_pci_root_remove(struct acpi_pci_root *root);
+
 static inline acpi_handle acpi_find_root_bridge_handle(struct pci_dev *pdev)
 {
 	struct pci_bus *pbus = pdev->bus;

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Bjorn Helgaas]

On Sat, Jun 16, 2012 at 4:44 PM, Yinghai Lu <yinghai@kernel.org> wrote:
> On Sat, Jun 16, 2012 at 2:48 PM, Bjorn Helgaas <bhelgaas@google.com> wrot=
e:
>>>
>>> We'd better to make all path share as most code as possible.
>>> 1. hostbridge scanning during boot -- early, it will check chipset and =
e820
>>> 2. MCFG checking during boot -- early, it will check e820
>>> 3. MCFG checking during boot -- late, it will check acpi pnp
>>> 4. _CBA checking for hotplug-able pci root bus but it is installed duri=
ng boot.
>>> 5. _CBA checking for hotplug-able pci root bus during run time.
>>>
>>> please keep mapping for all entries in MCFG table. aka 1, 2, 3.
>>> I have some local patches that will read ext pci conf space before scan=
 pci bus.
>>> please check attached one for nehalem-ioh.
>>
>> I don't think it's a requirement that Gerry keep your Nehalem patch
>> working. =A0Your intel_bus.c is not in the tree and you haven't provided
>> an explanation for why it should be.
>>
>> The only requirement I'm aware of for PCI config access before we
>> discover the host bridges via ACPI is for segment group 0, bus 0, as
>> mentioned in ACPI spec 5.0, sec 5.2.3.1, PDF page 143, and I think
>> that applies only to the first 0x100 bytes of config space. =A0I don't
>> think there's a requirement for access to the extended configuration
>> space (bytes 0x100-0xFFF). =A0I do not see a requirement that this
>> pre-host bridge access happen via MMCONFIG; as far as I can tell, we
>> can use the legacy 0xCF8/0xCFC mechanism.
>
> that one shot for intel host bridge resource discovery before root bus
> scanning,
> will need to access registers above 0x100.

I don't understand what you're saying.  Are you disagreeing with
something I said above?

As far as I know, we can rely on ACPI _CRS completely for host bridge
resources.  Are there exceptions?  What does "one shot for intel host
bridge resource discovery" mean?  Are there machines that are broken
because we don't have intel_bus.c?

> Current MCFG handling have some sanitary checking during probing.
>
> Now Jiang is trying to free result and cache it for two MCFG path 2/3.
> and later use cached and map again for entries from cached entries.
> but when use those cached entries sanitary of those entries are lost.
>
> so choice would be
> 1. cache all checked MMCFG result and use that later
> 2. or just leave current MCFG handling alone, just add _CBA support.
> like Jiang -v4 version.

Choice 2 sounds like a possibility.  I probably encouraged mucking
around in the current MCFG handling, but if we're not going to
actually clean anything up, there's not much point in touching it.

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Jiang Liu]

On 2012-6-17 9:55, Bjorn Helgaas wrote:
> On Sat, Jun 16, 2012 at 4:44 PM, Yinghai Lu <yinghai@kernel.org> wrote:
>> On Sat, Jun 16, 2012 at 2:48 PM, Bjorn Helgaas <bhelgaas@google.com> wrote:
>>>>
>>>> We'd better to make all path share as most code as possible.
>>>> 1. hostbridge scanning during boot -- early, it will check chipset and e820
>>>> 2. MCFG checking during boot -- early, it will check e820
>>>> 3. MCFG checking during boot -- late, it will check acpi pnp
>>>> 4. _CBA checking for hotplug-able pci root bus but it is installed during boot.
>>>> 5. _CBA checking for hotplug-able pci root bus during run time.
>>>>
>>>> please keep mapping for all entries in MCFG table. aka 1, 2, 3.
>>>> I have some local patches that will read ext pci conf space before scan pci bus.
>>>> please check attached one for nehalem-ioh.
>>>
>>> I don't think it's a requirement that Gerry keep your Nehalem patch
>>> working.  Your intel_bus.c is not in the tree and you haven't provided
>>> an explanation for why it should be.
>>>
>>> The only requirement I'm aware of for PCI config access before we
>>> discover the host bridges via ACPI is for segment group 0, bus 0, as
>>> mentioned in ACPI spec 5.0, sec 5.2.3.1, PDF page 143, and I think
>>> that applies only to the first 0x100 bytes of config space.  I don't
>>> think there's a requirement for access to the extended configuration
>>> space (bytes 0x100-0xFFF).  I do not see a requirement that this
>>> pre-host bridge access happen via MMCONFIG; as far as I can tell, we
>>> can use the legacy 0xCF8/0xCFC mechanism.
>>
>> that one shot for intel host bridge resource discovery before root bus
>> scanning,
>> will need to access registers above 0x100.
> 
> I don't understand what you're saying.  Are you disagreeing with
> something I said above?
> 
> As far as I know, we can rely on ACPI _CRS completely for host bridge
> resources.  Are there exceptions?  What does "one shot for intel host
> bridge resource discovery" mean?  Are there machines that are broken
> because we don't have intel_bus.c?

I have consulted Bob Moore about this topic before and Bob said that
they hasn't encountered a system on which the ACPICA has dependency 
on extended PCI configuration space yet.

>> Current MCFG handling have some sanitary checking during probing.
>>
>> Now Jiang is trying to free result and cache it for two MCFG path 2/3.
>> and later use cached and map again for entries from cached entries.
>> but when use those cached entries sanitary of those entries are lost.
>>
>> so choice would be
>> 1. cache all checked MMCFG result and use that later
>> 2. or just leave current MCFG handling alone, just add _CBA support.
>> like Jiang -v4 version.
> 
> Choice 2 sounds like a possibility.  I probably encouraged mucking
> around in the current MCFG handling, but if we're not going to
> actually clean anything up, there's not much point in touching it.
I prefer the second choice too. Backward compatibility is really
important on x86, so don't want to break anything here. I could
help to split the patch set into two parts: one for root bridge
hotplug and the other for cleanup. This time we could focus on 
the first patch to prepare for host bridge hotplug, and have 
more time to discuss about the cleanup work.

On the other hand, BIOS should report MMCONFIG information by _CBA
if a host bridge supports physical hotplug. So the cleanup only 
benefits two cases:
1) BIOS reports MMCONFIG information for hot-pluggable host bridges
   in MCFG table. This is really a BIOS bug. Currently we have no
   really x86 platforms in the field which support PCI host bridge
   hotplug yet. So it may be acceptable for OS to report such bugs
   and let BIOS people to fix them.
2) Account MMCONFIG information to specific host bridges. It does
   give better representation about the MMCONFIG resource usages,
   but it's on risk of breaking backward compatibilities.

So should we adopt the second solution here?

Thanks!
Gerry

Re: [PATCH v7 08/10] PCI, x86: add MMCFG information on demand

[Bjorn Helgaas]

On Sun, Jun 17, 2012 at 7:21 PM, Jiang Liu <jiang.liu@huawei.com> wrote:
> On 2012-6-17 9:55, Bjorn Helgaas wrote:
>> On Sat, Jun 16, 2012 at 4:44 PM, Yinghai Lu <yinghai@kernel.org> wrote:
>>> On Sat, Jun 16, 2012 at 2:48 PM, Bjorn Helgaas <bhelgaas@google.com> wr=
ote:
>>>>>
>>>>> We'd better to make all path share as most code as possible.
>>>>> 1. hostbridge scanning during boot -- early, it will check chipset an=
d e820
>>>>> 2. MCFG checking during boot -- early, it will check e820
>>>>> 3. MCFG checking during boot -- late, it will check acpi pnp
>>>>> 4. _CBA checking for hotplug-able pci root bus but it is installed du=
ring boot.
>>>>> 5. _CBA checking for hotplug-able pci root bus during run time.
>>>>>
>>>>> please keep mapping for all entries in MCFG table. aka 1, 2, 3.
>>>>> I have some local patches that will read ext pci conf space before sc=
an pci bus.
>>>>> please check attached one for nehalem-ioh.
>>>>
>>>> I don't think it's a requirement that Gerry keep your Nehalem patch
>>>> working. =A0Your intel_bus.c is not in the tree and you haven't provid=
ed
>>>> an explanation for why it should be.
>>>>
>>>> The only requirement I'm aware of for PCI config access before we
>>>> discover the host bridges via ACPI is for segment group 0, bus 0, as
>>>> mentioned in ACPI spec 5.0, sec 5.2.3.1, PDF page 143, and I think
>>>> that applies only to the first 0x100 bytes of config space. =A0I don't
>>>> think there's a requirement for access to the extended configuration
>>>> space (bytes 0x100-0xFFF). =A0I do not see a requirement that this
>>>> pre-host bridge access happen via MMCONFIG; as far as I can tell, we
>>>> can use the legacy 0xCF8/0xCFC mechanism.
>>>
>>> that one shot for intel host bridge resource discovery before root bus
>>> scanning,
>>> will need to access registers above 0x100.
>>
>> I don't understand what you're saying. =A0Are you disagreeing with
>> something I said above?
>>
>> As far as I know, we can rely on ACPI _CRS completely for host bridge
>> resources. =A0Are there exceptions? =A0What does "one shot for intel hos=
t
>> bridge resource discovery" mean? =A0Are there machines that are broken
>> because we don't have intel_bus.c?
>
> I have consulted Bob Moore about this topic before and Bob said that
> they hasn't encountered a system on which the ACPICA has dependency
> on extended PCI configuration space yet.
>
>>> Current MCFG handling have some sanitary checking during probing.
>>>
>>> Now Jiang is trying to free result and cache it for two MCFG path 2/3.
>>> and later use cached and map again for entries from cached entries.
>>> but when use those cached entries sanitary of those entries are lost.
>>>
>>> so choice would be
>>> 1. cache all checked MMCFG result and use that later
>>> 2. or just leave current MCFG handling alone, just add _CBA support.
>>> like Jiang -v4 version.
>>
>> Choice 2 sounds like a possibility. =A0I probably encouraged mucking
>> around in the current MCFG handling, but if we're not going to
>> actually clean anything up, there's not much point in touching it.
> I prefer the second choice too. Backward compatibility is really
> important on x86, so don't want to break anything here. I could
> help to split the patch set into two parts: one for root bridge
> hotplug and the other for cleanup. This time we could focus on
> the first patch to prepare for host bridge hotplug, and have
> more time to discuss about the cleanup work.
>
> On the other hand, BIOS should report MMCONFIG information by _CBA
> if a host bridge supports physical hotplug. So the cleanup only
> benefits two cases:
> 1) BIOS reports MMCONFIG information for hot-pluggable host bridges
> =A0 in MCFG table. This is really a BIOS bug. Currently we have no
> =A0 really x86 platforms in the field which support PCI host bridge
> =A0 hotplug yet. So it may be acceptable for OS to report such bugs
> =A0 and let BIOS people to fix them.
> 2) Account MMCONFIG information to specific host bridges. It does
> =A0 give better representation about the MMCONFIG resource usages,
> =A0 but it's on risk of breaking backward compatibilities.
>
> So should we adopt the second solution here?

Yes, let's try that.  I'm sorry I encouraged you to go on a wild goose chas=
e.