[RFC v2 0/7] PCI: hv: Support host bridge probing on ARM64

[RFC v2 0/7] PCI: hv: Support host bridge probing on ARM64

[Boqun Feng]

Hi Bjorn, Arnd and Marc,

This is the updated version of my first try to prepare for virtual PCI
support on Hyper-V ARM64:

	https://lore.kernel.org/lkml/20210319161956.2838291-1-boqun.feng@gmail.com/

, thanks a lot for your suggestions in the previous one.

This is still more an RFC, and trying to see what I'm on the correct
direction.

The basic problem we need to resolve is that ARM64 is an arch with
PCI_DOMAINS_GENERIC=y, so the bus sysdata is pci_config_window. However,
Hyper-V PCI provides a paravirtualized PCI interface, so there is no
actual pci_config_window for a PCI host bridge, so no information can be
retrieve from the pci_config_window of a Hyper-V virtual PCI bus.

This patchset mainly handle three things:

1)	PCI domain number. As suggested by Bjorn and Arnd, I introduce
	a member in pci_host_bridge, and Hyper-V can use it to set up
	the correct PCI domain number for the bus, while others remain
	the same behavior.

2)	MSI irq_domain. As suggested by Arnd, I use the
	GENERIC_MSI_IRQ_DOMAIN's dev_{set,get}_msi_domain() to allow
	Hyper-V set up the msi domain using generic code.

3)	pcibios_root_bridge_prepare(). This one is new, PCI core will
	call this during host bridge register, and it will access the
	pci_config_window, luckily the only field it accesses it the
	parent field, and I change the ARM64's behavior to treat the
	acpi device as NULL if pci_config_window::parent is NULL, so
	that Hyper-V can provide a all-zeroed pci_config_window.

With the above, we could enable the virtual PCI on Hyper-V ARM64 guest
with other code under development.

Comments and suggestions are welcome.

Regards,
Boqun

Arnd Bergmann (1):
  PCI: hv: Generify PCI probing

Boqun Feng (6):
  PCI: Introduce pci_host_bridge::domain_nr
  PCI: Allow msi domain set-up at host probing time
  PCI: hv: Use pci_host_bridge::domain_nr for PCI domain
  PCI: hv: Set up msi domain at bridge probing time
  PCI: arm64: Allow pci_config_window::parent to be NULL
  PCI: hv: Turn on the host bridge probing on ARM64

 arch/arm/kernel/bios32.c              |  2 +
 arch/arm/mach-dove/pcie.c             |  2 +
 arch/arm/mach-mv78xx0/pcie.c          |  2 +
 arch/arm/mach-orion5x/pci.c           |  2 +
 arch/arm64/kernel/pci.c               |  5 +-
 arch/mips/pci/pci-legacy.c            |  2 +
 arch/mips/pci/pci-xtalk-bridge.c      |  2 +
 drivers/pci/controller/pci-ftpci100.c |  2 +
 drivers/pci/controller/pci-hyperv.c   | 90 +++++++++++++++------------
 drivers/pci/controller/pci-mvebu.c    |  2 +
 drivers/pci/pci.c                     |  4 +-
 drivers/pci/probe.c                   | 12 +++-
 include/linux/pci.h                   | 11 +++-
 13 files changed, 89 insertions(+), 49 deletions(-)

-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RFC v2 1/7] PCI: Introduce pci_host_bridge::domain_nr

[Boqun Feng]

Currently we retrieve the PCI domain number of the host bridge from the
bus sysdata (or pci_config_window if PCI_DOMAINS_GENERIC=y). Actually
we have the information at PCI host bridge probing time, and it makes
sense that we store it into pci_host_bridge. One benefit of doing so is
the requirement for supporting PCI on Hyper-V for ARM64, because the
host bridge of Hyper-V doesnt' have pci_config_window, whereas ARM64 is
a PCI_DOMAINS_GENERIC=y arch, so we cannot retrieve the PCI domain
number from pci_config_window on ARM64 Hyper-V guest.

As the preparation for ARM64 Hyper-V PCI support, we introduce the
domain_nr in pci_host_bridge, and set it properly at probing time, then
for PCI_DOMAINS_GENERIC=y archs, bus domain numbers are set by the
bridge domain_nr.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 arch/arm/kernel/bios32.c              |  2 ++
 arch/arm/mach-dove/pcie.c             |  2 ++
 arch/arm/mach-mv78xx0/pcie.c          |  2 ++
 arch/arm/mach-orion5x/pci.c           |  2 ++
 arch/arm64/kernel/pci.c               |  3 +--
 arch/mips/pci/pci-legacy.c            |  2 ++
 arch/mips/pci/pci-xtalk-bridge.c      |  2 ++
 drivers/pci/controller/pci-ftpci100.c |  2 ++
 drivers/pci/controller/pci-mvebu.c    |  2 ++
 drivers/pci/pci.c                     |  4 ++--
 drivers/pci/probe.c                   |  7 ++++++-
 include/linux/pci.h                   | 11 ++++++++---
 12 files changed, 33 insertions(+), 8 deletions(-)

diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
index e7ef2b5bea9c..4942cd681e41 100644
--- a/arch/arm/kernel/bios32.c
+++ b/arch/arm/kernel/bios32.c
@@ -471,6 +471,8 @@ static void pcibios_init_hw(struct device *parent, struct hw_pci *hw,
 				bridge->sysdata = sys;
 				bridge->busnr = sys->busnr;
 				bridge->ops = hw->ops;
+				if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
+					bridge->domain_nr = pci_bus_find_domain_nr(sys, parent);
 
 				ret = pci_scan_root_bus_bridge(bridge);
 			}
diff --git a/arch/arm/mach-dove/pcie.c b/arch/arm/mach-dove/pcie.c
index ee91ac6b5ebf..92eb8484b49b 100644
--- a/arch/arm/mach-dove/pcie.c
+++ b/arch/arm/mach-dove/pcie.c
@@ -167,6 +167,8 @@ dove_pcie_scan_bus(int nr, struct pci_host_bridge *bridge)
 	bridge->sysdata = sys;
 	bridge->busnr = sys->busnr;
 	bridge->ops = &pcie_ops;
+	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
+		bridge->domain_nr = pci_bus_find_domain_nr(sys, NULL);
 
 	return pci_scan_root_bus_bridge(bridge);
 }
diff --git a/arch/arm/mach-mv78xx0/pcie.c b/arch/arm/mach-mv78xx0/pcie.c
index 636d84b40466..6703d394bcde 100644
--- a/arch/arm/mach-mv78xx0/pcie.c
+++ b/arch/arm/mach-mv78xx0/pcie.c
@@ -208,6 +208,8 @@ static int __init mv78xx0_pcie_scan_bus(int nr, struct pci_host_bridge *bridge)
 	bridge->sysdata = sys;
 	bridge->busnr = sys->busnr;
 	bridge->ops = &pcie_ops;
+	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
+		bridge->domain_nr = pci_bus_find_domain_nr(sys, NULL);
 
 	return pci_scan_root_bus_bridge(bridge);
 }
diff --git a/arch/arm/mach-orion5x/pci.c b/arch/arm/mach-orion5x/pci.c
index 76951bfbacf5..6257fbd4e705 100644
--- a/arch/arm/mach-orion5x/pci.c
+++ b/arch/arm/mach-orion5x/pci.c
@@ -563,6 +563,8 @@ int __init orion5x_pci_sys_scan_bus(int nr, struct pci_host_bridge *bridge)
 	bridge->dev.parent = NULL;
 	bridge->sysdata = sys;
 	bridge->busnr = sys->busnr;
+	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
+		bridge->domain_nr = pci_bus_find_domain_nr(sys, NULL);
 
 	if (nr == 0) {
 		bridge->ops = &pcie_ops;
diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index 1006ed2d7c60..e9a6eeb6a694 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -71,9 +71,8 @@ struct acpi_pci_generic_root_info {
 	struct pci_config_window	*cfg;	/* config space mapping */
 };
 
-int acpi_pci_bus_find_domain_nr(struct pci_bus *bus)
+int acpi_pci_bus_find_domain_nr(struct pci_config_window *cfg)
 {
-	struct pci_config_window *cfg = bus->sysdata;
 	struct acpi_device *adev = to_acpi_device(cfg->parent);
 	struct acpi_pci_root *root = acpi_driver_data(adev);
 
diff --git a/arch/mips/pci/pci-legacy.c b/arch/mips/pci/pci-legacy.c
index 39052de915f3..84ad482be22d 100644
--- a/arch/mips/pci/pci-legacy.c
+++ b/arch/mips/pci/pci-legacy.c
@@ -97,6 +97,8 @@ static void pcibios_scanbus(struct pci_controller *hose)
 	bridge->ops = hose->pci_ops;
 	bridge->swizzle_irq = pci_common_swizzle;
 	bridge->map_irq = pcibios_map_irq;
+	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
+		bridge->domain_nr = pci_bus_find_domain_nr(hose, NULL);
 	ret = pci_scan_root_bus_bridge(bridge);
 	if (ret) {
 		pci_free_host_bridge(bridge);
diff --git a/arch/mips/pci/pci-xtalk-bridge.c b/arch/mips/pci/pci-xtalk-bridge.c
index 50f7d42cca5a..23355ab720be 100644
--- a/arch/mips/pci/pci-xtalk-bridge.c
+++ b/arch/mips/pci/pci-xtalk-bridge.c
@@ -712,6 +712,8 @@ static int bridge_probe(struct platform_device *pdev)
 	host->ops = &bridge_pci_ops;
 	host->map_irq = bridge_map_irq;
 	host->swizzle_irq = pci_common_swizzle;
+	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
+		host->domain_nr = pci_bus_find_domain_nr(bc, dev);
 
 	err = pci_scan_root_bus_bridge(host);
 	if (err < 0)
diff --git a/drivers/pci/controller/pci-ftpci100.c b/drivers/pci/controller/pci-ftpci100.c
index da3cd216da00..cf6eec7f90e1 100644
--- a/drivers/pci/controller/pci-ftpci100.c
+++ b/drivers/pci/controller/pci-ftpci100.c
@@ -439,6 +439,8 @@ static int faraday_pci_probe(struct platform_device *pdev)
 	host->ops = &faraday_pci_ops;
 	p = pci_host_bridge_priv(host);
 	host->sysdata = p;
+	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
+		host->domain_nr = pci_bus_find_domain_nr(p, dev);
 	p->dev = dev;
 
 	/* Retrieve and enable optional clocks */
diff --git a/drivers/pci/controller/pci-mvebu.c b/drivers/pci/controller/pci-mvebu.c
index ed13e81cd691..b329ed2f0956 100644
--- a/drivers/pci/controller/pci-mvebu.c
+++ b/drivers/pci/controller/pci-mvebu.c
@@ -1122,6 +1122,8 @@ static int mvebu_pcie_probe(struct platform_device *pdev)
 	bridge->sysdata = pcie;
 	bridge->ops = &mvebu_pcie_ops;
 	bridge->align_resource = mvebu_pcie_align_resource;
+	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
+		bridge->domain_nr = pci_bus_find_domain_nr(pcie, dev);
 
 	return mvebu_pci_host_probe(bridge);
 }
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 16a17215f633..a249dbf78c34 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -6505,10 +6505,10 @@ static int of_pci_bus_find_domain_nr(struct device *parent)
 	return domain;
 }
 
-int pci_bus_find_domain_nr(struct pci_bus *bus, struct device *parent)
+int pci_bus_find_domain_nr(void *sysdata, struct device *parent)
 {
 	return acpi_disabled ? of_pci_bus_find_domain_nr(parent) :
-			       acpi_pci_bus_find_domain_nr(bus);
+			       acpi_pci_bus_find_domain_nr(sysdata);
 }
 #endif
 
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 953f15abc850..5e71cc5e1b6c 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -899,7 +899,7 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge)
 	bus->ops = bridge->ops;
 	bus->number = bus->busn_res.start = bridge->busnr;
 #ifdef CONFIG_PCI_DOMAINS_GENERIC
-	bus->domain_nr = pci_bus_find_domain_nr(bus, parent);
+	bus->domain_nr = bridge->domain_nr;
 #endif
 
 	b = pci_find_bus(pci_domain_nr(bus), bridge->busnr);
@@ -2974,6 +2974,8 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
 	bridge->sysdata = sysdata;
 	bridge->busnr = bus;
 	bridge->ops = ops;
+	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
+		bridge->domain_nr = pci_bus_find_domain_nr(sysdata, parent);
 
 	error = pci_register_host_bridge(bridge);
 	if (error < 0)
@@ -2992,6 +2994,9 @@ int pci_host_probe(struct pci_host_bridge *bridge)
 	struct pci_bus *bus, *child;
 	int ret;
 
+	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
+		bridge->domain_nr = pci_bus_find_domain_nr(bridge->sysdata, bridge->dev.parent);
+
 	ret = pci_scan_root_bus_bridge(bridge);
 	if (ret < 0) {
 		dev_err(bridge->dev.parent, "Scanning root bridge failed");
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 86c799c97b77..5bbd8417d219 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -534,6 +534,7 @@ struct pci_host_bridge {
 	struct pci_ops	*child_ops;
 	void		*sysdata;
 	int		busnr;
+	int		domain_nr;
 	struct list_head windows;	/* resource_entry */
 	struct list_head dma_ranges;	/* dma ranges resource list */
 	u8 (*swizzle_irq)(struct pci_dev *, u8 *); /* Platform IRQ swizzler */
@@ -1637,13 +1638,17 @@ static inline int pci_domain_nr(struct pci_bus *bus)
 {
 	return bus->domain_nr;
 }
+struct pci_config_window;
 #ifdef CONFIG_ACPI
-int acpi_pci_bus_find_domain_nr(struct pci_bus *bus);
+int acpi_pci_bus_find_domain_nr(struct pci_config_window *cfg);
 #else
-static inline int acpi_pci_bus_find_domain_nr(struct pci_bus *bus)
+static inline int acpi_pci_bus_find_domain_nr(struct pci_config_window *cfg)
 { return 0; }
 #endif
-int pci_bus_find_domain_nr(struct pci_bus *bus, struct device *parent);
+int pci_bus_find_domain_nr(void *sysdata, struct device *parent);
+#else
+static inline int pci_bus_find_domain_nr(void *sysdata, struct device *parent)
+{ return 0; }
 #endif
 
 /* Some architectures require additional setup to direct VGA traffic */
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RFC v2 2/7] PCI: Allow msi domain set-up at host probing time

[Boqun Feng]

For GENERIC_MSI_IRQ_DOMAIN drivers, we can set up the msi domain via
dev_set_msi_domain() at probing time, and drivers can use this more
generic way to set up the msi domain for the host bridge.

This is the preparation for ARM64 Hyper-V PCI support.

Originally-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 drivers/pci/probe.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 5e71cc5e1b6c..90afa05ab2f1 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -828,11 +828,14 @@ static struct irq_domain *pci_host_bridge_msi_domain(struct pci_bus *bus)
 {
 	struct irq_domain *d;
 
+	/* Default set by host bridge driver */
+	d = dev_get_msi_domain(bus->bridge);
 	/*
 	 * Any firmware interface that can resolve the msi_domain
 	 * should be called from here.
 	 */
-	d = pci_host_bridge_of_msi_domain(bus);
+	if (!d)
+		d = pci_host_bridge_of_msi_domain(bus);
 	if (!d)
 		d = pci_host_bridge_acpi_msi_domain(bus);
 
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RFC v2 3/7] PCI: hv: Generify PCI probing

[Boqun Feng]

From: Arnd Bergmann <arnd@arndb.de>

In order to support ARM64 Hyper-V PCI, we need to set up the bridge at
probing time because ARM64 is a PCI_DOMAIN_GENERIC arch and we don't
have pci_config_window (ARM64 sysdata) for a PCI root bus on Hyper-V, so
it's impossible to retrieve the information (e.g. PCI domains, irq
domains) from bus sysdata on ARM64 after creation.

Originally in create_root_hv_pci_bus(), pci_create_root_bus() is used to
create the root bus and the corresponding bridge based on x86 sysdata.
Now we create a bridge first and then call pci_scan_root_bus_bridge(),
which allows us to do the necessary set-ups for the bridge.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 drivers/pci/controller/pci-hyperv.c | 61 +++++++++++++++--------------
 1 file changed, 31 insertions(+), 30 deletions(-)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index 27a17a1e4a7c..27b922b4bb7b 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -450,6 +450,7 @@ enum hv_pcibus_state {
 
 struct hv_pcibus_device {
 	struct pci_sysdata sysdata;
+	struct pci_host_bridge *bridge;
 	/* Protocol version negotiated with the host */
 	enum pci_protocol_version_t protocol_version;
 	enum hv_pcibus_state state;
@@ -462,13 +463,10 @@ struct hv_pcibus_device {
 	struct resource *high_mmio_res;
 	struct completion *survey_event;
 	struct completion remove_event;
-	struct pci_bus *pci_bus;
 	spinlock_t config_lock;	/* Avoid two threads writing index page */
 	spinlock_t device_list_lock;	/* Protect lists below */
 	void __iomem *cfg_addr;
 
-	struct list_head resources_for_children;
-
 	struct list_head children;
 	struct list_head dr_list;
 
@@ -1804,7 +1802,7 @@ static void hv_pci_assign_slots(struct hv_pcibus_device *hbus)
 
 		slot_nr = PCI_SLOT(wslot_to_devfn(hpdev->desc.win_slot.slot));
 		snprintf(name, SLOT_NAME_SIZE, "%u", hpdev->desc.ser);
-		hpdev->pci_slot = pci_create_slot(hbus->pci_bus, slot_nr,
+		hpdev->pci_slot = pci_create_slot(hbus->bridge->bus, slot_nr,
 					  name, NULL);
 		if (IS_ERR(hpdev->pci_slot)) {
 			pr_warn("pci_create slot %s failed\n", name);
@@ -1834,7 +1832,7 @@ static void hv_pci_remove_slots(struct hv_pcibus_device *hbus)
 static void hv_pci_assign_numa_node(struct hv_pcibus_device *hbus)
 {
 	struct pci_dev *dev;
-	struct pci_bus *bus = hbus->pci_bus;
+	struct pci_bus *bus = hbus->bridge->bus;
 	struct hv_pci_dev *hv_dev;
 
 	list_for_each_entry(dev, &bus->devices, bus_list) {
@@ -1857,24 +1855,24 @@ static void hv_pci_assign_numa_node(struct hv_pcibus_device *hbus)
  */
 static int create_root_hv_pci_bus(struct hv_pcibus_device *hbus)
 {
-	/* Register the device */
-	hbus->pci_bus = pci_create_root_bus(&hbus->hdev->device,
-					    0, /* bus number is always zero */
-					    &hv_pcifront_ops,
-					    &hbus->sysdata,
-					    &hbus->resources_for_children);
-	if (!hbus->pci_bus)
-		return -ENODEV;
+	int error;
+	struct pci_host_bridge *bridge = hbus->bridge;
+
+	bridge->dev.parent = &hbus->hdev->device;
+	bridge->sysdata = &hbus->sysdata;
+	bridge->ops = &hv_pcifront_ops;
+	bridge->msi = &hbus->msi_chip;
+	bridge->msi->dev = &hbus->hdev->device;
 
-	hbus->pci_bus->msi = &hbus->msi_chip;
-	hbus->pci_bus->msi->dev = &hbus->hdev->device;
+	error = pci_scan_root_bus_bridge(bridge);
+	if (error)
+		return error;
 
 	pci_lock_rescan_remove();
-	pci_scan_child_bus(hbus->pci_bus);
 	hv_pci_assign_numa_node(hbus);
-	pci_bus_assign_resources(hbus->pci_bus);
+	pci_bus_assign_resources(bridge->bus);
 	hv_pci_assign_slots(hbus);
-	pci_bus_add_devices(hbus->pci_bus);
+	pci_bus_add_devices(bridge->bus);
 	pci_unlock_rescan_remove();
 	hbus->state = hv_pcibus_installed;
 	return 0;
@@ -2139,7 +2137,7 @@ static void pci_devices_present_work(struct work_struct *work)
 		 * because there may have been changes.
 		 */
 		pci_lock_rescan_remove();
-		pci_scan_child_bus(hbus->pci_bus);
+		pci_scan_child_bus(hbus->bridge->bus);
 		hv_pci_assign_numa_node(hbus);
 		hv_pci_assign_slots(hbus);
 		pci_unlock_rescan_remove();
@@ -2310,8 +2308,8 @@ static void hv_eject_device_work(struct work_struct *work)
 	/*
 	 * Ejection can come before or after the PCI bus has been set up, so
 	 * attempt to find it and tear down the bus state, if it exists.  This
-	 * must be done without constructs like pci_domain_nr(hbus->pci_bus)
-	 * because hbus->pci_bus may not exist yet.
+	 * must be done without constructs like pci_domain_nr(hbus->bridge->bus)
+	 * because hbus->bridge->bus may not exist yet.
 	 */
 	wslot = wslot_to_devfn(hpdev->desc.win_slot.slot);
 	pdev = pci_get_domain_bus_and_slot(hbus->sysdata.domain, 0, wslot);
@@ -2680,8 +2678,7 @@ static int hv_pci_allocate_bridge_windows(struct hv_pcibus_device *hbus)
 		/* Modify this resource to become a bridge window. */
 		hbus->low_mmio_res->flags |= IORESOURCE_WINDOW;
 		hbus->low_mmio_res->flags &= ~IORESOURCE_BUSY;
-		pci_add_resource(&hbus->resources_for_children,
-				 hbus->low_mmio_res);
+		pci_add_resource(&hbus->bridge->windows, hbus->low_mmio_res);
 	}
 
 	if (hbus->high_mmio_space) {
@@ -2700,8 +2697,7 @@ static int hv_pci_allocate_bridge_windows(struct hv_pcibus_device *hbus)
 		/* Modify this resource to become a bridge window. */
 		hbus->high_mmio_res->flags |= IORESOURCE_WINDOW;
 		hbus->high_mmio_res->flags &= ~IORESOURCE_BUSY;
-		pci_add_resource(&hbus->resources_for_children,
-				 hbus->high_mmio_res);
+		pci_add_resource(&hbus->bridge->windows, hbus->high_mmio_res);
 	}
 
 	return 0;
@@ -3031,6 +3027,7 @@ static void hv_put_dom_num(u16 dom)
 static int hv_pci_probe(struct hv_device *hdev,
 			const struct hv_vmbus_device_id *dev_id)
 {
+	struct pci_host_bridge *bridge;
 	struct hv_pcibus_device *hbus;
 	u16 dom_req, dom;
 	char *name;
@@ -3043,6 +3040,10 @@ static int hv_pci_probe(struct hv_device *hdev,
 	 */
 	BUILD_BUG_ON(sizeof(*hbus) > HV_HYP_PAGE_SIZE);
 
+	bridge = devm_pci_alloc_host_bridge(&hdev->device, 0);
+	if (!bridge)
+		return -ENOMEM;
+
 	/*
 	 * With the recent 59bb47985c1d ("mm, sl[aou]b: guarantee natural
 	 * alignment for kmalloc(power-of-two)"), kzalloc() is able to allocate
@@ -3064,6 +3065,8 @@ static int hv_pci_probe(struct hv_device *hdev,
 	hbus = kzalloc(HV_HYP_PAGE_SIZE, GFP_KERNEL);
 	if (!hbus)
 		return -ENOMEM;
+
+	hbus->bridge = bridge;
 	hbus->state = hv_pcibus_init;
 	hbus->wslot_res_allocated = -1;
 
@@ -3101,7 +3104,6 @@ static int hv_pci_probe(struct hv_device *hdev,
 	refcount_set(&hbus->remove_lock, 1);
 	INIT_LIST_HEAD(&hbus->children);
 	INIT_LIST_HEAD(&hbus->dr_list);
-	INIT_LIST_HEAD(&hbus->resources_for_children);
 	spin_lock_init(&hbus->config_lock);
 	spin_lock_init(&hbus->device_list_lock);
 	spin_lock_init(&hbus->retarget_msi_interrupt_lock);
@@ -3307,9 +3309,9 @@ static int hv_pci_remove(struct hv_device *hdev)
 	if (hbus->state == hv_pcibus_installed) {
 		/* Remove the bus from PCI's point of view. */
 		pci_lock_rescan_remove();
-		pci_stop_root_bus(hbus->pci_bus);
+		pci_stop_root_bus(hbus->bridge->bus);
 		hv_pci_remove_slots(hbus);
-		pci_remove_root_bus(hbus->pci_bus);
+		pci_remove_root_bus(hbus->bridge->bus);
 		pci_unlock_rescan_remove();
 		hbus->state = hv_pcibus_removed;
 	}
@@ -3320,7 +3322,6 @@ static int hv_pci_remove(struct hv_device *hdev)
 
 	iounmap(hbus->cfg_addr);
 	hv_free_config_window(hbus);
-	pci_free_resource_list(&hbus->resources_for_children);
 	hv_pci_free_bridge_windows(hbus);
 	irq_domain_remove(hbus->irq_domain);
 	irq_domain_free_fwnode(hbus->sysdata.fwnode);
@@ -3406,7 +3407,7 @@ static int hv_pci_restore_msi_msg(struct pci_dev *pdev, void *arg)
  */
 static void hv_pci_restore_msi_state(struct hv_pcibus_device *hbus)
 {
-	pci_walk_bus(hbus->pci_bus, hv_pci_restore_msi_msg, NULL);
+	pci_walk_bus(hbus->bridge->bus, hv_pci_restore_msi_msg, NULL);
 }
 
 static int hv_pci_resume(struct hv_device *hdev)
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RFC v2 4/7] PCI: hv: Use pci_host_bridge::domain_nr for PCI domain

[Boqun Feng]

No functional change, just store and maintain the PCI domain number in
the generic pci_host_bridge instead of x86 specific pci_sysdata.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 drivers/pci/controller/pci-hyperv.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index 27b922b4bb7b..05b73d5dfe9d 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -2312,7 +2312,7 @@ static void hv_eject_device_work(struct work_struct *work)
 	 * because hbus->bridge->bus may not exist yet.
 	 */
 	wslot = wslot_to_devfn(hpdev->desc.win_slot.slot);
-	pdev = pci_get_domain_bus_and_slot(hbus->sysdata.domain, 0, wslot);
+	pdev = pci_get_domain_bus_and_slot(hbus->bridge->domain_nr, 0, wslot);
 	if (pdev) {
 		pci_lock_rescan_remove();
 		pci_stop_and_remove_bus_device(pdev);
@@ -3098,6 +3098,7 @@ static int hv_pci_probe(struct hv_device *hdev,
 			 "PCI dom# 0x%hx has collision, using 0x%hx",
 			 dom_req, dom);
 
+	hbus->bridge->domain_nr = dom;
 	hbus->sysdata.domain = dom;
 
 	hbus->hdev = hdev;
@@ -3109,7 +3110,7 @@ static int hv_pci_probe(struct hv_device *hdev,
 	spin_lock_init(&hbus->retarget_msi_interrupt_lock);
 	init_completion(&hbus->remove_event);
 	hbus->wq = alloc_ordered_workqueue("hv_pci_%x", 0,
-					   hbus->sysdata.domain);
+					   hbus->bridge->domain_nr);
 	if (!hbus->wq) {
 		ret = -ENOMEM;
 		goto free_dom;
@@ -3236,7 +3237,7 @@ static int hv_pci_probe(struct hv_device *hdev,
 destroy_wq:
 	destroy_workqueue(hbus->wq);
 free_dom:
-	hv_put_dom_num(hbus->sysdata.domain);
+	hv_put_dom_num(hbus->bridge->domain_nr);
 free_bus:
 	kfree(hbus);
 	return ret;
@@ -3329,7 +3330,7 @@ static int hv_pci_remove(struct hv_device *hdev)
 	wait_for_completion(&hbus->remove_event);
 	destroy_workqueue(hbus->wq);
 
-	hv_put_dom_num(hbus->sysdata.domain);
+	hv_put_dom_num(hbus->bridge->domain_nr);
 
 	kfree(hbus);
 	return ret;
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RFC v2 5/7] PCI: hv: Set up msi domain at bridge probing time

[Boqun Feng]

Since PCI_HYPERV depends on PCI_MSI_IRQ_DOMAIN which selects
GENERIC_MSI_IRQ_DOMAIN, we can use dev_set_msi_domain() to set up the
msi irq domain at probing time, and this works for both x86 and ARM64.

Therefore use it as the preparation for ARM64 Hyper-V PCI support.

As a result, there is no need to set the pci_sysdata::fwnode which is
x86 specific. In addition, make hv_pcibus_device own the fwnode instead
of sysdata to make the code generic.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 drivers/pci/controller/pci-hyperv.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index 05b73d5dfe9d..4ec7839d0adf 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -451,6 +451,7 @@ enum hv_pcibus_state {
 struct hv_pcibus_device {
 	struct pci_sysdata sysdata;
 	struct pci_host_bridge *bridge;
+	struct fwnode_handle *fwnode;
 	/* Protocol version negotiated with the host */
 	enum pci_protocol_version_t protocol_version;
 	enum hv_pcibus_state state;
@@ -1571,7 +1572,7 @@ static int hv_pcie_init_irq_domain(struct hv_pcibus_device *hbus)
 	hbus->msi_info.handler = handle_edge_irq;
 	hbus->msi_info.handler_name = "edge";
 	hbus->msi_info.data = hbus;
-	hbus->irq_domain = pci_msi_create_irq_domain(hbus->sysdata.fwnode,
+	hbus->irq_domain = pci_msi_create_irq_domain(hbus->fwnode,
 						     &hbus->msi_info,
 						     x86_vector_domain);
 	if (!hbus->irq_domain) {
@@ -1580,6 +1581,8 @@ static int hv_pcie_init_irq_domain(struct hv_pcibus_device *hbus)
 		return -ENODEV;
 	}
 
+	dev_set_msi_domain(&hbus->bridge->dev, hbus->irq_domain);
+
 	return 0;
 }
 
@@ -3147,9 +3150,9 @@ static int hv_pci_probe(struct hv_device *hdev,
 		goto unmap;
 	}
 
-	hbus->sysdata.fwnode = irq_domain_alloc_named_fwnode(name);
+	hbus->fwnode = irq_domain_alloc_named_fwnode(name);
 	kfree(name);
-	if (!hbus->sysdata.fwnode) {
+	if (!hbus->fwnode) {
 		ret = -ENOMEM;
 		goto unmap;
 	}
@@ -3227,7 +3230,7 @@ static int hv_pci_probe(struct hv_device *hdev,
 free_irq_domain:
 	irq_domain_remove(hbus->irq_domain);
 free_fwnode:
-	irq_domain_free_fwnode(hbus->sysdata.fwnode);
+	irq_domain_free_fwnode(hbus->fwnode);
 unmap:
 	iounmap(hbus->cfg_addr);
 free_config:
@@ -3325,7 +3328,7 @@ static int hv_pci_remove(struct hv_device *hdev)
 	hv_free_config_window(hbus);
 	hv_pci_free_bridge_windows(hbus);
 	irq_domain_remove(hbus->irq_domain);
-	irq_domain_free_fwnode(hbus->sysdata.fwnode);
+	irq_domain_free_fwnode(hbus->fwnode);
 	put_hvpcibus(hbus);
 	wait_for_completion(&hbus->remove_event);
 	destroy_workqueue(hbus->wq);
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RFC v2 6/7] PCI: arm64: Allow pci_config_window::parent to be NULL

[Boqun Feng]

This is purely a hack, for ARM64 Hyper-V guest, there is no
corresponding ACPI device for the root bridge, so the best we can
provide is an all-zeroed pci_config_window, and in this case make
pcibios_root_bridge_prepare() act as the ACPI device is NULL.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 arch/arm64/kernel/pci.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index e9a6eeb6a694..f159df903ccb 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -83,7 +83,7 @@ int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
 {
 	if (!acpi_disabled) {
 		struct pci_config_window *cfg = bridge->bus->sysdata;
-		struct acpi_device *adev = to_acpi_device(cfg->parent);
+		struct acpi_device *adev = cfg->parent ? to_acpi_device(cfg->parent) : NULL;
 		struct device *bus_dev = &bridge->bus->dev;
 
 		ACPI_COMPANION_SET(&bridge->dev, adev);
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[RFC v2 7/7] PCI: hv: Turn on the host bridge probing on ARM64

[Boqun Feng]

Now we have everything we need, just provide a proper sysdata type for
the bus to use on ARM64 and everything else works.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 drivers/pci/controller/pci-hyperv.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index 4ec7839d0adf..75ff47bedf2a 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -40,6 +40,7 @@
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/delay.h>
 #include <linux/semaphore.h>
 #include <linux/irqdomain.h>
@@ -449,7 +450,11 @@ enum hv_pcibus_state {
 };
 
 struct hv_pcibus_device {
+#ifdef CONFIG_X86
 	struct pci_sysdata sysdata;
+#elif defined(CONFIG_ARM64)
+	struct pci_config_window sysdata;
+#endif
 	struct pci_host_bridge *bridge;
 	struct fwnode_handle *fwnode;
 	/* Protocol version negotiated with the host */
@@ -3102,7 +3107,9 @@ static int hv_pci_probe(struct hv_device *hdev,
 			 dom_req, dom);
 
 	hbus->bridge->domain_nr = dom;
+#ifdef CONFIG_X86
 	hbus->sysdata.domain = dom;
+#endif
 
 	hbus->hdev = hdev;
 	refcount_set(&hbus->remove_lock, 1);
-- 
2.30.2


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC v2 1/7] PCI: Introduce pci_host_bridge::domain_nr

[Mike Rapoport]

On Mon, May 03, 2021 at 10:46:29PM +0800, Boqun Feng wrote:
> Currently we retrieve the PCI domain number of the host bridge from the
> bus sysdata (or pci_config_window if PCI_DOMAINS_GENERIC=y). Actually
> we have the information at PCI host bridge probing time, and it makes
> sense that we store it into pci_host_bridge. One benefit of doing so is
> the requirement for supporting PCI on Hyper-V for ARM64, because the
> host bridge of Hyper-V doesnt' have pci_config_window, whereas ARM64 is
> a PCI_DOMAINS_GENERIC=y arch, so we cannot retrieve the PCI domain
> number from pci_config_window on ARM64 Hyper-V guest.
> 
> As the preparation for ARM64 Hyper-V PCI support, we introduce the
> domain_nr in pci_host_bridge, and set it properly at probing time, then
> for PCI_DOMAINS_GENERIC=y archs, bus domain numbers are set by the
> bridge domain_nr.
> 
> Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> ---
>  arch/arm/kernel/bios32.c              |  2 ++
>  arch/arm/mach-dove/pcie.c             |  2 ++
>  arch/arm/mach-mv78xx0/pcie.c          |  2 ++
>  arch/arm/mach-orion5x/pci.c           |  2 ++
>  arch/arm64/kernel/pci.c               |  3 +--
>  arch/mips/pci/pci-legacy.c            |  2 ++
>  arch/mips/pci/pci-xtalk-bridge.c      |  2 ++
>  drivers/pci/controller/pci-ftpci100.c |  2 ++
>  drivers/pci/controller/pci-mvebu.c    |  2 ++
>  drivers/pci/pci.c                     |  4 ++--
>  drivers/pci/probe.c                   |  7 ++++++-
>  include/linux/pci.h                   | 11 ++++++++---
>  12 files changed, 33 insertions(+), 8 deletions(-)
> 
> diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
> index e7ef2b5bea9c..4942cd681e41 100644
> --- a/arch/arm/kernel/bios32.c
> +++ b/arch/arm/kernel/bios32.c
> @@ -471,6 +471,8 @@ static void pcibios_init_hw(struct device *parent, struct hw_pci *hw,
>  				bridge->sysdata = sys;
>  				bridge->busnr = sys->busnr;
>  				bridge->ops = hw->ops;
> +				if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> +					bridge->domain_nr = pci_bus_find_domain_nr(sys, parent);
>  
>  				ret = pci_scan_root_bus_bridge(bridge);
>  			}
> diff --git a/arch/arm/mach-dove/pcie.c b/arch/arm/mach-dove/pcie.c
> index ee91ac6b5ebf..92eb8484b49b 100644
> --- a/arch/arm/mach-dove/pcie.c
> +++ b/arch/arm/mach-dove/pcie.c
> @@ -167,6 +167,8 @@ dove_pcie_scan_bus(int nr, struct pci_host_bridge *bridge)
>  	bridge->sysdata = sys;
>  	bridge->busnr = sys->busnr;
>  	bridge->ops = &pcie_ops;
> +	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> +		bridge->domain_nr = pci_bus_find_domain_nr(sys, NULL);

The check for CONFIG_PCI_DOMAINS_GENERIC is excessive because there is a
stub for pci_bus_find_domain_nr().

I'm not an expert in PCI, but maybe the repeated assignment of
bridge->domain_nr can live in the generic code, say, in
pci_scan_root_bus_bridge(). E.g. it will set the domain_nr when it is zero.

>  
>  	return pci_scan_root_bus_bridge(bridge);
>  }
> diff --git a/arch/arm/mach-mv78xx0/pcie.c b/arch/arm/mach-mv78xx0/pcie.c
> index 636d84b40466..6703d394bcde 100644
> --- a/arch/arm/mach-mv78xx0/pcie.c
> +++ b/arch/arm/mach-mv78xx0/pcie.c
> @@ -208,6 +208,8 @@ static int __init mv78xx0_pcie_scan_bus(int nr, struct pci_host_bridge *bridge)
>  	bridge->sysdata = sys;
>  	bridge->busnr = sys->busnr;
>  	bridge->ops = &pcie_ops;
> +	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> +		bridge->domain_nr = pci_bus_find_domain_nr(sys, NULL);
>  
>  	return pci_scan_root_bus_bridge(bridge);
>  }
> diff --git a/arch/arm/mach-orion5x/pci.c b/arch/arm/mach-orion5x/pci.c
> index 76951bfbacf5..6257fbd4e705 100644
> --- a/arch/arm/mach-orion5x/pci.c
> +++ b/arch/arm/mach-orion5x/pci.c
> @@ -563,6 +563,8 @@ int __init orion5x_pci_sys_scan_bus(int nr, struct pci_host_bridge *bridge)
>  	bridge->dev.parent = NULL;
>  	bridge->sysdata = sys;
>  	bridge->busnr = sys->busnr;
> +	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> +		bridge->domain_nr = pci_bus_find_domain_nr(sys, NULL);
>  
>  	if (nr == 0) {
>  		bridge->ops = &pcie_ops;
> diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
> index 1006ed2d7c60..e9a6eeb6a694 100644
> --- a/arch/arm64/kernel/pci.c
> +++ b/arch/arm64/kernel/pci.c
> @@ -71,9 +71,8 @@ struct acpi_pci_generic_root_info {
>  	struct pci_config_window	*cfg;	/* config space mapping */
>  };
>  
> -int acpi_pci_bus_find_domain_nr(struct pci_bus *bus)
> +int acpi_pci_bus_find_domain_nr(struct pci_config_window *cfg)
>  {
> -	struct pci_config_window *cfg = bus->sysdata;
>  	struct acpi_device *adev = to_acpi_device(cfg->parent);
>  	struct acpi_pci_root *root = acpi_driver_data(adev);
>  
> diff --git a/arch/mips/pci/pci-legacy.c b/arch/mips/pci/pci-legacy.c
> index 39052de915f3..84ad482be22d 100644
> --- a/arch/mips/pci/pci-legacy.c
> +++ b/arch/mips/pci/pci-legacy.c
> @@ -97,6 +97,8 @@ static void pcibios_scanbus(struct pci_controller *hose)
>  	bridge->ops = hose->pci_ops;
>  	bridge->swizzle_irq = pci_common_swizzle;
>  	bridge->map_irq = pcibios_map_irq;
> +	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> +		bridge->domain_nr = pci_bus_find_domain_nr(hose, NULL);
>  	ret = pci_scan_root_bus_bridge(bridge);
>  	if (ret) {
>  		pci_free_host_bridge(bridge);
> diff --git a/arch/mips/pci/pci-xtalk-bridge.c b/arch/mips/pci/pci-xtalk-bridge.c
> index 50f7d42cca5a..23355ab720be 100644
> --- a/arch/mips/pci/pci-xtalk-bridge.c
> +++ b/arch/mips/pci/pci-xtalk-bridge.c
> @@ -712,6 +712,8 @@ static int bridge_probe(struct platform_device *pdev)
>  	host->ops = &bridge_pci_ops;
>  	host->map_irq = bridge_map_irq;
>  	host->swizzle_irq = pci_common_swizzle;
> +	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> +		host->domain_nr = pci_bus_find_domain_nr(bc, dev);
>  
>  	err = pci_scan_root_bus_bridge(host);
>  	if (err < 0)
> diff --git a/drivers/pci/controller/pci-ftpci100.c b/drivers/pci/controller/pci-ftpci100.c
> index da3cd216da00..cf6eec7f90e1 100644
> --- a/drivers/pci/controller/pci-ftpci100.c
> +++ b/drivers/pci/controller/pci-ftpci100.c
> @@ -439,6 +439,8 @@ static int faraday_pci_probe(struct platform_device *pdev)
>  	host->ops = &faraday_pci_ops;
>  	p = pci_host_bridge_priv(host);
>  	host->sysdata = p;
> +	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> +		host->domain_nr = pci_bus_find_domain_nr(p, dev);
>  	p->dev = dev;
>  
>  	/* Retrieve and enable optional clocks */
> diff --git a/drivers/pci/controller/pci-mvebu.c b/drivers/pci/controller/pci-mvebu.c
> index ed13e81cd691..b329ed2f0956 100644
> --- a/drivers/pci/controller/pci-mvebu.c
> +++ b/drivers/pci/controller/pci-mvebu.c
> @@ -1122,6 +1122,8 @@ static int mvebu_pcie_probe(struct platform_device *pdev)
>  	bridge->sysdata = pcie;
>  	bridge->ops = &mvebu_pcie_ops;
>  	bridge->align_resource = mvebu_pcie_align_resource;
> +	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> +		bridge->domain_nr = pci_bus_find_domain_nr(pcie, dev);
>  
>  	return mvebu_pci_host_probe(bridge);
>  }
> diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
> index 16a17215f633..a249dbf78c34 100644
> --- a/drivers/pci/pci.c
> +++ b/drivers/pci/pci.c
> @@ -6505,10 +6505,10 @@ static int of_pci_bus_find_domain_nr(struct device *parent)
>  	return domain;
>  }
>  
> -int pci_bus_find_domain_nr(struct pci_bus *bus, struct device *parent)
> +int pci_bus_find_domain_nr(void *sysdata, struct device *parent)
>  {
>  	return acpi_disabled ? of_pci_bus_find_domain_nr(parent) :
> -			       acpi_pci_bus_find_domain_nr(bus);
> +			       acpi_pci_bus_find_domain_nr(sysdata);
>  }
>  #endif
>  
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index 953f15abc850..5e71cc5e1b6c 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -899,7 +899,7 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge)
>  	bus->ops = bridge->ops;
>  	bus->number = bus->busn_res.start = bridge->busnr;
>  #ifdef CONFIG_PCI_DOMAINS_GENERIC
> -	bus->domain_nr = pci_bus_find_domain_nr(bus, parent);
> +	bus->domain_nr = bridge->domain_nr;
>  #endif
>  
>  	b = pci_find_bus(pci_domain_nr(bus), bridge->busnr);
> @@ -2974,6 +2974,8 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus,
>  	bridge->sysdata = sysdata;
>  	bridge->busnr = bus;
>  	bridge->ops = ops;
> +	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> +		bridge->domain_nr = pci_bus_find_domain_nr(sysdata, parent);
>  
>  	error = pci_register_host_bridge(bridge);
>  	if (error < 0)
> @@ -2992,6 +2994,9 @@ int pci_host_probe(struct pci_host_bridge *bridge)
>  	struct pci_bus *bus, *child;
>  	int ret;
>  
> +	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> +		bridge->domain_nr = pci_bus_find_domain_nr(bridge->sysdata, bridge->dev.parent);
> +
>  	ret = pci_scan_root_bus_bridge(bridge);
>  	if (ret < 0) {
>  		dev_err(bridge->dev.parent, "Scanning root bridge failed");
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 86c799c97b77..5bbd8417d219 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -534,6 +534,7 @@ struct pci_host_bridge {
>  	struct pci_ops	*child_ops;
>  	void		*sysdata;
>  	int		busnr;
> +	int		domain_nr;
>  	struct list_head windows;	/* resource_entry */
>  	struct list_head dma_ranges;	/* dma ranges resource list */
>  	u8 (*swizzle_irq)(struct pci_dev *, u8 *); /* Platform IRQ swizzler */
> @@ -1637,13 +1638,17 @@ static inline int pci_domain_nr(struct pci_bus *bus)
>  {
>  	return bus->domain_nr;
>  }
> +struct pci_config_window;
>  #ifdef CONFIG_ACPI
> -int acpi_pci_bus_find_domain_nr(struct pci_bus *bus);
> +int acpi_pci_bus_find_domain_nr(struct pci_config_window *cfg);
>  #else
> -static inline int acpi_pci_bus_find_domain_nr(struct pci_bus *bus)
> +static inline int acpi_pci_bus_find_domain_nr(struct pci_config_window *cfg)
>  { return 0; }
>  #endif
> -int pci_bus_find_domain_nr(struct pci_bus *bus, struct device *parent);
> +int pci_bus_find_domain_nr(void *sysdata, struct device *parent);
> +#else
> +static inline int pci_bus_find_domain_nr(void *sysdata, struct device *parent)
> +{ return 0; }
>  #endif
>  
>  /* Some architectures require additional setup to direct VGA traffic */
> -- 
> 2.30.2
> 

-- 
Sincerely yours,
Mike.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC v2 1/7] PCI: Introduce pci_host_bridge::domain_nr

[Rob Herring]

On Tue, May 4, 2021 at 12:16 AM Mike Rapoport <rppt@kernel.org> wrote:
>
> On Mon, May 03, 2021 at 10:46:29PM +0800, Boqun Feng wrote:
> > Currently we retrieve the PCI domain number of the host bridge from the
> > bus sysdata (or pci_config_window if PCI_DOMAINS_GENERIC=y). Actually
> > we have the information at PCI host bridge probing time, and it makes
> > sense that we store it into pci_host_bridge. One benefit of doing so is
> > the requirement for supporting PCI on Hyper-V for ARM64, because the
> > host bridge of Hyper-V doesnt' have pci_config_window, whereas ARM64 is
> > a PCI_DOMAINS_GENERIC=y arch, so we cannot retrieve the PCI domain
> > number from pci_config_window on ARM64 Hyper-V guest.
> >
> > As the preparation for ARM64 Hyper-V PCI support, we introduce the
> > domain_nr in pci_host_bridge, and set it properly at probing time, then
> > for PCI_DOMAINS_GENERIC=y archs, bus domain numbers are set by the
> > bridge domain_nr.
> >
> > Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> > ---
> >  arch/arm/kernel/bios32.c              |  2 ++
> >  arch/arm/mach-dove/pcie.c             |  2 ++
> >  arch/arm/mach-mv78xx0/pcie.c          |  2 ++
> >  arch/arm/mach-orion5x/pci.c           |  2 ++
> >  arch/arm64/kernel/pci.c               |  3 +--
> >  arch/mips/pci/pci-legacy.c            |  2 ++
> >  arch/mips/pci/pci-xtalk-bridge.c      |  2 ++
> >  drivers/pci/controller/pci-ftpci100.c |  2 ++
> >  drivers/pci/controller/pci-mvebu.c    |  2 ++
> >  drivers/pci/pci.c                     |  4 ++--
> >  drivers/pci/probe.c                   |  7 ++++++-
> >  include/linux/pci.h                   | 11 ++++++++---
> >  12 files changed, 33 insertions(+), 8 deletions(-)
> >
> > diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
> > index e7ef2b5bea9c..4942cd681e41 100644
> > --- a/arch/arm/kernel/bios32.c
> > +++ b/arch/arm/kernel/bios32.c
> > @@ -471,6 +471,8 @@ static void pcibios_init_hw(struct device *parent, struct hw_pci *hw,
> >                               bridge->sysdata = sys;
> >                               bridge->busnr = sys->busnr;
> >                               bridge->ops = hw->ops;
> > +                             if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> > +                                     bridge->domain_nr = pci_bus_find_domain_nr(sys, parent);
> >
> >                               ret = pci_scan_root_bus_bridge(bridge);
> >                       }
> > diff --git a/arch/arm/mach-dove/pcie.c b/arch/arm/mach-dove/pcie.c
> > index ee91ac6b5ebf..92eb8484b49b 100644
> > --- a/arch/arm/mach-dove/pcie.c
> > +++ b/arch/arm/mach-dove/pcie.c
> > @@ -167,6 +167,8 @@ dove_pcie_scan_bus(int nr, struct pci_host_bridge *bridge)
> >       bridge->sysdata = sys;
> >       bridge->busnr = sys->busnr;
> >       bridge->ops = &pcie_ops;
> > +     if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> > +             bridge->domain_nr = pci_bus_find_domain_nr(sys, NULL);
>
> The check for CONFIG_PCI_DOMAINS_GENERIC is excessive because there is a
> stub for pci_bus_find_domain_nr().
>
> I'm not an expert in PCI, but maybe the repeated assignment of
> bridge->domain_nr can live in the generic code, say, in
> pci_scan_root_bus_bridge(). E.g. it will set the domain_nr when it is zero.

Yes. There's zero reason h/w drivers should care what the domain_nr is.

There's another issue with domains you should be aware of:

https://lore.kernel.org/linux-pci/20210425152155.mstuxndsoqdbdape@pali/

That may need to be fixed first because deferred probing could cause
the domain to increment each time you retry probe.

Rob

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC v2 1/7] PCI: Introduce pci_host_bridge::domain_nr

[Lorenzo Pieralisi]

On Tue, May 04, 2021 at 08:16:30AM +0300, Mike Rapoport wrote:
> On Mon, May 03, 2021 at 10:46:29PM +0800, Boqun Feng wrote:
> > Currently we retrieve the PCI domain number of the host bridge from the
> > bus sysdata (or pci_config_window if PCI_DOMAINS_GENERIC=y). Actually
> > we have the information at PCI host bridge probing time, and it makes
> > sense that we store it into pci_host_bridge. One benefit of doing so is
> > the requirement for supporting PCI on Hyper-V for ARM64, because the
> > host bridge of Hyper-V doesnt' have pci_config_window, whereas ARM64 is
> > a PCI_DOMAINS_GENERIC=y arch, so we cannot retrieve the PCI domain
> > number from pci_config_window on ARM64 Hyper-V guest.
> > 
> > As the preparation for ARM64 Hyper-V PCI support, we introduce the
> > domain_nr in pci_host_bridge, and set it properly at probing time, then
> > for PCI_DOMAINS_GENERIC=y archs, bus domain numbers are set by the
> > bridge domain_nr.
> > 
> > Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> > ---
> >  arch/arm/kernel/bios32.c              |  2 ++
> >  arch/arm/mach-dove/pcie.c             |  2 ++
> >  arch/arm/mach-mv78xx0/pcie.c          |  2 ++
> >  arch/arm/mach-orion5x/pci.c           |  2 ++
> >  arch/arm64/kernel/pci.c               |  3 +--
> >  arch/mips/pci/pci-legacy.c            |  2 ++
> >  arch/mips/pci/pci-xtalk-bridge.c      |  2 ++
> >  drivers/pci/controller/pci-ftpci100.c |  2 ++
> >  drivers/pci/controller/pci-mvebu.c    |  2 ++
> >  drivers/pci/pci.c                     |  4 ++--
> >  drivers/pci/probe.c                   |  7 ++++++-
> >  include/linux/pci.h                   | 11 ++++++++---
> >  12 files changed, 33 insertions(+), 8 deletions(-)
> > 
> > diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
> > index e7ef2b5bea9c..4942cd681e41 100644
> > --- a/arch/arm/kernel/bios32.c
> > +++ b/arch/arm/kernel/bios32.c
> > @@ -471,6 +471,8 @@ static void pcibios_init_hw(struct device *parent, struct hw_pci *hw,
> >  				bridge->sysdata = sys;
> >  				bridge->busnr = sys->busnr;
> >  				bridge->ops = hw->ops;
> > +				if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> > +					bridge->domain_nr = pci_bus_find_domain_nr(sys, parent);
> >  
> >  				ret = pci_scan_root_bus_bridge(bridge);
> >  			}
> > diff --git a/arch/arm/mach-dove/pcie.c b/arch/arm/mach-dove/pcie.c
> > index ee91ac6b5ebf..92eb8484b49b 100644
> > --- a/arch/arm/mach-dove/pcie.c
> > +++ b/arch/arm/mach-dove/pcie.c
> > @@ -167,6 +167,8 @@ dove_pcie_scan_bus(int nr, struct pci_host_bridge *bridge)
> >  	bridge->sysdata = sys;
> >  	bridge->busnr = sys->busnr;
> >  	bridge->ops = &pcie_ops;
> > +	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> > +		bridge->domain_nr = pci_bus_find_domain_nr(sys, NULL);
> 
> The check for CONFIG_PCI_DOMAINS_GENERIC is excessive because there is a
> stub for pci_bus_find_domain_nr().
> 
> I'm not an expert in PCI, but maybe the repeated assignment of
> bridge->domain_nr can live in the generic code, say, in
> pci_scan_root_bus_bridge(). E.g. it will set the domain_nr when it is zero.
> 
> >  

Yes, this churn should be avoided. We need a sentinel value to detect
whether the domain_nr is invalid (0 is a valid domain) so generic code
(ie pci_scan_root_bus_bridge() and friends) has to call generic
functions to get it (pci_bus_find_domain_nr()).

We can implement it as a flag or function pointer in the struct
pci_host_bridge, if the flag or function pointer is not set the
generic pci_bus_find_domain_nr() should be called.

Lorenzo

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC v2 6/7] PCI: arm64: Allow pci_config_window::parent to be NULL

[Bjorn Helgaas]

Make your subject something like this so it matches previous practice:

  arm64: PCI: ...

The "::" notation probably comes from C++, but doesn't really apply in
C.  In C, we would say "cfg.parent" or "cfg->parent".

But pci_config_window and cfg->parent are probably too low-level for
the subject anyway.  Seems like it should mention Hyper-V, for
instance.

On Mon, May 03, 2021 at 10:46:34PM +0800, Boqun Feng wrote:
> This is purely a hack, for ARM64 Hyper-V guest, there is no
> corresponding ACPI device for the root bridge, so the best we can
> provide is an all-zeroed pci_config_window, and in this case make
> pcibios_root_bridge_prepare() act as the ACPI device is NULL.

Why is there no ACPI device?  Is this a needless arch dependency?  Or
is this related to using DT instead of ACPI?

The cover letter hints that this might be related to
PCI_DOMAINS_GENERIC=y, but that doesn't sound like a very convincing
reason (and the cover letter can provide an overview, but the commit
logs of individual patches shouldn't assume knowledge of the cover
letter).

> Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> ---
>  arch/arm64/kernel/pci.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
> index e9a6eeb6a694..f159df903ccb 100644
> --- a/arch/arm64/kernel/pci.c
> +++ b/arch/arm64/kernel/pci.c
> @@ -83,7 +83,7 @@ int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
>  {
>  	if (!acpi_disabled) {
>  		struct pci_config_window *cfg = bridge->bus->sysdata;
> -		struct acpi_device *adev = to_acpi_device(cfg->parent);
> +		struct acpi_device *adev = cfg->parent ? to_acpi_device(cfg->parent) : NULL;
>  		struct device *bus_dev = &bridge->bus->dev;
>  
>  		ACPI_COMPANION_SET(&bridge->dev, adev);
> -- 
> 2.30.2
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC v2 6/7] PCI: arm64: Allow pci_config_window::parent to be NULL

[Boqun Feng]

On Thu, May 06, 2021 at 05:25:30PM -0500, Bjorn Helgaas wrote:
> Make your subject something like this so it matches previous practice:
> 
>   arm64: PCI: ...
> 

Got it.

> The "::" notation probably comes from C++, but doesn't really apply in
> C.  In C, we would say "cfg.parent" or "cfg->parent".
> 
> But pci_config_window and cfg->parent are probably too low-level for
> the subject anyway.  Seems like it should mention Hyper-V, for
> instance.
> 

I'm going to make the title something like:

   arm64: PCI: Support root bridge preparation for Hyper-V PCI

works for you? Also I will add comment inside the function as
explanation.

> On Mon, May 03, 2021 at 10:46:34PM +0800, Boqun Feng wrote:
> > This is purely a hack, for ARM64 Hyper-V guest, there is no
> > corresponding ACPI device for the root bridge, so the best we can
> > provide is an all-zeroed pci_config_window, and in this case make
> > pcibios_root_bridge_prepare() act as the ACPI device is NULL.
> 
> Why is there no ACPI device?  Is this a needless arch dependency?  Or
> is this related to using DT instead of ACPI?
> 

For Hyper-V virtual PCI host bridges, neither DT or ACPI is used to
describe them, a hypervisor-specific mechanism (VMBus) is used to
enumerate PCI host bridges. So actually on x86, Hyper-V PCI host
bridge's ACPI companion is set as NULL.

> The cover letter hints that this might be related to
> PCI_DOMAINS_GENERIC=y, but that doesn't sound like a very convincing
> reason (and the cover letter can provide an overview, but the commit
> logs of individual patches shouldn't assume knowledge of the cover
> letter).
> 

Ok, I will add a better explanation in the commit log in the next
version. Thanks!

Regards,
Boqun

> > Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> > ---
> >  arch/arm64/kernel/pci.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
> > index e9a6eeb6a694..f159df903ccb 100644
> > --- a/arch/arm64/kernel/pci.c
> > +++ b/arch/arm64/kernel/pci.c
> > @@ -83,7 +83,7 @@ int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
> >  {
> >  	if (!acpi_disabled) {
> >  		struct pci_config_window *cfg = bridge->bus->sysdata;
> > -		struct acpi_device *adev = to_acpi_device(cfg->parent);
> > +		struct acpi_device *adev = cfg->parent ? to_acpi_device(cfg->parent) : NULL;
> >  		struct device *bus_dev = &bridge->bus->dev;
> >  
> >  		ACPI_COMPANION_SET(&bridge->dev, adev);
> > -- 
> > 2.30.2
> > 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC v2 1/7] PCI: Introduce pci_host_bridge::domain_nr

[Boqun Feng]

[Copy Rob]

On Thu, May 06, 2021 at 11:52:45AM +0100, Lorenzo Pieralisi wrote:
> On Tue, May 04, 2021 at 08:16:30AM +0300, Mike Rapoport wrote:
> > On Mon, May 03, 2021 at 10:46:29PM +0800, Boqun Feng wrote:
> > > Currently we retrieve the PCI domain number of the host bridge from the
> > > bus sysdata (or pci_config_window if PCI_DOMAINS_GENERIC=y). Actually
> > > we have the information at PCI host bridge probing time, and it makes
> > > sense that we store it into pci_host_bridge. One benefit of doing so is
> > > the requirement for supporting PCI on Hyper-V for ARM64, because the
> > > host bridge of Hyper-V doesnt' have pci_config_window, whereas ARM64 is
> > > a PCI_DOMAINS_GENERIC=y arch, so we cannot retrieve the PCI domain
> > > number from pci_config_window on ARM64 Hyper-V guest.
> > > 
> > > As the preparation for ARM64 Hyper-V PCI support, we introduce the
> > > domain_nr in pci_host_bridge, and set it properly at probing time, then
> > > for PCI_DOMAINS_GENERIC=y archs, bus domain numbers are set by the
> > > bridge domain_nr.
> > > 
> > > Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> > > ---
> > >  arch/arm/kernel/bios32.c              |  2 ++
> > >  arch/arm/mach-dove/pcie.c             |  2 ++
> > >  arch/arm/mach-mv78xx0/pcie.c          |  2 ++
> > >  arch/arm/mach-orion5x/pci.c           |  2 ++
> > >  arch/arm64/kernel/pci.c               |  3 +--
> > >  arch/mips/pci/pci-legacy.c            |  2 ++
> > >  arch/mips/pci/pci-xtalk-bridge.c      |  2 ++
> > >  drivers/pci/controller/pci-ftpci100.c |  2 ++
> > >  drivers/pci/controller/pci-mvebu.c    |  2 ++
> > >  drivers/pci/pci.c                     |  4 ++--
> > >  drivers/pci/probe.c                   |  7 ++++++-
> > >  include/linux/pci.h                   | 11 ++++++++---
> > >  12 files changed, 33 insertions(+), 8 deletions(-)
> > > 
> > > diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
> > > index e7ef2b5bea9c..4942cd681e41 100644
> > > --- a/arch/arm/kernel/bios32.c
> > > +++ b/arch/arm/kernel/bios32.c
> > > @@ -471,6 +471,8 @@ static void pcibios_init_hw(struct device *parent, struct hw_pci *hw,
> > >  				bridge->sysdata = sys;
> > >  				bridge->busnr = sys->busnr;
> > >  				bridge->ops = hw->ops;
> > > +				if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> > > +					bridge->domain_nr = pci_bus_find_domain_nr(sys, parent);
> > >  
> > >  				ret = pci_scan_root_bus_bridge(bridge);
> > >  			}
> > > diff --git a/arch/arm/mach-dove/pcie.c b/arch/arm/mach-dove/pcie.c
> > > index ee91ac6b5ebf..92eb8484b49b 100644
> > > --- a/arch/arm/mach-dove/pcie.c
> > > +++ b/arch/arm/mach-dove/pcie.c
> > > @@ -167,6 +167,8 @@ dove_pcie_scan_bus(int nr, struct pci_host_bridge *bridge)
> > >  	bridge->sysdata = sys;
> > >  	bridge->busnr = sys->busnr;
> > >  	bridge->ops = &pcie_ops;
> > > +	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> > > +		bridge->domain_nr = pci_bus_find_domain_nr(sys, NULL);
> > 
> > The check for CONFIG_PCI_DOMAINS_GENERIC is excessive because there is a
> > stub for pci_bus_find_domain_nr().
> > 
> > I'm not an expert in PCI, but maybe the repeated assignment of
> > bridge->domain_nr can live in the generic code, say, in
> > pci_scan_root_bus_bridge(). E.g. it will set the domain_nr when it is zero.
> > 
> > >  
> 
> Yes, this churn should be avoided. We need a sentinel value to detect
> whether the domain_nr is invalid (0 is a valid domain) so generic code
> (ie pci_scan_root_bus_bridge() and friends) has to call generic
> functions to get it (pci_bus_find_domain_nr()).
> 

Agreed. Thank you all for the inputs.

According to [1], "PCI Conventional" has at most 256 PCI bus segments
and "PCI Express" has at most 65536 "PCI Segments Groups", so any value
outside [0, 65536] can be used as a sentinel. I'm planning to use -1
like:

	#define PCI_DOMAIN_NR_NOT_SET (-1)

	(in pci_alloc_host_bridge())
	bridge->domain_nr = PCI_DOMAIN_NR_NOT_SET;

	(in pci_register_host_bridge())
	if (bridge->domain_nr == PCI_DOMAIN_NR_NOT_SET)
		bridge->domain_nr = pci_bus_find_domain_nr(...);

Thoughts?

Regards,
Boqun

[1]: https://wiki.osdev.org/PCI_Express

> We can implement it as a flag or function pointer in the struct
> pci_host_bridge, if the flag or function pointer is not set the
> generic pci_bus_find_domain_nr() should be called.
> 
> Lorenzo

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC v2 1/7] PCI: Introduce pci_host_bridge::domain_nr

[Lorenzo Pieralisi]

On Mon, May 10, 2021 at 09:44:29PM +0800, Boqun Feng wrote:
> [Copy Rob]
> 
> On Thu, May 06, 2021 at 11:52:45AM +0100, Lorenzo Pieralisi wrote:
> > On Tue, May 04, 2021 at 08:16:30AM +0300, Mike Rapoport wrote:
> > > On Mon, May 03, 2021 at 10:46:29PM +0800, Boqun Feng wrote:
> > > > Currently we retrieve the PCI domain number of the host bridge from the
> > > > bus sysdata (or pci_config_window if PCI_DOMAINS_GENERIC=y). Actually
> > > > we have the information at PCI host bridge probing time, and it makes
> > > > sense that we store it into pci_host_bridge. One benefit of doing so is
> > > > the requirement for supporting PCI on Hyper-V for ARM64, because the
> > > > host bridge of Hyper-V doesnt' have pci_config_window, whereas ARM64 is
> > > > a PCI_DOMAINS_GENERIC=y arch, so we cannot retrieve the PCI domain
> > > > number from pci_config_window on ARM64 Hyper-V guest.
> > > > 
> > > > As the preparation for ARM64 Hyper-V PCI support, we introduce the
> > > > domain_nr in pci_host_bridge, and set it properly at probing time, then
> > > > for PCI_DOMAINS_GENERIC=y archs, bus domain numbers are set by the
> > > > bridge domain_nr.
> > > > 
> > > > Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> > > > ---
> > > >  arch/arm/kernel/bios32.c              |  2 ++
> > > >  arch/arm/mach-dove/pcie.c             |  2 ++
> > > >  arch/arm/mach-mv78xx0/pcie.c          |  2 ++
> > > >  arch/arm/mach-orion5x/pci.c           |  2 ++
> > > >  arch/arm64/kernel/pci.c               |  3 +--
> > > >  arch/mips/pci/pci-legacy.c            |  2 ++
> > > >  arch/mips/pci/pci-xtalk-bridge.c      |  2 ++
> > > >  drivers/pci/controller/pci-ftpci100.c |  2 ++
> > > >  drivers/pci/controller/pci-mvebu.c    |  2 ++
> > > >  drivers/pci/pci.c                     |  4 ++--
> > > >  drivers/pci/probe.c                   |  7 ++++++-
> > > >  include/linux/pci.h                   | 11 ++++++++---
> > > >  12 files changed, 33 insertions(+), 8 deletions(-)
> > > > 
> > > > diff --git a/arch/arm/kernel/bios32.c b/arch/arm/kernel/bios32.c
> > > > index e7ef2b5bea9c..4942cd681e41 100644
> > > > --- a/arch/arm/kernel/bios32.c
> > > > +++ b/arch/arm/kernel/bios32.c
> > > > @@ -471,6 +471,8 @@ static void pcibios_init_hw(struct device *parent, struct hw_pci *hw,
> > > >  				bridge->sysdata = sys;
> > > >  				bridge->busnr = sys->busnr;
> > > >  				bridge->ops = hw->ops;
> > > > +				if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> > > > +					bridge->domain_nr = pci_bus_find_domain_nr(sys, parent);
> > > >  
> > > >  				ret = pci_scan_root_bus_bridge(bridge);
> > > >  			}
> > > > diff --git a/arch/arm/mach-dove/pcie.c b/arch/arm/mach-dove/pcie.c
> > > > index ee91ac6b5ebf..92eb8484b49b 100644
> > > > --- a/arch/arm/mach-dove/pcie.c
> > > > +++ b/arch/arm/mach-dove/pcie.c
> > > > @@ -167,6 +167,8 @@ dove_pcie_scan_bus(int nr, struct pci_host_bridge *bridge)
> > > >  	bridge->sysdata = sys;
> > > >  	bridge->busnr = sys->busnr;
> > > >  	bridge->ops = &pcie_ops;
> > > > +	if (IS_ENABLED(CONFIG_PCI_DOMAINS_GENERIC))
> > > > +		bridge->domain_nr = pci_bus_find_domain_nr(sys, NULL);
> > > 
> > > The check for CONFIG_PCI_DOMAINS_GENERIC is excessive because there is a
> > > stub for pci_bus_find_domain_nr().
> > > 
> > > I'm not an expert in PCI, but maybe the repeated assignment of
> > > bridge->domain_nr can live in the generic code, say, in
> > > pci_scan_root_bus_bridge(). E.g. it will set the domain_nr when it is zero.
> > > 
> > > >  
> > 
> > Yes, this churn should be avoided. We need a sentinel value to detect
> > whether the domain_nr is invalid (0 is a valid domain) so generic code
> > (ie pci_scan_root_bus_bridge() and friends) has to call generic
> > functions to get it (pci_bus_find_domain_nr()).
> > 
> 
> Agreed. Thank you all for the inputs.
> 
> According to [1], "PCI Conventional" has at most 256 PCI bus segments
> and "PCI Express" has at most 65536 "PCI Segments Groups", so any value
> outside [0, 65536] can be used as a sentinel. I'm planning to use -1
> like:
> 
> 	#define PCI_DOMAIN_NR_NOT_SET (-1)
> 
> 	(in pci_alloc_host_bridge())
> 	bridge->domain_nr = PCI_DOMAIN_NR_NOT_SET;
> 
> 	(in pci_register_host_bridge())
> 	if (bridge->domain_nr == PCI_DOMAIN_NR_NOT_SET)
> 		bridge->domain_nr = pci_bus_find_domain_nr(...);

It should be fine. I'd move the check

if (bridge->domain_nr == PCI_DOMAIN_NR_NOT_SET)

in pci_bus_find_domain_nr()

to make the logic contained in there but that's a nit.

Lorenzo

> Thoughts?
> 
> Regards,
> Boqun
> 
> [1]: https://wiki.osdev.org/PCI_Express
> 
> > We can implement it as a flag or function pointer in the struct
> > pci_host_bridge, if the flag or function pointer is not set the
> > generic pci_bus_find_domain_nr() should be called.
> > 
> > Lorenzo

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel