linux-pci.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64
@ 2021-06-09 16:32 Boqun Feng
  2021-06-09 16:32 ` [RFC v3 1/7] PCI: Introduce domain_nr in pci_host_bridge Boqun Feng
                   ` (7 more replies)
  0 siblings, 8 replies; 12+ messages in thread
From: Boqun Feng @ 2021-06-09 16:32 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-arm-kernel, linux-kernel, linux-hyperv, linux-pci
  Cc: Catalin Marinas, Will Deacon, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Wei Liu, Lorenzo Pieralisi, Rob Herring,
	Clint Sbisa, Boqun Feng, Ard Biesheuvel, Sunil Muthuswamy

Hi Bjorn, Arnd and Marc,

This is the v3 for the preparation of virtual PCI support on Hyper-V
ARM64. Previous versions:

v1:	https://lore.kernel.org/lkml/20210319161956.2838291-1-boqun.feng@gmail.com/
v2:	https://lore.kernel.org/lkml/20210503144635.2297386-1-boqun.feng@gmail.com/

Changes since last version:

*	Use a sentinel value approach instead of calling
	pci_bus_find_domain_nr() for every CONFIG_PCI_DOMAIN_GENERIC=y
	arch as per suggestion from

*	Improve the commit log and comments for patch #6.

*	Rebase to the latest mainline.

The basic problem we need to resolve is that ARM64 is an arch with
PCI_DOMAINS_GENERIC=y, so the bus sysdata is pci_config_window. However,
Hyper-V PCI provides a paravirtualized PCI interface, so there is no
actual pci_config_window for a PCI host bridge, so no information can be
retrieve from the pci_config_window of a Hyper-V virtual PCI bus. Also
there is no corresponding ACPI device for the Hyper-V PCI root bridge.

With this patchset, we could enable the virtual PCI on Hyper-V ARM64
guest with other code under development.

Comments and suggestions are welcome.

Regards,
Boqun

Arnd Bergmann (1):
  PCI: hv: Generify PCI probing

Boqun Feng (6):
  PCI: Introduce domain_nr in pci_host_bridge
  PCI: Allow msi domain set-up at host probing time
  PCI: hv: Use pci_host_bridge::domain_nr for PCI domain
  PCI: hv: Set up msi domain at bridge probing time
  arm64: PCI: Support root bridge preparation for Hyper-V PCI
  PCI: hv: Turn on the host bridge probing on ARM64

 arch/arm64/kernel/pci.c             |  7 ++-
 drivers/pci/controller/pci-hyperv.c | 87 +++++++++++++++++------------
 drivers/pci/probe.c                 |  9 ++-
 include/linux/pci.h                 | 10 ++++
 4 files changed, 73 insertions(+), 40 deletions(-)

-- 
2.30.2


^ permalink raw reply	[flat|nested] 12+ messages in thread

* [RFC v3 1/7] PCI: Introduce domain_nr in pci_host_bridge
  2021-06-09 16:32 [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64 Boqun Feng
@ 2021-06-09 16:32 ` Boqun Feng
  2021-06-09 16:32 ` [RFC v3 2/7] PCI: Allow msi domain set-up at host probing time Boqun Feng
                   ` (6 subsequent siblings)
  7 siblings, 0 replies; 12+ messages in thread
From: Boqun Feng @ 2021-06-09 16:32 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-arm-kernel, linux-kernel, linux-hyperv, linux-pci
  Cc: Catalin Marinas, Will Deacon, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Wei Liu, Lorenzo Pieralisi, Rob Herring,
	Clint Sbisa, Boqun Feng, Ard Biesheuvel, Sunil Muthuswamy

Currently we retrieve the PCI domain number of the host bridge from the
bus sysdata (or pci_config_window if PCI_DOMAINS_GENERIC=y). Actually
we have the information at PCI host bridge probing time, and it makes
sense that we store it into pci_host_bridge. One benefit of doing so is
the requirement for supporting PCI on Hyper-V for ARM64, because the
host bridge of Hyper-V doesn't have pci_config_window, whereas ARM64 is
a PCI_DOMAINS_GENERIC=y arch, so we cannot retrieve the PCI domain
number from pci_config_window on ARM64 Hyper-V guest.

As the preparation for ARM64 Hyper-V PCI support, we introduce the
domain_nr in pci_host_bridge and a sentinel value to allow drivers to
set domain numbers properly at probing time. Currently
CONFIG_PCI_DOMAINS_GENERIC=y archs are only users of this
newly-introduced field.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 drivers/pci/probe.c |  4 +++-
 include/linux/pci.h | 10 ++++++++++
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 275204646c68..d2753097a1b0 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -594,6 +594,7 @@ static void pci_init_host_bridge(struct pci_host_bridge *bridge)
 	bridge->native_pme = 1;
 	bridge->native_ltr = 1;
 	bridge->native_dpc = 1;
+	bridge->domain_nr = PCI_DOMAIN_NR_NOT_SET;
 
 	device_initialize(&bridge->dev);
 }
@@ -898,7 +899,8 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge)
 	bus->ops = bridge->ops;
 	bus->number = bus->busn_res.start = bridge->busnr;
 #ifdef CONFIG_PCI_DOMAINS_GENERIC
-	bus->domain_nr = pci_bus_find_domain_nr(bus, parent);
+	if (bridge->domain_nr == PCI_DOMAIN_NR_NOT_SET)
+		bus->domain_nr = pci_bus_find_domain_nr(bus, parent);
 #endif
 
 	b = pci_find_bus(pci_domain_nr(bus), bridge->busnr);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 24306504226a..d85d41b5abbd 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -526,6 +526,15 @@ static inline int pci_channel_offline(struct pci_dev *pdev)
 	return (pdev->error_state != pci_channel_io_normal);
 }
 
+/*
+ * PCI Conventional has at most 256 PCI bus segments and PCI Express has at
+ * most 65536 "PCI Segments Groups", therefore -1 is not a valid PCI domain
+ * number, and can be used as a sentinel value indicating ->domain_nr is not
+ * set by the driver (and CONFIG_PCI_DOMAINS_GENERIC=y can set it in generic
+ * code).
+ */
+#define PCI_DOMAIN_NR_NOT_SET (-1)
+
 struct pci_host_bridge {
 	struct device	dev;
 	struct pci_bus	*bus;		/* Root bus */
@@ -533,6 +542,7 @@ struct pci_host_bridge {
 	struct pci_ops	*child_ops;
 	void		*sysdata;
 	int		busnr;
+	int		domain_nr;
 	struct list_head windows;	/* resource_entry */
 	struct list_head dma_ranges;	/* dma ranges resource list */
 	u8 (*swizzle_irq)(struct pci_dev *, u8 *); /* Platform IRQ swizzler */
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [RFC v3 2/7] PCI: Allow msi domain set-up at host probing time
  2021-06-09 16:32 [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64 Boqun Feng
  2021-06-09 16:32 ` [RFC v3 1/7] PCI: Introduce domain_nr in pci_host_bridge Boqun Feng
@ 2021-06-09 16:32 ` Boqun Feng
  2021-06-09 16:32 ` [RFC v3 3/7] PCI: hv: Generify PCI probing Boqun Feng
                   ` (5 subsequent siblings)
  7 siblings, 0 replies; 12+ messages in thread
From: Boqun Feng @ 2021-06-09 16:32 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-arm-kernel, linux-kernel, linux-hyperv, linux-pci
  Cc: Catalin Marinas, Will Deacon, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Wei Liu, Lorenzo Pieralisi, Rob Herring,
	Clint Sbisa, Boqun Feng, Ard Biesheuvel, Sunil Muthuswamy,
	Arnd Bergmann

For GENERIC_MSI_IRQ_DOMAIN drivers, we can set up the msi domain via
dev_set_msi_domain() at probing time, and drivers can use this more
generic way to set up the msi domain for the host bridge.

This is the preparation for ARM64 Hyper-V PCI support.

Originally-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 drivers/pci/probe.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index d2753097a1b0..38bee41dedcc 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -829,11 +829,14 @@ static struct irq_domain *pci_host_bridge_msi_domain(struct pci_bus *bus)
 {
 	struct irq_domain *d;
 
+	/* Default set by host bridge driver */
+	d = dev_get_msi_domain(bus->bridge);
 	/*
 	 * Any firmware interface that can resolve the msi_domain
 	 * should be called from here.
 	 */
-	d = pci_host_bridge_of_msi_domain(bus);
+	if (!d)
+		d = pci_host_bridge_of_msi_domain(bus);
 	if (!d)
 		d = pci_host_bridge_acpi_msi_domain(bus);
 
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [RFC v3 3/7] PCI: hv: Generify PCI probing
  2021-06-09 16:32 [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64 Boqun Feng
  2021-06-09 16:32 ` [RFC v3 1/7] PCI: Introduce domain_nr in pci_host_bridge Boqun Feng
  2021-06-09 16:32 ` [RFC v3 2/7] PCI: Allow msi domain set-up at host probing time Boqun Feng
@ 2021-06-09 16:32 ` Boqun Feng
  2021-06-09 16:32 ` [RFC v3 4/7] PCI: hv: Use pci_host_bridge::domain_nr for PCI domain Boqun Feng
                   ` (4 subsequent siblings)
  7 siblings, 0 replies; 12+ messages in thread
From: Boqun Feng @ 2021-06-09 16:32 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-arm-kernel, linux-kernel, linux-hyperv, linux-pci
  Cc: Catalin Marinas, Will Deacon, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Wei Liu, Lorenzo Pieralisi, Rob Herring,
	Clint Sbisa, Boqun Feng, Ard Biesheuvel, Sunil Muthuswamy,
	Arnd Bergmann

From: Arnd Bergmann <arnd@arndb.de>

In order to support ARM64 Hyper-V PCI, we need to set up the bridge at
probing time because ARM64 is a PCI_DOMAIN_GENERIC arch and we don't
have pci_config_window (ARM64 sysdata) for a PCI root bus on Hyper-V, so
it's impossible to retrieve the information (e.g. PCI domains, irq
domains) from bus sysdata on ARM64 after creation.

Originally in create_root_hv_pci_bus(), pci_create_root_bus() is used to
create the root bus and the corresponding bridge based on x86 sysdata.
Now we create a bridge first and then call pci_scan_root_bus_bridge(),
which allows us to do the necessary set-ups for the bridge.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 drivers/pci/controller/pci-hyperv.c | 58 +++++++++++++++--------------
 1 file changed, 30 insertions(+), 28 deletions(-)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index 6511648271b2..ee9f8e27e2e8 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -450,6 +450,7 @@ enum hv_pcibus_state {
 
 struct hv_pcibus_device {
 	struct pci_sysdata sysdata;
+	struct pci_host_bridge *bridge;
 	/* Protocol version negotiated with the host */
 	enum pci_protocol_version_t protocol_version;
 	enum hv_pcibus_state state;
@@ -462,13 +463,10 @@ struct hv_pcibus_device {
 	struct resource *high_mmio_res;
 	struct completion *survey_event;
 	struct completion remove_event;
-	struct pci_bus *pci_bus;
 	spinlock_t config_lock;	/* Avoid two threads writing index page */
 	spinlock_t device_list_lock;	/* Protect lists below */
 	void __iomem *cfg_addr;
 
-	struct list_head resources_for_children;
-
 	struct list_head children;
 	struct list_head dr_list;
 
@@ -1803,7 +1801,7 @@ static void hv_pci_assign_slots(struct hv_pcibus_device *hbus)
 
 		slot_nr = PCI_SLOT(wslot_to_devfn(hpdev->desc.win_slot.slot));
 		snprintf(name, SLOT_NAME_SIZE, "%u", hpdev->desc.ser);
-		hpdev->pci_slot = pci_create_slot(hbus->pci_bus, slot_nr,
+		hpdev->pci_slot = pci_create_slot(hbus->bridge->bus, slot_nr,
 					  name, NULL);
 		if (IS_ERR(hpdev->pci_slot)) {
 			pr_warn("pci_create slot %s failed\n", name);
@@ -1833,7 +1831,7 @@ static void hv_pci_remove_slots(struct hv_pcibus_device *hbus)
 static void hv_pci_assign_numa_node(struct hv_pcibus_device *hbus)
 {
 	struct pci_dev *dev;
-	struct pci_bus *bus = hbus->pci_bus;
+	struct pci_bus *bus = hbus->bridge->bus;
 	struct hv_pci_dev *hv_dev;
 
 	list_for_each_entry(dev, &bus->devices, bus_list) {
@@ -1856,21 +1854,22 @@ static void hv_pci_assign_numa_node(struct hv_pcibus_device *hbus)
  */
 static int create_root_hv_pci_bus(struct hv_pcibus_device *hbus)
 {
-	/* Register the device */
-	hbus->pci_bus = pci_create_root_bus(&hbus->hdev->device,
-					    0, /* bus number is always zero */
-					    &hv_pcifront_ops,
-					    &hbus->sysdata,
-					    &hbus->resources_for_children);
-	if (!hbus->pci_bus)
-		return -ENODEV;
+	int error;
+	struct pci_host_bridge *bridge = hbus->bridge;
+
+	bridge->dev.parent = &hbus->hdev->device;
+	bridge->sysdata = &hbus->sysdata;
+	bridge->ops = &hv_pcifront_ops;
+
+	error = pci_scan_root_bus_bridge(bridge);
+	if (error)
+		return error;
 
 	pci_lock_rescan_remove();
-	pci_scan_child_bus(hbus->pci_bus);
 	hv_pci_assign_numa_node(hbus);
-	pci_bus_assign_resources(hbus->pci_bus);
+	pci_bus_assign_resources(bridge->bus);
 	hv_pci_assign_slots(hbus);
-	pci_bus_add_devices(hbus->pci_bus);
+	pci_bus_add_devices(bridge->bus);
 	pci_unlock_rescan_remove();
 	hbus->state = hv_pcibus_installed;
 	return 0;
@@ -2135,7 +2134,7 @@ static void pci_devices_present_work(struct work_struct *work)
 		 * because there may have been changes.
 		 */
 		pci_lock_rescan_remove();
-		pci_scan_child_bus(hbus->pci_bus);
+		pci_scan_child_bus(hbus->bridge->bus);
 		hv_pci_assign_numa_node(hbus);
 		hv_pci_assign_slots(hbus);
 		pci_unlock_rescan_remove();
@@ -2306,8 +2305,8 @@ static void hv_eject_device_work(struct work_struct *work)
 	/*
 	 * Ejection can come before or after the PCI bus has been set up, so
 	 * attempt to find it and tear down the bus state, if it exists.  This
-	 * must be done without constructs like pci_domain_nr(hbus->pci_bus)
-	 * because hbus->pci_bus may not exist yet.
+	 * must be done without constructs like pci_domain_nr(hbus->bridge->bus)
+	 * because hbus->bridge->bus may not exist yet.
 	 */
 	wslot = wslot_to_devfn(hpdev->desc.win_slot.slot);
 	pdev = pci_get_domain_bus_and_slot(hbus->sysdata.domain, 0, wslot);
@@ -2676,8 +2675,7 @@ static int hv_pci_allocate_bridge_windows(struct hv_pcibus_device *hbus)
 		/* Modify this resource to become a bridge window. */
 		hbus->low_mmio_res->flags |= IORESOURCE_WINDOW;
 		hbus->low_mmio_res->flags &= ~IORESOURCE_BUSY;
-		pci_add_resource(&hbus->resources_for_children,
-				 hbus->low_mmio_res);
+		pci_add_resource(&hbus->bridge->windows, hbus->low_mmio_res);
 	}
 
 	if (hbus->high_mmio_space) {
@@ -2696,8 +2694,7 @@ static int hv_pci_allocate_bridge_windows(struct hv_pcibus_device *hbus)
 		/* Modify this resource to become a bridge window. */
 		hbus->high_mmio_res->flags |= IORESOURCE_WINDOW;
 		hbus->high_mmio_res->flags &= ~IORESOURCE_BUSY;
-		pci_add_resource(&hbus->resources_for_children,
-				 hbus->high_mmio_res);
+		pci_add_resource(&hbus->bridge->windows, hbus->high_mmio_res);
 	}
 
 	return 0;
@@ -3027,6 +3024,7 @@ static void hv_put_dom_num(u16 dom)
 static int hv_pci_probe(struct hv_device *hdev,
 			const struct hv_vmbus_device_id *dev_id)
 {
+	struct pci_host_bridge *bridge;
 	struct hv_pcibus_device *hbus;
 	u16 dom_req, dom;
 	char *name;
@@ -3039,6 +3037,10 @@ static int hv_pci_probe(struct hv_device *hdev,
 	 */
 	BUILD_BUG_ON(sizeof(*hbus) > HV_HYP_PAGE_SIZE);
 
+	bridge = devm_pci_alloc_host_bridge(&hdev->device, 0);
+	if (!bridge)
+		return -ENOMEM;
+
 	/*
 	 * With the recent 59bb47985c1d ("mm, sl[aou]b: guarantee natural
 	 * alignment for kmalloc(power-of-two)"), kzalloc() is able to allocate
@@ -3060,6 +3062,8 @@ static int hv_pci_probe(struct hv_device *hdev,
 	hbus = kzalloc(HV_HYP_PAGE_SIZE, GFP_KERNEL);
 	if (!hbus)
 		return -ENOMEM;
+
+	hbus->bridge = bridge;
 	hbus->state = hv_pcibus_init;
 	hbus->wslot_res_allocated = -1;
 
@@ -3097,7 +3101,6 @@ static int hv_pci_probe(struct hv_device *hdev,
 	refcount_set(&hbus->remove_lock, 1);
 	INIT_LIST_HEAD(&hbus->children);
 	INIT_LIST_HEAD(&hbus->dr_list);
-	INIT_LIST_HEAD(&hbus->resources_for_children);
 	spin_lock_init(&hbus->config_lock);
 	spin_lock_init(&hbus->device_list_lock);
 	spin_lock_init(&hbus->retarget_msi_interrupt_lock);
@@ -3303,9 +3306,9 @@ static int hv_pci_remove(struct hv_device *hdev)
 	if (hbus->state == hv_pcibus_installed) {
 		/* Remove the bus from PCI's point of view. */
 		pci_lock_rescan_remove();
-		pci_stop_root_bus(hbus->pci_bus);
+		pci_stop_root_bus(hbus->bridge->bus);
 		hv_pci_remove_slots(hbus);
-		pci_remove_root_bus(hbus->pci_bus);
+		pci_remove_root_bus(hbus->bridge->bus);
 		pci_unlock_rescan_remove();
 		hbus->state = hv_pcibus_removed;
 	}
@@ -3316,7 +3319,6 @@ static int hv_pci_remove(struct hv_device *hdev)
 
 	iounmap(hbus->cfg_addr);
 	hv_free_config_window(hbus);
-	pci_free_resource_list(&hbus->resources_for_children);
 	hv_pci_free_bridge_windows(hbus);
 	irq_domain_remove(hbus->irq_domain);
 	irq_domain_free_fwnode(hbus->sysdata.fwnode);
@@ -3402,7 +3404,7 @@ static int hv_pci_restore_msi_msg(struct pci_dev *pdev, void *arg)
  */
 static void hv_pci_restore_msi_state(struct hv_pcibus_device *hbus)
 {
-	pci_walk_bus(hbus->pci_bus, hv_pci_restore_msi_msg, NULL);
+	pci_walk_bus(hbus->bridge->bus, hv_pci_restore_msi_msg, NULL);
 }
 
 static int hv_pci_resume(struct hv_device *hdev)
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [RFC v3 4/7] PCI: hv: Use pci_host_bridge::domain_nr for PCI domain
  2021-06-09 16:32 [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64 Boqun Feng
                   ` (2 preceding siblings ...)
  2021-06-09 16:32 ` [RFC v3 3/7] PCI: hv: Generify PCI probing Boqun Feng
@ 2021-06-09 16:32 ` Boqun Feng
  2021-06-09 16:32 ` [RFC v3 5/7] PCI: hv: Set up msi domain at bridge probing time Boqun Feng
                   ` (3 subsequent siblings)
  7 siblings, 0 replies; 12+ messages in thread
From: Boqun Feng @ 2021-06-09 16:32 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-arm-kernel, linux-kernel, linux-hyperv, linux-pci
  Cc: Catalin Marinas, Will Deacon, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Wei Liu, Lorenzo Pieralisi, Rob Herring,
	Clint Sbisa, Boqun Feng, Ard Biesheuvel, Sunil Muthuswamy

No functional change, just store and maintain the PCI domain number in
the generic pci_host_bridge instead of x86 specific pci_sysdata.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 drivers/pci/controller/pci-hyperv.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index ee9f8e27e2e8..f9a644ae3b74 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -2309,7 +2309,7 @@ static void hv_eject_device_work(struct work_struct *work)
 	 * because hbus->bridge->bus may not exist yet.
 	 */
 	wslot = wslot_to_devfn(hpdev->desc.win_slot.slot);
-	pdev = pci_get_domain_bus_and_slot(hbus->sysdata.domain, 0, wslot);
+	pdev = pci_get_domain_bus_and_slot(hbus->bridge->domain_nr, 0, wslot);
 	if (pdev) {
 		pci_lock_rescan_remove();
 		pci_stop_and_remove_bus_device(pdev);
@@ -3095,6 +3095,7 @@ static int hv_pci_probe(struct hv_device *hdev,
 			 "PCI dom# 0x%hx has collision, using 0x%hx",
 			 dom_req, dom);
 
+	hbus->bridge->domain_nr = dom;
 	hbus->sysdata.domain = dom;
 
 	hbus->hdev = hdev;
@@ -3106,7 +3107,7 @@ static int hv_pci_probe(struct hv_device *hdev,
 	spin_lock_init(&hbus->retarget_msi_interrupt_lock);
 	init_completion(&hbus->remove_event);
 	hbus->wq = alloc_ordered_workqueue("hv_pci_%x", 0,
-					   hbus->sysdata.domain);
+					   hbus->bridge->domain_nr);
 	if (!hbus->wq) {
 		ret = -ENOMEM;
 		goto free_dom;
@@ -3233,7 +3234,7 @@ static int hv_pci_probe(struct hv_device *hdev,
 destroy_wq:
 	destroy_workqueue(hbus->wq);
 free_dom:
-	hv_put_dom_num(hbus->sysdata.domain);
+	hv_put_dom_num(hbus->bridge->domain_nr);
 free_bus:
 	kfree(hbus);
 	return ret;
@@ -3326,7 +3327,7 @@ static int hv_pci_remove(struct hv_device *hdev)
 	wait_for_completion(&hbus->remove_event);
 	destroy_workqueue(hbus->wq);
 
-	hv_put_dom_num(hbus->sysdata.domain);
+	hv_put_dom_num(hbus->bridge->domain_nr);
 
 	kfree(hbus);
 	return ret;
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [RFC v3 5/7] PCI: hv: Set up msi domain at bridge probing time
  2021-06-09 16:32 [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64 Boqun Feng
                   ` (3 preceding siblings ...)
  2021-06-09 16:32 ` [RFC v3 4/7] PCI: hv: Use pci_host_bridge::domain_nr for PCI domain Boqun Feng
@ 2021-06-09 16:32 ` Boqun Feng
  2021-06-09 16:32 ` [RFC v3 6/7] arm64: PCI: Support root bridge preparation for Hyper-V PCI Boqun Feng
                   ` (2 subsequent siblings)
  7 siblings, 0 replies; 12+ messages in thread
From: Boqun Feng @ 2021-06-09 16:32 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-arm-kernel, linux-kernel, linux-hyperv, linux-pci
  Cc: Catalin Marinas, Will Deacon, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Wei Liu, Lorenzo Pieralisi, Rob Herring,
	Clint Sbisa, Boqun Feng, Ard Biesheuvel, Sunil Muthuswamy

Since PCI_HYPERV depends on PCI_MSI_IRQ_DOMAIN which selects
GENERIC_MSI_IRQ_DOMAIN, we can use dev_set_msi_domain() to set up the
msi irq domain at probing time, and this works for both x86 and ARM64.

Therefore use it as the preparation for ARM64 Hyper-V PCI support.

As a result, there is no need to set the pci_sysdata::fwnode which is
x86 specific. In addition, make hv_pcibus_device own the fwnode instead
of sysdata to make the code generic.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 drivers/pci/controller/pci-hyperv.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index f9a644ae3b74..16a779ab9ed4 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -451,6 +451,7 @@ enum hv_pcibus_state {
 struct hv_pcibus_device {
 	struct pci_sysdata sysdata;
 	struct pci_host_bridge *bridge;
+	struct fwnode_handle *fwnode;
 	/* Protocol version negotiated with the host */
 	enum pci_protocol_version_t protocol_version;
 	enum hv_pcibus_state state;
@@ -1570,7 +1571,7 @@ static int hv_pcie_init_irq_domain(struct hv_pcibus_device *hbus)
 	hbus->msi_info.handler = handle_edge_irq;
 	hbus->msi_info.handler_name = "edge";
 	hbus->msi_info.data = hbus;
-	hbus->irq_domain = pci_msi_create_irq_domain(hbus->sysdata.fwnode,
+	hbus->irq_domain = pci_msi_create_irq_domain(hbus->fwnode,
 						     &hbus->msi_info,
 						     x86_vector_domain);
 	if (!hbus->irq_domain) {
@@ -1579,6 +1580,8 @@ static int hv_pcie_init_irq_domain(struct hv_pcibus_device *hbus)
 		return -ENODEV;
 	}
 
+	dev_set_msi_domain(&hbus->bridge->dev, hbus->irq_domain);
+
 	return 0;
 }
 
@@ -3144,9 +3147,9 @@ static int hv_pci_probe(struct hv_device *hdev,
 		goto unmap;
 	}
 
-	hbus->sysdata.fwnode = irq_domain_alloc_named_fwnode(name);
+	hbus->fwnode = irq_domain_alloc_named_fwnode(name);
 	kfree(name);
-	if (!hbus->sysdata.fwnode) {
+	if (!hbus->fwnode) {
 		ret = -ENOMEM;
 		goto unmap;
 	}
@@ -3224,7 +3227,7 @@ static int hv_pci_probe(struct hv_device *hdev,
 free_irq_domain:
 	irq_domain_remove(hbus->irq_domain);
 free_fwnode:
-	irq_domain_free_fwnode(hbus->sysdata.fwnode);
+	irq_domain_free_fwnode(hbus->fwnode);
 unmap:
 	iounmap(hbus->cfg_addr);
 free_config:
@@ -3322,7 +3325,7 @@ static int hv_pci_remove(struct hv_device *hdev)
 	hv_free_config_window(hbus);
 	hv_pci_free_bridge_windows(hbus);
 	irq_domain_remove(hbus->irq_domain);
-	irq_domain_free_fwnode(hbus->sysdata.fwnode);
+	irq_domain_free_fwnode(hbus->fwnode);
 	put_hvpcibus(hbus);
 	wait_for_completion(&hbus->remove_event);
 	destroy_workqueue(hbus->wq);
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [RFC v3 6/7] arm64: PCI: Support root bridge preparation for Hyper-V PCI
  2021-06-09 16:32 [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64 Boqun Feng
                   ` (4 preceding siblings ...)
  2021-06-09 16:32 ` [RFC v3 5/7] PCI: hv: Set up msi domain at bridge probing time Boqun Feng
@ 2021-06-09 16:32 ` Boqun Feng
  2021-06-09 16:32 ` [RFC v3 7/7] PCI: hv: Turn on the host bridge probing on ARM64 Boqun Feng
  2021-06-10 15:01 ` [RFC v3 0/7] PCI: hv: Support " Ard Biesheuvel
  7 siblings, 0 replies; 12+ messages in thread
From: Boqun Feng @ 2021-06-09 16:32 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-arm-kernel, linux-kernel, linux-hyperv, linux-pci
  Cc: Catalin Marinas, Will Deacon, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Wei Liu, Lorenzo Pieralisi, Rob Herring,
	Clint Sbisa, Boqun Feng, Ard Biesheuvel, Sunil Muthuswamy

Currently at root bridge preparation, the corresponding ACPI device will
be set as the companion, however for a Hyper-V virtual PCI root bridge,
there is no corresponding ACPI device, because a Hyper-V virtual PCI
root bridge is discovered via VMBus rather than ACPI table. In order to
support this, we need to make pcibios_root_bridge_prepare() work with
cfg->parent being NULL.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 arch/arm64/kernel/pci.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c
index 1006ed2d7c60..020bc268bf06 100644
--- a/arch/arm64/kernel/pci.c
+++ b/arch/arm64/kernel/pci.c
@@ -84,7 +84,12 @@ int pcibios_root_bridge_prepare(struct pci_host_bridge *bridge)
 {
 	if (!acpi_disabled) {
 		struct pci_config_window *cfg = bridge->bus->sysdata;
-		struct acpi_device *adev = to_acpi_device(cfg->parent);
+		/*
+		 * On Hyper-V, there is no corresponding APCI device for a root
+		 * bridge, therefore ->parent is set as NULL by the driver. Then
+		 * 'adev' should be NULL in this case.
+		 */
+		struct acpi_device *adev = cfg->parent ? to_acpi_device(cfg->parent) : NULL;
 		struct device *bus_dev = &bridge->bus->dev;
 
 		ACPI_COMPANION_SET(&bridge->dev, adev);
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [RFC v3 7/7] PCI: hv: Turn on the host bridge probing on ARM64
  2021-06-09 16:32 [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64 Boqun Feng
                   ` (5 preceding siblings ...)
  2021-06-09 16:32 ` [RFC v3 6/7] arm64: PCI: Support root bridge preparation for Hyper-V PCI Boqun Feng
@ 2021-06-09 16:32 ` Boqun Feng
  2021-06-10 15:01 ` [RFC v3 0/7] PCI: hv: Support " Ard Biesheuvel
  7 siblings, 0 replies; 12+ messages in thread
From: Boqun Feng @ 2021-06-09 16:32 UTC (permalink / raw)
  To: Bjorn Helgaas, linux-arm-kernel, linux-kernel, linux-hyperv, linux-pci
  Cc: Catalin Marinas, Will Deacon, K. Y. Srinivasan, Haiyang Zhang,
	Stephen Hemminger, Wei Liu, Lorenzo Pieralisi, Rob Herring,
	Clint Sbisa, Boqun Feng, Ard Biesheuvel, Sunil Muthuswamy

Now we have everything we need, just provide a proper sysdata type for
the bus to use on ARM64 and everything else works.

Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
---
 drivers/pci/controller/pci-hyperv.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index 16a779ab9ed4..271d0b6d4796 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -40,6 +40,7 @@
 #include <linux/kernel.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/pci-ecam.h>
 #include <linux/delay.h>
 #include <linux/semaphore.h>
 #include <linux/irqdomain.h>
@@ -449,7 +450,11 @@ enum hv_pcibus_state {
 };
 
 struct hv_pcibus_device {
+#ifdef CONFIG_X86
 	struct pci_sysdata sysdata;
+#elif defined(CONFIG_ARM64)
+	struct pci_config_window sysdata;
+#endif
 	struct pci_host_bridge *bridge;
 	struct fwnode_handle *fwnode;
 	/* Protocol version negotiated with the host */
@@ -3099,7 +3104,9 @@ static int hv_pci_probe(struct hv_device *hdev,
 			 dom_req, dom);
 
 	hbus->bridge->domain_nr = dom;
+#ifdef CONFIG_X86
 	hbus->sysdata.domain = dom;
+#endif
 
 	hbus->hdev = hdev;
 	refcount_set(&hbus->remove_lock, 1);
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64
  2021-06-09 16:32 [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64 Boqun Feng
                   ` (6 preceding siblings ...)
  2021-06-09 16:32 ` [RFC v3 7/7] PCI: hv: Turn on the host bridge probing on ARM64 Boqun Feng
@ 2021-06-10 15:01 ` Ard Biesheuvel
  2021-06-10 15:42   ` Marc Zyngier
  7 siblings, 1 reply; 12+ messages in thread
From: Ard Biesheuvel @ 2021-06-10 15:01 UTC (permalink / raw)
  To: Boqun Feng, Arnd Bergmann
  Cc: Bjorn Helgaas, Linux ARM, Linux Kernel Mailing List,
	linux-hyperv, PCI, Catalin Marinas, Will Deacon,
	K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger, Wei Liu,
	Lorenzo Pieralisi, Rob Herring, Clint Sbisa, Sunil Muthuswamy

On Wed, 9 Jun 2021 at 18:32, Boqun Feng <boqun.feng@gmail.com> wrote:
>
> Hi Bjorn, Arnd and Marc,
>

Instead of cc'ing Arnd, you cc'ed me (Ard)

> This is the v3 for the preparation of virtual PCI support on Hyper-V
> ARM64. Previous versions:
>
> v1:     https://lore.kernel.org/lkml/20210319161956.2838291-1-boqun.feng@gmail.com/
> v2:     https://lore.kernel.org/lkml/20210503144635.2297386-1-boqun.feng@gmail.com/
>
> Changes since last version:
>
> *       Use a sentinel value approach instead of calling
>         pci_bus_find_domain_nr() for every CONFIG_PCI_DOMAIN_GENERIC=y
>         arch as per suggestion from
>
> *       Improve the commit log and comments for patch #6.
>
> *       Rebase to the latest mainline.
>
> The basic problem we need to resolve is that ARM64 is an arch with
> PCI_DOMAINS_GENERIC=y, so the bus sysdata is pci_config_window. However,
> Hyper-V PCI provides a paravirtualized PCI interface, so there is no
> actual pci_config_window for a PCI host bridge, so no information can be
> retrieve from the pci_config_window of a Hyper-V virtual PCI bus. Also
> there is no corresponding ACPI device for the Hyper-V PCI root bridge.
>
> With this patchset, we could enable the virtual PCI on Hyper-V ARM64
> guest with other code under development.
>
> Comments and suggestions are welcome.
>
> Regards,
> Boqun
>
> Arnd Bergmann (1):
>   PCI: hv: Generify PCI probing
>
> Boqun Feng (6):
>   PCI: Introduce domain_nr in pci_host_bridge
>   PCI: Allow msi domain set-up at host probing time
>   PCI: hv: Use pci_host_bridge::domain_nr for PCI domain
>   PCI: hv: Set up msi domain at bridge probing time
>   arm64: PCI: Support root bridge preparation for Hyper-V PCI
>   PCI: hv: Turn on the host bridge probing on ARM64
>
>  arch/arm64/kernel/pci.c             |  7 ++-
>  drivers/pci/controller/pci-hyperv.c | 87 +++++++++++++++++------------
>  drivers/pci/probe.c                 |  9 ++-
>  include/linux/pci.h                 | 10 ++++
>  4 files changed, 73 insertions(+), 40 deletions(-)
>
> --
> 2.30.2
>

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64
  2021-06-10 15:01 ` [RFC v3 0/7] PCI: hv: Support " Ard Biesheuvel
@ 2021-06-10 15:42   ` Marc Zyngier
  2021-06-10 16:06     ` Boqun Feng
  0 siblings, 1 reply; 12+ messages in thread
From: Marc Zyngier @ 2021-06-10 15:42 UTC (permalink / raw)
  To: Ard Biesheuvel, Boqun Feng
  Cc: Boqun Feng, Arnd Bergmann, Bjorn Helgaas, Linux ARM,
	Linux Kernel Mailing List, linux-hyperv, PCI, Catalin Marinas,
	Will Deacon, K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Wei Liu, Lorenzo Pieralisi, Rob Herring, Clint Sbisa,
	Sunil Muthuswamy

On 2021-06-10 16:01, Ard Biesheuvel wrote:
> On Wed, 9 Jun 2021 at 18:32, Boqun Feng <boqun.feng@gmail.com> wrote:
>> 
>> Hi Bjorn, Arnd and Marc,
>> 
> 
> Instead of cc'ing Arnd, you cc'ed me (Ard)

And I don't know if you intended to Cc me, but you definitely didn't.

Thanks,

         M.

> 
>> This is the v3 for the preparation of virtual PCI support on Hyper-V
>> ARM64. Previous versions:
>> 
>> v1:     
>> https://lore.kernel.org/lkml/20210319161956.2838291-1-boqun.feng@gmail.com/
>> v2:     
>> https://lore.kernel.org/lkml/20210503144635.2297386-1-boqun.feng@gmail.com/
>> 
>> Changes since last version:
>> 
>> *       Use a sentinel value approach instead of calling
>>         pci_bus_find_domain_nr() for every CONFIG_PCI_DOMAIN_GENERIC=y
>>         arch as per suggestion from
>> 
>> *       Improve the commit log and comments for patch #6.
>> 
>> *       Rebase to the latest mainline.
>> 
>> The basic problem we need to resolve is that ARM64 is an arch with
>> PCI_DOMAINS_GENERIC=y, so the bus sysdata is pci_config_window. 
>> However,
>> Hyper-V PCI provides a paravirtualized PCI interface, so there is no
>> actual pci_config_window for a PCI host bridge, so no information can 
>> be
>> retrieve from the pci_config_window of a Hyper-V virtual PCI bus. Also
>> there is no corresponding ACPI device for the Hyper-V PCI root bridge.
>> 
>> With this patchset, we could enable the virtual PCI on Hyper-V ARM64
>> guest with other code under development.
>> 
>> Comments and suggestions are welcome.
>> 
>> Regards,
>> Boqun
>> 
>> Arnd Bergmann (1):
>>   PCI: hv: Generify PCI probing
>> 
>> Boqun Feng (6):
>>   PCI: Introduce domain_nr in pci_host_bridge
>>   PCI: Allow msi domain set-up at host probing time
>>   PCI: hv: Use pci_host_bridge::domain_nr for PCI domain
>>   PCI: hv: Set up msi domain at bridge probing time
>>   arm64: PCI: Support root bridge preparation for Hyper-V PCI
>>   PCI: hv: Turn on the host bridge probing on ARM64
>> 
>>  arch/arm64/kernel/pci.c             |  7 ++-
>>  drivers/pci/controller/pci-hyperv.c | 87 
>> +++++++++++++++++------------
>>  drivers/pci/probe.c                 |  9 ++-
>>  include/linux/pci.h                 | 10 ++++
>>  4 files changed, 73 insertions(+), 40 deletions(-)
>> 
>> --
>> 2.30.2
>> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

-- 
Who you jivin' with that Cosmik Debris?

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64
  2021-06-10 15:42   ` Marc Zyngier
@ 2021-06-10 16:06     ` Boqun Feng
  2021-06-10 16:20       ` Marc Zyngier
  0 siblings, 1 reply; 12+ messages in thread
From: Boqun Feng @ 2021-06-10 16:06 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: Ard Biesheuvel, Arnd Bergmann, Bjorn Helgaas, Linux ARM,
	Linux Kernel Mailing List, linux-hyperv, PCI, Catalin Marinas,
	Will Deacon, K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Wei Liu, Lorenzo Pieralisi, Rob Herring, Clint Sbisa,
	Sunil Muthuswamy

On Thu, Jun 10, 2021 at 04:42:45PM +0100, Marc Zyngier wrote:
> On 2021-06-10 16:01, Ard Biesheuvel wrote:
> > On Wed, 9 Jun 2021 at 18:32, Boqun Feng <boqun.feng@gmail.com> wrote:
> > > 
> > > Hi Bjorn, Arnd and Marc,
> > > 
> > 
> > Instead of cc'ing Arnd, you cc'ed me (Ard)
> 
> And I don't know if you intended to Cc me, but you definitely didn't.
> 

Weird.. seems my sending script got somewhere wrong. Apologies for you
both, and Arnd.. I did intend to Cc you and Arnd.

How do you want this to proceed? I could do a resend right now, or I
could wait for a few days (and see others' feedback) and send a V4 next
week. Sorry again ;-(

Regards,
Boqun

> Thanks,
> 
>         M.
> 
> > 
> > > This is the v3 for the preparation of virtual PCI support on Hyper-V
> > > ARM64. Previous versions:
> > > 
> > > v1:     https://lore.kernel.org/lkml/20210319161956.2838291-1-boqun.feng@gmail.com/
> > > v2:     https://lore.kernel.org/lkml/20210503144635.2297386-1-boqun.feng@gmail.com/
> > > 
> > > Changes since last version:
> > > 
> > > *       Use a sentinel value approach instead of calling
> > >         pci_bus_find_domain_nr() for every CONFIG_PCI_DOMAIN_GENERIC=y
> > >         arch as per suggestion from
> > > 
> > > *       Improve the commit log and comments for patch #6.
> > > 
> > > *       Rebase to the latest mainline.
> > > 
> > > The basic problem we need to resolve is that ARM64 is an arch with
> > > PCI_DOMAINS_GENERIC=y, so the bus sysdata is pci_config_window.
> > > However,
> > > Hyper-V PCI provides a paravirtualized PCI interface, so there is no
> > > actual pci_config_window for a PCI host bridge, so no information
> > > can be
> > > retrieve from the pci_config_window of a Hyper-V virtual PCI bus. Also
> > > there is no corresponding ACPI device for the Hyper-V PCI root bridge.
> > > 
> > > With this patchset, we could enable the virtual PCI on Hyper-V ARM64
> > > guest with other code under development.
> > > 
> > > Comments and suggestions are welcome.
> > > 
> > > Regards,
> > > Boqun
> > > 
> > > Arnd Bergmann (1):
> > >   PCI: hv: Generify PCI probing
> > > 
> > > Boqun Feng (6):
> > >   PCI: Introduce domain_nr in pci_host_bridge
> > >   PCI: Allow msi domain set-up at host probing time
> > >   PCI: hv: Use pci_host_bridge::domain_nr for PCI domain
> > >   PCI: hv: Set up msi domain at bridge probing time
> > >   arm64: PCI: Support root bridge preparation for Hyper-V PCI
> > >   PCI: hv: Turn on the host bridge probing on ARM64
> > > 
> > >  arch/arm64/kernel/pci.c             |  7 ++-
> > >  drivers/pci/controller/pci-hyperv.c | 87
> > > +++++++++++++++++------------
> > >  drivers/pci/probe.c                 |  9 ++-
> > >  include/linux/pci.h                 | 10 ++++
> > >  4 files changed, 73 insertions(+), 40 deletions(-)
> > > 
> > > --
> > > 2.30.2
> > > 
> > 
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 
> -- 
> Who you jivin' with that Cosmik Debris?

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64
  2021-06-10 16:06     ` Boqun Feng
@ 2021-06-10 16:20       ` Marc Zyngier
  0 siblings, 0 replies; 12+ messages in thread
From: Marc Zyngier @ 2021-06-10 16:20 UTC (permalink / raw)
  To: Boqun Feng
  Cc: Ard Biesheuvel, Arnd Bergmann, Bjorn Helgaas, Linux ARM,
	Linux Kernel Mailing List, linux-hyperv, PCI, Catalin Marinas,
	Will Deacon, K. Y. Srinivasan, Haiyang Zhang, Stephen Hemminger,
	Wei Liu, Lorenzo Pieralisi, Rob Herring, Clint Sbisa,
	Sunil Muthuswamy

On 2021-06-10 17:06, Boqun Feng wrote:
> On Thu, Jun 10, 2021 at 04:42:45PM +0100, Marc Zyngier wrote:
>> On 2021-06-10 16:01, Ard Biesheuvel wrote:
>> > On Wed, 9 Jun 2021 at 18:32, Boqun Feng <boqun.feng@gmail.com> wrote:
>> > >
>> > > Hi Bjorn, Arnd and Marc,
>> > >
>> >
>> > Instead of cc'ing Arnd, you cc'ed me (Ard)
>> 
>> And I don't know if you intended to Cc me, but you definitely didn't.
>> 
> 
> Weird.. seems my sending script got somewhere wrong. Apologies for you
> both, and Arnd.. I did intend to Cc you and Arnd.

No worries, it happens (I also used the wrong email address when
replying, so we're even).

> How do you want this to proceed? I could do a resend right now, or I
> could wait for a few days (and see others' feedback) and send a V4 next
> week. Sorry again ;-(

Let the current series simmer on the list for a few days, I can
always eyeball it there if I'm short of patches to review... ;-)

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2021-06-10 16:20 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-09 16:32 [RFC v3 0/7] PCI: hv: Support host bridge probing on ARM64 Boqun Feng
2021-06-09 16:32 ` [RFC v3 1/7] PCI: Introduce domain_nr in pci_host_bridge Boqun Feng
2021-06-09 16:32 ` [RFC v3 2/7] PCI: Allow msi domain set-up at host probing time Boqun Feng
2021-06-09 16:32 ` [RFC v3 3/7] PCI: hv: Generify PCI probing Boqun Feng
2021-06-09 16:32 ` [RFC v3 4/7] PCI: hv: Use pci_host_bridge::domain_nr for PCI domain Boqun Feng
2021-06-09 16:32 ` [RFC v3 5/7] PCI: hv: Set up msi domain at bridge probing time Boqun Feng
2021-06-09 16:32 ` [RFC v3 6/7] arm64: PCI: Support root bridge preparation for Hyper-V PCI Boqun Feng
2021-06-09 16:32 ` [RFC v3 7/7] PCI: hv: Turn on the host bridge probing on ARM64 Boqun Feng
2021-06-10 15:01 ` [RFC v3 0/7] PCI: hv: Support " Ard Biesheuvel
2021-06-10 15:42   ` Marc Zyngier
2021-06-10 16:06     ` Boqun Feng
2021-06-10 16:20       ` Marc Zyngier

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).