[PATCH RFC 0/7] Adding core support for wire-MSI bridges

[PATCH RFC 0/7] Adding core support for wire-MSI bridges

[Marc Zyngier]

There seems to be a new class of interrupt controller out there whose
sole purpose (apart from making everybody's life a nightmare) is to
turn wired interrupts into MSIs.

Instead of considering that the MSIs allocated to a device are for the
direct use of that device, we can turn this set of MSIs into a irq
domain, and use that domain to build a standard irqchip on top of
that.

This requires some (slightly ugly) surgery in both the generic MSI and
platform MSI layers, but the amount of code is actually relatively
small (about +150 LoC so far).

On top of that, we add a dummy driver for a such a bridge, hoping that
this will give enough information to driver authors so that they can
use this new feature. An even more stupid client driver is provided to
show the interrupt stack allocation:

     dummydev dummy-dev: Probing
     dummydev dummy-dev: Allocated IRQ35
     dummydev dummy-dev: Probing OK
     dummydev dummy-dev: IRQ35 hwirq 5 domain msichip_domain_ops
     dummydev dummy-dev: IRQ35 hwirq 0 domain msi_domain_ops
     dummydev dummy-dev: IRQ35 hwirq 8192 domain its_domain_ops
     dummydev dummy-dev: IRQ35 hwirq 8192 domain gic_irq_domain_ops

While this seems to work, it is far from being perfect:

- This is a prototype: error handling is rubbish, and there could be
  better abstractions to use.

- It relies on not declaring this bridge with IRQCHIP_DECLARE in order
  to prevent the DT layer from allocating interrupts behind our back.

- There is some probe ordering issues between the bridge and the wired
  interrupt device, leading to the use of -EPROBE_DEFER.

The last two points could be addressed directly in the OF layer, as
this is a generic device ordering issue (and people are already
working on that).

I'd welcome any comment on that approach (though I'm going to make
myself scarse over the next two weeks).

Marc Zyngier (7):
  platform-msi: Allow MSIs to be allocated in chunks
  platform-msi: Factor out allocation/free of private data
  irqdomain: Make irq_domain_alloc_irqs_recursive available
  genirq/msi: Make the .prepare callback reusable
  genirq/msi: Add msi_domain_populate_irqs
  platform-msi: Allow creation of a MSI-based stacked irq domain
  irqchip: [Example] dummy wired interrupt/MSI bridge driver

 drivers/base/platform-msi.c   | 185 ++++++++++++++++++++--------
 drivers/irqchip/Kconfig       |   7 ++
 drivers/irqchip/Makefile      |   1 +
 drivers/irqchip/irq-msichip.c | 271 ++++++++++++++++++++++++++++++++++++++++++
 include/linux/irqdomain.h     |   3 +
 include/linux/msi.h           |  16 +++
 kernel/irq/irqdomain.c        |   6 +-
 kernel/irq/msi.c              |  47 +++++++-
 8 files changed, 481 insertions(+), 55 deletions(-)
 create mode 100644 drivers/irqchip/irq-msichip.c

-- 
2.1.4

[PATCH RFC 1/7] platform-msi: Allow MSIs to be allocated in chunks

[Marc Zyngier]

MSIs for a given device are normally all allocated in one go.
Make sure the internal code can allocate them one at a time
if required.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/base/platform-msi.c | 32 ++++++++++++++++++++------------
 1 file changed, 20 insertions(+), 12 deletions(-)

diff --git a/drivers/base/platform-msi.c b/drivers/base/platform-msi.c
index 5df4575..6148c78 100644
--- a/drivers/base/platform-msi.c
+++ b/drivers/base/platform-msi.c
@@ -25,6 +25,7 @@
 #include <linux/slab.h>
 
 #define DEV_ID_SHIFT	24
+#define MAX_DEV_MSIS	(1 << (32 - DEV_ID_SHIFT))
 
 /*
  * Internal data structure containing a (made up, but unique) devid
@@ -110,13 +111,16 @@ static void platform_msi_update_chip_ops(struct msi_domain_info *info)
 		chip->irq_write_msi_msg = platform_msi_write_msg;
 }
 
-static void platform_msi_free_descs(struct device *dev)
+static void platform_msi_free_descs(struct device *dev, int base, int nvec)
 {
 	struct msi_desc *desc, *tmp;
 
 	list_for_each_entry_safe(desc, tmp, dev_to_msi_list(dev), list) {
-		list_del(&desc->list);
-		free_msi_entry(desc);
+		if (desc->platform.msi_index >= base &&
+		    desc->platform.msi_index < (base + nvec)) {
+			list_del(&desc->list);
+			free_msi_entry(desc);
+		}
 	}
 }
 
@@ -124,17 +128,22 @@ static int platform_msi_alloc_descs(struct device *dev, int nvec,
 				    struct platform_msi_priv_data *data)
 
 {
-	int i;
+	struct msi_desc *desc;
+	int i, base = 0;
 
-	for (i = 0; i < nvec; i++) {
-		struct msi_desc *desc;
+	if (!list_empty(dev_to_msi_list(dev))) {
+		desc = list_last_entry(dev_to_msi_list(dev),
+				       struct msi_desc, list);
+		base = desc->platform.msi_index + 1;
+	}
 
+	for (i = 0; i < nvec; i++) {
 		desc = alloc_msi_entry(dev);
 		if (!desc)
 			break;
 
 		desc->platform.msi_priv_data = data;
-		desc->platform.msi_index = i;
+		desc->platform.msi_index = base + i;
 		desc->nvec_used = 1;
 
 		list_add_tail(&desc->list, dev_to_msi_list(dev));
@@ -142,7 +151,7 @@ static int platform_msi_alloc_descs(struct device *dev, int nvec,
 
 	if (i != nvec) {
 		/* Clean up the mess */
-		platform_msi_free_descs(dev);
+		platform_msi_free_descs(dev, base, nvec);
 
 		return -ENOMEM;
 	}
@@ -201,8 +210,7 @@ int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
 	 * accordingly (which would impact the max number of MSI
 	 * capable devices).
 	 */
-	if (!dev->msi_domain || !write_msi_msg || !nvec ||
-	    nvec > (1 << (32 - DEV_ID_SHIFT)))
+	if (!dev->msi_domain || !write_msi_msg || !nvec || nvec > MAX_DEV_MSIS)
 		return -EINVAL;
 
 	if (dev->msi_domain->bus_token != DOMAIN_BUS_PLATFORM_MSI) {
@@ -238,7 +246,7 @@ int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
 	return 0;
 
 out_free_desc:
-	platform_msi_free_descs(dev);
+	platform_msi_free_descs(dev, 0, nvec);
 out_free_id:
 	ida_simple_remove(&platform_msi_devid_ida, priv_data->devid);
 out_free_data:
@@ -266,5 +274,5 @@ void platform_msi_domain_free_irqs(struct device *dev)
 	}
 
 	msi_domain_free_irqs(dev->msi_domain, dev);
-	platform_msi_free_descs(dev);
+	platform_msi_free_descs(dev, 0, MAX_DEV_MSIS);
 }
-- 
2.1.4

[PATCH RFC 2/7] platform-msi: Factor out allocation/free of private data

[Marc Zyngier]

As we're going to have multiple paths to allocate/free the
platform-msi private data, factor this out into separate
utility functions.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/base/platform-msi.c | 84 ++++++++++++++++++++++++++-------------------
 1 file changed, 48 insertions(+), 36 deletions(-)

diff --git a/drivers/base/platform-msi.c b/drivers/base/platform-msi.c
index 6148c78..92666cd 100644
--- a/drivers/base/platform-msi.c
+++ b/drivers/base/platform-msi.c
@@ -189,21 +189,11 @@ struct irq_domain *platform_msi_create_irq_domain(struct fwnode_handle *fwnode,
 	return domain;
 }
 
-/**
- * platform_msi_domain_alloc_irqs - Allocate MSI interrupts for @dev
- * @dev:		The device for which to allocate interrupts
- * @nvec:		The number of interrupts to allocate
- * @write_msi_msg:	Callback to write an interrupt message for @dev
- *
- * Returns:
- * Zero for success, or an error code in case of failure
- */
-int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
-				   irq_write_msi_msg_t write_msi_msg)
+static int platform_msi_alloc_priv_data(struct device *dev, unsigned int nvec,
+					irq_write_msi_msg_t write_msi_msg,
+					struct platform_msi_priv_data **data)
 {
-	struct platform_msi_priv_data *priv_data;
-	int err;
-
+	struct platform_msi_priv_data *datap;
 	/*
 	 * Limit the number of interrupts to 256 per device. Should we
 	 * need to bump this up, DEV_ID_SHIFT should be adjusted
@@ -222,22 +212,51 @@ int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
 	if (!list_empty(dev_to_msi_list(dev)))
 		return -EBUSY;
 
-	priv_data = kzalloc(sizeof(*priv_data), GFP_KERNEL);
-	if (!priv_data)
+	datap = *data = kzalloc(sizeof(**data), GFP_KERNEL);
+	if (!datap)
 		return -ENOMEM;
 
-	priv_data->devid = ida_simple_get(&platform_msi_devid_ida,
-					  0, 1 << DEV_ID_SHIFT, GFP_KERNEL);
-	if (priv_data->devid < 0) {
-		err = priv_data->devid;
-		goto out_free_data;
+	datap->devid = ida_simple_get(&platform_msi_devid_ida,
+				      0, 1 << DEV_ID_SHIFT, GFP_KERNEL);
+	if (datap->devid < 0) {
+		int err = datap->devid;
+		kfree(*data);
+		return err;
 	}
 
-	priv_data->write_msg = write_msi_msg;
+	datap->write_msg = write_msi_msg;
+
+	return 0;
+}
+
+static void platform_msi_free_priv_data(struct platform_msi_priv_data *data)
+{
+	ida_simple_remove(&platform_msi_devid_ida, data->devid);
+	kfree(data);
+}
+
+/**
+ * platform_msi_domain_alloc_irqs - Allocate MSI interrupts for @dev
+ * @dev:		The device for which to allocate interrupts
+ * @nvec:		The number of interrupts to allocate
+ * @write_msi_msg:	Callback to write an interrupt message for @dev
+ *
+ * Returns:
+ * Zero for success, or an error code in case of failure
+ */
+int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
+				   irq_write_msi_msg_t write_msi_msg)
+{
+	struct platform_msi_priv_data *priv_data;
+	int err;
+
+	err = platform_msi_alloc_priv_data(dev, nvec, write_msi_msg, &priv_data);
+	if (err)
+		return err;
 
 	err = platform_msi_alloc_descs(dev, nvec, priv_data);
 	if (err)
-		goto out_free_id;
+		goto out_free_priv_data;
 
 	err = msi_domain_alloc_irqs(dev->msi_domain, dev, nvec);
 	if (err)
@@ -247,10 +266,8 @@ int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
 
 out_free_desc:
 	platform_msi_free_descs(dev, 0, nvec);
-out_free_id:
-	ida_simple_remove(&platform_msi_devid_ida, priv_data->devid);
-out_free_data:
-	kfree(priv_data);
+out_free_priv_data:
+	platform_msi_free_priv_data(priv_data);
 
 	return err;
 }
@@ -261,16 +278,11 @@ out_free_data:
  */
 void platform_msi_domain_free_irqs(struct device *dev)
 {
-	struct msi_desc *desc;
-
-	desc = first_msi_entry(dev);
-	if (desc) {
-		struct platform_msi_priv_data *data;
-
-		data = desc->platform.msi_priv_data;
+	if (!list_empty(dev_to_msi_list(dev))) {
+		struct msi_desc *desc;
 
-		ida_simple_remove(&platform_msi_devid_ida, data->devid);
-		kfree(data);
+		desc = first_msi_entry(dev);
+		platform_msi_free_priv_data(desc->platform.msi_priv_data);
 	}
 
 	msi_domain_free_irqs(dev->msi_domain, dev);
-- 
2.1.4

[PATCH RFC 3/7] irqdomain: Make irq_domain_alloc_irqs_recursive available

[Marc Zyngier]

We are soon going to need the MSI layer to call into the domain
allocators. Instead of open coding this, make the standard
irq_domain_alloc_irqs_recursive function available to the MSI
layer.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/linux/irqdomain.h | 3 +++
 kernel/irq/irqdomain.c    | 6 +++---
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/include/linux/irqdomain.h b/include/linux/irqdomain.h
index d5e5c5b..cf96c6a 100644
--- a/include/linux/irqdomain.h
+++ b/include/linux/irqdomain.h
@@ -367,6 +367,9 @@ static inline int irq_domain_alloc_irqs(struct irq_domain *domain,
 	return __irq_domain_alloc_irqs(domain, -1, nr_irqs, node, arg, false);
 }
 
+extern int irq_domain_alloc_irqs_recursive(struct irq_domain *domain,
+					   unsigned int irq_base,
+					   unsigned int nr_irqs, void *arg);
 extern int irq_domain_set_hwirq_and_chip(struct irq_domain *domain,
 					 unsigned int virq,
 					 irq_hw_number_t hwirq,
diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
index 22aa961..1c9973e 100644
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -1125,9 +1125,9 @@ static void irq_domain_free_irqs_recursive(struct irq_domain *domain,
 	}
 }
 
-static int irq_domain_alloc_irqs_recursive(struct irq_domain *domain,
-					   unsigned int irq_base,
-					   unsigned int nr_irqs, void *arg)
+int irq_domain_alloc_irqs_recursive(struct irq_domain *domain,
+				    unsigned int irq_base,
+				    unsigned int nr_irqs, void *arg)
 {
 	int ret = 0;
 	struct irq_domain *parent = domain->parent;
-- 
2.1.4

[PATCH RFC 4/7] genirq/msi: Make the .prepare callback reusable

[Marc Zyngier]

The .prepare callbacks are so far only called from msi_domain_alloc_irqs.
In order to reuse that code, split that code and create a
msi_domain_prepare_irqs function that the existing code can call into.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/linux/msi.h |  4 ++++
 kernel/irq/msi.c    | 20 +++++++++++++++++---
 2 files changed, 21 insertions(+), 3 deletions(-)

diff --git a/include/linux/msi.h b/include/linux/msi.h
index 32a24b9..5bb36a9 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -277,6 +277,10 @@ struct irq_domain *platform_msi_create_irq_domain(struct fwnode_handle *fwnode,
 int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
 				   irq_write_msi_msg_t write_msi_msg);
 void platform_msi_domain_free_irqs(struct device *dev);
+
+/* When an MSI domain is used as an intermediate domain */
+int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev,
+			    int nvec, msi_alloc_info_t *args);
 #endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */
 
 #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c
index 95354bb..8e6e112 100644
--- a/kernel/irq/msi.c
+++ b/kernel/irq/msi.c
@@ -256,6 +256,22 @@ struct irq_domain *msi_create_irq_domain(struct fwnode_handle *fwnode,
 					   &msi_domain_ops, info);
 }
 
+int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev,
+			    int nvec, msi_alloc_info_t *arg)
+{
+	struct msi_domain_info *info = domain->host_data;
+	struct msi_domain_ops *ops = info->ops;
+	int ret;
+
+	ret = ops->msi_check(domain, info, dev);
+	if (ret == 0)
+		ret = ops->msi_prepare(domain, dev, nvec, arg);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
 /**
  * msi_domain_alloc_irqs - Allocate interrupts from a MSI interrupt domain
  * @domain:	The domain to allocate from
@@ -274,9 +290,7 @@ int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev,
 	struct msi_desc *desc;
 	int i, ret, virq = -1;
 
-	ret = ops->msi_check(domain, info, dev);
-	if (ret == 0)
-		ret = ops->msi_prepare(domain, dev, nvec, &arg);
+	ret = msi_domain_prepare_irqs(domain, dev, nvec, &arg);
 	if (ret)
 		return ret;
 
-- 
2.1.4

[PATCH RFC 5/7] genirq/msi: Add msi_domain_populate_irqs

[Marc Zyngier]

To be able to allocate interrupts from the MSI layer down,
add a new msi_domain_populate_irqs entry point.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/linux/msi.h |  2 ++
 kernel/irq/msi.c    | 27 +++++++++++++++++++++++++++
 2 files changed, 29 insertions(+)

diff --git a/include/linux/msi.h b/include/linux/msi.h
index 5bb36a9..e35558c 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -281,6 +281,8 @@ void platform_msi_domain_free_irqs(struct device *dev);
 /* When an MSI domain is used as an intermediate domain */
 int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev,
 			    int nvec, msi_alloc_info_t *args);
+int msi_domain_populate_irqs(struct irq_domain *domain, struct device *dev,
+			     int virq, int nvec, msi_alloc_info_t *args);
 #endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */
 
 #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c
index 8e6e112..53118e9 100644
--- a/kernel/irq/msi.c
+++ b/kernel/irq/msi.c
@@ -272,6 +272,33 @@ int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev,
 	return 0;
 }
 
+int msi_domain_populate_irqs(struct irq_domain *domain, struct device *dev,
+			     int virq, int nvec, msi_alloc_info_t *arg)
+{
+	struct msi_domain_info *info = domain->host_data;
+	struct msi_domain_ops *ops = info->ops;
+	struct msi_desc *desc;
+
+	for_each_msi_entry(desc, dev) {
+		int i, ret;
+
+		if (!(desc->irq >= virq && desc->irq < (virq + nvec)))
+			continue;
+
+		ops->set_desc(arg, desc);
+		/* Assumes the domain mutex is held! */
+		ret = irq_domain_alloc_irqs_recursive(domain, virq, nvec, arg);
+		/* FIXME: Error handling */
+		if (ret)
+			return ret;
+
+		for (i = 0; i < desc->nvec_used; i++)
+			irq_set_msi_desc_off(virq, i, desc);
+	}
+
+	return 0;
+}
+
 /**
  * msi_domain_alloc_irqs - Allocate interrupts from a MSI interrupt domain
  * @domain:	The domain to allocate from
-- 
2.1.4

[PATCH RFC 6/7] platform-msi: Allow creation of a MSI-based stacked irq domain

[Marc Zyngier]

We almost have all the needed bits to be able to create a irq domain
on top of a MSI domain.

For this, we enable a few things:
- the virq is stored in the msi_desc
- device, msi_alloc_info and domain-specific data
  are stored in the platform_pric_data structure
- we introduce a new API for platform-msi:

  /* Create a MSI-based domain */
  struct irq_domain *
  platform_msi_create_device_domain(struct device *dev,
                                    unsigned int nvec,
                                    irq_write_msi_msg_t write_msi_msg,
                                    const struct irq_domain_ops *ops,
                                    void *host_data);

  /* Allocate MSIs in an MSI domain */
  int platform_msi_domain_alloc(struct irq_domain *domain, unsigned int virq,
                              unsigned int nr_irqs);

  /* Obtain the host data passed to platform_msi_create_device_domain */
  void *platform_msi_get_host_data(struct irq_domain *domain);

This now allows a wired irq to MSI bridge to be created.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/base/platform-msi.c | 71 +++++++++++++++++++++++++++++++++++++++++++--
 include/linux/msi.h         | 10 +++++++
 2 files changed, 79 insertions(+), 2 deletions(-)

diff --git a/drivers/base/platform-msi.c b/drivers/base/platform-msi.c
index 92666cd..29e9b63 100644
--- a/drivers/base/platform-msi.c
+++ b/drivers/base/platform-msi.c
@@ -32,6 +32,9 @@
  * and the callback to write the MSI message.
  */
 struct platform_msi_priv_data {
+	struct device		*dev;
+	void 			*host_data;
+	msi_alloc_info_t	arg;
 	irq_write_msi_msg_t	write_msg;
 	int			devid;
 };
@@ -124,8 +127,9 @@ static void platform_msi_free_descs(struct device *dev, int base, int nvec)
 	}
 }
 
-static int platform_msi_alloc_descs(struct device *dev, int nvec,
-				    struct platform_msi_priv_data *data)
+static int platform_msi_alloc_descs_with_irq(struct device *dev, int virq,
+					     int nvec,
+					     struct platform_msi_priv_data *data)
 
 {
 	struct msi_desc *desc;
@@ -145,6 +149,7 @@ static int platform_msi_alloc_descs(struct device *dev, int nvec,
 		desc->platform.msi_priv_data = data;
 		desc->platform.msi_index = base + i;
 		desc->nvec_used = 1;
+		desc->irq = virq ? virq + i : 0;
 
 		list_add_tail(&desc->list, dev_to_msi_list(dev));
 	}
@@ -159,6 +164,13 @@ static int platform_msi_alloc_descs(struct device *dev, int nvec,
 	return 0;
 }
 
+static int platform_msi_alloc_descs(struct device *dev, int nvec,
+				    struct platform_msi_priv_data *data)
+
+{
+	return platform_msi_alloc_descs_with_irq(dev, 0, nvec, data);
+}
+
 /**
  * platform_msi_create_irq_domain - Create a platform MSI interrupt domain
  * @fwnode:		Optional fwnode of the interrupt controller
@@ -225,6 +237,7 @@ static int platform_msi_alloc_priv_data(struct device *dev, unsigned int nvec,
 	}
 
 	datap->write_msg = write_msi_msg;
+	datap->dev = dev;
 
 	return 0;
 }
@@ -288,3 +301,57 @@ void platform_msi_domain_free_irqs(struct device *dev)
 	msi_domain_free_irqs(dev->msi_domain, dev);
 	platform_msi_free_descs(dev, 0, MAX_DEV_MSIS);
 }
+
+void *platform_msi_get_host_data(struct irq_domain *domain)
+{
+	struct platform_msi_priv_data *data = domain->host_data;
+	return data->host_data;
+}
+
+struct irq_domain *
+platform_msi_create_device_domain(struct device *dev,
+				  unsigned int nvec,
+				  irq_write_msi_msg_t write_msi_msg,
+				  const struct irq_domain_ops *ops,
+				  void *host_data)
+{
+	struct platform_msi_priv_data *data;
+	struct irq_domain *domain;
+	int err;
+
+	err = platform_msi_alloc_priv_data(dev, nvec, write_msi_msg, &data);
+	if (err)
+		return NULL;
+
+	data->host_data = host_data;
+	domain = irq_domain_create_hierarchy(dev->msi_domain, 0, nvec,
+					     of_node_to_fwnode(dev->of_node),
+					     ops, data);
+	if (!domain) {
+		platform_msi_free_priv_data(data);
+		return NULL;
+	}
+
+	/* FIXME: Add better error handling... */
+	err = msi_domain_prepare_irqs(domain->parent, dev, nvec, &data->arg);
+	BUG_ON(err);
+
+	return domain;
+}
+
+int platform_msi_domain_alloc(struct irq_domain *domain, unsigned int virq,
+			      unsigned int nr_irqs)
+{
+	struct platform_msi_priv_data *data = domain->host_data;
+	int err;
+
+	err = platform_msi_alloc_descs_with_irq(data->dev, virq, nr_irqs, data);
+	if (err)
+		return err;
+
+	err = msi_domain_populate_irqs(domain->parent, data->dev,
+				       virq, nr_irqs, &data->arg);
+	/* FIXME: memory leak on error */
+	return err;
+}
+
diff --git a/include/linux/msi.h b/include/linux/msi.h
index e35558c..594b2113 100644
--- a/include/linux/msi.h
+++ b/include/linux/msi.h
@@ -172,6 +172,7 @@ struct msi_controller {
 #include <asm/msi.h>
 
 struct irq_domain;
+struct irq_domain_ops;
 struct irq_chip;
 struct device_node;
 struct fwnode_handle;
@@ -283,6 +284,15 @@ int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev,
 			    int nvec, msi_alloc_info_t *args);
 int msi_domain_populate_irqs(struct irq_domain *domain, struct device *dev,
 			     int virq, int nvec, msi_alloc_info_t *args);
+struct irq_domain *
+platform_msi_create_device_domain(struct device *dev,
+				  unsigned int nvec,
+				  irq_write_msi_msg_t write_msi_msg,
+				  const struct irq_domain_ops *ops,
+				  void *host_data);
+int platform_msi_domain_alloc(struct irq_domain *domain, unsigned int virq,
+			      unsigned int nr_irqs);
+void *platform_msi_get_host_data(struct irq_domain *domain);
 #endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */
 
 #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
-- 
2.1.4

[PATCH RFC 7/7] irqchip: [Example] dummy wired interrupt/MSI bridge driver

[Marc Zyngier]

In order to demonstrate how to put together a wire/MSI bridge,
add a dummy driver that doesn't do anything at all, except
for allocating interrupts.

It comes together with an even more stupid client driver that
allocates an interrupt and dump the hierarchy of that interrupt.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/Kconfig       |   7 ++
 drivers/irqchip/Makefile      |   1 +
 drivers/irqchip/irq-msichip.c | 271 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 279 insertions(+)
 create mode 100644 drivers/irqchip/irq-msichip.c

diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 4d7294e..73e38e6 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -193,3 +193,10 @@ config IRQ_MXS
 	def_bool y if MACH_ASM9260 || ARCH_MXS
 	select IRQ_DOMAIN
 	select STMP_DEVICE
+
+config DUMMY_MSICHIP
+	bool "Dummy irqchip on MSI"
+	depends on GENERIC_MSI_IRQ_DOMAIN
+	help
+	  Dummy stacked irqchip on top of platform MSI
+
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 177f78f..682e7dc 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -55,3 +55,4 @@ obj-$(CONFIG_RENESAS_H8S_INTC)		+= irq-renesas-h8s.o
 obj-$(CONFIG_ARCH_SA1100)		+= irq-sa11x0.o
 obj-$(CONFIG_INGENIC_IRQ)		+= irq-ingenic.o
 obj-$(CONFIG_IMX_GPCV2)			+= irq-imx-gpcv2.o
+obj-$(CONFIG_DUMMY_MSICHIP)		+= irq-msichip.o
diff --git a/drivers/irqchip/irq-msichip.c b/drivers/irqchip/irq-msichip.c
new file mode 100644
index 0000000..82b283b
--- /dev/null
+++ b/drivers/irqchip/irq-msichip.c
@@ -0,0 +1,271 @@
+/*
+ * Non-functionnal example for a wired irq <-> MSI bridge
+ *
+ * Copyright (C) 2015 ARM Limited, All Rights Reserved.
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+/*
+ * DT fragment to represent the MSI bridge:
+
+	intc: msichip {
+		compatible = "dummy,msichip";
+		num-msis = 32;
+		interrupt-controller;
+		interrupt-parent = <&gic>;
+		#interrupt-cells = <0x2>;
+		msi-parent = <&its 1234>;
+	};
+
+ * DT fragment to represent the device connected to the bridge:
+
+	dummy-dev {
+		compatible = "dummy,device";
+		interrupt-parent = <intc>;
+		interrupts = <0x5 0x1>;
+	};
+
+ * When "dummy,device" gets probed, it dumps the hierarchy for the
+ * interrupt it has allocated:
+
+	dummydev dummy-dev: Allocated IRQ35
+	dummydev dummy-dev: Probing OK
+	dummydev dummy-dev: IRQ35 hwirq 5 domain msichip_domain_ops
+	dummydev dummy-dev: IRQ35 hwirq 0 domain msi_domain_ops
+	dummydev dummy-dev: IRQ35 hwirq 8192 domain its_domain_ops
+	dummydev dummy-dev: IRQ35 hwirq 8192 domain gic_irq_domain_ops
+ */
+
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/irq.h>
+#include <linux/irqchip.h>
+#include <linux/irqdomain.h>
+#include <linux/module.h>
+#include <linux/msi.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_irq.h>
+#include <linux/of_address.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+static void msichip_mask(struct irq_data *data)
+{
+	/* Do something */
+}
+
+static void msichip_unmask(struct irq_data *data)
+{
+	/* Do something */
+}
+
+static void msichip_eoi(struct irq_data *data)
+{
+	/* Do something */
+}
+
+static int msichip_set_type(struct irq_data *data, unsigned int type)
+{
+	/* Do something */
+	return 0;
+}
+
+static int msichip_retrigger(struct irq_data *data)
+{
+	/* Do something */
+	return 0;
+}
+
+static int msichip_set_affinity(struct irq_data *data,
+				const struct cpumask *dest, bool force)
+{
+	/* Do something */
+	return 0;
+}
+
+static struct irq_chip msichip_chip = {
+	.name			= "MSICHIP",
+	.irq_mask		= msichip_mask,
+	.irq_unmask		= msichip_unmask,
+	.irq_eoi		= msichip_eoi,
+	.irq_set_type		= msichip_set_type,
+	.irq_retrigger		= msichip_retrigger,
+	.irq_set_affinity	= msichip_set_affinity,
+};
+
+static int msichip_domain_translate(struct irq_domain *d,
+				    struct irq_fwspec *fwspec,
+				    unsigned long *hwirq,
+				    unsigned int *type)
+{
+	if (is_of_node(fwspec->fwnode)) {
+		if (fwspec->param_count != 2)
+			return -EINVAL;
+
+		*hwirq = fwspec->param[0];
+		*type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+
+static int msichip_domain_alloc(struct irq_domain *domain, unsigned int virq,
+				   unsigned int nr_irqs, void *arg)
+{
+	int i, err;
+	irq_hw_number_t hwirq;
+	unsigned int type;
+	struct irq_fwspec *fwspec = arg;
+	void *data;
+
+	err = msichip_domain_translate(domain, fwspec, &hwirq, &type);
+	if (err)
+		return err;
+
+	err = platform_msi_domain_alloc(domain, virq, nr_irqs);
+	if (err)
+		return err;
+
+	data = platform_msi_get_host_data(domain);
+	for (i = 0; i < nr_irqs; i++)
+		irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
+					      &msichip_chip, data);
+
+	return 0;
+}
+
+static const struct irq_domain_ops msichip_domain_ops = {
+	.translate	= msichip_domain_translate,
+	.alloc		= msichip_domain_alloc,
+	.free		= irq_domain_free_irqs_common,
+};
+
+struct msichip_data {
+	/* Add whatever you fancy here */
+	struct platform_device *pdev;
+};
+
+static void msichip_write_msi_msg(struct msi_desc *desc, struct msi_msg *msg)
+{
+	/* Do the right thing */
+}
+
+static int msichip_probe(struct platform_device *pdev)
+{
+	struct irq_domain *domain;
+	struct msichip_data *msichip_data;
+	u32 num_msis;
+
+	dev_info(&pdev->dev, "Probing\n");
+	msichip_data = kzalloc(sizeof(*msichip_data), GFP_KERNEL);
+	if (!msichip_data)
+		return -ENOMEM;
+
+	msichip_data->pdev = pdev;
+
+	/* If there is no "num-msi" property, assume 64... */
+	if (of_property_read_u32(pdev->dev.of_node, "num-msis", &num_msis) < 0)
+		num_msis = 64;
+
+	dev_info(&pdev->dev, "allocating %d MSIs\n", num_msis);
+
+	domain = platform_msi_create_device_domain(&pdev->dev, num_msis,
+						   msichip_write_msi_msg,
+						   &msichip_domain_ops,
+						   msichip_data);
+
+	if (!domain){
+		kfree(msichip_data);
+		return -ENOMEM;
+	}
+
+	dev_info(&pdev->dev, "Probing OK\n");
+	return 0;
+}
+
+static const struct of_device_id msichip_of_match[] = {
+	{ .compatible = "dummy,msichip", },
+	{ },
+};
+MODULE_DEVICE_TABLE(of, msichip_of_match);
+
+static struct platform_driver msichip_driver = {
+	.driver = {
+		.name		= "msichip",
+		.of_match_table	= msichip_of_match,
+	},
+	.probe			= msichip_probe,
+};
+/* Do not define this as an irqchip */
+module_platform_driver(msichip_driver);
+
+
+
+/* Driver for a dummy device connected to the MSI bridge */
+static irqreturn_t dummydev_handler(int irq, void *dummy)
+{
+	return IRQ_HANDLED;
+}
+
+static void dummydev_dump_hierarchy(struct device *dev, int irq)
+{
+	struct irq_data *data = irq_get_irq_data(irq);
+
+	while(data) {
+		dev_info(dev, "IRQ%d hwirq %d domain %ps\n",
+			 data->irq, (int)data->hwirq, data->domain->ops);
+		data = data->parent_data;
+	}
+}
+
+static int dummydev_probe(struct platform_device *pdev)
+{
+	int irq;
+
+	dev_info(&pdev->dev, "Probing\n");
+	irq = irq_of_parse_and_map(pdev->dev.of_node, 0);
+	if (!irq) {
+		dev_err(&pdev->dev, "irq allocation failed, defering\n");
+		return -EPROBE_DEFER;
+	}
+
+	dev_info(&pdev->dev, "Allocated IRQ%d\n", irq);
+
+	if (request_irq(irq, dummydev_handler, 0, "dummydev", pdev))
+		return -EINVAL;
+
+	dev_info(&pdev->dev, "Probing OK\n");
+
+	dummydev_dump_hierarchy(&pdev->dev, irq);
+	return 0;
+}
+
+static const struct of_device_id dummydev_of_match[] = {
+	{ .compatible = "dummy,device", },
+	{ },
+};
+MODULE_DEVICE_TABLE(of, dummydev_of_match);
+
+static struct platform_driver dummydev_driver = {
+	.driver = {
+		.name		= "dummydev",
+		.of_match_table	= dummydev_of_match,
+	},
+	.probe			= dummydev_probe,
+};
+
+module_platform_driver(dummydev_driver);
-- 
2.1.4

Re: [PATCH RFC 0/7] Adding core support for wire-MSI bridges

[Arnd Bergmann]

On Thursday 15 October 2015 16:39:21 Marc Zyngier wrote:
> There seems to be a new class of interrupt controller out there whose
> sole purpose (apart from making everybody's life a nightmare) is to
> turn wired interrupts into MSIs.
> 
> Instead of considering that the MSIs allocated to a device are for the
> direct use of that device, we can turn this set of MSIs into a irq
> domain, and use that domain to build a standard irqchip on top of
> that.
> 
> This requires some (slightly ugly) surgery in both the generic MSI and
> platform MSI layers, but the amount of code is actually relatively
> small (about +150 LoC so far).
> 
> On top of that, we add a dummy driver for a such a bridge, hoping that
> this will give enough information to driver authors so that they can
> use this new feature. An even more stupid client driver is provided to
> show the interrupt stack allocation:

I'm pretty sure you've thought of this before and it doesn't work, but
can you explain why we can't just treat this as an edge-triggered
nested irqchip? As long as the weird hardware can be preconfigured
by the bootloader, the device that is attached to it shouldn't
care how the interrupt ends up at the CPU.

	Arnd

Re: [PATCH RFC 0/7] Adding core support for wire-MSI bridges

[Marc Zyngier]

Hi Arnd,

On 15/10/15 16:46, Arnd Bergmann wrote:
> On Thursday 15 October 2015 16:39:21 Marc Zyngier wrote:
>> There seems to be a new class of interrupt controller out there whose
>> sole purpose (apart from making everybody's life a nightmare) is to
>> turn wired interrupts into MSIs.
>>
>> Instead of considering that the MSIs allocated to a device are for the
>> direct use of that device, we can turn this set of MSIs into a irq
>> domain, and use that domain to build a standard irqchip on top of
>> that.
>>
>> This requires some (slightly ugly) surgery in both the generic MSI and
>> platform MSI layers, but the amount of code is actually relatively
>> small (about +150 LoC so far).
>>
>> On top of that, we add a dummy driver for a such a bridge, hoping that
>> this will give enough information to driver authors so that they can
>> use this new feature. An even more stupid client driver is provided to
>> show the interrupt stack allocation:
> 
> I'm pretty sure you've thought of this before and it doesn't work, but
> can you explain why we can't just treat this as an edge-triggered
> nested irqchip? As long as the weird hardware can be preconfigured
> by the bootloader, the device that is attached to it shouldn't
> care how the interrupt ends up at the CPU.

"Preconfigured" is the key word. While you can do something like that if
your hardware treats MSIs just as if they were wired interrupts
(something like GICv2m), it becomes far more hairy if the target of MSIs
is something like a GICv3 ITS (which is the case for HiSilicon mbigen).

The main reason is that the ITS relies on "translation tables" kept in
memory, which the OS has to configure, and handing over pre-configured
tables is not something I'm looking forward to doing. From a CPU point
of view, this is akin entering the kernel with the MMU already on and no
idmap...

The approach taken here is to make the MSI-ness explicit at the irqchip
level, and keep the interrupting device oblivious of that feature. Also,
this relies on the fact that we can have one MSI per wire, meaning that
we don't have to multiplex anything (no nested irqchip), and that we can
rely on hierarchical domains, which simplifies the code (at least for
the irqchip).

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

RE: [PATCH RFC 4/7] genirq/msi: Make the .prepare callback reusable

[Gabriele Paoloni]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="gb2312", Size: 3173 bytes --]

Hi Marc

It looks like this patch fails to apply to 4.3-rc5....

Gab

> -----Original Message-----
> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> owner@vger.kernel.org] On Behalf Of Marc Zyngier
> Sent: Thursday, October 15, 2015 4:39 PM
> To: Thomas Gleixner; Jiang Liu; Jason Cooper
> Cc: majun (F); linux-arm-kernel@lists.infradead.org; linux-
> pci@vger.kernel.org; linux-kernel@vger.kernel.org
> Subject: [PATCH RFC 4/7] genirq/msi: Make the .prepare callback
> reusable
> 
> The .prepare callbacks are so far only called from
> msi_domain_alloc_irqs.
> In order to reuse that code, split that code and create a
> msi_domain_prepare_irqs function that the existing code can call into.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  include/linux/msi.h |  4 ++++
>  kernel/irq/msi.c    | 20 +++++++++++++++++---
>  2 files changed, 21 insertions(+), 3 deletions(-)
> 
> diff --git a/include/linux/msi.h b/include/linux/msi.h
> index 32a24b9..5bb36a9 100644
> --- a/include/linux/msi.h
> +++ b/include/linux/msi.h
> @@ -277,6 +277,10 @@ struct irq_domain
> *platform_msi_create_irq_domain(struct fwnode_handle *fwnode,
>  int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int
> nvec,
>  				   irq_write_msi_msg_t write_msi_msg);
>  void platform_msi_domain_free_irqs(struct device *dev);
> +
> +/* When an MSI domain is used as an intermediate domain */
> +int msi_domain_prepare_irqs(struct irq_domain *domain, struct device
> *dev,
> +			    int nvec, msi_alloc_info_t *args);
>  #endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */
> 
>  #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
> diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c
> index 95354bb..8e6e112 100644
> --- a/kernel/irq/msi.c
> +++ b/kernel/irq/msi.c
> @@ -256,6 +256,22 @@ struct irq_domain *msi_create_irq_domain(struct
> fwnode_handle *fwnode,
>  					   &msi_domain_ops, info);
>  }
> 
> +int msi_domain_prepare_irqs(struct irq_domain *domain, struct device
> *dev,
> +			    int nvec, msi_alloc_info_t *arg)
> +{
> +	struct msi_domain_info *info = domain->host_data;
> +	struct msi_domain_ops *ops = info->ops;
> +	int ret;
> +
> +	ret = ops->msi_check(domain, info, dev);
> +	if (ret == 0)
> +		ret = ops->msi_prepare(domain, dev, nvec, arg);
> +	if (ret)
> +		return ret;
> +
> +	return 0;
> +}
> +
>  /**
>   * msi_domain_alloc_irqs - Allocate interrupts from a MSI interrupt
> domain
>   * @domain:	The domain to allocate from
> @@ -274,9 +290,7 @@ int msi_domain_alloc_irqs(struct irq_domain *domain,
> struct device *dev,
>  	struct msi_desc *desc;
>  	int i, ret, virq = -1;
> 
> -	ret = ops->msi_check(domain, info, dev);
> -	if (ret == 0)
> -		ret = ops->msi_prepare(domain, dev, nvec, &arg);
> +	ret = msi_domain_prepare_irqs(domain, dev, nvec, &arg);
>  	if (ret)
>  		return ret;
> 
> --
> 2.1.4
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [PATCH RFC 4/7] genirq/msi: Make the .prepare callback reusable

[Marc Zyngier]

On 15/10/15 18:24, Gabriele Paoloni wrote:
> Hi Marc
> 
> It looks like this patch fails to apply to 4.3-rc5....

Which is by design. Try tip/irq/core.

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH RFC 0/7] Adding core support for wire-MSI bridges

[Arnd Bergmann]

On Thursday 15 October 2015 17:01:02 Marc Zyngier wrote:
> 
> "Preconfigured" is the key word. While you can do something like that if
> your hardware treats MSIs just as if they were wired interrupts
> (something like GICv2m), it becomes far more hairy if the target of MSIs
> is something like a GICv3 ITS (which is the case for HiSilicon mbigen).
> 
> The main reason is that the ITS relies on "translation tables" kept in
> memory, which the OS has to configure, and handing over pre-configured
> tables is not something I'm looking forward to doing. From a CPU point
> of view, this is akin entering the kernel with the MMU already on and no
> idmap...
> 
> The approach taken here is to make the MSI-ness explicit at the irqchip
> level, and keep the interrupting device oblivious of that feature. Also,
> this relies on the fact that we can have one MSI per wire, meaning that
> we don't have to multiplex anything (no nested irqchip), and that we can
> rely on hierarchical domains, which simplifies the code (at least for
> the irqchip).
> 

Thanks, that already makes things much clearer. Just one more question:
why can't those translation tables be configured statically by the
irqchip driver? Is this all about being able to cut a few cycles
in case of virtualization? I would assume that once you have gone through
the overhead of having both an MSI and a normal interrupt line (with
the need for serialization vs DMA), you can just as well trap to user
space to deliver an IRQ to a guest.

	Arnd

Re: [PATCH RFC 0/7] Adding core support for wire-MSI bridges

[Jiang Liu]

On 2015/10/15 23:39, Marc Zyngier wrote:
> There seems to be a new class of interrupt controller out there whose
> sole purpose (apart from making everybody's life a nightmare) is to
> turn wired interrupts into MSIs.
> 
> Instead of considering that the MSIs allocated to a device are for the
> direct use of that device, we can turn this set of MSIs into a irq
> domain, and use that domain to build a standard irqchip on top of
> that.
Hi Marc,
	There's a working to enable Intel VMD storage device, which
has the similar requirement. Basically a PCIe hierarchy is hidden
behind a parent PCIe device, so we need to use the PCIe irqs on parent
to de-multiple PCIe IRQs from hidden PCIe devices. Seems a chance for
consolidation here.
	cc Keith Busch <keith.busch@intel.com> who is the author of
VMD device driver.
Thanks!
Gerry

> 
> This requires some (slightly ugly) surgery in both the generic MSI and
> platform MSI layers, but the amount of code is actually relatively
> small (about +150 LoC so far).
> 
> On top of that, we add a dummy driver for a such a bridge, hoping that
> this will give enough information to driver authors so that they can
> use this new feature. An even more stupid client driver is provided to
> show the interrupt stack allocation:
> 
>      dummydev dummy-dev: Probing
>      dummydev dummy-dev: Allocated IRQ35
>      dummydev dummy-dev: Probing OK
>      dummydev dummy-dev: IRQ35 hwirq 5 domain msichip_domain_ops
>      dummydev dummy-dev: IRQ35 hwirq 0 domain msi_domain_ops
>      dummydev dummy-dev: IRQ35 hwirq 8192 domain its_domain_ops
>      dummydev dummy-dev: IRQ35 hwirq 8192 domain gic_irq_domain_ops
> 
> While this seems to work, it is far from being perfect:
> 
> - This is a prototype: error handling is rubbish, and there could be
>   better abstractions to use.
> 
> - It relies on not declaring this bridge with IRQCHIP_DECLARE in order
>   to prevent the DT layer from allocating interrupts behind our back.
> 
> - There is some probe ordering issues between the bridge and the wired
>   interrupt device, leading to the use of -EPROBE_DEFER.
> 
> The last two points could be addressed directly in the OF layer, as
> this is a generic device ordering issue (and people are already
> working on that).
> 
> I'd welcome any comment on that approach (though I'm going to make
> myself scarse over the next two weeks).
> 
> Marc Zyngier (7):
>   platform-msi: Allow MSIs to be allocated in chunks
>   platform-msi: Factor out allocation/free of private data
>   irqdomain: Make irq_domain_alloc_irqs_recursive available
>   genirq/msi: Make the .prepare callback reusable
>   genirq/msi: Add msi_domain_populate_irqs
>   platform-msi: Allow creation of a MSI-based stacked irq domain
>   irqchip: [Example] dummy wired interrupt/MSI bridge driver
> 
>  drivers/base/platform-msi.c   | 185 ++++++++++++++++++++--------
>  drivers/irqchip/Kconfig       |   7 ++
>  drivers/irqchip/Makefile      |   1 +
>  drivers/irqchip/irq-msichip.c | 271 ++++++++++++++++++++++++++++++++++++++++++
>  include/linux/irqdomain.h     |   3 +
>  include/linux/msi.h           |  16 +++
>  kernel/irq/irqdomain.c        |   6 +-
>  kernel/irq/msi.c              |  47 +++++++-
>  8 files changed, 481 insertions(+), 55 deletions(-)
>  create mode 100644 drivers/irqchip/irq-msichip.c
> 

Re: [PATCH RFC 4/7] genirq/msi: Make the .prepare callback reusable

[Jiang Liu]

On 2015/10/15 23:39, Marc Zyngier wrote:
<snit>
> +int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev,
> +			    int nvec, msi_alloc_info_t *arg)
> +{
> +	struct msi_domain_info *info = domain->host_data;
> +	struct msi_domain_ops *ops = info->ops;
> +	int ret;
> +
> +	ret = ops->msi_check(domain, info, dev);
> +	if (ret == 0)
> +		ret = ops->msi_prepare(domain, dev, nvec, arg);
> +	if (ret)
> +		return ret;
> +
> +	return 0;
Seems a simple "return ret" is enough:)

Re: [PATCH RFC 2/7] platform-msi: Factor out allocation/free of private data

[Jiang Liu]

On 2015/10/15 23:39, Marc Zyngier wrote:
> As we're going to have multiple paths to allocate/free the
> platform-msi private data, factor this out into separate
> utility functions.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
>  drivers/base/platform-msi.c | 84 ++++++++++++++++++++++++++-------------------
>  1 file changed, 48 insertions(+), 36 deletions(-)
> 
> diff --git a/drivers/base/platform-msi.c b/drivers/base/platform-msi.c
> index 6148c78..92666cd 100644
> --- a/drivers/base/platform-msi.c
> +++ b/drivers/base/platform-msi.c
> @@ -189,21 +189,11 @@ struct irq_domain *platform_msi_create_irq_domain(struct fwnode_handle *fwnode,
>  	return domain;
>  }
>  
> -/**
> - * platform_msi_domain_alloc_irqs - Allocate MSI interrupts for @dev
> - * @dev:		The device for which to allocate interrupts
> - * @nvec:		The number of interrupts to allocate
> - * @write_msi_msg:	Callback to write an interrupt message for @dev
> - *
> - * Returns:
> - * Zero for success, or an error code in case of failure
> - */
> -int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
> -				   irq_write_msi_msg_t write_msi_msg)
> +static int platform_msi_alloc_priv_data(struct device *dev, unsigned int nvec,
> +					irq_write_msi_msg_t write_msi_msg,
> +					struct platform_msi_priv_data **data)
How about making platform_msi_alloc_priv_data() return a pointer
instead of an int, that may simplify the code a bit.

Re: [PATCH RFC 0/7] Adding core support for wire-MSI bridges

[Marc Zyngier]

On 15/10/15 20:16, Arnd Bergmann wrote:
> On Thursday 15 October 2015 17:01:02 Marc Zyngier wrote:
>>
>> "Preconfigured" is the key word. While you can do something like that if
>> your hardware treats MSIs just as if they were wired interrupts
>> (something like GICv2m), it becomes far more hairy if the target of MSIs
>> is something like a GICv3 ITS (which is the case for HiSilicon mbigen).
>>
>> The main reason is that the ITS relies on "translation tables" kept in
>> memory, which the OS has to configure, and handing over pre-configured
>> tables is not something I'm looking forward to doing. From a CPU point
>> of view, this is akin entering the kernel with the MMU already on and no
>> idmap...
>>
>> The approach taken here is to make the MSI-ness explicit at the irqchip
>> level, and keep the interrupting device oblivious of that feature. Also,
>> this relies on the fact that we can have one MSI per wire, meaning that
>> we don't have to multiplex anything (no nested irqchip), and that we can
>> rely on hierarchical domains, which simplifies the code (at least for
>> the irqchip).
>>
> 
> Thanks, that already makes things much clearer. Just one more question:
> why can't those translation tables be configured statically by the
> irqchip driver? Is this all about being able to cut a few cycles
> in case of virtualization? 

Having a static configuration, while doable, complicates things for
everybody else. The LPI number used by the irqchip would need to be put
an some "exclusion list" to make sure it is not reallocated for other
subsystems (e.g PCI). The translation tables also define the target CPU,
which could cause interesting problems once combined with CPU hotplug if
the ITS is not completely in control of it.

I'm not really getting your point about virtualization though.

> I would assume that once you have gone through the overhead of having
> both an MSI and a normal interrupt line (with the need for
> serialization vs DMA), you can just as well trap to user space to
> deliver an IRQ to a guest.

The whole idea behind this bridge is to move wired interrupts to the
periphery of a SoC. I don't think virtualization was part of the
equation, but of course I can't speak for the "geniuses" behind the idea.

Or maybe I'm reading your question the wrong way, which is entirely
possible given the lack of caffeine.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH RFC 0/7] Adding core support for wire-MSI bridges

[Arnd Bergmann]

On Friday 16 October 2015 09:03:14 Marc Zyngier wrote:
> On 15/10/15 20:16, Arnd Bergmann wrote:
> > On Thursday 15 October 2015 17:01:02 Marc Zyngier wrote:
> >>
> >> "Preconfigured" is the key word. While you can do something like that if
> >> your hardware treats MSIs just as if they were wired interrupts
> >> (something like GICv2m), it becomes far more hairy if the target of MSIs
> >> is something like a GICv3 ITS (which is the case for HiSilicon mbigen).
> >>
> >> The main reason is that the ITS relies on "translation tables" kept in
> >> memory, which the OS has to configure, and handing over pre-configured
> >> tables is not something I'm looking forward to doing. From a CPU point
> >> of view, this is akin entering the kernel with the MMU already on and no
> >> idmap...
> >>
> >> The approach taken here is to make the MSI-ness explicit at the irqchip
> >> level, and keep the interrupting device oblivious of that feature. Also,
> >> this relies on the fact that we can have one MSI per wire, meaning that
> >> we don't have to multiplex anything (no nested irqchip), and that we can
> >> rely on hierarchical domains, which simplifies the code (at least for
> >> the irqchip).
> >>
> > 
> > Thanks, that already makes things much clearer. Just one more question:
> > why can't those translation tables be configured statically by the
> > irqchip driver? Is this all about being able to cut a few cycles
> > in case of virtualization? 
> 
> Having a static configuration, while doable, complicates things for
> everybody else. The LPI number used by the irqchip would need to be put
> an some "exclusion list" to make sure it is not reallocated for other
> subsystems (e.g PCI). The translation tables also define the target CPU,
> which could cause interesting problems once combined with CPU hotplug if
> the ITS is not completely in control of it.
> 
> I'm not really getting your point about virtualization though.

I think I'm mainly still confused by how MSI is implemented on
the CPU side. Your explanation makes sense though.

> > I would assume that once you have gone through the overhead of having
> > both an MSI and a normal interrupt line (with the need for
> > serialization vs DMA), you can just as well trap to user space to
> > deliver an IRQ to a guest.
> 
> The whole idea behind this bridge is to move wired interrupts to the
> periphery of a SoC. I don't think virtualization was part of the
> equation, but of course I can't speak for the "geniuses" behind the idea.
> 
> Or maybe I'm reading your question the wrong way, which is entirely
> possible given the lack of caffeine.

No, I think I get it now.

	Arnd

Re: [PATCH RFC 0/7] Adding core support for wire-MSI bridges

[Marc Zyngier]

Hi Gerry,

On 16/10/15 02:55, Jiang Liu wrote:
> On 2015/10/15 23:39, Marc Zyngier wrote:
>> There seems to be a new class of interrupt controller out there whose
>> sole purpose (apart from making everybody's life a nightmare) is to
>> turn wired interrupts into MSIs.
>>
>> Instead of considering that the MSIs allocated to a device are for the
>> direct use of that device, we can turn this set of MSIs into a irq
>> domain, and use that domain to build a standard irqchip on top of
>> that.
> Hi Marc,
> 	There's a working to enable Intel VMD storage device, which
> has the similar requirement. Basically a PCIe hierarchy is hidden
> behind a parent PCIe device, so we need to use the PCIe irqs on parent
> to de-multiple PCIe IRQs from hidden PCIe devices. Seems a chance for
> consolidation here.

Do you know if there is a 1-1 mapping between the interrupts seen by the
parent device and those seen by the hidden devices? Or is it a case of
having to demultiplex the MSIs? Looks like the former, but I'd like to
be sure.

I just had a quick look at the code there:
https://lkml.org/lkml/2015/8/27/674

Is there anything more recent?

> 	cc Keith Busch <keith.busch@intel.com> who is the author of
> VMD device driver.

Sure, will do when I repost this (probably in a few weeks), and assuming
this fits the bill for Thomas and the MBIGEN folks.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH RFC 4/7] genirq/msi: Make the .prepare callback reusable

[Marc Zyngier]

On 16/10/15 06:45, Jiang Liu wrote:
> On 2015/10/15 23:39, Marc Zyngier wrote:
> <snit>
>> +int msi_domain_prepare_irqs(struct irq_domain *domain, struct device *dev,
>> +			    int nvec, msi_alloc_info_t *arg)
>> +{
>> +	struct msi_domain_info *info = domain->host_data;
>> +	struct msi_domain_ops *ops = info->ops;
>> +	int ret;
>> +
>> +	ret = ops->msi_check(domain, info, dev);
>> +	if (ret == 0)
>> +		ret = ops->msi_prepare(domain, dev, nvec, arg);
>> +	if (ret)
>> +		return ret;
>> +
>> +	return 0;
> Seems a simple "return ret" is enough:)

Ah! yes! ;-)

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH RFC 2/7] platform-msi: Factor out allocation/free of private data

[Marc Zyngier]

On 16/10/15 06:46, Jiang Liu wrote:
> On 2015/10/15 23:39, Marc Zyngier wrote:
>> As we're going to have multiple paths to allocate/free the
>> platform-msi private data, factor this out into separate
>> utility functions.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>>  drivers/base/platform-msi.c | 84 ++++++++++++++++++++++++++-------------------
>>  1 file changed, 48 insertions(+), 36 deletions(-)
>>
>> diff --git a/drivers/base/platform-msi.c b/drivers/base/platform-msi.c
>> index 6148c78..92666cd 100644
>> --- a/drivers/base/platform-msi.c
>> +++ b/drivers/base/platform-msi.c
>> @@ -189,21 +189,11 @@ struct irq_domain *platform_msi_create_irq_domain(struct fwnode_handle *fwnode,
>>  	return domain;
>>  }
>>  
>> -/**
>> - * platform_msi_domain_alloc_irqs - Allocate MSI interrupts for @dev
>> - * @dev:		The device for which to allocate interrupts
>> - * @nvec:		The number of interrupts to allocate
>> - * @write_msi_msg:	Callback to write an interrupt message for @dev
>> - *
>> - * Returns:
>> - * Zero for success, or an error code in case of failure
>> - */
>> -int platform_msi_domain_alloc_irqs(struct device *dev, unsigned int nvec,
>> -				   irq_write_msi_msg_t write_msi_msg)
>> +static int platform_msi_alloc_priv_data(struct device *dev, unsigned int nvec,
>> +					irq_write_msi_msg_t write_msi_msg,
>> +					struct platform_msi_priv_data **data)
> How about making platform_msi_alloc_priv_data() return a pointer
> instead of an int, that may simplify the code a bit.
> 

That's a good point. I could encode the error code in the return pointer
(PTR_ERR).

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

RE: [PATCH RFC 4/7] genirq/msi: Make the .prepare callback reusable

[Gabriele Paoloni]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="gb2312", Size: 1153 bytes --]

> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel-
> owner@vger.kernel.org] On Behalf Of Marc Zyngier
> Sent: Thursday, October 15, 2015 6:39 PM
> To: Gabriele Paoloni; Thomas Gleixner; Jiang Liu; Jason Cooper
> Cc: majun (F); linux-arm-kernel@lists.infradead.org; linux-
> pci@vger.kernel.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH RFC 4/7] genirq/msi: Make the .prepare callback
> reusable
> 
> On 15/10/15 18:24, Gabriele Paoloni wrote:
> > Hi Marc
> >
> > It looks like this patch fails to apply to 4.3-rc5....
> 
> Which is by design. Try tip/irq/core.

Oops sorry, got it!

Thanks, applies ok to tip/irq/core

Gab


> 
> 	M.
> --
> Jazz is not dead. It just smells funny...
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [PATCH RFC 7/7] irqchip: [Example] dummy wired interrupt/MSI bridge driver

[majun (F)]

Hi Marc:

在 2015/10/15 23:39, Marc Zyngier 写道:
> In order to demonstrate how to put together a wire/MSI bridge,
> add a dummy driver that doesn't do anything at all, except
> for allocating interrupts.
> 
> It comes together with an even more stupid client driver that
> allocates an interrupt and dump the hierarchy of that interrupt.
> 
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
[...]
> +
> +static int msichip_domain_alloc(struct irq_domain *domain, unsigned int virq,
> +				   unsigned int nr_irqs, void *arg)
> +{
> +	int i, err;
> +	irq_hw_number_t hwirq;
> +	unsigned int type;
> +	struct irq_fwspec *fwspec = arg;
> +	void *data;
> +
> +	err = msichip_domain_translate(domain, fwspec, &hwirq, &type);
> +	if (err)
> +		return err;
> +

.translate function already called once in  irq_domain_translate(),
I think we don't need call this fucntion one more time here.

> +	err = platform_msi_domain_alloc(domain, virq, nr_irqs);
> +	if (err)
> +		return err;
> +
> +	data = platform_msi_get_host_data(domain);
> +	for (i = 0; i < nr_irqs; i++)
> +		irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
> +					      &msichip_chip, data);
> +
> +	return 0;
> +}
> +
[...]
> +
> +static struct platform_driver msichip_driver = {
> +	.driver = {
> +		.name		= "msichip",
> +		.of_match_table	= msichip_of_match,
> +	},
> +	.probe			= msichip_probe,
> +};
> +/* Do not define this as an irqchip */
> +module_platform_driver(msichip_driver);
> +
> +

I think,for a interrupt controller, msichip driver initialization maybe is too late
for some devices which connect to this irqchip if we use module_platform_driver.

So, how about use the arch_initcall to register the msichip driver?

Thanks!
Ma Jun

Re: [PATCH RFC 7/7] irqchip: [Example] dummy wired interrupt/MSI bridge driver

[Marc Zyngier]

On 04/11/15 08:00, majun (F) wrote:
> Hi Marc:
> 
> 在 2015/10/15 23:39, Marc Zyngier 写道:
>> In order to demonstrate how to put together a wire/MSI bridge,
>> add a dummy driver that doesn't do anything at all, except
>> for allocating interrupts.
>>
>> It comes together with an even more stupid client driver that
>> allocates an interrupt and dump the hierarchy of that interrupt.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
> [...]
>> +
>> +static int msichip_domain_alloc(struct irq_domain *domain, unsigned int virq,
>> +				   unsigned int nr_irqs, void *arg)
>> +{
>> +	int i, err;
>> +	irq_hw_number_t hwirq;
>> +	unsigned int type;
>> +	struct irq_fwspec *fwspec = arg;
>> +	void *data;
>> +
>> +	err = msichip_domain_translate(domain, fwspec, &hwirq, &type);
>> +	if (err)
>> +		return err;
>> +
> 
> .translate function already called once in  irq_domain_translate(),
> I think we don't need call this fucntion one more time here.

if you don't translate it here, how do you obtain the hwirq that you
have to pass to irq_domain_set_hwirq_and_chip just below?

>> +	err = platform_msi_domain_alloc(domain, virq, nr_irqs);
>> +	if (err)
>> +		return err;
>> +
>> +	data = platform_msi_get_host_data(domain);
>> +	for (i = 0; i < nr_irqs; i++)
>> +		irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
>> +					      &msichip_chip, data);
>> +
>> +	return 0;
>> +}
>> +
> [...]
>> +
>> +static struct platform_driver msichip_driver = {
>> +	.driver = {
>> +		.name		= "msichip",
>> +		.of_match_table	= msichip_of_match,
>> +	},
>> +	.probe			= msichip_probe,
>> +};
>> +/* Do not define this as an irqchip */
>> +module_platform_driver(msichip_driver);
>> +
>> +
> 
> I think,for a interrupt controller, msichip driver initialization maybe is too late
> for some devices which connect to this irqchip if we use module_platform_driver.

That's a consequence of this design. This is why I insisted on the fact
that this is currently avoided by using deferred probe in drivers, and
that it should be solved by having a probe order. Either way, this is
not something that we can solve at that level (see the multiple proposal
for this on the various lists).

> So, how about use the arch_initcall to register the msichip driver?

You're only pushing the problem one level up. And you'll realize that
this is not enough for some random driver. This is not sustainable, and
must be addressed properly.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH RFC 0/7] Adding core support for wire-MSI bridges

[Thomas Gleixner]

Marc,

On Fri, 16 Oct 2015, Marc Zyngier wrote:
> On 16/10/15 02:55, Jiang Liu wrote:
> > 	There's a working to enable Intel VMD storage device, which
> > has the similar requirement. Basically a PCIe hierarchy is hidden
> > behind a parent PCIe device, so we need to use the PCIe irqs on parent
> > to de-multiple PCIe IRQs from hidden PCIe devices. Seems a chance for
> > consolidation here.
> 
> Do you know if there is a 1-1 mapping between the interrupts seen by the
> parent device and those seen by the hidden devices? Or is it a case of
> having to demultiplex the MSIs? Looks like the former, but I'd like to
> be sure.

Yes, it's a demultiplexer. No 1:1 mapping.

> Sure, will do when I repost this (probably in a few weeks), and assuming
> this fits the bill for Thomas and the MBIGEN folks.

It doesn't look that bad and the resulting mbigen stuff is way less
horrible than it was before. So I agree this is a possible solution to
the problem.

Thanks,

	tglx

RE: [PATCH RFC 7/7] irqchip: [Example] dummy wired interrupt/MSI bridge driver

[Gabriele Paoloni]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="gb2312", Size: 3682 bytes --]

Hi Marc

> -----Original Message-----
> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-owner@vger.kernel.org]
> On Behalf Of Marc Zyngier
> Sent: 04 November 2015 09:04
> To: majun (F); Thomas Gleixner; Jiang Liu; Jason Cooper
> Cc: linux-arm-kernel@lists.infradead.org; linux-pci@vger.kernel.org; linux-
> kernel@vger.kernel.org
> Subject: Re: [PATCH RFC 7/7] irqchip: [Example] dummy wired interrupt/MSI
> bridge driver
> 
> On 04/11/15 08:00, majun (F) wrote:
> > Hi Marc:
> >
> > ÔÚ 2015/10/15 23:39, Marc Zyngier дµÀ:
> >> In order to demonstrate how to put together a wire/MSI bridge,
> >> add a dummy driver that doesn't do anything at all, except
> >> for allocating interrupts.
> >>
> >> It comes together with an even more stupid client driver that
> >> allocates an interrupt and dump the hierarchy of that interrupt.
> >>
> >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >> ---
> > [...]
> >> +
> >> +static int msichip_domain_alloc(struct irq_domain *domain, unsigned int
> virq,
> >> +				   unsigned int nr_irqs, void *arg)
> >> +{
> >> +	int i, err;
> >> +	irq_hw_number_t hwirq;
> >> +	unsigned int type;
> >> +	struct irq_fwspec *fwspec = arg;
> >> +	void *data;
> >> +
> >> +	err = msichip_domain_translate(domain, fwspec, &hwirq, &type);
> >> +	if (err)
> >> +		return err;
> >> +
> >
> > .translate function already called once in  irq_domain_translate(),
> > I think we don't need call this fucntion one more time here.
> 
> if you don't translate it here, how do you obtain the hwirq that you
> have to pass to irq_domain_set_hwirq_and_chip just below?
> 
> >> +	err = platform_msi_domain_alloc(domain, virq, nr_irqs);
> >> +	if (err)
> >> +		return err;
> >> +
> >> +	data = platform_msi_get_host_data(domain);
> >> +	for (i = 0; i < nr_irqs; i++)
> >> +		irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
> >> +					      &msichip_chip, data);
> >> +
> >> +	return 0;
> >> +}
> >> +
> > [...]
> >> +
> >> +static struct platform_driver msichip_driver = {
> >> +	.driver = {
> >> +		.name		= "msichip",
> >> +		.of_match_table	= msichip_of_match,
> >> +	},
> >> +	.probe			= msichip_probe,
> >> +};
> >> +/* Do not define this as an irqchip */
> >> +module_platform_driver(msichip_driver);
> >> +
> >> +
> >
> > I think,for a interrupt controller, msichip driver initialization maybe is
> too late
> > for some devices which connect to this irqchip if we use
> module_platform_driver.
> 
> That's a consequence of this design. This is why I insisted on the fact
> that this is currently avoided by using deferred probe in drivers, and

Mmm using te deferred probe would mean to rework all the drivers of the
potential devices connected to mbi-gen...would that be sustainable/acceptable?

> that it should be solved by having a probe order. Either way, this is
> not something that we can solve at that level (see the multiple proposal
> for this on the various lists).

Could you point me to the relevant discussions for this...?

Thanks

Gab

> 
> > So, how about use the arch_initcall to register the msichip driver?
> 
> You're only pushing the problem one level up. And you'll realize that
> this is not enough for some random driver. This is not sustainable, and
> must be addressed properly.
> 
> Thanks,
> 
> 	M.
> --
> Jazz is not dead. It just smells funny...
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [PATCH RFC 7/7] irqchip: [Example] dummy wired interrupt/MSI bridge driver

[Marc Zyngier]

On 05/11/15 08:25, Gabriele Paoloni wrote:
> Hi Marc
> 
>> -----Original Message-----
>> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-owner@vger.kernel.org]
>> On Behalf Of Marc Zyngier
>> Sent: 04 November 2015 09:04
>> To: majun (F); Thomas Gleixner; Jiang Liu; Jason Cooper
>> Cc: linux-arm-kernel@lists.infradead.org; linux-pci@vger.kernel.org; linux-
>> kernel@vger.kernel.org
>> Subject: Re: [PATCH RFC 7/7] irqchip: [Example] dummy wired interrupt/MSI
>> bridge driver
>>
>> On 04/11/15 08:00, majun (F) wrote:

[...]

>>> I think,for a interrupt controller, msichip driver initialization maybe is
>> too late
>>> for some devices which connect to this irqchip if we use
>> module_platform_driver.
>>
>> That's a consequence of this design. This is why I insisted on the fact
>> that this is currently avoided by using deferred probe in drivers, and
> 
> Mmm using te deferred probe would mean to rework all the drivers of the
> potential devices connected to mbi-gen...would that be sustainable/acceptable?

I'm tempted to reply "Not my problem". Or rather, not a problem I'm
trying to solve right now (or any time soon).

I'm pretty sure that sprinkling -EPROBE_DEFER on all possible drivers
will result in a resounding NAK, which is is why I suggested that
someone with a vested interest dedicates some quality time helping those
who are trying to solve this issue for good.

>> that it should be solved by having a probe order. Either way, this is
>> not something that we can solve at that level (see the multiple proposal
>> for this on the various lists).
> 
> Could you point me to the relevant discussions for this...?

Google is, as always, your dearest friend. But here you go:

- LWN has some quality coverage of the KS discussions (assuming you're a
subscriber, otherwise you'll have to wait for another week):
http://lwn.net/Articles/662820/

- There is also Tomeu Vizoso's series, which itself builds upon other
previous attempts at solving this: https://lwn.net/Articles/658690/

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

RE: [PATCH RFC 7/7] irqchip: [Example] dummy wired interrupt/MSI bridge driver

[Gabriele Paoloni]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="gb2312", Size: 2891 bytes --]

> -----Original Message-----
> From: Marc Zyngier [mailto:marc.zyngier@arm.com]
> Sent: Thursday, November 05, 2015 9:36 AM
> To: Gabriele Paoloni; majun (F); Thomas Gleixner; Jiang Liu; Jason
> Cooper
> Cc: linux-arm-kernel@lists.infradead.org; linux-pci@vger.kernel.org;
> linux-kernel@vger.kernel.org
> Subject: Re: [PATCH RFC 7/7] irqchip: [Example] dummy wired
> interrupt/MSI bridge driver
> 
> On 05/11/15 08:25, Gabriele Paoloni wrote:
> > Hi Marc
> >
> >> -----Original Message-----
> >> From: linux-pci-owner@vger.kernel.org [mailto:linux-pci-
> owner@vger.kernel.org]
> >> On Behalf Of Marc Zyngier
> >> Sent: 04 November 2015 09:04
> >> To: majun (F); Thomas Gleixner; Jiang Liu; Jason Cooper
> >> Cc: linux-arm-kernel@lists.infradead.org; linux-pci@vger.kernel.org;
> linux-
> >> kernel@vger.kernel.org
> >> Subject: Re: [PATCH RFC 7/7] irqchip: [Example] dummy wired
> interrupt/MSI
> >> bridge driver
> >>
> >> On 04/11/15 08:00, majun (F) wrote:
> 
> [...]
> 
> >>> I think,for a interrupt controller, msichip driver initialization
> maybe is
> >> too late
> >>> for some devices which connect to this irqchip if we use
> >> module_platform_driver.
> >>
> >> That's a consequence of this design. This is why I insisted on the
> fact
> >> that this is currently avoided by using deferred probe in drivers,
> and
> >
> > Mmm using te deferred probe would mean to rework all the drivers of
> the
> > potential devices connected to mbi-gen...would that be
> sustainable/acceptable?
> 
> I'm tempted to reply "Not my problem". Or rather, not a problem I'm
> trying to solve right now (or any time soon).
> 
> I'm pretty sure that sprinkling -EPROBE_DEFER on all possible drivers
> will result in a resounding NAK, which is is why I suggested that
> someone with a vested interest dedicates some quality time helping
> those
> who are trying to solve this issue for good.

Yes you're right, makes perfect sense

> 
> >> that it should be solved by having a probe order. Either way, this
> is
> >> not something that we can solve at that level (see the multiple
> proposal
> >> for this on the various lists).
> >
> > Could you point me to the relevant discussions for this...?
> 
> Google is, as always, your dearest friend. But here you go:
> 
> - LWN has some quality coverage of the KS discussions (assuming you're
> a
> subscriber, otherwise you'll have to wait for another week):
> http://lwn.net/Articles/662820/
> 
> - There is also Tomeu Vizoso's series, which itself builds upon other
> previous attempts at solving this: https://lwn.net/Articles/658690/
> 

Great, many thanks for pointing them out.

I'll look into these.

Thanks again

Gab

> Thanks,
> 
> 	M.
> --
> Jazz is not dead. It just smells funny...
ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [PATCH RFC 0/7] Adding core support for wire-MSI bridges

[Marc Zyngier]

On 04/11/15 13:34, Thomas Gleixner wrote:
> Marc,
> 
> On Fri, 16 Oct 2015, Marc Zyngier wrote:
>> On 16/10/15 02:55, Jiang Liu wrote:
>>> 	There's a working to enable Intel VMD storage device, which
>>> has the similar requirement. Basically a PCIe hierarchy is hidden
>>> behind a parent PCIe device, so we need to use the PCIe irqs on parent
>>> to de-multiple PCIe IRQs from hidden PCIe devices. Seems a chance for
>>> consolidation here.
>>
>> Do you know if there is a 1-1 mapping between the interrupts seen by the
>> parent device and those seen by the hidden devices? Or is it a case of
>> having to demultiplex the MSIs? Looks like the former, but I'd like to
>> be sure.
> 
> Yes, it's a demultiplexer. No 1:1 mapping.

Right. This doesn't exactly fit the scheme I have so far (there is a 1:1
mapping between the wired interrupt and the MSI), but once we are able
to expose an MSI domain, it could be possible to construct the MSI
demultiplexer on top. That's a lot of layers! ;-)

>> Sure, will do when I repost this (probably in a few weeks), and assuming
>> this fits the bill for Thomas and the MBIGEN folks.
> 
> It doesn't look that bad and the resulting mbigen stuff is way less
> horrible than it was before. So I agree this is a possible solution to
> the problem.

Cool. I'll revisit that after the merge window.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH RFC 0/7] Adding core support for wire-MSI bridges

[Thomas Gleixner]

On Thu, 5 Nov 2015, Marc Zyngier wrote:
> On 04/11/15 13:34, Thomas Gleixner wrote:
> > Marc,
> > 
> > On Fri, 16 Oct 2015, Marc Zyngier wrote:
> >> On 16/10/15 02:55, Jiang Liu wrote:
> >>> 	There's a working to enable Intel VMD storage device, which
> >>> has the similar requirement. Basically a PCIe hierarchy is hidden
> >>> behind a parent PCIe device, so we need to use the PCIe irqs on parent
> >>> to de-multiple PCIe IRQs from hidden PCIe devices. Seems a chance for
> >>> consolidation here.
> >>
> >> Do you know if there is a 1-1 mapping between the interrupts seen by the
> >> parent device and those seen by the hidden devices? Or is it a case of
> >> having to demultiplex the MSIs? Looks like the former, but I'd like to
> >> be sure.
> > 
> > Yes, it's a demultiplexer. No 1:1 mapping.
> 
> Right. This doesn't exactly fit the scheme I have so far (there is a 1:1
> mapping between the wired interrupt and the MSI), but once we are able
> to expose an MSI domain, it could be possible to construct the MSI
> demultiplexer on top. That's a lot of layers! ;-)

Well for the demux case it doesn't make a lot of sense. It's not easy
to describe in a hierarchy. Having that parentless MSI domain for that
VMD case is simple enough.

Thanks,

	tglx