[patch 00/16] genirq: Hierarchical irq domains and generic MSI interrupt code

[patch 00/16] genirq: Hierarchical irq domains and generic MSI interrupt code

[Thomas Gleixner]

This is an extract from Jiangs various patch series which only
contains the generic irq and MSI parts w/o the x86 specific
modifications.

This is roughly what I plan to merge into the generic irq core, so the
various outstanding patches (irqchip/x86/...) can be based on this.

It's available from git as well:

    git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain

Note, that branch is not yet exposed to linux-next and subject to
modifications including rebasing.

Can all involved parties please have a close look and retest their
patches on top of this?

Jiang, you need to merge that into x86/apic for rebasing your series.

Thanks,

	tglx
----
 Documentation/IRQ-domain.txt           |   71 +++++
 arch/arm/mach-iop13xx/msi.c            |    2 
 arch/ia64/kernel/msi_ia64.c            |    4 
 arch/ia64/sn/kernel/msi_sn.c           |    4 
 arch/mips/pci/msi-octeon.c             |    2 
 arch/mips/pci/msi-xlp.c                |    2 
 arch/mips/pci/pci-xlr.c                |    2 
 arch/powerpc/platforms/cell/axon_msi.c |    2 
 arch/powerpc/platforms/powernv/pci.c   |    2 
 arch/powerpc/platforms/pseries/msi.c   |    2 
 arch/powerpc/sysdev/fsl_msi.c          |    2 
 arch/powerpc/sysdev/mpic_pasemi_msi.c  |    2 
 arch/powerpc/sysdev/mpic_u3msi.c       |    2 
 arch/powerpc/sysdev/ppc4xx_hsta_msi.c  |    2 
 arch/powerpc/sysdev/ppc4xx_msi.c       |    2 
 arch/s390/pci/pci.c                    |    2 
 arch/sparc/kernel/pci_msi.c            |    2 
 arch/tile/kernel/pci_gx.c              |    2 
 arch/x86/kernel/apic/io_apic.c         |    4 
 arch/x86/pci/xen.c                     |    4 
 drivers/iommu/irq_remapping.c          |    8 
 drivers/irqchip/irq-armada-370-xp.c    |    2 
 drivers/pci/Kconfig                    |    6 
 drivers/pci/host/pci-tegra.c           |    2 
 drivers/pci/host/pcie-designware.c     |    2 
 drivers/pci/host/pcie-rcar.c           |    2 
 drivers/pci/host/pcie-xilinx.c         |    2 
 drivers/pci/msi.c                      |  150 ++++++-----
 drivers/vfio/pci/vfio_pci_intrs.c      |    2 
 include/linux/irq.h                    |   33 ++
 include/linux/irqdomain.h              |   91 +++++++
 include/linux/irqhandler.h             |   14 +
 include/linux/msi.h                    |   53 +++-
 kernel/irq/Kconfig                     |   15 +
 kernel/irq/Makefile                    |    1 
 kernel/irq/chip.c                      |   37 ++
 kernel/irq/irqdomain.c                 |  418 +++++++++++++++++++++++++++++++--
 kernel/irq/manage.c                    |    2 
 kernel/irq/msi.c                       |  132 ++++++++++
 39 files changed, 959 insertions(+), 130 deletions(-)

[patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Thomas Gleixner]

[-- Attachment #1: irqdomain-introduce-new-interfaces-to-support-hierarchy-irqdomains.patch --]
[-- Type: text/plain, Size: 27391 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

We plan to use hierarchy irqdomain to suppport CPU vector assignment,
interrupt remapping controller, IO-APIC controller, MSI interrupt
and hypertransport interrupt etc on x86 platforms. So extend irqdomain
interfaces to support hierarchy irqdomain.

There are already many clients of current irqdomain interfaces.
To minimize the changes, we choose to introduce new version 2 interfaces
to support hierarchy instead of extending existing irqdomain interfaces.

According to Thomas's suggestion, the most important design decision is
to build hierarchy struct irq_data to support hierarchy irqdomain, so
hierarchy irqdomain related data could be saved in struct irq_data.
With support of hierarchy irq_data, we could also support stacked
irq_chips. This is most useful in case of set_affinity().

The new hierarchy irqdomain introduces following interfaces:
1) irq_domain_alloc_irqs()/irq_domain_free_irqs(): allocate/release IRQ
   and related resources.
2) __irq_domain_alloc_irqs(): a special version to support legacy IRQs.
3) irq_domain_activate_irq()/irq_domain_deactivate_irq(): program
   interrupt controllers to activate/deactivate interrupt.

There are also several help functions to ease irqdomain implemenations:
1) irq_domain_get_irq_data(): get irq_data associated with a specific
   irqdomain.
2) irq_domain_set_hwirq_and_chip(): save irqdomain specific data into
   irq_data.
3) irq_domain_alloc_irqs_parent()/irq_domain_free_irqs_parent(): invoke
   parent irqdomain's alloc/free callbacks.

We also changed irq_startup()/irq_shutdown() to invoke
irq_domain_activate_irq()/irq_domain_deactivate_irq() to program
interrupt controller when start/stop interrupts.

[ tglx: Folded parts of the later patch series in ]

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 Documentation/IRQ-domain.txt |   71 +++++++
 include/linux/irq.h          |    5 
 include/linux/irqdomain.h    |   86 +++++++++
 kernel/irq/Kconfig           |    5 
 kernel/irq/chip.c            |    3 
 kernel/irq/irqdomain.c       |  391 +++++++++++++++++++++++++++++++++++++++++--
 6 files changed, 545 insertions(+), 16 deletions(-)

Index: tip/Documentation/IRQ-domain.txt
===================================================================
--- tip.orig/Documentation/IRQ-domain.txt
+++ tip/Documentation/IRQ-domain.txt
@@ -151,3 +151,74 @@ used and no descriptor gets allocated it
 that the driver using the simple domain call irq_create_mapping()
 before any irq_find_mapping() since the latter will actually work
 for the static IRQ assignment case.
+
+==== Hierarchy IRQ domain ====
+On some architectures, there may be multiple interrupt controllers
+involved in delivering an interrupt from the device to the target CPU.
+Let's look at a typical interrupt delivering path on x86 platforms:
+
+Device --> IOAPIC -> Interrupt remapping Controller -> Local APIC -> CPU
+
+There are three interrupt controllers involved:
+1) IOAPIC controller
+2) Interrupt remapping controller
+3) Local APIC controller
+
+To support such a hardware topology and make software architecture match
+hardware architecture, an irq_domain data structure is built for each
+interrupt controller and those irq_domains are organized into hierarchy.
+When building irq_domain hierarchy, the irq_domain near to the device is
+child and the irq_domain near to CPU is parent. So a hierarchy structure
+as below will be built for the example above.
+	CPU Vector irq_domain (root irq_domain to manage CPU vectors)
+		^
+		|
+	Interrupt Remapping irq_domain (manage irq_remapping entries)
+		^
+		|
+	IOAPIC irq_domain (manage IOAPIC delivery entries/pins)
+
+There are four major interfaces to use hierarchy irq_domain:
+1) irq_domain_alloc_irqs(): allocate IRQ descriptors and interrupt
+   controller related resources to deliver these interrupts.
+2) irq_domain_free_irqs(): free IRQ descriptors and interrupt controller
+   related resources associated with these interrupts.
+3) irq_domain_activate_irq(): activate interrupt controller hardware to
+   deliver the interrupt.
+3) irq_domain_deactivate_irq(): deactivate interrupt controller hardware
+   to stop delivering the interrupt.
+
+Following changes are needed to support hierarchy irq_domain.
+1) a new field 'parent' is added to struct irq_domain; it's used to
+   maintain irq_domain hierarchy information.
+2) a new field 'parent_data' is added to struct irq_data; it's used to
+   build hierarchy irq_data to match hierarchy irq_domains. The irq_data
+   is used to store irq_domain pointer and hardware irq number.
+3) new callbacks are added to struct irq_domain_ops to support hierarchy
+   irq_domain operations.
+
+With support of hierarchy irq_domain and hierarchy irq_data ready, an
+irq_domain structure is built for each interrupt controller, and an
+irq_data structure is allocated for each irq_domain associated with an
+IRQ. Now we could go one step further to support stacked(hierarchy)
+irq_chip. That is, an irq_chip is associated with each irq_data along
+the hierarchy. A child irq_chip may implement a required action by
+itself or by cooperating with its parent irq_chip.
+
+With stacked irq_chip, interrupt controller driver only needs to deal
+with the hardware managed by itself and may ask for services from its
+parent irq_chip when needed. So we could achieve a much cleaner
+software architecture.
+
+For an interrupt controller driver to support hierarchy irq_domain, it
+needs to:
+1) Implement irq_domain_ops.alloc and irq_domain_ops.free
+2) Optionally implement irq_domain_ops.activate and
+   irq_domain_ops.deactivate.
+3) Optionally implement an irq_chip to manage the interrupt controller
+   hardware.
+4) No need to implement irq_domain_ops.map and irq_domain_ops.unmap,
+   they are unused with hierarchy irq_domain.
+
+Hierarchy irq_domain may also be used to support other architectures,
+such as ARM, ARM64 etc.
Index: tip/include/linux/irq.h
===================================================================
--- tip.orig/include/linux/irq.h
+++ tip/include/linux/irq.h
@@ -133,6 +133,8 @@ struct irq_domain;
  * @chip:		low level interrupt hardware access
  * @domain:		Interrupt translation domain; responsible for mapping
  *			between hwirq number and linux irq number.
+ * @parent_data:	pointer to parent struct irq_data to support hierarchy
+ *			irq_domain
  * @handler_data:	per-IRQ data for the irq_chip methods
  * @chip_data:		platform-specific per-chip private data for the chip
  *			methods, to allow shared chip implementations
@@ -151,6 +153,9 @@ struct irq_data {
 	unsigned int		state_use_accessors;
 	struct irq_chip		*chip;
 	struct irq_domain	*domain;
+#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
+	struct irq_data		*parent_data;
+#endif
 	void			*handler_data;
 	void			*chip_data;
 	struct msi_desc		*msi_desc;
Index: tip/include/linux/irqdomain.h
===================================================================
--- tip.orig/include/linux/irqdomain.h
+++ tip/include/linux/irqdomain.h
@@ -38,6 +38,8 @@
 struct device_node;
 struct irq_domain;
 struct of_device_id;
+struct irq_chip;
+struct irq_data;
 
 /* Number of irqs reserved for a legacy isa controller */
 #define NUM_ISA_INTERRUPTS	16
@@ -64,6 +66,16 @@ struct irq_domain_ops {
 	int (*xlate)(struct irq_domain *d, struct device_node *node,
 		     const u32 *intspec, unsigned int intsize,
 		     unsigned long *out_hwirq, unsigned int *out_type);
+
+#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
+	/* extended V2 interfaces to support hierarchy irq_domains */
+	int (*alloc)(struct irq_domain *d, unsigned int virq,
+		     unsigned int nr_irqs, void *arg);
+	void (*free)(struct irq_domain *d, unsigned int virq,
+		     unsigned int nr_irqs);
+	void (*activate)(struct irq_domain *d, struct irq_data *irq_data);
+	void (*deactivate)(struct irq_domain *d, struct irq_data *irq_data);
+#endif
 };
 
 extern struct irq_domain_ops irq_generic_chip_ops;
@@ -77,6 +89,7 @@ struct irq_domain_chip_generic;
  * @ops: pointer to irq_domain methods
  * @host_data: private data pointer for use by owner.  Not touched by irq_domain
  *             core code.
+ * @flags: host per irq_domain flags
  *
  * Optional elements
  * @of_node: Pointer to device tree nodes associated with the irq_domain. Used
@@ -84,6 +97,7 @@ struct irq_domain_chip_generic;
  * @gc: Pointer to a list of generic chips. There is a helper function for
  *      setting up one or more generic chips for interrupt controllers
  *      drivers using the generic chip library which uses this pointer.
+ * @parent: Pointer to parent irq_domain to support hierarchy irq_domains
  *
  * Revmap data, used internally by irq_domain
  * @revmap_direct_max_irq: The largest hwirq that can be set for controllers that
@@ -97,10 +111,14 @@ struct irq_domain {
 	const char *name;
 	const struct irq_domain_ops *ops;
 	void *host_data;
+	unsigned int flags;
 
 	/* Optional data */
 	struct device_node *of_node;
 	struct irq_domain_chip_generic *gc;
+#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
+	struct irq_domain *parent;
+#endif
 
 	/* reverse map data. The linear map gets appended to the irq_domain */
 	irq_hw_number_t hwirq_max;
@@ -110,6 +128,9 @@ struct irq_domain {
 	unsigned int linear_revmap[];
 };
 
+#define	IRQ_DOMAIN_FLAG_HIERARCHY	0x1
+#define	IRQ_DOMAIN_FLAG_ARCH1		0x10000
+
 #ifdef CONFIG_IRQ_DOMAIN
 struct irq_domain *__irq_domain_add(struct device_node *of_node, int size,
 				    irq_hw_number_t hwirq_max, int direct_max,
@@ -220,8 +241,73 @@ int irq_domain_xlate_onetwocell(struct i
 			const u32 *intspec, unsigned int intsize,
 			irq_hw_number_t *out_hwirq, unsigned int *out_type);
 
+/* V2 interfaces to support hierarchy IRQ domains. */
+extern struct irq_data *irq_domain_get_irq_data(struct irq_domain *domain,
+						unsigned int virq);
+#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
+extern int __irq_domain_alloc_irqs(struct irq_domain *domain, int irq_base,
+				   unsigned int nr_irqs, int node, void *arg,
+				   bool realloc);
+extern void irq_domain_free_irqs(unsigned int virq, unsigned int nr_irqs);
+extern void irq_domain_activate_irq(struct irq_data *irq_data);
+extern void irq_domain_deactivate_irq(struct irq_data *irq_data);
+
+static inline int irq_domain_alloc_irqs(struct irq_domain *domain,
+			unsigned int nr_irqs, int node, void *arg)
+{
+	return __irq_domain_alloc_irqs(domain, -1, nr_irqs, node, arg, false);
+}
+
+extern int irq_domain_set_hwirq_and_chip(struct irq_domain *domain,
+					 unsigned int virq,
+					 irq_hw_number_t hwirq,
+					 struct irq_chip *chip,
+					 void *chip_data);
+extern void irq_domain_reset_irq_data(struct irq_data *irq_data);
+extern void irq_domain_free_irqs_common(struct irq_domain *domain,
+					int virq, int nr_irqs);
+extern void irq_domain_free_irqs_top(struct irq_domain *domain,
+				     int virq, int nr_irqs);
+
+static inline int irq_domain_alloc_irqs_parent(struct irq_domain *domain,
+				int irq_base, unsigned int nr_irqs, void *arg)
+{
+	if (domain->parent && domain->parent->ops->alloc)
+		return domain->parent->ops->alloc(domain->parent, irq_base,
+						  nr_irqs, arg);
+	return -ENOSYS;
+}
+
+static inline void irq_domain_free_irqs_parent(struct irq_domain *domain,
+					int irq_base, unsigned int nr_irqs)
+{
+	if (domain->parent && domain->parent->ops->free)
+		domain->parent->ops->free(domain->parent, irq_base, nr_irqs);
+}
+
+static inline bool irq_domain_is_hierarchy(struct irq_domain *domain)
+{
+	return domain->flags & IRQ_DOMAIN_FLAG_HIERARCHY;
+}
+#else	/* CONFIG_IRQ_DOMAIN_HIERARCHY */
+static inline void irq_domain_activate_irq(struct irq_data *data) { }
+static inline void irq_domain_deactivate_irq(struct irq_data *data) { }
+static inline int irq_domain_alloc_irqs(struct irq_domain *domain,
+			unsigned int nr_irqs, int node, void *arg)
+{
+	return -1;
+}
+
+static inline bool irq_domain_is_hierarchy(struct irq_domain *domain)
+{
+	return false;
+}
+#endif	/* CONFIG_IRQ_DOMAIN_HIERARCHY */
+
 #else /* CONFIG_IRQ_DOMAIN */
 static inline void irq_dispose_mapping(unsigned int virq) { }
+static inline void irq_domain_activate_irq(struct irq_data *data) { }
+static inline void irq_domain_deactivate_irq(struct irq_data *data) { }
 #endif /* !CONFIG_IRQ_DOMAIN */
 
 #endif /* _LINUX_IRQDOMAIN_H */
Index: tip/kernel/irq/Kconfig
===================================================================
--- tip.orig/kernel/irq/Kconfig
+++ tip/kernel/irq/Kconfig
@@ -55,6 +55,11 @@ config GENERIC_IRQ_CHIP
 config IRQ_DOMAIN
 	bool
 
+# Support for hierarchical irq domains
+config IRQ_DOMAIN_HIERARCHY
+	bool
+	select IRQ_DOMAIN
+
 config HANDLE_DOMAIN_IRQ
 	bool
 
Index: tip/kernel/irq/chip.c
===================================================================
--- tip.orig/kernel/irq/chip.c
+++ tip/kernel/irq/chip.c
@@ -15,6 +15,7 @@
 #include <linux/module.h>
 #include <linux/interrupt.h>
 #include <linux/kernel_stat.h>
+#include <linux/irqdomain.h>
 
 #include <trace/events/irq.h>
 
@@ -178,6 +179,7 @@ int irq_startup(struct irq_desc *desc, b
 	irq_state_clr_disabled(desc);
 	desc->depth = 0;
 
+	irq_domain_activate_irq(&desc->irq_data);
 	if (desc->irq_data.chip->irq_startup) {
 		ret = desc->irq_data.chip->irq_startup(&desc->irq_data);
 		irq_state_clr_masked(desc);
@@ -199,6 +201,7 @@ void irq_shutdown(struct irq_desc *desc)
 		desc->irq_data.chip->irq_disable(&desc->irq_data);
 	else
 		desc->irq_data.chip->irq_mask(&desc->irq_data);
+	irq_domain_deactivate_irq(&desc->irq_data);
 	irq_state_set_masked(desc);
 }
 
Index: tip/kernel/irq/irqdomain.c
===================================================================
--- tip.orig/kernel/irq/irqdomain.c
+++ tip/kernel/irq/irqdomain.c
@@ -23,6 +23,10 @@ static DEFINE_MUTEX(irq_domain_mutex);
 static DEFINE_MUTEX(revmap_trees_mutex);
 static struct irq_domain *irq_default_domain;
 
+static int irq_domain_alloc_descs(int virq, unsigned int nr_irqs,
+				  irq_hw_number_t hwirq, int node);
+static void irq_domain_check_hierarchy(struct irq_domain *domain);
+
 /**
  * __irq_domain_add() - Allocate a new irq_domain data structure
  * @of_node: optional device-tree node of the interrupt controller
@@ -30,7 +34,7 @@ static struct irq_domain *irq_default_do
  * @hwirq_max: Maximum number of interrupts supported by controller
  * @direct_max: Maximum value of direct maps; Use ~0 for no limit; 0 for no
  *              direct mapping
- * @ops: map/unmap domain callbacks
+ * @ops: domain callbacks
  * @host_data: Controller private data pointer
  *
  * Allocates and initialize and irq_domain structure.
@@ -56,6 +60,7 @@ struct irq_domain *__irq_domain_add(stru
 	domain->hwirq_max = hwirq_max;
 	domain->revmap_size = size;
 	domain->revmap_direct_max_irq = direct_max;
+	irq_domain_check_hierarchy(domain);
 
 	mutex_lock(&irq_domain_mutex);
 	list_add(&domain->link, &irq_domain_list);
@@ -109,7 +114,7 @@ EXPORT_SYMBOL_GPL(irq_domain_remove);
  * @first_irq: first number of irq block assigned to the domain,
  *	pass zero to assign irqs on-the-fly. If first_irq is non-zero, then
  *	pre-map all of the irqs in the domain to virqs starting at first_irq.
- * @ops: map/unmap domain callbacks
+ * @ops: domain callbacks
  * @host_data: Controller private data pointer
  *
  * Allocates an irq_domain, and optionally if first_irq is positive then also
@@ -174,10 +179,8 @@ struct irq_domain *irq_domain_add_legacy
 
 	domain = __irq_domain_add(of_node, first_hwirq + size,
 				  first_hwirq + size, 0, ops, host_data);
-	if (!domain)
-		return NULL;
-
-	irq_domain_associate_many(domain, first_irq, first_hwirq, size);
+	if (domain)
+		irq_domain_associate_many(domain, first_irq, first_hwirq, size);
 
 	return domain;
 }
@@ -388,7 +391,6 @@ EXPORT_SYMBOL_GPL(irq_create_direct_mapp
 unsigned int irq_create_mapping(struct irq_domain *domain,
 				irq_hw_number_t hwirq)
 {
-	unsigned int hint;
 	int virq;
 
 	pr_debug("irq_create_mapping(0x%p, 0x%lx)\n", domain, hwirq);
@@ -410,12 +412,8 @@ unsigned int irq_create_mapping(struct i
 	}
 
 	/* Allocate a virtual interrupt number */
-	hint = hwirq % nr_irqs;
-	if (hint == 0)
-		hint++;
-	virq = irq_alloc_desc_from(hint, of_node_to_nid(domain->of_node));
-	if (virq <= 0)
-		virq = irq_alloc_desc_from(1, of_node_to_nid(domain->of_node));
+	virq = irq_domain_alloc_descs(-1, 1, hwirq,
+				      of_node_to_nid(domain->of_node));
 	if (virq <= 0) {
 		pr_debug("-> virq allocation failed\n");
 		return 0;
@@ -471,7 +469,7 @@ unsigned int irq_create_of_mapping(struc
 	struct irq_domain *domain;
 	irq_hw_number_t hwirq;
 	unsigned int type = IRQ_TYPE_NONE;
-	unsigned int virq;
+	int virq;
 
 	domain = irq_data->np ? irq_find_host(irq_data->np) : irq_default_domain;
 	if (!domain) {
@@ -480,6 +478,11 @@ unsigned int irq_create_of_mapping(struc
 		return 0;
 	}
 
+	if (irq_domain_is_hierarchy(domain)) {
+		virq = irq_domain_alloc_irqs(domain, 1, NUMA_NO_NODE, irq_data);
+		return virq <= 0 ? 0 : virq;
+	}
+
 	/* If domain has no translation, then we assume interrupt line */
 	if (domain->ops->xlate == NULL)
 		hwirq = irq_data->args[0];
@@ -540,8 +543,8 @@ unsigned int irq_find_mapping(struct irq
 		return 0;
 
 	if (hwirq < domain->revmap_direct_max_irq) {
-		data = irq_get_irq_data(hwirq);
-		if (data && (data->domain == domain) && (data->hwirq == hwirq))
+		data = irq_domain_get_irq_data(domain, hwirq);
+		if (data && data->hwirq == hwirq)
 			return hwirq;
 	}
 
@@ -709,3 +712,359 @@ const struct irq_domain_ops irq_domain_s
 	.xlate = irq_domain_xlate_onetwocell,
 };
 EXPORT_SYMBOL_GPL(irq_domain_simple_ops);
+
+static int irq_domain_alloc_descs(int virq, unsigned int cnt,
+				  irq_hw_number_t hwirq, int node)
+{
+	unsigned int hint;
+
+	if (virq >= 0) {
+		virq = irq_alloc_descs(virq, virq, cnt, node);
+	} else {
+		hint = hwirq % nr_irqs;
+		if (hint == 0)
+			hint++;
+		virq = irq_alloc_descs_from(hint, cnt, node);
+		if (virq <= 0 && hint > 1)
+			virq = irq_alloc_descs_from(1, cnt, node);
+	}
+
+	return virq;
+}
+
+#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
+static void irq_domain_insert_irq(int virq)
+{
+	struct irq_data *data;
+
+	for (data = irq_get_irq_data(virq); data; data = data->parent_data) {
+		struct irq_domain *domain = data->domain;
+		irq_hw_number_t hwirq = data->hwirq;
+
+		if (hwirq < domain->revmap_size) {
+			domain->linear_revmap[hwirq] = virq;
+		} else {
+			mutex_lock(&revmap_trees_mutex);
+			radix_tree_insert(&domain->revmap_tree, hwirq, data);
+			mutex_unlock(&revmap_trees_mutex);
+		}
+
+		/* If not already assigned, give the domain the chip's name */
+		if (!domain->name && data->chip)
+			domain->name = data->chip->name;
+	}
+
+	irq_clear_status_flags(virq, IRQ_NOREQUEST);
+}
+
+static void irq_domain_remove_irq(int virq)
+{
+	struct irq_data *data;
+
+	irq_set_status_flags(virq, IRQ_NOREQUEST);
+	irq_set_chip_and_handler(virq, NULL, NULL);
+	synchronize_irq(virq);
+	smp_mb();
+
+	for (data = irq_get_irq_data(virq); data; data = data->parent_data) {
+		struct irq_domain *domain = data->domain;
+		irq_hw_number_t hwirq = data->hwirq;
+
+		if (hwirq < domain->revmap_size) {
+			domain->linear_revmap[hwirq] = 0;
+		} else {
+			mutex_lock(&revmap_trees_mutex);
+			radix_tree_delete(&domain->revmap_tree, hwirq);
+			mutex_unlock(&revmap_trees_mutex);
+		}
+	}
+}
+
+static struct irq_data *irq_domain_insert_irq_data(struct irq_domain *domain,
+						   struct irq_data *child)
+{
+	struct irq_data *irq_data;
+
+	irq_data = kzalloc_node(sizeof(*irq_data), GFP_KERNEL, child->node);
+	if (irq_data) {
+		child->parent_data = irq_data;
+		irq_data->irq = child->irq;
+		irq_data->node = child->node;
+		irq_data->domain = domain;
+	}
+
+	return irq_data;
+}
+
+static void irq_domain_free_irq_data(unsigned int virq, unsigned int nr_irqs)
+{
+	int i;
+	struct irq_data *irq_data, *tmp;
+
+	for (i = 0; i < nr_irqs; i++) {
+		irq_data = irq_get_irq_data(virq + i);
+		tmp = irq_data->parent_data;
+		irq_data->parent_data = NULL;
+		irq_data->domain = NULL;
+
+		while (tmp) {
+			irq_data = tmp;
+			tmp = tmp->parent_data;
+			kfree(irq_data);
+		}
+	}
+}
+
+static int irq_domain_alloc_irq_data(struct irq_domain *domain,
+				     unsigned int virq, unsigned int nr_irqs)
+{
+	int i;
+	struct irq_data *irq_data;
+	struct irq_domain *parent;
+
+	/* The outermost irq_data is embedded in struct irq_desc */
+	for (i = 0; i < nr_irqs; i++) {
+		irq_data = irq_get_irq_data(virq + i);
+		irq_data->domain = domain;
+
+		for (parent = domain->parent; parent; parent = parent->parent) {
+			irq_data = irq_domain_insert_irq_data(parent, irq_data);
+			if (!irq_data) {
+				irq_domain_free_irq_data(virq, i + 1);
+				return -ENOMEM;
+			}
+		}
+	}
+
+	return 0;
+}
+
+/**
+ * irq_domain_get_irq_data - Get irq_data associated with @virq and @domain
+ * @domain: domain to match
+ * @virq: IRQ number to get irq_data
+ */
+struct irq_data *irq_domain_get_irq_data(struct irq_domain *domain,
+					 unsigned int virq)
+{
+	struct irq_data *irq_data;
+
+	for (irq_data = irq_get_irq_data(virq); irq_data;
+	     irq_data = irq_data->parent_data)
+		if (irq_data->domain == domain)
+			return irq_data;
+
+	return NULL;
+}
+
+int irq_domain_set_hwirq_and_chip(struct irq_domain *domain, unsigned int virq,
+				  irq_hw_number_t hwirq, struct irq_chip *chip,
+				  void *chip_data)
+{
+	struct irq_data *irq_data = irq_domain_get_irq_data(domain, virq);
+
+	if (!irq_data)
+		return -ENOENT;
+
+	irq_data->hwirq = hwirq;
+	irq_data->chip = chip ? chip : &no_irq_chip;
+	irq_data->chip_data = chip_data;
+
+	return 0;
+}
+
+void irq_domain_reset_irq_data(struct irq_data *irq_data)
+{
+	irq_data->hwirq = 0;
+	irq_data->chip = &no_irq_chip;
+	irq_data->chip_data = NULL;
+}
+
+void irq_domain_free_irqs_common(struct irq_domain *domain, int virq,
+				 int nr_irqs)
+{
+	int i;
+	struct irq_data *irq_data;
+
+	for (i = 0; i < nr_irqs; i++) {
+		irq_data = irq_domain_get_irq_data(domain, virq + i);
+		if (irq_data)
+			irq_domain_reset_irq_data(irq_data);
+	}
+	irq_domain_free_irqs_parent(domain, virq, nr_irqs);
+}
+
+void irq_domain_free_irqs_top(struct irq_domain *domain, int virq,
+			      int nr_irqs)
+{
+	int i;
+
+	for (i = 0; i < nr_irqs; i++) {
+		irq_set_handler_data(virq + i, NULL);
+		irq_set_handler(virq + i, NULL);
+	}
+	irq_domain_free_irqs_common(domain, virq, nr_irqs);
+}
+
+/**
+ * __irq_domain_alloc_irqs - Allocate IRQs from domain
+ * @domain: domain to allocate from
+ * @irq_base: allocate specified IRQ nubmer if irq_base >= 0
+ * @nr_irqs: number of IRQs to allocate
+ * @node: NUMA node id for memory allocation
+ * @arg: domain specific argument
+ * @realloc: IRQ descriptors have already been allocated if true
+ *
+ * Allocate IRQ numbers and initialized all data structures to support
+ * hiearchy IRQ domains.
+ * Parameter @realloc is mainly to support legacy IRQs.
+ * Returns error code or allocated IRQ number
+ *
+ * The whole process to setup an IRQ has been split into two steps.
+ * The first step, __irq_domain_alloc_irqs(), is to allocate IRQ
+ * descriptor and required hardware resources. The second step,
+ * irq_domain_activate_irq(), is to program hardwares with preallocated
+ * resources. In this way, it's easier to rollback when failing to
+ * allocate resources.
+ */
+int __irq_domain_alloc_irqs(struct irq_domain *domain, int irq_base,
+			    unsigned int nr_irqs, int node, void *arg,
+			    bool realloc)
+{
+	int i, ret, virq;
+
+	if (domain == NULL) {
+		domain = irq_default_domain;
+		if (WARN(!domain, "domain is NULL; cannot allocate IRQ\n"))
+			return -EINVAL;
+	}
+
+	if (!domain->ops->alloc) {
+		pr_debug("domain->ops->alloc() is NULL\n");
+		return -ENOSYS;
+	}
+
+	if (realloc && irq_base >= 0) {
+		virq = irq_base;
+	} else {
+		virq = irq_domain_alloc_descs(irq_base, nr_irqs, 0, node);
+		if (virq < 0) {
+			pr_debug("cannot allocate IRQ(base %d, count %d)\n",
+				 irq_base, nr_irqs);
+			return virq;
+		}
+	}
+
+	if (irq_domain_alloc_irq_data(domain, virq, nr_irqs)) {
+		pr_debug("cannot allocate memory for IRQ%d\n", virq);
+		ret = -ENOMEM;
+		goto out_free_desc;
+	}
+
+	mutex_lock(&irq_domain_mutex);
+	ret = domain->ops->alloc(domain, virq, nr_irqs, arg);
+	if (ret < 0) {
+		mutex_unlock(&irq_domain_mutex);
+		goto out_free_irq_data;
+	}
+	for (i = 0; i < nr_irqs; i++)
+		irq_domain_insert_irq(virq + i);
+	mutex_unlock(&irq_domain_mutex);
+
+	return virq;
+
+out_free_irq_data:
+	irq_domain_free_irq_data(virq, nr_irqs);
+out_free_desc:
+	irq_free_descs(virq, nr_irqs);
+	return ret;
+}
+
+/**
+ * irq_domain_free_irqs - Free IRQ number and associated data structures
+ * @virq: base IRQ number
+ * @nr_irqs: number of IRQs to free
+ */
+void irq_domain_free_irqs(unsigned int virq, unsigned int nr_irqs)
+{
+	int i;
+	struct irq_data *data = irq_get_irq_data(virq);
+
+	if (WARN(!data || !data->domain || !data->domain->ops->free,
+		 "NULL pointer, cannot free irq\n"))
+		return;
+
+	mutex_lock(&irq_domain_mutex);
+	for (i = 0; i < nr_irqs; i++)
+		irq_domain_remove_irq(virq + i);
+	data->domain->ops->free(data->domain, virq, nr_irqs);
+	mutex_unlock(&irq_domain_mutex);
+
+	irq_domain_free_irq_data(virq, nr_irqs);
+	irq_free_descs(virq, nr_irqs);
+}
+
+/**
+ * irq_domain_activate_irq - Call domain_ops->activate recursively to activate
+ *			     interrupt
+ * @irq_data: outermost irq_data associated with interrupt
+ *
+ * This is the second step to call domain_ops->activate to program interrupt
+ * controllers, so the interrupt could actually get delivered.
+ */
+void irq_domain_activate_irq(struct irq_data *irq_data)
+{
+	if (irq_data && irq_data->domain) {
+		struct irq_domain *domain = irq_data->domain;
+
+		if (irq_data->parent_data)
+			irq_domain_activate_irq(irq_data->parent_data);
+		if (domain->ops->activate)
+			domain->ops->activate(domain, irq_data);
+	}
+}
+
+/**
+ * irq_domain_deactivate_irq - Call domain_ops->deactivate recursively to
+ *			       deactivate interrupt
+ * @irq_data: outermost irq_data associated with interrupt
+ *
+ * It calls domain_ops->deactivate to program interrupt controllers to disable
+ * interrupt delivery.
+ */
+void irq_domain_deactivate_irq(struct irq_data *irq_data)
+{
+	if (irq_data && irq_data->domain) {
+		struct irq_domain *domain = irq_data->domain;
+
+		if (domain->ops->deactivate)
+			domain->ops->deactivate(domain, irq_data);
+		if (irq_data->parent_data)
+			irq_domain_deactivate_irq(irq_data->parent_data);
+	}
+}
+
+static void irq_domain_check_hierarchy(struct irq_domain *domain)
+{
+	/* Hierarchy irq_domains must implement callback alloc() */
+	if (domain->ops->alloc)
+		domain->flags |= IRQ_DOMAIN_FLAG_HIERARCHY;
+}
+#else	/* CONFIG_IRQ_DOMAIN_HIERARCHY */
+/**
+ * irq_domain_get_irq_data - Get irq_data associated with @virq and @domain
+ * @domain: domain to match
+ * @virq: IRQ number to get irq_data
+ */
+struct irq_data *irq_domain_get_irq_data(struct irq_domain *domain,
+					 unsigned int virq)
+{
+	struct irq_data *irq_data = irq_get_irq_data(virq);
+
+	return (irq_data && irq_data->domain == domain) ? irq_data : NULL;
+}
+
+static void irq_domain_check_hierarchy(struct irq_domain *domain)
+{
+}
+#endif	/* CONFIG_IRQ_DOMAIN_HIERARCHY */

[patch 02/16] irqdomain: Do irq_find_mapping and set_type for hierarchy irqdomain in case OF

[Thomas Gleixner]

[-- Attachment #1: irqdomain-do-irq_find_mapping-and-set_type-for-hierarchy-irqdomain-in-case-of.patch --]
[-- Type: text/plain, Size: 2116 bytes --]

From: Yingjoe Chen <yingjoe.chen@mediatek.com>

It is possible to call irq_create_of_mapping to create/translate the
same IRQ from DT for multiple times. Perform irq_find_mapping check
and set_type for hierarchy irqdomain in irq_create_of_mapping() to
avoid duplicate these functionality in all outer most irqdomain.

Signed-off-by: Yingjoe Chen <yingjoe.chen@mediatek.com>
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/irq/irqdomain.c |   27 ++++++++++++++++++---------
 1 file changed, 18 insertions(+), 9 deletions(-)

diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
index 610891e63844..4811bee1b8a7 100644
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -478,11 +478,6 @@ unsigned int irq_create_of_mapping(struct of_phandle_args *irq_data)
 		return 0;
 	}
 
-	if (irq_domain_is_hierarchy(domain)) {
-		virq = irq_domain_alloc_irqs(domain, 1, NUMA_NO_NODE, irq_data);
-		return virq <= 0 ? 0 : virq;
-	}
-
 	/* If domain has no translation, then we assume interrupt line */
 	if (domain->ops->xlate == NULL)
 		hwirq = irq_data->args[0];
@@ -492,10 +487,24 @@ unsigned int irq_create_of_mapping(struct of_phandle_args *irq_data)
 			return 0;
 	}
 
-	/* Create mapping */
-	virq = irq_create_mapping(domain, hwirq);
-	if (!virq)
-		return virq;
+	if (irq_domain_is_hierarchy(domain)) {
+		/*
+		 * If we've already configured this interrupt,
+		 * don't do it again, or hell will break loose.
+		 */
+		virq = irq_find_mapping(domain, hwirq);
+		if (virq)
+			return virq;
+
+		virq = irq_domain_alloc_irqs(domain, 1, NUMA_NO_NODE, irq_data);
+		if (virq <= 0)
+			return 0;
+	} else {
+		/* Create mapping */
+		virq = irq_create_mapping(domain, hwirq);
+		if (!virq)
+			return virq;
+	}
 
 	/* Set type if specified and different than the current one */
 	if (type != IRQ_TYPE_NONE &&
-- 
1.7.10.4

[patch 03/16] genirq: Introduce helper functions to support stacked irq_chip

[Thomas Gleixner]

[-- Attachment #1: genirq-introduce-helper-functions-to-support-stacked-irq_chip.patch --]
[-- Type: text/plain, Size: 1935 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

Now we already support hierarchy irq_data, so introduce several helpers
to support stacked irq_chips.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/irq.h |    5 +++++
 kernel/irq/chip.c   |   17 +++++++++++++++++
 2 files changed, 22 insertions(+)

diff --git a/include/linux/irq.h b/include/linux/irq.h
index 13ba412ce3a0..0adcbbbf2e87 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -443,6 +443,11 @@ extern void handle_percpu_devid_irq(unsigned int irq, struct irq_desc *desc);
 extern void handle_bad_irq(unsigned int irq, struct irq_desc *desc);
 extern void handle_nested_irq(unsigned int irq);
 
+#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
+extern void irq_chip_ack_parent(struct irq_data *data);
+extern int irq_chip_retrigger_hierarchy(struct irq_data *data);
+#endif
+
 /* Handling of unhandled and spurious interrupts: */
 extern void note_interrupt(unsigned int irq, struct irq_desc *desc,
 			   irqreturn_t action_ret);
diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index 72a93086216b..12f3e72449eb 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -850,3 +850,20 @@ void irq_cpu_offline(void)
 		raw_spin_unlock_irqrestore(&desc->lock, flags);
 	}
 }
+
+#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
+void irq_chip_ack_parent(struct irq_data *data)
+{
+	data = data->parent_data;
+	data->chip->irq_ack(data);
+}
+
+int irq_chip_retrigger_hierarchy(struct irq_data *data)
+{
+	for (data = data->parent_data; data; data = data->parent_data)
+		if (data->chip && data->chip->irq_retrigger)
+			return data->chip->irq_retrigger(data);
+
+	return -ENOSYS;
+}
+#endif
-- 
1.7.10.4

[patch 04/16] genirq: Introduce irq_chip.irq_compose_msi_msg() to support stacked irqchip

[Thomas Gleixner]

[-- Attachment #1: genirq-introduce-irq_chip-irq_compose_msi_msg-to-support-stacked-irqchip.patch --]
[-- Type: text/plain, Size: 2683 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

Add callback irq_compose_msi_msg to struct irq_chip, which will be used
to support stacked irqchip.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/irq.h |    5 +++++
 kernel/irq/chip.c   |   17 +++++++++++++++++
 2 files changed, 22 insertions(+)

diff --git a/include/linux/irq.h b/include/linux/irq.h
index 0adcbbbf2e87..536b7fc6c8f4 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -29,6 +29,7 @@ struct seq_file;
 struct module;
 struct irq_desc;
 struct irq_data;
+struct msi_msg;
 typedef	void (*irq_flow_handler_t)(unsigned int irq,
 					    struct irq_desc *desc);
 typedef	void (*irq_preflow_handler_t)(struct irq_data *data);
@@ -320,6 +321,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
  *				any other callback related to this irq
  * @irq_release_resources:	optional to release resources acquired with
  *				irq_request_resources
+ * @irq_compose_msi_msg:	optional to compose message content for MSI
  * @flags:		chip specific flags
  */
 struct irq_chip {
@@ -356,6 +358,8 @@ struct irq_chip {
 	int		(*irq_request_resources)(struct irq_data *data);
 	void		(*irq_release_resources)(struct irq_data *data);
 
+	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
+
 	unsigned long	flags;
 };
 
@@ -443,6 +447,7 @@ extern void handle_percpu_devid_irq(unsigned int irq, struct irq_desc *desc);
 extern void handle_bad_irq(unsigned int irq, struct irq_desc *desc);
 extern void handle_nested_irq(unsigned int irq);
 
+extern int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg);
 #ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
 extern void irq_chip_ack_parent(struct irq_data *data);
 extern int irq_chip_retrigger_hierarchy(struct irq_data *data);
diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index 12f3e72449eb..8f362db17a8a 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -867,3 +867,20 @@ int irq_chip_retrigger_hierarchy(struct irq_data *data)
 	return -ENOSYS;
 }
 #endif
+
+int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
+{
+	struct irq_data *pos = NULL;
+
+#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
+	for (; data; data = data->parent_data)
+#endif
+		if (data->chip && data->chip->irq_compose_msi_msg)
+			pos = data;
+	if (!pos)
+		return -ENOSYS;
+
+	pos->chip->irq_compose_msi_msg(pos, msg);
+
+	return 0;
+}
-- 
1.7.10.4

[patch 05/16] genirq: Add IRQ_SET_MASK_OK_DONE to support stacked irqchip

[Thomas Gleixner]

[-- Attachment #1: genirq-add-irq_set_mask_ok_done-to-support-stacked-irqchip.patch --]
[-- Type: text/plain, Size: 2075 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

Add IRQ_SET_MASK_OK_DONE in addition to IRQ_SET_MASK_OK and
IRQ_SET_MASK_OK_NOCOPY to support stacked irqchip. IRQ_SET_MASK_OK_DONE
is the same as IRQ_SET_MASK_OK to irq core. To stacked irqchip, it means
that ascendant irqchips have done all the work and no more handling
needed in descendant irqchips.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/irq.h |    4 ++++
 kernel/irq/manage.c |    2 ++
 2 files changed, 6 insertions(+)

diff --git a/include/linux/irq.h b/include/linux/irq.h
index 536b7fc6c8f4..041edd6dc409 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -114,10 +114,14 @@ enum {
  *
  * IRQ_SET_MASK_OK	- OK, core updates irq_data.affinity
  * IRQ_SET_MASK_NOCPY	- OK, chip did update irq_data.affinity
+ * IRQ_SET_MASK_OK_DONE	- Same as IRQ_SET_MASK_OK for core. Special code to
+ *			  support stacked irqchips, which indicates skipping
+ *			  all descendent irqchips.
  */
 enum {
 	IRQ_SET_MASK_OK = 0,
 	IRQ_SET_MASK_OK_NOCOPY,
+	IRQ_SET_MASK_OK_DONE,
 };
 
 struct msi_desc;
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index 0a9104b4608b..80692373abd6 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -183,6 +183,7 @@ int irq_do_set_affinity(struct irq_data *data, const struct cpumask *mask,
 	ret = chip->irq_set_affinity(data, mask, force);
 	switch (ret) {
 	case IRQ_SET_MASK_OK:
+	case IRQ_SET_MASK_OK_DONE:
 		cpumask_copy(data->affinity, mask);
 	case IRQ_SET_MASK_OK_NOCOPY:
 		irq_set_thread_affinity(desc);
@@ -600,6 +601,7 @@ int __irq_set_trigger(struct irq_desc *desc, unsigned int irq,
 
 	switch (ret) {
 	case IRQ_SET_MASK_OK:
+	case IRQ_SET_MASK_OK_DONE:
 		irqd_clear(&desc->irq_data, IRQD_TRIGGER_MASK);
 		irqd_set(&desc->irq_data, flags);
 
-- 
1.7.10.4

[patch 06/16] genirq: Split out flow handler typedefs into seperate header file

[Thomas Gleixner]

[-- Attachment #1: genirq-splitout-handler-typedef-into-seperate-header.patch --]
[-- Type: text/plain, Size: 1437 bytes --]

Required to avoid circular include dependencies.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/irq.h        |    6 +-----
 include/linux/irqhandler.h |   14 ++++++++++++++
 2 files changed, 15 insertions(+), 5 deletions(-)

Index: tip/include/linux/irq.h
===================================================================
--- tip.orig/include/linux/irq.h
+++ tip/include/linux/irq.h
@@ -15,6 +15,7 @@
 #include <linux/spinlock.h>
 #include <linux/cpumask.h>
 #include <linux/gfp.h>
+#include <linux/irqhandler.h>
 #include <linux/irqreturn.h>
 #include <linux/irqnr.h>
 #include <linux/errno.h>
@@ -27,12 +28,7 @@
 
 struct seq_file;
 struct module;
-struct irq_desc;
-struct irq_data;
 struct msi_msg;
-typedef	void (*irq_flow_handler_t)(unsigned int irq,
-					    struct irq_desc *desc);
-typedef	void (*irq_preflow_handler_t)(struct irq_data *data);
 
 /*
  * IRQ line status.
Index: tip/include/linux/irqhandler.h
===================================================================
--- /dev/null
+++ tip/include/linux/irqhandler.h
@@ -0,0 +1,14 @@
+#ifndef _LINUX_IRQHANDLER_H
+#define _LINUX_IRQHANDLER_H
+
+/*
+ * Interrupt flow handler typedefs are defined here to avoid circular
+ * include dependencies.
+ */
+
+struct irq_desc;
+struct irq_data;
+typedef	void (*irq_flow_handler_t)(unsigned int irq, struct irq_desc *desc);
+typedef	void (*irq_preflow_handler_t)(struct irq_data *data);
+
+#endif

[patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Thomas Gleixner]

[-- Attachment #1: rfc-part4-v1-02-17-genirq-introduce-helper-irq_domain_set_info-to-reduce-duplicated-code.patch --]
[-- Type: text/plain, Size: 2099 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/irqdomain.h |    5 +++++
 kernel/irq/irqdomain.c    |   10 ++++++++++
 2 files changed, 15 insertions(+)

Index: tip/include/linux/irqdomain.h
===================================================================
--- tip.orig/include/linux/irqdomain.h
+++ tip/include/linux/irqdomain.h
@@ -33,6 +33,7 @@
 #define _LINUX_IRQDOMAIN_H
 
 #include <linux/types.h>
+#include <linux/irqhandler.h>
 #include <linux/radix-tree.h>
 
 struct device_node;
@@ -263,6 +264,10 @@ extern int irq_domain_set_hwirq_and_chip
 					 irq_hw_number_t hwirq,
 					 struct irq_chip *chip,
 					 void *chip_data);
+extern void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
+				irq_hw_number_t hwirq, struct irq_chip *chip,
+				void *chip_data, irq_flow_handler_t handler,
+				void *handler_data, const char *handler_name);
 extern void irq_domain_reset_irq_data(struct irq_data *irq_data);
 extern void irq_domain_free_irqs_common(struct irq_domain *domain,
 					int virq, int nr_irqs);
Index: tip/kernel/irq/irqdomain.c
===================================================================
--- tip.orig/kernel/irq/irqdomain.c
+++ tip/kernel/irq/irqdomain.c
@@ -882,6 +882,16 @@ int irq_domain_set_hwirq_and_chip(struct
 	return 0;
 }
 
+void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
+			 irq_hw_number_t hwirq, struct irq_chip *chip,
+			 void *chip_data, irq_flow_handler_t handler,
+			 void *handler_data, const char *handler_name)
+{
+	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
+	__irq_set_handler(virq, handler, 0, handler_name);
+	irq_set_handler_data(virq, handler_data);
+}
+
 void irq_domain_reset_irq_data(struct irq_data *irq_data)
 {
 	irq_data->hwirq = 0;

[patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Thomas Gleixner]

[-- Attachment #1: rfc-part4-v1-06-17-genirq-introduce-callback-irq_chip-irq_write_msi_msg.patch --]
[-- Type: text/plain, Size: 1803 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

Introduce callback irq_chip.irq_write_msi_msg, so we can share common
code among MSI alike interrupt controllers, such as HPET and DMAR.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/irq.h |    8 ++++++++
 1 file changed, 8 insertions(+)

Index: tip/include/linux/irq.h
===================================================================
--- tip.orig/include/linux/irq.h
+++ tip/include/linux/irq.h
@@ -322,6 +322,7 @@ static inline irq_hw_number_t irqd_to_hw
  * @irq_release_resources:	optional to release resources acquired with
  *				irq_request_resources
  * @irq_compose_msi_msg:	optional to compose message content for MSI
+ * @irq_write_msi_msg:	optional to write message content for MSI
  * @flags:		chip specific flags
  */
 struct irq_chip {
@@ -359,6 +360,7 @@ struct irq_chip {
 	void		(*irq_release_resources)(struct irq_data *data);
 
 	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
+	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
 
 	unsigned long	flags;
 };
@@ -453,6 +455,12 @@ extern void irq_chip_ack_parent(struct i
 extern int irq_chip_retrigger_hierarchy(struct irq_data *data);
 #endif
 
+static inline void irq_chip_write_msi_msg(struct irq_data *data,
+					  struct msi_msg *msg)
+{
+	data->chip->irq_write_msi_msg(data, msg);
+}
+
 /* Handling of unhandled and spurious interrupts: */
 extern void note_interrupt(unsigned int irq, struct irq_desc *desc,
 			   irqreturn_t action_ret);

[patch 09/16] genirq: Add generic msi irq domain support

[Thomas Gleixner]

[-- Attachment #1: genirq-add-generic-msi-irq-support.patch --]
[-- Type: text/plain, Size: 6142 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

Implement the basic functions for MSI interrupt support with
hierarchical interrupt domains.

[ tglx: Extracted and combined from several patches ]

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/msi.h |   35 +++++++++++++++
 kernel/irq/Kconfig  |   10 ++++
 kernel/irq/Makefile |    1 
 kernel/irq/msi.c    |  119 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 165 insertions(+)

Index: tip/include/linux/msi.h
===================================================================
--- tip.orig/include/linux/msi.h
+++ tip/include/linux/msi.h
@@ -77,4 +77,39 @@ struct msi_chip {
 	void (*teardown_irq)(struct msi_chip *chip, unsigned int irq);
 };
 
+#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
+struct irq_domain;
+struct irq_chip;
+struct device_node;
+struct msi_domain_info;
+
+struct msi_domain_ops {
+	void		(*calc_hwirq)(struct msi_domain_info *info, void *arg,
+				      struct msi_desc *desc);
+	irq_hw_number_t	(*get_hwirq)(struct msi_domain_info *info, void *arg);
+	int		(*msi_init)(struct irq_domain *domain,
+				    struct msi_domain_info *info,
+				    unsigned int virq, irq_hw_number_t hwirq,
+				    void *arg);
+	void		(*msi_free)(struct irq_domain *domain,
+				    struct msi_domain_info *info,
+				    unsigned int virq);
+};
+
+struct msi_domain_info {
+	struct msi_domain_ops	*ops;
+	struct irq_chip		*chip;
+	void			*data;
+};
+
+int msi_domain_set_affinity(struct irq_data *data, const struct cpumask *mask,
+			    bool force);
+
+struct irq_domain *msi_create_irq_domain(struct device_node *of_node,
+					 struct msi_domain_info *info,
+					 struct irq_domain *parent);
+struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain);
+
+#endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */
+
 #endif /* LINUX_MSI_H */
Index: tip/kernel/irq/Kconfig
===================================================================
--- tip.orig/kernel/irq/Kconfig
+++ tip/kernel/irq/Kconfig
@@ -60,6 +60,16 @@ config IRQ_DOMAIN_HIERARCHY
 	bool
 	select IRQ_DOMAIN
 
+# Generic MSI interrupt support
+config GENERIC_MSI_IRQ
+	bool
+
+# Generic MSI hierarchical interrupt domain support
+config GENERIC_MSI_IRQ_DOMAIN
+	bool
+	select IRQ_DOMAIN_HIERARCHY
+	select GENERIC_MSI_IRQ
+
 config HANDLE_DOMAIN_IRQ
 	bool
 
Index: tip/kernel/irq/Makefile
===================================================================
--- tip.orig/kernel/irq/Makefile
+++ tip/kernel/irq/Makefile
@@ -6,3 +6,4 @@ obj-$(CONFIG_IRQ_DOMAIN) += irqdomain.o
 obj-$(CONFIG_PROC_FS) += proc.o
 obj-$(CONFIG_GENERIC_PENDING_IRQ) += migration.o
 obj-$(CONFIG_PM_SLEEP) += pm.o
+obj-$(CONFIG_GENERIC_MSI_IRQ) += msi.o
Index: tip/kernel/irq/msi.c
===================================================================
--- /dev/null
+++ tip/kernel/irq/msi.c
@@ -0,0 +1,119 @@
+/*
+ * linux/kernel/irq/msi.c
+ *
+ * Copyright (C) 2014 Intel Corp.
+ * Author: Jiang Liu <jiang.liu@linux.intel.com>
+ *
+ * This file is licensed under GPLv2.
+ *
+ * This file contains common code to support Message Signalled Interrupt for
+ * PCI compatible and non PCI compatible devices.
+ */
+#include <linux/irq.h>
+#include <linux/irqdomain.h>
+#include <linux/msi.h>
+
+#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
+int msi_domain_set_affinity(struct irq_data *irq_data,
+			    const struct cpumask *mask, bool force)
+{
+	struct irq_data *parent = irq_data->parent_data;
+	struct msi_msg msg;
+	int ret;
+
+	ret = parent->chip->irq_set_affinity(parent, mask, force);
+	if (ret >= 0 && ret != IRQ_SET_MASK_OK_DONE) {
+		BUG_ON(irq_chip_compose_msi_msg(irq_data, &msg));
+		irq_chip_write_msi_msg(irq_data, &msg);
+	}
+
+	return ret;
+}
+
+static void msi_domain_activate(struct irq_domain *domain,
+				struct irq_data *irq_data)
+{
+	struct msi_msg msg;
+
+	BUG_ON(irq_chip_compose_msi_msg(irq_data, &msg));
+	irq_chip_write_msi_msg(irq_data, &msg);
+}
+
+static void msi_domain_deactivate(struct irq_domain *domain,
+				  struct irq_data *irq_data)
+{
+	struct msi_msg msg;
+
+	memset(&msg, 0, sizeof(msg));
+	irq_chip_write_msi_msg(irq_data, &msg);
+}
+
+static int msi_domain_alloc(struct irq_domain *domain, unsigned int virq,
+			    unsigned int nr_irqs, void *arg)
+{
+	struct msi_domain_info *info = domain->host_data;
+	struct msi_domain_ops *ops = info->ops;
+	irq_hw_number_t hwirq = ops->get_hwirq(info, arg);
+	int i, ret;
+
+	if (irq_find_mapping(domain, hwirq) > 0)
+		return -EEXIST;
+
+	ret = irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, arg);
+	if (ret < 0)
+		return ret;
+
+	for (i = 0; i < nr_irqs; i++) {
+		ret = ops->msi_init(domain, info, virq + i, hwirq + i, arg);
+		if (ret < 0) {
+			if (ops->msi_free) {
+				for (i--; i > 0; i--)
+					ops->msi_free(domain, info, virq + i);
+			}
+			irq_domain_free_irqs_top(domain, virq, nr_irqs);
+			return ret;
+		}
+	}
+
+	return 0;
+}
+
+static void msi_domain_free(struct irq_domain *domain, unsigned int virq,
+			    unsigned int nr_irqs)
+{
+	struct msi_domain_info *info = domain->host_data;
+	int i;
+
+	if (info->ops->msi_free) {
+		for (i = 0; i < nr_irqs; i++)
+			info->ops->msi_free(domain, info, virq + i);
+	}
+	irq_domain_free_irqs_top(domain, virq, nr_irqs);
+}
+
+static struct irq_domain_ops msi_domain_ops = {
+	.alloc		= msi_domain_alloc,
+	.free		= msi_domain_free,
+	.activate	= msi_domain_activate,
+	.deactivate	= msi_domain_deactivate,
+};
+
+struct irq_domain *msi_create_irq_domain(struct device_node *of_node,
+					 struct msi_domain_info *info,
+					 struct irq_domain *parent)
+{
+	struct irq_domain *domain;
+
+	domain = irq_domain_add_tree(of_node, &msi_domain_ops, info);
+	if (domain)
+		domain->parent = parent;
+
+	return domain;
+}
+
+struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain)
+{
+	return (struct msi_domain_info *)domain->host_data;
+}
+
+#endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */

[patch 10/16] PCI/MSI: Move cached entry functions to irq core

[Thomas Gleixner]

[-- Attachment #1: PCI-MSI-Move-cached-entry-functions-to-irq-core.patch --]
[-- Type: text/plain, Size: 2352 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

Required to support non PCI based MSI.

[ tglx: Extracted from Jiangs patch series ]

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 drivers/pci/Kconfig |    1 +
 drivers/pci/msi.c   |   18 ------------------
 kernel/irq/msi.c    |   13 +++++++++++++
 3 files changed, 14 insertions(+), 18 deletions(-)

Index: tip/drivers/pci/Kconfig
===================================================================
--- tip.orig/drivers/pci/Kconfig
+++ tip/drivers/pci/Kconfig
@@ -4,6 +4,7 @@
 config PCI_MSI
 	bool "Message Signaled Interrupts (MSI and MSI-X)"
 	depends on PCI
+	select GENERIC_MSI_IRQ
 	help
 	   This allows device drivers to enable MSI (Message Signaled
 	   Interrupts).  Message Signaled Interrupts enable a device to
Index: tip/drivers/pci/msi.c
===================================================================
--- tip.orig/drivers/pci/msi.c
+++ tip/drivers/pci/msi.c
@@ -286,24 +286,6 @@ void read_msi_msg(unsigned int irq, stru
 	__read_msi_msg(entry, msg);
 }
 
-void __get_cached_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
-{
-	/* Assert that the cache is valid, assuming that
-	 * valid messages are not all-zeroes. */
-	BUG_ON(!(entry->msg.address_hi | entry->msg.address_lo |
-		 entry->msg.data));
-
-	*msg = entry->msg;
-}
-
-void get_cached_msi_msg(unsigned int irq, struct msi_msg *msg)
-{
-	struct msi_desc *entry = irq_get_msi_desc(irq);
-
-	__get_cached_msi_msg(entry, msg);
-}
-EXPORT_SYMBOL_GPL(get_cached_msi_msg);
-
 void __write_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
 {
 	if (entry->dev->current_state != PCI_D0) {
Index: tip/kernel/irq/msi.c
===================================================================
--- tip.orig/kernel/irq/msi.c
+++ tip/kernel/irq/msi.c
@@ -13,6 +13,19 @@
 #include <linux/irqdomain.h>
 #include <linux/msi.h>
 
+void __get_cached_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
+{
+	*msg = entry->msg;
+}
+
+void get_cached_msi_msg(unsigned int irq, struct msi_msg *msg)
+{
+	struct msi_desc *entry = irq_get_msi_desc(irq);
+
+	__get_cached_msi_msg(entry, msg);
+}
+EXPORT_SYMBOL_GPL(get_cached_msi_msg);
+
 #ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
 int msi_domain_set_affinity(struct irq_data *irq_data,
 			    const struct cpumask *mask, bool force)

[patch 11/16] PCI/MSI: Remove unnecessary braces around single statements

[Thomas Gleixner]

[-- Attachment #1: pci-msi-remove-unnecessary-braces-around-single-statements.patch --]
[-- Type: text/plain, Size: 1682 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

Per Documentation/CodingStyle, don't use braces around single statements.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 drivers/pci/msi.c |    9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

Index: tip/drivers/pci/msi.c
===================================================================
--- tip.orig/drivers/pci/msi.c
+++ tip/drivers/pci/msi.c
@@ -244,9 +244,8 @@ void default_restore_msi_irqs(struct pci
 {
 	struct msi_desc *entry;
 
-	list_for_each_entry(entry, &dev->msi_list, list) {
+	list_for_each_entry(entry, &dev->msi_list, list)
 		default_restore_msi_irq(dev, entry->irq);
-	}
 }
 
 void __read_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
@@ -433,9 +432,8 @@ static void __pci_restore_msix_state(str
 				PCI_MSIX_FLAGS_ENABLE | PCI_MSIX_FLAGS_MASKALL);
 
 	arch_restore_msi_irqs(dev);
-	list_for_each_entry(entry, &dev->msi_list, list) {
+	list_for_each_entry(entry, &dev->msi_list, list)
 		msix_mask_irq(entry, entry->masked);
-	}
 
 	msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_MASKALL, 0);
 }
@@ -479,9 +477,8 @@ static int populate_msi_sysfs(struct pci
 	int count = 0;
 
 	/* Determine how many msi entries we have */
-	list_for_each_entry(entry, &pdev->msi_list, list) {
+	list_for_each_entry(entry, &pdev->msi_list, list)
 		++num_msi;
-	}
 	if (!num_msi)
 		return 0;
 

[patch 12/16] PCI/MSI: Simplify PCI MSI code by initializing msi_desc.nvec_used earlier

[Thomas Gleixner]

[-- Attachment #1: pci-msi-simplify-pci-msi-code-by-initializing-msi_desc-nvec_used-earlier.patch --]
[-- Type: text/plain, Size: 4546 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

Simplify PCI MSI code by initializing msi_desc.nvec_used and
msi_desc.msi_attrib.multiple when creating MSI descriptors.

Also remove redundant checks in IRQ remapping drivers, PCI MSI core
already guarantees these.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 drivers/iommu/irq_remapping.c |    8 --------
 drivers/pci/msi.c             |   40 +++++++++++++++-------------------------
 2 files changed, 15 insertions(+), 33 deletions(-)

Index: tip/drivers/iommu/irq_remapping.c
===================================================================
--- tip.orig/drivers/iommu/irq_remapping.c
+++ tip/drivers/iommu/irq_remapping.c
@@ -56,19 +56,13 @@ static int do_setup_msi_irqs(struct pci_
 	unsigned int irq;
 	struct msi_desc *msidesc;
 
-	WARN_ON(!list_is_singular(&dev->msi_list));
 	msidesc = list_entry(dev->msi_list.next, struct msi_desc, list);
-	WARN_ON(msidesc->irq);
-	WARN_ON(msidesc->msi_attrib.multiple);
-	WARN_ON(msidesc->nvec_used);
 
 	irq = irq_alloc_hwirqs(nvec, dev_to_node(&dev->dev));
 	if (irq == 0)
 		return -ENOSPC;
 
 	nvec_pow2 = __roundup_pow_of_two(nvec);
-	msidesc->nvec_used = nvec;
-	msidesc->msi_attrib.multiple = ilog2(nvec_pow2);
 	for (sub_handle = 0; sub_handle < nvec; sub_handle++) {
 		if (!sub_handle) {
 			index = msi_alloc_remapped_irq(dev, irq, nvec_pow2);
@@ -96,8 +90,6 @@ error:
 	 * IRQs from tearing down again in default_teardown_msi_irqs()
 	 */
 	msidesc->irq = 0;
-	msidesc->nvec_used = 0;
-	msidesc->msi_attrib.multiple = 0;
 
 	return ret;
 }
Index: tip/drivers/pci/msi.c
===================================================================
--- tip.orig/drivers/pci/msi.c
+++ tip/drivers/pci/msi.c
@@ -85,19 +85,13 @@ int __weak arch_setup_msi_irqs(struct pc
  */
 void default_teardown_msi_irqs(struct pci_dev *dev)
 {
+	int i;
 	struct msi_desc *entry;
 
-	list_for_each_entry(entry, &dev->msi_list, list) {
-		int i, nvec;
-		if (entry->irq == 0)
-			continue;
-		if (entry->nvec_used)
-			nvec = entry->nvec_used;
-		else
-			nvec = 1 << entry->msi_attrib.multiple;
-		for (i = 0; i < nvec; i++)
-			arch_teardown_msi_irq(entry->irq + i);
-	}
+	list_for_each_entry(entry, &dev->msi_list, list)
+		if (entry->irq)
+			for (i = 0; i < entry->nvec_used; i++)
+				arch_teardown_msi_irq(entry->irq + i);
 }
 
 void __weak arch_teardown_msi_irqs(struct pci_dev *dev)
@@ -335,19 +329,12 @@ static void free_msi_irqs(struct pci_dev
 	struct msi_desc *entry, *tmp;
 	struct attribute **msi_attrs;
 	struct device_attribute *dev_attr;
-	int count = 0;
+	int i, count = 0;
 
-	list_for_each_entry(entry, &dev->msi_list, list) {
-		int i, nvec;
-		if (!entry->irq)
-			continue;
-		if (entry->nvec_used)
-			nvec = entry->nvec_used;
-		else
-			nvec = 1 << entry->msi_attrib.multiple;
-		for (i = 0; i < nvec; i++)
-			BUG_ON(irq_has_action(entry->irq + i));
-	}
+	list_for_each_entry(entry, &dev->msi_list, list)
+		if (entry->irq)
+			for (i = 0; i < entry->nvec_used; i++)
+				BUG_ON(irq_has_action(entry->irq + i));
 
 	arch_teardown_msi_irqs(dev);
 
@@ -538,7 +525,7 @@ error_attrs:
 	return ret;
 }
 
-static struct msi_desc *msi_setup_entry(struct pci_dev *dev)
+static struct msi_desc *msi_setup_entry(struct pci_dev *dev, int nvec)
 {
 	u16 control;
 	struct msi_desc *entry;
@@ -556,6 +543,8 @@ static struct msi_desc *msi_setup_entry(
 	entry->msi_attrib.maskbit	= !!(control & PCI_MSI_FLAGS_MASKBIT);
 	entry->msi_attrib.default_irq	= dev->irq;	/* Save IOAPIC IRQ */
 	entry->msi_attrib.multi_cap	= (control & PCI_MSI_FLAGS_QMASK) >> 1;
+	entry->msi_attrib.multiple	= ilog2(__roundup_pow_of_two(nvec));
+	entry->nvec_used		= nvec;
 
 	if (control & PCI_MSI_FLAGS_64BIT)
 		entry->mask_pos = dev->msi_cap + PCI_MSI_MASK_64;
@@ -588,7 +577,7 @@ static int msi_capability_init(struct pc
 
 	msi_set_enable(dev, 0);	/* Disable MSI during set up */
 
-	entry = msi_setup_entry(dev);
+	entry = msi_setup_entry(dev, nvec);
 	if (!entry)
 		return -ENOMEM;
 
@@ -659,6 +648,7 @@ static int msix_setup_entries(struct pci
 		entry->msi_attrib.entry_nr	= entries[i].entry;
 		entry->msi_attrib.default_irq	= dev->irq;
 		entry->mask_base		= base;
+		entry->nvec_used		= 1;
 
 		list_add_tail(&entry->list, &dev->msi_list);
 	}

[patch 13/16] PCI/MSI: Kill redundant call of irq_set_msi_desc() for MSI-X interrupts

[Thomas Gleixner]

[-- Attachment #1: pci-msi-kill-redundant-call-of-irq_set_msi_desc-for-msi-x-interrupts.patch --]
[-- Type: text/plain, Size: 1076 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

It is the repsonsibility of arch_setup_msi_irq()/arch_setup_msi_irqs()
to call irq_set_msi_desc() to associate IRQ descriptors and MSI
descriptors. Kill the redundant call of irq_set_msi_desc() for MSI-X
interrupts in the PCI MSI core.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 drivers/pci/msi.c |    1 -
 1 file changed, 1 deletion(-)

Index: tip/drivers/pci/msi.c
===================================================================
--- tip.orig/drivers/pci/msi.c
+++ tip/drivers/pci/msi.c
@@ -667,7 +667,6 @@ static void msix_program_entries(struct
 						PCI_MSIX_ENTRY_VECTOR_CTRL;
 
 		entries[i].vector = entry->irq;
-		irq_set_msi_desc(entry->irq, entry);
 		entry->masked = readl(entry->mask_base + offset);
 		msix_mask_irq(entry, 1);
 		i++;

[patch 14/16] PCI/MSI: Rename __read_msi_msg() to __pci_read_msi_msg()

[Thomas Gleixner]

[-- Attachment #1: rfc-part4-v1-11-17-pci-msi-rename-__read_msi_msg-as-__pci_read_msi_msg.patch --]
[-- Type: text/plain, Size: 3439 bytes --]

Rename __read_msi_msg() to __pci_read_msi_msg() and kill unused
read_msi_msg(). It's a preparation to separate generic MSI code from
PCI core.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/powerpc/platforms/pseries/msi.c |    2 +-
 arch/x86/pci/xen.c                   |    2 +-
 drivers/pci/msi.c                    |    9 +--------
 include/linux/msi.h                  |    6 ++++--
 4 files changed, 7 insertions(+), 12 deletions(-)

Index: tip/arch/powerpc/platforms/pseries/msi.c
===================================================================
--- tip.orig/arch/powerpc/platforms/pseries/msi.c
+++ tip/arch/powerpc/platforms/pseries/msi.c
@@ -476,7 +476,7 @@ again:
 		irq_set_msi_desc(virq, entry);
 
 		/* Read config space back so we can restore after reset */
-		__read_msi_msg(entry, &msg);
+		__pci_read_msi_msg(entry, &msg);
 		entry->msg = msg;
 	}
 
Index: tip/arch/x86/pci/xen.c
===================================================================
--- tip.orig/arch/x86/pci/xen.c
+++ tip/arch/x86/pci/xen.c
@@ -229,7 +229,7 @@ static int xen_hvm_setup_msi_irqs(struct
 		return 1;
 
 	list_for_each_entry(msidesc, &dev->msi_list, list) {
-		__read_msi_msg(msidesc, &msg);
+		__pci_read_msi_msg(msidesc, &msg);
 		pirq = MSI_ADDR_EXT_DEST_ID(msg.address_hi) |
 			((msg.address_lo >> MSI_ADDR_DEST_ID_SHIFT) & 0xff);
 		if (msg.data != XEN_PIRQ_MSI_DATA ||
Index: tip/drivers/pci/msi.c
===================================================================
--- tip.orig/drivers/pci/msi.c
+++ tip/drivers/pci/msi.c
@@ -242,7 +242,7 @@ void default_restore_msi_irqs(struct pci
 		default_restore_msi_irq(dev, entry->irq);
 }
 
-void __read_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
+void __pci_read_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
 {
 	BUG_ON(entry->dev->current_state != PCI_D0);
 
@@ -272,13 +272,6 @@ void __read_msi_msg(struct msi_desc *ent
 	}
 }
 
-void read_msi_msg(unsigned int irq, struct msi_msg *msg)
-{
-	struct msi_desc *entry = irq_get_msi_desc(irq);
-
-	__read_msi_msg(entry, msg);
-}
-
 void __write_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
 {
 	if (entry->dev->current_state != PCI_D0) {
Index: tip/include/linux/msi.h
===================================================================
--- tip.orig/include/linux/msi.h
+++ tip/include/linux/msi.h
@@ -15,10 +15,8 @@ struct irq_data;
 struct msi_desc;
 void mask_msi_irq(struct irq_data *data);
 void unmask_msi_irq(struct irq_data *data);
-void __read_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
 void __get_cached_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
 void __write_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
-void read_msi_msg(unsigned int irq, struct msi_msg *msg);
 void get_cached_msi_msg(unsigned int irq, struct msi_msg *msg);
 void write_msi_msg(unsigned int irq, struct msi_msg *msg);
 
@@ -48,6 +46,10 @@ struct msi_desc {
 	struct msi_msg msg;
 };
 
+#ifdef CONFIG_PCI_MSI
+void __pci_read_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
+#endif
+
 /*
  * The arch hooks to setup up msi irqs. Those functions are
  * implemented as weak symbols so that they /can/ be overriden by

[patch 15/16] PCI/MSI: Rename write_msi_msg() to pci_write_msi_msg()

[Thomas Gleixner]

[-- Attachment #1: rfc-part4-v1-12-17-pci-msi-rename-__write_msi_msg-as-__pci_write_msi_msg.patch --]
[-- Type: text/plain, Size: 14899 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

Rename write_msi_msg() to pci_write_msi_msg() to mark it as PCI
specific.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/arm/mach-iop13xx/msi.c            |    2 +-
 arch/ia64/kernel/msi_ia64.c            |    4 ++--
 arch/ia64/sn/kernel/msi_sn.c           |    4 ++--
 arch/mips/pci/msi-octeon.c             |    2 +-
 arch/mips/pci/msi-xlp.c                |    2 +-
 arch/mips/pci/pci-xlr.c                |    2 +-
 arch/powerpc/platforms/cell/axon_msi.c |    2 +-
 arch/powerpc/platforms/powernv/pci.c   |    2 +-
 arch/powerpc/sysdev/fsl_msi.c          |    2 +-
 arch/powerpc/sysdev/mpic_pasemi_msi.c  |    2 +-
 arch/powerpc/sysdev/mpic_u3msi.c       |    2 +-
 arch/powerpc/sysdev/ppc4xx_hsta_msi.c  |    2 +-
 arch/powerpc/sysdev/ppc4xx_msi.c       |    2 +-
 arch/s390/pci/pci.c                    |    2 +-
 arch/sparc/kernel/pci_msi.c            |    2 +-
 arch/tile/kernel/pci_gx.c              |    2 +-
 arch/x86/kernel/apic/io_apic.c         |    4 ++--
 arch/x86/pci/xen.c                     |    2 +-
 drivers/irqchip/irq-armada-370-xp.c    |    2 +-
 drivers/pci/host/pci-tegra.c           |    2 +-
 drivers/pci/host/pcie-designware.c     |    2 +-
 drivers/pci/host/pcie-rcar.c           |    2 +-
 drivers/pci/host/pcie-xilinx.c         |    2 +-
 drivers/pci/msi.c                      |   11 +++++------
 drivers/vfio/pci/vfio_pci_intrs.c      |    2 +-
 include/linux/msi.h                    |    4 ++--
 26 files changed, 34 insertions(+), 35 deletions(-)

Index: tip/arch/arm/mach-iop13xx/msi.c
===================================================================
--- tip.orig/arch/arm/mach-iop13xx/msi.c
+++ tip/arch/arm/mach-iop13xx/msi.c
@@ -153,7 +153,7 @@ int arch_setup_msi_irq(struct pci_dev *p
 	id = iop13xx_cpu_id();
 	msg.data = (id << IOP13XX_MU_MIMR_CORE_SELECT) | (irq & 0x7f);
 
-	write_msi_msg(irq, &msg);
+	pci_write_msi_msg(irq, &msg);
 	irq_set_chip_and_handler(irq, &iop13xx_msi_chip, handle_simple_irq);
 
 	return 0;
Index: tip/arch/ia64/kernel/msi_ia64.c
===================================================================
--- tip.orig/arch/ia64/kernel/msi_ia64.c
+++ tip/arch/ia64/kernel/msi_ia64.c
@@ -35,7 +35,7 @@ static int ia64_set_msi_irq_affinity(str
 	data |= MSI_DATA_VECTOR(irq_to_vector(irq));
 	msg.data = data;
 
-	write_msi_msg(irq, &msg);
+	pci_write_msi_msg(irq, &msg);
 	cpumask_copy(idata->affinity, cpumask_of(cpu));
 
 	return 0;
@@ -71,7 +71,7 @@ int ia64_setup_msi_irq(struct pci_dev *p
 		MSI_DATA_DELIVERY_FIXED |
 		MSI_DATA_VECTOR(vector);
 
-	write_msi_msg(irq, &msg);
+	pci_write_msi_msg(irq, &msg);
 	irq_set_chip_and_handler(irq, &ia64_msi_chip, handle_edge_irq);
 
 	return 0;
Index: tip/arch/ia64/sn/kernel/msi_sn.c
===================================================================
--- tip.orig/arch/ia64/sn/kernel/msi_sn.c
+++ tip/arch/ia64/sn/kernel/msi_sn.c
@@ -145,7 +145,7 @@ int sn_setup_msi_irq(struct pci_dev *pde
 	msg.data = 0x100 + irq;
 
 	irq_set_msi_desc(irq, entry);
-	write_msi_msg(irq, &msg);
+	pci_write_msi_msg(irq, &msg);
 	irq_set_chip_and_handler(irq, &sn_msi_chip, handle_edge_irq);
 
 	return 0;
@@ -205,7 +205,7 @@ static int sn_set_msi_irq_affinity(struc
 	msg.address_hi = (u32)(bus_addr >> 32);
 	msg.address_lo = (u32)(bus_addr & 0x00000000ffffffff);
 
-	write_msi_msg(irq, &msg);
+	pci_write_msi_msg(irq, &msg);
 	cpumask_copy(data->affinity, cpu_mask);
 
 	return 0;
Index: tip/arch/mips/pci/msi-octeon.c
===================================================================
--- tip.orig/arch/mips/pci/msi-octeon.c
+++ tip/arch/mips/pci/msi-octeon.c
@@ -178,7 +178,7 @@ msi_irq_allocated:
 	pci_write_config_word(dev, dev->msi_cap + PCI_MSI_FLAGS, control);
 
 	irq_set_msi_desc(irq, desc);
-	write_msi_msg(irq, &msg);
+	pci_write_msi_msg(irq, &msg);
 	return 0;
 }
 
Index: tip/arch/mips/pci/msi-xlp.c
===================================================================
--- tip.orig/arch/mips/pci/msi-xlp.c
+++ tip/arch/mips/pci/msi-xlp.c
@@ -345,7 +345,7 @@ static int xlp_setup_msi(uint64_t lnkbas
 	if (ret < 0)
 		return ret;
 
-	write_msi_msg(xirq, &msg);
+	pci_write_msi_msg(xirq, &msg);
 	return 0;
 }
 
Index: tip/arch/mips/pci/pci-xlr.c
===================================================================
--- tip.orig/arch/mips/pci/pci-xlr.c
+++ tip/arch/mips/pci/pci-xlr.c
@@ -260,7 +260,7 @@ int arch_setup_msi_irq(struct pci_dev *d
 	if (ret < 0)
 		return ret;
 
-	write_msi_msg(irq, &msg);
+	pci_write_msi_msg(irq, &msg);
 	return 0;
 }
 #endif
Index: tip/arch/powerpc/platforms/cell/axon_msi.c
===================================================================
--- tip.orig/arch/powerpc/platforms/cell/axon_msi.c
+++ tip/arch/powerpc/platforms/cell/axon_msi.c
@@ -279,7 +279,7 @@ static int axon_msi_setup_msi_irqs(struc
 
 		irq_set_msi_desc(virq, entry);
 		msg.data = virq;
-		write_msi_msg(virq, &msg);
+		pci_write_msi_msg(virq, &msg);
 	}
 
 	return 0;
Index: tip/arch/powerpc/platforms/powernv/pci.c
===================================================================
--- tip.orig/arch/powerpc/platforms/powernv/pci.c
+++ tip/arch/powerpc/platforms/powernv/pci.c
@@ -91,7 +91,7 @@ static int pnv_setup_msi_irqs(struct pci
 			return rc;
 		}
 		irq_set_msi_desc(virq, entry);
-		write_msi_msg(virq, &msg);
+		pci_write_msi_msg(virq, &msg);
 	}
 	return 0;
 }
Index: tip/arch/powerpc/sysdev/fsl_msi.c
===================================================================
--- tip.orig/arch/powerpc/sysdev/fsl_msi.c
+++ tip/arch/powerpc/sysdev/fsl_msi.c
@@ -242,7 +242,7 @@ static int fsl_setup_msi_irqs(struct pci
 		irq_set_msi_desc(virq, entry);
 
 		fsl_compose_msi_msg(pdev, hwirq, &msg, msi_data);
-		write_msi_msg(virq, &msg);
+		pci_write_msi_msg(virq, &msg);
 	}
 	return 0;
 
Index: tip/arch/powerpc/sysdev/mpic_pasemi_msi.c
===================================================================
--- tip.orig/arch/powerpc/sysdev/mpic_pasemi_msi.c
+++ tip/arch/powerpc/sysdev/mpic_pasemi_msi.c
@@ -136,7 +136,7 @@ static int pasemi_msi_setup_msi_irqs(str
 		 * register to generate MSI [512...1023]
 		 */
 		msg.data = hwirq-0x200;
-		write_msi_msg(virq, &msg);
+		pci_write_msi_msg(virq, &msg);
 	}
 
 	return 0;
Index: tip/arch/powerpc/sysdev/mpic_u3msi.c
===================================================================
--- tip.orig/arch/powerpc/sysdev/mpic_u3msi.c
+++ tip/arch/powerpc/sysdev/mpic_u3msi.c
@@ -171,7 +171,7 @@ static int u3msi_setup_msi_irqs(struct p
 		printk("u3msi: allocated virq 0x%x (hw 0x%x) addr 0x%lx\n",
 			  virq, hwirq, (unsigned long)addr);
 		msg.data = hwirq;
-		write_msi_msg(virq, &msg);
+		pci_write_msi_msg(virq, &msg);
 
 		hwirq++;
 	}
Index: tip/arch/powerpc/sysdev/ppc4xx_hsta_msi.c
===================================================================
--- tip.orig/arch/powerpc/sysdev/ppc4xx_hsta_msi.c
+++ tip/arch/powerpc/sysdev/ppc4xx_hsta_msi.c
@@ -85,7 +85,7 @@ static int hsta_setup_msi_irqs(struct pc
 			msi_bitmap_free_hwirqs(&ppc4xx_hsta_msi.bmp, irq, 1);
 			return -EINVAL;
 		}
-		write_msi_msg(hwirq, &msg);
+		pci_write_msi_msg(hwirq, &msg);
 	}
 
 	return 0;
Index: tip/arch/powerpc/sysdev/ppc4xx_msi.c
===================================================================
--- tip.orig/arch/powerpc/sysdev/ppc4xx_msi.c
+++ tip/arch/powerpc/sysdev/ppc4xx_msi.c
@@ -116,7 +116,7 @@ static int ppc4xx_setup_msi_irqs(struct
 
 		irq_set_msi_desc(virq, entry);
 		msg.data = int_no;
-		write_msi_msg(virq, &msg);
+		pci_write_msi_msg(virq, &msg);
 	}
 	return 0;
 }
Index: tip/arch/s390/pci/pci.c
===================================================================
--- tip.orig/arch/s390/pci/pci.c
+++ tip/arch/s390/pci/pci.c
@@ -403,7 +403,7 @@ int arch_setup_msi_irqs(struct pci_dev *
 		msg.data = hwirq;
 		msg.address_lo = zdev->msi_addr & 0xffffffff;
 		msg.address_hi = zdev->msi_addr >> 32;
-		write_msi_msg(irq, &msg);
+		pci_write_msi_msg(irq, &msg);
 		airq_iv_set_data(zdev->aibv, hwirq, irq);
 		hwirq++;
 	}
Index: tip/arch/sparc/kernel/pci_msi.c
===================================================================
--- tip.orig/arch/sparc/kernel/pci_msi.c
+++ tip/arch/sparc/kernel/pci_msi.c
@@ -161,7 +161,7 @@ static int sparc64_setup_msi_irq(unsigne
 	msg.data = msi;
 
 	irq_set_msi_desc(*irq_p, entry);
-	write_msi_msg(*irq_p, &msg);
+	pci_write_msi_msg(*irq_p, &msg);
 
 	return 0;
 
Index: tip/arch/tile/kernel/pci_gx.c
===================================================================
--- tip.orig/arch/tile/kernel/pci_gx.c
+++ tip/arch/tile/kernel/pci_gx.c
@@ -1590,7 +1590,7 @@ int arch_setup_msi_irq(struct pci_dev *p
 	msg.address_hi = msi_addr >> 32;
 	msg.address_lo = msi_addr & 0xffffffff;
 
-	write_msi_msg(irq, &msg);
+	pci_write_msi_msg(irq, &msg);
 	irq_set_chip_and_handler(irq, &tilegx_msi_chip, handle_level_irq);
 	irq_set_handler_data(irq, controller);
 
Index: tip/arch/x86/kernel/apic/io_apic.c
===================================================================
--- tip.orig/arch/x86/kernel/apic/io_apic.c
+++ tip/arch/x86/kernel/apic/io_apic.c
@@ -3158,7 +3158,7 @@ msi_set_affinity(struct irq_data *data,
 	msg.address_lo &= ~MSI_ADDR_DEST_ID_MASK;
 	msg.address_lo |= MSI_ADDR_DEST_ID(dest);
 
-	__write_msi_msg(data->msi_desc, &msg);
+	__pci_write_msi_msg(data->msi_desc, &msg);
 
 	return IRQ_SET_MASK_OK_NOCOPY;
 }
@@ -3196,7 +3196,7 @@ int setup_msi_irq(struct pci_dev *dev, s
 	 * MSI message denotes a contiguous group of IRQs, written for 0th IRQ.
 	 */
 	if (!irq_offset)
-		write_msi_msg(irq, &msg);
+		pci_write_msi_msg(irq, &msg);
 
 	setup_remapped_irq(irq, irq_cfg(irq), chip);
 
Index: tip/arch/x86/pci/xen.c
===================================================================
--- tip.orig/arch/x86/pci/xen.c
+++ tip/arch/x86/pci/xen.c
@@ -240,7 +240,7 @@ static int xen_hvm_setup_msi_irqs(struct
 				goto error;
 			}
 			xen_msi_compose_msg(dev, pirq, &msg);
-			__write_msi_msg(msidesc, &msg);
+			__pci_write_msi_msg(msidesc, &msg);
 			dev_dbg(&dev->dev, "xen: msi bound to pirq=%d\n", pirq);
 		} else {
 			dev_dbg(&dev->dev,
Index: tip/drivers/irqchip/irq-armada-370-xp.c
===================================================================
--- tip.orig/drivers/irqchip/irq-armada-370-xp.c
+++ tip/drivers/irqchip/irq-armada-370-xp.c
@@ -157,7 +157,7 @@ static int armada_370_xp_setup_msi_irq(s
 	msg.address_hi = 0;
 	msg.data = 0xf00 | (hwirq + 16);
 
-	write_msi_msg(virq, &msg);
+	pci_write_msi_msg(virq, &msg);
 	return 0;
 }
 
Index: tip/drivers/pci/host/pci-tegra.c
===================================================================
--- tip.orig/drivers/pci/host/pci-tegra.c
+++ tip/drivers/pci/host/pci-tegra.c
@@ -1308,7 +1308,7 @@ static int tegra_msi_setup_irq(struct ms
 	msg.address_hi = 0;
 	msg.data = hwirq;
 
-	write_msi_msg(irq, &msg);
+	pci_write_msi_msg(irq, &msg);
 
 	return 0;
 }
Index: tip/drivers/pci/host/pcie-designware.c
===================================================================
--- tip.orig/drivers/pci/host/pcie-designware.c
+++ tip/drivers/pci/host/pcie-designware.c
@@ -298,7 +298,7 @@ static int dw_msi_setup_irq(struct msi_c
 	else
 		msg.data = pos;
 
-	write_msi_msg(irq, &msg);
+	pci_write_msi_msg(irq, &msg);
 
 	return 0;
 }
Index: tip/drivers/pci/host/pcie-rcar.c
===================================================================
--- tip.orig/drivers/pci/host/pcie-rcar.c
+++ tip/drivers/pci/host/pcie-rcar.c
@@ -647,7 +647,7 @@ static int rcar_msi_setup_irq(struct msi
 	msg.address_hi = rcar_pci_read_reg(pcie, PCIEMSIAUR);
 	msg.data = hwirq;
 
-	write_msi_msg(irq, &msg);
+	pci_write_msi_msg(irq, &msg);
 
 	return 0;
 }
Index: tip/drivers/pci/host/pcie-xilinx.c
===================================================================
--- tip.orig/drivers/pci/host/pcie-xilinx.c
+++ tip/drivers/pci/host/pcie-xilinx.c
@@ -374,7 +374,7 @@ static int xilinx_pcie_msi_setup_irq(str
 	msg.address_lo = msg_addr;
 	msg.data = irq;
 
-	write_msi_msg(irq, &msg);
+	pci_write_msi_msg(irq, &msg);
 
 	return 0;
 }
Index: tip/drivers/pci/msi.c
===================================================================
--- tip.orig/drivers/pci/msi.c
+++ tip/drivers/pci/msi.c
@@ -26,7 +26,6 @@ static int pci_msi_enable = 1;
 
 #define msix_table_size(flags)	((flags & PCI_MSIX_FLAGS_QSIZE) + 1)
 
-
 /* Arch hooks */
 
 int __weak arch_setup_msi_irq(struct pci_dev *dev, struct msi_desc *desc)
@@ -114,7 +113,7 @@ static void default_restore_msi_irq(stru
 	}
 
 	if (entry)
-		__write_msi_msg(entry, &entry->msg);
+		__pci_write_msi_msg(entry, &entry->msg);
 }
 
 void __weak arch_restore_msi_irqs(struct pci_dev *dev)
@@ -272,7 +271,7 @@ void __pci_read_msi_msg(struct msi_desc
 	}
 }
 
-void __write_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
+void __pci_write_msi_msg(struct msi_desc *entry, struct msi_msg *msg)
 {
 	if (entry->dev->current_state != PCI_D0) {
 		/* Don't touch the hardware now */
@@ -309,13 +308,13 @@ void __write_msi_msg(struct msi_desc *en
 	entry->msg = *msg;
 }
 
-void write_msi_msg(unsigned int irq, struct msi_msg *msg)
+void pci_write_msi_msg(unsigned int irq, struct msi_msg *msg)
 {
 	struct msi_desc *entry = irq_get_msi_desc(irq);
 
-	__write_msi_msg(entry, msg);
+	__pci_write_msi_msg(entry, msg);
 }
-EXPORT_SYMBOL_GPL(write_msi_msg);
+EXPORT_SYMBOL_GPL(pci_write_msi_msg);
 
 static void free_msi_irqs(struct pci_dev *dev)
 {
Index: tip/drivers/vfio/pci/vfio_pci_intrs.c
===================================================================
--- tip.orig/drivers/vfio/pci/vfio_pci_intrs.c
+++ tip/drivers/vfio/pci/vfio_pci_intrs.c
@@ -560,7 +560,7 @@ static int vfio_msi_set_vector_signal(st
 		struct msi_msg msg;
 
 		get_cached_msi_msg(irq, &msg);
-		write_msi_msg(irq, &msg);
+		pci_write_msi_msg(irq, &msg);
 	}
 
 	ret = request_irq(irq, vfio_msihandler, 0,
Index: tip/include/linux/msi.h
===================================================================
--- tip.orig/include/linux/msi.h
+++ tip/include/linux/msi.h
@@ -16,9 +16,7 @@ struct msi_desc;
 void mask_msi_irq(struct irq_data *data);
 void unmask_msi_irq(struct irq_data *data);
 void __get_cached_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
-void __write_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
 void get_cached_msi_msg(unsigned int irq, struct msi_msg *msg);
-void write_msi_msg(unsigned int irq, struct msi_msg *msg);
 
 struct msi_desc {
 	struct {
@@ -48,6 +46,8 @@ struct msi_desc {
 
 #ifdef CONFIG_PCI_MSI
 void __pci_read_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
+void __pci_write_msi_msg(struct msi_desc *entry, struct msi_msg *msg);
+void pci_write_msi_msg(unsigned int irq, struct msi_msg *msg);
 #endif
 
 /*

[patch 16/16] PCI/MSI: Enhance core to support hierarchy irqdomain

[Thomas Gleixner]

[-- Attachment #1: pci-msi-enhance-core-to-support-hierarchy-irqdomain.patch --]
[-- Type: text/plain, Size: 3955 bytes --]

From: Jiang Liu <jiang.liu@linux.intel.com>

Enhance PCI MSI core to support hierarchy irqdomain, so the common
code can be shared across architectures.

[ tglx: Extracted and combined from several patches ]

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
Cc: Yijing Wang <wangyijing@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 drivers/pci/Kconfig |    5 ++++
 drivers/pci/msi.c   |   62 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/msi.h |    8 ++++++
 3 files changed, 75 insertions(+)

Index: tip/drivers/pci/Kconfig
===================================================================
--- tip.orig/drivers/pci/Kconfig
+++ tip/drivers/pci/Kconfig
@@ -17,6 +17,11 @@ config PCI_MSI
 
 	   If you don't know what to do here, say Y.
 
+config PCI_MSI_IRQ_DOMAIN
+	bool
+	depends on PCI_MSI
+	select GENERIC_MSI_IRQ_DOMAIN
+
 config PCI_DEBUG
 	bool "PCI Debugging"
 	depends on PCI && DEBUG_KERNEL
Index: tip/drivers/pci/msi.c
===================================================================
--- tip.orig/drivers/pci/msi.c
+++ tip/drivers/pci/msi.c
@@ -19,6 +19,7 @@
 #include <linux/errno.h>
 #include <linux/io.h>
 #include <linux/slab.h>
+#include <linux/irqdomain.h>
 
 #include "pci.h"
 
@@ -1072,3 +1073,64 @@ int pci_enable_msix_range(struct pci_dev
 	return nvec;
 }
 EXPORT_SYMBOL(pci_enable_msix_range);
+
+#ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
+void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg *msg)
+{
+	struct msi_desc *desc = irq_data->msi_desc;
+
+	/*
+	 * MSI-X message is written per-IRQ.
+	 * MSI message denotes a contiguous group of IRQs, written for 0th IRQ.
+	 */
+	if (desc->irq == irq_data->irq)
+		pci_write_msi_msg(desc, msg);
+}
+
+/*
+ * Generate a unique ID number for each possible MSI source, the ID number
+ * is only used within the irqdomain.
+ */
+irq_hw_number_t pci_msi_domain_calc_hwirq(struct pci_dev *dev,
+					  struct msi_desc *desc)
+{
+	return (irq_hw_number_t)desc->msi_attrib.entry_nr |
+		PCI_DEVID(dev->bus->number, dev->devfn) << 11 |
+		(pci_domain_nr(dev->bus) & 0xFFFFFFFF) << 27;
+}
+
+int pci_msi_domain_alloc_irqs(struct irq_domain *domain, int type,
+			      struct pci_dev *dev, void *arg)
+{
+	struct msi_domain_info *info = domain->host_data;
+	int node = dev_to_node(&dev->dev);
+	struct msi_desc *desc;
+	int i, virq;
+
+	list_for_each_entry(desc, &dev->msi_list, list) {
+		if (info->ops->calc_hwirq)
+			info->ops->calc_hwirq(info, arg, desc);
+
+		virq = irq_domain_alloc_irqs(domain, desc->nvec_used,
+					     node, arg);
+		if (virq < 0) {
+			/* Special handling for pci_enable_msi_range(). */
+			if (type == PCI_CAP_ID_MSI && desc->nvec_used > 1)
+				return 1;
+			else
+				return -ENOSPC;
+		}
+		for (i = 0; i < desc->nvec_used; i++)
+			irq_set_msi_desc_off(virq, i, desc);
+	}
+
+	list_for_each_entry(desc, &dev->msi_list, list)
+		if (desc->nvec_used == 1)
+			dev_dbg(&dev->dev, "irq %d for MSI/MSI-X\n", virq);
+		else
+			dev_dbg(&dev->dev, "irq [%d-%d] for MSI/MSI-X\n",
+				virq, virq + desc->nvec_used - 1);
+
+	return 0;
+}
+#endif /* CONFIG_PCI_MSI_IRQ_DOMAIN */
Index: tip/include/linux/msi.h
===================================================================
--- tip.orig/include/linux/msi.h
+++ tip/include/linux/msi.h
@@ -112,4 +112,12 @@ struct msi_domain_info *msi_get_domain_i
 
 #endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */
 
+#ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
+void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg *msg);
+int pci_msi_domain_alloc_irqs(struct irq_domain *domain, int type,
+			      struct pci_dev *dev, void *arg);
+irq_hw_number_t pci_msi_domain_calc_hwirq(struct pci_dev *dev,
+					  struct msi_desc *desc);
+#endif /* CONFIG_PCI_MSI_IRQ_DOMAIN */
+
 #endif /* LINUX_MSI_H */

Re: [patch 00/16] genirq: Hierarchical irq domains and generic MSI interrupt code

[Yingjoe Chen]

On Wed, 2014-11-12 at 13:42 +0000, Thomas Gleixner wrote:
> This is an extract from Jiangs various patch series which only
> contains the generic irq and MSI parts w/o the x86 specific
> modifications.
> 
> This is roughly what I plan to merge into the generic irq core, so the
> various outstanding patches (irqchip/x86/...) can be based on this.
> 
> It's available from git as well:
> 
>     git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain
> 
> Note, that branch is not yet exposed to linux-next and subject to
> modifications including rebasing.

Hi,

I checked the branch and it is just plain v3.18-rc4. Is this branch
ready now?

https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/log/?h=irq/irqdomain

Joe.C

Re: [patch 00/16] genirq: Hierarchical irq domains and generic MSI interrupt code

[Thomas Gleixner]

On Wed, 12 Nov 2014, Yingjoe Chen wrote:
> On Wed, 2014-11-12 at 13:42 +0000, Thomas Gleixner wrote:
> > This is an extract from Jiangs various patch series which only
> > contains the generic irq and MSI parts w/o the x86 specific
> > modifications.
> > 
> > This is roughly what I plan to merge into the generic irq core, so the
> > various outstanding patches (irqchip/x86/...) can be based on this.
> > 
> > It's available from git as well:
> > 
> >     git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain
> > 
> > Note, that branch is not yet exposed to linux-next and subject to
> > modifications including rebasing.
> 
> Hi,
> 
> I checked the branch and it is just plain v3.18-rc4. Is this branch
> ready now?
> 
> https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/log/?h=irq/irqdomain

It's in the master repository so it should be propagated towards the
public interfaces soon.

Thanks,

	tglx

Re: [patch 16/16] PCI/MSI: Enhance core to support hierarchy irqdomain

[Marc Zyngier]

Hi Thomas,

On 12/11/14 13:43, Thomas Gleixner wrote:
> From: Jiang Liu <jiang.liu@linux.intel.com>
> 
> Enhance PCI MSI core to support hierarchy irqdomain, so the common
> code can be shared across architectures.
> 
> [ tglx: Extracted and combined from several patches ]
> 
> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Grant Likely <grant.likely@linaro.org>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
> Cc: Yijing Wang <wangyijing@huawei.com>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  drivers/pci/Kconfig |    5 ++++
>  drivers/pci/msi.c   |   62 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>  include/linux/msi.h |    8 ++++++
>  3 files changed, 75 insertions(+)
> 
> Index: tip/drivers/pci/Kconfig
> ===================================================================
> --- tip.orig/drivers/pci/Kconfig
> +++ tip/drivers/pci/Kconfig
> @@ -17,6 +17,11 @@ config PCI_MSI
>  
>  	   If you don't know what to do here, say Y.
>  
> +config PCI_MSI_IRQ_DOMAIN
> +	bool
> +	depends on PCI_MSI
> +	select GENERIC_MSI_IRQ_DOMAIN
> +
>  config PCI_DEBUG
>  	bool "PCI Debugging"
>  	depends on PCI && DEBUG_KERNEL
> Index: tip/drivers/pci/msi.c
> ===================================================================
> --- tip.orig/drivers/pci/msi.c
> +++ tip/drivers/pci/msi.c
> @@ -19,6 +19,7 @@
>  #include <linux/errno.h>
>  #include <linux/io.h>
>  #include <linux/slab.h>
> +#include <linux/irqdomain.h>
>  
>  #include "pci.h"
>  
> @@ -1072,3 +1073,64 @@ int pci_enable_msix_range(struct pci_dev
>  	return nvec;
>  }
>  EXPORT_SYMBOL(pci_enable_msix_range);
> +
> +#ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
> +void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg *msg)
> +{
> +	struct msi_desc *desc = irq_data->msi_desc;
> +
> +	/*
> +	 * MSI-X message is written per-IRQ.
> +	 * MSI message denotes a contiguous group of IRQs, written for 0th IRQ.
> +	 */
> +	if (desc->irq == irq_data->irq)
> +		pci_write_msi_msg(desc, msg);

My compiler barks at this. Shouldn't that be either:
		pci_write_msi_msg(desc->irq, msg);
or
		__pci_write_msi_msg(desc, msg);
?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [patch 16/16] PCI/MSI: Enhance core to support hierarchy irqdomain

[Thomas Gleixner]

On Wed, 12 Nov 2014, Marc Zyngier wrote:
> > +#ifdef CONFIG_PCI_MSI_IRQ_DOMAIN
> > +void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg *msg)
> > +{
> > +	struct msi_desc *desc = irq_data->msi_desc;
> > +
> > +	/*
> > +	 * MSI-X message is written per-IRQ.
> > +	 * MSI message denotes a contiguous group of IRQs, written for 0th IRQ.
> > +	 */
> > +	if (desc->irq == irq_data->irq)
> > +		pci_write_msi_msg(desc, msg);
> 
> My compiler barks at this. Shouldn't that be either:
> 		pci_write_msi_msg(desc->irq, msg);
> or
> 		__pci_write_msi_msg(desc, msg);

The latter.

Thanks,

	tglx

Re: [patch 15/16] PCI/MSI: Rename write_msi_msg() to pci_write_msi_msg()

[Jiang Liu]

Hi Thomas,
	Could you please help to fold this into patch 15?
Regards!
Gerry
PCI, MSI: Replace write_msi_msg() with pci_write_msi_msg()

Commit e5f1a59c4e12 "PCI/MSI: Rename write_msi_msg() to pci_write_msi_msg()"
missed one instance of write_msi_msg() in file arch/mips/pci/msi-xlp.c.

Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
 arch/mips/pci/msi-xlp.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/mips/pci/msi-xlp.c b/arch/mips/pci/msi-xlp.c
index 262001ee1554..75bceb99e9a6 100644
--- a/arch/mips/pci/msi-xlp.c
+++ b/arch/mips/pci/msi-xlp.c
@@ -446,7 +446,7 @@ static int xlp_setup_msix(uint64_t lnkbase, int
node, int link,
        if (ret < 0)
                return ret;

-       write_msi_msg(xirq, &msg);
+       pci_write_msi_msg(xirq, &msg);
        return 0;
 }

--


On 2014/11/12 21:43, Thomas Gleixner wrote:

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Yingjoe Chen]

On Wed, 2014-11-12 at 13:43 +0000, Thomas Gleixner wrote:
> plain text document attachment
> (rfc-part4-v1-02-17-genirq-introduce-helper-irq_domain_set_info-to-reduce-duplicated-code.patch)
> From: Jiang Liu <jiang.liu@linux.intel.com>
> 
> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Grant Likely <grant.likely@linaro.org>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
> Cc: Yijing Wang <wangyijing@huawei.com>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  include/linux/irqdomain.h |    5 +++++
>  kernel/irq/irqdomain.c    |   10 ++++++++++
>  2 files changed, 15 insertions(+)
> 
> Index: tip/include/linux/irqdomain.h
> ===================================================================
> --- tip.orig/include/linux/irqdomain.h
> +++ tip/include/linux/irqdomain.h
> @@ -33,6 +33,7 @@
>  #define _LINUX_IRQDOMAIN_H
>  
>  #include <linux/types.h>
> +#include <linux/irqhandler.h>
>  #include <linux/radix-tree.h>
>  
>  struct device_node;
> @@ -263,6 +264,10 @@ extern int irq_domain_set_hwirq_and_chip
>  					 irq_hw_number_t hwirq,
>  					 struct irq_chip *chip,
>  					 void *chip_data);
> +extern void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
> +				irq_hw_number_t hwirq, struct irq_chip *chip,
> +				void *chip_data, irq_flow_handler_t handler,
> +				void *handler_data, const char *handler_name);
>  extern void irq_domain_reset_irq_data(struct irq_data *irq_data);
>  extern void irq_domain_free_irqs_common(struct irq_domain *domain,
>  					int virq, int nr_irqs);
> Index: tip/kernel/irq/irqdomain.c
> ===================================================================
> --- tip.orig/kernel/irq/irqdomain.c
> +++ tip/kernel/irq/irqdomain.c
> @@ -882,6 +882,16 @@ int irq_domain_set_hwirq_and_chip(struct
>  	return 0;
>  }
>  
> +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
> +			 irq_hw_number_t hwirq, struct irq_chip *chip,
> +			 void *chip_data, irq_flow_handler_t handler,
> +			 void *handler_data, const char *handler_name)
> +{
> +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
> +	__irq_set_handler(virq, handler, 0, handler_name);
> +	irq_set_handler_data(virq, handler_data);
> +}
> +

Hi,

While trying to use this function, I'm not sure about its interface.
This function have 8 parameters but only save 3 function calls. After
checking uses in Jiang's original patch, I think this make code harder
to understand.

Joe.C

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Jiang Liu]

On 2014/11/13 17:57, Yingjoe Chen wrote:
> On Wed, 2014-11-12 at 13:43 +0000, Thomas Gleixner wrote:
>> Index: tip/kernel/irq/irqdomain.c
>> ===================================================================
>> --- tip.orig/kernel/irq/irqdomain.c
>> +++ tip/kernel/irq/irqdomain.c
>> @@ -882,6 +882,16 @@ int irq_domain_set_hwirq_and_chip(struct
>>  	return 0;
>>  }
>>  
>> +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
>> +			 irq_hw_number_t hwirq, struct irq_chip *chip,
>> +			 void *chip_data, irq_flow_handler_t handler,
>> +			 void *handler_data, const char *handler_name)
>> +{
>> +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
>> +	__irq_set_handler(virq, handler, 0, handler_name);
>> +	irq_set_handler_data(virq, handler_data);
>> +}
>> +
> 
> Hi,
> 
> While trying to use this function, I'm not sure about its interface.
> This function have 8 parameters but only save 3 function calls. After
> checking uses in Jiang's original patch, I think this make code harder
> to understand.
Hi Joe,
	It's true, but we also want to reduce duplicated code:(
Regards!
Gerry
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Marc Zyngier]

On 13/11/14 10:00, Jiang Liu wrote:
> On 2014/11/13 17:57, Yingjoe Chen wrote:
>> On Wed, 2014-11-12 at 13:43 +0000, Thomas Gleixner wrote:
>>> Index: tip/kernel/irq/irqdomain.c
>>> ===================================================================
>>> --- tip.orig/kernel/irq/irqdomain.c
>>> +++ tip/kernel/irq/irqdomain.c
>>> @@ -882,6 +882,16 @@ int irq_domain_set_hwirq_and_chip(struct
>>>  	return 0;
>>>  }
>>>  
>>> +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
>>> +			 irq_hw_number_t hwirq, struct irq_chip *chip,
>>> +			 void *chip_data, irq_flow_handler_t handler,
>>> +			 void *handler_data, const char *handler_name)
>>> +{
>>> +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
>>> +	__irq_set_handler(virq, handler, 0, handler_name);
>>> +	irq_set_handler_data(virq, handler_data);
>>> +}
>>> +
>>
>> Hi,
>>
>> While trying to use this function, I'm not sure about its interface.
>> This function have 8 parameters but only save 3 function calls. After
>> checking uses in Jiang's original patch, I think this make code harder
>> to understand.
> Hi Joe,
> 	It's true, but we also want to reduce duplicated code:(

I'm in two minds about this one.

Yes, it looks horrible (to be fair, I hate anything that has more than 3
or 4 parameters, because I can never remember what they're for).

On the other hand, it gives you a nice check-point in your code, with
all the bits you need to set, or at least consider. Because it depends
on so many things that you may have to construct/obtain, you quickly
realize that there is not that many locations in your stack that fits
these requirements. Yes, this a twisted reasoning.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Marc Zyngier]

On 12/11/14 13:43, Thomas Gleixner wrote:
> From: Jiang Liu <jiang.liu@linux.intel.com>
> 
> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Grant Likely <grant.likely@linaro.org>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
> Cc: Yijing Wang <wangyijing@huawei.com>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  include/linux/irqdomain.h |    5 +++++
>  kernel/irq/irqdomain.c    |   10 ++++++++++
>  2 files changed, 15 insertions(+)
> 
> Index: tip/include/linux/irqdomain.h
> ===================================================================
> --- tip.orig/include/linux/irqdomain.h
> +++ tip/include/linux/irqdomain.h
> @@ -33,6 +33,7 @@
>  #define _LINUX_IRQDOMAIN_H
>  
>  #include <linux/types.h>
> +#include <linux/irqhandler.h>
>  #include <linux/radix-tree.h>
>  
>  struct device_node;
> @@ -263,6 +264,10 @@ extern int irq_domain_set_hwirq_and_chip
>  					 irq_hw_number_t hwirq,
>  					 struct irq_chip *chip,
>  					 void *chip_data);
> +extern void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
> +				irq_hw_number_t hwirq, struct irq_chip *chip,
> +				void *chip_data, irq_flow_handler_t handler,
> +				void *handler_data, const char *handler_name);
>  extern void irq_domain_reset_irq_data(struct irq_data *irq_data);
>  extern void irq_domain_free_irqs_common(struct irq_domain *domain,
>  					int virq, int nr_irqs);
> Index: tip/kernel/irq/irqdomain.c
> ===================================================================
> --- tip.orig/kernel/irq/irqdomain.c
> +++ tip/kernel/irq/irqdomain.c
> @@ -882,6 +882,16 @@ int irq_domain_set_hwirq_and_chip(struct
>  	return 0;
>  }
>  
> +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
> +			 irq_hw_number_t hwirq, struct irq_chip *chip,
> +			 void *chip_data, irq_flow_handler_t handler,
> +			 void *handler_data, const char *handler_name)
> +{
> +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
> +	__irq_set_handler(virq, handler, 0, handler_name);
> +	irq_set_handler_data(virq, handler_data);
> +}
> +

We still have the issue that, depending on where in the stack this is
 called, this will succeed or fail: If this is called from the inner
irqchip, __irq_set_handler() will fail, as it will look at the outer
domain as the (desc->irq_data.chip == &no_irq_chip) test fails (we
haven't set the top level yet).

I have this very imperfect workaround in my tree:

diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index d028b34..91e6515 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -731,7 +731,16 @@ __irq_set_handler(unsigned int irq, irq_flow_handler_t handle, int is_chained,
 	if (!handle) {
 		handle = handle_bad_irq;
 	} else {
-		if (WARN_ON(desc->irq_data.chip == &no_irq_chip))
+#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY
+		struct irq_data *irq_data = &desc->irq_data;
+		while (irq_data) {
+			if (irq_data->chip != &no_irq_chip)
+				break;
+			irq_data = irq_data->parent_data;
+		}
+#endif
+
+		if (WARN_ON(!irq_data || irq_data->chip == &no_irq_chip))
 			goto out;
 	}
 
Which translate into: If there is at least one irqchip in the domain,
it will probably sort itself out. Not ideal. Any real solution to
this problem?

GICv2 faces this exact problem, as some of its interrupts are used
directly, and some others are used through the MSI domain. In the
GIC driver, it is almost impossible to find out...

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Jiang Liu]

On 2014/11/14 23:31, Marc Zyngier wrote:
> On 12/11/14 13:43, Thomas Gleixner wrote:
>> From: Jiang Liu <jiang.liu@linux.intel.com>
>>
>> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>> Cc: Grant Likely <grant.likely@linaro.org>
>> Cc: Marc Zyngier <marc.zyngier@arm.com>
>> Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
>> Cc: Yijing Wang <wangyijing@huawei.com>
>> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
>> ---
>>  include/linux/irqdomain.h |    5 +++++
>>  kernel/irq/irqdomain.c    |   10 ++++++++++
>>  2 files changed, 15 insertions(+)
>>
>> Index: tip/include/linux/irqdomain.h
>> ===================================================================
>> --- tip.orig/include/linux/irqdomain.h
>> +++ tip/include/linux/irqdomain.h
>> @@ -33,6 +33,7 @@
>>  #define _LINUX_IRQDOMAIN_H
>>  
>>  #include <linux/types.h>
>> +#include <linux/irqhandler.h>
>>  #include <linux/radix-tree.h>
>>  
>>  struct device_node;
>> @@ -263,6 +264,10 @@ extern int irq_domain_set_hwirq_and_chip
>>  					 irq_hw_number_t hwirq,
>>  					 struct irq_chip *chip,
>>  					 void *chip_data);
>> +extern void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
>> +				irq_hw_number_t hwirq, struct irq_chip *chip,
>> +				void *chip_data, irq_flow_handler_t handler,
>> +				void *handler_data, const char *handler_name);
>>  extern void irq_domain_reset_irq_data(struct irq_data *irq_data);
>>  extern void irq_domain_free_irqs_common(struct irq_domain *domain,
>>  					int virq, int nr_irqs);
>> Index: tip/kernel/irq/irqdomain.c
>> ===================================================================
>> --- tip.orig/kernel/irq/irqdomain.c
>> +++ tip/kernel/irq/irqdomain.c
>> @@ -882,6 +882,16 @@ int irq_domain_set_hwirq_and_chip(struct
>>  	return 0;
>>  }
>>  
>> +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
>> +			 irq_hw_number_t hwirq, struct irq_chip *chip,
>> +			 void *chip_data, irq_flow_handler_t handler,
>> +			 void *handler_data, const char *handler_name)
>> +{
>> +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
>> +	__irq_set_handler(virq, handler, 0, handler_name);
>> +	irq_set_handler_data(virq, handler_data);
>> +}
>> +
> 
> We still have the issue that, depending on where in the stack this is
>  called, this will succeed or fail: If this is called from the inner
> irqchip, __irq_set_handler() will fail, as it will look at the outer
> domain as the (desc->irq_data.chip == &no_irq_chip) test fails (we
> haven't set the top level yet).
> 
> I have this very imperfect workaround in my tree:
> 
> diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
> index d028b34..91e6515 100644
> --- a/kernel/irq/chip.c
> +++ b/kernel/irq/chip.c
> @@ -731,7 +731,16 @@ __irq_set_handler(unsigned int irq, irq_flow_handler_t handle, int is_chained,
>  	if (!handle) {
>  		handle = handle_bad_irq;
>  	} else {
> -		if (WARN_ON(desc->irq_data.chip == &no_irq_chip))
> +#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY
> +		struct irq_data *irq_data = &desc->irq_data;
> +		while (irq_data) {
> +			if (irq_data->chip != &no_irq_chip)
> +				break;
> +			irq_data = irq_data->parent_data;
> +		}
> +#endif
> +
> +		if (WARN_ON(!irq_data || irq_data->chip == &no_irq_chip))
>  			goto out;
>  	}
>  
> Which translate into: If there is at least one irqchip in the domain,
> it will probably sort itself out. Not ideal. Any real solution to
> this problem?
> 
> GICv2 faces this exact problem, as some of its interrupts are used
> directly, and some others are used through the MSI domain. In the
> GIC driver, it is almost impossible to find out...
Hi Marc,
	I prefer the above solution to relax the warning conditions.
Changing the calling order in irq_domain_ops->alloc() looks a little
strange, and other interrupt drivers may still run into the same issue.
Regards!
Gerry

> 
> Thanks,
> 
> 	M.
> 

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Marc Zyngier]

On 14/11/14 15:41, Jiang Liu wrote:
> On 2014/11/14 23:31, Marc Zyngier wrote:
>> On 12/11/14 13:43, Thomas Gleixner wrote:
>>> From: Jiang Liu <jiang.liu@linux.intel.com>
>>>
>>> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
>>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>>> Cc: Grant Likely <grant.likely@linaro.org>
>>> Cc: Marc Zyngier <marc.zyngier@arm.com>
>>> Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
>>> Cc: Yijing Wang <wangyijing@huawei.com>
>>> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
>>> ---
>>>  include/linux/irqdomain.h |    5 +++++
>>>  kernel/irq/irqdomain.c    |   10 ++++++++++
>>>  2 files changed, 15 insertions(+)
>>>
>>> Index: tip/include/linux/irqdomain.h
>>> ===================================================================
>>> --- tip.orig/include/linux/irqdomain.h
>>> +++ tip/include/linux/irqdomain.h
>>> @@ -33,6 +33,7 @@
>>>  #define _LINUX_IRQDOMAIN_H
>>>  
>>>  #include <linux/types.h>
>>> +#include <linux/irqhandler.h>
>>>  #include <linux/radix-tree.h>
>>>  
>>>  struct device_node;
>>> @@ -263,6 +264,10 @@ extern int irq_domain_set_hwirq_and_chip
>>>  					 irq_hw_number_t hwirq,
>>>  					 struct irq_chip *chip,
>>>  					 void *chip_data);
>>> +extern void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
>>> +				irq_hw_number_t hwirq, struct irq_chip *chip,
>>> +				void *chip_data, irq_flow_handler_t handler,
>>> +				void *handler_data, const char *handler_name);
>>>  extern void irq_domain_reset_irq_data(struct irq_data *irq_data);
>>>  extern void irq_domain_free_irqs_common(struct irq_domain *domain,
>>>  					int virq, int nr_irqs);
>>> Index: tip/kernel/irq/irqdomain.c
>>> ===================================================================
>>> --- tip.orig/kernel/irq/irqdomain.c
>>> +++ tip/kernel/irq/irqdomain.c
>>> @@ -882,6 +882,16 @@ int irq_domain_set_hwirq_and_chip(struct
>>>  	return 0;
>>>  }
>>>  
>>> +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
>>> +			 irq_hw_number_t hwirq, struct irq_chip *chip,
>>> +			 void *chip_data, irq_flow_handler_t handler,
>>> +			 void *handler_data, const char *handler_name)
>>> +{
>>> +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
>>> +	__irq_set_handler(virq, handler, 0, handler_name);
>>> +	irq_set_handler_data(virq, handler_data);
>>> +}
>>> +
>>
>> We still have the issue that, depending on where in the stack this is
>>  called, this will succeed or fail: If this is called from the inner
>> irqchip, __irq_set_handler() will fail, as it will look at the outer
>> domain as the (desc->irq_data.chip == &no_irq_chip) test fails (we
>> haven't set the top level yet).
>>
>> I have this very imperfect workaround in my tree:
>>
>> diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
>> index d028b34..91e6515 100644
>> --- a/kernel/irq/chip.c
>> +++ b/kernel/irq/chip.c
>> @@ -731,7 +731,16 @@ __irq_set_handler(unsigned int irq, irq_flow_handler_t handle, int is_chained,
>>  	if (!handle) {
>>  		handle = handle_bad_irq;
>>  	} else {
>> -		if (WARN_ON(desc->irq_data.chip == &no_irq_chip))
>> +#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY
>> +		struct irq_data *irq_data = &desc->irq_data;
>> +		while (irq_data) {
>> +			if (irq_data->chip != &no_irq_chip)
>> +				break;
>> +			irq_data = irq_data->parent_data;
>> +		}
>> +#endif
>> +
>> +		if (WARN_ON(!irq_data || irq_data->chip == &no_irq_chip))
>>  			goto out;
>>  	}
>>  
>> Which translate into: If there is at least one irqchip in the domain,
>> it will probably sort itself out. Not ideal. Any real solution to
>> this problem?
>>
>> GICv2 faces this exact problem, as some of its interrupts are used
>> directly, and some others are used through the MSI domain. In the
>> GIC driver, it is almost impossible to find out...
> Hi Marc,
> 	I prefer the above solution to relax the warning conditions.
> Changing the calling order in irq_domain_ops->alloc() looks a little
> strange, and other interrupt drivers may still run into the same issue.

OK. Where do we from from there? Do you want a proper patch, or will you
fold this into the existing code?

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Jiang Liu]

On 2014/11/15 1:35, Marc Zyngier wrote:
> On 14/11/14 15:41, Jiang Liu wrote:
>> On 2014/11/14 23:31, Marc Zyngier wrote:
>>> On 12/11/14 13:43, Thomas Gleixner wrote:
>>>> From: Jiang Liu <jiang.liu@linux.intel.com>
>>>>
>>>> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
>>>> Cc: Bjorn Helgaas <bhelgaas@google.com>
>>>> Cc: Grant Likely <grant.likely@linaro.org>
>>>> Cc: Marc Zyngier <marc.zyngier@arm.com>
>>>> Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
>>>> Cc: Yijing Wang <wangyijing@huawei.com>
>>>> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
>>>> ---
>>>>  include/linux/irqdomain.h |    5 +++++
>>>>  kernel/irq/irqdomain.c    |   10 ++++++++++
>>>>  2 files changed, 15 insertions(+)
>>>>
>>>> Index: tip/include/linux/irqdomain.h
>>>> ===================================================================
>>>> --- tip.orig/include/linux/irqdomain.h
>>>> +++ tip/include/linux/irqdomain.h
>>>> @@ -33,6 +33,7 @@
>>>>  #define _LINUX_IRQDOMAIN_H
>>>>  
>>>>  #include <linux/types.h>
>>>> +#include <linux/irqhandler.h>
>>>>  #include <linux/radix-tree.h>
>>>>  
>>>>  struct device_node;
>>>> @@ -263,6 +264,10 @@ extern int irq_domain_set_hwirq_and_chip
>>>>  					 irq_hw_number_t hwirq,
>>>>  					 struct irq_chip *chip,
>>>>  					 void *chip_data);
>>>> +extern void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
>>>> +				irq_hw_number_t hwirq, struct irq_chip *chip,
>>>> +				void *chip_data, irq_flow_handler_t handler,
>>>> +				void *handler_data, const char *handler_name);
>>>>  extern void irq_domain_reset_irq_data(struct irq_data *irq_data);
>>>>  extern void irq_domain_free_irqs_common(struct irq_domain *domain,
>>>>  					int virq, int nr_irqs);
>>>> Index: tip/kernel/irq/irqdomain.c
>>>> ===================================================================
>>>> --- tip.orig/kernel/irq/irqdomain.c
>>>> +++ tip/kernel/irq/irqdomain.c
>>>> @@ -882,6 +882,16 @@ int irq_domain_set_hwirq_and_chip(struct
>>>>  	return 0;
>>>>  }
>>>>  
>>>> +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
>>>> +			 irq_hw_number_t hwirq, struct irq_chip *chip,
>>>> +			 void *chip_data, irq_flow_handler_t handler,
>>>> +			 void *handler_data, const char *handler_name)
>>>> +{
>>>> +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
>>>> +	__irq_set_handler(virq, handler, 0, handler_name);
>>>> +	irq_set_handler_data(virq, handler_data);
>>>> +}
>>>> +
>>>
>>> We still have the issue that, depending on where in the stack this is
>>>  called, this will succeed or fail: If this is called from the inner
>>> irqchip, __irq_set_handler() will fail, as it will look at the outer
>>> domain as the (desc->irq_data.chip == &no_irq_chip) test fails (we
>>> haven't set the top level yet).
>>>
>>> I have this very imperfect workaround in my tree:
>>>
>>> diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
>>> index d028b34..91e6515 100644
>>> --- a/kernel/irq/chip.c
>>> +++ b/kernel/irq/chip.c
>>> @@ -731,7 +731,16 @@ __irq_set_handler(unsigned int irq, irq_flow_handler_t handle, int is_chained,
>>>  	if (!handle) {
>>>  		handle = handle_bad_irq;
>>>  	} else {
>>> -		if (WARN_ON(desc->irq_data.chip == &no_irq_chip))
>>> +#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY
>>> +		struct irq_data *irq_data = &desc->irq_data;
>>> +		while (irq_data) {
>>> +			if (irq_data->chip != &no_irq_chip)
>>> +				break;
>>> +			irq_data = irq_data->parent_data;
>>> +		}
>>> +#endif
>>> +
>>> +		if (WARN_ON(!irq_data || irq_data->chip == &no_irq_chip))
>>>  			goto out;
>>>  	}
>>>  
>>> Which translate into: If there is at least one irqchip in the domain,
>>> it will probably sort itself out. Not ideal. Any real solution to
>>> this problem?
>>>
>>> GICv2 faces this exact problem, as some of its interrupts are used
>>> directly, and some others are used through the MSI domain. In the
>>> GIC driver, it is almost impossible to find out...
>> Hi Marc,
>> 	I prefer the above solution to relax the warning conditions.
>> Changing the calling order in irq_domain_ops->alloc() looks a little
>> strange, and other interrupt drivers may still run into the same issue.
> 
> OK. Where do we from from there? Do you want a proper patch, or will you
> fold this into the existing code?
A patch will be great:)

> 
> Thanks,
> 
> 	M.
> 

Re: [patch 00/16] genirq: Hierarchical irq domains and generic MSI interrupt code

[Yun Wu (Abel)]

Hi Thomas, Jiang,

I finally get some time on this, hopefully not too late...
In brief, I like the part of stacked domain, and have some opinions on
the part of refactoring MSI. Please check inline comments.

Regards,
	Abel

On 2014/11/12 21:42, Thomas Gleixner wrote:

> This is an extract from Jiangs various patch series which only
> contains the generic irq and MSI parts w/o the x86 specific
> modifications.
> 
> This is roughly what I plan to merge into the generic irq core, so the
> various outstanding patches (irqchip/x86/...) can be based on this.
> 
> It's available from git as well:
> 
>     git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain
> 
> Note, that branch is not yet exposed to linux-next and subject to
> modifications including rebasing.
> 
> Can all involved parties please have a close look and retest their
> patches on top of this?
> 
> Jiang, you need to merge that into x86/apic for rebasing your series.
> 
> Thanks,
> 
> 	tglx
> ----
>  Documentation/IRQ-domain.txt           |   71 +++++
>  arch/arm/mach-iop13xx/msi.c            |    2 
>  arch/ia64/kernel/msi_ia64.c            |    4 
>  arch/ia64/sn/kernel/msi_sn.c           |    4 
>  arch/mips/pci/msi-octeon.c             |    2 
>  arch/mips/pci/msi-xlp.c                |    2 
>  arch/mips/pci/pci-xlr.c                |    2 
>  arch/powerpc/platforms/cell/axon_msi.c |    2 
>  arch/powerpc/platforms/powernv/pci.c   |    2 
>  arch/powerpc/platforms/pseries/msi.c   |    2 
>  arch/powerpc/sysdev/fsl_msi.c          |    2 
>  arch/powerpc/sysdev/mpic_pasemi_msi.c  |    2 
>  arch/powerpc/sysdev/mpic_u3msi.c       |    2 
>  arch/powerpc/sysdev/ppc4xx_hsta_msi.c  |    2 
>  arch/powerpc/sysdev/ppc4xx_msi.c       |    2 
>  arch/s390/pci/pci.c                    |    2 
>  arch/sparc/kernel/pci_msi.c            |    2 
>  arch/tile/kernel/pci_gx.c              |    2 
>  arch/x86/kernel/apic/io_apic.c         |    4 
>  arch/x86/pci/xen.c                     |    4 
>  drivers/iommu/irq_remapping.c          |    8 
>  drivers/irqchip/irq-armada-370-xp.c    |    2 
>  drivers/pci/Kconfig                    |    6 
>  drivers/pci/host/pci-tegra.c           |    2 
>  drivers/pci/host/pcie-designware.c     |    2 
>  drivers/pci/host/pcie-rcar.c           |    2 
>  drivers/pci/host/pcie-xilinx.c         |    2 
>  drivers/pci/msi.c                      |  150 ++++++-----
>  drivers/vfio/pci/vfio_pci_intrs.c      |    2 
>  include/linux/irq.h                    |   33 ++
>  include/linux/irqdomain.h              |   91 +++++++
>  include/linux/irqhandler.h             |   14 +
>  include/linux/msi.h                    |   53 +++-
>  kernel/irq/Kconfig                     |   15 +
>  kernel/irq/Makefile                    |    1 
>  kernel/irq/chip.c                      |   37 ++
>  kernel/irq/irqdomain.c                 |  418 +++++++++++++++++++++++++++++++--
>  kernel/irq/manage.c                    |    2 
>  kernel/irq/msi.c                       |  132 ++++++++++
>  39 files changed, 959 insertions(+), 130 deletions(-)
> 
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> .
> 
> 

Re: [patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Yun Wu (Abel)]

Hi Thomas, Jiang,
On 2014/11/12 21:42, Thomas Gleixner wrote:

> From: Jiang Liu <jiang.liu@linux.intel.com>
> Index: tip/kernel/irq/chip.c
> ===================================================================
> --- tip.orig/kernel/irq/chip.c
> +++ tip/kernel/irq/chip.c
> @@ -15,6 +15,7 @@
>  #include <linux/module.h>
>  #include <linux/interrupt.h>
>  #include <linux/kernel_stat.h>
> +#include <linux/irqdomain.h>
>  
>  #include <trace/events/irq.h>
>  
> @@ -178,6 +179,7 @@ int irq_startup(struct irq_desc *desc, b
>  	irq_state_clr_disabled(desc);
>  	desc->depth = 0;
>  
> +	irq_domain_activate_irq(&desc->irq_data);

I'm not sure why this is needed, please help me out.. :)

Thanks,
	Abel

>  	if (desc->irq_data.chip->irq_startup) {
>  		ret = desc->irq_data.chip->irq_startup(&desc->irq_data);
>  		irq_state_clr_masked(desc);
> @@ -199,6 +201,7 @@ void irq_shutdown(struct irq_desc *desc)
>  		desc->irq_data.chip->irq_disable(&desc->irq_data);
>  	else
>  		desc->irq_data.chip->irq_mask(&desc->irq_data);
> +	irq_domain_deactivate_irq(&desc->irq_data);
>  	irq_state_set_masked(desc);
>  }
>  

Re: [patch 04/16] genirq: Introduce irq_chip.irq_compose_msi_msg() to support stacked irqchip

[Yun Wu (Abel)]

On 2014/11/12 21:42, Thomas Gleixner wrote:

> From: Jiang Liu <jiang.liu@linux.intel.com>
> 
> Add callback irq_compose_msi_msg to struct irq_chip, which will be used
> to support stacked irqchip.
> 
> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Grant Likely <grant.likely@linaro.org>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
> Cc: Yijing Wang <wangyijing@huawei.com>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  include/linux/irq.h |    5 +++++
>  kernel/irq/chip.c   |   17 +++++++++++++++++
>  2 files changed, 22 insertions(+)
> 
> diff --git a/include/linux/irq.h b/include/linux/irq.h
> index 0adcbbbf2e87..536b7fc6c8f4 100644
> --- a/include/linux/irq.h
> +++ b/include/linux/irq.h
> @@ -29,6 +29,7 @@ struct seq_file;
>  struct module;
>  struct irq_desc;
>  struct irq_data;
> +struct msi_msg;
>  typedef	void (*irq_flow_handler_t)(unsigned int irq,
>  					    struct irq_desc *desc);
>  typedef	void (*irq_preflow_handler_t)(struct irq_data *data);
> @@ -320,6 +321,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
>   *				any other callback related to this irq
>   * @irq_release_resources:	optional to release resources acquired with
>   *				irq_request_resources
> + * @irq_compose_msi_msg:	optional to compose message content for MSI
>   * @flags:		chip specific flags
>   */
>  struct irq_chip {
> @@ -356,6 +358,8 @@ struct irq_chip {
>  	int		(*irq_request_resources)(struct irq_data *data);
>  	void		(*irq_release_resources)(struct irq_data *data);
>  
> +	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
> +
>  	unsigned long	flags;
>  };
>  
> @@ -443,6 +447,7 @@ extern void handle_percpu_devid_irq(unsigned int irq, struct irq_desc *desc);
>  extern void handle_bad_irq(unsigned int irq, struct irq_desc *desc);
>  extern void handle_nested_irq(unsigned int irq);
>  
> +extern int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg);
>  #ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>  extern void irq_chip_ack_parent(struct irq_data *data);
>  extern int irq_chip_retrigger_hierarchy(struct irq_data *data);
> diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
> index 12f3e72449eb..8f362db17a8a 100644
> --- a/kernel/irq/chip.c
> +++ b/kernel/irq/chip.c
> @@ -867,3 +867,20 @@ int irq_chip_retrigger_hierarchy(struct irq_data *data)
>  	return -ENOSYS;
>  }
>  #endif
> +
> +int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
> +{
> +	struct irq_data *pos = NULL;
> +
> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
> +	for (; data; data = data->parent_data)
> +#endif
> +		if (data->chip && data->chip->irq_compose_msi_msg)
> +			pos = data;
> +	if (!pos)
> +		return -ENOSYS;
> +
> +	pos->chip->irq_compose_msi_msg(pos, msg);
> +
> +	return 0;
> +}

Adding message composing routine to struct irq_chip is OK to me, and it should
be because it is interrupt controllers' duty to compose messages (so that they
can parse the messages correctly without any pre-defined rules that endpoint
devices absolutely need not to know).
However a problem comes out when deciding which parameters should be passed to
this routine. A message can associate with multiple interrupts, which makes me
think composing messages for each interrupt is not that appropriate. And we
can take a look at the new routine irq_chip_compose_msi_msg(). It is called by
msi_domain_activate() which will be called by irq_domain_activate_irq() in
irq_startup() for each interrupt descriptor, result in composing a message for
each interrupt, right? (Unless requiring a judge on the parameter @data when
implementing the irq_compose_msi_msg() callback that only compose message for
the first entry of that message. But I really don't like that...)

Regards,
	Abel

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Yun Wu (Abel)]

Hi Thomas, Jiang,
On 2014/11/12 21:43, Thomas Gleixner wrote:

> From: Jiang Liu <jiang.liu@linux.intel.com>
[...]
> +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
> +			 irq_hw_number_t hwirq, struct irq_chip *chip,
> +			 void *chip_data, irq_flow_handler_t handler,
> +			 void *handler_data, const char *handler_name)
> +{
> +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
> +	__irq_set_handler(virq, handler, 0, handler_name);
> +	irq_set_handler_data(virq, handler_data);
> +}

When stacked domain enabled, there will be a semantic shift to the linux interrupt
identifiers. The @virq now delivers much more than before.
More specifically, now we need both @virq and @domain, rather than only @irq, to
determine which irq_data we want to configure. And once we configure @irq without
providing the exact domain, it means we are configuring all the domains related to
that @irq. So I think this routine just messed all things up.

Regards,
	Abel

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

Hi Thomas, Jiang,
On 2014/11/12 21:43, Thomas Gleixner wrote:

> From: Jiang Liu <jiang.liu@linux.intel.com>
> 
> Introduce callback irq_chip.irq_write_msi_msg, so we can share common
> code among MSI alike interrupt controllers, such as HPET and DMAR.
> 
> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Grant Likely <grant.likely@linaro.org>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
> Cc: Yijing Wang <wangyijing@huawei.com>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  include/linux/irq.h |    8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> Index: tip/include/linux/irq.h
> ===================================================================
> --- tip.orig/include/linux/irq.h
> +++ tip/include/linux/irq.h
> @@ -322,6 +322,7 @@ static inline irq_hw_number_t irqd_to_hw
>   * @irq_release_resources:	optional to release resources acquired with
>   *				irq_request_resources
>   * @irq_compose_msi_msg:	optional to compose message content for MSI
> + * @irq_write_msi_msg:	optional to write message content for MSI
>   * @flags:		chip specific flags
>   */
>  struct irq_chip {
> @@ -359,6 +360,7 @@ struct irq_chip {
>  	void		(*irq_release_resources)(struct irq_data *data);
>  
>  	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
> +	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);

Hmm... It's really weird.
I don't think it's the interrupt controllers' responsibility to write messages
for all the endpoint devices since the methods of configuring message registers
may different between these devices. And theoretically, the endpoint devices
themselves should take the responsibility to configure their message registers.
To say the least, the write_msg callback here still need to call some certain
interfaces provided by the corresponding device.
There would be lots of ARM new devices capable of sending message based interrupts
to interrupt controllers, does all the drivers of the devices need to expose a
write_msg callback to interrupt controllers?

Regards,
	Abel

>  
>  	unsigned long	flags;
>  };
> @@ -453,6 +455,12 @@ extern void irq_chip_ack_parent(struct i
>  extern int irq_chip_retrigger_hierarchy(struct irq_data *data);
>  #endif
>  
> +static inline void irq_chip_write_msi_msg(struct irq_data *data,
> +					  struct msi_msg *msg)
> +{
> +	data->chip->irq_write_msi_msg(data, msg);
> +}
> +
>  /* Handling of unhandled and spurious interrupts: */
>  extern void note_interrupt(unsigned int irq, struct irq_desc *desc,
>  			   irqreturn_t action_ret);
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> .
> 
> 

Re: [patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Thomas Gleixner]

On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
> Hi Thomas, Jiang,
> On 2014/11/12 21:42, Thomas Gleixner wrote:
> 
> > From: Jiang Liu <jiang.liu@linux.intel.com>
> > Index: tip/kernel/irq/chip.c
> > ===================================================================
> > --- tip.orig/kernel/irq/chip.c
> > +++ tip/kernel/irq/chip.c
> > @@ -15,6 +15,7 @@
> >  #include <linux/module.h>
> >  #include <linux/interrupt.h>
> >  #include <linux/kernel_stat.h>
> > +#include <linux/irqdomain.h>
> >  
> >  #include <trace/events/irq.h>
> >  
> > @@ -178,6 +179,7 @@ int irq_startup(struct irq_desc *desc, b
> >  	irq_state_clr_disabled(desc);
> >  	desc->depth = 0;
> >  
> > +	irq_domain_activate_irq(&desc->irq_data);
> 
> I'm not sure why this is needed, please help me out.. :)

Because it makes the whole error handling in stacked allocations way
simpler.

We allocate and reserve resources, but do not program the hardware up
to the point where request_irq and therefor irq_startup() is invoked.

So when in the allocation/reservation phase any of the stack level
fails we just have to undo the allocations/reservations and not any
hardware settings.

That also solves the issue that depending on the stacking we might not
be able to program the hardware during the allocation because all
stack levels need to be allocated/reserved before we can figure out
which hardware configuration we need for the various levels.

So we decided to postpone the actual hardware programming to the point
where the interrupt actually gets used.

Thanks,

	tglx

Re: [patch 04/16] genirq: Introduce irq_chip.irq_compose_msi_msg() to support stacked irqchip

[Thomas Gleixner]

On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
> On 2014/11/12 21:42, Thomas Gleixner wrote:
> > +int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
> > +{
> > +	struct irq_data *pos = NULL;
> > +
> > +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
> > +	for (; data; data = data->parent_data)
> > +#endif
> > +		if (data->chip && data->chip->irq_compose_msi_msg)
> > +			pos = data;
> > +	if (!pos)
> > +		return -ENOSYS;
> > +
> > +	pos->chip->irq_compose_msi_msg(pos, msg);
> > +
> > +	return 0;
> > +}
> 
> Adding message composing routine to struct irq_chip is OK to me, and it should
> be because it is interrupt controllers' duty to compose messages (so that they
> can parse the messages correctly without any pre-defined rules that endpoint
> devices absolutely need not to know).
> However a problem comes out when deciding which parameters should be passed to
> this routine. A message can associate with multiple interrupts, which makes me
> think composing messages for each interrupt is not that appropriate. And we
> can take a look at the new routine irq_chip_compose_msi_msg(). It is called by
> msi_domain_activate() which will be called by irq_domain_activate_irq() in
> irq_startup() for each interrupt descriptor, result in composing a message for
> each interrupt, right? (Unless requiring a judge on the parameter @data when
> implementing the irq_compose_msi_msg() callback that only compose message for
> the first entry of that message. But I really don't like that...)

No, that's not correct. You are looking at some random stale version
of this. The current state of affairs is in 

   git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain

See also https://lkml.org/lkml/2014/11/17/764

In activate we write the message, which is the right point to do so.

Thanks,

	tglx

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Thomas Gleixner]

On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:

> Hi Thomas, Jiang,
> On 2014/11/12 21:43, Thomas Gleixner wrote:
> 
> > From: Jiang Liu <jiang.liu@linux.intel.com>
> [...]
> > +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
> > +			 irq_hw_number_t hwirq, struct irq_chip *chip,
> > +			 void *chip_data, irq_flow_handler_t handler,
> > +			 void *handler_data, const char *handler_name)
> > +{
> > +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
> > +	__irq_set_handler(virq, handler, 0, handler_name);
> > +	irq_set_handler_data(virq, handler_data);
> > +}
> 
> When stacked domain enabled, there will be a semantic shift to the linux interrupt
> identifiers. The @virq now delivers much more than before.
> More specifically, now we need both @virq and @domain, rather than only @irq, to
> determine which irq_data we want to configure. And once we configure @irq without
> providing the exact domain, it means we are configuring all the domains related to
> that @irq. So I think this routine just messed all things up.

You can mess up anything by using an interface in the wrong way. Open
coding will not make that harder.

Thanks,

	tglx

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Thomas Gleixner]

On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
> On 2014/11/12 21:43, Thomas Gleixner wrote:
> >  struct irq_chip {
> > @@ -359,6 +360,7 @@ struct irq_chip {
> >  	void		(*irq_release_resources)(struct irq_data *data);
> >  
> >  	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
> > +	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
> 
> Hmm... It's really weird.
> I don't think it's the interrupt controllers' responsibility to write messages
> for all the endpoint devices since the methods of configuring message registers
> may different between these devices. And theoretically, the endpoint devices
> themselves should take the responsibility to configure their message registers.
> To say the least, the write_msg callback here still need to call some certain
> interfaces provided by the corresponding device.
>
> There would be lots of ARM new devices capable of sending message
> based interrupts to interrupt controllers, does all the drivers of
> the devices need to expose a write_msg callback to interrupt
> controllers?

Well, writing the message _IS_ part of the interrupt controller.

So in order to enable non PCI based MSI we want to have generic
infrastructure with minimal per device/device class callbacks and of
course you need to provide that callback for your special device.

We already have non PCI based MSI controllers in x86 today and we need
to handle the whole stuff with tons of copied coded extra for each of
those. So consolidating it into common infrastructure allows us to get
rid of the pointless copied code and reduce the per device effort to
the relevant hardware specific callbacks. irq_write_msi_msg being one
of those.

Thanks,

	tglx

Re: [patch 04/16] genirq: Introduce irq_chip.irq_compose_msi_msg() to support stacked irqchip

[Yun Wu (Abel)]

On 2014/11/18 18:02, Thomas Gleixner wrote:

> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>> +int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
>>> +{
>>> +	struct irq_data *pos = NULL;
>>> +
>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>> +	for (; data; data = data->parent_data)
>>> +#endif
>>> +		if (data->chip && data->chip->irq_compose_msi_msg)
>>> +			pos = data;
>>> +	if (!pos)
>>> +		return -ENOSYS;
>>> +
>>> +	pos->chip->irq_compose_msi_msg(pos, msg);
>>> +
>>> +	return 0;
>>> +}
>>
>> Adding message composing routine to struct irq_chip is OK to me, and it should
>> be because it is interrupt controllers' duty to compose messages (so that they
>> can parse the messages correctly without any pre-defined rules that endpoint
>> devices absolutely need not to know).
>> However a problem comes out when deciding which parameters should be passed to
>> this routine. A message can associate with multiple interrupts, which makes me
>> think composing messages for each interrupt is not that appropriate. And we
>> can take a look at the new routine irq_chip_compose_msi_msg(). It is called by
>> msi_domain_activate() which will be called by irq_domain_activate_irq() in
>> irq_startup() for each interrupt descriptor, result in composing a message for
>> each interrupt, right? (Unless requiring a judge on the parameter @data when
>> implementing the irq_compose_msi_msg() callback that only compose message for
>> the first entry of that message. But I really don't like that...)
> 
> No, that's not correct. You are looking at some random stale version
> of this. The current state of affairs is in 
> 
>    git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain
> 
> See also https://lkml.org/lkml/2014/11/17/764
> 
> In activate we write the message, which is the right point to do so.
> 

I checked the current state, it seems to be the same.
Yes, the decision of postponing the actual hardware programming to the point
where the interrupt actually gets used is right, but here above I was talking
another thing.
As I mentioned, a message can associate with multiple interrupts. Enabling
any of them will call irq_startup(). So if we don't want to compose or write
messages repeatedly, we'd better require performing some checks before
activating the interrupts.

Thanks,
	Abel

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Yun Wu (Abel)]

On 2014/11/18 18:03, Thomas Gleixner wrote:

> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
> 
>> Hi Thomas, Jiang,
>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>
>>> From: Jiang Liu <jiang.liu@linux.intel.com>
>> [...]
>>> +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
>>> +			 irq_hw_number_t hwirq, struct irq_chip *chip,
>>> +			 void *chip_data, irq_flow_handler_t handler,
>>> +			 void *handler_data, const char *handler_name)
>>> +{
>>> +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
>>> +	__irq_set_handler(virq, handler, 0, handler_name);
>>> +	irq_set_handler_data(virq, handler_data);
>>> +}
>>
>> When stacked domain enabled, there will be a semantic shift to the linux interrupt
>> identifiers. The @virq now delivers much more than before.
>> More specifically, now we need both @virq and @domain, rather than only @irq, to
>> determine which irq_data we want to configure. And once we configure @irq without
>> providing the exact domain, it means we are configuring all the domains related to
>> that @irq. So I think this routine just messed all things up.
> 
> You can mess up anything by using an interface in the wrong way. Open
> coding will not make that harder.
> 

But what's the correct way to use this interface?

Thanks,
	Abel

Re: [patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Yun Wu (Abel)]

On 2014/11/18 17:54, Thomas Gleixner wrote:

> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>> Hi Thomas, Jiang,
>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>
>>> From: Jiang Liu <jiang.liu@linux.intel.com>
>>> Index: tip/kernel/irq/chip.c
>>> ===================================================================
>>> --- tip.orig/kernel/irq/chip.c
>>> +++ tip/kernel/irq/chip.c
>>> @@ -15,6 +15,7 @@
>>>  #include <linux/module.h>
>>>  #include <linux/interrupt.h>
>>>  #include <linux/kernel_stat.h>
>>> +#include <linux/irqdomain.h>
>>>  
>>>  #include <trace/events/irq.h>
>>>  
>>> @@ -178,6 +179,7 @@ int irq_startup(struct irq_desc *desc, b
>>>  	irq_state_clr_disabled(desc);
>>>  	desc->depth = 0;
>>>  
>>> +	irq_domain_activate_irq(&desc->irq_data);
>>
>> I'm not sure why this is needed, please help me out.. :)
> 
> Because it makes the whole error handling in stacked allocations way
> simpler.
> 
> We allocate and reserve resources, but do not program the hardware up
> to the point where request_irq and therefor irq_startup() is invoked.
> 
> So when in the allocation/reservation phase any of the stack level
> fails we just have to undo the allocations/reservations and not any
> hardware settings.
> 
> That also solves the issue that depending on the stacking we might not
> be able to program the hardware during the allocation because all
> stack levels need to be allocated/reserved before we can figure out
> which hardware configuration we need for the various levels.
> 
> So we decided to postpone the actual hardware programming to the point
> where the interrupt actually gets used.
> 

OK, I got it.

Thanks,
	Abel

Re: [patch 09/16] genirq: Add generic msi irq domain support

[Yun Wu (Abel)]

On 2014/11/12 21:43, Thomas Gleixner wrote:

> From: Jiang Liu <jiang.liu@linux.intel.com>
> 
> Implement the basic functions for MSI interrupt support with
> hierarchical interrupt domains.
> 
> [ tglx: Extracted and combined from several patches ]
> 
> Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Grant Likely <grant.likely@linaro.org>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Yingjoe Chen <yingjoe.chen@mediatek.com>
> Cc: Yijing Wang <wangyijing@huawei.com>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  include/linux/msi.h |   35 +++++++++++++++
>  kernel/irq/Kconfig  |   10 ++++
>  kernel/irq/Makefile |    1 
>  kernel/irq/msi.c    |  119 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 165 insertions(+)
> 
> Index: tip/include/linux/msi.h
> ===================================================================
> --- tip.orig/include/linux/msi.h
> +++ tip/include/linux/msi.h
> @@ -77,4 +77,39 @@ struct msi_chip {
>  	void (*teardown_irq)(struct msi_chip *chip, unsigned int irq);
>  };
>  
> +#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
> +struct irq_domain;
> +struct irq_chip;
> +struct device_node;
> +struct msi_domain_info;
> +
> +struct msi_domain_ops {
> +	void		(*calc_hwirq)(struct msi_domain_info *info, void *arg,
> +				      struct msi_desc *desc);
> +	irq_hw_number_t	(*get_hwirq)(struct msi_domain_info *info, void *arg);
> +	int		(*msi_init)(struct irq_domain *domain,
> +				    struct msi_domain_info *info,
> +				    unsigned int virq, irq_hw_number_t hwirq,
> +				    void *arg);
> +	void		(*msi_free)(struct irq_domain *domain,
> +				    struct msi_domain_info *info,
> +				    unsigned int virq);
> +};
> +
> +struct msi_domain_info {
> +	struct msi_domain_ops	*ops;
> +	struct irq_chip		*chip;
> +	void			*data;
> +};
> +
> +int msi_domain_set_affinity(struct irq_data *data, const struct cpumask *mask,
> +			    bool force);
> +
> +struct irq_domain *msi_create_irq_domain(struct device_node *of_node,
> +					 struct msi_domain_info *info,
> +					 struct irq_domain *parent);
> +struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain);
> +
> +#endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */
> +
>  #endif /* LINUX_MSI_H */
> Index: tip/kernel/irq/Kconfig
> ===================================================================
> --- tip.orig/kernel/irq/Kconfig
> +++ tip/kernel/irq/Kconfig
> @@ -60,6 +60,16 @@ config IRQ_DOMAIN_HIERARCHY
>  	bool
>  	select IRQ_DOMAIN
>  
> +# Generic MSI interrupt support
> +config GENERIC_MSI_IRQ
> +	bool
> +
> +# Generic MSI hierarchical interrupt domain support
> +config GENERIC_MSI_IRQ_DOMAIN
> +	bool
> +	select IRQ_DOMAIN_HIERARCHY
> +	select GENERIC_MSI_IRQ
> +
>  config HANDLE_DOMAIN_IRQ
>  	bool
>  
> Index: tip/kernel/irq/Makefile
> ===================================================================
> --- tip.orig/kernel/irq/Makefile
> +++ tip/kernel/irq/Makefile
> @@ -6,3 +6,4 @@ obj-$(CONFIG_IRQ_DOMAIN) += irqdomain.o
>  obj-$(CONFIG_PROC_FS) += proc.o
>  obj-$(CONFIG_GENERIC_PENDING_IRQ) += migration.o
>  obj-$(CONFIG_PM_SLEEP) += pm.o
> +obj-$(CONFIG_GENERIC_MSI_IRQ) += msi.o
> Index: tip/kernel/irq/msi.c
> ===================================================================
> --- /dev/null
> +++ tip/kernel/irq/msi.c
> @@ -0,0 +1,119 @@
> +/*
> + * linux/kernel/irq/msi.c
> + *
> + * Copyright (C) 2014 Intel Corp.
> + * Author: Jiang Liu <jiang.liu@linux.intel.com>
> + *
> + * This file is licensed under GPLv2.
> + *
> + * This file contains common code to support Message Signalled Interrupt for
> + * PCI compatible and non PCI compatible devices.
> + */
> +#include <linux/irq.h>
> +#include <linux/irqdomain.h>
> +#include <linux/msi.h>
> +
> +#ifdef CONFIG_GENERIC_MSI_IRQ_DOMAIN
> +int msi_domain_set_affinity(struct irq_data *irq_data,
> +			    const struct cpumask *mask, bool force)
> +{
> +	struct irq_data *parent = irq_data->parent_data;
> +	struct msi_msg msg;
> +	int ret;
> +
> +	ret = parent->chip->irq_set_affinity(parent, mask, force);
> +	if (ret >= 0 && ret != IRQ_SET_MASK_OK_DONE) {
> +		BUG_ON(irq_chip_compose_msi_msg(irq_data, &msg));
> +		irq_chip_write_msi_msg(irq_data, &msg);
> +	}
> +
> +	return ret;
> +}
> +
> +static void msi_domain_activate(struct irq_domain *domain,
> +				struct irq_data *irq_data)
> +{
> +	struct msi_msg msg;
> +
> +	BUG_ON(irq_chip_compose_msi_msg(irq_data, &msg));
> +	irq_chip_write_msi_msg(irq_data, &msg);
> +}
> +
> +static void msi_domain_deactivate(struct irq_domain *domain,
> +				  struct irq_data *irq_data)
> +{
> +	struct msi_msg msg;
> +
> +	memset(&msg, 0, sizeof(msg));
> +	irq_chip_write_msi_msg(irq_data, &msg);
> +}
> +
> +static int msi_domain_alloc(struct irq_domain *domain, unsigned int virq,
> +			    unsigned int nr_irqs, void *arg)
> +{
> +	struct msi_domain_info *info = domain->host_data;
> +	struct msi_domain_ops *ops = info->ops;
> +	irq_hw_number_t hwirq = ops->get_hwirq(info, arg);
> +	int i, ret;
> +
> +	if (irq_find_mapping(domain, hwirq) > 0)
> +		return -EEXIST;

I think we need to check range here.
I fixed this by applying a patch showed in the bottom.

> +
> +	ret = irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, arg);
> +	if (ret < 0)
> +		return ret;
> +
> +	for (i = 0; i < nr_irqs; i++) {
> +		ret = ops->msi_init(domain, info, virq + i, hwirq + i, arg);
> +		if (ret < 0) {
> +			if (ops->msi_free) {
> +				for (i--; i > 0; i--)

while (i--) ?

Regards,
	Abel

> +					ops->msi_free(domain, info, virq + i);
> +			}
> +			irq_domain_free_irqs_top(domain, virq, nr_irqs);
> +			return ret;
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static void msi_domain_free(struct irq_domain *domain, unsigned int virq,
> +			    unsigned int nr_irqs)
> +{
> +	struct msi_domain_info *info = domain->host_data;
> +	int i;
> +
> +	if (info->ops->msi_free) {
> +		for (i = 0; i < nr_irqs; i++)
> +			info->ops->msi_free(domain, info, virq + i);
> +	}
> +	irq_domain_free_irqs_top(domain, virq, nr_irqs);
> +}
> +
> +static struct irq_domain_ops msi_domain_ops = {
> +	.alloc		= msi_domain_alloc,
> +	.free		= msi_domain_free,
> +	.activate	= msi_domain_activate,
> +	.deactivate	= msi_domain_deactivate,
> +};
> +
> +struct irq_domain *msi_create_irq_domain(struct device_node *of_node,
> +					 struct msi_domain_info *info,
> +					 struct irq_domain *parent)
> +{
> +	struct irq_domain *domain;
> +
> +	domain = irq_domain_add_tree(of_node, &msi_domain_ops, info);
> +	if (domain)
> +		domain->parent = parent;
> +
> +	return domain;
> +}
> +
> +struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain)
> +{
> +	return (struct msi_domain_info *)domain->host_data;
> +}
> +
> +#endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */
> 
> 

diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
index cc18182..6d1be00 100644
--- a/kernel/irq/irqdomain.c
+++ b/kernel/irq/irqdomain.c
@@ -568,6 +568,34 @@ unsigned int irq_find_mapping(struct irq_domain *domain,
 }
 EXPORT_SYMBOL_GPL(irq_find_mapping);

+/**
+ * irq_find_mapping_many() - Find a range of linux irqs from a range of hwirqs.
+ * @domain:	domain owning these hardware interrupts
+ * @hwirq:	start hardware irq number in @domain space
+ * @count:	number of interrupts to find
+ *
+ * Returns the first linux irq number on success, or 0 when not found, or error.
+ */
+int irq_find_mapping_many(struct irq_domain *domain, irq_hw_number_t hwirq,
+			  int count)
+{
+	unsigned int from;
+	int i;
+
+	for (i = from = 0; i < count; i++) {
+		if (!from) {
+			from = irq_find_mapping(domain, hwirq + i);
+			if (from && i)
+				return -1;
+		} else if ((from + i) != irq_find_mapping(domain, hwirq + i)) {
+			return -1;
+		}
+	}
+
+	return from;
+}
+EXPORT_SYMBOL_GPL(irq_find_mapping_many);
+
 #ifdef CONFIG_IRQ_DOMAIN_DEBUG
 static int virq_debug_show(struct seq_file *m, void *private)
 {

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Jiang Liu]

On 2014/11/18 19:47, Yun Wu (Abel) wrote:
> On 2014/11/18 18:03, Thomas Gleixner wrote:
> 
>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>
>>> Hi Thomas, Jiang,
>>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>
>>>> From: Jiang Liu <jiang.liu@linux.intel.com>
>>> [...]
>>>> +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
>>>> +			 irq_hw_number_t hwirq, struct irq_chip *chip,
>>>> +			 void *chip_data, irq_flow_handler_t handler,
>>>> +			 void *handler_data, const char *handler_name)
>>>> +{
>>>> +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
>>>> +	__irq_set_handler(virq, handler, 0, handler_name);
>>>> +	irq_set_handler_data(virq, handler_data);
>>>> +}
>>>
>>> When stacked domain enabled, there will be a semantic shift to the linux interrupt
>>> identifiers. The @virq now delivers much more than before.
>>> More specifically, now we need both @virq and @domain, rather than only @irq, to
>>> determine which irq_data we want to configure. And once we configure @irq without
>>> providing the exact domain, it means we are configuring all the domains related to
>>> that @irq. So I think this routine just messed all things up.
>>
>> You can mess up anything by using an interface in the wrong way. Open
>> coding will not make that harder.
>>
> 
> But what's the correct way to use this interface?
Hi Yun,
	It's to be used by interrupt controller drivers to implement
hierarchy irqdomains.
Regards!
Gerry

> 
> Thanks,
> 	Abel
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [patch 04/16] genirq: Introduce irq_chip.irq_compose_msi_msg() to support stacked irqchip

[Jiang Liu]

On 2014/11/18 19:47, Yun Wu (Abel) wrote:
> On 2014/11/18 18:02, Thomas Gleixner wrote:
> 
>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>>> +int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
>>>> +{
>>>> +	struct irq_data *pos = NULL;
>>>> +
>>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>>> +	for (; data; data = data->parent_data)
>>>> +#endif
>>>> +		if (data->chip && data->chip->irq_compose_msi_msg)
>>>> +			pos = data;
>>>> +	if (!pos)
>>>> +		return -ENOSYS;
>>>> +
>>>> +	pos->chip->irq_compose_msi_msg(pos, msg);
>>>> +
>>>> +	return 0;
>>>> +}
>>>
>>> Adding message composing routine to struct irq_chip is OK to me, and it should
>>> be because it is interrupt controllers' duty to compose messages (so that they
>>> can parse the messages correctly without any pre-defined rules that endpoint
>>> devices absolutely need not to know).
>>> However a problem comes out when deciding which parameters should be passed to
>>> this routine. A message can associate with multiple interrupts, which makes me
>>> think composing messages for each interrupt is not that appropriate. And we
>>> can take a look at the new routine irq_chip_compose_msi_msg(). It is called by
>>> msi_domain_activate() which will be called by irq_domain_activate_irq() in
>>> irq_startup() for each interrupt descriptor, result in composing a message for
>>> each interrupt, right? (Unless requiring a judge on the parameter @data when
>>> implementing the irq_compose_msi_msg() callback that only compose message for
>>> the first entry of that message. But I really don't like that...)
>>
>> No, that's not correct. You are looking at some random stale version
>> of this. The current state of affairs is in 
>>
>>    git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain
>>
>> See also https://lkml.org/lkml/2014/11/17/764
>>
>> In activate we write the message, which is the right point to do so.
>>
> 
> I checked the current state, it seems to be the same.
> Yes, the decision of postponing the actual hardware programming to the point
> where the interrupt actually gets used is right, but here above I was talking
> another thing.
> As I mentioned, a message can associate with multiple interrupts. Enabling
> any of them will call irq_startup(). So if we don't want to compose or write
> messages repeatedly, we'd better require performing some checks before
> activating the interrupts.
Hi Yun,
	Seems you are talking about the case of multiple MSI support.
Yes, we have special treatment for multiple MSI, which only writes PCI
MSI registers when starting up the first MSI interrupt.
void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg
*msg)
{
        struct msi_desc *desc = irq_data->msi_desc;

        /*
         * For MSI-X desc->irq is always equal to irq_data->irq. For
         * MSI only the first interrupt of MULTI MSI passes the test.
         */
        if (desc->irq == irq_data->irq)
                __pci_write_msi_msg(desc, msg);
}
Regards!
Gerry

> 
> Thanks,
> 	Abel
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [patch 09/16] genirq: Add generic msi irq domain support

[Jiang Liu]

On 2014/11/18 20:07, Yun Wu (Abel) wrote:
> On 2014/11/12 21:43, Thomas Gleixner wrote:
> 
>> From: Jiang Liu <jiang.liu@linux.intel.com>
<snip>
>> +static int msi_domain_alloc(struct irq_domain *domain, unsigned int virq,
>> +			    unsigned int nr_irqs, void *arg)
>> +{
>> +	struct msi_domain_info *info = domain->host_data;
>> +	struct msi_domain_ops *ops = info->ops;
>> +	irq_hw_number_t hwirq = ops->get_hwirq(info, arg);
>> +	int i, ret;
>> +
>> +	if (irq_find_mapping(domain, hwirq) > 0)
>> +		return -EEXIST;
> 
> I think we need to check range here.
> I fixed this by applying a patch showed in the bottom.
Hi Yun,
	Yes, we have some assumptions here about the way the interface
will be used for. Will improve it in future patches.
Regards!
Gerry
> 
>> +
>> +	ret = irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, arg);
>> +	if (ret < 0)
>> +		return ret;
>> +
>> +	for (i = 0; i < nr_irqs; i++) {
>> +		ret = ops->msi_init(domain, info, virq + i, hwirq + i, arg);
>> +		if (ret < 0) {
>> +			if (ops->msi_free) {
>> +				for (i--; i > 0; i--)
> 
> while (i--) ?
> 
> Regards,
> 	Abel
> 
>> +					ops->msi_free(domain, info, virq + i);
>> +			}
>> +			irq_domain_free_irqs_top(domain, virq, nr_irqs);
>> +			return ret;
>> +		}
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> +static void msi_domain_free(struct irq_domain *domain, unsigned int virq,
>> +			    unsigned int nr_irqs)
>> +{
>> +	struct msi_domain_info *info = domain->host_data;
>> +	int i;
>> +
>> +	if (info->ops->msi_free) {
>> +		for (i = 0; i < nr_irqs; i++)
>> +			info->ops->msi_free(domain, info, virq + i);
>> +	}
>> +	irq_domain_free_irqs_top(domain, virq, nr_irqs);
>> +}
>> +
>> +static struct irq_domain_ops msi_domain_ops = {
>> +	.alloc		= msi_domain_alloc,
>> +	.free		= msi_domain_free,
>> +	.activate	= msi_domain_activate,
>> +	.deactivate	= msi_domain_deactivate,
>> +};
>> +
>> +struct irq_domain *msi_create_irq_domain(struct device_node *of_node,
>> +					 struct msi_domain_info *info,
>> +					 struct irq_domain *parent)
>> +{
>> +	struct irq_domain *domain;
>> +
>> +	domain = irq_domain_add_tree(of_node, &msi_domain_ops, info);
>> +	if (domain)
>> +		domain->parent = parent;
>> +
>> +	return domain;
>> +}
>> +
>> +struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain)
>> +{
>> +	return (struct msi_domain_info *)domain->host_data;
>> +}
>> +
>> +#endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */
>>
>>
> 
> diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
> index cc18182..6d1be00 100644
> --- a/kernel/irq/irqdomain.c
> +++ b/kernel/irq/irqdomain.c
> @@ -568,6 +568,34 @@ unsigned int irq_find_mapping(struct irq_domain *domain,
>  }
>  EXPORT_SYMBOL_GPL(irq_find_mapping);
> 
> +/**
> + * irq_find_mapping_many() - Find a range of linux irqs from a range of hwirqs.
> + * @domain:	domain owning these hardware interrupts
> + * @hwirq:	start hardware irq number in @domain space
> + * @count:	number of interrupts to find
> + *
> + * Returns the first linux irq number on success, or 0 when not found, or error.
> + */
> +int irq_find_mapping_many(struct irq_domain *domain, irq_hw_number_t hwirq,
> +			  int count)
> +{
> +	unsigned int from;
> +	int i;
> +
> +	for (i = from = 0; i < count; i++) {
> +		if (!from) {
> +			from = irq_find_mapping(domain, hwirq + i);
> +			if (from && i)
> +				return -1;
> +		} else if ((from + i) != irq_find_mapping(domain, hwirq + i)) {
> +			return -1;
> +		}
> +	}
> +
> +	return from;
> +}
> +EXPORT_SYMBOL_GPL(irq_find_mapping_many);
> +
>  #ifdef CONFIG_IRQ_DOMAIN_DEBUG
>  static int virq_debug_show(struct seq_file *m, void *private)
>  {
> 
> 
> 

Re: [patch 04/16] genirq: Introduce irq_chip.irq_compose_msi_msg() to support stacked irqchip

[Yun Wu (Abel)]

On 2014/11/18 20:43, Jiang Liu wrote:

> On 2014/11/18 19:47, Yun Wu (Abel) wrote:
>> On 2014/11/18 18:02, Thomas Gleixner wrote:
>>
>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>>>> +int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
>>>>> +{
>>>>> +	struct irq_data *pos = NULL;
>>>>> +
>>>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>>>> +	for (; data; data = data->parent_data)
>>>>> +#endif
>>>>> +		if (data->chip && data->chip->irq_compose_msi_msg)
>>>>> +			pos = data;
>>>>> +	if (!pos)
>>>>> +		return -ENOSYS;
>>>>> +
>>>>> +	pos->chip->irq_compose_msi_msg(pos, msg);
>>>>> +
>>>>> +	return 0;
>>>>> +}
>>>>
>>>> Adding message composing routine to struct irq_chip is OK to me, and it should
>>>> be because it is interrupt controllers' duty to compose messages (so that they
>>>> can parse the messages correctly without any pre-defined rules that endpoint
>>>> devices absolutely need not to know).
>>>> However a problem comes out when deciding which parameters should be passed to
>>>> this routine. A message can associate with multiple interrupts, which makes me
>>>> think composing messages for each interrupt is not that appropriate. And we
>>>> can take a look at the new routine irq_chip_compose_msi_msg(). It is called by
>>>> msi_domain_activate() which will be called by irq_domain_activate_irq() in
>>>> irq_startup() for each interrupt descriptor, result in composing a message for
>>>> each interrupt, right? (Unless requiring a judge on the parameter @data when
>>>> implementing the irq_compose_msi_msg() callback that only compose message for
>>>> the first entry of that message. But I really don't like that...)
>>>
>>> No, that's not correct. You are looking at some random stale version
>>> of this. The current state of affairs is in 
>>>
>>>    git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain
>>>
>>> See also https://lkml.org/lkml/2014/11/17/764
>>>
>>> In activate we write the message, which is the right point to do so.
>>>
>>
>> I checked the current state, it seems to be the same.
>> Yes, the decision of postponing the actual hardware programming to the point
>> where the interrupt actually gets used is right, but here above I was talking
>> another thing.
>> As I mentioned, a message can associate with multiple interrupts. Enabling
>> any of them will call irq_startup(). So if we don't want to compose or write
>> messages repeatedly, we'd better require performing some checks before
>> activating the interrupts.
> Hi Yun,
> 	Seems you are talking about the case of multiple MSI support.
> Yes, we have special treatment for multiple MSI, which only writes PCI
> MSI registers when starting up the first MSI interrupt.
> void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg
> *msg)
> {
>         struct msi_desc *desc = irq_data->msi_desc;
> 
>         /*
>          * For MSI-X desc->irq is always equal to irq_data->irq. For
>          * MSI only the first interrupt of MULTI MSI passes the test.
>          */
>         if (desc->irq == irq_data->irq)
>                 __pci_write_msi_msg(desc, msg);
> }


Yes, I picked the case of multiple MSI support.
The check should also be performed when composing messages. That's why
I don't like its parameters. The @data only indicates one interrupt,
while I prefer doing compose/write in the unit of message descriptor.

Thanks,
	Abel

Re: [patch 04/16] genirq: Introduce irq_chip.irq_compose_msi_msg() to support stacked irqchip

[Jiang Liu]

On 2014/11/18 21:16, Yun Wu (Abel) wrote:
> On 2014/11/18 20:43, Jiang Liu wrote:
> 
>> On 2014/11/18 19:47, Yun Wu (Abel) wrote:
>>> On 2014/11/18 18:02, Thomas Gleixner wrote:
>>>
>>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>>>>> +int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
>>>>>> +{
>>>>>> +	struct irq_data *pos = NULL;
>>>>>> +
>>>>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>>>>> +	for (; data; data = data->parent_data)
>>>>>> +#endif
>>>>>> +		if (data->chip && data->chip->irq_compose_msi_msg)
>>>>>> +			pos = data;
>>>>>> +	if (!pos)
>>>>>> +		return -ENOSYS;
>>>>>> +
>>>>>> +	pos->chip->irq_compose_msi_msg(pos, msg);
>>>>>> +
>>>>>> +	return 0;
>>>>>> +}
>>>>>
>>>>> Adding message composing routine to struct irq_chip is OK to me, and it should
>>>>> be because it is interrupt controllers' duty to compose messages (so that they
>>>>> can parse the messages correctly without any pre-defined rules that endpoint
>>>>> devices absolutely need not to know).
>>>>> However a problem comes out when deciding which parameters should be passed to
>>>>> this routine. A message can associate with multiple interrupts, which makes me
>>>>> think composing messages for each interrupt is not that appropriate. And we
>>>>> can take a look at the new routine irq_chip_compose_msi_msg(). It is called by
>>>>> msi_domain_activate() which will be called by irq_domain_activate_irq() in
>>>>> irq_startup() for each interrupt descriptor, result in composing a message for
>>>>> each interrupt, right? (Unless requiring a judge on the parameter @data when
>>>>> implementing the irq_compose_msi_msg() callback that only compose message for
>>>>> the first entry of that message. But I really don't like that...)
>>>>
>>>> No, that's not correct. You are looking at some random stale version
>>>> of this. The current state of affairs is in 
>>>>
>>>>    git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain
>>>>
>>>> See also https://lkml.org/lkml/2014/11/17/764
>>>>
>>>> In activate we write the message, which is the right point to do so.
>>>>
>>>
>>> I checked the current state, it seems to be the same.
>>> Yes, the decision of postponing the actual hardware programming to the point
>>> where the interrupt actually gets used is right, but here above I was talking
>>> another thing.
>>> As I mentioned, a message can associate with multiple interrupts. Enabling
>>> any of them will call irq_startup(). So if we don't want to compose or write
>>> messages repeatedly, we'd better require performing some checks before
>>> activating the interrupts.
>> Hi Yun,
>> 	Seems you are talking about the case of multiple MSI support.
>> Yes, we have special treatment for multiple MSI, which only writes PCI
>> MSI registers when starting up the first MSI interrupt.
>> void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg
>> *msg)
>> {
>>         struct msi_desc *desc = irq_data->msi_desc;
>>
>>         /*
>>          * For MSI-X desc->irq is always equal to irq_data->irq. For
>>          * MSI only the first interrupt of MULTI MSI passes the test.
>>          */
>>         if (desc->irq == irq_data->irq)
>>                 __pci_write_msi_msg(desc, msg);
>> }
> 
> 
> Yes, I picked the case of multiple MSI support.
> The check should also be performed when composing messages. That's why
> I don't like its parameters. The @data only indicates one interrupt,
> while I prefer doing compose/write in the unit of message descriptor.
Hi Yun,
	The common abstraction is that every message interrupt could be
controlled independently, so have compose_msi_msg()/write_msi_msg() per
interrupt. MSI is abstracted as an special message signaled interrupt
with hardware limitation where multiple interrupts sharing the same
hardware registers. So we filter in pci_msi_domain_write_msg(). On the
other handle, the generic MSI framework caches msi_msg in msi_desc,
so we don't filter compose_msi_msg().
Regards!
Gerry

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Yun Wu (Abel)]

On 2014/11/18 20:38, Jiang Liu wrote:

> On 2014/11/18 19:47, Yun Wu (Abel) wrote:
>> On 2014/11/18 18:03, Thomas Gleixner wrote:
>>
>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>
>>>> Hi Thomas, Jiang,
>>>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>>
>>>>> From: Jiang Liu <jiang.liu@linux.intel.com>
>>>> [...]
>>>>> +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
>>>>> +			 irq_hw_number_t hwirq, struct irq_chip *chip,
>>>>> +			 void *chip_data, irq_flow_handler_t handler,
>>>>> +			 void *handler_data, const char *handler_name)
>>>>> +{
>>>>> +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
>>>>> +	__irq_set_handler(virq, handler, 0, handler_name);
>>>>> +	irq_set_handler_data(virq, handler_data);
>>>>> +}
>>>>
>>>> When stacked domain enabled, there will be a semantic shift to the linux interrupt
>>>> identifiers. The @virq now delivers much more than before.
>>>> More specifically, now we need both @virq and @domain, rather than only @irq, to
>>>> determine which irq_data we want to configure. And once we configure @irq without
>>>> providing the exact domain, it means we are configuring all the domains related to
>>>> that @irq. So I think this routine just messed all things up.
>>>
>>> You can mess up anything by using an interface in the wrong way. Open
>>> coding will not make that harder.
>>>
>>
>> But what's the correct way to use this interface?
>
> 	It's to be used by interrupt controller drivers to implement
> hierarchy irqdomains.
> 

Each time an interrupt domain calls this, the previous (@handler, @handler_name,
@handler_data) will be overrode. It's because the routines __irq_set_handler()
and irq_set_handler_data() only configure top level (without Marc's fix which is
not ideal). Is this what we really want to see?

Thanks,
	Abel

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

On 2014/11/18 18:19, Thomas Gleixner wrote:

> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>  struct irq_chip {
>>> @@ -359,6 +360,7 @@ struct irq_chip {
>>>  	void		(*irq_release_resources)(struct irq_data *data);
>>>  
>>>  	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>> +	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>
>> Hmm... It's really weird.
>> I don't think it's the interrupt controllers' responsibility to write messages
>> for all the endpoint devices since the methods of configuring message registers
>> may different between these devices. And theoretically, the endpoint devices
>> themselves should take the responsibility to configure their message registers.
>> To say the least, the write_msg callback here still need to call some certain
>> interfaces provided by the corresponding device.
>>
>> There would be lots of ARM new devices capable of sending message
>> based interrupts to interrupt controllers, does all the drivers of
>> the devices need to expose a write_msg callback to interrupt
>> controllers?
> 
> Well, writing the message _IS_ part of the interrupt controller.
> 
> So in order to enable non PCI based MSI we want to have generic
> infrastructure with minimal per device/device class callbacks and of
> course you need to provide that callback for your special device.
> 
> We already have non PCI based MSI controllers in x86 today and we need
> to handle the whole stuff with tons of copied coded extra for each of
> those. So consolidating it into common infrastructure allows us to get
> rid of the pointless copied code and reduce the per device effort to
> the relevant hardware specific callbacks. irq_write_msi_msg being one
> of those.
> 

At least, we have the same goal.
I will illustrate my thoughts by an example.
The current code is something like:

Device A
========
void A_write_msg() { ... }

Group B
(a group of devices behave same on writing messages, i.e. PCI)
=======
void B_write_msg() { ... }

Controller
==========
irq_chip.irq_write_msi_msg () {
	if (A)
		A_write_msg();
	if (B)
		B_write_msg();
}

It's horrible when new devices come out, since we need to modify the
controller part for each new device.
What I suggested is:

MSI Core
========
struct msi_ops { .write_msg, };
struct msi_desc { .msi_ops, };

write_msg() {
	X = get_dev();
	irq_chip.compose_msg(X);	// IRQ chips' responsibility
	X_msi_ops.write_msg();		// nothing to do with IRQ chips
}

Device A
========
void A_write_msg() { ... }
A_msi_ops.write_msg = A_write_msg;

Group B
=======
void B_write_msg() { ... }
B_msi_ops.write_msg = B_write_msg;

Please correct me if I misunderstood anything.

Thanks,
	Abel

Re: [patch 07/16] genirq: Introduce helper irq_domain_set_info() to reduce duplicated code

[Jiang Liu]

On 2014/11/18 21:28, Yun Wu (Abel) wrote:
> On 2014/11/18 20:38, Jiang Liu wrote:
> 
>> On 2014/11/18 19:47, Yun Wu (Abel) wrote:
>>> On 2014/11/18 18:03, Thomas Gleixner wrote:
>>>
>>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>>
>>>>> Hi Thomas, Jiang,
>>>>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>>>
>>>>>> From: Jiang Liu <jiang.liu@linux.intel.com>
>>>>> [...]
>>>>>> +void irq_domain_set_info(struct irq_domain *domain, unsigned int virq,
>>>>>> +			 irq_hw_number_t hwirq, struct irq_chip *chip,
>>>>>> +			 void *chip_data, irq_flow_handler_t handler,
>>>>>> +			 void *handler_data, const char *handler_name)
>>>>>> +{
>>>>>> +	irq_domain_set_hwirq_and_chip(domain, virq, hwirq, chip, chip_data);
>>>>>> +	__irq_set_handler(virq, handler, 0, handler_name);
>>>>>> +	irq_set_handler_data(virq, handler_data);
>>>>>> +}
>>>>>
>>>>> When stacked domain enabled, there will be a semantic shift to the linux interrupt
>>>>> identifiers. The @virq now delivers much more than before.
>>>>> More specifically, now we need both @virq and @domain, rather than only @irq, to
>>>>> determine which irq_data we want to configure. And once we configure @irq without
>>>>> providing the exact domain, it means we are configuring all the domains related to
>>>>> that @irq. So I think this routine just messed all things up.
>>>>
>>>> You can mess up anything by using an interface in the wrong way. Open
>>>> coding will not make that harder.
>>>>
>>>
>>> But what's the correct way to use this interface?
>>
>> 	It's to be used by interrupt controller drivers to implement
>> hierarchy irqdomains.
>>
> 
> Each time an interrupt domain calls this, the previous (@handler, @handler_name,
> @handler_data) will be overrode. It's because the routines __irq_set_handler()
> and irq_set_handler_data() only configure top level (without Marc's fix which is
> not ideal). Is this what we really want to see?
Hi Yun,
	There are different interfaces for different purposes.
irq_domain_set_hwirq_and_chip() sets hwirq, irq_chip and irq_chip_data.
irq_domain_set_info() sets handler and handler_data in addition to
irq_domain_set_hwirq_and_chip().
	Which irqdomain to call irq_domain_set_info() to set irq_handler
is determined by the interrupt hierarchy. And at least on irqdomain in
the hierarchy must set flow handler for the irq.
Thanks!
Gerry

> 
> Thanks,
> 	Abel
> 

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Jiang Liu]

On 2014/11/18 21:33, Yun Wu (Abel) wrote:
> On 2014/11/18 18:19, Thomas Gleixner wrote:
> 
>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>>  struct irq_chip {
>>>> @@ -359,6 +360,7 @@ struct irq_chip {
>>>>  	void		(*irq_release_resources)(struct irq_data *data);
>>>>  
>>>>  	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>> +	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>
>>> Hmm... It's really weird.
>>> I don't think it's the interrupt controllers' responsibility to write messages
>>> for all the endpoint devices since the methods of configuring message registers
>>> may different between these devices. And theoretically, the endpoint devices
>>> themselves should take the responsibility to configure their message registers.
>>> To say the least, the write_msg callback here still need to call some certain
>>> interfaces provided by the corresponding device.
>>>
>>> There would be lots of ARM new devices capable of sending message
>>> based interrupts to interrupt controllers, does all the drivers of
>>> the devices need to expose a write_msg callback to interrupt
>>> controllers?
>>
>> Well, writing the message _IS_ part of the interrupt controller.
>>
>> So in order to enable non PCI based MSI we want to have generic
>> infrastructure with minimal per device/device class callbacks and of
>> course you need to provide that callback for your special device.
>>
>> We already have non PCI based MSI controllers in x86 today and we need
>> to handle the whole stuff with tons of copied coded extra for each of
>> those. So consolidating it into common infrastructure allows us to get
>> rid of the pointless copied code and reduce the per device effort to
>> the relevant hardware specific callbacks. irq_write_msi_msg being one
>> of those.
>>
> 
> At least, we have the same goal.
> I will illustrate my thoughts by an example.
> The current code is something like:
> 
> Device A
> ========
> void A_write_msg() { ... }
> 
> Group B
> (a group of devices behave same on writing messages, i.e. PCI)
> =======
> void B_write_msg() { ... }
> 
> Controller
> ==========
> irq_chip.irq_write_msi_msg () {
> 	if (A)
> 		A_write_msg();
> 	if (B)
> 		B_write_msg();
> }
> 
> It's horrible when new devices come out, since we need to modify the
> controller part for each new device.
> What I suggested is:
> 
> MSI Core
> ========
> struct msi_ops { .write_msg, };
> struct msi_desc { .msi_ops, };
> 
> write_msg() {
> 	X = get_dev();
> 	irq_chip.compose_msg(X);	// IRQ chips' responsibility
> 	X_msi_ops.write_msg();		// nothing to do with IRQ chips
> }
> 
> Device A
> ========
> void A_write_msg() { ... }
> A_msi_ops.write_msg = A_write_msg;
> 
> Group B
> =======
> void B_write_msg() { ... }
> B_msi_ops.write_msg = B_write_msg;
> 
> Please correct me if I misunderstood anything.
Hi Yun,
	We provide an irq_chip for each type of interrupt controller
instead of devices. For the example mentioned above, if device A
and Group B has different interrupt controllers, we just need to
implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
to suitable callbacks.
	The framework already achieves what you you want:)
Regards!
Gerry
> 
> Thanks,
> 	Abel
> 

Re: [patch 04/16] genirq: Introduce irq_chip.irq_compose_msi_msg() to support stacked irqchip

[Yun Wu (Abel)]

On 2014/11/18 21:25, Jiang Liu wrote:

> On 2014/11/18 21:16, Yun Wu (Abel) wrote:
>> On 2014/11/18 20:43, Jiang Liu wrote:
>>
>>> On 2014/11/18 19:47, Yun Wu (Abel) wrote:
>>>> On 2014/11/18 18:02, Thomas Gleixner wrote:
>>>>
>>>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>>>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>>>>>> +int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
>>>>>>> +{
>>>>>>> +	struct irq_data *pos = NULL;
>>>>>>> +
>>>>>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>>>>>> +	for (; data; data = data->parent_data)
>>>>>>> +#endif
>>>>>>> +		if (data->chip && data->chip->irq_compose_msi_msg)
>>>>>>> +			pos = data;
>>>>>>> +	if (!pos)
>>>>>>> +		return -ENOSYS;
>>>>>>> +
>>>>>>> +	pos->chip->irq_compose_msi_msg(pos, msg);
>>>>>>> +
>>>>>>> +	return 0;
>>>>>>> +}
>>>>>>
>>>>>> Adding message composing routine to struct irq_chip is OK to me, and it should
>>>>>> be because it is interrupt controllers' duty to compose messages (so that they
>>>>>> can parse the messages correctly without any pre-defined rules that endpoint
>>>>>> devices absolutely need not to know).
>>>>>> However a problem comes out when deciding which parameters should be passed to
>>>>>> this routine. A message can associate with multiple interrupts, which makes me
>>>>>> think composing messages for each interrupt is not that appropriate. And we
>>>>>> can take a look at the new routine irq_chip_compose_msi_msg(). It is called by
>>>>>> msi_domain_activate() which will be called by irq_domain_activate_irq() in
>>>>>> irq_startup() for each interrupt descriptor, result in composing a message for
>>>>>> each interrupt, right? (Unless requiring a judge on the parameter @data when
>>>>>> implementing the irq_compose_msi_msg() callback that only compose message for
>>>>>> the first entry of that message. But I really don't like that...)
>>>>>
>>>>> No, that's not correct. You are looking at some random stale version
>>>>> of this. The current state of affairs is in 
>>>>>
>>>>>    git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain
>>>>>
>>>>> See also https://lkml.org/lkml/2014/11/17/764
>>>>>
>>>>> In activate we write the message, which is the right point to do so.
>>>>>
>>>>
>>>> I checked the current state, it seems to be the same.
>>>> Yes, the decision of postponing the actual hardware programming to the point
>>>> where the interrupt actually gets used is right, but here above I was talking
>>>> another thing.
>>>> As I mentioned, a message can associate with multiple interrupts. Enabling
>>>> any of them will call irq_startup(). So if we don't want to compose or write
>>>> messages repeatedly, we'd better require performing some checks before
>>>> activating the interrupts.
>>> Hi Yun,
>>> 	Seems you are talking about the case of multiple MSI support.
>>> Yes, we have special treatment for multiple MSI, which only writes PCI
>>> MSI registers when starting up the first MSI interrupt.
>>> void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg
>>> *msg)
>>> {
>>>         struct msi_desc *desc = irq_data->msi_desc;
>>>
>>>         /*
>>>          * For MSI-X desc->irq is always equal to irq_data->irq. For
>>>          * MSI only the first interrupt of MULTI MSI passes the test.
>>>          */
>>>         if (desc->irq == irq_data->irq)
>>>                 __pci_write_msi_msg(desc, msg);
>>> }
>>
>>
>> Yes, I picked the case of multiple MSI support.
>> The check should also be performed when composing messages. That's why
>> I don't like its parameters. The @data only indicates one interrupt,
>> while I prefer doing compose/write in the unit of message descriptor.
> Hi Yun,
> 	The common abstraction is that every message interrupt could be
> controlled independently, so have compose_msi_msg()/write_msi_msg() per
> interrupt. MSI is abstracted as an special message signaled interrupt
> with hardware limitation where multiple interrupts sharing the same
> hardware registers. So we filter in pci_msi_domain_write_msg(). On the
> other handle, the generic MSI framework caches msi_msg in msi_desc,
> so we don't filter compose_msi_msg().
> 

It's true that every message interrupt could be controlled independently,
I mean, by enable/disable/mask/unmask. But the message data & address are
shared among the interrupts of that message.
Despite the detailed hardware implementation, MSI and MSI-X are the same
thing in software view, that is a message related with several consecutive
interrupts. And the core MSI infrastructure you want to build should not
be based on any hardware assumptions.

Thanks,
	Abel

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Jiang Liu]

On 2014/11/18 21:33, Yun Wu (Abel) wrote:
> On 2014/11/18 18:19, Thomas Gleixner wrote:
> 
>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>>  struct irq_chip {
>>>> @@ -359,6 +360,7 @@ struct irq_chip {
>>>>  	void		(*irq_release_resources)(struct irq_data *data);
>>>>  
>>>>  	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>> +	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>
>>> Hmm... It's really weird.
>>> I don't think it's the interrupt controllers' responsibility to write messages
>>> for all the endpoint devices since the methods of configuring message registers
>>> may different between these devices. And theoretically, the endpoint devices
>>> themselves should take the responsibility to configure their message registers.
>>> To say the least, the write_msg callback here still need to call some certain
>>> interfaces provided by the corresponding device.
>>>
>>> There would be lots of ARM new devices capable of sending message
>>> based interrupts to interrupt controllers, does all the drivers of
>>> the devices need to expose a write_msg callback to interrupt
>>> controllers?
>>
>> Well, writing the message _IS_ part of the interrupt controller.
>>
>> So in order to enable non PCI based MSI we want to have generic
>> infrastructure with minimal per device/device class callbacks and of
>> course you need to provide that callback for your special device.
>>
>> We already have non PCI based MSI controllers in x86 today and we need
>> to handle the whole stuff with tons of copied coded extra for each of
>> those. So consolidating it into common infrastructure allows us to get
>> rid of the pointless copied code and reduce the per device effort to
>> the relevant hardware specific callbacks. irq_write_msi_msg being one
>> of those.
>>
> 
> At least, we have the same goal.
> I will illustrate my thoughts by an example.
> The current code is something like:
> 
> Device A
> ========
> void A_write_msg() { ... }
> 
> Group B
> (a group of devices behave same on writing messages, i.e. PCI)
> =======
> void B_write_msg() { ... }
> 
> Controller
> ==========
> irq_chip.irq_write_msi_msg () {
> 	if (A)
> 		A_write_msg();
> 	if (B)
> 		B_write_msg();
> }
> 
> It's horrible when new devices come out, since we need to modify the
> controller part for each new device.
> What I suggested is:
> 
> MSI Core
> ========
> struct msi_ops { .write_msg, };
> struct msi_desc { .msi_ops, };
> 
> write_msg() {
> 	X = get_dev();
> 	irq_chip.compose_msg(X);	// IRQ chips' responsibility
> 	X_msi_ops.write_msg();		// nothing to do with IRQ chips
> }
> 
> Device A
> ========
> void A_write_msg() { ... }
> A_msi_ops.write_msg = A_write_msg;
> 
> Group B
> =======
> void B_write_msg() { ... }
> B_msi_ops.write_msg = B_write_msg;
> 
> Please correct me if I misunderstood anything.
Please take a look at following file, which uses the common MSI
framework to support PCI, DMAR and HPET interrupt on x86.
https://github.com/jiangliu/linux/blob/irqdomain/p2v7/arch/x86/kernel/apic/msi.c
Regards!
Gerry

> 
> Thanks,
> 	Abel
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

On 2014/11/18 21:43, Jiang Liu wrote:

> On 2014/11/18 21:33, Yun Wu (Abel) wrote:
>> On 2014/11/18 18:19, Thomas Gleixner wrote:
>>
>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>>>  struct irq_chip {
>>>>> @@ -359,6 +360,7 @@ struct irq_chip {
>>>>>  	void		(*irq_release_resources)(struct irq_data *data);
>>>>>  
>>>>>  	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>> +	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>
>>>> Hmm... It's really weird.
>>>> I don't think it's the interrupt controllers' responsibility to write messages
>>>> for all the endpoint devices since the methods of configuring message registers
>>>> may different between these devices. And theoretically, the endpoint devices
>>>> themselves should take the responsibility to configure their message registers.
>>>> To say the least, the write_msg callback here still need to call some certain
>>>> interfaces provided by the corresponding device.
>>>>
>>>> There would be lots of ARM new devices capable of sending message
>>>> based interrupts to interrupt controllers, does all the drivers of
>>>> the devices need to expose a write_msg callback to interrupt
>>>> controllers?
>>>
>>> Well, writing the message _IS_ part of the interrupt controller.
>>>
>>> So in order to enable non PCI based MSI we want to have generic
>>> infrastructure with minimal per device/device class callbacks and of
>>> course you need to provide that callback for your special device.
>>>
>>> We already have non PCI based MSI controllers in x86 today and we need
>>> to handle the whole stuff with tons of copied coded extra for each of
>>> those. So consolidating it into common infrastructure allows us to get
>>> rid of the pointless copied code and reduce the per device effort to
>>> the relevant hardware specific callbacks. irq_write_msi_msg being one
>>> of those.
>>>
>>
>> At least, we have the same goal.
>> I will illustrate my thoughts by an example.
>> The current code is something like:
>>
>> Device A
>> ========
>> void A_write_msg() { ... }
>>
>> Group B
>> (a group of devices behave same on writing messages, i.e. PCI)
>> =======
>> void B_write_msg() { ... }
>>
>> Controller
>> ==========
>> irq_chip.irq_write_msi_msg () {
>> 	if (A)
>> 		A_write_msg();
>> 	if (B)
>> 		B_write_msg();
>> }
>>
>> It's horrible when new devices come out, since we need to modify the
>> controller part for each new device.
>> What I suggested is:
>>
>> MSI Core
>> ========
>> struct msi_ops { .write_msg, };
>> struct msi_desc { .msi_ops, };
>>
>> write_msg() {
>> 	X = get_dev();
>> 	irq_chip.compose_msg(X);	// IRQ chips' responsibility
>> 	X_msi_ops.write_msg();		// nothing to do with IRQ chips
>> }
>>
>> Device A
>> ========
>> void A_write_msg() { ... }
>> A_msi_ops.write_msg = A_write_msg;
>>
>> Group B
>> =======
>> void B_write_msg() { ... }
>> B_msi_ops.write_msg = B_write_msg;
>>
>> Please correct me if I misunderstood anything.
> Hi Yun,
> 	We provide an irq_chip for each type of interrupt controller
> instead of devices. For the example mentioned above, if device A
> and Group B has different interrupt controllers, we just need to
> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
> to suitable callbacks.
> 	The framework already achieves what you you want:)

What if device A and group B have the same interrupt controller?

Regards,
	Abel

Re: [patch 09/16] genirq: Add generic msi irq domain support

[Yun Wu (Abel)]

On 2014/11/18 20:49, Jiang Liu wrote:

> 
> On 2014/11/18 20:07, Yun Wu (Abel) wrote:
>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>
>>> From: Jiang Liu <jiang.liu@linux.intel.com>
> <snip>
>>> +static int msi_domain_alloc(struct irq_domain *domain, unsigned int virq,
>>> +			    unsigned int nr_irqs, void *arg)
>>> +{
>>> +	struct msi_domain_info *info = domain->host_data;
>>> +	struct msi_domain_ops *ops = info->ops;
>>> +	irq_hw_number_t hwirq = ops->get_hwirq(info, arg);
>>> +	int i, ret;
>>> +
>>> +	if (irq_find_mapping(domain, hwirq) > 0)
>>> +		return -EEXIST;
>>
>> I think we need to check range here.
>> I fixed this by applying a patch showed in the bottom.
> Hi Yun,
> 	Yes, we have some assumptions here about the way the interface
> will be used for. Will improve it in future patches.
> Regards!
> Gerry
>>

OK. Just note that don't forget to check my second inline comment below. :)

>>> +
>>> +	ret = irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, arg);
>>> +	if (ret < 0)
>>> +		return ret;
>>> +
>>> +	for (i = 0; i < nr_irqs; i++) {
>>> +		ret = ops->msi_init(domain, info, virq + i, hwirq + i, arg);
>>> +		if (ret < 0) {
>>> +			if (ops->msi_free) {
>>> +				for (i--; i > 0; i--)
>>
>> while (i--) ?
>>
Here.


Thanks,
	Abel

>>
>>> +					ops->msi_free(domain, info, virq + i);
>>> +			}
>>> +			irq_domain_free_irqs_top(domain, virq, nr_irqs);
>>> +			return ret;
>>> +		}
>>> +	}
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void msi_domain_free(struct irq_domain *domain, unsigned int virq,
>>> +			    unsigned int nr_irqs)
>>> +{
>>> +	struct msi_domain_info *info = domain->host_data;
>>> +	int i;
>>> +
>>> +	if (info->ops->msi_free) {
>>> +		for (i = 0; i < nr_irqs; i++)
>>> +			info->ops->msi_free(domain, info, virq + i);
>>> +	}
>>> +	irq_domain_free_irqs_top(domain, virq, nr_irqs);
>>> +}
>>> +
>>> +static struct irq_domain_ops msi_domain_ops = {
>>> +	.alloc		= msi_domain_alloc,
>>> +	.free		= msi_domain_free,
>>> +	.activate	= msi_domain_activate,
>>> +	.deactivate	= msi_domain_deactivate,
>>> +};
>>> +
>>> +struct irq_domain *msi_create_irq_domain(struct device_node *of_node,
>>> +					 struct msi_domain_info *info,
>>> +					 struct irq_domain *parent)
>>> +{
>>> +	struct irq_domain *domain;
>>> +
>>> +	domain = irq_domain_add_tree(of_node, &msi_domain_ops, info);
>>> +	if (domain)
>>> +		domain->parent = parent;
>>> +
>>> +	return domain;
>>> +}
>>> +
>>> +struct msi_domain_info *msi_get_domain_info(struct irq_domain *domain)
>>> +{
>>> +	return (struct msi_domain_info *)domain->host_data;
>>> +}
>>> +
>>> +#endif /* CONFIG_GENERIC_MSI_IRQ_DOMAIN */
>>>
>>>
>>
>> diff --git a/kernel/irq/irqdomain.c b/kernel/irq/irqdomain.c
>> index cc18182..6d1be00 100644
>> --- a/kernel/irq/irqdomain.c
>> +++ b/kernel/irq/irqdomain.c
>> @@ -568,6 +568,34 @@ unsigned int irq_find_mapping(struct irq_domain *domain,
>>  }
>>  EXPORT_SYMBOL_GPL(irq_find_mapping);
>>
>> +/**
>> + * irq_find_mapping_many() - Find a range of linux irqs from a range of hwirqs.
>> + * @domain:	domain owning these hardware interrupts
>> + * @hwirq:	start hardware irq number in @domain space
>> + * @count:	number of interrupts to find
>> + *
>> + * Returns the first linux irq number on success, or 0 when not found, or error.
>> + */
>> +int irq_find_mapping_many(struct irq_domain *domain, irq_hw_number_t hwirq,
>> +			  int count)
>> +{
>> +	unsigned int from;
>> +	int i;
>> +
>> +	for (i = from = 0; i < count; i++) {
>> +		if (!from) {
>> +			from = irq_find_mapping(domain, hwirq + i);
>> +			if (from && i)
>> +				return -1;
>> +		} else if ((from + i) != irq_find_mapping(domain, hwirq + i)) {
>> +			return -1;
>> +		}
>> +	}
>> +
>> +	return from;
>> +}
>> +EXPORT_SYMBOL_GPL(irq_find_mapping_many);
>> +
>>  #ifdef CONFIG_IRQ_DOMAIN_DEBUG
>>  static int virq_debug_show(struct seq_file *m, void *private)
>>  {
>>
>>
>>
> 
> .
> 

Re: [patch 04/16] genirq: Introduce irq_chip.irq_compose_msi_msg() to support stacked irqchip

[Jiang Liu]

On 2014/11/18 21:48, Yun Wu (Abel) wrote:
> On 2014/11/18 21:25, Jiang Liu wrote:
> 
>> On 2014/11/18 21:16, Yun Wu (Abel) wrote:
>>> On 2014/11/18 20:43, Jiang Liu wrote:
>>>
>>>> On 2014/11/18 19:47, Yun Wu (Abel) wrote:
>>>>> On 2014/11/18 18:02, Thomas Gleixner wrote:
>>>>>
>>>>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>>>>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>>>>>>> +int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
>>>>>>>> +{
>>>>>>>> +	struct irq_data *pos = NULL;
>>>>>>>> +
>>>>>>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>>>>>>> +	for (; data; data = data->parent_data)
>>>>>>>> +#endif
>>>>>>>> +		if (data->chip && data->chip->irq_compose_msi_msg)
>>>>>>>> +			pos = data;
>>>>>>>> +	if (!pos)
>>>>>>>> +		return -ENOSYS;
>>>>>>>> +
>>>>>>>> +	pos->chip->irq_compose_msi_msg(pos, msg);
>>>>>>>> +
>>>>>>>> +	return 0;
>>>>>>>> +}
>>>>>>>
>>>>>>> Adding message composing routine to struct irq_chip is OK to me, and it should
>>>>>>> be because it is interrupt controllers' duty to compose messages (so that they
>>>>>>> can parse the messages correctly without any pre-defined rules that endpoint
>>>>>>> devices absolutely need not to know).
>>>>>>> However a problem comes out when deciding which parameters should be passed to
>>>>>>> this routine. A message can associate with multiple interrupts, which makes me
>>>>>>> think composing messages for each interrupt is not that appropriate. And we
>>>>>>> can take a look at the new routine irq_chip_compose_msi_msg(). It is called by
>>>>>>> msi_domain_activate() which will be called by irq_domain_activate_irq() in
>>>>>>> irq_startup() for each interrupt descriptor, result in composing a message for
>>>>>>> each interrupt, right? (Unless requiring a judge on the parameter @data when
>>>>>>> implementing the irq_compose_msi_msg() callback that only compose message for
>>>>>>> the first entry of that message. But I really don't like that...)
>>>>>>
>>>>>> No, that's not correct. You are looking at some random stale version
>>>>>> of this. The current state of affairs is in 
>>>>>>
>>>>>>    git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain
>>>>>>
>>>>>> See also https://lkml.org/lkml/2014/11/17/764
>>>>>>
>>>>>> In activate we write the message, which is the right point to do so.
>>>>>>
>>>>>
>>>>> I checked the current state, it seems to be the same.
>>>>> Yes, the decision of postponing the actual hardware programming to the point
>>>>> where the interrupt actually gets used is right, but here above I was talking
>>>>> another thing.
>>>>> As I mentioned, a message can associate with multiple interrupts. Enabling
>>>>> any of them will call irq_startup(). So if we don't want to compose or write
>>>>> messages repeatedly, we'd better require performing some checks before
>>>>> activating the interrupts.
>>>> Hi Yun,
>>>> 	Seems you are talking about the case of multiple MSI support.
>>>> Yes, we have special treatment for multiple MSI, which only writes PCI
>>>> MSI registers when starting up the first MSI interrupt.
>>>> void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg
>>>> *msg)
>>>> {
>>>>         struct msi_desc *desc = irq_data->msi_desc;
>>>>
>>>>         /*
>>>>          * For MSI-X desc->irq is always equal to irq_data->irq. For
>>>>          * MSI only the first interrupt of MULTI MSI passes the test.
>>>>          */
>>>>         if (desc->irq == irq_data->irq)
>>>>                 __pci_write_msi_msg(desc, msg);
>>>> }
>>>
>>>
>>> Yes, I picked the case of multiple MSI support.
>>> The check should also be performed when composing messages. That's why
>>> I don't like its parameters. The @data only indicates one interrupt,
>>> while I prefer doing compose/write in the unit of message descriptor.
>> Hi Yun,
>> 	The common abstraction is that every message interrupt could be
>> controlled independently, so have compose_msi_msg()/write_msi_msg() per
>> interrupt. MSI is abstracted as an special message signaled interrupt
>> with hardware limitation where multiple interrupts sharing the same
>> hardware registers. So we filter in pci_msi_domain_write_msg(). On the
>> other handle, the generic MSI framework caches msi_msg in msi_desc,
>> so we don't filter compose_msi_msg().
>>
> 
> It's true that every message interrupt could be controlled independently,
> I mean, by enable/disable/mask/unmask. But the message data & address are
> shared among the interrupts of that message.
> Despite the detailed hardware implementation, MSI and MSI-X are the same
> thing in software view, that is a message related with several consecutive
> interrupts. And the core MSI infrastructure you want to build should not
> be based on any hardware assumptions.
That's the key point. We abstract MSI as using a message to control an
interrupt source instead of controlling several consecutive interrupts.
PCI MSI is just a special case which controls a group of consecutive
interrupts all together due to hardware limitation.

> 
> Thanks,
> 	Abel
> 
> 

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Jiang Liu]

On 2014/11/18 21:52, Yun Wu (Abel) wrote:
> On 2014/11/18 21:43, Jiang Liu wrote:
> 
>> On 2014/11/18 21:33, Yun Wu (Abel) wrote:
>>> On 2014/11/18 18:19, Thomas Gleixner wrote:
>>>
>>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>>>>  struct irq_chip {
>>>>>> @@ -359,6 +360,7 @@ struct irq_chip {
>>>>>>  	void		(*irq_release_resources)(struct irq_data *data);
>>>>>>  
>>>>>>  	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>>> +	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>>
>>>>> Hmm... It's really weird.
>>>>> I don't think it's the interrupt controllers' responsibility to write messages
>>>>> for all the endpoint devices since the methods of configuring message registers
>>>>> may different between these devices. And theoretically, the endpoint devices
>>>>> themselves should take the responsibility to configure their message registers.
>>>>> To say the least, the write_msg callback here still need to call some certain
>>>>> interfaces provided by the corresponding device.
>>>>>
>>>>> There would be lots of ARM new devices capable of sending message
>>>>> based interrupts to interrupt controllers, does all the drivers of
>>>>> the devices need to expose a write_msg callback to interrupt
>>>>> controllers?
>>>>
>>>> Well, writing the message _IS_ part of the interrupt controller.
>>>>
>>>> So in order to enable non PCI based MSI we want to have generic
>>>> infrastructure with minimal per device/device class callbacks and of
>>>> course you need to provide that callback for your special device.
>>>>
>>>> We already have non PCI based MSI controllers in x86 today and we need
>>>> to handle the whole stuff with tons of copied coded extra for each of
>>>> those. So consolidating it into common infrastructure allows us to get
>>>> rid of the pointless copied code and reduce the per device effort to
>>>> the relevant hardware specific callbacks. irq_write_msi_msg being one
>>>> of those.
>>>>
>>>
>>> At least, we have the same goal.
>>> I will illustrate my thoughts by an example.
>>> The current code is something like:
>>>
>>> Device A
>>> ========
>>> void A_write_msg() { ... }
>>>
>>> Group B
>>> (a group of devices behave same on writing messages, i.e. PCI)
>>> =======
>>> void B_write_msg() { ... }
>>>
>>> Controller
>>> ==========
>>> irq_chip.irq_write_msi_msg () {
>>> 	if (A)
>>> 		A_write_msg();
>>> 	if (B)
>>> 		B_write_msg();
>>> }
>>>
>>> It's horrible when new devices come out, since we need to modify the
>>> controller part for each new device.
>>> What I suggested is:
>>>
>>> MSI Core
>>> ========
>>> struct msi_ops { .write_msg, };
>>> struct msi_desc { .msi_ops, };
>>>
>>> write_msg() {
>>> 	X = get_dev();
>>> 	irq_chip.compose_msg(X);	// IRQ chips' responsibility
>>> 	X_msi_ops.write_msg();		// nothing to do with IRQ chips
>>> }
>>>
>>> Device A
>>> ========
>>> void A_write_msg() { ... }
>>> A_msi_ops.write_msg = A_write_msg;
>>>
>>> Group B
>>> =======
>>> void B_write_msg() { ... }
>>> B_msi_ops.write_msg = B_write_msg;
>>>
>>> Please correct me if I misunderstood anything.
>> Hi Yun,
>> 	We provide an irq_chip for each type of interrupt controller
>> instead of devices. For the example mentioned above, if device A
>> and Group B has different interrupt controllers, we just need to
>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
>> to suitable callbacks.
>> 	The framework already achieves what you you want:)
> 
> What if device A and group B have the same interrupt controller?
Device doesn't care about interrupt controllers, it just cares about
interrupts used by itself. It's the interrupt source (controller)
enumerators' responsibility to discover interrupt source, associate
methods to control the interrupt source and assign irq numbers for
interrupt sources.
There are two ways to associate irq numbers with device:
1) arch code/bus drivers statically assigns irq number for devices
   when creating device objects, such as PCI legacy interrupt
   (INTA-INTD), IOAPIC interrupts.
2) device drivers ask interrupt source enumerators to dynamically
   create irq numbers, such pci_enable_msix_range() and friends.
So device driver definitely doesn't need to known about irq_chip
methods to control interrupt sources.
Regards!
Gerry

Re: [patch 04/16] genirq: Introduce irq_chip.irq_compose_msi_msg() to support stacked irqchip

[Yun Wu (Abel)]

On 2014/11/18 21:55, Jiang Liu wrote:

> On 2014/11/18 21:48, Yun Wu (Abel) wrote:
>> On 2014/11/18 21:25, Jiang Liu wrote:
>>
>>> On 2014/11/18 21:16, Yun Wu (Abel) wrote:
>>>> On 2014/11/18 20:43, Jiang Liu wrote:
>>>>
>>>>> On 2014/11/18 19:47, Yun Wu (Abel) wrote:
>>>>>> On 2014/11/18 18:02, Thomas Gleixner wrote:
>>>>>>
>>>>>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>>>>>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>>>>>>>> +int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
>>>>>>>>> +{
>>>>>>>>> +	struct irq_data *pos = NULL;
>>>>>>>>> +
>>>>>>>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>>>>>>>> +	for (; data; data = data->parent_data)
>>>>>>>>> +#endif
>>>>>>>>> +		if (data->chip && data->chip->irq_compose_msi_msg)
>>>>>>>>> +			pos = data;
>>>>>>>>> +	if (!pos)
>>>>>>>>> +		return -ENOSYS;
>>>>>>>>> +
>>>>>>>>> +	pos->chip->irq_compose_msi_msg(pos, msg);
>>>>>>>>> +
>>>>>>>>> +	return 0;
>>>>>>>>> +}
>>>>>>>>
>>>>>>>> Adding message composing routine to struct irq_chip is OK to me, and it should
>>>>>>>> be because it is interrupt controllers' duty to compose messages (so that they
>>>>>>>> can parse the messages correctly without any pre-defined rules that endpoint
>>>>>>>> devices absolutely need not to know).
>>>>>>>> However a problem comes out when deciding which parameters should be passed to
>>>>>>>> this routine. A message can associate with multiple interrupts, which makes me
>>>>>>>> think composing messages for each interrupt is not that appropriate. And we
>>>>>>>> can take a look at the new routine irq_chip_compose_msi_msg(). It is called by
>>>>>>>> msi_domain_activate() which will be called by irq_domain_activate_irq() in
>>>>>>>> irq_startup() for each interrupt descriptor, result in composing a message for
>>>>>>>> each interrupt, right? (Unless requiring a judge on the parameter @data when
>>>>>>>> implementing the irq_compose_msi_msg() callback that only compose message for
>>>>>>>> the first entry of that message. But I really don't like that...)
>>>>>>>
>>>>>>> No, that's not correct. You are looking at some random stale version
>>>>>>> of this. The current state of affairs is in 
>>>>>>>
>>>>>>>    git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain
>>>>>>>
>>>>>>> See also https://lkml.org/lkml/2014/11/17/764
>>>>>>>
>>>>>>> In activate we write the message, which is the right point to do so.
>>>>>>>
>>>>>>
>>>>>> I checked the current state, it seems to be the same.
>>>>>> Yes, the decision of postponing the actual hardware programming to the point
>>>>>> where the interrupt actually gets used is right, but here above I was talking
>>>>>> another thing.
>>>>>> As I mentioned, a message can associate with multiple interrupts. Enabling
>>>>>> any of them will call irq_startup(). So if we don't want to compose or write
>>>>>> messages repeatedly, we'd better require performing some checks before
>>>>>> activating the interrupts.
>>>>> Hi Yun,
>>>>> 	Seems you are talking about the case of multiple MSI support.
>>>>> Yes, we have special treatment for multiple MSI, which only writes PCI
>>>>> MSI registers when starting up the first MSI interrupt.
>>>>> void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg
>>>>> *msg)
>>>>> {
>>>>>         struct msi_desc *desc = irq_data->msi_desc;
>>>>>
>>>>>         /*
>>>>>          * For MSI-X desc->irq is always equal to irq_data->irq. For
>>>>>          * MSI only the first interrupt of MULTI MSI passes the test.
>>>>>          */
>>>>>         if (desc->irq == irq_data->irq)
>>>>>                 __pci_write_msi_msg(desc, msg);
>>>>> }
>>>>
>>>>
>>>> Yes, I picked the case of multiple MSI support.
>>>> The check should also be performed when composing messages. That's why
>>>> I don't like its parameters. The @data only indicates one interrupt,
>>>> while I prefer doing compose/write in the unit of message descriptor.
>>> Hi Yun,
>>> 	The common abstraction is that every message interrupt could be
>>> controlled independently, so have compose_msi_msg()/write_msi_msg() per
>>> interrupt. MSI is abstracted as an special message signaled interrupt
>>> with hardware limitation where multiple interrupts sharing the same
>>> hardware registers. So we filter in pci_msi_domain_write_msg(). On the
>>> other handle, the generic MSI framework caches msi_msg in msi_desc,
>>> so we don't filter compose_msi_msg().
>>>
>>
>> It's true that every message interrupt could be controlled independently,
>> I mean, by enable/disable/mask/unmask. But the message data & address are
>> shared among the interrupts of that message.
>> Despite the detailed hardware implementation, MSI and MSI-X are the same
>> thing in software view, that is a message related with several consecutive
>> interrupts. And the core MSI infrastructure you want to build should not
>> be based on any hardware assumptions.
> That's the key point. We abstract MSI as using a message to control an
> interrupt source instead of controlling several consecutive interrupts.
> PCI MSI is just a special case which controls a group of consecutive
> interrupts all together due to hardware limitation.
> 

Oh, I see. We abstract it in different ways...
And sounds like you treat multiple MSI as a broken implementation?

	Abel

Re: [patch 04/16] genirq: Introduce irq_chip.irq_compose_msi_msg() to support stacked irqchip

[Jiang Liu]

On 2014/11/18 22:03, Yun Wu (Abel) wrote:
> On 2014/11/18 21:55, Jiang Liu wrote:
> 
>> On 2014/11/18 21:48, Yun Wu (Abel) wrote:
>>> On 2014/11/18 21:25, Jiang Liu wrote:
>>>
>>>> On 2014/11/18 21:16, Yun Wu (Abel) wrote:
>>>>> On 2014/11/18 20:43, Jiang Liu wrote:
>>>>>
>>>>>> On 2014/11/18 19:47, Yun Wu (Abel) wrote:
>>>>>>> On 2014/11/18 18:02, Thomas Gleixner wrote:
>>>>>>>
>>>>>>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>>>>>>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>>>>>>>>> +int irq_chip_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
>>>>>>>>>> +{
>>>>>>>>>> +	struct irq_data *pos = NULL;
>>>>>>>>>> +
>>>>>>>>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>>>>>>>>> +	for (; data; data = data->parent_data)
>>>>>>>>>> +#endif
>>>>>>>>>> +		if (data->chip && data->chip->irq_compose_msi_msg)
>>>>>>>>>> +			pos = data;>>>>>>>>> +	if (!pos)
>>>>>>>>>> +		return -ENOSYS;
>>>>>>>>>> +
>>>>>>>>>> +	pos->chip->irq_compose_msi_msg(pos, msg);
>>>>>>>>>> +
>>>>>>>>>> +	return 0;
>>>>>>>>>> +}
>>>>>>>>>
>>>>>>>>> Adding message composing routine to struct irq_chip is OK to me, and it should
>>>>>>>>> be because it is interrupt controllers' duty to compose messages (so that they
>>>>>>>>> can parse the messages correctly without any pre-defined rules that endpoint
>>>>>>>>> devices absolutely need not to know).
>>>>>>>>> However a problem comes out when deciding which parameters should be passed to
>>>>>>>>> this routine. A message can associate with multiple interrupts, which makes me
>>>>>>>>> think composing messages for each interrupt is not that appropriate. And we
>>>>>>>>> can take a look at the new routine irq_chip_compose_msi_msg(). It is called by
>>>>>>>>> msi_domain_activate() which will be called by irq_domain_activate_irq() in
>>>>>>>>> irq_startup() for each interrupt descriptor, result in composing a message for
>>>>>>>>> each interrupt, right? (Unless requiring a judge on the parameter @data when
>>>>>>>>> implementing the irq_compose_msi_msg() callback that only compose message for
>>>>>>>>> the first entry of that message. But I really don't like that...)
>>>>>>>>
>>>>>>>> No, that's not correct. You are looking at some random stale version
>>>>>>>> of this. The current state of affairs is in 
>>>>>>>>
>>>>>>>>    git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/irqdomain
>>>>>>>>
>>>>>>>> See also https://lkml.org/lkml/2014/11/17/764
>>>>>>>>
>>>>>>>> In activate we write the message, which is the right point to do so.
>>>>>>>>
>>>>>>>
>>>>>>> I checked the current state, it seems to be the same.
>>>>>>> Yes, the decision of postponing the actual hardware programming to the point
>>>>>>> where the interrupt actually gets used is right, but here above I was talking
>>>>>>> another thing.
>>>>>>> As I mentioned, a message can associate with multiple interrupts. Enabling
>>>>>>> any of them will call irq_startup(). So if we don't want to compose or write
>>>>>>> messages repeatedly, we'd better require performing some checks before
>>>>>>> activating the interrupts.
>>>>>> Hi Yun,
>>>>>> 	Seems you are talking about the case of multiple MSI support.
>>>>>> Yes, we have special treatment for multiple MSI, which only writes PCI
>>>>>> MSI registers when starting up the first MSI interrupt.
>>>>>> void pci_msi_domain_write_msg(struct irq_data *irq_data, struct msi_msg
>>>>>> *msg)
>>>>>> {
>>>>>>         struct msi_desc *desc = irq_data->msi_desc;
>>>>>>
>>>>>>         /*
>>>>>>          * For MSI-X desc->irq is always equal to irq_data->irq. For
>>>>>>          * MSI only the first interrupt of MULTI MSI passes the test.
>>>>>>          */
>>>>>>         if (desc->irq == irq_data->irq)
>>>>>>                 __pci_write_msi_msg(desc, msg);
>>>>>> }
>>>>>
>>>>>
>>>>> Yes, I picked the case of multiple MSI support.
>>>>> The check should also be performed when composing messages. That's why
>>>>> I don't like its parameters. The @data only indicates one interrupt,
>>>>> while I prefer doing compose/write in the unit of message descriptor.
>>>> Hi Yun,
>>>> 	The common abstraction is that every message interrupt could be
>>>> controlled independently, so have compose_msi_msg()/write_msi_msg() per
>>>> interrupt. MSI is abstracted as an special message signaled interrupt
>>>> with hardware limitation where multiple interrupts sharing the same
>>>> hardware registers. So we filter in pci_msi_domain_write_msg(). On the
>>>> other handle, the generic MSI framework caches msi_msg in msi_desc,
>>>> so we don't filter compose_msi_msg().
>>>>
>>>
>>> It's true that every message interrupt could be controlled independently,
>>> I mean, by enable/disable/mask/unmask. But the message data & address are
>>> shared among the interrupts of that message.
>>> Despite the detailed hardware implementation, MSI and MSI-X are the same
>>> thing in software view, that is a message related with several consecutive
>>> interrupts. And the core MSI infrastructure you want to build should not
>>> be based on any hardware assumptions.
>> That's the key point. We abstract MSI as using a message to control an
>> interrupt source instead of controlling several consecutive interrupts.
>> PCI MSI is just a special case which controls a group of consecutive
>> interrupts all together due to hardware limitation.
>>
> 
> Oh, I see. We abstract it in different ways...
> And sounds like you treat multiple MSI as a broken implementation?
Basically, yes:)

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Jiang Liu]

On 2014/11/18 22:03, Jiang Liu wrote:
> On 2014/11/18 21:52, Yun Wu (Abel) wrote:
>> On 2014/11/18 21:43, Jiang Liu wrote:
>>
>>> On 2014/11/18 21:33, Yun Wu (Abel) wrote:
>>> Hi Yun,
>>> 	We provide an irq_chip for each type of interrupt controller
>>> instead of devices. For the example mentioned above, if device A
>>> and Group B has different interrupt controllers, we just need to
>>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
>>> to suitable callbacks.
>>> 	The framework already achieves what you you want:)
>>
>> What if device A and group B have the same interrupt controller?
> Device doesn't care about interrupt controllers, it just cares about
> interrupts used by itself. It's the interrupt source (controller)
> enumerators' responsibility to discover interrupt source, associate
> methods to control the interrupt source and assign irq numbers for
> interrupt sources.
> There are two ways to associate irq numbers with device:
> 1) arch code/bus drivers statically assigns irq number for devices
>    when creating device objects, such as PCI legacy interrupt
>    (INTA-INTD), IOAPIC interrupts.
> 2) device drivers ask interrupt source enumerators to dynamically
>    create irq numbers, such pci_enable_msix_range() and friends.
> So device driver definitely doesn't need to known about irq_chip
> methods to control interrupt sources.
Please refer to
http://www.xisanqi.net/year/2014/pdf/2014-pdf-09-reconstruct-x86-interrupt-architecture.pdf
for more information
about the hierarchy irqdomain background:)

> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Thomas Gleixner]

On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
> On 2014/11/18 21:43, Jiang Liu wrote:
> > 	We provide an irq_chip for each type of interrupt controller
> > instead of devices. For the example mentioned above, if device A
> > and Group B has different interrupt controllers, we just need to
> > implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
> > to suitable callbacks.
> > 	The framework already achieves what you you want:)
> 
> What if device A and group B have the same interrupt controller?

Well, if write_msg() is different they are hardly the same.

Thanks,

	tglx

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

On 2014/11/18 22:03, Jiang Liu wrote:

> On 2014/11/18 21:52, Yun Wu (Abel) wrote:
>> On 2014/11/18 21:43, Jiang Liu wrote:
>>
>>> On 2014/11/18 21:33, Yun Wu (Abel) wrote:
>>>> On 2014/11/18 18:19, Thomas Gleixner wrote:
>>>>
>>>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>>>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>>>>>  struct irq_chip {
>>>>>>> @@ -359,6 +360,7 @@ struct irq_chip {
>>>>>>>  	void		(*irq_release_resources)(struct irq_data *data);
>>>>>>>  
>>>>>>>  	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>>>> +	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>>>
>>>>>> Hmm... It's really weird.
>>>>>> I don't think it's the interrupt controllers' responsibility to write messages
>>>>>> for all the endpoint devices since the methods of configuring message registers
>>>>>> may different between these devices. And theoretically, the endpoint devices
>>>>>> themselves should take the responsibility to configure their message registers.
>>>>>> To say the least, the write_msg callback here still need to call some certain
>>>>>> interfaces provided by the corresponding device.
>>>>>>
>>>>>> There would be lots of ARM new devices capable of sending message
>>>>>> based interrupts to interrupt controllers, does all the drivers of
>>>>>> the devices need to expose a write_msg callback to interrupt
>>>>>> controllers?
>>>>>
>>>>> Well, writing the message _IS_ part of the interrupt controller.
>>>>>
>>>>> So in order to enable non PCI based MSI we want to have generic
>>>>> infrastructure with minimal per device/device class callbacks and of
>>>>> course you need to provide that callback for your special device.
>>>>>
>>>>> We already have non PCI based MSI controllers in x86 today and we need
>>>>> to handle the whole stuff with tons of copied coded extra for each of
>>>>> those. So consolidating it into common infrastructure allows us to get
>>>>> rid of the pointless copied code and reduce the per device effort to
>>>>> the relevant hardware specific callbacks. irq_write_msi_msg being one
>>>>> of those.
>>>>>
>>>>
>>>> At least, we have the same goal.
>>>> I will illustrate my thoughts by an example.
>>>> The current code is something like:
>>>>
>>>> Device A
>>>> ========
>>>> void A_write_msg() { ... }
>>>>
>>>> Group B
>>>> (a group of devices behave same on writing messages, i.e. PCI)
>>>> =======
>>>> void B_write_msg() { ... }
>>>>
>>>> Controller
>>>> ==========
>>>> irq_chip.irq_write_msi_msg () {
>>>> 	if (A)
>>>> 		A_write_msg();
>>>> 	if (B)
>>>> 		B_write_msg();
>>>> }
>>>>
>>>> It's horrible when new devices come out, since we need to modify the
>>>> controller part for each new device.
>>>> What I suggested is:
>>>>
>>>> MSI Core
>>>> ========
>>>> struct msi_ops { .write_msg, };
>>>> struct msi_desc { .msi_ops, };
>>>>
>>>> write_msg() {
>>>> 	X = get_dev();
>>>> 	irq_chip.compose_msg(X);	// IRQ chips' responsibility
>>>> 	X_msi_ops.write_msg();		// nothing to do with IRQ chips
>>>> }
>>>>
>>>> Device A
>>>> ========
>>>> void A_write_msg() { ... }
>>>> A_msi_ops.write_msg = A_write_msg;
>>>>
>>>> Group B
>>>> =======
>>>> void B_write_msg() { ... }
>>>> B_msi_ops.write_msg = B_write_msg;
>>>>
>>>> Please correct me if I misunderstood anything.
>>> Hi Yun,
>>> 	We provide an irq_chip for each type of interrupt controller
>>> instead of devices. For the example mentioned above, if device A
>>> and Group B has different interrupt controllers, we just need to
>>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
>>> to suitable callbacks.
>>> 	The framework already achieves what you you want:)
>>
>> What if device A and group B have the same interrupt controller?
> Device doesn't care about interrupt controllers, it just cares about
> interrupts used by itself. It's the interrupt source (controller)
> enumerators' responsibility to discover interrupt source, associate
> methods to control the interrupt source and assign irq numbers for
> interrupt sources.

Yes, indeed.

> There are two ways to associate irq numbers with device:
> 1) arch code/bus drivers statically assigns irq number for devices
>    when creating device objects, such as PCI legacy interrupt
>    (INTA-INTD), IOAPIC interrupts.

And OF interfaces.

> 2) device drivers ask interrupt source enumerators to dynamically
>    create irq numbers, such pci_enable_msix_range() and friends.
> So device driver definitely doesn't need to known about irq_chip
> methods to control interrupt sources.
> 

The above you described is absolutely right, but not the things I want
to know. :)
Take GICv3 ITS for example, it deals with both PCI and non PCI message
interrupts. IIUC, several irq_chips need to be implemented in the ITS
driver (i.e. pci_msi_chip, A_msi_chip and B_msi_chip). What should we
do to the ITS driver if new MSI-capable devices come out?

Regards,
	Abel

Re: [patch 09/16] genirq: Add generic msi irq domain support

[Thomas Gleixner]

On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
> On 2014/11/18 20:49, Jiang Liu wrote:
> >>> +	ret = irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, arg);
> >>> +	if (ret < 0)
> >>> +		return ret;
> >>> +
> >>> +	for (i = 0; i < nr_irqs; i++) {
> >>> +		ret = ops->msi_init(domain, info, virq + i, hwirq + i, arg);
> >>> +		if (ret < 0) {
> >>> +			if (ops->msi_free) {
> >>> +				for (i--; i > 0; i--)
> >>
> >> while (i--) ?

The allocation for 'i' failed. So we don't free i. You missed the real
bug here:

> >>> +				for (i--; i > 0; i--)

Must be i >= 0.

Thanks,

	tglx

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Jiang Liu]

On 2014/11/18 22:22, Yun Wu (Abel) wrote:
> On 2014/11/18 22:03, Jiang Liu wrote:
> 
>> On 2014/11/18 21:52, Yun Wu (Abel) wrote:
>>> On 2014/11/18 21:43, Jiang Liu wrote:
>>>
>>>> On 2014/11/18 21:33, Yun Wu (Abel) wrote:
>>>>> On 2014/11/18 18:19, Thomas Gleixner wrote:
>>>>>
>>>>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>>>>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>>>>>>  struct irq_chip {
>>>>>>>> @@ -359,6 +360,7 @@ struct irq_chip {
>>>>>>>>  	void		(*irq_release_resources)(struct irq_data *data);
>>>>>>>>  
>>>>>>>>  	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>>>>> +	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>>>>
>>>>>>> Hmm... It's really weird.
>>>>>>> I don't think it's the interrupt controllers' responsibility to write messages
>>>>>>> for all the endpoint devices since the methods of configuring message registers
>>>>>>> may different between these devices. And theoretically, the endpoint devices
>>>>>>> themselves should take the responsibility to configure their message registers.
>>>>>>> To say the least, the write_msg callback here still need to call some certain
>>>>>>> interfaces provided by the corresponding device.
>>>>>>>
>>>>>>> There would be lots of ARM new devices capable of sending message
>>>>>>> based interrupts to interrupt controllers, does all the drivers of
>>>>>>> the devices need to expose a write_msg callback to interrupt
>>>>>>> controllers?
>>>>>>
>>>>>> Well, writing the message _IS_ part of the interrupt controller.
>>>>>>
>>>>>> So in order to enable non PCI based MSI we want to have generic
>>>>>> infrastructure with minimal per device/device class callbacks and of
>>>>>> course you need to provide that callback for your special device.
>>>>>>
>>>>>> We already have non PCI based MSI controllers in x86 today and we need
>>>>>> to handle the whole stuff with tons of copied coded extra for each of
>>>>>> those. So consolidating it into common infrastructure allows us to get
>>>>>> rid of the pointless copied code and reduce the per device effort to
>>>>>> the relevant hardware specific callbacks. irq_write_msi_msg being one
>>>>>> of those.
>>>>>>
>>>>>
>>>>> At least, we have the same goal.
>>>>> I will illustrate my thoughts by an example.
>>>>> The current code is something like:
>>>>>
>>>>> Device A
>>>>> ========
>>>>> void A_write_msg() { ... }
>>>>>
>>>>> Group B
>>>>> (a group of devices behave same on writing messages, i.e. PCI)
>>>>> =======
>>>>> void B_write_msg() { ... }
>>>>>
>>>>> Controller
>>>>> ==========
>>>>> irq_chip.irq_write_msi_msg () {
>>>>> 	if (A)
>>>>> 		A_write_msg();
>>>>> 	if (B)
>>>>> 		B_write_msg();
>>>>> }
>>>>>
>>>>> It's horrible when new devices come out, since we need to modify the
>>>>> controller part for each new device.
>>>>> What I suggested is:
>>>>>
>>>>> MSI Core
>>>>> ========
>>>>> struct msi_ops { .write_msg, };
>>>>> struct msi_desc { .msi_ops, };
>>>>>
>>>>> write_msg() {
>>>>> 	X = get_dev();
>>>>> 	irq_chip.compose_msg(X);	// IRQ chips' responsibility
>>>>> 	X_msi_ops.write_msg();		// nothing to do with IRQ chips
>>>>> }
>>>>>
>>>>> Device A
>>>>> ========
>>>>> void A_write_msg() { ... }
>>>>> A_msi_ops.write_msg = A_write_msg;
>>>>>
>>>>> Group B
>>>>> =======
>>>>> void B_write_msg() { ... }
>>>>> B_msi_ops.write_msg = B_write_msg;
>>>>>
>>>>> Please correct me if I misunderstood anything.
>>>> Hi Yun,
>>>> 	We provide an irq_chip for each type of interrupt controller
>>>> instead of devices. For the example mentioned above, if device A
>>>> and Group B has different interrupt controllers, we just need to
>>>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
>>>> to suitable callbacks.
>>>> 	The framework already achieves what you you want:)
>>>
>>> What if device A and group B have the same interrupt controller?
>> Device doesn't care about interrupt controllers, it just cares about
>> interrupts used by itself. It's the interrupt source (controller)
>> enumerators' responsibility to discover interrupt source, associate
>> methods to control the interrupt source and assign irq numbers for
>> interrupt sources.
> 
> Yes, indeed.
> 
>> There are two ways to associate irq numbers with device:
>> 1) arch code/bus drivers statically assigns irq number for devices
>>    when creating device objects, such as PCI legacy interrupt
>>    (INTA-INTD), IOAPIC interrupts.
> 
> And OF interfaces.
> 
>> 2) device drivers ask interrupt source enumerators to dynamically
>>    create irq numbers, such pci_enable_msix_range() and friends.
>> So device driver definitely doesn't need to known about irq_chip
>> methods to control interrupt sources.
>>
> 
> The above you described is absolutely right, but not the things I want
> to know. :)
> Take GICv3 ITS for example, it deals with both PCI and non PCI message
> interrupts. IIUC, several irq_chips need to be implemented in the ITS
> driver (i.e. pci_msi_chip, A_msi_chip and B_msi_chip). What should we
> do to the ITS driver if new MSI-capable devices come out?
Marc has posted a patchset to enable ITS based on the hierarchy
irqdomain framework, please refer to "[PATCH 00/15] arm64: PCI/MSI:
GICv3 ITS support (stacked domain edition)" at
https://lkml.org/lkml/2014/11/11/620

> 
> Regards,
> 	Abel
> 

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Thomas Gleixner]

On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:

Can you please trim the messages when you're replying?

> The above you described is absolutely right, but not the things I want
> to know. :)
> Take GICv3 ITS for example, it deals with both PCI and non PCI message
> interrupts. IIUC, several irq_chips need to be implemented in the ITS
> driver (i.e. pci_msi_chip, A_msi_chip and B_msi_chip). What should we
> do to the ITS driver if new MSI-capable devices come out?

You seem to miss the stacking here

PCI-MSI	->
A-MSI	->   ITS  -> GIC
B-MSI	->

So each of the device types has its own MSI controller. Each of them
will have their own callbacks and are backed by the underlying ITS/GIC
implementation.

And that's the only sensible solution.

Thanks,

	tglx

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

On 2014/11/18 22:19, Thomas Gleixner wrote:

> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>> On 2014/11/18 21:43, Jiang Liu wrote:
>>> 	We provide an irq_chip for each type of interrupt controller
>>> instead of devices. For the example mentioned above, if device A
>>> and Group B has different interrupt controllers, we just need to
>>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
>>> to suitable callbacks.
>>> 	The framework already achieves what you you want:)
>>
>> What if device A and group B have the same interrupt controller?
> 
> Well, if write_msg() is different they are hardly the same.
> 

The GICv3 ITS now deals with both PCI and non PCI message interrupts.
We can't require the new devices behave writing message in a same way.
What we can do is to abstract all the endpoints' behavior, and I
provided one abstraction in an earlier reply.

Thanks,
	Abel

Re: [patch 09/16] genirq: Add generic msi irq domain support

[Yun Wu (Abel)]

On 2014/11/18 22:24, Thomas Gleixner wrote:

> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>> On 2014/11/18 20:49, Jiang Liu wrote:
>>>>> +	ret = irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, arg);
>>>>> +	if (ret < 0)
>>>>> +		return ret;
>>>>> +
>>>>> +	for (i = 0; i < nr_irqs; i++) {
>>>>> +		ret = ops->msi_init(domain, info, virq + i, hwirq + i, arg);
>>>>> +		if (ret < 0) {
>>>>> +			if (ops->msi_free) {
>>>>> +				for (i--; i > 0; i--)
>>>>
>>>> while (i--) ?
> 
> The allocation for 'i' failed. So we don't free i. You missed the real
> bug here:
> 
>>>>> +				for (i--; i > 0; i--)
> 
> Must be i >= 0.
> 

"for (i--; i >= 0; i--)" is just the same as "while (i--)".

Thanks,
	Abel

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

On 2014/11/18 22:29, Jiang Liu wrote:

> 
> 
> On 2014/11/18 22:22, Yun Wu (Abel) wrote:
>> On 2014/11/18 22:03, Jiang Liu wrote:
>>
>>> On 2014/11/18 21:52, Yun Wu (Abel) wrote:
>>>> On 2014/11/18 21:43, Jiang Liu wrote:
>>>>
>>>>> On 2014/11/18 21:33, Yun Wu (Abel) wrote:
>>>>>> On 2014/11/18 18:19, Thomas Gleixner wrote:
>>>>>>
>>>>>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>>>>>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>>>>>>>  struct irq_chip {
>>>>>>>>> @@ -359,6 +360,7 @@ struct irq_chip {
>>>>>>>>>  	void		(*irq_release_resources)(struct irq_data *data);
>>>>>>>>>  
>>>>>>>>>  	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>>>>>> +	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>>>>>
>>>>>>>> Hmm... It's really weird.
>>>>>>>> I don't think it's the interrupt controllers' responsibility to write messages
>>>>>>>> for all the endpoint devices since the methods of configuring message registers
>>>>>>>> may different between these devices. And theoretically, the endpoint devices
>>>>>>>> themselves should take the responsibility to configure their message registers.
>>>>>>>> To say the least, the write_msg callback here still need to call some certain
>>>>>>>> interfaces provided by the corresponding device.
>>>>>>>>
>>>>>>>> There would be lots of ARM new devices capable of sending message
>>>>>>>> based interrupts to interrupt controllers, does all the drivers of
>>>>>>>> the devices need to expose a write_msg callback to interrupt
>>>>>>>> controllers?
>>>>>>>
>>>>>>> Well, writing the message _IS_ part of the interrupt controller.
>>>>>>>
>>>>>>> So in order to enable non PCI based MSI we want to have generic
>>>>>>> infrastructure with minimal per device/device class callbacks and of
>>>>>>> course you need to provide that callback for your special device.
>>>>>>>
>>>>>>> We already have non PCI based MSI controllers in x86 today and we need
>>>>>>> to handle the whole stuff with tons of copied coded extra for each of
>>>>>>> those. So consolidating it into common infrastructure allows us to get
>>>>>>> rid of the pointless copied code and reduce the per device effort to
>>>>>>> the relevant hardware specific callbacks. irq_write_msi_msg being one
>>>>>>> of those.
>>>>>>>
>>>>>>
>>>>>> At least, we have the same goal.
>>>>>> I will illustrate my thoughts by an example.
>>>>>> The current code is something like:
>>>>>>
>>>>>> Device A
>>>>>> ========
>>>>>> void A_write_msg() { ... }
>>>>>>
>>>>>> Group B
>>>>>> (a group of devices behave same on writing messages, i.e. PCI)
>>>>>> =======
>>>>>> void B_write_msg() { ... }
>>>>>>
>>>>>> Controller
>>>>>> ==========
>>>>>> irq_chip.irq_write_msi_msg () {
>>>>>> 	if (A)
>>>>>> 		A_write_msg();
>>>>>> 	if (B)
>>>>>> 		B_write_msg();
>>>>>> }
>>>>>>
>>>>>> It's horrible when new devices come out, since we need to modify the
>>>>>> controller part for each new device.
>>>>>> What I suggested is:
>>>>>>
>>>>>> MSI Core
>>>>>> ========
>>>>>> struct msi_ops { .write_msg, };
>>>>>> struct msi_desc { .msi_ops, };
>>>>>>
>>>>>> write_msg() {
>>>>>> 	X = get_dev();
>>>>>> 	irq_chip.compose_msg(X);	// IRQ chips' responsibility
>>>>>> 	X_msi_ops.write_msg();		// nothing to do with IRQ chips
>>>>>> }
>>>>>>
>>>>>> Device A
>>>>>> ========
>>>>>> void A_write_msg() { ... }
>>>>>> A_msi_ops.write_msg = A_write_msg;
>>>>>>
>>>>>> Group B
>>>>>> =======
>>>>>> void B_write_msg() { ... }
>>>>>> B_msi_ops.write_msg = B_write_msg;
>>>>>>
>>>>>> Please correct me if I misunderstood anything.
>>>>> Hi Yun,
>>>>> 	We provide an irq_chip for each type of interrupt controller
>>>>> instead of devices. For the example mentioned above, if device A
>>>>> and Group B has different interrupt controllers, we just need to
>>>>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
>>>>> to suitable callbacks.
>>>>> 	The framework already achieves what you you want:)
>>>>
>>>> What if device A and group B have the same interrupt controller?
>>> Device doesn't care about interrupt controllers, it just cares about
>>> interrupts used by itself. It's the interrupt source (controller)
>>> enumerators' responsibility to discover interrupt source, associate
>>> methods to control the interrupt source and assign irq numbers for
>>> interrupt sources.
>>
>> Yes, indeed.
>>
>>> There are two ways to associate irq numbers with device:
>>> 1) arch code/bus drivers statically assigns irq number for devices
>>>    when creating device objects, such as PCI legacy interrupt
>>>    (INTA-INTD), IOAPIC interrupts.
>>
>> And OF interfaces.
>>
>>> 2) device drivers ask interrupt source enumerators to dynamically
>>>    create irq numbers, such pci_enable_msix_range() and friends.
>>> So device driver definitely doesn't need to known about irq_chip
>>> methods to control interrupt sources.
>>>
>>
>> The above you described is absolutely right, but not the things I want
>> to know. :)
>> Take GICv3 ITS for example, it deals with both PCI and non PCI message
>> interrupts. IIUC, several irq_chips need to be implemented in the ITS
>> driver (i.e. pci_msi_chip, A_msi_chip and B_msi_chip). What should we
>> do to the ITS driver if new MSI-capable devices come out?
> Marc has posted a patchset to enable ITS based on the hierarchy
> irqdomain framework, please refer to "[PATCH 00/15] arm64: PCI/MSI:
> GICv3 ITS support (stacked domain edition)" at
> https://lkml.org/lkml/2014/11/11/620
> 

IIUC, Marc's patch now only supports PCI MSI/MSI-X...

Thanks,
	Abel

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Jiang Liu]

On 2014/11/18 22:34, Yun Wu (Abel) wrote:
> On 2014/11/18 22:19, Thomas Gleixner wrote:
> 
>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>> On 2014/11/18 21:43, Jiang Liu wrote:
>>>> 	We provide an irq_chip for each type of interrupt controller
>>>> instead of devices. For the example mentioned above, if device A
>>>> and Group B has different interrupt controllers, we just need to
>>>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
>>>> to suitable callbacks.
>>>> 	The framework already achieves what you you want:)
>>>
>>> What if device A and group B have the same interrupt controller?
>>
>> Well, if write_msg() is different they are hardly the same.
>>
> 
> The GICv3 ITS now deals with both PCI and non PCI message interrupts.
> We can't require the new devices behave writing message in a same way.
> What we can do is to abstract all the endpoints' behavior, and I
> provided one abstraction in an earlier reply.
It should be easy to extend:)
Actually, x86 interrupt remapping drivers already support two types of
MSIs, one is PCI MSI/MSIX, another is HPET interrupt.
> 
> Thanks,
> 	Abel
> 

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Marc Zyngier]

On Tue, Nov 18 2014 at  2:46:02 pm GMT, "Yun Wu (Abel)" <wuyun.wu@huawei.com> wrote:
> On 2014/11/18 22:29, Jiang Liu wrote:
>
>> 
>> 
>> On 2014/11/18 22:22, Yun Wu (Abel) wrote:
>>> On 2014/11/18 22:03, Jiang Liu wrote:
>>>
>>>> On 2014/11/18 21:52, Yun Wu (Abel) wrote:
>>>>> On 2014/11/18 21:43, Jiang Liu wrote:
>>>>>
>>>>>> On 2014/11/18 21:33, Yun Wu (Abel) wrote:
>>>>>>> On 2014/11/18 18:19, Thomas Gleixner wrote:
>>>>>>>
>>>>>>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>>>>>>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>>>>>>>>  struct irq_chip {
>>>>>>>>>> @@ -359,6 +360,7 @@ struct irq_chip {
>>>>>>>>>>  	void		(*irq_release_resources)(struct irq_data *data);
>>>>>>>>>>  
>>>>>>>>>>  	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>>>>>>> +	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>>>>>>
>>>>>>>>> Hmm... It's really weird.
>>>>>>>>> I don't think it's the interrupt controllers' responsibility
>>>>>>>>> to write messages
>>>>>>>>> for all the endpoint devices since the methods of configuring
>>>>>>>>> message registers
>>>>>>>>> may different between these devices. And theoretically, the
>>>>>>>>> endpoint devices
>>>>>>>>> themselves should take the responsibility to configure their
>>>>>>>>> message registers.
>>>>>>>>> To say the least, the write_msg callback here still need to
>>>>>>>>> call some certain
>>>>>>>>> interfaces provided by the corresponding device.
>>>>>>>>>
>>>>>>>>> There would be lots of ARM new devices capable of sending message
>>>>>>>>> based interrupts to interrupt controllers, does all the drivers of
>>>>>>>>> the devices need to expose a write_msg callback to interrupt
>>>>>>>>> controllers?
>>>>>>>>
>>>>>>>> Well, writing the message _IS_ part of the interrupt controller.
>>>>>>>>
>>>>>>>> So in order to enable non PCI based MSI we want to have generic
>>>>>>>> infrastructure with minimal per device/device class callbacks and of
>>>>>>>> course you need to provide that callback for your special device.
>>>>>>>>
>>>>>>>> We already have non PCI based MSI controllers in x86 today and we need
>>>>>>>> to handle the whole stuff with tons of copied coded extra for each of
>>>>>>>> those. So consolidating it into common infrastructure allows us to get
>>>>>>>> rid of the pointless copied code and reduce the per device effort to
>>>>>>>> the relevant hardware specific callbacks. irq_write_msi_msg being one
>>>>>>>> of those.
>>>>>>>>
>>>>>>>
>>>>>>> At least, we have the same goal.
>>>>>>> I will illustrate my thoughts by an example.
>>>>>>> The current code is something like:
>>>>>>>
>>>>>>> Device A
>>>>>>> ========
>>>>>>> void A_write_msg() { ... }
>>>>>>>
>>>>>>> Group B
>>>>>>> (a group of devices behave same on writing messages, i.e. PCI)
>>>>>>> =======
>>>>>>> void B_write_msg() { ... }
>>>>>>>
>>>>>>> Controller
>>>>>>> ==========
>>>>>>> irq_chip.irq_write_msi_msg () {
>>>>>>> 	if (A)
>>>>>>> 		A_write_msg();
>>>>>>> 	if (B)
>>>>>>> 		B_write_msg();
>>>>>>> }
>>>>>>>
>>>>>>> It's horrible when new devices come out, since we need to modify the
>>>>>>> controller part for each new device.
>>>>>>> What I suggested is:
>>>>>>>
>>>>>>> MSI Core
>>>>>>> ========
>>>>>>> struct msi_ops { .write_msg, };
>>>>>>> struct msi_desc { .msi_ops, };
>>>>>>>
>>>>>>> write_msg() {
>>>>>>> 	X = get_dev();
>>>>>>> 	irq_chip.compose_msg(X);	// IRQ chips' responsibility
>>>>>>> 	X_msi_ops.write_msg();		// nothing to do with IRQ chips
>>>>>>> }
>>>>>>>
>>>>>>> Device A
>>>>>>> ========
>>>>>>> void A_write_msg() { ... }
>>>>>>> A_msi_ops.write_msg = A_write_msg;
>>>>>>>
>>>>>>> Group B
>>>>>>> =======
>>>>>>> void B_write_msg() { ... }
>>>>>>> B_msi_ops.write_msg = B_write_msg;
>>>>>>>
>>>>>>> Please correct me if I misunderstood anything.
>>>>>> Hi Yun,
>>>>>> 	We provide an irq_chip for each type of interrupt controller
>>>>>> instead of devices. For the example mentioned above, if device A
>>>>>> and Group B has different interrupt controllers, we just need to
>>>>>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
>>>>>> to suitable callbacks.
>>>>>> 	The framework already achieves what you you want:)
>>>>>
>>>>> What if device A and group B have the same interrupt controller?
>>>> Device doesn't care about interrupt controllers, it just cares about
>>>> interrupts used by itself. It's the interrupt source (controller)
>>>> enumerators' responsibility to discover interrupt source, associate
>>>> methods to control the interrupt source and assign irq numbers for
>>>> interrupt sources.
>>>
>>> Yes, indeed.
>>>
>>>> There are two ways to associate irq numbers with device:
>>>> 1) arch code/bus drivers statically assigns irq number for devices
>>>>    when creating device objects, such as PCI legacy interrupt
>>>>    (INTA-INTD), IOAPIC interrupts.
>>>
>>> And OF interfaces.
>>>
>>>> 2) device drivers ask interrupt source enumerators to dynamically
>>>>    create irq numbers, such pci_enable_msix_range() and friends.
>>>> So device driver definitely doesn't need to known about irq_chip
>>>> methods to control interrupt sources.
>>>>
>>>
>>> The above you described is absolutely right, but not the things I want
>>> to know. :)
>>> Take GICv3 ITS for example, it deals with both PCI and non PCI message
>>> interrupts. IIUC, several irq_chips need to be implemented in the ITS
>>> driver (i.e. pci_msi_chip, A_msi_chip and B_msi_chip). What should we
>>> do to the ITS driver if new MSI-capable devices come out?
>> Marc has posted a patchset to enable ITS based on the hierarchy
>> irqdomain framework, please refer to "[PATCH 00/15] arm64: PCI/MSI:
>> GICv3 ITS support (stacked domain edition)" at
>> https://lkml.org/lkml/2014/11/11/620
>> 
>
> IIUC, Marc's patch now only supports PCI MSI/MSI-X...

Indeed, and the current solution makes is relatively easy to plug in
non-PCI MSI. Just don't plug the ITS into the *PCI* MSI framework when
you encounter such a thing.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny.

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Marc Zyngier]

On Tue, Nov 18 2014 at  2:34:44 pm GMT, "Yun Wu (Abel)" <wuyun.wu@huawei.com> wrote:
> On 2014/11/18 22:19, Thomas Gleixner wrote:
>
>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>> On 2014/11/18 21:43, Jiang Liu wrote:
>>>> 	We provide an irq_chip for each type of interrupt controller
>>>> instead of devices. For the example mentioned above, if device A
>>>> and Group B has different interrupt controllers, we just need to
>>>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
>>>> to suitable callbacks.
>>>> 	The framework already achieves what you you want:)
>>>
>>> What if device A and group B have the same interrupt controller?
>> 
>> Well, if write_msg() is different they are hardly the same.
>> 
>
> The GICv3 ITS now deals with both PCI and non PCI message interrupts.
> We can't require the new devices behave writing message in a same way.
> What we can do is to abstract all the endpoints' behavior, and I
> provided one abstraction in an earlier reply.

This is why the framework gives you the opportunity to provide methods
that:
- compose the message
- program the message into the device

None of that has to be PCI specific, and gives you a clean
abstraction. The framework only gives you a number of shortcuts for PCI
MSI, because that's what most people care about.

Thanks,

        M.
-- 
Jazz is not dead. It just smells funny.

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

On 2014/11/19 1:14, Marc Zyngier wrote:

> On Tue, Nov 18 2014 at  2:46:02 pm GMT, "Yun Wu (Abel)" <wuyun.wu@huawei.com> wrote:
[...]
>> IIUC, Marc's patch now only supports PCI MSI/MSI-X...
> 
> Indeed, and the current solution makes is relatively easy to plug in
> non-PCI MSI. Just don't plug the ITS into the *PCI* MSI framework when
> you encounter such a thing.
> 

I am looking forward to that. :)
I guess the main reason you haven't plugged in non-PCI MSI is the way
of obtaining device ids used in searching DT table.

Thanks,
	Abel

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

On 2014/11/19 1:21, Marc Zyngier wrote:

> On Tue, Nov 18 2014 at  2:34:44 pm GMT, "Yun Wu (Abel)" <wuyun.wu@huawei.com> wrote:
>> On 2014/11/18 22:19, Thomas Gleixner wrote:
>>
>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>> On 2014/11/18 21:43, Jiang Liu wrote:
>>>>> 	We provide an irq_chip for each type of interrupt controller
>>>>> instead of devices. For the example mentioned above, if device A
>>>>> and Group B has different interrupt controllers, we just need to
>>>>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
>>>>> to suitable callbacks.
>>>>> 	The framework already achieves what you you want:)
>>>>
>>>> What if device A and group B have the same interrupt controller?
>>>
>>> Well, if write_msg() is different they are hardly the same.
>>>
>>
>> The GICv3 ITS now deals with both PCI and non PCI message interrupts.
>> We can't require the new devices behave writing message in a same way.
>> What we can do is to abstract all the endpoints' behavior, and I
>> provided one abstraction in an earlier reply.
> 
> This is why the framework gives you the opportunity to provide methods
> that:
> - compose the message
> - program the message into the device
> 
> None of that has to be PCI specific, and gives you a clean
> abstraction. The framework only gives you a number of shortcuts for PCI
> MSI, because that's what most people care about.
> 

Indeed, and I never said Jiang's patches don't work, I was just thinking
that they were not that perfect.

Thanks,
	Abel

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

On 2014/11/18 22:52, Jiang Liu wrote:

> On 2014/11/18 22:34, Yun Wu (Abel) wrote:
>> On 2014/11/18 22:19, Thomas Gleixner wrote:
>>
>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>> On 2014/11/18 21:43, Jiang Liu wrote:
>>>>> 	We provide an irq_chip for each type of interrupt controller
>>>>> instead of devices. For the example mentioned above, if device A
>>>>> and Group B has different interrupt controllers, we just need to
>>>>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
>>>>> to suitable callbacks.
>>>>> 	The framework already achieves what you you want:)
>>>>
>>>> What if device A and group B have the same interrupt controller?
>>>
>>> Well, if write_msg() is different they are hardly the same.
>>>
>>
>> The GICv3 ITS now deals with both PCI and non PCI message interrupts.
>> We can't require the new devices behave writing message in a same way.
>> What we can do is to abstract all the endpoints' behavior, and I
>> provided one abstraction in an earlier reply.
> It should be easy to extend:)
> Actually, x86 interrupt remapping drivers already support two types of
> MSIs, one is PCI MSI/MSIX, another is HPET interrupt.


Well, if there are one hundred types, I don't think it's as easy as you
thought to extend. Of course we can doubt the possibility of being hundred,
but tens or twenties is reasonably possible lying under the fact we have
already startet to integrate the MSI registers (or some other form to store
information) into the individual devices.

Thanks,
	Abel

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

On 2014/11/18 22:32, Thomas Gleixner wrote:

> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
> 
> Can you please trim the messages when you're replying?
> 
>> The above you described is absolutely right, but not the things I want
>> to know. :)
>> Take GICv3 ITS for example, it deals with both PCI and non PCI message
>> interrupts. IIUC, several irq_chips need to be implemented in the ITS
>> driver (i.e. pci_msi_chip, A_msi_chip and B_msi_chip). What should we
>> do to the ITS driver if new MSI-capable devices come out?
> 
> You seem to miss the stacking here
> 
> PCI-MSI	->
> A-MSI	->   ITS  -> GIC
> B-MSI	->
> 
> So each of the device types has its own MSI controller. Each of them
> will have their own callbacks and are backed by the underlying ITS/GIC
> implementation.

Yes, this hits the key point. Once a new device type becomes available,
we need to add pieces of code outside the new device's driver to make
it work, which in my opinion is due to lack of core infrastructure.
More specifically, the core infrastructure needs to support mechanism
of MSI, not the various types of devices.

> 
> And that's the only sensible solution.
> 

It's sensible, but not perfect.
What I suggested is to add a generic layer:

PCI-MSI	->
A-MSI	-> (generic layer) -> ITS -> GICR
B-MSI	->

The PCI/A/B/... passes its hardware properties to the generic layer which
gets configurations ready by calling ITS's domain/chip callbacks. When
a new device type arrives, the only thing need to do is to implement the
driver of that device, with nothing to do with the generic layer or ITS.

Thanks,
	Abel

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Jiang Liu]

On 2014/11/19 14:57, Yun Wu (Abel) wrote:
> On 2014/11/18 22:32, Thomas Gleixner wrote:
> 
>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>
>> Can you please trim the messages when you're replying?
>>
>>> The above you described is absolutely right, but not the things I want
>>> to know. :)
>>> Take GICv3 ITS for example, it deals with both PCI and non PCI message
>>> interrupts. IIUC, several irq_chips need to be implemented in the ITS
>>> driver (i.e. pci_msi_chip, A_msi_chip and B_msi_chip). What should we
>>> do to the ITS driver if new MSI-capable devices come out?
>>
>> You seem to miss the stacking here
>>
>> PCI-MSI	->
>> A-MSI	->   ITS  -> GIC
>> B-MSI	->
>>
>> So each of the device types has its own MSI controller. Each of them
>> will have their own callbacks and are backed by the underlying ITS/GIC
>> implementation.
> 
> Yes, this hits the key point. Once a new device type becomes available,
> we need to add pieces of code outside the new device's driver to make
> it work, which in my opinion is due to lack of core infrastructure.
> More specifically, the core infrastructure needs to support mechanism
> of MSI, not the various types of devices.
It depends on the implementation details whether we need to change ITS
to support new devices. One possible way to avoid changing ITS to
support new devices is:
1) ITS only allocates and manages interrupt translation tables and pass
   some type of identifier back to MSI controller drivers.
2) Each MSI controller driver implements its own compose_msi_msg()
   and write_msi_msg() according to hardware specifications.
By this way, there is no need to change ITS to support new types of
MSI controllers.

On x86 systems, interrupt remapping is optional and we also want to hide
implementation details, so we chose another solution:
1) interrupt remapping driver allocates and manages remapping resources,
   it also implements different compose_msi_msg() to support different
   MSI controllers.
2) MSI controller implement write_msi_msg(), it also implements
   compose_msi_msg() if interrupt remapping is disabled.
By this way, we do need to change interrupt remapping driver(ITS on ARM)
to support new type of MSI controllers.

So it totally depends on how you implement the hierarchy irqdomain,
there are no limitations from the framework side.

For the proposal to add some type of ops structure into struct msi_desc,
actually I have a serial of patches to support device specific
mask/unmask/write/compose/startup/shutdown operations by adding
msi_desc.chip_ops. But there's no actual users of those interfaces
currently and we don't want to over-engineering, so I abandoned
that patch set. I may rework on it  when enabling generic MSI
support later.
Regards!
Gerry
> 
>>
>> And that's the only sensible solution.
>>
> 
> It's sensible, but not perfect.
> What I suggested is to add a generic layer:
> 
> PCI-MSI	->
> A-MSI	-> (generic layer) -> ITS -> GICR
> B-MSI	->
> 
> The PCI/A/B/... passes its hardware properties to the generic layer which
> gets configurations ready by calling ITS's domain/chip callbacks. When
> a new device type arrives, the only thing need to do is to implement the
> driver of that device, with nothing to do with the generic layer or ITS.
> 
> Thanks,
> 	Abel
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Marc Zyngier]

On Wed, Nov 19 2014 at  3:38:09 am GMT, "Yun Wu (Abel)" <wuyun.wu@huawei.com> wrote:
> On 2014/11/19 1:14, Marc Zyngier wrote:
>
>> On Tue, Nov 18 2014 at  2:46:02 pm GMT, "Yun Wu (Abel)" <wuyun.wu@huawei.com> wrote:
> [...]
>>> IIUC, Marc's patch now only supports PCI MSI/MSI-X...
>> 
>> Indeed, and the current solution makes is relatively easy to plug in
>> non-PCI MSI. Just don't plug the ITS into the *PCI* MSI framework when
>> you encounter such a thing.
>> 
>
> I am looking forward to that. :)

I really don't.

> I guess the main reason you haven't plugged in non-PCI MSI is the way
> of obtaining device ids used in searching DT table.

That, and also:
- No way to carve out LPI ranges for non-PCI allocations, should some
ITS mandate specific ranges
- No description of how to program the generating device, it required
- No platform with such requirements aiming for upstream support

To put it mildly, non-PCI support on the ITS is a rather long way
away, and it has nothing to do with the code. It has everything to do
with specifying what we want to support, and how we describe it in
DT. Until then, not much is going to happen.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny.

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Marc Zyngier]

On Wed, Nov 19 2014 at  6:57:25 am GMT, "Yun Wu (Abel)" <wuyun.wu@huawei.com> wrote:
> On 2014/11/18 22:32, Thomas Gleixner wrote:
>
>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>> 
>> Can you please trim the messages when you're replying?
>> 
>>> The above you described is absolutely right, but not the things I want
>>> to know. :)
>>> Take GICv3 ITS for example, it deals with both PCI and non PCI message
>>> interrupts. IIUC, several irq_chips need to be implemented in the ITS
>>> driver (i.e. pci_msi_chip, A_msi_chip and B_msi_chip). What should we
>>> do to the ITS driver if new MSI-capable devices come out?
>> 
>> You seem to miss the stacking here
>> 
>> PCI-MSI	->
>> A-MSI	->   ITS  -> GIC
>> B-MSI	->
>> 
>> So each of the device types has its own MSI controller. Each of them
>> will have their own callbacks and are backed by the underlying ITS/GIC
>> implementation.
>
> Yes, this hits the key point. Once a new device type becomes available,
> we need to add pieces of code outside the new device's driver to make
> it work, which in my opinion is due to lack of core infrastructure.
> More specifically, the core infrastructure needs to support mechanism
> of MSI, not the various types of devices.
>
>> 
>> And that's the only sensible solution.
>> 
>
> It's sensible, but not perfect.
> What I suggested is to add a generic layer:
>
> PCI-MSI	->
> A-MSI	-> (generic layer) -> ITS -> GICR
> B-MSI	->
>
> The PCI/A/B/... passes its hardware properties to the generic layer which
> gets configurations ready by calling ITS's domain/chip callbacks. When
> a new device type arrives, the only thing need to do is to implement the
> driver of that device, with nothing to do with the generic layer or ITS.

I really don't get your "generic layer" story. To me, it looks like a
glorified set of function pointers. And that's exactly what stacked
domains are giving you:

A-MSI -> ITS -> GIC

This "A-MSI" is responsible for:
- implementing the "prepare" callback, which allocates the ITT
- implementing the "irq_compose_msi_msg"

Hardly anything to change in the ITS driver, and I can probably make it
so that you don't even have to write a single line of code in the ITS
driver.

If the generic MSI layer we now have is not enough for you, then please
submit detailed use cases.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny.

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Thomas Gleixner]

On Wed, 19 Nov 2014, Yun Wu (Abel) wrote:
> On 2014/11/18 22:52, Jiang Liu wrote:
> 
> > On 2014/11/18 22:34, Yun Wu (Abel) wrote:
> >> On 2014/11/18 22:19, Thomas Gleixner wrote:
> >>
> >>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
> >>>> On 2014/11/18 21:43, Jiang Liu wrote:
> >>>>> 	We provide an irq_chip for each type of interrupt controller
> >>>>> instead of devices. For the example mentioned above, if device A
> >>>>> and Group B has different interrupt controllers, we just need to
> >>>>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
> >>>>> to suitable callbacks.
> >>>>> 	The framework already achieves what you you want:)
> >>>>
> >>>> What if device A and group B have the same interrupt controller?
> >>>
> >>> Well, if write_msg() is different they are hardly the same.
> >>>
> >>
> >> The GICv3 ITS now deals with both PCI and non PCI message interrupts.
> >> We can't require the new devices behave writing message in a same way.
> >> What we can do is to abstract all the endpoints' behavior, and I
> >> provided one abstraction in an earlier reply.
> > It should be easy to extend:)
> > Actually, x86 interrupt remapping drivers already support two types of
> > MSIs, one is PCI MSI/MSIX, another is HPET interrupt.
> 
> 
> Well, if there are one hundred types, I don't think it's as easy as you
> thought to extend. Of course we can doubt the possibility of being hundred,
> but tens or twenties is reasonably possible lying under the fact we have
> already startet to integrate the MSI registers (or some other form to store
> information) into the individual devices.

If your hardware designers decided to come up with 20+ different ways
to implement MSI support, then it's not a reason to inflict completely
non-sensical crap into the core infrastucture.

The infrastructure we provided is optimized for sane use cases, while
it allows you to deal with totally brainwrecked hardware designs. 

If you have 20 different ways, you need 20 different controllers for
it, whether you like it or not. No magic 'generic' abstraction will
solve that for you.

Thanks,

	tglx

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Thomas Gleixner]

On Wed, 19 Nov 2014, Yun Wu (Abel) wrote:
> On 2014/11/19 1:21, Marc Zyngier wrote:
> > This is why the framework gives you the opportunity to provide methods
> > that:
> > - compose the message
> > - program the message into the device
> > 
> > None of that has to be PCI specific, and gives you a clean
> > abstraction. The framework only gives you a number of shortcuts for PCI
> > MSI, because that's what most people care about.
> > 
> 
> Indeed, and I never said Jiang's patches don't work, I was just thinking
> that they were not that perfect.

But your magic extra layer of indirection is perfect? It's not, it
just violates sane layering in order to support braindead hardware
designs.

Thanks,

	tglx

Re: [patch 09/16] genirq: Add generic msi irq domain support

[Jiang Liu]

On 2014/11/18 22:24, Thomas Gleixner wrote:
> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>> On 2014/11/18 20:49, Jiang Liu wrote:
>>>>> +	ret = irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, arg);
>>>>> +	if (ret < 0)
>>>>> +		return ret;
>>>>> +
>>>>> +	for (i = 0; i < nr_irqs; i++) {
>>>>> +		ret = ops->msi_init(domain, info, virq + i, hwirq + i, arg);
>>>>> +		if (ret < 0) {
>>>>> +			if (ops->msi_free) {
>>>>> +				for (i--; i > 0; i--)
>>>>
>>>> while (i--) ?
> 
> The allocation for 'i' failed. So we don't free i. You missed the real
> bug here:
> 
>>>>> +				for (i--; i > 0; i--)
> 
> Must be i >= 0.
Hi Thomas,
	You will take care of this bug, right? Or should I send a patch
for it?
Regards!
Gerry

Re: [patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Yun Wu (Abel)]

Hi Thomas, Jiang,
On 2014/11/12 21:42, Thomas Gleixner wrote:

> From: Jiang Liu <jiang.liu@linux.intel.com>
> 
[...]
>  /* Number of irqs reserved for a legacy isa controller */
>  #define NUM_ISA_INTERRUPTS	16
> @@ -64,6 +66,16 @@ struct irq_domain_ops {
>  	int (*xlate)(struct irq_domain *d, struct device_node *node,
>  		     const u32 *intspec, unsigned int intsize,
>  		     unsigned long *out_hwirq, unsigned int *out_type);
> +
> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
> +	/* extended V2 interfaces to support hierarchy irq_domains */
> +	int (*alloc)(struct irq_domain *d, unsigned int virq,
> +		     unsigned int nr_irqs, void *arg);
> +	void (*free)(struct irq_domain *d, unsigned int virq,
> +		     unsigned int nr_irqs);
> +	void (*activate)(struct irq_domain *d, struct irq_data *irq_data);
> +	void (*deactivate)(struct irq_domain *d, struct irq_data *irq_data);

What's the usage of the parameter domain reference in activate/deactivate?
I think the purpose of the two callbacks is to activate/deactivate the
irq_data->hwirq in irq_data->domain. If so, the first parameter @domain is
required to be equal to irq_data->domain (which makes @domain useless).
Besides, the main responsibility of interrupt domains is to manage mappings
between hardware and linux interrupt numbers, so would it be better if move
the two callbacks into struct irq_chip?

Thanks,
	Abel.

> +#endif
>  };
>  
>  extern struct irq_domain_ops irq_generic_chip_ops;
> @@ -77,6 +89,7 @@ struct irq_domain_chip_generic;
>   * @ops: pointer to irq_domain methods
>   * @host_data: private data pointer for use by owner.  Not touched by irq_domain
>   *             core code.
> + * @flags: host per irq_domain flags
>   *
>   * Optional elements
>   * @of_node: Pointer to device tree nodes associated with the irq_domain. Used
> @@ -84,6 +97,7 @@ struct irq_domain_chip_generic;
>   * @gc: Pointer to a list of generic chips. There is a helper function for
>   *      setting up one or more generic chips for interrupt controllers
>   *      drivers using the generic chip library which uses this pointer.
> + * @parent: Pointer to parent irq_domain to support hierarchy irq_domains
>   *
>   * Revmap data, used internally by irq_domain
>   * @revmap_direct_max_irq: The largest hwirq that can be set for controllers that
> @@ -97,10 +111,14 @@ struct irq_domain {
>  	const char *name;
>  	const struct irq_domain_ops *ops;
>  	void *host_data;
> +	unsigned int flags;
>  
>  	/* Optional data */
>  	struct device_node *of_node;
>  	struct irq_domain_chip_generic *gc;
> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
> +	struct irq_domain *parent;
> +#endif
>  
>  	/* reverse map data. The linear map gets appended to the irq_domain */
>  	irq_hw_number_t hwirq_max;
> @@ -110,6 +128,9 @@ struct irq_domain {
>  	unsigned int linear_revmap[];
>  };
>  
> +#define	IRQ_DOMAIN_FLAG_HIERARCHY	0x1
> +#define	IRQ_DOMAIN_FLAG_ARCH1		0x10000
> +
>  #ifdef CONFIG_IRQ_DOMAIN
>  struct irq_domain *__irq_domain_add(struct device_node *of_node, int size,
>  				    irq_hw_number_t hwirq_max, int direct_max,
> @@ -220,8 +241,73 @@ int irq_domain_xlate_onetwocell(struct i
>  			const u32 *intspec, unsigned int intsize,
>  			irq_hw_number_t *out_hwirq, unsigned int *out_type);
>  
> +/* V2 interfaces to support hierarchy IRQ domains. */
> +extern struct irq_data *irq_domain_get_irq_data(struct irq_domain *domain,
> +						unsigned int virq);
> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
> +extern int __irq_domain_alloc_irqs(struct irq_domain *domain, int irq_base,
> +				   unsigned int nr_irqs, int node, void *arg,
> +				   bool realloc);
> +extern void irq_domain_free_irqs(unsigned int virq, unsigned int nr_irqs);
> +extern void irq_domain_activate_irq(struct irq_data *irq_data);
> +extern void irq_domain_deactivate_irq(struct irq_data *irq_data);
> +
> +static inline int irq_domain_alloc_irqs(struct irq_domain *domain,
> +			unsigned int nr_irqs, int node, void *arg)
> +{
> +	return __irq_domain_alloc_irqs(domain, -1, nr_irqs, node, arg, false);
> +}
> +
> +extern int irq_domain_set_hwirq_and_chip(struct irq_domain *domain,
> +					 unsigned int virq,
> +					 irq_hw_number_t hwirq,
> +					 struct irq_chip *chip,
> +					 void *chip_data);
> +extern void irq_domain_reset_irq_data(struct irq_data *irq_data);
> +extern void irq_domain_free_irqs_common(struct irq_domain *domain,
> +					int virq, int nr_irqs);
> +extern void irq_domain_free_irqs_top(struct irq_domain *domain,
> +				     int virq, int nr_irqs);
> +
> +static inline int irq_domain_alloc_irqs_parent(struct irq_domain *domain,
> +				int irq_base, unsigned int nr_irqs, void *arg)
> +{
> +	if (domain->parent && domain->parent->ops->alloc)
> +		return domain->parent->ops->alloc(domain->parent, irq_base,
> +						  nr_irqs, arg);
> +	return -ENOSYS;
> +}
> +
> +static inline void irq_domain_free_irqs_parent(struct irq_domain *domain,
> +					int irq_base, unsigned int nr_irqs)
> +{
> +	if (domain->parent && domain->parent->ops->free)
> +		domain->parent->ops->free(domain->parent, irq_base, nr_irqs);
> +}
> +
> +static inline bool irq_domain_is_hierarchy(struct irq_domain *domain)
> +{
> +	return domain->flags & IRQ_DOMAIN_FLAG_HIERARCHY;
> +}
> +#else	/* CONFIG_IRQ_DOMAIN_HIERARCHY */
> +static inline void irq_domain_activate_irq(struct irq_data *data) { }
> +static inline void irq_domain_deactivate_irq(struct irq_data *data) { }
> +static inline int irq_domain_alloc_irqs(struct irq_domain *domain,
> +			unsigned int nr_irqs, int node, void *arg)
> +{
> +	return -1;
> +}
> +
> +static inline bool irq_domain_is_hierarchy(struct irq_domain *domain)
> +{
> +	return false;
> +}
> +#endif	/* CONFIG_IRQ_DOMAIN_HIERARCHY */
> +
>  #else /* CONFIG_IRQ_DOMAIN */
>  static inline void irq_dispose_mapping(unsigned int virq) { }
> +static inline void irq_domain_activate_irq(struct irq_data *data) { }
> +static inline void irq_domain_deactivate_irq(struct irq_data *data) { }
>  #endif /* !CONFIG_IRQ_DOMAIN */
>  
>  #endif /* _LINUX_IRQDOMAIN_H */
> Index: tip/kernel/irq/Kconfig
> ===================================================================
> --- tip.orig/kernel/irq/Kconfig
> +++ tip/kernel/irq/Kconfig
> @@ -55,6 +55,11 @@ config GENERIC_IRQ_CHIP
>  config IRQ_DOMAIN
>  	bool
>  
> +# Support for hierarchical irq domains
> +config IRQ_DOMAIN_HIERARCHY
> +	bool
> +	select IRQ_DOMAIN
> +
>  config HANDLE_DOMAIN_IRQ
>  	bool
>  
> Index: tip/kernel/irq/chip.c
> ===================================================================
> --- tip.orig/kernel/irq/chip.c
> +++ tip/kernel/irq/chip.c
> @@ -15,6 +15,7 @@
>  #include <linux/module.h>
>  #include <linux/interrupt.h>
>  #include <linux/kernel_stat.h>
> +#include <linux/irqdomain.h>
>  
>  #include <trace/events/irq.h>
>  
> @@ -178,6 +179,7 @@ int irq_startup(struct irq_desc *desc, b
>  	irq_state_clr_disabled(desc);
>  	desc->depth = 0;
>  
> +	irq_domain_activate_irq(&desc->irq_data);
>  	if (desc->irq_data.chip->irq_startup) {
>  		ret = desc->irq_data.chip->irq_startup(&desc->irq_data);
>  		irq_state_clr_masked(desc);
> @@ -199,6 +201,7 @@ void irq_shutdown(struct irq_desc *desc)
>  		desc->irq_data.chip->irq_disable(&desc->irq_data);
>  	else
>  		desc->irq_data.chip->irq_mask(&desc->irq_data);
> +	irq_domain_deactivate_irq(&desc->irq_data);
>  	irq_state_set_masked(desc);
>  }
>  
> Index: tip/kernel/irq/irqdomain.c
> ===================================================================
> --- tip.orig/kernel/irq/irqdomain.c
> +++ tip/kernel/irq/irqdomain.c
> @@ -23,6 +23,10 @@ static DEFINE_MUTEX(irq_domain_mutex);
>  static DEFINE_MUTEX(revmap_trees_mutex);
>  static struct irq_domain *irq_default_domain;
>  
> +static int irq_domain_alloc_descs(int virq, unsigned int nr_irqs,
> +				  irq_hw_number_t hwirq, int node);
> +static void irq_domain_check_hierarchy(struct irq_domain *domain);
> +
>  /**
>   * __irq_domain_add() - Allocate a new irq_domain data structure
>   * @of_node: optional device-tree node of the interrupt controller
> @@ -30,7 +34,7 @@ static struct irq_domain *irq_default_do
>   * @hwirq_max: Maximum number of interrupts supported by controller
>   * @direct_max: Maximum value of direct maps; Use ~0 for no limit; 0 for no
>   *              direct mapping
> - * @ops: map/unmap domain callbacks
> + * @ops: domain callbacks
>   * @host_data: Controller private data pointer
>   *
>   * Allocates and initialize and irq_domain structure.
> @@ -56,6 +60,7 @@ struct irq_domain *__irq_domain_add(stru
>  	domain->hwirq_max = hwirq_max;
>  	domain->revmap_size = size;
>  	domain->revmap_direct_max_irq = direct_max;
> +	irq_domain_check_hierarchy(domain);
>  
>  	mutex_lock(&irq_domain_mutex);
>  	list_add(&domain->link, &irq_domain_list);
> @@ -109,7 +114,7 @@ EXPORT_SYMBOL_GPL(irq_domain_remove);
>   * @first_irq: first number of irq block assigned to the domain,
>   *	pass zero to assign irqs on-the-fly. If first_irq is non-zero, then
>   *	pre-map all of the irqs in the domain to virqs starting at first_irq.
> - * @ops: map/unmap domain callbacks
> + * @ops: domain callbacks
>   * @host_data: Controller private data pointer
>   *
>   * Allocates an irq_domain, and optionally if first_irq is positive then also
> @@ -174,10 +179,8 @@ struct irq_domain *irq_domain_add_legacy
>  
>  	domain = __irq_domain_add(of_node, first_hwirq + size,
>  				  first_hwirq + size, 0, ops, host_data);
> -	if (!domain)
> -		return NULL;
> -
> -	irq_domain_associate_many(domain, first_irq, first_hwirq, size);
> +	if (domain)
> +		irq_domain_associate_many(domain, first_irq, first_hwirq, size);
>  
>  	return domain;
>  }
> @@ -388,7 +391,6 @@ EXPORT_SYMBOL_GPL(irq_create_direct_mapp
>  unsigned int irq_create_mapping(struct irq_domain *domain,
>  				irq_hw_number_t hwirq)
>  {
> -	unsigned int hint;
>  	int virq;
>  
>  	pr_debug("irq_create_mapping(0x%p, 0x%lx)\n", domain, hwirq);
> @@ -410,12 +412,8 @@ unsigned int irq_create_mapping(struct i
>  	}
>  
>  	/* Allocate a virtual interrupt number */
> -	hint = hwirq % nr_irqs;
> -	if (hint == 0)
> -		hint++;
> -	virq = irq_alloc_desc_from(hint, of_node_to_nid(domain->of_node));
> -	if (virq <= 0)
> -		virq = irq_alloc_desc_from(1, of_node_to_nid(domain->of_node));
> +	virq = irq_domain_alloc_descs(-1, 1, hwirq,
> +				      of_node_to_nid(domain->of_node));
>  	if (virq <= 0) {
>  		pr_debug("-> virq allocation failed\n");
>  		return 0;
> @@ -471,7 +469,7 @@ unsigned int irq_create_of_mapping(struc
>  	struct irq_domain *domain;
>  	irq_hw_number_t hwirq;
>  	unsigned int type = IRQ_TYPE_NONE;
> -	unsigned int virq;
> +	int virq;
>  
>  	domain = irq_data->np ? irq_find_host(irq_data->np) : irq_default_domain;
>  	if (!domain) {
> @@ -480,6 +478,11 @@ unsigned int irq_create_of_mapping(struc
>  		return 0;
>  	}
>  
> +	if (irq_domain_is_hierarchy(domain)) {
> +		virq = irq_domain_alloc_irqs(domain, 1, NUMA_NO_NODE, irq_data);
> +		return virq <= 0 ? 0 : virq;
> +	}
> +
>  	/* If domain has no translation, then we assume interrupt line */
>  	if (domain->ops->xlate == NULL)
>  		hwirq = irq_data->args[0];
> @@ -540,8 +543,8 @@ unsigned int irq_find_mapping(struct irq
>  		return 0;
>  
>  	if (hwirq < domain->revmap_direct_max_irq) {
> -		data = irq_get_irq_data(hwirq);
> -		if (data && (data->domain == domain) && (data->hwirq == hwirq))
> +		data = irq_domain_get_irq_data(domain, hwirq);
> +		if (data && data->hwirq == hwirq)
>  			return hwirq;
>  	}
>  
> @@ -709,3 +712,359 @@ const struct irq_domain_ops irq_domain_s
>  	.xlate = irq_domain_xlate_onetwocell,
>  };
>  EXPORT_SYMBOL_GPL(irq_domain_simple_ops);
> +
> +static int irq_domain_alloc_descs(int virq, unsigned int cnt,
> +				  irq_hw_number_t hwirq, int node)
> +{
> +	unsigned int hint;
> +
> +	if (virq >= 0) {
> +		virq = irq_alloc_descs(virq, virq, cnt, node);
> +	} else {
> +		hint = hwirq % nr_irqs;
> +		if (hint == 0)
> +			hint++;
> +		virq = irq_alloc_descs_from(hint, cnt, node);
> +		if (virq <= 0 && hint > 1)
> +			virq = irq_alloc_descs_from(1, cnt, node);
> +	}
> +
> +	return virq;
> +}
> +
> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
> +static void irq_domain_insert_irq(int virq)
> +{
> +	struct irq_data *data;
> +
> +	for (data = irq_get_irq_data(virq); data; data = data->parent_data) {
> +		struct irq_domain *domain = data->domain;
> +		irq_hw_number_t hwirq = data->hwirq;
> +
> +		if (hwirq < domain->revmap_size) {
> +			domain->linear_revmap[hwirq] = virq;
> +		} else {
> +			mutex_lock(&revmap_trees_mutex);
> +			radix_tree_insert(&domain->revmap_tree, hwirq, data);
> +			mutex_unlock(&revmap_trees_mutex);
> +		}
> +
> +		/* If not already assigned, give the domain the chip's name */
> +		if (!domain->name && data->chip)
> +			domain->name = data->chip->name;
> +	}
> +
> +	irq_clear_status_flags(virq, IRQ_NOREQUEST);
> +}
> +
> +static void irq_domain_remove_irq(int virq)
> +{
> +	struct irq_data *data;
> +
> +	irq_set_status_flags(virq, IRQ_NOREQUEST);
> +	irq_set_chip_and_handler(virq, NULL, NULL);
> +	synchronize_irq(virq);
> +	smp_mb();
> +
> +	for (data = irq_get_irq_data(virq); data; data = data->parent_data) {
> +		struct irq_domain *domain = data->domain;
> +		irq_hw_number_t hwirq = data->hwirq;
> +
> +		if (hwirq < domain->revmap_size) {
> +			domain->linear_revmap[hwirq] = 0;
> +		} else {
> +			mutex_lock(&revmap_trees_mutex);
> +			radix_tree_delete(&domain->revmap_tree, hwirq);
> +			mutex_unlock(&revmap_trees_mutex);
> +		}
> +	}
> +}
> +
> +static struct irq_data *irq_domain_insert_irq_data(struct irq_domain *domain,
> +						   struct irq_data *child)
> +{
> +	struct irq_data *irq_data;
> +
> +	irq_data = kzalloc_node(sizeof(*irq_data), GFP_KERNEL, child->node);
> +	if (irq_data) {
> +		child->parent_data = irq_data;
> +		irq_data->irq = child->irq;
> +		irq_data->node = child->node;
> +		irq_data->domain = domain;
> +	}
> +
> +	return irq_data;
> +}
> +
> +static void irq_domain_free_irq_data(unsigned int virq, unsigned int nr_irqs)
> +{
> +	int i;
> +	struct irq_data *irq_data, *tmp;
> +
> +	for (i = 0; i < nr_irqs; i++) {
> +		irq_data = irq_get_irq_data(virq + i);
> +		tmp = irq_data->parent_data;
> +		irq_data->parent_data = NULL;
> +		irq_data->domain = NULL;
> +
> +		while (tmp) {
> +			irq_data = tmp;
> +			tmp = tmp->parent_data;
> +			kfree(irq_data);
> +		}
> +	}
> +}
> +
> +static int irq_domain_alloc_irq_data(struct irq_domain *domain,
> +				     unsigned int virq, unsigned int nr_irqs)
> +{
> +	int i;
> +	struct irq_data *irq_data;
> +	struct irq_domain *parent;
> +
> +	/* The outermost irq_data is embedded in struct irq_desc */
> +	for (i = 0; i < nr_irqs; i++) {
> +		irq_data = irq_get_irq_data(virq + i);
> +		irq_data->domain = domain;
> +
> +		for (parent = domain->parent; parent; parent = parent->parent) {
> +			irq_data = irq_domain_insert_irq_data(parent, irq_data);
> +			if (!irq_data) {
> +				irq_domain_free_irq_data(virq, i + 1);
> +				return -ENOMEM;
> +			}
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +/**
> + * irq_domain_get_irq_data - Get irq_data associated with @virq and @domain
> + * @domain: domain to match
> + * @virq: IRQ number to get irq_data
> + */
> +struct irq_data *irq_domain_get_irq_data(struct irq_domain *domain,
> +					 unsigned int virq)
> +{
> +	struct irq_data *irq_data;
> +
> +	for (irq_data = irq_get_irq_data(virq); irq_data;
> +	     irq_data = irq_data->parent_data)
> +		if (irq_data->domain == domain)
> +			return irq_data;
> +
> +	return NULL;
> +}
> +
> +int irq_domain_set_hwirq_and_chip(struct irq_domain *domain, unsigned int virq,
> +				  irq_hw_number_t hwirq, struct irq_chip *chip,
> +				  void *chip_data)
> +{
> +	struct irq_data *irq_data = irq_domain_get_irq_data(domain, virq);
> +
> +	if (!irq_data)
> +		return -ENOENT;
> +
> +	irq_data->hwirq = hwirq;
> +	irq_data->chip = chip ? chip : &no_irq_chip;
> +	irq_data->chip_data = chip_data;
> +
> +	return 0;
> +}
> +
> +void irq_domain_reset_irq_data(struct irq_data *irq_data)
> +{
> +	irq_data->hwirq = 0;
> +	irq_data->chip = &no_irq_chip;
> +	irq_data->chip_data = NULL;
> +}
> +
> +void irq_domain_free_irqs_common(struct irq_domain *domain, int virq,
> +				 int nr_irqs)
> +{
> +	int i;
> +	struct irq_data *irq_data;
> +
> +	for (i = 0; i < nr_irqs; i++) {
> +		irq_data = irq_domain_get_irq_data(domain, virq + i);
> +		if (irq_data)
> +			irq_domain_reset_irq_data(irq_data);
> +	}
> +	irq_domain_free_irqs_parent(domain, virq, nr_irqs);
> +}
> +
> +void irq_domain_free_irqs_top(struct irq_domain *domain, int virq,
> +			      int nr_irqs)
> +{
> +	int i;
> +
> +	for (i = 0; i < nr_irqs; i++) {
> +		irq_set_handler_data(virq + i, NULL);
> +		irq_set_handler(virq + i, NULL);
> +	}
> +	irq_domain_free_irqs_common(domain, virq, nr_irqs);
> +}
> +
> +/**
> + * __irq_domain_alloc_irqs - Allocate IRQs from domain
> + * @domain: domain to allocate from
> + * @irq_base: allocate specified IRQ nubmer if irq_base >= 0
> + * @nr_irqs: number of IRQs to allocate
> + * @node: NUMA node id for memory allocation
> + * @arg: domain specific argument
> + * @realloc: IRQ descriptors have already been allocated if true
> + *
> + * Allocate IRQ numbers and initialized all data structures to support
> + * hiearchy IRQ domains.
> + * Parameter @realloc is mainly to support legacy IRQs.
> + * Returns error code or allocated IRQ number
> + *
> + * The whole process to setup an IRQ has been split into two steps.
> + * The first step, __irq_domain_alloc_irqs(), is to allocate IRQ
> + * descriptor and required hardware resources. The second step,
> + * irq_domain_activate_irq(), is to program hardwares with preallocated
> + * resources. In this way, it's easier to rollback when failing to
> + * allocate resources.
> + */
> +int __irq_domain_alloc_irqs(struct irq_domain *domain, int irq_base,
> +			    unsigned int nr_irqs, int node, void *arg,
> +			    bool realloc)
> +{
> +	int i, ret, virq;
> +
> +	if (domain == NULL) {
> +		domain = irq_default_domain;
> +		if (WARN(!domain, "domain is NULL; cannot allocate IRQ\n"))
> +			return -EINVAL;
> +	}
> +
> +	if (!domain->ops->alloc) {
> +		pr_debug("domain->ops->alloc() is NULL\n");
> +		return -ENOSYS;
> +	}
> +
> +	if (realloc && irq_base >= 0) {
> +		virq = irq_base;
> +	} else {
> +		virq = irq_domain_alloc_descs(irq_base, nr_irqs, 0, node);
> +		if (virq < 0) {
> +			pr_debug("cannot allocate IRQ(base %d, count %d)\n",
> +				 irq_base, nr_irqs);
> +			return virq;
> +		}
> +	}
> +
> +	if (irq_domain_alloc_irq_data(domain, virq, nr_irqs)) {
> +		pr_debug("cannot allocate memory for IRQ%d\n", virq);
> +		ret = -ENOMEM;
> +		goto out_free_desc;
> +	}
> +
> +	mutex_lock(&irq_domain_mutex);
> +	ret = domain->ops->alloc(domain, virq, nr_irqs, arg);
> +	if (ret < 0) {
> +		mutex_unlock(&irq_domain_mutex);
> +		goto out_free_irq_data;
> +	}
> +	for (i = 0; i < nr_irqs; i++)
> +		irq_domain_insert_irq(virq + i);
> +	mutex_unlock(&irq_domain_mutex);
> +
> +	return virq;
> +
> +out_free_irq_data:
> +	irq_domain_free_irq_data(virq, nr_irqs);
> +out_free_desc:
> +	irq_free_descs(virq, nr_irqs);
> +	return ret;
> +}
> +
> +/**
> + * irq_domain_free_irqs - Free IRQ number and associated data structures
> + * @virq: base IRQ number
> + * @nr_irqs: number of IRQs to free
> + */
> +void irq_domain_free_irqs(unsigned int virq, unsigned int nr_irqs)
> +{
> +	int i;
> +	struct irq_data *data = irq_get_irq_data(virq);
> +
> +	if (WARN(!data || !data->domain || !data->domain->ops->free,
> +		 "NULL pointer, cannot free irq\n"))
> +		return;
> +
> +	mutex_lock(&irq_domain_mutex);
> +	for (i = 0; i < nr_irqs; i++)
> +		irq_domain_remove_irq(virq + i);
> +	data->domain->ops->free(data->domain, virq, nr_irqs);
> +	mutex_unlock(&irq_domain_mutex);
> +
> +	irq_domain_free_irq_data(virq, nr_irqs);
> +	irq_free_descs(virq, nr_irqs);
> +}
> +
> +/**
> + * irq_domain_activate_irq - Call domain_ops->activate recursively to activate
> + *			     interrupt
> + * @irq_data: outermost irq_data associated with interrupt
> + *
> + * This is the second step to call domain_ops->activate to program interrupt
> + * controllers, so the interrupt could actually get delivered.
> + */
> +void irq_domain_activate_irq(struct irq_data *irq_data)
> +{
> +	if (irq_data && irq_data->domain) {
> +		struct irq_domain *domain = irq_data->domain;
> +
> +		if (irq_data->parent_data)
> +			irq_domain_activate_irq(irq_data->parent_data);
> +		if (domain->ops->activate)
> +			domain->ops->activate(domain, irq_data);
> +	}
> +}
> +
> +/**
> + * irq_domain_deactivate_irq - Call domain_ops->deactivate recursively to
> + *			       deactivate interrupt
> + * @irq_data: outermost irq_data associated with interrupt
> + *
> + * It calls domain_ops->deactivate to program interrupt controllers to disable
> + * interrupt delivery.
> + */
> +void irq_domain_deactivate_irq(struct irq_data *irq_data)
> +{
> +	if (irq_data && irq_data->domain) {
> +		struct irq_domain *domain = irq_data->domain;
> +
> +		if (domain->ops->deactivate)
> +			domain->ops->deactivate(domain, irq_data);
> +		if (irq_data->parent_data)
> +			irq_domain_deactivate_irq(irq_data->parent_data);
> +	}
> +}
> +
> +static void irq_domain_check_hierarchy(struct irq_domain *domain)
> +{
> +	/* Hierarchy irq_domains must implement callback alloc() */
> +	if (domain->ops->alloc)
> +		domain->flags |= IRQ_DOMAIN_FLAG_HIERARCHY;
> +}
> +#else	/* CONFIG_IRQ_DOMAIN_HIERARCHY */
> +/**
> + * irq_domain_get_irq_data - Get irq_data associated with @virq and @domain
> + * @domain: domain to match
> + * @virq: IRQ number to get irq_data
> + */
> +struct irq_data *irq_domain_get_irq_data(struct irq_domain *domain,
> +					 unsigned int virq)
> +{
> +	struct irq_data *irq_data = irq_get_irq_data(virq);
> +
> +	return (irq_data && irq_data->domain == domain) ? irq_data : NULL;
> +}
> +
> +static void irq_domain_check_hierarchy(struct irq_domain *domain)
> +{
> +}
> +#endif	/* CONFIG_IRQ_DOMAIN_HIERARCHY */
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> .
> 
> 

Re: [patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Thomas Gleixner]

On Mon, 24 Nov 2014, Yun Wu (Abel) wrote:
> Hi Thomas, Jiang,
> On 2014/11/12 21:42, Thomas Gleixner wrote:
> 
> > From: Jiang Liu <jiang.liu@linux.intel.com>
> > 
> [...]
> >  /* Number of irqs reserved for a legacy isa controller */
> >  #define NUM_ISA_INTERRUPTS	16
> > @@ -64,6 +66,16 @@ struct irq_domain_ops {
> >  	int (*xlate)(struct irq_domain *d, struct device_node *node,
> >  		     const u32 *intspec, unsigned int intsize,
> >  		     unsigned long *out_hwirq, unsigned int *out_type);
> > +
> > +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
> > +	/* extended V2 interfaces to support hierarchy irq_domains */
> > +	int (*alloc)(struct irq_domain *d, unsigned int virq,
> > +		     unsigned int nr_irqs, void *arg);
> > +	void (*free)(struct irq_domain *d, unsigned int virq,
> > +		     unsigned int nr_irqs);
> > +	void (*activate)(struct irq_domain *d, struct irq_data *irq_data);
> > +	void (*deactivate)(struct irq_domain *d, struct irq_data *irq_data);
> 
> What's the usage of the parameter domain reference in activate/deactivate?
> I think the purpose of the two callbacks is to activate/deactivate the
> irq_data->hwirq in irq_data->domain. If so, the first parameter @domain is
> required to be equal to irq_data->domain (which makes @domain useless).
> Besides, the main responsibility of interrupt domains is to manage mappings
> between hardware and linux interrupt numbers, so would it be better if move
> the two callbacks into struct irq_chip?

No. It's not a function of the irq_chip to activate/deactivate a
hierarchy. As I explained you before:

The existing irqdomain code maps between hardware and virtual
interrupts and thereby activates the interrupt in hardware.

In the hierarchical case we do not touch the hardware in the
allocation step, so we need to activate the allocated interrupt in the
hardware before we can use it. And that's clearly a domain interface
not a irq chip issue.

Thanks,

	tglx

Re: [patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Yun Wu (Abel)]

On 2014/11/24 21:13, Thomas Gleixner wrote:

> On Mon, 24 Nov 2014, Yun Wu (Abel) wrote:
>> Hi Thomas, Jiang,
>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>
>>> From: Jiang Liu <jiang.liu@linux.intel.com>
>>>
>> [...]
>>>  /* Number of irqs reserved for a legacy isa controller */
>>>  #define NUM_ISA_INTERRUPTS	16
>>> @@ -64,6 +66,16 @@ struct irq_domain_ops {
>>>  	int (*xlate)(struct irq_domain *d, struct device_node *node,
>>>  		     const u32 *intspec, unsigned int intsize,
>>>  		     unsigned long *out_hwirq, unsigned int *out_type);
>>> +
>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>> +	/* extended V2 interfaces to support hierarchy irq_domains */
>>> +	int (*alloc)(struct irq_domain *d, unsigned int virq,
>>> +		     unsigned int nr_irqs, void *arg);
>>> +	void (*free)(struct irq_domain *d, unsigned int virq,
>>> +		     unsigned int nr_irqs);
>>> +	void (*activate)(struct irq_domain *d, struct irq_data *irq_data);
>>> +	void (*deactivate)(struct irq_domain *d, struct irq_data *irq_data);
>>
>> What's the usage of the parameter domain reference in activate/deactivate?
>> I think the purpose of the two callbacks is to activate/deactivate the
>> irq_data->hwirq in irq_data->domain. If so, the first parameter @domain is
>> required to be equal to irq_data->domain (which makes @domain useless).
>> Besides, the main responsibility of interrupt domains is to manage mappings
>> between hardware and linux interrupt numbers, so would it be better if move
>> the two callbacks into struct irq_chip?
> 
> No. It's not a function of the irq_chip to activate/deactivate a
> hierarchy. As I explained you before:
> 
> The existing irqdomain code maps between hardware and virtual
> interrupts and thereby activates the interrupt in hardware.
> 
> In the hierarchical case we do not touch the hardware in the
> allocation step, so we need to activate the allocated interrupt in the
> hardware before we can use it. And that's clearly a domain interface
> not a irq chip issue.
> 

Makes sense, now the interrupt domain seems to be the best place.
And when the @domain parameter can be really useful? I haven't see
anyone using it so far.

Thanks,
	Abel

Re: [patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Jiang Liu]

On 2014/11/24 22:01, Yun Wu (Abel) wrote:
> On 2014/11/24 21:13, Thomas Gleixner wrote:
> 
>> On Mon, 24 Nov 2014, Yun Wu (Abel) wrote:
>>> Hi Thomas, Jiang,
>>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>>
>>>> From: Jiang Liu <jiang.liu@linux.intel.com>
>>>>
>>> [...]
>>>>  /* Number of irqs reserved for a legacy isa controller */
>>>>  #define NUM_ISA_INTERRUPTS	16
>>>> @@ -64,6 +66,16 @@ struct irq_domain_ops {
>>>>  	int (*xlate)(struct irq_domain *d, struct device_node *node,
>>>>  		     const u32 *intspec, unsigned int intsize,
>>>>  		     unsigned long *out_hwirq, unsigned int *out_type);
>>>> +
>>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>>> +	/* extended V2 interfaces to support hierarchy irq_domains */
>>>> +	int (*alloc)(struct irq_domain *d, unsigned int virq,
>>>> +		     unsigned int nr_irqs, void *arg);
>>>> +	void (*free)(struct irq_domain *d, unsigned int virq,
>>>> +		     unsigned int nr_irqs);
>>>> +	void (*activate)(struct irq_domain *d, struct irq_data *irq_data);
>>>> +	void (*deactivate)(struct irq_domain *d, struct irq_data *irq_data);
>>>
>>> What's the usage of the parameter domain reference in activate/deactivate?
>>> I think the purpose of the two callbacks is to activate/deactivate the
>>> irq_data->hwirq in irq_data->domain. If so, the first parameter @domain is
>>> required to be equal to irq_data->domain (which makes @domain useless).
>>> Besides, the main responsibility of interrupt domains is to manage mappings
>>> between hardware and linux interrupt numbers, so would it be better if move
>>> the two callbacks into struct irq_chip?
>>
>> No. It's not a function of the irq_chip to activate/deactivate a
>> hierarchy. As I explained you before:
>>
>> The existing irqdomain code maps between hardware and virtual
>> interrupts and thereby activates the interrupt in hardware.
>>
>> In the hierarchical case we do not touch the hardware in the
>> allocation step, so we need to activate the allocated interrupt in the
>> hardware before we can use it. And that's clearly a domain interface
>> not a irq chip issue.
>>
> 
> Makes sense, now the interrupt domain seems to be the best place.
> And when the @domain parameter can be really useful? I haven't see
> anyone using it so far.
We will use it for IOAPIC on x86, as below:
void mp_irqdomain_deactivate(struct irq_domain *domain,
                             struct irq_data *irq_data)
{
        ioapic_mask_entry(mp_irqdomain_ioapic_idx(domain),
                          (int)irq_data->hwirq);
}

>From an object oriented point of view, we pass the object as the
first parameter. It's true that we could retrieve domain from
irq_data->domain instead of explicitly passing it in, but that
will cause irqdomain interfaces depends on irq_data, not sounds
a good situation:)
Thanks!
Gerry
> 
> Thanks,
> 	Abel
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Yun Wu (Abel)]

On 2014/11/24 22:11, Jiang Liu wrote:

> On 2014/11/24 22:01, Yun Wu (Abel) wrote:
>> On 2014/11/24 21:13, Thomas Gleixner wrote:
>>
>>> On Mon, 24 Nov 2014, Yun Wu (Abel) wrote:
>>>> Hi Thomas, Jiang,
>>>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>>>
>>>>> From: Jiang Liu <jiang.liu@linux.intel.com>
>>>>>
>>>> [...]
>>>>>  /* Number of irqs reserved for a legacy isa controller */
>>>>>  #define NUM_ISA_INTERRUPTS	16
>>>>> @@ -64,6 +66,16 @@ struct irq_domain_ops {
>>>>>  	int (*xlate)(struct irq_domain *d, struct device_node *node,
>>>>>  		     const u32 *intspec, unsigned int intsize,
>>>>>  		     unsigned long *out_hwirq, unsigned int *out_type);
>>>>> +
>>>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>>>> +	/* extended V2 interfaces to support hierarchy irq_domains */
>>>>> +	int (*alloc)(struct irq_domain *d, unsigned int virq,
>>>>> +		     unsigned int nr_irqs, void *arg);
>>>>> +	void (*free)(struct irq_domain *d, unsigned int virq,
>>>>> +		     unsigned int nr_irqs);
>>>>> +	void (*activate)(struct irq_domain *d, struct irq_data *irq_data);
>>>>> +	void (*deactivate)(struct irq_domain *d, struct irq_data *irq_data);
>>>>
>>>> What's the usage of the parameter domain reference in activate/deactivate?
>>>> I think the purpose of the two callbacks is to activate/deactivate the
>>>> irq_data->hwirq in irq_data->domain. If so, the first parameter @domain is
>>>> required to be equal to irq_data->domain (which makes @domain useless).
>>>> Besides, the main responsibility of interrupt domains is to manage mappings
>>>> between hardware and linux interrupt numbers, so would it be better if move
>>>> the two callbacks into struct irq_chip?
>>>
>>> No. It's not a function of the irq_chip to activate/deactivate a
>>> hierarchy. As I explained you before:
>>>
>>> The existing irqdomain code maps between hardware and virtual
>>> interrupts and thereby activates the interrupt in hardware.
>>>
>>> In the hierarchical case we do not touch the hardware in the
>>> allocation step, so we need to activate the allocated interrupt in the
>>> hardware before we can use it. And that's clearly a domain interface
>>> not a irq chip issue.
>>>
>>
>> Makes sense, now the interrupt domain seems to be the best place.
>> And when the @domain parameter can be really useful? I haven't see
>> anyone using it so far.
> We will use it for IOAPIC on x86, as below:
> void mp_irqdomain_deactivate(struct irq_domain *domain,
>                              struct irq_data *irq_data)
> {
>         ioapic_mask_entry(mp_irqdomain_ioapic_idx(domain),
>                           (int)irq_data->hwirq);
> }
> 
>>From an object oriented point of view, we pass the object as the
> first parameter. It's true that we could retrieve domain from
> irq_data->domain instead of explicitly passing it in, but that
> will cause irqdomain interfaces depends on irq_data, not sounds
> a good situation:)

Hi Gerry,

Is there any possibility that domain doesn't equal to irq_data->domain?
I'm a little confused..

Thanks,
	Abel

Re: [patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Thomas Gleixner]

On Mon, 24 Nov 2014, Yun Wu (Abel) wrote:
> On 2014/11/24 21:13, Thomas Gleixner wrote:
> > In the hierarchical case we do not touch the hardware in the
> > allocation step, so we need to activate the allocated interrupt in the
> > hardware before we can use it. And that's clearly a domain interface
> > not a irq chip issue.
> > 
> 
> Makes sense, now the interrupt domain seems to be the best place.
> And when the @domain parameter can be really useful? I haven't see
> anyone using it so far.

All irqdomain callbacks take the domain pointer as their first
parameter. So for consistency sake we made it that way.

You can argue in circles about whether the domain argument could be
removed. It's going to stay for now as it does not matter at all.

Thanks,

	tglx

Re: [patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Jiang Liu]

On 2014/11/24 22:19, Yun Wu (Abel) wrote:
> On 2014/11/24 22:11, Jiang Liu wrote:
> 
>> On 2014/11/24 22:01, Yun Wu (Abel) wrote:
>>> On 2014/11/24 21:13, Thomas Gleixner wrote:
>>>
>>>> On Mon, 24 Nov 2014, Yun Wu (Abel) wrote:
>>>>> Hi Thomas, Jiang,
>>>>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>>>>
>>>>>> From: Jiang Liu <jiang.liu@linux.intel.com>
>>>>>>
>>>>> [...]
>>>>>>  /* Number of irqs reserved for a legacy isa controller */
>>>>>>  #define NUM_ISA_INTERRUPTS	16
>>>>>> @@ -64,6 +66,16 @@ struct irq_domain_ops {
>>>>>>  	int (*xlate)(struct irq_domain *d, struct device_node *node,
>>>>>>  		     const u32 *intspec, unsigned int intsize,
>>>>>>  		     unsigned long *out_hwirq, unsigned int *out_type);
>>>>>> +
>>>>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>>>>> +	/* extended V2 interfaces to support hierarchy irq_domains */
>>>>>> +	int (*alloc)(struct irq_domain *d, unsigned int virq,
>>>>>> +		     unsigned int nr_irqs, void *arg);
>>>>>> +	void (*free)(struct irq_domain *d, unsigned int virq,
>>>>>> +		     unsigned int nr_irqs);
>>>>>> +	void (*activate)(struct irq_domain *d, struct irq_data *irq_data);
>>>>>> +	void (*deactivate)(struct irq_domain *d, struct irq_data *irq_data);
>>>>>
>>>>> What's the usage of the parameter domain reference in activate/deactivate?
>>>>> I think the purpose of the two callbacks is to activate/deactivate the
>>>>> irq_data->hwirq in irq_data->domain. If so, the first parameter @domain is
>>>>> required to be equal to irq_data->domain (which makes @domain useless).
>>>>> Besides, the main responsibility of interrupt domains is to manage mappings
>>>>> between hardware and linux interrupt numbers, so would it be better if move
>>>>> the two callbacks into struct irq_chip?
>>>>
>>>> No. It's not a function of the irq_chip to activate/deactivate a
>>>> hierarchy. As I explained you before:
>>>>
>>>> The existing irqdomain code maps between hardware and virtual
>>>> interrupts and thereby activates the interrupt in hardware.
>>>>
>>>> In the hierarchical case we do not touch the hardware in the
>>>> allocation step, so we need to activate the allocated interrupt in the
>>>> hardware before we can use it. And that's clearly a domain interface
>>>> not a irq chip issue.
>>>>
>>>
>>> Makes sense, now the interrupt domain seems to be the best place.
>>> And when the @domain parameter can be really useful? I haven't see
>>> anyone using it so far.
>> We will use it for IOAPIC on x86, as below:
>> void mp_irqdomain_deactivate(struct irq_domain *domain,
>>                              struct irq_data *irq_data)
>> {
>>         ioapic_mask_entry(mp_irqdomain_ioapic_idx(domain),
>>                           (int)irq_data->hwirq);
>> }
>>
>> >From an object oriented point of view, we pass the object as the
>> first parameter. It's true that we could retrieve domain from
>> irq_data->domain instead of explicitly passing it in, but that
>> will cause irqdomain interfaces depends on irq_data, not sounds
>> a good situation:)
> 
> Hi Gerry,
> 
> Is there any possibility that domain doesn't equal to irq_data->domain?
> I'm a little confused..
Hi Yun,
	Currently they are always the same, but we don't want irqdomain
interfaces make assumption of struct irq_data. If it will bring big
performance improvement, we will try to kill the first parameter,
otherwise we may prefer keeping irqdomain interfaces clear.
Thanks!
Gerry

Re: [patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Yun Wu (Abel)]

On 2014/11/24 22:32, Thomas Gleixner wrote:

> On Mon, 24 Nov 2014, Yun Wu (Abel) wrote:
>> On 2014/11/24 21:13, Thomas Gleixner wrote:
>>> In the hierarchical case we do not touch the hardware in the
>>> allocation step, so we need to activate the allocated interrupt in the
>>> hardware before we can use it. And that's clearly a domain interface
>>> not a irq chip issue.
>>>
>>
>> Makes sense, now the interrupt domain seems to be the best place.
>> And when the @domain parameter can be really useful? I haven't see
>> anyone using it so far.
> 
> All irqdomain callbacks take the domain pointer as their first
> parameter. So for consistency sake we made it that way.
> 
> You can argue in circles about whether the domain argument could be
> removed. It's going to stay for now as it does not matter at all.
> 

Yes, you are right.

Thanks,
	Abel

Re: [patch 01/16] irqdomain: Introduce new interfaces to support hierarchy irqdomains

[Yun Wu (Abel)]

On 2014/11/24 22:33, Jiang Liu wrote:

> On 2014/11/24 22:19, Yun Wu (Abel) wrote:
>> On 2014/11/24 22:11, Jiang Liu wrote:
>>
>>> On 2014/11/24 22:01, Yun Wu (Abel) wrote:
>>>> On 2014/11/24 21:13, Thomas Gleixner wrote:
>>>>
>>>>> On Mon, 24 Nov 2014, Yun Wu (Abel) wrote:
>>>>>> Hi Thomas, Jiang,
>>>>>> On 2014/11/12 21:42, Thomas Gleixner wrote:
>>>>>>
>>>>>>> From: Jiang Liu <jiang.liu@linux.intel.com>
>>>>>>>
>>>>>> [...]
>>>>>>>  /* Number of irqs reserved for a legacy isa controller */
>>>>>>>  #define NUM_ISA_INTERRUPTS	16
>>>>>>> @@ -64,6 +66,16 @@ struct irq_domain_ops {
>>>>>>>  	int (*xlate)(struct irq_domain *d, struct device_node *node,
>>>>>>>  		     const u32 *intspec, unsigned int intsize,
>>>>>>>  		     unsigned long *out_hwirq, unsigned int *out_type);
>>>>>>> +
>>>>>>> +#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
>>>>>>> +	/* extended V2 interfaces to support hierarchy irq_domains */
>>>>>>> +	int (*alloc)(struct irq_domain *d, unsigned int virq,
>>>>>>> +		     unsigned int nr_irqs, void *arg);
>>>>>>> +	void (*free)(struct irq_domain *d, unsigned int virq,
>>>>>>> +		     unsigned int nr_irqs);
>>>>>>> +	void (*activate)(struct irq_domain *d, struct irq_data *irq_data);
>>>>>>> +	void (*deactivate)(struct irq_domain *d, struct irq_data *irq_data);
>>>>>>
>>>>>> What's the usage of the parameter domain reference in activate/deactivate?
>>>>>> I think the purpose of the two callbacks is to activate/deactivate the
>>>>>> irq_data->hwirq in irq_data->domain. If so, the first parameter @domain is
>>>>>> required to be equal to irq_data->domain (which makes @domain useless).
>>>>>> Besides, the main responsibility of interrupt domains is to manage mappings
>>>>>> between hardware and linux interrupt numbers, so would it be better if move
>>>>>> the two callbacks into struct irq_chip?
>>>>>
>>>>> No. It's not a function of the irq_chip to activate/deactivate a
>>>>> hierarchy. As I explained you before:
>>>>>
>>>>> The existing irqdomain code maps between hardware and virtual
>>>>> interrupts and thereby activates the interrupt in hardware.
>>>>>
>>>>> In the hierarchical case we do not touch the hardware in the
>>>>> allocation step, so we need to activate the allocated interrupt in the
>>>>> hardware before we can use it. And that's clearly a domain interface
>>>>> not a irq chip issue.
>>>>>
>>>>
>>>> Makes sense, now the interrupt domain seems to be the best place.
>>>> And when the @domain parameter can be really useful? I haven't see
>>>> anyone using it so far.
>>> We will use it for IOAPIC on x86, as below:
>>> void mp_irqdomain_deactivate(struct irq_domain *domain,
>>>                              struct irq_data *irq_data)
>>> {
>>>         ioapic_mask_entry(mp_irqdomain_ioapic_idx(domain),
>>>                           (int)irq_data->hwirq);
>>> }
>>>
>>> >From an object oriented point of view, we pass the object as the
>>> first parameter. It's true that we could retrieve domain from
>>> irq_data->domain instead of explicitly passing it in, but that
>>> will cause irqdomain interfaces depends on irq_data, not sounds
>>> a good situation:)
>>
>> Hi Gerry,
>>
>> Is there any possibility that domain doesn't equal to irq_data->domain?
>> I'm a little confused..
> Hi Yun,
> 	Currently they are always the same, but we don't want irqdomain
> interfaces make assumption of struct irq_data. If it will bring big
> performance improvement, we will try to kill the first parameter,
> otherwise we may prefer keeping irqdomain interfaces clear.

OK, let's keep it as is. :)

Thanks,
	Abel

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

[-- Attachment #1: Type: text/plain, Size: 2694 bytes --]

On 2014/11/19 19:11, Thomas Gleixner wrote:

> On Wed, 19 Nov 2014, Yun Wu (Abel) wrote:
>> On 2014/11/19 1:21, Marc Zyngier wrote:
>>> This is why the framework gives you the opportunity to provide methods
>>> that:
>>> - compose the message
>>> - program the message into the device
>>>
>>> None of that has to be PCI specific, and gives you a clean
>>> abstraction. The framework only gives you a number of shortcuts for PCI
>>> MSI, because that's what most people care about.
>>>
>>
>> Indeed, and I never said Jiang's patches don't work, I was just thinking
>> that they were not that perfect.
> 
> But your magic extra layer of indirection is perfect? It's not, it
> just violates sane layering in order to support braindead hardware
> designs.
> 

Hi Thomas, Gerry and Marc,

I spent last two weeks implementing and testing my original idea about making
the sub domains generic, based on stacked domain feature. Now it comes real,
please see the attached patch.

With this patch applied, I think things will get easier.

For drivers of interrupt controllers, they need to implement:
a) struct irq_chip, gets associated in domain's map/alloc callback
b) struct irq_domain, with corresponding operations
c) create sub generic MBI domain of IRQ domain to deal with all MBI types.
This changes almost nothing of the current code.

For drivers of MBI-capable devices, they need to implement:
a) MBI operations, like mask/unmask or setting message.
This will remove current ugly arch-specific code by organizing the device
behavior into a generic structure used in generic MBI layer.

The MBI generic code will build the bridge between the two groups. So when a
new driver need to implement, either controller's or endpoint's, just code
the corresponding 'need' described above with no more work to do.

This patch (also with several other patches) is tested on Hisilicon ARM64 SoC,
with non PCI devices capable of message based interrupts. The PCI part is not
tested because it needs large refactoring work to do. So yes, the testing work
is not sufficient, but I think the patch is enough to present what I really
wanted to express. :)

A new term introduced by the patch named Message Based Interrupt (MBI) is used
for presenting the generic MSIs (which does help me avoid conflicting with the
existing code). Actually the new name is proposed by Marc several months ago,
suggesting that MSI implies too much about PCI. I think it's a good idea to use
MBI in generic code and make the MSI/MSI-x a wrapper of MBI inside the PCI core.
Anyway, naming is not the key point yet.

Finally, yes, my thoughts is not perfect, but I am just trying to make it better.

Best regards and thanks,
	Abel

[-- Attachment #2: 0001-MBI-initial-support-for-message-based-interrupts.patch --]
[-- Type: text/plain, Size: 16844 bytes --]

>From 64c5440f685440bb7d92ab716013f9f54f21bca2 Mon Sep 17 00:00:00 2001
From: Yun Wu <wuyun.wu@huawei.com>
Date: Wed, 10 Dec 2014 10:32:58 +0800
Subject: [PATCH] MBI: initial support for message based interrupts

This patch provides initial support for Message Based Interrupt (MBI),
which is a write from the device to a special address which causes an
interrupt to be received by the CPU.

MBI is a generic mechanism not specific to any architectures or any
subsystems. MSI/MSI-X defined in PCI specification are special MBIs.

Signed-off-by: Yun Wu <wuyun.wu@huawei.com>
---
 arch/arm64/Kconfig     |   1 +
 include/linux/device.h |   3 +
 include/linux/irq.h    |  22 ++++
 include/linux/mbi.h    |  95 ++++++++++++++++++
 kernel/irq/Kconfig     |   5 +
 kernel/irq/Makefile    |   1 +
 kernel/irq/chip.c      |  66 ++++++++++++
 kernel/irq/mbi.c       | 260 +++++++++++++++++++++++++++++++++++++++++++++++++
 8 files changed, 453 insertions(+)
 create mode 100644 include/linux/mbi.h
 create mode 100644 kernel/irq/mbi.c

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 1f49c288..ef55541 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -13,6 +13,7 @@ config ARM64
 	select ARM_GIC
 	select AUDIT_ARCH_COMPAT_GENERIC
 	select ARM_GIC_V3
+	select MBI
 	select ARM_GIC_V3_ITS if PCI_MSI
 	select BUILDTIME_EXTABLE_SORT
 	select CLONE_BACKWARDS
diff --git a/include/linux/device.h b/include/linux/device.h
index ce1f2160..e0618c2 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -715,6 +715,7 @@ struct acpi_dev_node {
  * 		gone away. This should be set by the allocator of the
  * 		device (i.e. the bus driver that discovered the device).
  * @iommu_group: IOMMU group the device belongs to.
+ * @mbi:	Pointer to the data of message based interrupts.
  *
  * @offline_disabled: If set, the device is permanently online.
  * @offline:	Set after successful invocation of bus type's .offline().
@@ -794,6 +795,8 @@ struct device {
 	void	(*release)(struct device *dev);
 	struct iommu_group	*iommu_group;
 
+	struct mbi_data		*mbi;
+
 	bool			offline_disabled:1;
 	bool			offline:1;
 };
diff --git a/include/linux/irq.h b/include/linux/irq.h
index 8badf34..92e25e6 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -28,6 +28,7 @@
 
 struct seq_file;
 struct module;
+struct mbi_msg;
 struct msi_msg;
 
 /*
@@ -120,6 +121,7 @@ enum {
 	IRQ_SET_MASK_OK_DONE,
 };
 
+struct mbi_desc;
 struct msi_desc;
 struct irq_domain;
 
@@ -139,6 +141,7 @@ struct irq_domain;
  * @handler_data:	per-IRQ data for the irq_chip methods
  * @chip_data:		platform-specific per-chip private data for the chip
  *			methods, to allow shared chip implementations
+ * @mbi_desc:		MBI descriptor
  * @msi_desc:		MSI descriptor
  * @affinity:		IRQ affinity on SMP
  *
@@ -159,6 +162,7 @@ struct irq_data {
 #endif
 	void			*handler_data;
 	void			*chip_data;
+	struct mbi_desc		*mbi_desc;
 	struct msi_desc		*msi_desc;
 	cpumask_var_t		affinity;
 };
@@ -323,6 +327,7 @@ static inline irq_hw_number_t irqd_to_hwirq(struct irq_data *d)
  *				irq_request_resources
  * @irq_compose_msi_msg:	optional to compose message content for MSI
  * @irq_write_msi_msg:	optional to write message content for MSI
+ * @irq_compose_msg:	optional to write message content for MBI
  * @flags:		chip specific flags
  */
 struct irq_chip {
@@ -362,6 +367,8 @@ struct irq_chip {
 	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
 	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
 
+	void		(*irq_compose_msg)(struct irq_data *data, struct mbi_msg *msg);
+
 	unsigned long	flags;
 };
 
@@ -567,10 +574,14 @@ extern int irq_set_chip(unsigned int irq, struct irq_chip *chip);
 extern int irq_set_handler_data(unsigned int irq, void *data);
 extern int irq_set_chip_data(unsigned int irq, void *data);
 extern int irq_set_irq_type(unsigned int irq, unsigned int type);
+extern int irq_set_mbi_desc(unsigned int irq, struct mbi_desc *entry);
+extern int irq_set_mbi_desc_range(unsigned int irq, struct mbi_desc *entry,
+				  unsigned int count);
 extern int irq_set_msi_desc(unsigned int irq, struct msi_desc *entry);
 extern int irq_set_msi_desc_off(unsigned int irq_base, unsigned int irq_offset,
 				struct msi_desc *entry);
 extern struct irq_data *irq_get_irq_data(unsigned int irq);
+extern int irq_compose_mbi_msg(struct irq_data *data, struct mbi_msg *msg);
 
 static inline struct irq_chip *irq_get_chip(unsigned int irq)
 {
@@ -605,6 +616,17 @@ static inline void *irq_data_get_irq_handler_data(struct irq_data *d)
 	return d->handler_data;
 }
 
+static inline struct mbi_desc *irq_get_mbi_desc(unsigned int irq)
+{
+	struct irq_data *d = irq_get_irq_data(irq);
+	return d ? d->mbi_desc : NULL;
+}
+
+static inline struct mbi_desc *irq_data_get_mbi(struct irq_data *d)
+{
+	return d->mbi_desc;
+}
+
 static inline struct msi_desc *irq_get_msi_desc(unsigned int irq)
 {
 	struct irq_data *d = irq_get_irq_data(irq);
diff --git a/include/linux/mbi.h b/include/linux/mbi.h
new file mode 100644
index 0000000..075a453
--- /dev/null
+++ b/include/linux/mbi.h
@@ -0,0 +1,95 @@
+#ifndef _LINUX_MBI_H
+#define _LINUX_MBI_H
+
+#include <linux/device.h>
+#include <linux/irq.h>
+#include <linux/irqdomain.h>
+
+struct mbi_data;
+
+/**
+ * struct mbi_msg - MBI message descriptor
+ *
+ * @address_lo:	lower 32bit value of MBI address register
+ * @address_hi:	higher 32bit value of MBI address register
+ * @data:	data value of MBI data register
+ */
+struct mbi_msg {
+	u32	address_lo;
+	u32	address_hi;
+	u32	data;
+};
+
+/**
+ * struct mbi_desc - Message Based Interrupt (MBI) descriptor
+ *
+ * @entry:	element in device MBI list
+ * @mbi:	the device MBI data
+ * @data:	message-specific unique data to identify an MBI
+ * @msg:	message registers info of the MBI
+ * @irq:	base linux interrupt number of the MBI
+ * @nvec:	number of interrupts supported
+ */
+struct mbi_desc {
+	struct list_head	entry;
+	struct mbi_data		*mbi;
+	void			*data;
+	struct mbi_msg		msg;
+	unsigned int		irq;
+	unsigned int		nvec;
+};
+
+/**
+ * struct mbi_ops - MBI functions of MBI-capable device
+ *
+ * @write_msg:	write message registers for an MBI
+ * @mask_irq:	mask an MBI interrupt
+ * @unmask_irq:	unmask an MBI interrupt
+ */
+struct mbi_ops {
+	void	(*write_msg)(struct mbi_desc *desc, struct mbi_msg *msg);
+	void	(*mask_irq)(struct mbi_desc *desc, int id);
+	void	(*unmask_irq)(struct mbi_desc *desc, int id);
+};
+
+/**
+ * struct mbi_data - MBI information of a device
+ *
+ * @dev:	the device owned the MBI data
+ * @ops:	operations of the MBI-capable device
+ * @lock:	spinlock for @mbi_list
+ * @mbi_list:	list of MBI descriptors
+ *
+ * One MBI data for each MBI-capable device, holding device specific
+ * information.
+ */
+struct mbi_data {
+	struct device		*dev;
+	struct mbi_ops		*ops;
+	raw_spinlock_t		lock;
+	struct list_head	mbi_list;
+};
+
+/**
+ * struct mbi_domain_info - Infomation prepared for MBI domain
+ *
+ * @dev:	the device owned the MBI data
+ * @ops:	operations of the MBI-capable device
+ * @data:	message-specific data to identify an MBI
+ * @nvec:	maximum number of supported interrupts by the device
+ *
+ * The MBI-capable devices should provide this to their parent MBI
+ * domains when initializing MBIs.
+ */
+struct mbi_domain_info {
+	struct device		*dev;
+	struct mbi_ops		*ops;
+	void			*data;
+	unsigned int		nvec;
+};
+
+/* Create hierarchy MBI domain for interrupt controllers */
+struct irq_domain *mbi_create_irq_domain(struct device_node *of_node,
+					 struct irq_domain *parent);
+
+#endif /* _LINUX_MBI_H */
diff --git a/kernel/irq/Kconfig b/kernel/irq/Kconfig
index 9a76e3b..d1071cf 100644
--- a/kernel/irq/Kconfig
+++ b/kernel/irq/Kconfig
@@ -60,6 +60,11 @@ config IRQ_DOMAIN_HIERARCHY
 	bool
 	select IRQ_DOMAIN
 
+# Support for Message Based Interrupt (MBI)
+config MBI
+	bool "Support Message Based Interrupt (MBI)"
+	select IRQ_DOMAIN_HIERARCHY
+
 # Generic MSI interrupt support
 config GENERIC_MSI_IRQ
 	bool
diff --git a/kernel/irq/Makefile b/kernel/irq/Makefile
index d121235..1e5f4b8 100644
--- a/kernel/irq/Makefile
+++ b/kernel/irq/Makefile
@@ -6,4 +6,5 @@ obj-$(CONFIG_IRQ_DOMAIN) += irqdomain.o
 obj-$(CONFIG_PROC_FS) += proc.o
 obj-$(CONFIG_GENERIC_PENDING_IRQ) += migration.o
 obj-$(CONFIG_PM_SLEEP) += pm.o
+obj-$(CONFIG_MBI) += mbi.o
 obj-$(CONFIG_GENERIC_MSI_IRQ) += msi.o
diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index 6f1c7a5..756d277 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -90,6 +90,45 @@ int irq_set_handler_data(unsigned int irq, void *data)
 EXPORT_SYMBOL(irq_set_handler_data);
 
 /**
+ *	irq_set_mbi_desc - set MBI descriptor for an irq
+ *	@irq:	Interrupt number
+ *	@entry:	Pointer to MBI descriptor
+ */
+int irq_set_mbi_desc(unsigned int irq, struct mbi_desc *entry)
+{
+	unsigned long flags;
+	struct irq_desc *desc = irq_get_desc_lock(irq, &flags, IRQ_GET_DESC_CHECK_GLOBAL);
+
+	if (!desc)
+		return -EINVAL;
+	desc->irq_data.mbi_desc = entry;
+	irq_put_desc_unlock(desc, flags);
+	return 0;
+}
+EXPORT_SYMBOL(irq_set_mbi_desc);
+
+/**
+ *	irq_set_mbi_desc_range - set MBI descriptor for a range of irqs
+ *	@irq:	Interrupt number
+ *	@entry:	Pointer to MBI descriptor
+ *	@count:	Number of interrupts to be set
+ */
+int irq_set_mbi_desc_range(unsigned int irq, struct mbi_desc *entry,
+			   unsigned int count)
+{
+	int i, ret = 0;
+
+	for (i = 0; i < count; i++) {
+		ret = irq_set_mbi_desc(irq + i, entry);
+		if (ret)
+			break;
+	}
+
+	return ret;
+}
+EXPORT_SYMBOL(irq_set_mbi_desc_range);
+
+/**
  *	irq_set_msi_desc_off - set MSI descriptor data for an irq at offset
  *	@irq_base:	Interrupt number base
  *	@irq_offset:	Interrupt number offset
@@ -152,6 +191,33 @@ struct irq_data *irq_get_irq_data(unsigned int irq)
 }
 EXPORT_SYMBOL_GPL(irq_get_irq_data);
 
+/**
+ * irq_compose_mbi_msg - Compose mbi message for an irq chip
+ * @data:	Pointer to interrupt specific data
+ * @msg:	Pointer to the MBI message
+ *
+ * For hierarchical domains we find the first chip in the hierarchy
+ * which implements the irq_compose_msg callback. For non hierarchical
+ * we use the top level chip.
+ */
+int irq_compose_mbi_msg(struct irq_data *data, struct mbi_msg *msg)
+{
+	struct irq_data *pos = NULL;
+
+#ifdef	CONFIG_IRQ_DOMAIN_HIERARCHY
+	for (; data; data = data->parent_data)
+#endif
+		if (data->chip && data->chip->irq_compose_msg)
+			pos = data;
+	if (!pos)
+		return -ENOSYS;
+
+	pos->chip->irq_compose_msg(pos, msg);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(irq_compose_mbi_msg);
+
 static void irq_state_clr_disabled(struct irq_desc *desc)
 {
 	irqd_clear(&desc->irq_data, IRQD_IRQ_DISABLED);
diff --git a/kernel/irq/mbi.c b/kernel/irq/mbi.c
new file mode 100644
index 0000000..1b1f108
--- /dev/null
+++ b/kernel/irq/mbi.c
@@ -0,0 +1,260 @@
+/**
+ * Copyright (C) 2014, Yun Wu <wuyun.wu@huawei.com>
+ *
+ * This file contains the generic interfaces for Message Based
+ * Interrupts (MBI).
+ */
+
+#include <linux/interrupt.h>
+#include <linux/mbi.h>
+#include <linux/slab.h>
+
+/**
+ * mbi_find_desc() - search an MBI descriptor from MBI data
+ * @mbi:	MBI data of the device
+ * @data:	the unique MBI-specific data
+ */
+static struct mbi_desc *mbi_find_desc(struct device *dev, void *data)
+{
+	struct mbi_desc *desc;
+
+	list_for_each_entry(desc, &dev->mbi->mbi_list, entry) {
+		if (desc->data == data)
+			return desc;
+	}
+
+	return NULL;
+}
+
+/**
+ * mbi_init() - initialize MBI data
+ * @dev:	the device owned the MBI data
+ * @ops:	operations of the MBI-capable device
+ * @pmbi:	pointer to the pointer of MBI data
+ */
+static int mbi_init(struct device *dev, struct mbi_ops *ops,
+		    struct mbi_data **pmbi)
+{
+	struct mbi_data *mbi = dev->mbi;
+
+	if (mbi)
+		goto out_done;
+	if (!ops)
+		return -EINVAL;
+
+	mbi = kzalloc(sizeof(*mbi), GFP_KERNEL);
+	if (!mbi)
+		return -ENOMEM;
+
+	mbi->dev = dev;
+	mbi->ops = ops;
+	raw_spin_lock_init(&mbi->lock);
+	INIT_LIST_HEAD(&mbi->mbi_list);
+	dev->mbi = mbi;
+
+out_done:
+	*pmbi = mbi;
+	return 0;
+}
+
+/**
+ * mbi_free_desc() - free an MBI descriptor
+ * @desc:	the MBI descriptor to be freed
+ */
+static void mbi_free_desc(struct mbi_desc *desc)
+{
+	struct mbi_data *mbi;
+
+	mbi = desc ? desc->mbi : NULL;
+	BUG_ON(!mbi);
+
+	raw_spin_lock(&mbi->lock);
+	list_del(&desc->entry);
+	if (list_empty(&mbi->mbi_list))
+		mbi->dev->mbi = NULL;
+	raw_spin_unlock(&mbi->lock);
+
+	kfree(desc);
+	kfree(mbi);
+}
+
+/**
+ * mbi_alloc_desc() - allocate an MBI descriptor
+ * @info:	infomation prepared for the MBI domain
+ * @virq:	the base linux interrupt number of the MBI
+ * @pdesc:	pointer to the pointer of MBI descriptor
+ */
+static int mbi_alloc_desc(struct mbi_domain_info *info, unsigned int virq,
+			  struct mbi_desc **pdesc)
+{
+	struct mbi_data *mbi = NULL;
+	struct mbi_desc *desc;
+	int ret;
+
+	ret = mbi_init(info->dev, info->ops, &mbi);
+	if (ret)
+		return ret;
+
+	desc = mbi_find_desc(info->dev, info->data);
+	if (desc)
+		goto out_done;
+
+	desc = kzalloc(sizeof(*desc), GFP_KERNEL);
+	if (!desc)
+		return -ENOMEM;
+
+	desc->mbi = mbi;
+	desc->irq = virq;
+	desc->data = info->data;
+	desc->nvec = info->nvec;
+
+	raw_spin_lock(&mbi->lock);
+	list_add(&desc->entry, &mbi->mbi_list);
+	raw_spin_unlock(&mbi->lock);
+
+out_done:
+	*pdesc = desc;
+	return 0;
+}
+
+/**
+ * mbi_config_irq() - config one MBI interrupt
+ * @data:	irq_data of the MBI interrupt
+ * @mask:	type of configuration, mask if true, otherwise unmask
+ */
+static void mbi_config_irq(struct irq_data *data, int mask)
+{
+	struct mbi_desc *desc = data->mbi_desc;
+	struct mbi_ops *ops = desc->mbi->ops;
+	unsigned int id = data->irq - desc->irq;
+	void (*fn)(struct mbi_desc *, int) = NULL;
+
+	if (ops)
+		fn = mask ? ops->mask_irq : ops->unmask_irq;
+	if (fn)
+		fn(desc, id);
+}
+
+static void mbi_mask_irq(struct irq_data *data)
+{
+	mbi_config_irq(data, 1);
+}
+
+static void mbi_unmask_irq(struct irq_data *data)
+{
+	mbi_config_irq(data, 0);
+}
+
+static void mbi_write_msg(struct mbi_desc *desc, struct mbi_msg *msg)
+{
+	desc->mbi->ops->write_msg(desc, msg);
+	desc->msg = *msg;
+}
+
+static int mbi_set_affinity(struct irq_data *irq_data,
+			    const struct cpumask *mask, bool force)
+{
+	struct irq_data *parent = irq_data->parent_data;
+	struct mbi_msg msg;
+	int ret;
+
+	ret = parent->chip->irq_set_affinity(parent, mask, force);
+	if (ret >= 0 && ret != IRQ_SET_MASK_OK_DONE) {
+		BUG_ON(irq_compose_mbi_msg(irq_data, &msg));
+		mbi_write_msg(irq_data->mbi_desc, &msg);
+	}
+
+	return ret;
+}
+
+static struct irq_chip mbi_irq_chip = {
+	.name = "MBI",
+	.irq_unmask = mbi_unmask_irq,
+	.irq_mask = mbi_mask_irq,
+	.irq_ack = irq_chip_ack_parent,
+	.irq_set_affinity = mbi_set_affinity,
+	.irq_retrigger = irq_chip_retrigger_hierarchy,
+	.flags = IRQCHIP_SKIP_SET_WAKE,
+};
+
+static void mbi_domain_activate(struct irq_domain *domain, struct irq_data *data)
+{
+	struct mbi_desc *desc = data->mbi_desc;
+	struct mbi_msg msg;
+
+	WARN_ON(domain != data->domain);
+	BUG_ON(irq_compose_mbi_msg(data, &msg));
+	mbi_write_msg(desc, &msg);
+}
+
+static void mbi_domain_deactivate(struct irq_domain *domain, struct irq_data *data)
+{
+	struct mbi_desc *desc = data->mbi_desc;
+	struct mbi_msg msg;
+
+	WARN_ON(domain != data->domain);
+	memset(&msg, 0, sizeof(msg));
+	mbi_write_msg(desc, &msg);
+}
+
+static int mbi_domain_alloc(struct irq_domain *domain, unsigned int virq,
+			    unsigned int nr_irqs, void *arg)
+{
+	struct mbi_domain_info *info = arg;
+	struct mbi_desc *desc;
+	int ret, i;
+
+	ret = mbi_alloc_desc(info, virq, &desc);
+	if (ret)
+		return ret;
+
+	ret = irq_set_mbi_desc_range(virq, desc, nr_irqs);
+	if (unlikely(ret))
+		goto out_free_desc;
+
+	for (i = 0; i < nr_irqs; i++)
+		irq_domain_set_hwirq_and_chip(domain, virq + i, virq + i,
+					      &mbi_irq_chip, NULL);
+
+	ret = irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, NULL);
+	if (ret)
+		goto out_unset_desc;
+
+	return 0;
+
+out_unset_desc:
+	irq_set_mbi_desc_range(virq, NULL, nr_irqs);
+out_free_desc:
+	mbi_free_desc(desc);
+	return ret;
+}
+
+static void mbi_domain_free(struct irq_domain *domain, unsigned int virq,
+			    unsigned int nr_irqs)
+{
+	struct mbi_desc *desc = irq_get_mbi_desc(virq);
+
+	irq_set_mbi_desc_range(virq, NULL, nr_irqs);
+	mbi_free_desc(desc);
+	irq_domain_free_irqs_top(domain, virq, nr_irqs);
+}
+
+static struct irq_domain_ops mbi_domain_ops = {
+	.alloc = mbi_domain_alloc,
+	.free = mbi_domain_free,
+	.activate = mbi_domain_activate,
+	.deactivate = mbi_domain_deactivate,
+};
+
+struct irq_domain *mbi_create_irq_domain(struct device_node *of_node,
+					 struct irq_domain *parent)
+{
+	struct irq_domain *domain;
+
+	domain = irq_domain_add_tree(of_node, &mbi_domain_ops, NULL);
+	if (!domain)
+		return NULL;
+
+	domain->parent = parent;
+	return domain;
+}
-- 
1.8.0

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

On 2014/11/19 17:20, Marc Zyngier wrote:

> On Wed, Nov 19 2014 at  6:57:25 am GMT, "Yun Wu (Abel)" <wuyun.wu@huawei.com> wrote:
>> On 2014/11/18 22:32, Thomas Gleixner wrote:
>>
>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>
>>> Can you please trim the messages when you're replying?
>>>
>>>> The above you described is absolutely right, but not the things I want
>>>> to know. :)
>>>> Take GICv3 ITS for example, it deals with both PCI and non PCI message
>>>> interrupts. IIUC, several irq_chips need to be implemented in the ITS
>>>> driver (i.e. pci_msi_chip, A_msi_chip and B_msi_chip). What should we
>>>> do to the ITS driver if new MSI-capable devices come out?
>>>
>>> You seem to miss the stacking here
>>>
>>> PCI-MSI	->
>>> A-MSI	->   ITS  -> GIC
>>> B-MSI	->
>>>
>>> So each of the device types has its own MSI controller. Each of them
>>> will have their own callbacks and are backed by the underlying ITS/GIC
>>> implementation.
>>
>> Yes, this hits the key point. Once a new device type becomes available,
>> we need to add pieces of code outside the new device's driver to make
>> it work, which in my opinion is due to lack of core infrastructure.
>> More specifically, the core infrastructure needs to support mechanism
>> of MSI, not the various types of devices.
>>
>>>
>>> And that's the only sensible solution.
>>>
>>
>> It's sensible, but not perfect.
>> What I suggested is to add a generic layer:
>>
>> PCI-MSI	->
>> A-MSI	-> (generic layer) -> ITS -> GICR
>> B-MSI	->
>>
>> The PCI/A/B/... passes its hardware properties to the generic layer which
>> gets configurations ready by calling ITS's domain/chip callbacks. When
>> a new device type arrives, the only thing need to do is to implement the
>> driver of that device, with nothing to do with the generic layer or ITS.
> 
> I really don't get your "generic layer" story. To me, it looks like a
> glorified set of function pointers. And that's exactly what stacked
> domains are giving you:
> 
> A-MSI -> ITS -> GIC
> 
> This "A-MSI" is responsible for:
> - implementing the "prepare" callback, which allocates the ITT
> - implementing the "irq_compose_msi_msg"
> 
> Hardly anything to change in the ITS driver, and I can probably make it
> so that you don't even have to write a single line of code in the ITS
> driver.
> 
> If the generic MSI layer we now have is not enough for you, then please
> submit detailed use cases.
> 

Hi Marc,

As I said, I never thought Gerry's patch don't work, I am just trying to
make it better. :)

As to the "generic layer" story, please check the following URL:
https://lkml.org/lkml/2014/12/10/93

Thanks,
	Abel

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Thomas Gleixner]

On Wed, 10 Dec 2014, Yun Wu (Abel) wrote:
> On 2014/11/19 19:11, Thomas Gleixner wrote:
> I spent last two weeks implementing and testing my original idea about making
> the sub domains generic, based on stacked domain feature. Now it comes real,
> please see the attached patch.

Can you please send patches inline? Attached is a pain to reply to.
 
> With this patch applied, I think things will get easier.

I don't see what gets easier. It's just another infrastructure which
is painfully similar to MSI.

> This patch (also with several other patches) is tested on Hisilicon ARM64 SoC,
> with non PCI devices capable of message based interrupts. The PCI part is not
> tested because it needs large refactoring work to do. So yes, the testing work
> is not sufficient, but I think the patch is enough to present what I really
> wanted to express. :)

Not really.

If you provide proper patches which make use of it and most important
a proper refactoring of the PCI/MSI side then we can discuss that, but
for now it's just handwaving.

Thanks,

	tglx

Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

[Yun Wu (Abel)]

On 2014/12/10 18:25, Thomas Gleixner wrote:

> On Wed, 10 Dec 2014, Yun Wu (Abel) wrote:
>> On 2014/11/19 19:11, Thomas Gleixner wrote:
>> I spent last two weeks implementing and testing my original idea about making
>> the sub domains generic, based on stacked domain feature. Now it comes real,
>> please see the attached patch.
> 
> Can you please send patches inline? Attached is a pain to reply to.

Sure, why not.

>  
>> With this patch applied, I think things will get easier.
> 
> I don't see what gets easier. It's just another infrastructure which
> is painfully similar to MSI.

Then please help me feel that pain when I post inline patches, thanks. :)

> 
>> This patch (also with several other patches) is tested on Hisilicon ARM64 SoC,
>> with non PCI devices capable of message based interrupts. The PCI part is not
>> tested because it needs large refactoring work to do. So yes, the testing work
>> is not sufficient, but I think the patch is enough to present what I really
>> wanted to express. :)
> 
> Not really.
> 
> If you provide proper patches which make use of it and most important
> a proper refactoring of the PCI/MSI side then we can discuss that, but
> for now it's just handwaving.
> 

OK, I will start a new thread when I finished PCI/MSI refactoring work.

Thanks,
	Abel