[PATCH 0/3] Amazon's Annapurna Labs Fabric Interrupt Controller

[PATCH 0/3] Amazon's Annapurna Labs Fabric Interrupt Controller

[Talel Shenhar]

This series introduces support for Amazon's Annapurna Labs Fabric Interrupt
Controller.

The Amazon's Annapurna Labs FIC (Fabric Interrupt Controller) has 32
inputs/sources. The output of this interrupt controller can be legacy-wired
output or, in case embedded inside PCIe devices, msi-x message. This FIC may
be cascaded into another FIC or connected directly to the main CPU
Interrupt Controller (e.g. GIC).

The FIC is a flexible HW unit that is embedded inside different parts of
the Amazon's Annapurna Labs chips. It can be a simple level 2 interrupt
controller which is then configured as a wired interrupt controller that
aggregates events from different units, or, while embedded inside PCIe
device, it can generate MSI-X messages based on the tables configured to
that PCIe device or can be configured to generate wired interrupt. 



Talel Shenhar (3):
  dt-bindings: interrupt-controller: Amazon's Annapurna Labs FIC
  irqchip: al-fic: Introduce Amazon's Annapurna Labs Fabric Interrupt
    Controller Driver
  irqchip: al-fic: Introducing support for MSI-X

 .../interrupt-controller/amazon,al-fic.txt         |  22 ++
 MAINTAINERS                                        |   7 +
 drivers/irqchip/Kconfig                            |  11 +
 drivers/irqchip/Makefile                           |   1 +
 drivers/irqchip/irq-al-fic.c                       | 388 +++++++++++++++++++++
 include/linux/irqchip/al-fic.h                     |  23 ++
 6 files changed, 452 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt
 create mode 100644 drivers/irqchip/irq-al-fic.c
 create mode 100644 include/linux/irqchip/al-fic.h

-- 
2.7.4

[PATCH 0/3] Amazon's Annapurna Labs Fabric Interrupt Controller

[Talel Shenhar]

This series introduces support for Amazon's Annapurna Labs Fabric Interrupt
Controller.

The Amazon's Annapurna Labs FIC (Fabric Interrupt Controller) has 32
inputs/sources. The output of this interrupt controller can be legacy-wired
output or, in case embedded inside PCIe devices, msi-x message. This FIC may
be cascaded into another FIC or connected directly to the main CPU
Interrupt Controller (e.g. GIC).

The FIC is a flexible HW unit that is embedded inside different parts of
the Amazon's Annapurna Labs chips. It can be a simple level 2 interrupt
controller which is then configured as a wired interrupt controller that
aggregates events from different units, or, while embedded inside PCIe
device, it can generate MSI-X messages based on the tables configured to
that PCIe device or can be configured to generate wired interrupt. 



Talel Shenhar (3):
  dt-bindings: interrupt-controller: Amazon's Annapurna Labs FIC
  irqchip: al-fic: Introduce Amazon's Annapurna Labs Fabric Interrupt
    Controller Driver
  irqchip: al-fic: Introducing support for MSI-X

 .../interrupt-controller/amazon,al-fic.txt         |  22 ++
 MAINTAINERS                                        |   7 +
 drivers/irqchip/Kconfig                            |  11 +
 drivers/irqchip/Makefile                           |   1 +
 drivers/irqchip/irq-al-fic.c                       | 388 +++++++++++++++++++++
 include/linux/irqchip/al-fic.h                     |  23 ++
 6 files changed, 452 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt
 create mode 100644 drivers/irqchip/irq-al-fic.c
 create mode 100644 include/linux/irqchip/al-fic.h

-- 
2.7.4

[PATCH 1/3] dt-bindings: interrupt-controller: Amazon's Annapurna Labs FIC

[Talel Shenhar]

Document Amazon's Annapurna Labs Fabric Interrupt Controller SoC binding.

Signed-off-by: Talel Shenhar <talel@amazon.com>
---
 .../interrupt-controller/amazon,al-fic.txt         | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt

diff --git a/Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt b/Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt
new file mode 100644
index 0000000..a2f31a6
--- /dev/null
+++ b/Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt
@@ -0,0 +1,22 @@
+Amazon's Annapurna Labs Fabric Interrupt Controller
+
+Required properties:
+
+- compatible: should be "amazon,al-fic"
+- reg: physical base address and size of the registers
+- interrupt-controller: identifies the node as an interrupt controller
+- #interrupt-cells: must be 2.
+- interrupt-parent: specifies the parent interrupt controller.
+- interrupts: describes which input line in the interrupt parent, this
+  fic's output is connected to.
+
+Example:
+
+amazon_fic: amazon_fic {
+	compatible = "amazon,al-fic";
+	interrupt-controller;
+	#interrupt-cells = <1>;
+	reg = <0x0 0xfd8a8500 0x0 0x1000>;
+	interrupt-parent = <&gic>;
+	interrupts = <GIC_SPI 0x0 IRQ_TYPE_LEVEL_HIGH>;
+};
-- 
2.7.4

[PATCH 1/3] dt-bindings: interrupt-controller: Amazon's Annapurna Labs FIC

[Talel Shenhar]

Document Amazon's Annapurna Labs Fabric Interrupt Controller SoC binding.

Signed-off-by: Talel Shenhar <talel@amazon.com>
---
 .../interrupt-controller/amazon,al-fic.txt         | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt

diff --git a/Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt b/Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt
new file mode 100644
index 0000000..a2f31a6
--- /dev/null
+++ b/Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt
@@ -0,0 +1,22 @@
+Amazon's Annapurna Labs Fabric Interrupt Controller
+
+Required properties:
+
+- compatible: should be "amazon,al-fic"
+- reg: physical base address and size of the registers
+- interrupt-controller: identifies the node as an interrupt controller
+- #interrupt-cells: must be 2.
+- interrupt-parent: specifies the parent interrupt controller.
+- interrupts: describes which input line in the interrupt parent, this
+  fic's output is connected to.
+
+Example:
+
+amazon_fic: amazon_fic {
+	compatible = "amazon,al-fic";
+	interrupt-controller;
+	#interrupt-cells = <1>;
+	reg = <0x0 0xfd8a8500 0x0 0x1000>;
+	interrupt-parent = <&gic>;
+	interrupts = <GIC_SPI 0x0 IRQ_TYPE_LEVEL_HIGH>;
+};
-- 
2.7.4

[PATCH 2/3] irqchip: al-fic: Introduce Amazon's Annapurna Labs Fabric Interrupt Controller Driver

[Talel Shenhar]

The Amazon's Annapurna Labs Fabric Interrupt Controller has 32 inputs
lines. A FIC (Fabric Interrupt Controller) may be cascaded into another FIC
or directly to the main CPU Interrupt Controller (e.g. GIC).

This driver introduces support for initialization based on device-tree
compatibility and run-time for dynamically added FICs (e.g. FICs embedded
to PCIe devices).

Signed-off-by: Talel Shenhar <talel@amazon.com>
---
 MAINTAINERS                    |   7 +
 drivers/irqchip/Kconfig        |  11 ++
 drivers/irqchip/Makefile       |   1 +
 drivers/irqchip/irq-al-fic.c   | 346 +++++++++++++++++++++++++++++++++++++++++
 include/linux/irqchip/al-fic.h |  21 +++
 5 files changed, 386 insertions(+)
 create mode 100644 drivers/irqchip/irq-al-fic.c
 create mode 100644 include/linux/irqchip/al-fic.h

diff --git a/MAINTAINERS b/MAINTAINERS
index f485597..c28a97f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1209,6 +1209,13 @@ S:	Maintained
 F:	Documentation/devicetree/bindings/interrupt-controller/arm,vic.txt
 F:	drivers/irqchip/irq-vic.c
 
+AMAZON ANNAPURNA LABS FIC DRIVER
+M:	Talel Shenhar <talel@amazon.com>
+S:	Maintained
+F:	Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt
+F:	drivers/irqchip/irq-al-fic.c
+F:	include/linux/irqchip/al-fic.h
+
 ARM SMMU DRIVERS
 M:	Will Deacon <will.deacon@arm.com>
 R:	Robin Murphy <robin.murphy@arm.com>
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 51a5ef0..1e51f0f 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -89,6 +89,17 @@ config ALPINE_MSI
 	select PCI_MSI
 	select GENERIC_IRQ_CHIP
 
+config AL_FIC
+	bool "Amazon's Annapurna Labs Fabric Interrupt Controller"
+	depends on OF || COMPILE_TEST
+	select GENERIC_IRQ_CHIP
+	select IRQ_DOMAIN
+	select GENERIC_IRQ_MULTI_HANDLER
+	select IRQ_DOMAIN_HIERARCHY
+	select SPARSE_IRQ
+	help
+	  Support Amazon's Annapurna Labs Fabric Interrupt Controller.
+
 config ATMEL_AIC_IRQ
 	bool
 	select GENERIC_IRQ_CHIP
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 794c13d..a20eba5 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-$(CONFIG_IRQCHIP)			+= irqchip.o
 
+obj-$(CONFIG_AL_FIC)			+= irq-al-fic.o
 obj-$(CONFIG_ALPINE_MSI)		+= irq-alpine-msi.o
 obj-$(CONFIG_ATH79)			+= irq-ath79-cpu.o
 obj-$(CONFIG_ATH79)			+= irq-ath79-misc.o
diff --git a/drivers/irqchip/irq-al-fic.c b/drivers/irqchip/irq-al-fic.c
new file mode 100644
index 0000000..d881d42
--- /dev/null
+++ b/drivers/irqchip/irq-al-fic.c
@@ -0,0 +1,346 @@
+// SPDX-License-Identifier: GPL-2.0
+/**
+ * Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/irqchip/al-fic.h>
+#include <linux/irq.h>
+#include <linux/irqchip.h>
+#include <linux/irqchip/chained_irq.h>
+#include <linux/irqdomain.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+
+/* FIC Registers */
+#define AL_FIC_CAUSE		0x00
+#define AL_FIC_MASK		0x10
+#define AL_FIC_CONTROL		0x28
+
+#define CONTROL_TRIGGER_RISING	BIT(3)
+#define CONTROL_MASK_MSI_X	BIT(5)
+
+#define NR_FIC_IRQS 32
+
+MODULE_AUTHOR("Talel Shenhar");
+MODULE_DESCRIPTION("Amazon's Annapurna Labs Interrupt Controller Driver");
+MODULE_LICENSE("GPL v2");
+
+enum al_fic_state {
+	AL_FIC_CLEAN = 0,
+	AL_FIC_CONFIGURED_LEVEL,
+	AL_FIC_CONFIGURED_RAISING_EDGE,
+};
+
+struct al_fic {
+	void __iomem *base;
+	struct irq_domain *domain;
+	const char *name;
+	unsigned int parent_irq;
+	enum al_fic_state state;
+};
+
+static void al_fic_set_trigger(struct al_fic *fic,
+			       struct irq_chip_generic *gc,
+			       enum al_fic_state new_state)
+{
+	irq_flow_handler_t handler;
+	u32 control = readl(fic->base + AL_FIC_CONTROL);
+
+	if (new_state == AL_FIC_CONFIGURED_LEVEL) {
+		handler = handle_level_irq;
+		control &= ~CONTROL_TRIGGER_RISING;
+	} else {
+		handler = handle_edge_irq;
+		control |= CONTROL_TRIGGER_RISING;
+	}
+	gc->chip_types->handler = handler;
+	fic->state = new_state;
+	writel(control, fic->base + AL_FIC_CONTROL);
+}
+
+static int al_fic_irq_set_type(struct irq_data *data, unsigned int flow_type)
+{
+	struct irq_chip_generic *gc = irq_data_get_irq_chip_data(data);
+	struct al_fic *fic = gc->private;
+	enum al_fic_state new_state;
+	int ret = 0;
+
+	irq_gc_lock(gc);
+
+	if (!(flow_type & IRQ_TYPE_LEVEL_HIGH) &&
+	    !(flow_type & IRQ_TYPE_EDGE_RISING)) {
+		pr_err("fic doesn't support flow type %d\n", flow_type);
+		ret = -EPERM;
+		goto err;
+	}
+
+	new_state = (flow_type & IRQ_TYPE_LEVEL_HIGH) ?
+		AL_FIC_CONFIGURED_LEVEL : AL_FIC_CONFIGURED_RAISING_EDGE;
+
+	/* A given FIC instance can be either all level or all edge triggered.
+	 * This is generally fixed depending on what pieces of HW it's wired up
+	 * to.
+	 *
+	 * We configure it based on the sensitivity of the first source
+	 * being setup, and reject any subsequent attempt at configuring it in a
+	 * different way.
+	 */
+	if (fic->state == AL_FIC_CLEAN) {
+		al_fic_set_trigger(fic, gc, new_state);
+	} else if (fic->state != new_state) {
+		pr_err("fic %s state already configured to %d\n",
+		       fic->name, fic->state);
+		ret = -EPERM;
+		goto err;
+	}
+
+err:
+	irq_gc_unlock(gc);
+
+	return ret;
+}
+
+static void al_fic_irq_handler(struct irq_desc *desc)
+{
+	struct al_fic *fic = irq_desc_get_handler_data(desc);
+	struct irq_domain *domain = fic->domain;
+	struct irq_chip *irqchip = irq_desc_get_chip(desc);
+	struct irq_chip_generic *gc = irq_get_domain_generic_chip(domain, 0);
+	unsigned long pending;
+	unsigned int irq;
+	u32 hwirq;
+
+	chained_irq_enter(irqchip, desc);
+
+	pending = readl(fic->base + AL_FIC_CAUSE);
+	pending &= ~gc->mask_cache;
+
+	for_each_set_bit(hwirq, &pending, domain->revmap_size) {
+		irq = irq_find_mapping(domain, hwirq);
+		generic_handle_irq(irq);
+	}
+
+	chained_irq_exit(irqchip, desc);
+}
+
+static int al_fic_register(struct device_node *node,
+			   struct al_fic *fic)
+{
+	struct irq_chip_generic *gc;
+	int ret;
+
+	fic->domain = irq_domain_add_linear(node,
+					    NR_FIC_IRQS,
+					    &irq_generic_chip_ops,
+					    fic);
+	if (!fic->domain) {
+		pr_err("fail to add irq domain\n");
+		return -ENOMEM;
+	}
+
+	ret = irq_alloc_domain_generic_chips(fic->domain,
+					     NR_FIC_IRQS,
+					     1, fic->name,
+					     handle_level_irq,
+					     0, 0, IRQ_GC_INIT_MASK_CACHE);
+	if (ret) {
+		pr_err("fail to allocate generic chip (%d)\n", ret);
+		goto err_domain_remove;
+	}
+
+	gc = irq_get_domain_generic_chip(fic->domain, 0);
+	gc->reg_base = fic->base;
+	gc->chip_types->regs.mask = AL_FIC_MASK;
+	gc->chip_types->regs.ack = AL_FIC_CAUSE;
+	gc->chip_types->chip.irq_mask = irq_gc_mask_set_bit;
+	gc->chip_types->chip.irq_unmask = irq_gc_mask_clr_bit;
+	gc->chip_types->chip.irq_ack = irq_gc_ack_clr_bit;
+	gc->chip_types->chip.irq_set_type = al_fic_irq_set_type;
+	gc->chip_types->chip.flags = IRQCHIP_SKIP_SET_WAKE;
+	gc->private = fic;
+
+	irq_set_chained_handler_and_data(fic->parent_irq,
+					 al_fic_irq_handler,
+					 fic);
+	return 0;
+
+err_domain_remove:
+	irq_domain_remove(fic->domain);
+
+	return ret;
+}
+
+/**
+ * al_fic_wire_get_domain() - returns the irq_domain assigned to this fic
+ * @fic: fic context configured to Legacy mode
+ *
+ * A fic configured to Legacy mode requires domain to map the hwirqs to virtual
+ * irqs. This API supplies the domain allocated to this fic.
+ *
+ * Returns the irq_domain or ERR_PTR on error.
+ */
+struct irq_domain *al_fic_wire_get_domain(struct al_fic *fic)
+{
+	if (!fic)
+		return ERR_PTR(-EINVAL);
+	if (!fic->domain)
+		return ERR_PTR(-EPERM);
+
+	return fic->domain;
+}
+EXPORT_SYMBOL_GPL(al_fic_wire_get_domain);
+
+static void al_fic_hw_init(struct al_fic *fic)
+{
+	u32 control = CONTROL_MASK_MSI_X;
+
+	/* mask out all interrupts */
+	writel(0xFFFFFFFF, fic->base + AL_FIC_MASK);
+
+	/* clear any pending interrupt */
+	writel(0, fic->base + AL_FIC_CAUSE);
+
+	writel(control, fic->base + AL_FIC_CONTROL);
+}
+
+/**
+ * al_fic_wire_init() - initialize and configure fic in wire mode
+ * @of_node: optional pointer to interrupt controller's device tree node.
+ * @base: mmio to fic register
+ * @name: name of the fic
+ * @parent_irq: interrupt of parent
+ *
+ * This API will configure the fic hardware to to work in wire mode.
+ * In wire mode, fic hardware is generating a wire ("wired") interrupt.
+ * Interrupt can be generated based on positive edge or level - configuration is
+ * to be determined based on connected hardware to this fic.
+ *
+ * Returns fic context that allows the user to obtain the irq_domain by using
+ * al_fic_wire_get_domain().
+ */
+struct al_fic *al_fic_wire_init(struct device_node *node,
+				void __iomem *base,
+				const char *name,
+				unsigned int parent_irq)
+{
+	struct al_fic *fic;
+	int ret;
+
+	if (!base)
+		return ERR_PTR(-EINVAL);
+
+	fic = kzalloc(sizeof(*fic), GFP_KERNEL);
+	if (!fic)
+		return ERR_PTR(-ENOMEM);
+
+	fic->base = base;
+	fic->parent_irq = parent_irq;
+	fic->name = (name ?: "al-fic-wire");
+
+	al_fic_hw_init(fic);
+
+	ret = al_fic_register(node, fic);
+	if (ret) {
+		pr_err("fail to register irqchip\n");
+		goto err_free;
+	}
+
+	pr_debug("%s initialized successfully in Legacy mode (parent-irq=%u)\n",
+		 fic->name, parent_irq);
+
+	return fic;
+
+err_free:
+	kfree(fic);
+	return ERR_PTR(ret);
+}
+EXPORT_SYMBOL_GPL(al_fic_wire_init);
+
+/**
+ * al_fic_cleanup() - free all resources allocated by fic
+ * @fic: pointer to fic context
+ *
+ * This API will release the domain (in case of wire mode) and will free the
+ * context allocated for this fic.
+ * Caller should call this API only after all interrupts were released.
+ *
+ * Returns 0 on success or negative error code.
+ */
+int al_fic_cleanup(struct al_fic *fic)
+{
+	if (!fic)
+		return -EINVAL;
+
+	if (fic->domain) {
+		struct irq_chip_generic *gc;
+
+		gc = irq_get_domain_generic_chip(fic->domain, 0);
+		if (IS_ERR(gc)) {
+			pr_err("%s was not able to grab gc for fic (%lu)\n",
+			       __func__, PTR_ERR(gc));
+			return PTR_ERR(gc);
+		}
+
+		irq_destroy_generic_chip(gc, 0, 0, 0);
+		irq_domain_remove(fic->domain);
+	}
+
+	kfree(fic);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(al_fic_cleanup);
+
+static int __init al_fic_init_dt(struct device_node *node,
+				 struct device_node *parent)
+{
+	int ret;
+	void __iomem *base;
+	unsigned int parent_irq;
+	struct al_fic *fic;
+
+	if (!parent) {
+		pr_err("%s: unsupported - device require a parent\n",
+		       node->name);
+		return -EINVAL;
+	}
+
+	base = of_iomap(node, 0);
+	if (!base) {
+		pr_err("%s: fail to map memory\n", node->name);
+		return -ENOMEM;
+	}
+
+	parent_irq = irq_of_parse_and_map(node, 0);
+	if (!parent_irq) {
+		pr_err("%s: fail to map irq\n", node->name);
+		ret = -EINVAL;
+		goto err_unmap;
+	}
+
+	fic = al_fic_wire_init(node,
+			       base,
+			       node->name,
+			       parent_irq);
+	if (IS_ERR(fic)) {
+		pr_err("%s: fail to initialize irqchip (%lu)\n",
+		       node->name,
+		       PTR_ERR(fic));
+		ret = PTR_ERR(fic);
+		goto err_irq_dispose;
+	}
+
+	return 0;
+
+err_irq_dispose:
+	irq_dispose_mapping(parent_irq);
+err_unmap:
+	iounmap(base);
+
+	return ret;
+}
+
+IRQCHIP_DECLARE(al_fic, "amazon,al-fic", al_fic_init_dt);
diff --git a/include/linux/irqchip/al-fic.h b/include/linux/irqchip/al-fic.h
new file mode 100644
index 0000000..0833749
--- /dev/null
+++ b/include/linux/irqchip/al-fic.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/**
+ * Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ */
+
+#ifndef _AL_FIC_H_
+#define _AL_FIC_H_
+
+#include <linux/irqdomain.h>
+
+struct al_fic;
+
+struct irq_domain *al_fic_wire_get_domain(struct al_fic *fic);
+
+struct al_fic *al_fic_wire_init(struct device_node *node,
+				void __iomem *base,
+				const char *name,
+				unsigned int parent_irq);
+int al_fic_cleanup(struct al_fic *fic);
+
+#endif
-- 
2.7.4

[PATCH 2/3] irqchip: al-fic: Introduce Amazon's Annapurna Labs Fabric Interrupt Controller Driver

[Talel Shenhar]

The Amazon's Annapurna Labs Fabric Interrupt Controller has 32 inputs
lines. A FIC (Fabric Interrupt Controller) may be cascaded into another FIC
or directly to the main CPU Interrupt Controller (e.g. GIC).

This driver introduces support for initialization based on device-tree
compatibility and run-time for dynamically added FICs (e.g. FICs embedded
to PCIe devices).

Signed-off-by: Talel Shenhar <talel@amazon.com>
---
 MAINTAINERS                    |   7 +
 drivers/irqchip/Kconfig        |  11 ++
 drivers/irqchip/Makefile       |   1 +
 drivers/irqchip/irq-al-fic.c   | 346 +++++++++++++++++++++++++++++++++++++++++
 include/linux/irqchip/al-fic.h |  21 +++
 5 files changed, 386 insertions(+)
 create mode 100644 drivers/irqchip/irq-al-fic.c
 create mode 100644 include/linux/irqchip/al-fic.h

diff --git a/MAINTAINERS b/MAINTAINERS
index f485597..c28a97f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1209,6 +1209,13 @@ S:	Maintained
 F:	Documentation/devicetree/bindings/interrupt-controller/arm,vic.txt
 F:	drivers/irqchip/irq-vic.c
 
+AMAZON ANNAPURNA LABS FIC DRIVER
+M:	Talel Shenhar <talel@amazon.com>
+S:	Maintained
+F:	Documentation/devicetree/bindings/interrupt-controller/amazon,al-fic.txt
+F:	drivers/irqchip/irq-al-fic.c
+F:	include/linux/irqchip/al-fic.h
+
 ARM SMMU DRIVERS
 M:	Will Deacon <will.deacon@arm.com>
 R:	Robin Murphy <robin.murphy@arm.com>
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 51a5ef0..1e51f0f 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -89,6 +89,17 @@ config ALPINE_MSI
 	select PCI_MSI
 	select GENERIC_IRQ_CHIP
 
+config AL_FIC
+	bool "Amazon's Annapurna Labs Fabric Interrupt Controller"
+	depends on OF || COMPILE_TEST
+	select GENERIC_IRQ_CHIP
+	select IRQ_DOMAIN
+	select GENERIC_IRQ_MULTI_HANDLER
+	select IRQ_DOMAIN_HIERARCHY
+	select SPARSE_IRQ
+	help
+	  Support Amazon's Annapurna Labs Fabric Interrupt Controller.
+
 config ATMEL_AIC_IRQ
 	bool
 	select GENERIC_IRQ_CHIP
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 794c13d..a20eba5 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-$(CONFIG_IRQCHIP)			+= irqchip.o
 
+obj-$(CONFIG_AL_FIC)			+= irq-al-fic.o
 obj-$(CONFIG_ALPINE_MSI)		+= irq-alpine-msi.o
 obj-$(CONFIG_ATH79)			+= irq-ath79-cpu.o
 obj-$(CONFIG_ATH79)			+= irq-ath79-misc.o
diff --git a/drivers/irqchip/irq-al-fic.c b/drivers/irqchip/irq-al-fic.c
new file mode 100644
index 0000000..d881d42
--- /dev/null
+++ b/drivers/irqchip/irq-al-fic.c
@@ -0,0 +1,346 @@
+// SPDX-License-Identifier: GPL-2.0
+/**
+ * Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/irqchip/al-fic.h>
+#include <linux/irq.h>
+#include <linux/irqchip.h>
+#include <linux/irqchip/chained_irq.h>
+#include <linux/irqdomain.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+
+/* FIC Registers */
+#define AL_FIC_CAUSE		0x00
+#define AL_FIC_MASK		0x10
+#define AL_FIC_CONTROL		0x28
+
+#define CONTROL_TRIGGER_RISING	BIT(3)
+#define CONTROL_MASK_MSI_X	BIT(5)
+
+#define NR_FIC_IRQS 32
+
+MODULE_AUTHOR("Talel Shenhar");
+MODULE_DESCRIPTION("Amazon's Annapurna Labs Interrupt Controller Driver");
+MODULE_LICENSE("GPL v2");
+
+enum al_fic_state {
+	AL_FIC_CLEAN = 0,
+	AL_FIC_CONFIGURED_LEVEL,
+	AL_FIC_CONFIGURED_RAISING_EDGE,
+};
+
+struct al_fic {
+	void __iomem *base;
+	struct irq_domain *domain;
+	const char *name;
+	unsigned int parent_irq;
+	enum al_fic_state state;
+};
+
+static void al_fic_set_trigger(struct al_fic *fic,
+			       struct irq_chip_generic *gc,
+			       enum al_fic_state new_state)
+{
+	irq_flow_handler_t handler;
+	u32 control = readl(fic->base + AL_FIC_CONTROL);
+
+	if (new_state == AL_FIC_CONFIGURED_LEVEL) {
+		handler = handle_level_irq;
+		control &= ~CONTROL_TRIGGER_RISING;
+	} else {
+		handler = handle_edge_irq;
+		control |= CONTROL_TRIGGER_RISING;
+	}
+	gc->chip_types->handler = handler;
+	fic->state = new_state;
+	writel(control, fic->base + AL_FIC_CONTROL);
+}
+
+static int al_fic_irq_set_type(struct irq_data *data, unsigned int flow_type)
+{
+	struct irq_chip_generic *gc = irq_data_get_irq_chip_data(data);
+	struct al_fic *fic = gc->private;
+	enum al_fic_state new_state;
+	int ret = 0;
+
+	irq_gc_lock(gc);
+
+	if (!(flow_type & IRQ_TYPE_LEVEL_HIGH) &&
+	    !(flow_type & IRQ_TYPE_EDGE_RISING)) {
+		pr_err("fic doesn't support flow type %d\n", flow_type);
+		ret = -EPERM;
+		goto err;
+	}
+
+	new_state = (flow_type & IRQ_TYPE_LEVEL_HIGH) ?
+		AL_FIC_CONFIGURED_LEVEL : AL_FIC_CONFIGURED_RAISING_EDGE;
+
+	/* A given FIC instance can be either all level or all edge triggered.
+	 * This is generally fixed depending on what pieces of HW it's wired up
+	 * to.
+	 *
+	 * We configure it based on the sensitivity of the first source
+	 * being setup, and reject any subsequent attempt at configuring it in a
+	 * different way.
+	 */
+	if (fic->state == AL_FIC_CLEAN) {
+		al_fic_set_trigger(fic, gc, new_state);
+	} else if (fic->state != new_state) {
+		pr_err("fic %s state already configured to %d\n",
+		       fic->name, fic->state);
+		ret = -EPERM;
+		goto err;
+	}
+
+err:
+	irq_gc_unlock(gc);
+
+	return ret;
+}
+
+static void al_fic_irq_handler(struct irq_desc *desc)
+{
+	struct al_fic *fic = irq_desc_get_handler_data(desc);
+	struct irq_domain *domain = fic->domain;
+	struct irq_chip *irqchip = irq_desc_get_chip(desc);
+	struct irq_chip_generic *gc = irq_get_domain_generic_chip(domain, 0);
+	unsigned long pending;
+	unsigned int irq;
+	u32 hwirq;
+
+	chained_irq_enter(irqchip, desc);
+
+	pending = readl(fic->base + AL_FIC_CAUSE);
+	pending &= ~gc->mask_cache;
+
+	for_each_set_bit(hwirq, &pending, domain->revmap_size) {
+		irq = irq_find_mapping(domain, hwirq);
+		generic_handle_irq(irq);
+	}
+
+	chained_irq_exit(irqchip, desc);
+}
+
+static int al_fic_register(struct device_node *node,
+			   struct al_fic *fic)
+{
+	struct irq_chip_generic *gc;
+	int ret;
+
+	fic->domain = irq_domain_add_linear(node,
+					    NR_FIC_IRQS,
+					    &irq_generic_chip_ops,
+					    fic);
+	if (!fic->domain) {
+		pr_err("fail to add irq domain\n");
+		return -ENOMEM;
+	}
+
+	ret = irq_alloc_domain_generic_chips(fic->domain,
+					     NR_FIC_IRQS,
+					     1, fic->name,
+					     handle_level_irq,
+					     0, 0, IRQ_GC_INIT_MASK_CACHE);
+	if (ret) {
+		pr_err("fail to allocate generic chip (%d)\n", ret);
+		goto err_domain_remove;
+	}
+
+	gc = irq_get_domain_generic_chip(fic->domain, 0);
+	gc->reg_base = fic->base;
+	gc->chip_types->regs.mask = AL_FIC_MASK;
+	gc->chip_types->regs.ack = AL_FIC_CAUSE;
+	gc->chip_types->chip.irq_mask = irq_gc_mask_set_bit;
+	gc->chip_types->chip.irq_unmask = irq_gc_mask_clr_bit;
+	gc->chip_types->chip.irq_ack = irq_gc_ack_clr_bit;
+	gc->chip_types->chip.irq_set_type = al_fic_irq_set_type;
+	gc->chip_types->chip.flags = IRQCHIP_SKIP_SET_WAKE;
+	gc->private = fic;
+
+	irq_set_chained_handler_and_data(fic->parent_irq,
+					 al_fic_irq_handler,
+					 fic);
+	return 0;
+
+err_domain_remove:
+	irq_domain_remove(fic->domain);
+
+	return ret;
+}
+
+/**
+ * al_fic_wire_get_domain() - returns the irq_domain assigned to this fic
+ * @fic: fic context configured to Legacy mode
+ *
+ * A fic configured to Legacy mode requires domain to map the hwirqs to virtual
+ * irqs. This API supplies the domain allocated to this fic.
+ *
+ * Returns the irq_domain or ERR_PTR on error.
+ */
+struct irq_domain *al_fic_wire_get_domain(struct al_fic *fic)
+{
+	if (!fic)
+		return ERR_PTR(-EINVAL);
+	if (!fic->domain)
+		return ERR_PTR(-EPERM);
+
+	return fic->domain;
+}
+EXPORT_SYMBOL_GPL(al_fic_wire_get_domain);
+
+static void al_fic_hw_init(struct al_fic *fic)
+{
+	u32 control = CONTROL_MASK_MSI_X;
+
+	/* mask out all interrupts */
+	writel(0xFFFFFFFF, fic->base + AL_FIC_MASK);
+
+	/* clear any pending interrupt */
+	writel(0, fic->base + AL_FIC_CAUSE);
+
+	writel(control, fic->base + AL_FIC_CONTROL);
+}
+
+/**
+ * al_fic_wire_init() - initialize and configure fic in wire mode
+ * @of_node: optional pointer to interrupt controller's device tree node.
+ * @base: mmio to fic register
+ * @name: name of the fic
+ * @parent_irq: interrupt of parent
+ *
+ * This API will configure the fic hardware to to work in wire mode.
+ * In wire mode, fic hardware is generating a wire ("wired") interrupt.
+ * Interrupt can be generated based on positive edge or level - configuration is
+ * to be determined based on connected hardware to this fic.
+ *
+ * Returns fic context that allows the user to obtain the irq_domain by using
+ * al_fic_wire_get_domain().
+ */
+struct al_fic *al_fic_wire_init(struct device_node *node,
+				void __iomem *base,
+				const char *name,
+				unsigned int parent_irq)
+{
+	struct al_fic *fic;
+	int ret;
+
+	if (!base)
+		return ERR_PTR(-EINVAL);
+
+	fic = kzalloc(sizeof(*fic), GFP_KERNEL);
+	if (!fic)
+		return ERR_PTR(-ENOMEM);
+
+	fic->base = base;
+	fic->parent_irq = parent_irq;
+	fic->name = (name ?: "al-fic-wire");
+
+	al_fic_hw_init(fic);
+
+	ret = al_fic_register(node, fic);
+	if (ret) {
+		pr_err("fail to register irqchip\n");
+		goto err_free;
+	}
+
+	pr_debug("%s initialized successfully in Legacy mode (parent-irq=%u)\n",
+		 fic->name, parent_irq);
+
+	return fic;
+
+err_free:
+	kfree(fic);
+	return ERR_PTR(ret);
+}
+EXPORT_SYMBOL_GPL(al_fic_wire_init);
+
+/**
+ * al_fic_cleanup() - free all resources allocated by fic
+ * @fic: pointer to fic context
+ *
+ * This API will release the domain (in case of wire mode) and will free the
+ * context allocated for this fic.
+ * Caller should call this API only after all interrupts were released.
+ *
+ * Returns 0 on success or negative error code.
+ */
+int al_fic_cleanup(struct al_fic *fic)
+{
+	if (!fic)
+		return -EINVAL;
+
+	if (fic->domain) {
+		struct irq_chip_generic *gc;
+
+		gc = irq_get_domain_generic_chip(fic->domain, 0);
+		if (IS_ERR(gc)) {
+			pr_err("%s was not able to grab gc for fic (%lu)\n",
+			       __func__, PTR_ERR(gc));
+			return PTR_ERR(gc);
+		}
+
+		irq_destroy_generic_chip(gc, 0, 0, 0);
+		irq_domain_remove(fic->domain);
+	}
+
+	kfree(fic);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(al_fic_cleanup);
+
+static int __init al_fic_init_dt(struct device_node *node,
+				 struct device_node *parent)
+{
+	int ret;
+	void __iomem *base;
+	unsigned int parent_irq;
+	struct al_fic *fic;
+
+	if (!parent) {
+		pr_err("%s: unsupported - device require a parent\n",
+		       node->name);
+		return -EINVAL;
+	}
+
+	base = of_iomap(node, 0);
+	if (!base) {
+		pr_err("%s: fail to map memory\n", node->name);
+		return -ENOMEM;
+	}
+
+	parent_irq = irq_of_parse_and_map(node, 0);
+	if (!parent_irq) {
+		pr_err("%s: fail to map irq\n", node->name);
+		ret = -EINVAL;
+		goto err_unmap;
+	}
+
+	fic = al_fic_wire_init(node,
+			       base,
+			       node->name,
+			       parent_irq);
+	if (IS_ERR(fic)) {
+		pr_err("%s: fail to initialize irqchip (%lu)\n",
+		       node->name,
+		       PTR_ERR(fic));
+		ret = PTR_ERR(fic);
+		goto err_irq_dispose;
+	}
+
+	return 0;
+
+err_irq_dispose:
+	irq_dispose_mapping(parent_irq);
+err_unmap:
+	iounmap(base);
+
+	return ret;
+}
+
+IRQCHIP_DECLARE(al_fic, "amazon,al-fic", al_fic_init_dt);
diff --git a/include/linux/irqchip/al-fic.h b/include/linux/irqchip/al-fic.h
new file mode 100644
index 0000000..0833749
--- /dev/null
+++ b/include/linux/irqchip/al-fic.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/**
+ * Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ */
+
+#ifndef _AL_FIC_H_
+#define _AL_FIC_H_
+
+#include <linux/irqdomain.h>
+
+struct al_fic;
+
+struct irq_domain *al_fic_wire_get_domain(struct al_fic *fic);
+
+struct al_fic *al_fic_wire_init(struct device_node *node,
+				void __iomem *base,
+				const char *name,
+				unsigned int parent_irq);
+int al_fic_cleanup(struct al_fic *fic);
+
+#endif
-- 
2.7.4

[PATCH 3/3] irqchip: al-fic: Introducing support for MSI-X

[Talel Shenhar]

The FIC supports either a (single) wired output, or generation of an MSI-X
interrupt per input (for cases where it is embedded in a PCIe device,
hence, allowing the PCIe drivers to call this API).
This patch introduces the support for allowing the configuration of MSI-X
instead of a wire interrupt.

Signed-off-by: Talel Shenhar <talel@amazon.com>
---
 drivers/irqchip/irq-al-fic.c   | 48 +++++++++++++++++++++++++++++++++++++++---
 include/linux/irqchip/al-fic.h |  2 ++
 2 files changed, 47 insertions(+), 3 deletions(-)

diff --git a/drivers/irqchip/irq-al-fic.c b/drivers/irqchip/irq-al-fic.c
index d881d42..e49b912 100644
--- a/drivers/irqchip/irq-al-fic.c
+++ b/drivers/irqchip/irq-al-fic.c
@@ -19,6 +19,7 @@
 #define AL_FIC_MASK		0x10
 #define AL_FIC_CONTROL		0x28
 
+#define CONTROL_AUTO_CLEAR	BIT(2)
 #define CONTROL_TRIGGER_RISING	BIT(3)
 #define CONTROL_MASK_MSI_X	BIT(5)
 
@@ -193,9 +194,11 @@ struct irq_domain *al_fic_wire_get_domain(struct al_fic *fic)
 }
 EXPORT_SYMBOL_GPL(al_fic_wire_get_domain);
 
-static void al_fic_hw_init(struct al_fic *fic)
+static void al_fic_hw_init(struct al_fic *fic,
+			   int use_msi)
 {
-	u32 control = CONTROL_MASK_MSI_X;
+	u32 control = (use_msi ? (CONTROL_AUTO_CLEAR | CONTROL_TRIGGER_RISING) :
+		       CONTROL_MASK_MSI_X);
 
 	/* mask out all interrupts */
 	writel(0xFFFFFFFF, fic->base + AL_FIC_MASK);
@@ -240,7 +243,7 @@ struct al_fic *al_fic_wire_init(struct device_node *node,
 	fic->parent_irq = parent_irq;
 	fic->name = (name ?: "al-fic-wire");
 
-	al_fic_hw_init(fic);
+	al_fic_hw_init(fic, false);
 
 	ret = al_fic_register(node, fic);
 	if (ret) {
@@ -260,6 +263,45 @@ struct al_fic *al_fic_wire_init(struct device_node *node,
 EXPORT_SYMBOL_GPL(al_fic_wire_init);
 
 /**
+ * al_fic_msi_x_init() - initialize and configure fic in msi-x mode
+ * @base: mmio to fic register
+ * @name: name of the fic
+ *
+ * This API will configure the fic hardware to to work in msi-x mode.
+ * msi-x fic is to be configured for fics that are embedded inside AL PCIE EP.
+ * Those kind of fic are aware of the fact that they live inside PCIE and
+ * familiar with the MSI-X table which is configured as part of
+ * pci_enable_msix_range() and friends.
+ * Interrupt can be generated based on a positive edge or level - configuration
+ * is to be determined based on connected hardware to this fic.
+ *
+ * Returns pointer to fic context or ERR_PTR in case of error.
+ */
+struct al_fic *al_fic_msi_x_init(void __iomem *base,
+				 const char *name)
+{
+	struct al_fic *fic;
+
+	if (!base)
+		return ERR_PTR(-EINVAL);
+
+	fic = kzalloc(sizeof(*fic), GFP_KERNEL);
+	if (!fic)
+		return ERR_PTR(-ENOMEM);
+
+	fic->base = base;
+	fic->name = (name ?: "al-fic-full-fledged");
+
+	al_fic_hw_init(fic, true);
+
+	pr_debug("%s initialized successfully in Full-Fledged mode\n",
+		 fic->name);
+
+	return fic;
+}
+EXPORT_SYMBOL_GPL(al_fic_msi_x_init);
+
+/**
  * al_fic_cleanup() - free all resources allocated by fic
  * @fic: pointer to fic context
  *
diff --git a/include/linux/irqchip/al-fic.h b/include/linux/irqchip/al-fic.h
index 0833749..a2e89ff 100644
--- a/include/linux/irqchip/al-fic.h
+++ b/include/linux/irqchip/al-fic.h
@@ -16,6 +16,8 @@ struct al_fic *al_fic_wire_init(struct device_node *node,
 				void __iomem *base,
 				const char *name,
 				unsigned int parent_irq);
+struct al_fic *al_fic_msi_x_init(void __iomem *base,
+				 const char *name);
 int al_fic_cleanup(struct al_fic *fic);
 
 #endif
-- 
2.7.4

[PATCH 3/3] irqchip: al-fic: Introducing support for MSI-X

[Talel Shenhar]

The FIC supports either a (single) wired output, or generation of an MSI-X
interrupt per input (for cases where it is embedded in a PCIe device,
hence, allowing the PCIe drivers to call this API).
This patch introduces the support for allowing the configuration of MSI-X
instead of a wire interrupt.

Signed-off-by: Talel Shenhar <talel@amazon.com>
---
 drivers/irqchip/irq-al-fic.c   | 48 +++++++++++++++++++++++++++++++++++++++---
 include/linux/irqchip/al-fic.h |  2 ++
 2 files changed, 47 insertions(+), 3 deletions(-)

diff --git a/drivers/irqchip/irq-al-fic.c b/drivers/irqchip/irq-al-fic.c
index d881d42..e49b912 100644
--- a/drivers/irqchip/irq-al-fic.c
+++ b/drivers/irqchip/irq-al-fic.c
@@ -19,6 +19,7 @@
 #define AL_FIC_MASK		0x10
 #define AL_FIC_CONTROL		0x28
 
+#define CONTROL_AUTO_CLEAR	BIT(2)
 #define CONTROL_TRIGGER_RISING	BIT(3)
 #define CONTROL_MASK_MSI_X	BIT(5)
 
@@ -193,9 +194,11 @@ struct irq_domain *al_fic_wire_get_domain(struct al_fic *fic)
 }
 EXPORT_SYMBOL_GPL(al_fic_wire_get_domain);
 
-static void al_fic_hw_init(struct al_fic *fic)
+static void al_fic_hw_init(struct al_fic *fic,
+			   int use_msi)
 {
-	u32 control = CONTROL_MASK_MSI_X;
+	u32 control = (use_msi ? (CONTROL_AUTO_CLEAR | CONTROL_TRIGGER_RISING) :
+		       CONTROL_MASK_MSI_X);
 
 	/* mask out all interrupts */
 	writel(0xFFFFFFFF, fic->base + AL_FIC_MASK);
@@ -240,7 +243,7 @@ struct al_fic *al_fic_wire_init(struct device_node *node,
 	fic->parent_irq = parent_irq;
 	fic->name = (name ?: "al-fic-wire");
 
-	al_fic_hw_init(fic);
+	al_fic_hw_init(fic, false);
 
 	ret = al_fic_register(node, fic);
 	if (ret) {
@@ -260,6 +263,45 @@ struct al_fic *al_fic_wire_init(struct device_node *node,
 EXPORT_SYMBOL_GPL(al_fic_wire_init);
 
 /**
+ * al_fic_msi_x_init() - initialize and configure fic in msi-x mode
+ * @base: mmio to fic register
+ * @name: name of the fic
+ *
+ * This API will configure the fic hardware to to work in msi-x mode.
+ * msi-x fic is to be configured for fics that are embedded inside AL PCIE EP.
+ * Those kind of fic are aware of the fact that they live inside PCIE and
+ * familiar with the MSI-X table which is configured as part of
+ * pci_enable_msix_range() and friends.
+ * Interrupt can be generated based on a positive edge or level - configuration
+ * is to be determined based on connected hardware to this fic.
+ *
+ * Returns pointer to fic context or ERR_PTR in case of error.
+ */
+struct al_fic *al_fic_msi_x_init(void __iomem *base,
+				 const char *name)
+{
+	struct al_fic *fic;
+
+	if (!base)
+		return ERR_PTR(-EINVAL);
+
+	fic = kzalloc(sizeof(*fic), GFP_KERNEL);
+	if (!fic)
+		return ERR_PTR(-ENOMEM);
+
+	fic->base = base;
+	fic->name = (name ?: "al-fic-full-fledged");
+
+	al_fic_hw_init(fic, true);
+
+	pr_debug("%s initialized successfully in Full-Fledged mode\n",
+		 fic->name);
+
+	return fic;
+}
+EXPORT_SYMBOL_GPL(al_fic_msi_x_init);
+
+/**
  * al_fic_cleanup() - free all resources allocated by fic
  * @fic: pointer to fic context
  *
diff --git a/include/linux/irqchip/al-fic.h b/include/linux/irqchip/al-fic.h
index 0833749..a2e89ff 100644
--- a/include/linux/irqchip/al-fic.h
+++ b/include/linux/irqchip/al-fic.h
@@ -16,6 +16,8 @@ struct al_fic *al_fic_wire_init(struct device_node *node,
 				void __iomem *base,
 				const char *name,
 				unsigned int parent_irq);
+struct al_fic *al_fic_msi_x_init(void __iomem *base,
+				 const char *name);
 int al_fic_cleanup(struct al_fic *fic);
 
 #endif
-- 
2.7.4

Re: [PATCH 2/3] irqchip: al-fic: Introduce Amazon's Annapurna Labs Fabric Interrupt Controller Driver

[Greg KH]

On Wed, Jun 05, 2019 at 09:54:12AM +0300, Talel Shenhar wrote:
> --- /dev/null
> +++ b/include/linux/irqchip/al-fic.h
> @@ -0,0 +1,21 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/**
> + * Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
> + */
> +
> +#ifndef _AL_FIC_H_
> +#define _AL_FIC_H_
> +
> +#include <linux/irqdomain.h>
> +
> +struct al_fic;
> +
> +struct irq_domain *al_fic_wire_get_domain(struct al_fic *fic);
> +
> +struct al_fic *al_fic_wire_init(struct device_node *node,
> +				void __iomem *base,
> +				const char *name,
> +				unsigned int parent_irq);
> +int al_fic_cleanup(struct al_fic *fic);

Who is using these new functions?  We don't add new apis that no one
uses :(

thanks,

greg k-h

Re: [PATCH 3/3] irqchip: al-fic: Introducing support for MSI-X

[Greg KH]

On Wed, Jun 05, 2019 at 09:54:13AM +0300, Talel Shenhar wrote:
> The FIC supports either a (single) wired output, or generation of an MSI-X
> interrupt per input (for cases where it is embedded in a PCIe device,
> hence, allowing the PCIe drivers to call this API).
> This patch introduces the support for allowing the configuration of MSI-X
> instead of a wire interrupt.
> 
> Signed-off-by: Talel Shenhar <talel@amazon.com>
> ---
>  drivers/irqchip/irq-al-fic.c   | 48 +++++++++++++++++++++++++++++++++++++++---
>  include/linux/irqchip/al-fic.h |  2 ++
>  2 files changed, 47 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/irqchip/irq-al-fic.c b/drivers/irqchip/irq-al-fic.c
> index d881d42..e49b912 100644
> --- a/drivers/irqchip/irq-al-fic.c
> +++ b/drivers/irqchip/irq-al-fic.c
> @@ -19,6 +19,7 @@
>  #define AL_FIC_MASK		0x10
>  #define AL_FIC_CONTROL		0x28
>  
> +#define CONTROL_AUTO_CLEAR	BIT(2)
>  #define CONTROL_TRIGGER_RISING	BIT(3)
>  #define CONTROL_MASK_MSI_X	BIT(5)
>  
> @@ -193,9 +194,11 @@ struct irq_domain *al_fic_wire_get_domain(struct al_fic *fic)
>  }
>  EXPORT_SYMBOL_GPL(al_fic_wire_get_domain);
>  
> -static void al_fic_hw_init(struct al_fic *fic)
> +static void al_fic_hw_init(struct al_fic *fic,
> +			   int use_msi)
>  {
> -	u32 control = CONTROL_MASK_MSI_X;
> +	u32 control = (use_msi ? (CONTROL_AUTO_CLEAR | CONTROL_TRIGGER_RISING) :
> +		       CONTROL_MASK_MSI_X);
>  
>  	/* mask out all interrupts */
>  	writel(0xFFFFFFFF, fic->base + AL_FIC_MASK);
> @@ -240,7 +243,7 @@ struct al_fic *al_fic_wire_init(struct device_node *node,
>  	fic->parent_irq = parent_irq;
>  	fic->name = (name ?: "al-fic-wire");
>  
> -	al_fic_hw_init(fic);
> +	al_fic_hw_init(fic, false);
>  
>  	ret = al_fic_register(node, fic);
>  	if (ret) {
> @@ -260,6 +263,45 @@ struct al_fic *al_fic_wire_init(struct device_node *node,
>  EXPORT_SYMBOL_GPL(al_fic_wire_init);
>  
>  /**
> + * al_fic_msi_x_init() - initialize and configure fic in msi-x mode
> + * @base: mmio to fic register
> + * @name: name of the fic
> + *
> + * This API will configure the fic hardware to to work in msi-x mode.
> + * msi-x fic is to be configured for fics that are embedded inside AL PCIE EP.
> + * Those kind of fic are aware of the fact that they live inside PCIE and
> + * familiar with the MSI-X table which is configured as part of
> + * pci_enable_msix_range() and friends.
> + * Interrupt can be generated based on a positive edge or level - configuration
> + * is to be determined based on connected hardware to this fic.
> + *
> + * Returns pointer to fic context or ERR_PTR in case of error.
> + */
> +struct al_fic *al_fic_msi_x_init(void __iomem *base,
> +				 const char *name)
> +{
> +	struct al_fic *fic;
> +
> +	if (!base)
> +		return ERR_PTR(-EINVAL);
> +
> +	fic = kzalloc(sizeof(*fic), GFP_KERNEL);
> +	if (!fic)
> +		return ERR_PTR(-ENOMEM);
> +
> +	fic->base = base;
> +	fic->name = (name ?: "al-fic-full-fledged");
> +
> +	al_fic_hw_init(fic, true);
> +
> +	pr_debug("%s initialized successfully in Full-Fledged mode\n",
> +		 fic->name);
> +
> +	return fic;
> +}
> +EXPORT_SYMBOL_GPL(al_fic_msi_x_init);
> +
> +/**
>   * al_fic_cleanup() - free all resources allocated by fic
>   * @fic: pointer to fic context
>   *
> diff --git a/include/linux/irqchip/al-fic.h b/include/linux/irqchip/al-fic.h
> index 0833749..a2e89ff 100644
> --- a/include/linux/irqchip/al-fic.h
> +++ b/include/linux/irqchip/al-fic.h
> @@ -16,6 +16,8 @@ struct al_fic *al_fic_wire_init(struct device_node *node,
>  				void __iomem *base,
>  				const char *name,
>  				unsigned int parent_irq);
> +struct al_fic *al_fic_msi_x_init(void __iomem *base,
> +				 const char *name);

Who uses this new function?

thanks,

greg k-h

Re: [PATCH 2/3] irqchip: al-fic: Introduce Amazon's Annapurna Labs Fabric Interrupt Controller Driver

[Benjamin Herrenschmidt]

On Wed, 2019-06-05 at 09:59 +0200, Greg KH wrote:
> 
> > +struct irq_domain *al_fic_wire_get_domain(struct al_fic *fic);
> > +
> > +struct al_fic *al_fic_wire_init(struct device_node *node,
> > +				void __iomem *base,
> > +				const char *name,
> > +				unsigned int parent_irq);
> > +int al_fic_cleanup(struct al_fic *fic);
> 
> Who is using these new functions?  We don't add new apis that no one
> uses :(

They will be used by subsequent driver submissions but those aren't
quite ready yet, so we can hold onto patch 3 for now until they are.

Cheers,
Ben.

Re: [PATCH 2/3] irqchip: al-fic: Introduce Amazon's Annapurna Labs Fabric Interrupt Controller Driver

[Greg KH]

On Thu, Jun 06, 2019 at 07:55:43AM +1000, Benjamin Herrenschmidt wrote:
> On Wed, 2019-06-05 at 09:59 +0200, Greg KH wrote:
> > 
> > > +struct irq_domain *al_fic_wire_get_domain(struct al_fic *fic);
> > > +
> > > +struct al_fic *al_fic_wire_init(struct device_node *node,
> > > +				void __iomem *base,
> > > +				const char *name,
> > > +				unsigned int parent_irq);
> > > +int al_fic_cleanup(struct al_fic *fic);
> > 
> > Who is using these new functions?  We don't add new apis that no one
> > uses :(
> 
> They will be used by subsequent driver submissions but those aren't
> quite ready yet, so we can hold onto patch 3 for now until they are.

Patch 2 also should have these removed :)

You know we don't add new apis until we have a real, in-kernel user for
them...

thanks,

greg k-h

Re: [PATCH 2/3] irqchip: al-fic: Introduce Amazon's Annapurna Labs Fabric Interrupt Controller Driver

[Benjamin Herrenschmidt]

On Thu, 2019-06-06 at 08:37 +0200, Greg KH wrote:
> On Thu, Jun 06, 2019 at 07:55:43AM +1000, Benjamin Herrenschmidt wrote:
> > On Wed, 2019-06-05 at 09:59 +0200, Greg KH wrote:
> > > 
> > > > +struct irq_domain *al_fic_wire_get_domain(struct al_fic *fic);
> > > > +
> > > > +struct al_fic *al_fic_wire_init(struct device_node *node,
> > > > +				void __iomem *base,
> > > > +				const char *name,
> > > > +				unsigned int parent_irq);
> > > > +int al_fic_cleanup(struct al_fic *fic);
> > > 
> > > Who is using these new functions?  We don't add new apis that no one
> > > uses :(
> > 
> > They will be used by subsequent driver submissions but those aren't
> > quite ready yet, so we can hold onto patch 3 for now until they are.
> 
> Patch 2 also should have these removed :)

That's a mistake, that export should have been in patch3. Talel, pls
fix that in your next spin.

> You know we don't add new apis until we have a real, in-kernel user for
> them...

Yup, the user are going to be drivers in other subsystems, so
coordination is a bit tricky, which is I think why Talel wanted to
submit that now, but the patches for those other drivers aren't quite
ready yet so we can hold onto that one for the time being.

It's nothing nefarious :-) Just coordination issues.

Cheers,
Ben.

> thanks,
> 
> greg k-h

Re: [PATCH 2/3] irqchip: al-fic: Introduce Amazon's Annapurna Labs Fabric Interrupt Controller Driver

[Benjamin Herrenschmidt]

On Thu, 2019-06-06 at 16:47 +1000, Benjamin Herrenschmidt wrote:
> On Thu, 2019-06-06 at 08:37 +0200, Greg KH wrote:
> > On Thu, Jun 06, 2019 at 07:55:43AM +1000, Benjamin Herrenschmidt wrote:
> > > On Wed, 2019-06-05 at 09:59 +0200, Greg KH wrote:
> > > > 
> > > > > +struct irq_domain *al_fic_wire_get_domain(struct al_fic *fic);
> > > > > +
> > > > > +struct al_fic *al_fic_wire_init(struct device_node *node,
> > > > > +				void __iomem *base,
> > > > > +				const char *name,
> > > > > +				unsigned int parent_irq);
> > > > > +int al_fic_cleanup(struct al_fic *fic);
> > > > 
> > > > Who is using these new functions?  We don't add new apis that no one
> > > > uses :(
> > > 
> > > They will be used by subsequent driver submissions but those aren't
> > > quite ready yet, so we can hold onto patch 3 for now until they are.
> > 
> > Patch 2 also should have these removed :)
> 
> That's a mistake, that export should have been in patch3. Talel, pls
> fix that in your next spin.

Actually that's already fixed in v2 of the series. The API have been
removed for now.

Cheers,
Ben.