* [PATCH 0/2 V6] irqchip: gic: Introduce ARM GICv2m MSI(-X) support
@ 2014-09-14 6:30 ` suravee.suthikulpanit
0 siblings, 0 replies; 17+ messages in thread
From: suravee.suthikulpanit @ 2014-09-14 6:30 UTC (permalink / raw)
To: marc.zyngier, mark.rutland, jason
Cc: pawel.moll, Catalin.Marinas, Will.Deacon, tglx,
Harish.Kasiviswanathan, linux-arm-kernel, linux-pci,
linux-kernel, linux-doc, devicetree, Suravee Suthikulpanit
From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
This patch set introduces support for MSI(-X) in GICv2m specification,
which is implemented in some variation of GIC400.
This depends on and has been tested with the following patch set which
implements PCI supports for ARM64:
* https://lkml.org/lkml/2014/8/12/394
* https://lkml.org/lkml/2014/8/12/361
Changes in V6:
* Fix the logic in patch 1 in drivers/irqchip/irq-gic.c: gic_of_init()
to print warning message when gicv2m_of_init fails instead of
returning error since v2m is optional.
* Minor clean up
Changes in V5:
* Rebase to git://git.infradead.org/users/jcooper/linux.git irqchip/core
Marc Zyngier suggestions:
* Only use GICv2m irq_chip for MSI interrupts.
* Simplify logic to support multi-MSI in arch/arm64/kernel/msi.c.
* Modify gicv2m_setup_msi_irq() to also handle multi-MSI.
Mark Rutlan suggestions:
* V4 patch set did not support multiple MSI register frame within a GIC.
Although, the proposed GICv2m device tree binding should be able to
handle the case. Mark was questioning on how we can properly handle
this in the code. Therefore, I try to implement this by iterating through
the subnodes and look for msi-controller property. Once found, the code
parses v2m register frame information and store it in the v2m_list of
each gic_chip_data.
Jingoo han suggestions:
* Misc clean up.
Suravee Suthikulpanit (2):
irqchip: gic: Add supports for ARM GICv2m MSI(-X)
irqchip: gicv2m: Add support for multiple MSI for ARM64 GICv2m
Documentation/devicetree/bindings/arm/gic.txt | 39 +++
arch/arm64/kernel/Makefile | 1 +
arch/arm64/kernel/msi.c | 41 ++++
drivers/irqchip/Kconfig | 7 +
drivers/irqchip/Makefile | 1 +
drivers/irqchip/irq-gic-v2m.c | 328 ++++++++++++++++++++++++++
drivers/irqchip/irq-gic.c | 88 ++++---
drivers/irqchip/irq-gic.h | 51 ++++
8 files changed, 528 insertions(+), 28 deletions(-)
create mode 100644 arch/arm64/kernel/msi.c
create mode 100644 drivers/irqchip/irq-gic-v2m.c
create mode 100644 drivers/irqchip/irq-gic.h
--
1.9.3
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH 0/2 V6] irqchip: gic: Introduce ARM GICv2m MSI(-X) support
@ 2014-09-14 6:30 ` suravee.suthikulpanit
0 siblings, 0 replies; 17+ messages in thread
From: suravee.suthikulpanit @ 2014-09-14 6:30 UTC (permalink / raw)
To: marc.zyngier, mark.rutland, jason
Cc: pawel.moll, Catalin.Marinas, Will.Deacon, tglx,
Harish.Kasiviswanathan, linux-arm-kernel, linux-pci,
linux-kernel, linux-doc, devicetree, Suravee Suthikulpanit
From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
This patch set introduces support for MSI(-X) in GICv2m specification,
which is implemented in some variation of GIC400.
This depends on and has been tested with the following patch set which
implements PCI supports for ARM64:
* https://lkml.org/lkml/2014/8/12/394
* https://lkml.org/lkml/2014/8/12/361
Changes in V6:
* Fix the logic in patch 1 in drivers/irqchip/irq-gic.c: gic_of_init()
to print warning message when gicv2m_of_init fails instead of
returning error since v2m is optional.
* Minor clean up
Changes in V5:
* Rebase to git://git.infradead.org/users/jcooper/linux.git irqchip/core
Marc Zyngier suggestions:
* Only use GICv2m irq_chip for MSI interrupts.
* Simplify logic to support multi-MSI in arch/arm64/kernel/msi.c.
* Modify gicv2m_setup_msi_irq() to also handle multi-MSI.
Mark Rutlan suggestions:
* V4 patch set did not support multiple MSI register frame within a GIC.
Although, the proposed GICv2m device tree binding should be able to
handle the case. Mark was questioning on how we can properly handle
this in the code. Therefore, I try to implement this by iterating through
the subnodes and look for msi-controller property. Once found, the code
parses v2m register frame information and store it in the v2m_list of
each gic_chip_data.
Jingoo han suggestions:
* Misc clean up.
Suravee Suthikulpanit (2):
irqchip: gic: Add supports for ARM GICv2m MSI(-X)
irqchip: gicv2m: Add support for multiple MSI for ARM64 GICv2m
Documentation/devicetree/bindings/arm/gic.txt | 39 +++
arch/arm64/kernel/Makefile | 1 +
arch/arm64/kernel/msi.c | 41 ++++
drivers/irqchip/Kconfig | 7 +
drivers/irqchip/Makefile | 1 +
drivers/irqchip/irq-gic-v2m.c | 328 ++++++++++++++++++++++++++
drivers/irqchip/irq-gic.c | 88 ++++---
drivers/irqchip/irq-gic.h | 51 ++++
8 files changed, 528 insertions(+), 28 deletions(-)
create mode 100644 arch/arm64/kernel/msi.c
create mode 100644 drivers/irqchip/irq-gic-v2m.c
create mode 100644 drivers/irqchip/irq-gic.h
--
1.9.3
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH 0/2 V6] irqchip: gic: Introduce ARM GICv2m MSI(-X) support
@ 2014-09-14 6:30 ` suravee.suthikulpanit
0 siblings, 0 replies; 17+ messages in thread
From: suravee.suthikulpanit at amd.com @ 2014-09-14 6:30 UTC (permalink / raw)
To: linux-arm-kernel
From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
This patch set introduces support for MSI(-X) in GICv2m specification,
which is implemented in some variation of GIC400.
This depends on and has been tested with the following patch set which
implements PCI supports for ARM64:
* https://lkml.org/lkml/2014/8/12/394
* https://lkml.org/lkml/2014/8/12/361
Changes in V6:
* Fix the logic in patch 1 in drivers/irqchip/irq-gic.c: gic_of_init()
to print warning message when gicv2m_of_init fails instead of
returning error since v2m is optional.
* Minor clean up
Changes in V5:
* Rebase to git://git.infradead.org/users/jcooper/linux.git irqchip/core
Marc Zyngier suggestions:
* Only use GICv2m irq_chip for MSI interrupts.
* Simplify logic to support multi-MSI in arch/arm64/kernel/msi.c.
* Modify gicv2m_setup_msi_irq() to also handle multi-MSI.
Mark Rutlan suggestions:
* V4 patch set did not support multiple MSI register frame within a GIC.
Although, the proposed GICv2m device tree binding should be able to
handle the case. Mark was questioning on how we can properly handle
this in the code. Therefore, I try to implement this by iterating through
the subnodes and look for msi-controller property. Once found, the code
parses v2m register frame information and store it in the v2m_list of
each gic_chip_data.
Jingoo han suggestions:
* Misc clean up.
Suravee Suthikulpanit (2):
irqchip: gic: Add supports for ARM GICv2m MSI(-X)
irqchip: gicv2m: Add support for multiple MSI for ARM64 GICv2m
Documentation/devicetree/bindings/arm/gic.txt | 39 +++
arch/arm64/kernel/Makefile | 1 +
arch/arm64/kernel/msi.c | 41 ++++
drivers/irqchip/Kconfig | 7 +
drivers/irqchip/Makefile | 1 +
drivers/irqchip/irq-gic-v2m.c | 328 ++++++++++++++++++++++++++
drivers/irqchip/irq-gic.c | 88 ++++---
drivers/irqchip/irq-gic.h | 51 ++++
8 files changed, 528 insertions(+), 28 deletions(-)
create mode 100644 arch/arm64/kernel/msi.c
create mode 100644 drivers/irqchip/irq-gic-v2m.c
create mode 100644 drivers/irqchip/irq-gic.h
--
1.9.3
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH 1/2 V6] irqchip: gic: Add supports for ARM GICv2m MSI(-X)
2014-09-14 6:30 ` suravee.suthikulpanit
(?)
@ 2014-09-14 6:30 ` suravee.suthikulpanit
-1 siblings, 0 replies; 17+ messages in thread
From: suravee.suthikulpanit @ 2014-09-14 6:30 UTC (permalink / raw)
To: marc.zyngier, mark.rutland, jason
Cc: pawel.moll, Catalin.Marinas, Will.Deacon, tglx,
Harish.Kasiviswanathan, linux-arm-kernel, linux-pci,
linux-kernel, linux-doc, devicetree, Suravee Suthikulpanit,
Mark Rutland, Marc Zyngier
From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
ARM GICv2m specification extends GICv2 to support MSI(-X) with
a new set of register frame. This patch introduces support for
the non-secure GICv2m register frame. Currently, GICV2m is available
in certain version of GIC-400.
The patch introduces a new property in ARM gic binding, the v2m subnode.
It is optional.
Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Cc: Mark Rutland <Mark.Rutland@arm.com>
Cc: Marc Zyngier <Marc.Zyngier@arm.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Catalin Marinas <Catalin.Marinas@arm.com>
Cc: Will Deacon <Will.Deacon@arm.com>
---
Documentation/devicetree/bindings/arm/gic.txt | 39 ++++
drivers/irqchip/Kconfig | 7 +
drivers/irqchip/Makefile | 1 +
drivers/irqchip/irq-gic-v2m.c | 261 ++++++++++++++++++++++++++
drivers/irqchip/irq-gic.c | 88 ++++++---
drivers/irqchip/irq-gic.h | 51 +++++
6 files changed, 419 insertions(+), 28 deletions(-)
create mode 100644 drivers/irqchip/irq-gic-v2m.c
create mode 100644 drivers/irqchip/irq-gic.h
diff --git a/Documentation/devicetree/bindings/arm/gic.txt b/Documentation/devicetree/bindings/arm/gic.txt
index c7d2fa1..a461591 100644
--- a/Documentation/devicetree/bindings/arm/gic.txt
+++ b/Documentation/devicetree/bindings/arm/gic.txt
@@ -96,3 +96,42 @@ Example:
<0x2c006000 0x2000>;
interrupts = <1 9 0xf04>;
};
+
+
+* GICv2m extension for MSI/MSI-x support (Optional)
+
+Certain revision of GIC-400 supports MSI/MSI-x via V2M register frame.
+This is enabled by specifying v2m sub-node.
+
+Required properties:
+
+- msi-controller : Identifies the node as an MSI controller.
+
+- reg : GICv2m MSI interface register base and size
+
+Example:
+
+ interrupt-controller@e1101000 {
+ compatible = "arm,gic-400";
+ #interrupt-cells = <3>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-controller;
+ interrupts = <1 8 0xf04>;
+ ranges = <0 0 0 0xe1100000 0 0x100000>;
+ reg = <0x0 0xe1110000 0 0x01000>,
+ <0x0 0xe112f000 0 0x02000>,
+ <0x0 0xe1140000 0 0x10000>,
+ <0x0 0xe1160000 0 0x10000>;
+ v2m0 {
+ msi-controller;
+ reg = <0x0 0x80000 0 0x1000>;
+ };
+
+ ....
+
+ v2mN {
+ msi-controller;
+ reg = <0x0 0x90000 0 0x1000>;
+ };
+ };
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index b8632bf..ddb31ee 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -7,6 +7,13 @@ config ARM_GIC
select IRQ_DOMAIN
select MULTI_IRQ_HANDLER
+config ARM_GIC_V2M
+ bool
+ select IRQ_DOMAIN
+ select MULTI_IRQ_HANDLER
+ depends on ARM_GIC
+ depends on PCI && PCI_MSI
+
config GIC_NON_BANKED
bool
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 73052ba..3bda951 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -17,6 +17,7 @@ obj-$(CONFIG_ARCH_SUNXI) += irq-sun4i.o
obj-$(CONFIG_ARCH_SUNXI) += irq-sunxi-nmi.o
obj-$(CONFIG_ARCH_SPEAR3XX) += spear-shirq.o
obj-$(CONFIG_ARM_GIC) += irq-gic.o irq-gic-common.o
+obj-$(CONFIG_ARM_GIC_V2M) += irq-gic-v2m.o
obj-$(CONFIG_ARM_GIC_V3) += irq-gic-v3.o irq-gic-common.o
obj-$(CONFIG_ARM_NVIC) += irq-nvic.o
obj-$(CONFIG_ARM_VIC) += irq-vic.o
diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
new file mode 100644
index 0000000..499bfb9
--- /dev/null
+++ b/drivers/irqchip/irq-gic-v2m.c
@@ -0,0 +1,261 @@
+/*
+ * ARM GIC v2m MSI(-X) support
+ * Support for Message Signaled Interrupts for systems that
+ * implement ARM Generic Interrupt Controller: GICv2m.
+ *
+ * Copyright (C) 2014 Advanced Micro Devices, Inc.
+ * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
+ * Harish Kasiviswanathan <harish.kasiviswanathan@amd.com>
+ * Brandon Anderson <brandon.anderson@amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ */
+
+#define pr_fmt(fmt) "GICv2m: " fmt
+
+#include <linux/bitmap.h>
+#include <linux/irq.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_pci.h>
+#include <linux/pci.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#include "irqchip.h"
+#include "irq-gic.h"
+
+/*
+* MSI_TYPER:
+* [31:26] Reserved
+* [25:16] lowest SPI assigned to MSI
+* [15:10] Reserved
+* [9:0] Numer of SPIs assigned to MSI
+*/
+#define V2M_MSI_TYPER 0x008
+#define V2M_MSI_TYPER_BASE_SHIFT 16
+#define V2M_MSI_TYPER_BASE_MASK 0x3FF
+#define V2M_MSI_TYPER_NUM_MASK 0x3FF
+#define V2M_MSI_SETSPI_NS 0x040
+#define V2M_MIN_SPI 32
+#define V2M_MAX_SPI 1019
+
+/*
+ * alloc_msi_irq - Allocate MSIs from available MSI bitmap.
+ * @data: Pointer to v2m_data
+ * @nvec: Number of interrupts to allocate
+ * @irq: Pointer to the allocated irq
+ *
+ * Allocates interrupts only if the contiguous range of MSIs
+ * with specified nvec are available. Otherwise return the number
+ * of available interrupts. If none are available, then returns -ENOENT.
+ */
+static int alloc_msi_irq(struct v2m_data *data, int nvec, int *irq)
+{
+ int size = data->nr_spis;
+ int next = size, i = nvec, ret;
+
+ /* We should never allocate more than available nr_spis */
+ if (i >= size)
+ i = size;
+
+ spin_lock(&data->msi_cnt_lock);
+
+ for (; i > 0; i--) {
+ next = bitmap_find_next_zero_area(data->bm,
+ size, 0, i, 0);
+ if (next < size)
+ break;
+ }
+
+ if (i != nvec) {
+ ret = i ? : -ENOENT;
+ } else {
+ bitmap_set(data->bm, next, nvec);
+ *irq = data->spi_start + next;
+ ret = 0;
+ }
+
+ spin_unlock(&data->msi_cnt_lock);
+
+ return ret;
+}
+
+static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
+{
+ int pos;
+ struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
+
+ spin_lock(&data->msi_cnt_lock);
+
+ pos = irq - data->spi_start;
+ if (pos >= 0 && pos < data->nr_spis)
+ bitmap_clear(data->bm, pos, 1);
+
+ spin_unlock(&data->msi_cnt_lock);
+}
+
+static int gicv2m_setup_msi_irq(struct msi_chip *chip,
+ struct pci_dev *pdev,
+ struct msi_desc *desc)
+{
+ int irq, avail;
+ struct msi_msg msg;
+ phys_addr_t addr;
+ struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
+
+ if (!desc) {
+ dev_err(&pdev->dev,
+ "GICv2m: MSI setup failed. Invalid msi descriptor\n");
+ return -EINVAL;
+ }
+
+ avail = alloc_msi_irq(data, 1, &irq);
+ if (avail != 0) {
+ dev_err(&pdev->dev,
+ "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
+ return -ENOSPC;
+ }
+
+ irq_set_chip_data(irq, chip);
+ irq_set_msi_desc(irq, desc);
+ irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
+
+ addr = data->res.start + V2M_MSI_SETSPI_NS;
+ msg.address_hi = (u32)(addr >> 32);
+ msg.address_lo = (u32)(addr);
+ msg.data = irq;
+ write_msi_msg(irq, &msg);
+
+ return 0;
+}
+
+static void gicv2m_mask_irq(struct irq_data *d)
+{
+ gic_mask_irq(d);
+ if (d->msi_desc)
+ mask_msi_irq(d);
+}
+
+static void gicv2m_unmask_irq(struct irq_data *d)
+{
+ gic_unmask_irq(d);
+ if (d->msi_desc)
+ unmask_msi_irq(d);
+}
+
+#ifdef CONFIG_OF
+
+static int __init
+v2m_init(struct device_node *node, struct v2m_data *v2m,
+ struct resource *res)
+{
+ int ret;
+ unsigned int typer;
+
+ v2m->msi_chip.owner = THIS_MODULE;
+ v2m->msi_chip.of_node = node;
+ v2m->msi_chip.setup_irq = gicv2m_setup_msi_irq;
+ v2m->msi_chip.teardown_irq = gicv2m_teardown_msi_irq;
+ ret = of_pci_msi_chip_add(&v2m->msi_chip);
+ if (ret) {
+ pr_info("Failed to add msi_chip.\n");
+ return ret;
+ }
+
+ v2m->base = ioremap(res->start, resource_size(res));
+ if (!v2m->base) {
+ pr_err("Failed to map GICv2m resource\n");
+ return -EINVAL;
+ }
+
+ memcpy(&v2m->res, res, sizeof(struct resource));
+
+ typer = readl_relaxed(v2m->base + V2M_MSI_TYPER);
+ if (!typer) {
+ pr_warn("Failed to read V2M_MSI_TYPER register\n");
+ return -EINVAL;
+ }
+
+ v2m->spi_start = (typer >> V2M_MSI_TYPER_BASE_SHIFT) &
+ V2M_MSI_TYPER_BASE_MASK;
+ v2m->nr_spis = typer & V2M_MSI_TYPER_NUM_MASK;
+ if ((v2m->spi_start < V2M_MIN_SPI) || (v2m->nr_spis >= V2M_MAX_SPI)) {
+ pr_err("Invalid MSI_TYPER (%#x)\n", typer);
+ return -EINVAL;
+ }
+
+ v2m->bm = kzalloc(sizeof(long) * BITS_TO_LONGS(v2m->nr_spis),
+ GFP_KERNEL);
+ if (!v2m->bm) {
+ pr_err("Failed to allocate MSI bitmap\n");
+ return -ENOMEM;
+ }
+
+ spin_lock_init(&v2m->msi_cnt_lock);
+
+ pr_info("SPI range [%d:%d]\n",
+ v2m->spi_start, (v2m->spi_start + v2m->nr_spis));
+
+ return 0;
+}
+
+int __init gicv2m_of_init(struct device_node *node,
+ struct gic_chip_data *gic,
+ struct irq_chip *v2m_chip)
+{
+ int ret = 0;
+ struct v2m_data *v2m;
+ struct resource res;
+ struct device_node *child = NULL;
+
+ INIT_LIST_HEAD(&gic->v2m_list);
+
+ for (;;) {
+ child = of_get_next_child(node, child);
+ if (!child)
+ break;
+
+ if (!of_find_property(child, "msi-controller", NULL))
+ continue;
+
+ ret = of_address_to_resource(child, 0, &res);
+ if (ret) {
+ pr_err("Failed to allocate v2m resource.\n");
+ break;
+ }
+
+ v2m = kzalloc(sizeof(struct v2m_data), GFP_KERNEL);
+ if (!v2m) {
+ pr_err("Failed to allocate struct v2m_data.\n");
+ return -ENOMEM;
+ }
+
+ ret = v2m_init(child, v2m, &res);
+ if (ret)
+ break;
+
+ v2m->gic = gic;
+
+ pr_info(" %s: [%lx : %lx]\n", child->name,
+ (unsigned long)res.start, (unsigned long)res.end);
+
+ list_add_tail(&v2m->list, &gic->v2m_list);
+ }
+
+ if (ret && list_empty(&gic->v2m_list)) {
+ pr_err("MSI resource not found\n");
+ return -EINVAL;
+ }
+
+ v2m_chip->irq_mask = gicv2m_mask_irq;
+ v2m_chip->irq_unmask = gicv2m_unmask_irq;
+
+ return ret;
+}
+
+#endif /* CONFIG_OF */
diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 4b959e6..f490599 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -46,30 +46,9 @@
#include <asm/smp_plat.h>
#include "irq-gic-common.h"
+#include "irq-gic.h"
#include "irqchip.h"
-union gic_base {
- void __iomem *common_base;
- void __percpu * __iomem *percpu_base;
-};
-
-struct gic_chip_data {
- union gic_base dist_base;
- union gic_base cpu_base;
-#ifdef CONFIG_CPU_PM
- u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
- u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
- u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
- u32 __percpu *saved_ppi_enable;
- u32 __percpu *saved_ppi_conf;
-#endif
- struct irq_domain *domain;
- unsigned int gic_irqs;
-#ifdef CONFIG_GIC_NON_BANKED
- void __iomem *(*get_base)(union gic_base *);
-#endif
-};
-
static DEFINE_RAW_SPINLOCK(irq_controller_lock);
/*
@@ -131,15 +110,36 @@ static inline void gic_set_base_accessor(struct gic_chip_data *data,
#define gic_set_base_accessor(d, f)
#endif
+static inline
+struct gic_chip_data *irq_data_get_gic_chip_data(struct irq_data *d)
+{
+ struct gic_chip_data *gic_data;
+ struct msi_chip *mchip;
+ struct v2m_data *v2mdat;
+
+ /*
+ * For MSI, irq_data.chip_data points to struct msi_chip.
+ * For non-MSI, irq_data.chip_data points to struct gic_chip_data.
+ */
+ if (d->msi_desc) {
+ mchip = irq_data_get_irq_chip_data(d);
+ v2mdat = container_of(mchip, struct v2m_data, msi_chip);
+ gic_data = v2mdat->gic;
+ } else {
+ gic_data = irq_data_get_irq_chip_data(d);
+ }
+ return gic_data;
+}
+
static inline void __iomem *gic_dist_base(struct irq_data *d)
{
- struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
+ struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
return gic_data_dist_base(gic_data);
}
static inline void __iomem *gic_cpu_base(struct irq_data *d)
{
- struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
+ struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
return gic_data_cpu_base(gic_data);
}
@@ -151,7 +151,7 @@ static inline unsigned int gic_irq(struct irq_data *d)
/*
* Routines to acknowledge, disable and enable interrupts
*/
-static void gic_mask_irq(struct irq_data *d)
+void gic_mask_irq(struct irq_data *d)
{
u32 mask = 1 << (gic_irq(d) % 32);
@@ -162,7 +162,7 @@ static void gic_mask_irq(struct irq_data *d)
raw_spin_unlock(&irq_controller_lock);
}
-static void gic_unmask_irq(struct irq_data *d)
+void gic_unmask_irq(struct irq_data *d)
{
u32 mask = 1 << (gic_irq(d) % 32);
@@ -325,6 +325,15 @@ static struct irq_chip gic_chip = {
.irq_set_wake = gic_set_wake,
};
+static struct irq_chip v2m_chip = {
+ .name = "GICv2m",
+ .irq_eoi = gic_eoi_irq,
+ .irq_set_type = gic_set_type,
+#ifdef CONFIG_SMP
+ .irq_set_affinity = gic_set_affinity,
+#endif
+};
+
void __init gic_cascade_irq(unsigned int gic_nr, unsigned int irq)
{
if (gic_nr >= MAX_GIC_NR)
@@ -767,6 +776,24 @@ void __init gic_init_physaddr(struct device_node *node)
static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
irq_hw_number_t hw)
{
+ struct gic_chip_data *gic = d->host_data;
+ struct v2m_data *v2m;
+
+ irq_set_chip_data(irq, gic);
+
+#ifdef CONFIG_ARM_GIC_V2M
+ list_for_each_entry(v2m, &gic->v2m_list, list) {
+ if (hw >= v2m->spi_start &&
+ hw < v2m->spi_start + v2m->nr_spis) {
+
+ irq_set_chip_and_handler(irq, &v2m_chip,
+ handle_fasteoi_irq);
+ set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
+ return 0;
+ }
+ }
+#endif
+
if (hw < 32) {
irq_set_percpu_devid(irq);
irq_set_chip_and_handler(irq, &gic_chip,
@@ -779,7 +806,6 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
gic_routable_irq_domain_ops->map(d, irq, hw);
}
- irq_set_chip_data(irq, d->host_data);
return 0;
}
@@ -996,7 +1022,7 @@ gic_of_init(struct device_node *node, struct device_node *parent)
void __iomem *cpu_base;
void __iomem *dist_base;
u32 percpu_offset;
- int irq;
+ int irq, ret;
if (WARN_ON(!node))
return -ENODEV;
@@ -1010,6 +1036,12 @@ gic_of_init(struct device_node *node, struct device_node *parent)
if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
percpu_offset = 0;
+ if (IS_ENABLED(CONFIG_ARM_GIC_V2M)) {
+ ret = gicv2m_of_init(node, &gic_data[gic_cnt], &v2m_chip);
+ if (ret)
+ pr_warn("Warning: Failed to enable GICv2m support.\n");
+ }
+
gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset, node);
if (!gic_cnt)
gic_init_physaddr(node);
diff --git a/drivers/irqchip/irq-gic.h b/drivers/irqchip/irq-gic.h
new file mode 100644
index 0000000..1f4a2b0
--- /dev/null
+++ b/drivers/irqchip/irq-gic.h
@@ -0,0 +1,51 @@
+#ifndef _IRQ_GIC_H_
+#define _IRQ_GIC_H_
+
+#include <linux/msi.h>
+#include <linux/pci.h>
+
+union gic_base {
+ void __iomem *common_base;
+ void __percpu * __iomem *percpu_base;
+};
+
+struct gic_chip_data;
+
+struct v2m_data {
+#ifdef CONFIG_ARM_GIC_V2M
+ struct list_head list;
+ spinlock_t msi_cnt_lock;
+ struct msi_chip msi_chip;
+ struct resource res; /* GICv2m resource */
+ void __iomem *base; /* GICv2m virt address */
+ unsigned int spi_start; /* The SPI number that MSIs start */
+ unsigned int nr_spis; /* The number of SPIs for MSIs */
+ unsigned long *bm; /* MSI vector bitmap */
+ struct gic_chip_data *gic;
+#endif
+};
+
+struct gic_chip_data {
+ union gic_base dist_base;
+ union gic_base cpu_base;
+#ifdef CONFIG_CPU_PM
+ u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
+ u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
+ u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
+ u32 __percpu *saved_ppi_enable;
+ u32 __percpu *saved_ppi_conf;
+#endif
+ struct irq_domain *domain;
+ unsigned int gic_irqs;
+#ifdef CONFIG_GIC_NON_BANKED
+ void __iomem *(*get_base)(union gic_base *);
+#endif
+ struct list_head v2m_list;
+};
+
+void gic_mask_irq(struct irq_data *d);
+void gic_unmask_irq(struct irq_data *d);
+int gicv2m_of_init(struct device_node *node, struct gic_chip_data *gic,
+ struct irq_chip *v2m_chip) __init;
+
+#endif /* _IRQ_GIC_H_ */
--
1.9.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH 1/2 V6] irqchip: gic: Add supports for ARM GICv2m MSI(-X)
@ 2014-09-14 6:30 ` suravee.suthikulpanit
0 siblings, 0 replies; 17+ messages in thread
From: suravee.suthikulpanit @ 2014-09-14 6:30 UTC (permalink / raw)
To: marc.zyngier, mark.rutland, jason
Cc: pawel.moll, Catalin.Marinas, Will.Deacon, tglx,
Harish.Kasiviswanathan, linux-arm-kernel, linux-pci,
linux-kernel, linux-doc, devicetree, Suravee Suthikulpanit,
Mark Rutland, Marc Zyngier
From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
ARM GICv2m specification extends GICv2 to support MSI(-X) with
a new set of register frame. This patch introduces support for
the non-secure GICv2m register frame. Currently, GICV2m is available
in certain version of GIC-400.
The patch introduces a new property in ARM gic binding, the v2m subnode.
It is optional.
Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Cc: Mark Rutland <Mark.Rutland@arm.com>
Cc: Marc Zyngier <Marc.Zyngier@arm.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Catalin Marinas <Catalin.Marinas@arm.com>
Cc: Will Deacon <Will.Deacon@arm.com>
---
Documentation/devicetree/bindings/arm/gic.txt | 39 ++++
drivers/irqchip/Kconfig | 7 +
drivers/irqchip/Makefile | 1 +
drivers/irqchip/irq-gic-v2m.c | 261 ++++++++++++++++++++++++++
drivers/irqchip/irq-gic.c | 88 ++++++---
drivers/irqchip/irq-gic.h | 51 +++++
6 files changed, 419 insertions(+), 28 deletions(-)
create mode 100644 drivers/irqchip/irq-gic-v2m.c
create mode 100644 drivers/irqchip/irq-gic.h
diff --git a/Documentation/devicetree/bindings/arm/gic.txt b/Documentation/devicetree/bindings/arm/gic.txt
index c7d2fa1..a461591 100644
--- a/Documentation/devicetree/bindings/arm/gic.txt
+++ b/Documentation/devicetree/bindings/arm/gic.txt
@@ -96,3 +96,42 @@ Example:
<0x2c006000 0x2000>;
interrupts = <1 9 0xf04>;
};
+
+
+* GICv2m extension for MSI/MSI-x support (Optional)
+
+Certain revision of GIC-400 supports MSI/MSI-x via V2M register frame.
+This is enabled by specifying v2m sub-node.
+
+Required properties:
+
+- msi-controller : Identifies the node as an MSI controller.
+
+- reg : GICv2m MSI interface register base and size
+
+Example:
+
+ interrupt-controller@e1101000 {
+ compatible = "arm,gic-400";
+ #interrupt-cells = <3>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-controller;
+ interrupts = <1 8 0xf04>;
+ ranges = <0 0 0 0xe1100000 0 0x100000>;
+ reg = <0x0 0xe1110000 0 0x01000>,
+ <0x0 0xe112f000 0 0x02000>,
+ <0x0 0xe1140000 0 0x10000>,
+ <0x0 0xe1160000 0 0x10000>;
+ v2m0 {
+ msi-controller;
+ reg = <0x0 0x80000 0 0x1000>;
+ };
+
+ ....
+
+ v2mN {
+ msi-controller;
+ reg = <0x0 0x90000 0 0x1000>;
+ };
+ };
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index b8632bf..ddb31ee 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -7,6 +7,13 @@ config ARM_GIC
select IRQ_DOMAIN
select MULTI_IRQ_HANDLER
+config ARM_GIC_V2M
+ bool
+ select IRQ_DOMAIN
+ select MULTI_IRQ_HANDLER
+ depends on ARM_GIC
+ depends on PCI && PCI_MSI
+
config GIC_NON_BANKED
bool
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 73052ba..3bda951 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -17,6 +17,7 @@ obj-$(CONFIG_ARCH_SUNXI) += irq-sun4i.o
obj-$(CONFIG_ARCH_SUNXI) += irq-sunxi-nmi.o
obj-$(CONFIG_ARCH_SPEAR3XX) += spear-shirq.o
obj-$(CONFIG_ARM_GIC) += irq-gic.o irq-gic-common.o
+obj-$(CONFIG_ARM_GIC_V2M) += irq-gic-v2m.o
obj-$(CONFIG_ARM_GIC_V3) += irq-gic-v3.o irq-gic-common.o
obj-$(CONFIG_ARM_NVIC) += irq-nvic.o
obj-$(CONFIG_ARM_VIC) += irq-vic.o
diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
new file mode 100644
index 0000000..499bfb9
--- /dev/null
+++ b/drivers/irqchip/irq-gic-v2m.c
@@ -0,0 +1,261 @@
+/*
+ * ARM GIC v2m MSI(-X) support
+ * Support for Message Signaled Interrupts for systems that
+ * implement ARM Generic Interrupt Controller: GICv2m.
+ *
+ * Copyright (C) 2014 Advanced Micro Devices, Inc.
+ * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
+ * Harish Kasiviswanathan <harish.kasiviswanathan@amd.com>
+ * Brandon Anderson <brandon.anderson@amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ */
+
+#define pr_fmt(fmt) "GICv2m: " fmt
+
+#include <linux/bitmap.h>
+#include <linux/irq.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_pci.h>
+#include <linux/pci.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#include "irqchip.h"
+#include "irq-gic.h"
+
+/*
+* MSI_TYPER:
+* [31:26] Reserved
+* [25:16] lowest SPI assigned to MSI
+* [15:10] Reserved
+* [9:0] Numer of SPIs assigned to MSI
+*/
+#define V2M_MSI_TYPER 0x008
+#define V2M_MSI_TYPER_BASE_SHIFT 16
+#define V2M_MSI_TYPER_BASE_MASK 0x3FF
+#define V2M_MSI_TYPER_NUM_MASK 0x3FF
+#define V2M_MSI_SETSPI_NS 0x040
+#define V2M_MIN_SPI 32
+#define V2M_MAX_SPI 1019
+
+/*
+ * alloc_msi_irq - Allocate MSIs from available MSI bitmap.
+ * @data: Pointer to v2m_data
+ * @nvec: Number of interrupts to allocate
+ * @irq: Pointer to the allocated irq
+ *
+ * Allocates interrupts only if the contiguous range of MSIs
+ * with specified nvec are available. Otherwise return the number
+ * of available interrupts. If none are available, then returns -ENOENT.
+ */
+static int alloc_msi_irq(struct v2m_data *data, int nvec, int *irq)
+{
+ int size = data->nr_spis;
+ int next = size, i = nvec, ret;
+
+ /* We should never allocate more than available nr_spis */
+ if (i >= size)
+ i = size;
+
+ spin_lock(&data->msi_cnt_lock);
+
+ for (; i > 0; i--) {
+ next = bitmap_find_next_zero_area(data->bm,
+ size, 0, i, 0);
+ if (next < size)
+ break;
+ }
+
+ if (i != nvec) {
+ ret = i ? : -ENOENT;
+ } else {
+ bitmap_set(data->bm, next, nvec);
+ *irq = data->spi_start + next;
+ ret = 0;
+ }
+
+ spin_unlock(&data->msi_cnt_lock);
+
+ return ret;
+}
+
+static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
+{
+ int pos;
+ struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
+
+ spin_lock(&data->msi_cnt_lock);
+
+ pos = irq - data->spi_start;
+ if (pos >= 0 && pos < data->nr_spis)
+ bitmap_clear(data->bm, pos, 1);
+
+ spin_unlock(&data->msi_cnt_lock);
+}
+
+static int gicv2m_setup_msi_irq(struct msi_chip *chip,
+ struct pci_dev *pdev,
+ struct msi_desc *desc)
+{
+ int irq, avail;
+ struct msi_msg msg;
+ phys_addr_t addr;
+ struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
+
+ if (!desc) {
+ dev_err(&pdev->dev,
+ "GICv2m: MSI setup failed. Invalid msi descriptor\n");
+ return -EINVAL;
+ }
+
+ avail = alloc_msi_irq(data, 1, &irq);
+ if (avail != 0) {
+ dev_err(&pdev->dev,
+ "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
+ return -ENOSPC;
+ }
+
+ irq_set_chip_data(irq, chip);
+ irq_set_msi_desc(irq, desc);
+ irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
+
+ addr = data->res.start + V2M_MSI_SETSPI_NS;
+ msg.address_hi = (u32)(addr >> 32);
+ msg.address_lo = (u32)(addr);
+ msg.data = irq;
+ write_msi_msg(irq, &msg);
+
+ return 0;
+}
+
+static void gicv2m_mask_irq(struct irq_data *d)
+{
+ gic_mask_irq(d);
+ if (d->msi_desc)
+ mask_msi_irq(d);
+}
+
+static void gicv2m_unmask_irq(struct irq_data *d)
+{
+ gic_unmask_irq(d);
+ if (d->msi_desc)
+ unmask_msi_irq(d);
+}
+
+#ifdef CONFIG_OF
+
+static int __init
+v2m_init(struct device_node *node, struct v2m_data *v2m,
+ struct resource *res)
+{
+ int ret;
+ unsigned int typer;
+
+ v2m->msi_chip.owner = THIS_MODULE;
+ v2m->msi_chip.of_node = node;
+ v2m->msi_chip.setup_irq = gicv2m_setup_msi_irq;
+ v2m->msi_chip.teardown_irq = gicv2m_teardown_msi_irq;
+ ret = of_pci_msi_chip_add(&v2m->msi_chip);
+ if (ret) {
+ pr_info("Failed to add msi_chip.\n");
+ return ret;
+ }
+
+ v2m->base = ioremap(res->start, resource_size(res));
+ if (!v2m->base) {
+ pr_err("Failed to map GICv2m resource\n");
+ return -EINVAL;
+ }
+
+ memcpy(&v2m->res, res, sizeof(struct resource));
+
+ typer = readl_relaxed(v2m->base + V2M_MSI_TYPER);
+ if (!typer) {
+ pr_warn("Failed to read V2M_MSI_TYPER register\n");
+ return -EINVAL;
+ }
+
+ v2m->spi_start = (typer >> V2M_MSI_TYPER_BASE_SHIFT) &
+ V2M_MSI_TYPER_BASE_MASK;
+ v2m->nr_spis = typer & V2M_MSI_TYPER_NUM_MASK;
+ if ((v2m->spi_start < V2M_MIN_SPI) || (v2m->nr_spis >= V2M_MAX_SPI)) {
+ pr_err("Invalid MSI_TYPER (%#x)\n", typer);
+ return -EINVAL;
+ }
+
+ v2m->bm = kzalloc(sizeof(long) * BITS_TO_LONGS(v2m->nr_spis),
+ GFP_KERNEL);
+ if (!v2m->bm) {
+ pr_err("Failed to allocate MSI bitmap\n");
+ return -ENOMEM;
+ }
+
+ spin_lock_init(&v2m->msi_cnt_lock);
+
+ pr_info("SPI range [%d:%d]\n",
+ v2m->spi_start, (v2m->spi_start + v2m->nr_spis));
+
+ return 0;
+}
+
+int __init gicv2m_of_init(struct device_node *node,
+ struct gic_chip_data *gic,
+ struct irq_chip *v2m_chip)
+{
+ int ret = 0;
+ struct v2m_data *v2m;
+ struct resource res;
+ struct device_node *child = NULL;
+
+ INIT_LIST_HEAD(&gic->v2m_list);
+
+ for (;;) {
+ child = of_get_next_child(node, child);
+ if (!child)
+ break;
+
+ if (!of_find_property(child, "msi-controller", NULL))
+ continue;
+
+ ret = of_address_to_resource(child, 0, &res);
+ if (ret) {
+ pr_err("Failed to allocate v2m resource.\n");
+ break;
+ }
+
+ v2m = kzalloc(sizeof(struct v2m_data), GFP_KERNEL);
+ if (!v2m) {
+ pr_err("Failed to allocate struct v2m_data.\n");
+ return -ENOMEM;
+ }
+
+ ret = v2m_init(child, v2m, &res);
+ if (ret)
+ break;
+
+ v2m->gic = gic;
+
+ pr_info(" %s: [%lx : %lx]\n", child->name,
+ (unsigned long)res.start, (unsigned long)res.end);
+
+ list_add_tail(&v2m->list, &gic->v2m_list);
+ }
+
+ if (ret && list_empty(&gic->v2m_list)) {
+ pr_err("MSI resource not found\n");
+ return -EINVAL;
+ }
+
+ v2m_chip->irq_mask = gicv2m_mask_irq;
+ v2m_chip->irq_unmask = gicv2m_unmask_irq;
+
+ return ret;
+}
+
+#endif /* CONFIG_OF */
diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 4b959e6..f490599 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -46,30 +46,9 @@
#include <asm/smp_plat.h>
#include "irq-gic-common.h"
+#include "irq-gic.h"
#include "irqchip.h"
-union gic_base {
- void __iomem *common_base;
- void __percpu * __iomem *percpu_base;
-};
-
-struct gic_chip_data {
- union gic_base dist_base;
- union gic_base cpu_base;
-#ifdef CONFIG_CPU_PM
- u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
- u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
- u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
- u32 __percpu *saved_ppi_enable;
- u32 __percpu *saved_ppi_conf;
-#endif
- struct irq_domain *domain;
- unsigned int gic_irqs;
-#ifdef CONFIG_GIC_NON_BANKED
- void __iomem *(*get_base)(union gic_base *);
-#endif
-};
-
static DEFINE_RAW_SPINLOCK(irq_controller_lock);
/*
@@ -131,15 +110,36 @@ static inline void gic_set_base_accessor(struct gic_chip_data *data,
#define gic_set_base_accessor(d, f)
#endif
+static inline
+struct gic_chip_data *irq_data_get_gic_chip_data(struct irq_data *d)
+{
+ struct gic_chip_data *gic_data;
+ struct msi_chip *mchip;
+ struct v2m_data *v2mdat;
+
+ /*
+ * For MSI, irq_data.chip_data points to struct msi_chip.
+ * For non-MSI, irq_data.chip_data points to struct gic_chip_data.
+ */
+ if (d->msi_desc) {
+ mchip = irq_data_get_irq_chip_data(d);
+ v2mdat = container_of(mchip, struct v2m_data, msi_chip);
+ gic_data = v2mdat->gic;
+ } else {
+ gic_data = irq_data_get_irq_chip_data(d);
+ }
+ return gic_data;
+}
+
static inline void __iomem *gic_dist_base(struct irq_data *d)
{
- struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
+ struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
return gic_data_dist_base(gic_data);
}
static inline void __iomem *gic_cpu_base(struct irq_data *d)
{
- struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
+ struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
return gic_data_cpu_base(gic_data);
}
@@ -151,7 +151,7 @@ static inline unsigned int gic_irq(struct irq_data *d)
/*
* Routines to acknowledge, disable and enable interrupts
*/
-static void gic_mask_irq(struct irq_data *d)
+void gic_mask_irq(struct irq_data *d)
{
u32 mask = 1 << (gic_irq(d) % 32);
@@ -162,7 +162,7 @@ static void gic_mask_irq(struct irq_data *d)
raw_spin_unlock(&irq_controller_lock);
}
-static void gic_unmask_irq(struct irq_data *d)
+void gic_unmask_irq(struct irq_data *d)
{
u32 mask = 1 << (gic_irq(d) % 32);
@@ -325,6 +325,15 @@ static struct irq_chip gic_chip = {
.irq_set_wake = gic_set_wake,
};
+static struct irq_chip v2m_chip = {
+ .name = "GICv2m",
+ .irq_eoi = gic_eoi_irq,
+ .irq_set_type = gic_set_type,
+#ifdef CONFIG_SMP
+ .irq_set_affinity = gic_set_affinity,
+#endif
+};
+
void __init gic_cascade_irq(unsigned int gic_nr, unsigned int irq)
{
if (gic_nr >= MAX_GIC_NR)
@@ -767,6 +776,24 @@ void __init gic_init_physaddr(struct device_node *node)
static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
irq_hw_number_t hw)
{
+ struct gic_chip_data *gic = d->host_data;
+ struct v2m_data *v2m;
+
+ irq_set_chip_data(irq, gic);
+
+#ifdef CONFIG_ARM_GIC_V2M
+ list_for_each_entry(v2m, &gic->v2m_list, list) {
+ if (hw >= v2m->spi_start &&
+ hw < v2m->spi_start + v2m->nr_spis) {
+
+ irq_set_chip_and_handler(irq, &v2m_chip,
+ handle_fasteoi_irq);
+ set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
+ return 0;
+ }
+ }
+#endif
+
if (hw < 32) {
irq_set_percpu_devid(irq);
irq_set_chip_and_handler(irq, &gic_chip,
@@ -779,7 +806,6 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
gic_routable_irq_domain_ops->map(d, irq, hw);
}
- irq_set_chip_data(irq, d->host_data);
return 0;
}
@@ -996,7 +1022,7 @@ gic_of_init(struct device_node *node, struct device_node *parent)
void __iomem *cpu_base;
void __iomem *dist_base;
u32 percpu_offset;
- int irq;
+ int irq, ret;
if (WARN_ON(!node))
return -ENODEV;
@@ -1010,6 +1036,12 @@ gic_of_init(struct device_node *node, struct device_node *parent)
if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
percpu_offset = 0;
+ if (IS_ENABLED(CONFIG_ARM_GIC_V2M)) {
+ ret = gicv2m_of_init(node, &gic_data[gic_cnt], &v2m_chip);
+ if (ret)
+ pr_warn("Warning: Failed to enable GICv2m support.\n");
+ }
+
gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset, node);
if (!gic_cnt)
gic_init_physaddr(node);
diff --git a/drivers/irqchip/irq-gic.h b/drivers/irqchip/irq-gic.h
new file mode 100644
index 0000000..1f4a2b0
--- /dev/null
+++ b/drivers/irqchip/irq-gic.h
@@ -0,0 +1,51 @@
+#ifndef _IRQ_GIC_H_
+#define _IRQ_GIC_H_
+
+#include <linux/msi.h>
+#include <linux/pci.h>
+
+union gic_base {
+ void __iomem *common_base;
+ void __percpu * __iomem *percpu_base;
+};
+
+struct gic_chip_data;
+
+struct v2m_data {
+#ifdef CONFIG_ARM_GIC_V2M
+ struct list_head list;
+ spinlock_t msi_cnt_lock;
+ struct msi_chip msi_chip;
+ struct resource res; /* GICv2m resource */
+ void __iomem *base; /* GICv2m virt address */
+ unsigned int spi_start; /* The SPI number that MSIs start */
+ unsigned int nr_spis; /* The number of SPIs for MSIs */
+ unsigned long *bm; /* MSI vector bitmap */
+ struct gic_chip_data *gic;
+#endif
+};
+
+struct gic_chip_data {
+ union gic_base dist_base;
+ union gic_base cpu_base;
+#ifdef CONFIG_CPU_PM
+ u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
+ u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
+ u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
+ u32 __percpu *saved_ppi_enable;
+ u32 __percpu *saved_ppi_conf;
+#endif
+ struct irq_domain *domain;
+ unsigned int gic_irqs;
+#ifdef CONFIG_GIC_NON_BANKED
+ void __iomem *(*get_base)(union gic_base *);
+#endif
+ struct list_head v2m_list;
+};
+
+void gic_mask_irq(struct irq_data *d);
+void gic_unmask_irq(struct irq_data *d);
+int gicv2m_of_init(struct device_node *node, struct gic_chip_data *gic,
+ struct irq_chip *v2m_chip) __init;
+
+#endif /* _IRQ_GIC_H_ */
--
1.9.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH 1/2 V6] irqchip: gic: Add supports for ARM GICv2m MSI(-X)
@ 2014-09-14 6:30 ` suravee.suthikulpanit
0 siblings, 0 replies; 17+ messages in thread
From: suravee.suthikulpanit at amd.com @ 2014-09-14 6:30 UTC (permalink / raw)
To: linux-arm-kernel
From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
ARM GICv2m specification extends GICv2 to support MSI(-X) with
a new set of register frame. This patch introduces support for
the non-secure GICv2m register frame. Currently, GICV2m is available
in certain version of GIC-400.
The patch introduces a new property in ARM gic binding, the v2m subnode.
It is optional.
Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Cc: Mark Rutland <Mark.Rutland@arm.com>
Cc: Marc Zyngier <Marc.Zyngier@arm.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Catalin Marinas <Catalin.Marinas@arm.com>
Cc: Will Deacon <Will.Deacon@arm.com>
---
Documentation/devicetree/bindings/arm/gic.txt | 39 ++++
drivers/irqchip/Kconfig | 7 +
drivers/irqchip/Makefile | 1 +
drivers/irqchip/irq-gic-v2m.c | 261 ++++++++++++++++++++++++++
drivers/irqchip/irq-gic.c | 88 ++++++---
drivers/irqchip/irq-gic.h | 51 +++++
6 files changed, 419 insertions(+), 28 deletions(-)
create mode 100644 drivers/irqchip/irq-gic-v2m.c
create mode 100644 drivers/irqchip/irq-gic.h
diff --git a/Documentation/devicetree/bindings/arm/gic.txt b/Documentation/devicetree/bindings/arm/gic.txt
index c7d2fa1..a461591 100644
--- a/Documentation/devicetree/bindings/arm/gic.txt
+++ b/Documentation/devicetree/bindings/arm/gic.txt
@@ -96,3 +96,42 @@ Example:
<0x2c006000 0x2000>;
interrupts = <1 9 0xf04>;
};
+
+
+* GICv2m extension for MSI/MSI-x support (Optional)
+
+Certain revision of GIC-400 supports MSI/MSI-x via V2M register frame.
+This is enabled by specifying v2m sub-node.
+
+Required properties:
+
+- msi-controller : Identifies the node as an MSI controller.
+
+- reg : GICv2m MSI interface register base and size
+
+Example:
+
+ interrupt-controller at e1101000 {
+ compatible = "arm,gic-400";
+ #interrupt-cells = <3>;
+ #address-cells = <2>;
+ #size-cells = <2>;
+ interrupt-controller;
+ interrupts = <1 8 0xf04>;
+ ranges = <0 0 0 0xe1100000 0 0x100000>;
+ reg = <0x0 0xe1110000 0 0x01000>,
+ <0x0 0xe112f000 0 0x02000>,
+ <0x0 0xe1140000 0 0x10000>,
+ <0x0 0xe1160000 0 0x10000>;
+ v2m0 {
+ msi-controller;
+ reg = <0x0 0x80000 0 0x1000>;
+ };
+
+ ....
+
+ v2mN {
+ msi-controller;
+ reg = <0x0 0x90000 0 0x1000>;
+ };
+ };
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index b8632bf..ddb31ee 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -7,6 +7,13 @@ config ARM_GIC
select IRQ_DOMAIN
select MULTI_IRQ_HANDLER
+config ARM_GIC_V2M
+ bool
+ select IRQ_DOMAIN
+ select MULTI_IRQ_HANDLER
+ depends on ARM_GIC
+ depends on PCI && PCI_MSI
+
config GIC_NON_BANKED
bool
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 73052ba..3bda951 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -17,6 +17,7 @@ obj-$(CONFIG_ARCH_SUNXI) += irq-sun4i.o
obj-$(CONFIG_ARCH_SUNXI) += irq-sunxi-nmi.o
obj-$(CONFIG_ARCH_SPEAR3XX) += spear-shirq.o
obj-$(CONFIG_ARM_GIC) += irq-gic.o irq-gic-common.o
+obj-$(CONFIG_ARM_GIC_V2M) += irq-gic-v2m.o
obj-$(CONFIG_ARM_GIC_V3) += irq-gic-v3.o irq-gic-common.o
obj-$(CONFIG_ARM_NVIC) += irq-nvic.o
obj-$(CONFIG_ARM_VIC) += irq-vic.o
diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
new file mode 100644
index 0000000..499bfb9
--- /dev/null
+++ b/drivers/irqchip/irq-gic-v2m.c
@@ -0,0 +1,261 @@
+/*
+ * ARM GIC v2m MSI(-X) support
+ * Support for Message Signaled Interrupts for systems that
+ * implement ARM Generic Interrupt Controller: GICv2m.
+ *
+ * Copyright (C) 2014 Advanced Micro Devices, Inc.
+ * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
+ * Harish Kasiviswanathan <harish.kasiviswanathan@amd.com>
+ * Brandon Anderson <brandon.anderson@amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ */
+
+#define pr_fmt(fmt) "GICv2m: " fmt
+
+#include <linux/bitmap.h>
+#include <linux/irq.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of_address.h>
+#include <linux/of_pci.h>
+#include <linux/pci.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#include "irqchip.h"
+#include "irq-gic.h"
+
+/*
+* MSI_TYPER:
+* [31:26] Reserved
+* [25:16] lowest SPI assigned to MSI
+* [15:10] Reserved
+* [9:0] Numer of SPIs assigned to MSI
+*/
+#define V2M_MSI_TYPER 0x008
+#define V2M_MSI_TYPER_BASE_SHIFT 16
+#define V2M_MSI_TYPER_BASE_MASK 0x3FF
+#define V2M_MSI_TYPER_NUM_MASK 0x3FF
+#define V2M_MSI_SETSPI_NS 0x040
+#define V2M_MIN_SPI 32
+#define V2M_MAX_SPI 1019
+
+/*
+ * alloc_msi_irq - Allocate MSIs from available MSI bitmap.
+ * @data: Pointer to v2m_data
+ * @nvec: Number of interrupts to allocate
+ * @irq: Pointer to the allocated irq
+ *
+ * Allocates interrupts only if the contiguous range of MSIs
+ * with specified nvec are available. Otherwise return the number
+ * of available interrupts. If none are available, then returns -ENOENT.
+ */
+static int alloc_msi_irq(struct v2m_data *data, int nvec, int *irq)
+{
+ int size = data->nr_spis;
+ int next = size, i = nvec, ret;
+
+ /* We should never allocate more than available nr_spis */
+ if (i >= size)
+ i = size;
+
+ spin_lock(&data->msi_cnt_lock);
+
+ for (; i > 0; i--) {
+ next = bitmap_find_next_zero_area(data->bm,
+ size, 0, i, 0);
+ if (next < size)
+ break;
+ }
+
+ if (i != nvec) {
+ ret = i ? : -ENOENT;
+ } else {
+ bitmap_set(data->bm, next, nvec);
+ *irq = data->spi_start + next;
+ ret = 0;
+ }
+
+ spin_unlock(&data->msi_cnt_lock);
+
+ return ret;
+}
+
+static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
+{
+ int pos;
+ struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
+
+ spin_lock(&data->msi_cnt_lock);
+
+ pos = irq - data->spi_start;
+ if (pos >= 0 && pos < data->nr_spis)
+ bitmap_clear(data->bm, pos, 1);
+
+ spin_unlock(&data->msi_cnt_lock);
+}
+
+static int gicv2m_setup_msi_irq(struct msi_chip *chip,
+ struct pci_dev *pdev,
+ struct msi_desc *desc)
+{
+ int irq, avail;
+ struct msi_msg msg;
+ phys_addr_t addr;
+ struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
+
+ if (!desc) {
+ dev_err(&pdev->dev,
+ "GICv2m: MSI setup failed. Invalid msi descriptor\n");
+ return -EINVAL;
+ }
+
+ avail = alloc_msi_irq(data, 1, &irq);
+ if (avail != 0) {
+ dev_err(&pdev->dev,
+ "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
+ return -ENOSPC;
+ }
+
+ irq_set_chip_data(irq, chip);
+ irq_set_msi_desc(irq, desc);
+ irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
+
+ addr = data->res.start + V2M_MSI_SETSPI_NS;
+ msg.address_hi = (u32)(addr >> 32);
+ msg.address_lo = (u32)(addr);
+ msg.data = irq;
+ write_msi_msg(irq, &msg);
+
+ return 0;
+}
+
+static void gicv2m_mask_irq(struct irq_data *d)
+{
+ gic_mask_irq(d);
+ if (d->msi_desc)
+ mask_msi_irq(d);
+}
+
+static void gicv2m_unmask_irq(struct irq_data *d)
+{
+ gic_unmask_irq(d);
+ if (d->msi_desc)
+ unmask_msi_irq(d);
+}
+
+#ifdef CONFIG_OF
+
+static int __init
+v2m_init(struct device_node *node, struct v2m_data *v2m,
+ struct resource *res)
+{
+ int ret;
+ unsigned int typer;
+
+ v2m->msi_chip.owner = THIS_MODULE;
+ v2m->msi_chip.of_node = node;
+ v2m->msi_chip.setup_irq = gicv2m_setup_msi_irq;
+ v2m->msi_chip.teardown_irq = gicv2m_teardown_msi_irq;
+ ret = of_pci_msi_chip_add(&v2m->msi_chip);
+ if (ret) {
+ pr_info("Failed to add msi_chip.\n");
+ return ret;
+ }
+
+ v2m->base = ioremap(res->start, resource_size(res));
+ if (!v2m->base) {
+ pr_err("Failed to map GICv2m resource\n");
+ return -EINVAL;
+ }
+
+ memcpy(&v2m->res, res, sizeof(struct resource));
+
+ typer = readl_relaxed(v2m->base + V2M_MSI_TYPER);
+ if (!typer) {
+ pr_warn("Failed to read V2M_MSI_TYPER register\n");
+ return -EINVAL;
+ }
+
+ v2m->spi_start = (typer >> V2M_MSI_TYPER_BASE_SHIFT) &
+ V2M_MSI_TYPER_BASE_MASK;
+ v2m->nr_spis = typer & V2M_MSI_TYPER_NUM_MASK;
+ if ((v2m->spi_start < V2M_MIN_SPI) || (v2m->nr_spis >= V2M_MAX_SPI)) {
+ pr_err("Invalid MSI_TYPER (%#x)\n", typer);
+ return -EINVAL;
+ }
+
+ v2m->bm = kzalloc(sizeof(long) * BITS_TO_LONGS(v2m->nr_spis),
+ GFP_KERNEL);
+ if (!v2m->bm) {
+ pr_err("Failed to allocate MSI bitmap\n");
+ return -ENOMEM;
+ }
+
+ spin_lock_init(&v2m->msi_cnt_lock);
+
+ pr_info("SPI range [%d:%d]\n",
+ v2m->spi_start, (v2m->spi_start + v2m->nr_spis));
+
+ return 0;
+}
+
+int __init gicv2m_of_init(struct device_node *node,
+ struct gic_chip_data *gic,
+ struct irq_chip *v2m_chip)
+{
+ int ret = 0;
+ struct v2m_data *v2m;
+ struct resource res;
+ struct device_node *child = NULL;
+
+ INIT_LIST_HEAD(&gic->v2m_list);
+
+ for (;;) {
+ child = of_get_next_child(node, child);
+ if (!child)
+ break;
+
+ if (!of_find_property(child, "msi-controller", NULL))
+ continue;
+
+ ret = of_address_to_resource(child, 0, &res);
+ if (ret) {
+ pr_err("Failed to allocate v2m resource.\n");
+ break;
+ }
+
+ v2m = kzalloc(sizeof(struct v2m_data), GFP_KERNEL);
+ if (!v2m) {
+ pr_err("Failed to allocate struct v2m_data.\n");
+ return -ENOMEM;
+ }
+
+ ret = v2m_init(child, v2m, &res);
+ if (ret)
+ break;
+
+ v2m->gic = gic;
+
+ pr_info(" %s: [%lx : %lx]\n", child->name,
+ (unsigned long)res.start, (unsigned long)res.end);
+
+ list_add_tail(&v2m->list, &gic->v2m_list);
+ }
+
+ if (ret && list_empty(&gic->v2m_list)) {
+ pr_err("MSI resource not found\n");
+ return -EINVAL;
+ }
+
+ v2m_chip->irq_mask = gicv2m_mask_irq;
+ v2m_chip->irq_unmask = gicv2m_unmask_irq;
+
+ return ret;
+}
+
+#endif /* CONFIG_OF */
diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
index 4b959e6..f490599 100644
--- a/drivers/irqchip/irq-gic.c
+++ b/drivers/irqchip/irq-gic.c
@@ -46,30 +46,9 @@
#include <asm/smp_plat.h>
#include "irq-gic-common.h"
+#include "irq-gic.h"
#include "irqchip.h"
-union gic_base {
- void __iomem *common_base;
- void __percpu * __iomem *percpu_base;
-};
-
-struct gic_chip_data {
- union gic_base dist_base;
- union gic_base cpu_base;
-#ifdef CONFIG_CPU_PM
- u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
- u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
- u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
- u32 __percpu *saved_ppi_enable;
- u32 __percpu *saved_ppi_conf;
-#endif
- struct irq_domain *domain;
- unsigned int gic_irqs;
-#ifdef CONFIG_GIC_NON_BANKED
- void __iomem *(*get_base)(union gic_base *);
-#endif
-};
-
static DEFINE_RAW_SPINLOCK(irq_controller_lock);
/*
@@ -131,15 +110,36 @@ static inline void gic_set_base_accessor(struct gic_chip_data *data,
#define gic_set_base_accessor(d, f)
#endif
+static inline
+struct gic_chip_data *irq_data_get_gic_chip_data(struct irq_data *d)
+{
+ struct gic_chip_data *gic_data;
+ struct msi_chip *mchip;
+ struct v2m_data *v2mdat;
+
+ /*
+ * For MSI, irq_data.chip_data points to struct msi_chip.
+ * For non-MSI, irq_data.chip_data points to struct gic_chip_data.
+ */
+ if (d->msi_desc) {
+ mchip = irq_data_get_irq_chip_data(d);
+ v2mdat = container_of(mchip, struct v2m_data, msi_chip);
+ gic_data = v2mdat->gic;
+ } else {
+ gic_data = irq_data_get_irq_chip_data(d);
+ }
+ return gic_data;
+}
+
static inline void __iomem *gic_dist_base(struct irq_data *d)
{
- struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
+ struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
return gic_data_dist_base(gic_data);
}
static inline void __iomem *gic_cpu_base(struct irq_data *d)
{
- struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
+ struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
return gic_data_cpu_base(gic_data);
}
@@ -151,7 +151,7 @@ static inline unsigned int gic_irq(struct irq_data *d)
/*
* Routines to acknowledge, disable and enable interrupts
*/
-static void gic_mask_irq(struct irq_data *d)
+void gic_mask_irq(struct irq_data *d)
{
u32 mask = 1 << (gic_irq(d) % 32);
@@ -162,7 +162,7 @@ static void gic_mask_irq(struct irq_data *d)
raw_spin_unlock(&irq_controller_lock);
}
-static void gic_unmask_irq(struct irq_data *d)
+void gic_unmask_irq(struct irq_data *d)
{
u32 mask = 1 << (gic_irq(d) % 32);
@@ -325,6 +325,15 @@ static struct irq_chip gic_chip = {
.irq_set_wake = gic_set_wake,
};
+static struct irq_chip v2m_chip = {
+ .name = "GICv2m",
+ .irq_eoi = gic_eoi_irq,
+ .irq_set_type = gic_set_type,
+#ifdef CONFIG_SMP
+ .irq_set_affinity = gic_set_affinity,
+#endif
+};
+
void __init gic_cascade_irq(unsigned int gic_nr, unsigned int irq)
{
if (gic_nr >= MAX_GIC_NR)
@@ -767,6 +776,24 @@ void __init gic_init_physaddr(struct device_node *node)
static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
irq_hw_number_t hw)
{
+ struct gic_chip_data *gic = d->host_data;
+ struct v2m_data *v2m;
+
+ irq_set_chip_data(irq, gic);
+
+#ifdef CONFIG_ARM_GIC_V2M
+ list_for_each_entry(v2m, &gic->v2m_list, list) {
+ if (hw >= v2m->spi_start &&
+ hw < v2m->spi_start + v2m->nr_spis) {
+
+ irq_set_chip_and_handler(irq, &v2m_chip,
+ handle_fasteoi_irq);
+ set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
+ return 0;
+ }
+ }
+#endif
+
if (hw < 32) {
irq_set_percpu_devid(irq);
irq_set_chip_and_handler(irq, &gic_chip,
@@ -779,7 +806,6 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
gic_routable_irq_domain_ops->map(d, irq, hw);
}
- irq_set_chip_data(irq, d->host_data);
return 0;
}
@@ -996,7 +1022,7 @@ gic_of_init(struct device_node *node, struct device_node *parent)
void __iomem *cpu_base;
void __iomem *dist_base;
u32 percpu_offset;
- int irq;
+ int irq, ret;
if (WARN_ON(!node))
return -ENODEV;
@@ -1010,6 +1036,12 @@ gic_of_init(struct device_node *node, struct device_node *parent)
if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
percpu_offset = 0;
+ if (IS_ENABLED(CONFIG_ARM_GIC_V2M)) {
+ ret = gicv2m_of_init(node, &gic_data[gic_cnt], &v2m_chip);
+ if (ret)
+ pr_warn("Warning: Failed to enable GICv2m support.\n");
+ }
+
gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset, node);
if (!gic_cnt)
gic_init_physaddr(node);
diff --git a/drivers/irqchip/irq-gic.h b/drivers/irqchip/irq-gic.h
new file mode 100644
index 0000000..1f4a2b0
--- /dev/null
+++ b/drivers/irqchip/irq-gic.h
@@ -0,0 +1,51 @@
+#ifndef _IRQ_GIC_H_
+#define _IRQ_GIC_H_
+
+#include <linux/msi.h>
+#include <linux/pci.h>
+
+union gic_base {
+ void __iomem *common_base;
+ void __percpu * __iomem *percpu_base;
+};
+
+struct gic_chip_data;
+
+struct v2m_data {
+#ifdef CONFIG_ARM_GIC_V2M
+ struct list_head list;
+ spinlock_t msi_cnt_lock;
+ struct msi_chip msi_chip;
+ struct resource res; /* GICv2m resource */
+ void __iomem *base; /* GICv2m virt address */
+ unsigned int spi_start; /* The SPI number that MSIs start */
+ unsigned int nr_spis; /* The number of SPIs for MSIs */
+ unsigned long *bm; /* MSI vector bitmap */
+ struct gic_chip_data *gic;
+#endif
+};
+
+struct gic_chip_data {
+ union gic_base dist_base;
+ union gic_base cpu_base;
+#ifdef CONFIG_CPU_PM
+ u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
+ u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
+ u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
+ u32 __percpu *saved_ppi_enable;
+ u32 __percpu *saved_ppi_conf;
+#endif
+ struct irq_domain *domain;
+ unsigned int gic_irqs;
+#ifdef CONFIG_GIC_NON_BANKED
+ void __iomem *(*get_base)(union gic_base *);
+#endif
+ struct list_head v2m_list;
+};
+
+void gic_mask_irq(struct irq_data *d);
+void gic_unmask_irq(struct irq_data *d);
+int gicv2m_of_init(struct device_node *node, struct gic_chip_data *gic,
+ struct irq_chip *v2m_chip) __init;
+
+#endif /* _IRQ_GIC_H_ */
--
1.9.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH 2/2 V6] irqchip: gicv2m: Add support for multiple MSI for ARM64 GICv2m
2014-09-14 6:30 ` suravee.suthikulpanit
(?)
@ 2014-09-14 6:30 ` suravee.suthikulpanit
-1 siblings, 0 replies; 17+ messages in thread
From: suravee.suthikulpanit @ 2014-09-14 6:30 UTC (permalink / raw)
To: marc.zyngier, mark.rutland, jason
Cc: pawel.moll, Catalin.Marinas, Will.Deacon, tglx,
Harish.Kasiviswanathan, linux-arm-kernel, linux-pci,
linux-kernel, linux-doc, devicetree, Suravee Suthikulpanit,
Mark Rutland, Marc Zyngier
From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
This patch extend GICv2m MSI to support multiple MSI in ARM64.
This requires the common arch_setup_msi_irqs() to be overwriten
with ARM64 version which does not return 1 for PCI_CAP_ID_MSI and
nvec > 1.
Cc: Mark Rutland <Mark.Rutland@arm.com>
Cc: Marc Zyngier <Marc.Zyngier@arm.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Catalin Marinas <Catalin.Marinas@arm.com>
Cc: Will Deacon <Will.Deacon@arm.com>
Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
---
arch/arm64/kernel/Makefile | 1 +
arch/arm64/kernel/msi.c | 41 ++++++++++++++++++++
drivers/irqchip/irq-gic-v2m.c | 87 ++++++++++++++++++++++++++++++++++++++-----
3 files changed, 119 insertions(+), 10 deletions(-)
create mode 100644 arch/arm64/kernel/msi.c
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index df7ef87..a921c42 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -29,6 +29,7 @@ arm64-obj-$(CONFIG_ARM64_CPU_SUSPEND) += sleep.o suspend.o
arm64-obj-$(CONFIG_JUMP_LABEL) += jump_label.o
arm64-obj-$(CONFIG_KGDB) += kgdb.o
arm64-obj-$(CONFIG_EFI) += efi.o efi-stub.o efi-entry.o
+arm64-obj-$(CONFIG_PCI_MSI) += msi.o
obj-y += $(arm64-obj-y) vdso/
obj-m += $(arm64-obj-m)
diff --git a/arch/arm64/kernel/msi.c b/arch/arm64/kernel/msi.c
new file mode 100644
index 0000000..a295862
--- /dev/null
+++ b/arch/arm64/kernel/msi.c
@@ -0,0 +1,41 @@
+/*
+ * ARM64 architectural MSI implemention
+ *
+ * Support for Message Signalelled Interrupts for systems that
+ * implement ARM Generic Interrupt Controller: GICv2m.
+ *
+ * Copyright (C) 2014 Advanced Micro Devices, Inc.
+ * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ */
+
+#include <linux/irq.h>
+#include <linux/msi.h>
+#include <linux/pci.h>
+
+/*
+ * ARM64 function for seting up MSI irqs.
+ * Based on driver/pci/msi.c: arch_setup_msi_irqs().
+ *
+ * Note:
+ * Current implementation assumes that all interrupt controller used in
+ * ARM64 architecture _MUST_ supports multi-MSI.
+ */
+int arm64_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
+{
+ struct msi_desc *entry;
+ int ret;
+
+ list_for_each_entry(entry, &dev->msi_list, list) {
+ ret = arch_setup_msi_irq(dev, entry);
+ if (ret < 0)
+ return ret;
+ if (ret > 0)
+ return -ENOSPC;
+ }
+
+ return 0;
+}
diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
index 499bfb9..4ddb47d 100644
--- a/drivers/irqchip/irq-gic-v2m.c
+++ b/drivers/irqchip/irq-gic-v2m.c
@@ -99,13 +99,26 @@ static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
spin_unlock(&data->msi_cnt_lock);
}
+static int gicv2m_msi_get_vec_count(struct pci_dev *pdev, struct msi_desc *desc)
+{
+ int ret = -EINVAL;
+#ifdef CONFIG_PCI_MSI
+ if (desc->msi_attrib.is_msix)
+ ret = pci_msix_vec_count(pdev);
+ else
+ ret = pci_msi_vec_count(pdev);
+#endif
+ return ret;
+}
+
static int gicv2m_setup_msi_irq(struct msi_chip *chip,
struct pci_dev *pdev,
struct msi_desc *desc)
{
- int irq, avail;
+ int i, irq, nvec, avail;
struct msi_msg msg;
phys_addr_t addr;
+ struct msi_desc *entry;
struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
if (!desc) {
@@ -114,16 +127,70 @@ static int gicv2m_setup_msi_irq(struct msi_chip *chip,
return -EINVAL;
}
- avail = alloc_msi_irq(data, 1, &irq);
- if (avail != 0) {
- dev_err(&pdev->dev,
- "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
- return -ENOSPC;
- }
+ if (desc->msi_attrib.is_msix) {
+ /**
+ * For MSIx:
+ * We allocate one irq at a time
+ */
+ avail = alloc_msi_irq(data, 1, &irq);
+ if (avail != 0) {
+ dev_err(&pdev->dev,
+ "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
+ return -ENOSPC;
+ }
- irq_set_chip_data(irq, chip);
- irq_set_msi_desc(irq, desc);
- irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
+ irq_set_chip_data(irq, chip);
+ irq_set_msi_desc(irq, desc);
+ irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
+ } else {
+ /**
+ * For MSI and Multi-MSI:
+ * All requested irqs are allocated and setup at
+ * once. Subsequent calls to this function would simply return
+ * success. This is to avoid having to implement a separate
+ * function for setting up multiple irqs.
+ */
+ BUG_ON(list_empty(&pdev->msi_list));
+ WARN_ON(!list_is_singular(&pdev->msi_list));
+
+ nvec = gicv2m_msi_get_vec_count(pdev, desc);
+ if (WARN_ON(nvec <= 0))
+ return nvec;
+
+ entry = list_first_entry(&pdev->msi_list,
+ struct msi_desc, list);
+
+ if ((nvec > 1) && (entry->msi_attrib.multiple))
+ return 0;
+
+ avail = alloc_msi_irq(data, nvec, &irq);
+ if (avail != 0) {
+ dev_err(&pdev->dev,
+ "GICv2m: Failed to allocate %d irqs.\n", nvec);
+ return avail;
+ }
+
+ if (nvec > 1) {
+ /* Set lowest of the new interrupts assigned
+ * to the PCI device
+ */
+ entry->nvec_used = nvec;
+ entry->msi_attrib.multiple = ilog2(
+ __roundup_pow_of_two(nvec));
+ }
+
+ for (i = 0; i < nvec; i++) {
+ irq_set_chip_data(irq+i, chip);
+ if (irq_set_msi_desc_off(irq, i, entry)) {
+ dev_err(&pdev->dev,
+ "GICv2m: Failed to set up MSI irq %d\n",
+ (irq+i));
+ return -EINVAL;
+ }
+
+ irq_set_irq_type((irq+i), IRQ_TYPE_EDGE_RISING);
+ }
+ }
addr = data->res.start + V2M_MSI_SETSPI_NS;
msg.address_hi = (u32)(addr >> 32);
--
1.9.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH 2/2 V6] irqchip: gicv2m: Add support for multiple MSI for ARM64 GICv2m
@ 2014-09-14 6:30 ` suravee.suthikulpanit
0 siblings, 0 replies; 17+ messages in thread
From: suravee.suthikulpanit @ 2014-09-14 6:30 UTC (permalink / raw)
To: marc.zyngier, mark.rutland, jason
Cc: pawel.moll, Catalin.Marinas, Will.Deacon, tglx,
Harish.Kasiviswanathan, linux-arm-kernel, linux-pci,
linux-kernel, linux-doc, devicetree, Suravee Suthikulpanit,
Mark Rutland, Marc Zyngier
From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
This patch extend GICv2m MSI to support multiple MSI in ARM64.
This requires the common arch_setup_msi_irqs() to be overwriten
with ARM64 version which does not return 1 for PCI_CAP_ID_MSI and
nvec > 1.
Cc: Mark Rutland <Mark.Rutland@arm.com>
Cc: Marc Zyngier <Marc.Zyngier@arm.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Catalin Marinas <Catalin.Marinas@arm.com>
Cc: Will Deacon <Will.Deacon@arm.com>
Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
---
arch/arm64/kernel/Makefile | 1 +
arch/arm64/kernel/msi.c | 41 ++++++++++++++++++++
drivers/irqchip/irq-gic-v2m.c | 87 ++++++++++++++++++++++++++++++++++++++-----
3 files changed, 119 insertions(+), 10 deletions(-)
create mode 100644 arch/arm64/kernel/msi.c
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index df7ef87..a921c42 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -29,6 +29,7 @@ arm64-obj-$(CONFIG_ARM64_CPU_SUSPEND) += sleep.o suspend.o
arm64-obj-$(CONFIG_JUMP_LABEL) += jump_label.o
arm64-obj-$(CONFIG_KGDB) += kgdb.o
arm64-obj-$(CONFIG_EFI) += efi.o efi-stub.o efi-entry.o
+arm64-obj-$(CONFIG_PCI_MSI) += msi.o
obj-y += $(arm64-obj-y) vdso/
obj-m += $(arm64-obj-m)
diff --git a/arch/arm64/kernel/msi.c b/arch/arm64/kernel/msi.c
new file mode 100644
index 0000000..a295862
--- /dev/null
+++ b/arch/arm64/kernel/msi.c
@@ -0,0 +1,41 @@
+/*
+ * ARM64 architectural MSI implemention
+ *
+ * Support for Message Signalelled Interrupts for systems that
+ * implement ARM Generic Interrupt Controller: GICv2m.
+ *
+ * Copyright (C) 2014 Advanced Micro Devices, Inc.
+ * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ */
+
+#include <linux/irq.h>
+#include <linux/msi.h>
+#include <linux/pci.h>
+
+/*
+ * ARM64 function for seting up MSI irqs.
+ * Based on driver/pci/msi.c: arch_setup_msi_irqs().
+ *
+ * Note:
+ * Current implementation assumes that all interrupt controller used in
+ * ARM64 architecture _MUST_ supports multi-MSI.
+ */
+int arm64_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
+{
+ struct msi_desc *entry;
+ int ret;
+
+ list_for_each_entry(entry, &dev->msi_list, list) {
+ ret = arch_setup_msi_irq(dev, entry);
+ if (ret < 0)
+ return ret;
+ if (ret > 0)
+ return -ENOSPC;
+ }
+
+ return 0;
+}
diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
index 499bfb9..4ddb47d 100644
--- a/drivers/irqchip/irq-gic-v2m.c
+++ b/drivers/irqchip/irq-gic-v2m.c
@@ -99,13 +99,26 @@ static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
spin_unlock(&data->msi_cnt_lock);
}
+static int gicv2m_msi_get_vec_count(struct pci_dev *pdev, struct msi_desc *desc)
+{
+ int ret = -EINVAL;
+#ifdef CONFIG_PCI_MSI
+ if (desc->msi_attrib.is_msix)
+ ret = pci_msix_vec_count(pdev);
+ else
+ ret = pci_msi_vec_count(pdev);
+#endif
+ return ret;
+}
+
static int gicv2m_setup_msi_irq(struct msi_chip *chip,
struct pci_dev *pdev,
struct msi_desc *desc)
{
- int irq, avail;
+ int i, irq, nvec, avail;
struct msi_msg msg;
phys_addr_t addr;
+ struct msi_desc *entry;
struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
if (!desc) {
@@ -114,16 +127,70 @@ static int gicv2m_setup_msi_irq(struct msi_chip *chip,
return -EINVAL;
}
- avail = alloc_msi_irq(data, 1, &irq);
- if (avail != 0) {
- dev_err(&pdev->dev,
- "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
- return -ENOSPC;
- }
+ if (desc->msi_attrib.is_msix) {
+ /**
+ * For MSIx:
+ * We allocate one irq at a time
+ */
+ avail = alloc_msi_irq(data, 1, &irq);
+ if (avail != 0) {
+ dev_err(&pdev->dev,
+ "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
+ return -ENOSPC;
+ }
- irq_set_chip_data(irq, chip);
- irq_set_msi_desc(irq, desc);
- irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
+ irq_set_chip_data(irq, chip);
+ irq_set_msi_desc(irq, desc);
+ irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
+ } else {
+ /**
+ * For MSI and Multi-MSI:
+ * All requested irqs are allocated and setup at
+ * once. Subsequent calls to this function would simply return
+ * success. This is to avoid having to implement a separate
+ * function for setting up multiple irqs.
+ */
+ BUG_ON(list_empty(&pdev->msi_list));
+ WARN_ON(!list_is_singular(&pdev->msi_list));
+
+ nvec = gicv2m_msi_get_vec_count(pdev, desc);
+ if (WARN_ON(nvec <= 0))
+ return nvec;
+
+ entry = list_first_entry(&pdev->msi_list,
+ struct msi_desc, list);
+
+ if ((nvec > 1) && (entry->msi_attrib.multiple))
+ return 0;
+
+ avail = alloc_msi_irq(data, nvec, &irq);
+ if (avail != 0) {
+ dev_err(&pdev->dev,
+ "GICv2m: Failed to allocate %d irqs.\n", nvec);
+ return avail;
+ }
+
+ if (nvec > 1) {
+ /* Set lowest of the new interrupts assigned
+ * to the PCI device
+ */
+ entry->nvec_used = nvec;
+ entry->msi_attrib.multiple = ilog2(
+ __roundup_pow_of_two(nvec));
+ }
+
+ for (i = 0; i < nvec; i++) {
+ irq_set_chip_data(irq+i, chip);
+ if (irq_set_msi_desc_off(irq, i, entry)) {
+ dev_err(&pdev->dev,
+ "GICv2m: Failed to set up MSI irq %d\n",
+ (irq+i));
+ return -EINVAL;
+ }
+
+ irq_set_irq_type((irq+i), IRQ_TYPE_EDGE_RISING);
+ }
+ }
addr = data->res.start + V2M_MSI_SETSPI_NS;
msg.address_hi = (u32)(addr >> 32);
--
1.9.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PATCH 2/2 V6] irqchip: gicv2m: Add support for multiple MSI for ARM64 GICv2m
@ 2014-09-14 6:30 ` suravee.suthikulpanit
0 siblings, 0 replies; 17+ messages in thread
From: suravee.suthikulpanit at amd.com @ 2014-09-14 6:30 UTC (permalink / raw)
To: linux-arm-kernel
From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
This patch extend GICv2m MSI to support multiple MSI in ARM64.
This requires the common arch_setup_msi_irqs() to be overwriten
with ARM64 version which does not return 1 for PCI_CAP_ID_MSI and
nvec > 1.
Cc: Mark Rutland <Mark.Rutland@arm.com>
Cc: Marc Zyngier <Marc.Zyngier@arm.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Catalin Marinas <Catalin.Marinas@arm.com>
Cc: Will Deacon <Will.Deacon@arm.com>
Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
---
arch/arm64/kernel/Makefile | 1 +
arch/arm64/kernel/msi.c | 41 ++++++++++++++++++++
drivers/irqchip/irq-gic-v2m.c | 87 ++++++++++++++++++++++++++++++++++++++-----
3 files changed, 119 insertions(+), 10 deletions(-)
create mode 100644 arch/arm64/kernel/msi.c
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index df7ef87..a921c42 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -29,6 +29,7 @@ arm64-obj-$(CONFIG_ARM64_CPU_SUSPEND) += sleep.o suspend.o
arm64-obj-$(CONFIG_JUMP_LABEL) += jump_label.o
arm64-obj-$(CONFIG_KGDB) += kgdb.o
arm64-obj-$(CONFIG_EFI) += efi.o efi-stub.o efi-entry.o
+arm64-obj-$(CONFIG_PCI_MSI) += msi.o
obj-y += $(arm64-obj-y) vdso/
obj-m += $(arm64-obj-m)
diff --git a/arch/arm64/kernel/msi.c b/arch/arm64/kernel/msi.c
new file mode 100644
index 0000000..a295862
--- /dev/null
+++ b/arch/arm64/kernel/msi.c
@@ -0,0 +1,41 @@
+/*
+ * ARM64 architectural MSI implemention
+ *
+ * Support for Message Signalelled Interrupts for systems that
+ * implement ARM Generic Interrupt Controller: GICv2m.
+ *
+ * Copyright (C) 2014 Advanced Micro Devices, Inc.
+ * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ */
+
+#include <linux/irq.h>
+#include <linux/msi.h>
+#include <linux/pci.h>
+
+/*
+ * ARM64 function for seting up MSI irqs.
+ * Based on driver/pci/msi.c: arch_setup_msi_irqs().
+ *
+ * Note:
+ * Current implementation assumes that all interrupt controller used in
+ * ARM64 architecture _MUST_ supports multi-MSI.
+ */
+int arm64_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
+{
+ struct msi_desc *entry;
+ int ret;
+
+ list_for_each_entry(entry, &dev->msi_list, list) {
+ ret = arch_setup_msi_irq(dev, entry);
+ if (ret < 0)
+ return ret;
+ if (ret > 0)
+ return -ENOSPC;
+ }
+
+ return 0;
+}
diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
index 499bfb9..4ddb47d 100644
--- a/drivers/irqchip/irq-gic-v2m.c
+++ b/drivers/irqchip/irq-gic-v2m.c
@@ -99,13 +99,26 @@ static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
spin_unlock(&data->msi_cnt_lock);
}
+static int gicv2m_msi_get_vec_count(struct pci_dev *pdev, struct msi_desc *desc)
+{
+ int ret = -EINVAL;
+#ifdef CONFIG_PCI_MSI
+ if (desc->msi_attrib.is_msix)
+ ret = pci_msix_vec_count(pdev);
+ else
+ ret = pci_msi_vec_count(pdev);
+#endif
+ return ret;
+}
+
static int gicv2m_setup_msi_irq(struct msi_chip *chip,
struct pci_dev *pdev,
struct msi_desc *desc)
{
- int irq, avail;
+ int i, irq, nvec, avail;
struct msi_msg msg;
phys_addr_t addr;
+ struct msi_desc *entry;
struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
if (!desc) {
@@ -114,16 +127,70 @@ static int gicv2m_setup_msi_irq(struct msi_chip *chip,
return -EINVAL;
}
- avail = alloc_msi_irq(data, 1, &irq);
- if (avail != 0) {
- dev_err(&pdev->dev,
- "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
- return -ENOSPC;
- }
+ if (desc->msi_attrib.is_msix) {
+ /**
+ * For MSIx:
+ * We allocate one irq at a time
+ */
+ avail = alloc_msi_irq(data, 1, &irq);
+ if (avail != 0) {
+ dev_err(&pdev->dev,
+ "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
+ return -ENOSPC;
+ }
- irq_set_chip_data(irq, chip);
- irq_set_msi_desc(irq, desc);
- irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
+ irq_set_chip_data(irq, chip);
+ irq_set_msi_desc(irq, desc);
+ irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
+ } else {
+ /**
+ * For MSI and Multi-MSI:
+ * All requested irqs are allocated and setup at
+ * once. Subsequent calls to this function would simply return
+ * success. This is to avoid having to implement a separate
+ * function for setting up multiple irqs.
+ */
+ BUG_ON(list_empty(&pdev->msi_list));
+ WARN_ON(!list_is_singular(&pdev->msi_list));
+
+ nvec = gicv2m_msi_get_vec_count(pdev, desc);
+ if (WARN_ON(nvec <= 0))
+ return nvec;
+
+ entry = list_first_entry(&pdev->msi_list,
+ struct msi_desc, list);
+
+ if ((nvec > 1) && (entry->msi_attrib.multiple))
+ return 0;
+
+ avail = alloc_msi_irq(data, nvec, &irq);
+ if (avail != 0) {
+ dev_err(&pdev->dev,
+ "GICv2m: Failed to allocate %d irqs.\n", nvec);
+ return avail;
+ }
+
+ if (nvec > 1) {
+ /* Set lowest of the new interrupts assigned
+ * to the PCI device
+ */
+ entry->nvec_used = nvec;
+ entry->msi_attrib.multiple = ilog2(
+ __roundup_pow_of_two(nvec));
+ }
+
+ for (i = 0; i < nvec; i++) {
+ irq_set_chip_data(irq+i, chip);
+ if (irq_set_msi_desc_off(irq, i, entry)) {
+ dev_err(&pdev->dev,
+ "GICv2m: Failed to set up MSI irq %d\n",
+ (irq+i));
+ return -EINVAL;
+ }
+
+ irq_set_irq_type((irq+i), IRQ_TYPE_EDGE_RISING);
+ }
+ }
addr = data->res.start + V2M_MSI_SETSPI_NS;
msg.address_hi = (u32)(addr >> 32);
--
1.9.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PATCH 1/2 V6] irqchip: gic: Add supports for ARM GICv2m MSI(-X)
2014-09-14 6:30 ` suravee.suthikulpanit
(?)
(?)
@ 2014-09-15 19:26 ` Marc Zyngier
-1 siblings, 0 replies; 17+ messages in thread
From: Marc Zyngier @ 2014-09-15 19:26 UTC (permalink / raw)
To: suravee.suthikulpanit
Cc: Mark Rutland, jason, Pawel Moll, Catalin Marinas, Will Deacon,
tglx, Harish.Kasiviswanathan, linux-arm-kernel, linux-pci,
linux-kernel, linux-doc, devicetree
On Sun, Sep 14 2014 at 07:30:15 AM, "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com> wrote:
Hi Suravee,
> From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>
> ARM GICv2m specification extends GICv2 to support MSI(-X) with
> a new set of register frame. This patch introduces support for
> the non-secure GICv2m register frame. Currently, GICV2m is available
> in certain version of GIC-400.
>
> The patch introduces a new property in ARM gic binding, the v2m subnode.
> It is optional.
Overall, this looks much better than the previous versions I've
reviewed. There is still a number of points I'd like to see addressed,
see below.
>
> Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> Cc: Mark Rutland <Mark.Rutland@arm.com>
> Cc: Marc Zyngier <Marc.Zyngier@arm.com>
> Cc: Jason Cooper <jason@lakedaemon.net>
> Cc: Catalin Marinas <Catalin.Marinas@arm.com>
> Cc: Will Deacon <Will.Deacon@arm.com>
> ---
> Documentation/devicetree/bindings/arm/gic.txt | 39 ++++
> drivers/irqchip/Kconfig | 7 +
> drivers/irqchip/Makefile | 1 +
> drivers/irqchip/irq-gic-v2m.c | 261 ++++++++++++++++++++++++++
> drivers/irqchip/irq-gic.c | 88 ++++++---
> drivers/irqchip/irq-gic.h | 51 +++++
> 6 files changed, 419 insertions(+), 28 deletions(-)
> create mode 100644 drivers/irqchip/irq-gic-v2m.c
> create mode 100644 drivers/irqchip/irq-gic.h
>
> diff --git a/Documentation/devicetree/bindings/arm/gic.txt b/Documentation/devicetree/bindings/arm/gic.txt
> index c7d2fa1..a461591 100644
> --- a/Documentation/devicetree/bindings/arm/gic.txt
> +++ b/Documentation/devicetree/bindings/arm/gic.txt
> @@ -96,3 +96,42 @@ Example:
> <0x2c006000 0x2000>;
> interrupts = <1 9 0xf04>;
> };
> +
> +
> +* GICv2m extension for MSI/MSI-x support (Optional)
> +
> +Certain revision of GIC-400 supports MSI/MSI-x via V2M register frame.
> +This is enabled by specifying v2m sub-node.
> +
> +Required properties:
> +
> +- msi-controller : Identifies the node as an MSI controller.
> +
> +- reg : GICv2m MSI interface register base and size
> +
> +Example:
> +
> + interrupt-controller@e1101000 {
> + compatible = "arm,gic-400";
> + #interrupt-cells = <3>;
> + #address-cells = <2>;
> + #size-cells = <2>;
> + interrupt-controller;
> + interrupts = <1 8 0xf04>;
> + ranges = <0 0 0 0xe1100000 0 0x100000>;
> + reg = <0x0 0xe1110000 0 0x01000>,
> + <0x0 0xe112f000 0 0x02000>,
> + <0x0 0xe1140000 0 0x10000>,
> + <0x0 0xe1160000 0 0x10000>;
> + v2m0 {
I believe the convention is to have
v2m0 : v2m@0x80000 {
> + msi-controller;
> + reg = <0x0 0x80000 0 0x1000>;
I think having a "compatible" property would be nice, and would help
handle variants if they ever happen.
> + };
> +
> + ....
> +
> + v2mN {
> + msi-controller;
> + reg = <0x0 0x90000 0 0x1000>;
> + };
> + };
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index b8632bf..ddb31ee 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -7,6 +7,13 @@ config ARM_GIC
> select IRQ_DOMAIN
> select MULTI_IRQ_HANDLER
>
> +config ARM_GIC_V2M
> + bool
> + select IRQ_DOMAIN
> + select MULTI_IRQ_HANDLER
> + depends on ARM_GIC
> + depends on PCI && PCI_MSI
> +
> config GIC_NON_BANKED
> bool
>
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index 73052ba..3bda951 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -17,6 +17,7 @@ obj-$(CONFIG_ARCH_SUNXI) += irq-sun4i.o
> obj-$(CONFIG_ARCH_SUNXI) += irq-sunxi-nmi.o
> obj-$(CONFIG_ARCH_SPEAR3XX) += spear-shirq.o
> obj-$(CONFIG_ARM_GIC) += irq-gic.o irq-gic-common.o
> +obj-$(CONFIG_ARM_GIC_V2M) += irq-gic-v2m.o
> obj-$(CONFIG_ARM_GIC_V3) += irq-gic-v3.o irq-gic-common.o
> obj-$(CONFIG_ARM_NVIC) += irq-nvic.o
> obj-$(CONFIG_ARM_VIC) += irq-vic.o
> diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
> new file mode 100644
> index 0000000..499bfb9
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic-v2m.c
> @@ -0,0 +1,261 @@
> +/*
> + * ARM GIC v2m MSI(-X) support
> + * Support for Message Signaled Interrupts for systems that
> + * implement ARM Generic Interrupt Controller: GICv2m.
> + *
> + * Copyright (C) 2014 Advanced Micro Devices, Inc.
> + * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> + * Harish Kasiviswanathan <harish.kasiviswanathan@amd.com>
> + * Brandon Anderson <brandon.anderson@amd.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published
> + * by the Free Software Foundation.
> + */
> +
> +#define pr_fmt(fmt) "GICv2m: " fmt
> +
> +#include <linux/bitmap.h>
> +#include <linux/irq.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_pci.h>
> +#include <linux/pci.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +#include <linux/spinlock.h>
> +
> +#include "irqchip.h"
> +#include "irq-gic.h"
> +
> +/*
> +* MSI_TYPER:
> +* [31:26] Reserved
> +* [25:16] lowest SPI assigned to MSI
> +* [15:10] Reserved
> +* [9:0] Numer of SPIs assigned to MSI
> +*/
> +#define V2M_MSI_TYPER 0x008
> +#define V2M_MSI_TYPER_BASE_SHIFT 16
> +#define V2M_MSI_TYPER_BASE_MASK 0x3FF
> +#define V2M_MSI_TYPER_NUM_MASK 0x3FF
> +#define V2M_MSI_SETSPI_NS 0x040
> +#define V2M_MIN_SPI 32
> +#define V2M_MAX_SPI 1019
> +
> +/*
> + * alloc_msi_irq - Allocate MSIs from available MSI bitmap.
> + * @data: Pointer to v2m_data
> + * @nvec: Number of interrupts to allocate
> + * @irq: Pointer to the allocated irq
> + *
> + * Allocates interrupts only if the contiguous range of MSIs
> + * with specified nvec are available. Otherwise return the number
> + * of available interrupts. If none are available, then returns -ENOENT.
> + */
> +static int alloc_msi_irq(struct v2m_data *data, int nvec, int *irq)
> +{
> + int size = data->nr_spis;
> + int next = size, i = nvec, ret;
> +
> + /* We should never allocate more than available nr_spis */
> + if (i >= size)
> + i = size;
> +
> + spin_lock(&data->msi_cnt_lock);
> +
> + for (; i > 0; i--) {
> + next = bitmap_find_next_zero_area(data->bm,
> + size, 0, i, 0);
> + if (next < size)
> + break;
> + }
> +
> + if (i != nvec) {
> + ret = i ? : -ENOENT;
> + } else {
> + bitmap_set(data->bm, next, nvec);
> + *irq = data->spi_start + next;
> + ret = 0;
> + }
> +
> + spin_unlock(&data->msi_cnt_lock);
> +
> + return ret;
> +}
> +
> +static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
> +{
> + int pos;
> + struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> + spin_lock(&data->msi_cnt_lock);
> +
> + pos = irq - data->spi_start;
> + if (pos >= 0 && pos < data->nr_spis)
> + bitmap_clear(data->bm, pos, 1);
> +
> + spin_unlock(&data->msi_cnt_lock);
> +}
> +
> +static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> + struct pci_dev *pdev,
> + struct msi_desc *desc)
> +{
> + int irq, avail;
> + struct msi_msg msg;
> + phys_addr_t addr;
> + struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> + if (!desc) {
> + dev_err(&pdev->dev,
> + "GICv2m: MSI setup failed. Invalid msi descriptor\n");
Isn't the "GICv2m" header already provided by the pr_fmt at the
beginning of the file?
> + return -EINVAL;
> + }
> +
> + avail = alloc_msi_irq(data, 1, &irq);
> + if (avail != 0) {
> + dev_err(&pdev->dev,
> + "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> + return -ENOSPC;
> + }
> +
> + irq_set_chip_data(irq, chip);
> + irq_set_msi_desc(irq, desc);
> + irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> +
> + addr = data->res.start + V2M_MSI_SETSPI_NS;
> + msg.address_hi = (u32)(addr >> 32);
> + msg.address_lo = (u32)(addr);
> + msg.data = irq;
> + write_msi_msg(irq, &msg);
> +
> + return 0;
> +}
> +
> +static void gicv2m_mask_irq(struct irq_data *d)
> +{
> + gic_mask_irq(d);
> + if (d->msi_desc)
> + mask_msi_irq(d);
> +}
> +
> +static void gicv2m_unmask_irq(struct irq_data *d)
> +{
> + gic_unmask_irq(d);
> + if (d->msi_desc)
> + unmask_msi_irq(d);
> +}
> +
> +#ifdef CONFIG_OF
I think you can loose the CONFIG_OF here. There is no valid
configuration without OF on arm64.
> +
> +static int __init
> +v2m_init(struct device_node *node, struct v2m_data *v2m,
> + struct resource *res)
> +{
> + int ret;
> + unsigned int typer;
> +
> + v2m->msi_chip.owner = THIS_MODULE;
> + v2m->msi_chip.of_node = node;
> + v2m->msi_chip.setup_irq = gicv2m_setup_msi_irq;
> + v2m->msi_chip.teardown_irq = gicv2m_teardown_msi_irq;
> + ret = of_pci_msi_chip_add(&v2m->msi_chip);
> + if (ret) {
> + pr_info("Failed to add msi_chip.\n");
> + return ret;
> + }
> +
> + v2m->base = ioremap(res->start, resource_size(res));
> + if (!v2m->base) {
> + pr_err("Failed to map GICv2m resource\n");
> + return -EINVAL;
Beware, you're leaving with a registered msi_chip. You probably want to
remove it on the error path.
> + }
> +
> + memcpy(&v2m->res, res, sizeof(struct resource));
> +
> + typer = readl_relaxed(v2m->base + V2M_MSI_TYPER);
> + if (!typer) {
> + pr_warn("Failed to read V2M_MSI_TYPER register\n");
> + return -EINVAL;
And here leaving the v2m widget mapped.
> + }
> +
> + v2m->spi_start = (typer >> V2M_MSI_TYPER_BASE_SHIFT) &
> + V2M_MSI_TYPER_BASE_MASK;
> + v2m->nr_spis = typer & V2M_MSI_TYPER_NUM_MASK;
> + if ((v2m->spi_start < V2M_MIN_SPI) || (v2m->nr_spis >= V2M_MAX_SPI)) {
> + pr_err("Invalid MSI_TYPER (%#x)\n", typer);
> + return -EINVAL;
Blah. The whole error handling of this function is b0rked.
> + }
> +
> + v2m->bm = kzalloc(sizeof(long) * BITS_TO_LONGS(v2m->nr_spis),
> + GFP_KERNEL);
> + if (!v2m->bm) {
> + pr_err("Failed to allocate MSI bitmap\n");
> + return -ENOMEM;
> + }
> +
> + spin_lock_init(&v2m->msi_cnt_lock);
> +
> + pr_info("SPI range [%d:%d]\n",
> + v2m->spi_start, (v2m->spi_start + v2m->nr_spis));
> +
> + return 0;
> +}
> +
> +int __init gicv2m_of_init(struct device_node *node,
> + struct gic_chip_data *gic,
> + struct irq_chip *v2m_chip)
> +{
> + int ret = 0;
> + struct v2m_data *v2m;
> + struct resource res;
> + struct device_node *child = NULL;
> +
> + INIT_LIST_HEAD(&gic->v2m_list);
> +
> + for (;;) {
> + child = of_get_next_child(node, child);
I don't see any of_node_put() in this function. You'll need to fix
this...
> + if (!child)
> + break;
> +
> + if (!of_find_property(child, "msi-controller", NULL))
> + continue;
> +
> + ret = of_address_to_resource(child, 0, &res);
> + if (ret) {
> + pr_err("Failed to allocate v2m resource.\n");
> + break;
> + }
> +
> + v2m = kzalloc(sizeof(struct v2m_data), GFP_KERNEL);
> + if (!v2m) {
> + pr_err("Failed to allocate struct v2m_data.\n");
> + return -ENOMEM;
> + }
> +
> + ret = v2m_init(child, v2m, &res);
> + if (ret)
> + break;
> +
> + v2m->gic = gic;
> +
> + pr_info(" %s: [%lx : %lx]\n", child->name,
> + (unsigned long)res.start, (unsigned long)res.end);
> +
> + list_add_tail(&v2m->list, &gic->v2m_list);
> + }
> +
> + if (ret && list_empty(&gic->v2m_list)) {
> + pr_err("MSI resource not found\n");
> + return -EINVAL;
> + }
> +
> + v2m_chip->irq_mask = gicv2m_mask_irq;
> + v2m_chip->irq_unmask = gicv2m_unmask_irq;
> +
Again, the whole error handling is evil. You can end-up allocating a few
v2m structures, registering them, and yet have kzalloc failing at some
point. At that stage, you exit early, not setting the mas/unmask
methods. Boom.
> + return ret;
> +}
> +
> +#endif /* CONFIG_OF */
> diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
> index 4b959e6..f490599 100644
> --- a/drivers/irqchip/irq-gic.c
> +++ b/drivers/irqchip/irq-gic.c
> @@ -46,30 +46,9 @@
> #include <asm/smp_plat.h>
>
> #include "irq-gic-common.h"
> +#include "irq-gic.h"
> #include "irqchip.h"
>
> -union gic_base {
> - void __iomem *common_base;
> - void __percpu * __iomem *percpu_base;
> -};
> -
> -struct gic_chip_data {
> - union gic_base dist_base;
> - union gic_base cpu_base;
> -#ifdef CONFIG_CPU_PM
> - u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> - u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> - u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> - u32 __percpu *saved_ppi_enable;
> - u32 __percpu *saved_ppi_conf;
> -#endif
> - struct irq_domain *domain;
> - unsigned int gic_irqs;
> -#ifdef CONFIG_GIC_NON_BANKED
> - void __iomem *(*get_base)(union gic_base *);
> -#endif
> -};
> -
> static DEFINE_RAW_SPINLOCK(irq_controller_lock);
>
> /*
> @@ -131,15 +110,36 @@ static inline void gic_set_base_accessor(struct gic_chip_data *data,
> #define gic_set_base_accessor(d, f)
> #endif
>
> +static inline
> +struct gic_chip_data *irq_data_get_gic_chip_data(struct irq_data *d)
> +{
> + struct gic_chip_data *gic_data;
> + struct msi_chip *mchip;
> + struct v2m_data *v2mdat;
> +
> + /*
> + * For MSI, irq_data.chip_data points to struct msi_chip.
> + * For non-MSI, irq_data.chip_data points to struct gic_chip_data.
> + */
> + if (d->msi_desc) {
> + mchip = irq_data_get_irq_chip_data(d);
> + v2mdat = container_of(mchip, struct v2m_data, msi_chip);
> + gic_data = v2mdat->gic;
> + } else {
> + gic_data = irq_data_get_irq_chip_data(d);
> + }
> + return gic_data;
> +}
> +
> static inline void __iomem *gic_dist_base(struct irq_data *d)
> {
> - struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> + struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
> return gic_data_dist_base(gic_data);
> }
>
> static inline void __iomem *gic_cpu_base(struct irq_data *d)
> {
> - struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> + struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
> return gic_data_cpu_base(gic_data);
> }
>
> @@ -151,7 +151,7 @@ static inline unsigned int gic_irq(struct irq_data *d)
> /*
> * Routines to acknowledge, disable and enable interrupts
> */
> -static void gic_mask_irq(struct irq_data *d)
> +void gic_mask_irq(struct irq_data *d)
I'm still very uncomfortable with exporting the mask/unmask methods
outside of this file... Maybe it is not a big deal, but I see evil
everywhere.
> {
> u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -162,7 +162,7 @@ static void gic_mask_irq(struct irq_data *d)
> raw_spin_unlock(&irq_controller_lock);
> }
>
> -static void gic_unmask_irq(struct irq_data *d)
> +void gic_unmask_irq(struct irq_data *d)
> {
> u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -325,6 +325,15 @@ static struct irq_chip gic_chip = {
> .irq_set_wake = gic_set_wake,
> };
>
> +static struct irq_chip v2m_chip = {
> + .name = "GICv2m",
> + .irq_eoi = gic_eoi_irq,
> + .irq_set_type = gic_set_type,
> +#ifdef CONFIG_SMP
> + .irq_set_affinity = gic_set_affinity,
> +#endif
> +};
> +
> void __init gic_cascade_irq(unsigned int gic_nr, unsigned int irq)
> {
> if (gic_nr >= MAX_GIC_NR)
> @@ -767,6 +776,24 @@ void __init gic_init_physaddr(struct device_node *node)
> static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
> irq_hw_number_t hw)
> {
> + struct gic_chip_data *gic = d->host_data;
> + struct v2m_data *v2m;
> +
> + irq_set_chip_data(irq, gic);
>
> +#ifdef CONFIG_ARM_GIC_V2M
> + list_for_each_entry(v2m, &gic->v2m_list, list) {
> + if (hw >= v2m->spi_start &&
> + hw < v2m->spi_start + v2m->nr_spis) {
> +
> + irq_set_chip_and_handler(irq, &v2m_chip,
> + handle_fasteoi_irq);
> + set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
> + return 0;
> + }
> + }
> +#endif
> +
> if (hw < 32) {
> irq_set_percpu_devid(irq);
> irq_set_chip_and_handler(irq, &gic_chip,
> @@ -779,7 +806,6 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
>
> gic_routable_irq_domain_ops->map(d, irq, hw);
> }
> - irq_set_chip_data(irq, d->host_data);
These two hunks make the code a bit hard to read:
- First, message-based SPIs,
- Then PPIs
- and finally the rest of the SPIs.
Can you come up with a construct that keeps everything in a logical
order (PPIs, and then SPIs)? Not a big deal but it would make the code
nicer.
> return 0;
> }
>
> @@ -996,7 +1022,7 @@ gic_of_init(struct device_node *node, struct device_node *parent)
> void __iomem *cpu_base;
> void __iomem *dist_base;
> u32 percpu_offset;
> - int irq;
> + int irq, ret;
>
> if (WARN_ON(!node))
> return -ENODEV;
> @@ -1010,6 +1036,12 @@ gic_of_init(struct device_node *node, struct device_node *parent)
> if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
> percpu_offset = 0;
>
> + if (IS_ENABLED(CONFIG_ARM_GIC_V2M)) {
> + ret = gicv2m_of_init(node, &gic_data[gic_cnt], &v2m_chip);
> + if (ret)
> + pr_warn("Warning: Failed to enable GICv2m support.\n");
> + }
> +
> gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset, node);
> if (!gic_cnt)
> gic_init_physaddr(node);
> diff --git a/drivers/irqchip/irq-gic.h b/drivers/irqchip/irq-gic.h
> new file mode 100644
> index 0000000..1f4a2b0
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic.h
> @@ -0,0 +1,51 @@
> +#ifndef _IRQ_GIC_H_
> +#define _IRQ_GIC_H_
> +
> +#include <linux/msi.h>
> +#include <linux/pci.h>
> +
> +union gic_base {
> + void __iomem *common_base;
> + void __percpu * __iomem *percpu_base;
> +};
> +
> +struct gic_chip_data;
> +
> +struct v2m_data {
> +#ifdef CONFIG_ARM_GIC_V2M
> + struct list_head list;
> + spinlock_t msi_cnt_lock;
> + struct msi_chip msi_chip;
> + struct resource res; /* GICv2m resource */
> + void __iomem *base; /* GICv2m virt address */
> + unsigned int spi_start; /* The SPI number that MSIs start */
> + unsigned int nr_spis; /* The number of SPIs for MSIs */
> + unsigned long *bm; /* MSI vector bitmap */
> + struct gic_chip_data *gic;
> +#endif
> +};
> +
> +struct gic_chip_data {
> + union gic_base dist_base;
> + union gic_base cpu_base;
> +#ifdef CONFIG_CPU_PM
> + u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> + u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> + u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> + u32 __percpu *saved_ppi_enable;
> + u32 __percpu *saved_ppi_conf;
> +#endif
> + struct irq_domain *domain;
> + unsigned int gic_irqs;
> +#ifdef CONFIG_GIC_NON_BANKED
> + void __iomem *(*get_base)(union gic_base *);
> +#endif
> + struct list_head v2m_list;
> +};
Maybe wrap this list_head around #ifdef/#endif?
> +
> +void gic_mask_irq(struct irq_data *d);
> +void gic_unmask_irq(struct irq_data *d);
> +int gicv2m_of_init(struct device_node *node, struct gic_chip_data *gic,
> + struct irq_chip *v2m_chip) __init;
> +
> +#endif /* _IRQ_GIC_H_ */
> --
> 1.9.3
>
>
--
Jazz is not dead. It just smells funny.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 1/2 V6] irqchip: gic: Add supports for ARM GICv2m MSI(-X)
@ 2014-09-15 19:26 ` Marc Zyngier
0 siblings, 0 replies; 17+ messages in thread
From: Marc Zyngier @ 2014-09-15 19:26 UTC (permalink / raw)
To: suravee.suthikulpanit
Cc: Mark Rutland, jason, Pawel Moll, Catalin Marinas, Will Deacon,
tglx, Harish.Kasiviswanathan, linux-arm-kernel, linux-pci,
linux-kernel, linux-doc, devicetree
On Sun, Sep 14 2014 at 07:30:15 AM, "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com> wrote:
Hi Suravee,
> From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>
> ARM GICv2m specification extends GICv2 to support MSI(-X) with
> a new set of register frame. This patch introduces support for
> the non-secure GICv2m register frame. Currently, GICV2m is available
> in certain version of GIC-400.
>
> The patch introduces a new property in ARM gic binding, the v2m subnode.
> It is optional.
Overall, this looks much better than the previous versions I've
reviewed. There is still a number of points I'd like to see addressed,
see below.
>
> Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> Cc: Mark Rutland <Mark.Rutland@arm.com>
> Cc: Marc Zyngier <Marc.Zyngier@arm.com>
> Cc: Jason Cooper <jason@lakedaemon.net>
> Cc: Catalin Marinas <Catalin.Marinas@arm.com>
> Cc: Will Deacon <Will.Deacon@arm.com>
> ---
> Documentation/devicetree/bindings/arm/gic.txt | 39 ++++
> drivers/irqchip/Kconfig | 7 +
> drivers/irqchip/Makefile | 1 +
> drivers/irqchip/irq-gic-v2m.c | 261 ++++++++++++++++++++++++++
> drivers/irqchip/irq-gic.c | 88 ++++++---
> drivers/irqchip/irq-gic.h | 51 +++++
> 6 files changed, 419 insertions(+), 28 deletions(-)
> create mode 100644 drivers/irqchip/irq-gic-v2m.c
> create mode 100644 drivers/irqchip/irq-gic.h
>
> diff --git a/Documentation/devicetree/bindings/arm/gic.txt b/Documentation/devicetree/bindings/arm/gic.txt
> index c7d2fa1..a461591 100644
> --- a/Documentation/devicetree/bindings/arm/gic.txt
> +++ b/Documentation/devicetree/bindings/arm/gic.txt
> @@ -96,3 +96,42 @@ Example:
> <0x2c006000 0x2000>;
> interrupts = <1 9 0xf04>;
> };
> +
> +
> +* GICv2m extension for MSI/MSI-x support (Optional)
> +
> +Certain revision of GIC-400 supports MSI/MSI-x via V2M register frame.
> +This is enabled by specifying v2m sub-node.
> +
> +Required properties:
> +
> +- msi-controller : Identifies the node as an MSI controller.
> +
> +- reg : GICv2m MSI interface register base and size
> +
> +Example:
> +
> + interrupt-controller@e1101000 {
> + compatible = "arm,gic-400";
> + #interrupt-cells = <3>;
> + #address-cells = <2>;
> + #size-cells = <2>;
> + interrupt-controller;
> + interrupts = <1 8 0xf04>;
> + ranges = <0 0 0 0xe1100000 0 0x100000>;
> + reg = <0x0 0xe1110000 0 0x01000>,
> + <0x0 0xe112f000 0 0x02000>,
> + <0x0 0xe1140000 0 0x10000>,
> + <0x0 0xe1160000 0 0x10000>;
> + v2m0 {
I believe the convention is to have
v2m0 : v2m@0x80000 {
> + msi-controller;
> + reg = <0x0 0x80000 0 0x1000>;
I think having a "compatible" property would be nice, and would help
handle variants if they ever happen.
> + };
> +
> + ....
> +
> + v2mN {
> + msi-controller;
> + reg = <0x0 0x90000 0 0x1000>;
> + };
> + };
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index b8632bf..ddb31ee 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -7,6 +7,13 @@ config ARM_GIC
> select IRQ_DOMAIN
> select MULTI_IRQ_HANDLER
>
> +config ARM_GIC_V2M
> + bool
> + select IRQ_DOMAIN
> + select MULTI_IRQ_HANDLER
> + depends on ARM_GIC
> + depends on PCI && PCI_MSI
> +
> config GIC_NON_BANKED
> bool
>
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index 73052ba..3bda951 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -17,6 +17,7 @@ obj-$(CONFIG_ARCH_SUNXI) += irq-sun4i.o
> obj-$(CONFIG_ARCH_SUNXI) += irq-sunxi-nmi.o
> obj-$(CONFIG_ARCH_SPEAR3XX) += spear-shirq.o
> obj-$(CONFIG_ARM_GIC) += irq-gic.o irq-gic-common.o
> +obj-$(CONFIG_ARM_GIC_V2M) += irq-gic-v2m.o
> obj-$(CONFIG_ARM_GIC_V3) += irq-gic-v3.o irq-gic-common.o
> obj-$(CONFIG_ARM_NVIC) += irq-nvic.o
> obj-$(CONFIG_ARM_VIC) += irq-vic.o
> diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
> new file mode 100644
> index 0000000..499bfb9
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic-v2m.c
> @@ -0,0 +1,261 @@
> +/*
> + * ARM GIC v2m MSI(-X) support
> + * Support for Message Signaled Interrupts for systems that
> + * implement ARM Generic Interrupt Controller: GICv2m.
> + *
> + * Copyright (C) 2014 Advanced Micro Devices, Inc.
> + * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> + * Harish Kasiviswanathan <harish.kasiviswanathan@amd.com>
> + * Brandon Anderson <brandon.anderson@amd.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published
> + * by the Free Software Foundation.
> + */
> +
> +#define pr_fmt(fmt) "GICv2m: " fmt
> +
> +#include <linux/bitmap.h>
> +#include <linux/irq.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_pci.h>
> +#include <linux/pci.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +#include <linux/spinlock.h>
> +
> +#include "irqchip.h"
> +#include "irq-gic.h"
> +
> +/*
> +* MSI_TYPER:
> +* [31:26] Reserved
> +* [25:16] lowest SPI assigned to MSI
> +* [15:10] Reserved
> +* [9:0] Numer of SPIs assigned to MSI
> +*/
> +#define V2M_MSI_TYPER 0x008
> +#define V2M_MSI_TYPER_BASE_SHIFT 16
> +#define V2M_MSI_TYPER_BASE_MASK 0x3FF
> +#define V2M_MSI_TYPER_NUM_MASK 0x3FF
> +#define V2M_MSI_SETSPI_NS 0x040
> +#define V2M_MIN_SPI 32
> +#define V2M_MAX_SPI 1019
> +
> +/*
> + * alloc_msi_irq - Allocate MSIs from available MSI bitmap.
> + * @data: Pointer to v2m_data
> + * @nvec: Number of interrupts to allocate
> + * @irq: Pointer to the allocated irq
> + *
> + * Allocates interrupts only if the contiguous range of MSIs
> + * with specified nvec are available. Otherwise return the number
> + * of available interrupts. If none are available, then returns -ENOENT.
> + */
> +static int alloc_msi_irq(struct v2m_data *data, int nvec, int *irq)
> +{
> + int size = data->nr_spis;
> + int next = size, i = nvec, ret;
> +
> + /* We should never allocate more than available nr_spis */
> + if (i >= size)
> + i = size;
> +
> + spin_lock(&data->msi_cnt_lock);
> +
> + for (; i > 0; i--) {
> + next = bitmap_find_next_zero_area(data->bm,
> + size, 0, i, 0);
> + if (next < size)
> + break;
> + }
> +
> + if (i != nvec) {
> + ret = i ? : -ENOENT;
> + } else {
> + bitmap_set(data->bm, next, nvec);
> + *irq = data->spi_start + next;
> + ret = 0;
> + }
> +
> + spin_unlock(&data->msi_cnt_lock);
> +
> + return ret;
> +}
> +
> +static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
> +{
> + int pos;
> + struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> + spin_lock(&data->msi_cnt_lock);
> +
> + pos = irq - data->spi_start;
> + if (pos >= 0 && pos < data->nr_spis)
> + bitmap_clear(data->bm, pos, 1);
> +
> + spin_unlock(&data->msi_cnt_lock);
> +}
> +
> +static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> + struct pci_dev *pdev,
> + struct msi_desc *desc)
> +{
> + int irq, avail;
> + struct msi_msg msg;
> + phys_addr_t addr;
> + struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> + if (!desc) {
> + dev_err(&pdev->dev,
> + "GICv2m: MSI setup failed. Invalid msi descriptor\n");
Isn't the "GICv2m" header already provided by the pr_fmt at the
beginning of the file?
> + return -EINVAL;
> + }
> +
> + avail = alloc_msi_irq(data, 1, &irq);
> + if (avail != 0) {
> + dev_err(&pdev->dev,
> + "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> + return -ENOSPC;
> + }
> +
> + irq_set_chip_data(irq, chip);
> + irq_set_msi_desc(irq, desc);
> + irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> +
> + addr = data->res.start + V2M_MSI_SETSPI_NS;
> + msg.address_hi = (u32)(addr >> 32);
> + msg.address_lo = (u32)(addr);
> + msg.data = irq;
> + write_msi_msg(irq, &msg);
> +
> + return 0;
> +}
> +
> +static void gicv2m_mask_irq(struct irq_data *d)
> +{
> + gic_mask_irq(d);
> + if (d->msi_desc)
> + mask_msi_irq(d);
> +}
> +
> +static void gicv2m_unmask_irq(struct irq_data *d)
> +{
> + gic_unmask_irq(d);
> + if (d->msi_desc)
> + unmask_msi_irq(d);
> +}
> +
> +#ifdef CONFIG_OF
I think you can loose the CONFIG_OF here. There is no valid
configuration without OF on arm64.
> +
> +static int __init
> +v2m_init(struct device_node *node, struct v2m_data *v2m,
> + struct resource *res)
> +{
> + int ret;
> + unsigned int typer;
> +
> + v2m->msi_chip.owner = THIS_MODULE;
> + v2m->msi_chip.of_node = node;
> + v2m->msi_chip.setup_irq = gicv2m_setup_msi_irq;
> + v2m->msi_chip.teardown_irq = gicv2m_teardown_msi_irq;
> + ret = of_pci_msi_chip_add(&v2m->msi_chip);
> + if (ret) {
> + pr_info("Failed to add msi_chip.\n");
> + return ret;
> + }
> +
> + v2m->base = ioremap(res->start, resource_size(res));
> + if (!v2m->base) {
> + pr_err("Failed to map GICv2m resource\n");
> + return -EINVAL;
Beware, you're leaving with a registered msi_chip. You probably want to
remove it on the error path.
> + }
> +
> + memcpy(&v2m->res, res, sizeof(struct resource));
> +
> + typer = readl_relaxed(v2m->base + V2M_MSI_TYPER);
> + if (!typer) {
> + pr_warn("Failed to read V2M_MSI_TYPER register\n");
> + return -EINVAL;
And here leaving the v2m widget mapped.
> + }
> +
> + v2m->spi_start = (typer >> V2M_MSI_TYPER_BASE_SHIFT) &
> + V2M_MSI_TYPER_BASE_MASK;
> + v2m->nr_spis = typer & V2M_MSI_TYPER_NUM_MASK;
> + if ((v2m->spi_start < V2M_MIN_SPI) || (v2m->nr_spis >= V2M_MAX_SPI)) {
> + pr_err("Invalid MSI_TYPER (%#x)\n", typer);
> + return -EINVAL;
Blah. The whole error handling of this function is b0rked.
> + }
> +
> + v2m->bm = kzalloc(sizeof(long) * BITS_TO_LONGS(v2m->nr_spis),
> + GFP_KERNEL);
> + if (!v2m->bm) {
> + pr_err("Failed to allocate MSI bitmap\n");
> + return -ENOMEM;
> + }
> +
> + spin_lock_init(&v2m->msi_cnt_lock);
> +
> + pr_info("SPI range [%d:%d]\n",
> + v2m->spi_start, (v2m->spi_start + v2m->nr_spis));
> +
> + return 0;
> +}
> +
> +int __init gicv2m_of_init(struct device_node *node,
> + struct gic_chip_data *gic,
> + struct irq_chip *v2m_chip)
> +{
> + int ret = 0;
> + struct v2m_data *v2m;
> + struct resource res;
> + struct device_node *child = NULL;
> +
> + INIT_LIST_HEAD(&gic->v2m_list);
> +
> + for (;;) {
> + child = of_get_next_child(node, child);
I don't see any of_node_put() in this function. You'll need to fix
this...
> + if (!child)
> + break;
> +
> + if (!of_find_property(child, "msi-controller", NULL))
> + continue;
> +
> + ret = of_address_to_resource(child, 0, &res);
> + if (ret) {
> + pr_err("Failed to allocate v2m resource.\n");
> + break;
> + }
> +
> + v2m = kzalloc(sizeof(struct v2m_data), GFP_KERNEL);
> + if (!v2m) {
> + pr_err("Failed to allocate struct v2m_data.\n");
> + return -ENOMEM;
> + }
> +
> + ret = v2m_init(child, v2m, &res);
> + if (ret)
> + break;
> +
> + v2m->gic = gic;
> +
> + pr_info(" %s: [%lx : %lx]\n", child->name,
> + (unsigned long)res.start, (unsigned long)res.end);
> +
> + list_add_tail(&v2m->list, &gic->v2m_list);
> + }
> +
> + if (ret && list_empty(&gic->v2m_list)) {
> + pr_err("MSI resource not found\n");
> + return -EINVAL;
> + }
> +
> + v2m_chip->irq_mask = gicv2m_mask_irq;
> + v2m_chip->irq_unmask = gicv2m_unmask_irq;
> +
Again, the whole error handling is evil. You can end-up allocating a few
v2m structures, registering them, and yet have kzalloc failing at some
point. At that stage, you exit early, not setting the mas/unmask
methods. Boom.
> + return ret;
> +}
> +
> +#endif /* CONFIG_OF */
> diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
> index 4b959e6..f490599 100644
> --- a/drivers/irqchip/irq-gic.c
> +++ b/drivers/irqchip/irq-gic.c
> @@ -46,30 +46,9 @@
> #include <asm/smp_plat.h>
>
> #include "irq-gic-common.h"
> +#include "irq-gic.h"
> #include "irqchip.h"
>
> -union gic_base {
> - void __iomem *common_base;
> - void __percpu * __iomem *percpu_base;
> -};
> -
> -struct gic_chip_data {
> - union gic_base dist_base;
> - union gic_base cpu_base;
> -#ifdef CONFIG_CPU_PM
> - u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> - u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> - u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> - u32 __percpu *saved_ppi_enable;
> - u32 __percpu *saved_ppi_conf;
> -#endif
> - struct irq_domain *domain;
> - unsigned int gic_irqs;
> -#ifdef CONFIG_GIC_NON_BANKED
> - void __iomem *(*get_base)(union gic_base *);
> -#endif
> -};
> -
> static DEFINE_RAW_SPINLOCK(irq_controller_lock);
>
> /*
> @@ -131,15 +110,36 @@ static inline void gic_set_base_accessor(struct gic_chip_data *data,
> #define gic_set_base_accessor(d, f)
> #endif
>
> +static inline
> +struct gic_chip_data *irq_data_get_gic_chip_data(struct irq_data *d)
> +{
> + struct gic_chip_data *gic_data;
> + struct msi_chip *mchip;
> + struct v2m_data *v2mdat;
> +
> + /*
> + * For MSI, irq_data.chip_data points to struct msi_chip.
> + * For non-MSI, irq_data.chip_data points to struct gic_chip_data.
> + */
> + if (d->msi_desc) {
> + mchip = irq_data_get_irq_chip_data(d);
> + v2mdat = container_of(mchip, struct v2m_data, msi_chip);
> + gic_data = v2mdat->gic;
> + } else {
> + gic_data = irq_data_get_irq_chip_data(d);
> + }
> + return gic_data;
> +}
> +
> static inline void __iomem *gic_dist_base(struct irq_data *d)
> {
> - struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> + struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
> return gic_data_dist_base(gic_data);
> }
>
> static inline void __iomem *gic_cpu_base(struct irq_data *d)
> {
> - struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> + struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
> return gic_data_cpu_base(gic_data);
> }
>
> @@ -151,7 +151,7 @@ static inline unsigned int gic_irq(struct irq_data *d)
> /*
> * Routines to acknowledge, disable and enable interrupts
> */
> -static void gic_mask_irq(struct irq_data *d)
> +void gic_mask_irq(struct irq_data *d)
I'm still very uncomfortable with exporting the mask/unmask methods
outside of this file... Maybe it is not a big deal, but I see evil
everywhere.
> {
> u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -162,7 +162,7 @@ static void gic_mask_irq(struct irq_data *d)
> raw_spin_unlock(&irq_controller_lock);
> }
>
> -static void gic_unmask_irq(struct irq_data *d)
> +void gic_unmask_irq(struct irq_data *d)
> {
> u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -325,6 +325,15 @@ static struct irq_chip gic_chip = {
> .irq_set_wake = gic_set_wake,
> };
>
> +static struct irq_chip v2m_chip = {
> + .name = "GICv2m",
> + .irq_eoi = gic_eoi_irq,
> + .irq_set_type = gic_set_type,
> +#ifdef CONFIG_SMP
> + .irq_set_affinity = gic_set_affinity,
> +#endif
> +};
> +
> void __init gic_cascade_irq(unsigned int gic_nr, unsigned int irq)
> {
> if (gic_nr >= MAX_GIC_NR)
> @@ -767,6 +776,24 @@ void __init gic_init_physaddr(struct device_node *node)
> static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
> irq_hw_number_t hw)
> {
> + struct gic_chip_data *gic = d->host_data;
> + struct v2m_data *v2m;
> +
> + irq_set_chip_data(irq, gic);
>
> +#ifdef CONFIG_ARM_GIC_V2M
> + list_for_each_entry(v2m, &gic->v2m_list, list) {
> + if (hw >= v2m->spi_start &&
> + hw < v2m->spi_start + v2m->nr_spis) {
> +
> + irq_set_chip_and_handler(irq, &v2m_chip,
> + handle_fasteoi_irq);
> + set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
> + return 0;
> + }
> + }
> +#endif
> +
> if (hw < 32) {
> irq_set_percpu_devid(irq);
> irq_set_chip_and_handler(irq, &gic_chip,
> @@ -779,7 +806,6 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
>
> gic_routable_irq_domain_ops->map(d, irq, hw);
> }
> - irq_set_chip_data(irq, d->host_data);
These two hunks make the code a bit hard to read:
- First, message-based SPIs,
- Then PPIs
- and finally the rest of the SPIs.
Can you come up with a construct that keeps everything in a logical
order (PPIs, and then SPIs)? Not a big deal but it would make the code
nicer.
> return 0;
> }
>
> @@ -996,7 +1022,7 @@ gic_of_init(struct device_node *node, struct device_node *parent)
> void __iomem *cpu_base;
> void __iomem *dist_base;
> u32 percpu_offset;
> - int irq;
> + int irq, ret;
>
> if (WARN_ON(!node))
> return -ENODEV;
> @@ -1010,6 +1036,12 @@ gic_of_init(struct device_node *node, struct device_node *parent)
> if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
> percpu_offset = 0;
>
> + if (IS_ENABLED(CONFIG_ARM_GIC_V2M)) {
> + ret = gicv2m_of_init(node, &gic_data[gic_cnt], &v2m_chip);
> + if (ret)
> + pr_warn("Warning: Failed to enable GICv2m support.\n");
> + }
> +
> gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset, node);
> if (!gic_cnt)
> gic_init_physaddr(node);
> diff --git a/drivers/irqchip/irq-gic.h b/drivers/irqchip/irq-gic.h
> new file mode 100644
> index 0000000..1f4a2b0
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic.h
> @@ -0,0 +1,51 @@
> +#ifndef _IRQ_GIC_H_
> +#define _IRQ_GIC_H_
> +
> +#include <linux/msi.h>
> +#include <linux/pci.h>
> +
> +union gic_base {
> + void __iomem *common_base;
> + void __percpu * __iomem *percpu_base;
> +};
> +
> +struct gic_chip_data;
> +
> +struct v2m_data {
> +#ifdef CONFIG_ARM_GIC_V2M
> + struct list_head list;
> + spinlock_t msi_cnt_lock;
> + struct msi_chip msi_chip;
> + struct resource res; /* GICv2m resource */
> + void __iomem *base; /* GICv2m virt address */
> + unsigned int spi_start; /* The SPI number that MSIs start */
> + unsigned int nr_spis; /* The number of SPIs for MSIs */
> + unsigned long *bm; /* MSI vector bitmap */
> + struct gic_chip_data *gic;
> +#endif
> +};
> +
> +struct gic_chip_data {
> + union gic_base dist_base;
> + union gic_base cpu_base;
> +#ifdef CONFIG_CPU_PM
> + u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> + u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> + u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> + u32 __percpu *saved_ppi_enable;
> + u32 __percpu *saved_ppi_conf;
> +#endif
> + struct irq_domain *domain;
> + unsigned int gic_irqs;
> +#ifdef CONFIG_GIC_NON_BANKED
> + void __iomem *(*get_base)(union gic_base *);
> +#endif
> + struct list_head v2m_list;
> +};
Maybe wrap this list_head around #ifdef/#endif?
> +
> +void gic_mask_irq(struct irq_data *d);
> +void gic_unmask_irq(struct irq_data *d);
> +int gicv2m_of_init(struct device_node *node, struct gic_chip_data *gic,
> + struct irq_chip *v2m_chip) __init;
> +
> +#endif /* _IRQ_GIC_H_ */
> --
> 1.9.3
>
>
--
Jazz is not dead. It just smells funny.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 1/2 V6] irqchip: gic: Add supports for ARM GICv2m MSI(-X)
@ 2014-09-15 19:26 ` Marc Zyngier
0 siblings, 0 replies; 17+ messages in thread
From: Marc Zyngier @ 2014-09-15 19:26 UTC (permalink / raw)
To: suravee.suthikulpanit
Cc: Mark Rutland, jason, Pawel Moll, Catalin Marinas, Will Deacon,
tglx, Harish.Kasiviswanathan, linux-arm-kernel, linux-pci,
linux-kernel, linux-doc, devicetree
On Sun, Sep 14 2014 at 07:30:15 AM, "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com> wrote:
Hi Suravee,
> From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>
> ARM GICv2m specification extends GICv2 to support MSI(-X) with
> a new set of register frame. This patch introduces support for
> the non-secure GICv2m register frame. Currently, GICV2m is available
> in certain version of GIC-400.
>
> The patch introduces a new property in ARM gic binding, the v2m subnode.
> It is optional.
Overall, this looks much better than the previous versions I've
reviewed. There is still a number of points I'd like to see addressed,
see below.
>
> Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> Cc: Mark Rutland <Mark.Rutland@arm.com>
> Cc: Marc Zyngier <Marc.Zyngier@arm.com>
> Cc: Jason Cooper <jason@lakedaemon.net>
> Cc: Catalin Marinas <Catalin.Marinas@arm.com>
> Cc: Will Deacon <Will.Deacon@arm.com>
> ---
> Documentation/devicetree/bindings/arm/gic.txt | 39 ++++
> drivers/irqchip/Kconfig | 7 +
> drivers/irqchip/Makefile | 1 +
> drivers/irqchip/irq-gic-v2m.c | 261 ++++++++++++++++++++++++++
> drivers/irqchip/irq-gic.c | 88 ++++++---
> drivers/irqchip/irq-gic.h | 51 +++++
> 6 files changed, 419 insertions(+), 28 deletions(-)
> create mode 100644 drivers/irqchip/irq-gic-v2m.c
> create mode 100644 drivers/irqchip/irq-gic.h
>
> diff --git a/Documentation/devicetree/bindings/arm/gic.txt b/Documentation/devicetree/bindings/arm/gic.txt
> index c7d2fa1..a461591 100644
> --- a/Documentation/devicetree/bindings/arm/gic.txt
> +++ b/Documentation/devicetree/bindings/arm/gic.txt
> @@ -96,3 +96,42 @@ Example:
> <0x2c006000 0x2000>;
> interrupts = <1 9 0xf04>;
> };
> +
> +
> +* GICv2m extension for MSI/MSI-x support (Optional)
> +
> +Certain revision of GIC-400 supports MSI/MSI-x via V2M register frame.
> +This is enabled by specifying v2m sub-node.
> +
> +Required properties:
> +
> +- msi-controller : Identifies the node as an MSI controller.
> +
> +- reg : GICv2m MSI interface register base and size
> +
> +Example:
> +
> + interrupt-controller@e1101000 {
> + compatible = "arm,gic-400";
> + #interrupt-cells = <3>;
> + #address-cells = <2>;
> + #size-cells = <2>;
> + interrupt-controller;
> + interrupts = <1 8 0xf04>;
> + ranges = <0 0 0 0xe1100000 0 0x100000>;
> + reg = <0x0 0xe1110000 0 0x01000>,
> + <0x0 0xe112f000 0 0x02000>,
> + <0x0 0xe1140000 0 0x10000>,
> + <0x0 0xe1160000 0 0x10000>;
> + v2m0 {
I believe the convention is to have
v2m0 : v2m@0x80000 {
> + msi-controller;
> + reg = <0x0 0x80000 0 0x1000>;
I think having a "compatible" property would be nice, and would help
handle variants if they ever happen.
> + };
> +
> + ....
> +
> + v2mN {
> + msi-controller;
> + reg = <0x0 0x90000 0 0x1000>;
> + };
> + };
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index b8632bf..ddb31ee 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -7,6 +7,13 @@ config ARM_GIC
> select IRQ_DOMAIN
> select MULTI_IRQ_HANDLER
>
> +config ARM_GIC_V2M
> + bool
> + select IRQ_DOMAIN
> + select MULTI_IRQ_HANDLER
> + depends on ARM_GIC
> + depends on PCI && PCI_MSI
> +
> config GIC_NON_BANKED
> bool
>
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index 73052ba..3bda951 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -17,6 +17,7 @@ obj-$(CONFIG_ARCH_SUNXI) += irq-sun4i.o
> obj-$(CONFIG_ARCH_SUNXI) += irq-sunxi-nmi.o
> obj-$(CONFIG_ARCH_SPEAR3XX) += spear-shirq.o
> obj-$(CONFIG_ARM_GIC) += irq-gic.o irq-gic-common.o
> +obj-$(CONFIG_ARM_GIC_V2M) += irq-gic-v2m.o
> obj-$(CONFIG_ARM_GIC_V3) += irq-gic-v3.o irq-gic-common.o
> obj-$(CONFIG_ARM_NVIC) += irq-nvic.o
> obj-$(CONFIG_ARM_VIC) += irq-vic.o
> diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
> new file mode 100644
> index 0000000..499bfb9
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic-v2m.c
> @@ -0,0 +1,261 @@
> +/*
> + * ARM GIC v2m MSI(-X) support
> + * Support for Message Signaled Interrupts for systems that
> + * implement ARM Generic Interrupt Controller: GICv2m.
> + *
> + * Copyright (C) 2014 Advanced Micro Devices, Inc.
> + * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> + * Harish Kasiviswanathan <harish.kasiviswanathan@amd.com>
> + * Brandon Anderson <brandon.anderson@amd.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published
> + * by the Free Software Foundation.
> + */
> +
> +#define pr_fmt(fmt) "GICv2m: " fmt
> +
> +#include <linux/bitmap.h>
> +#include <linux/irq.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_pci.h>
> +#include <linux/pci.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +#include <linux/spinlock.h>
> +
> +#include "irqchip.h"
> +#include "irq-gic.h"
> +
> +/*
> +* MSI_TYPER:
> +* [31:26] Reserved
> +* [25:16] lowest SPI assigned to MSI
> +* [15:10] Reserved
> +* [9:0] Numer of SPIs assigned to MSI
> +*/
> +#define V2M_MSI_TYPER 0x008
> +#define V2M_MSI_TYPER_BASE_SHIFT 16
> +#define V2M_MSI_TYPER_BASE_MASK 0x3FF
> +#define V2M_MSI_TYPER_NUM_MASK 0x3FF
> +#define V2M_MSI_SETSPI_NS 0x040
> +#define V2M_MIN_SPI 32
> +#define V2M_MAX_SPI 1019
> +
> +/*
> + * alloc_msi_irq - Allocate MSIs from available MSI bitmap.
> + * @data: Pointer to v2m_data
> + * @nvec: Number of interrupts to allocate
> + * @irq: Pointer to the allocated irq
> + *
> + * Allocates interrupts only if the contiguous range of MSIs
> + * with specified nvec are available. Otherwise return the number
> + * of available interrupts. If none are available, then returns -ENOENT.
> + */
> +static int alloc_msi_irq(struct v2m_data *data, int nvec, int *irq)
> +{
> + int size = data->nr_spis;
> + int next = size, i = nvec, ret;
> +
> + /* We should never allocate more than available nr_spis */
> + if (i >= size)
> + i = size;
> +
> + spin_lock(&data->msi_cnt_lock);
> +
> + for (; i > 0; i--) {
> + next = bitmap_find_next_zero_area(data->bm,
> + size, 0, i, 0);
> + if (next < size)
> + break;
> + }
> +
> + if (i != nvec) {
> + ret = i ? : -ENOENT;
> + } else {
> + bitmap_set(data->bm, next, nvec);
> + *irq = data->spi_start + next;
> + ret = 0;
> + }
> +
> + spin_unlock(&data->msi_cnt_lock);
> +
> + return ret;
> +}
> +
> +static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
> +{
> + int pos;
> + struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> + spin_lock(&data->msi_cnt_lock);
> +
> + pos = irq - data->spi_start;
> + if (pos >= 0 && pos < data->nr_spis)
> + bitmap_clear(data->bm, pos, 1);
> +
> + spin_unlock(&data->msi_cnt_lock);
> +}
> +
> +static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> + struct pci_dev *pdev,
> + struct msi_desc *desc)
> +{
> + int irq, avail;
> + struct msi_msg msg;
> + phys_addr_t addr;
> + struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> + if (!desc) {
> + dev_err(&pdev->dev,
> + "GICv2m: MSI setup failed. Invalid msi descriptor\n");
Isn't the "GICv2m" header already provided by the pr_fmt at the
beginning of the file?
> + return -EINVAL;
> + }
> +
> + avail = alloc_msi_irq(data, 1, &irq);
> + if (avail != 0) {
> + dev_err(&pdev->dev,
> + "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> + return -ENOSPC;
> + }
> +
> + irq_set_chip_data(irq, chip);
> + irq_set_msi_desc(irq, desc);
> + irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> +
> + addr = data->res.start + V2M_MSI_SETSPI_NS;
> + msg.address_hi = (u32)(addr >> 32);
> + msg.address_lo = (u32)(addr);
> + msg.data = irq;
> + write_msi_msg(irq, &msg);
> +
> + return 0;
> +}
> +
> +static void gicv2m_mask_irq(struct irq_data *d)
> +{
> + gic_mask_irq(d);
> + if (d->msi_desc)
> + mask_msi_irq(d);
> +}
> +
> +static void gicv2m_unmask_irq(struct irq_data *d)
> +{
> + gic_unmask_irq(d);
> + if (d->msi_desc)
> + unmask_msi_irq(d);
> +}
> +
> +#ifdef CONFIG_OF
I think you can loose the CONFIG_OF here. There is no valid
configuration without OF on arm64.
> +
> +static int __init
> +v2m_init(struct device_node *node, struct v2m_data *v2m,
> + struct resource *res)
> +{
> + int ret;
> + unsigned int typer;
> +
> + v2m->msi_chip.owner = THIS_MODULE;
> + v2m->msi_chip.of_node = node;
> + v2m->msi_chip.setup_irq = gicv2m_setup_msi_irq;
> + v2m->msi_chip.teardown_irq = gicv2m_teardown_msi_irq;
> + ret = of_pci_msi_chip_add(&v2m->msi_chip);
> + if (ret) {
> + pr_info("Failed to add msi_chip.\n");
> + return ret;
> + }
> +
> + v2m->base = ioremap(res->start, resource_size(res));
> + if (!v2m->base) {
> + pr_err("Failed to map GICv2m resource\n");
> + return -EINVAL;
Beware, you're leaving with a registered msi_chip. You probably want to
remove it on the error path.
> + }
> +
> + memcpy(&v2m->res, res, sizeof(struct resource));
> +
> + typer = readl_relaxed(v2m->base + V2M_MSI_TYPER);
> + if (!typer) {
> + pr_warn("Failed to read V2M_MSI_TYPER register\n");
> + return -EINVAL;
And here leaving the v2m widget mapped.
> + }
> +
> + v2m->spi_start = (typer >> V2M_MSI_TYPER_BASE_SHIFT) &
> + V2M_MSI_TYPER_BASE_MASK;
> + v2m->nr_spis = typer & V2M_MSI_TYPER_NUM_MASK;
> + if ((v2m->spi_start < V2M_MIN_SPI) || (v2m->nr_spis >= V2M_MAX_SPI)) {
> + pr_err("Invalid MSI_TYPER (%#x)\n", typer);
> + return -EINVAL;
Blah. The whole error handling of this function is b0rked.
> + }
> +
> + v2m->bm = kzalloc(sizeof(long) * BITS_TO_LONGS(v2m->nr_spis),
> + GFP_KERNEL);
> + if (!v2m->bm) {
> + pr_err("Failed to allocate MSI bitmap\n");
> + return -ENOMEM;
> + }
> +
> + spin_lock_init(&v2m->msi_cnt_lock);
> +
> + pr_info("SPI range [%d:%d]\n",
> + v2m->spi_start, (v2m->spi_start + v2m->nr_spis));
> +
> + return 0;
> +}
> +
> +int __init gicv2m_of_init(struct device_node *node,
> + struct gic_chip_data *gic,
> + struct irq_chip *v2m_chip)
> +{
> + int ret = 0;
> + struct v2m_data *v2m;
> + struct resource res;
> + struct device_node *child = NULL;
> +
> + INIT_LIST_HEAD(&gic->v2m_list);
> +
> + for (;;) {
> + child = of_get_next_child(node, child);
I don't see any of_node_put() in this function. You'll need to fix
this...
> + if (!child)
> + break;
> +
> + if (!of_find_property(child, "msi-controller", NULL))
> + continue;
> +
> + ret = of_address_to_resource(child, 0, &res);
> + if (ret) {
> + pr_err("Failed to allocate v2m resource.\n");
> + break;
> + }
> +
> + v2m = kzalloc(sizeof(struct v2m_data), GFP_KERNEL);
> + if (!v2m) {
> + pr_err("Failed to allocate struct v2m_data.\n");
> + return -ENOMEM;
> + }
> +
> + ret = v2m_init(child, v2m, &res);
> + if (ret)
> + break;
> +
> + v2m->gic = gic;
> +
> + pr_info(" %s: [%lx : %lx]\n", child->name,
> + (unsigned long)res.start, (unsigned long)res.end);
> +
> + list_add_tail(&v2m->list, &gic->v2m_list);
> + }
> +
> + if (ret && list_empty(&gic->v2m_list)) {
> + pr_err("MSI resource not found\n");
> + return -EINVAL;
> + }
> +
> + v2m_chip->irq_mask = gicv2m_mask_irq;
> + v2m_chip->irq_unmask = gicv2m_unmask_irq;
> +
Again, the whole error handling is evil. You can end-up allocating a few
v2m structures, registering them, and yet have kzalloc failing at some
point. At that stage, you exit early, not setting the mas/unmask
methods. Boom.
> + return ret;
> +}
> +
> +#endif /* CONFIG_OF */
> diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
> index 4b959e6..f490599 100644
> --- a/drivers/irqchip/irq-gic.c
> +++ b/drivers/irqchip/irq-gic.c
> @@ -46,30 +46,9 @@
> #include <asm/smp_plat.h>
>
> #include "irq-gic-common.h"
> +#include "irq-gic.h"
> #include "irqchip.h"
>
> -union gic_base {
> - void __iomem *common_base;
> - void __percpu * __iomem *percpu_base;
> -};
> -
> -struct gic_chip_data {
> - union gic_base dist_base;
> - union gic_base cpu_base;
> -#ifdef CONFIG_CPU_PM
> - u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> - u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> - u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> - u32 __percpu *saved_ppi_enable;
> - u32 __percpu *saved_ppi_conf;
> -#endif
> - struct irq_domain *domain;
> - unsigned int gic_irqs;
> -#ifdef CONFIG_GIC_NON_BANKED
> - void __iomem *(*get_base)(union gic_base *);
> -#endif
> -};
> -
> static DEFINE_RAW_SPINLOCK(irq_controller_lock);
>
> /*
> @@ -131,15 +110,36 @@ static inline void gic_set_base_accessor(struct gic_chip_data *data,
> #define gic_set_base_accessor(d, f)
> #endif
>
> +static inline
> +struct gic_chip_data *irq_data_get_gic_chip_data(struct irq_data *d)
> +{
> + struct gic_chip_data *gic_data;
> + struct msi_chip *mchip;
> + struct v2m_data *v2mdat;
> +
> + /*
> + * For MSI, irq_data.chip_data points to struct msi_chip.
> + * For non-MSI, irq_data.chip_data points to struct gic_chip_data.
> + */
> + if (d->msi_desc) {
> + mchip = irq_data_get_irq_chip_data(d);
> + v2mdat = container_of(mchip, struct v2m_data, msi_chip);
> + gic_data = v2mdat->gic;
> + } else {
> + gic_data = irq_data_get_irq_chip_data(d);
> + }
> + return gic_data;
> +}
> +
> static inline void __iomem *gic_dist_base(struct irq_data *d)
> {
> - struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> + struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
> return gic_data_dist_base(gic_data);
> }
>
> static inline void __iomem *gic_cpu_base(struct irq_data *d)
> {
> - struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> + struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
> return gic_data_cpu_base(gic_data);
> }
>
> @@ -151,7 +151,7 @@ static inline unsigned int gic_irq(struct irq_data *d)
> /*
> * Routines to acknowledge, disable and enable interrupts
> */
> -static void gic_mask_irq(struct irq_data *d)
> +void gic_mask_irq(struct irq_data *d)
I'm still very uncomfortable with exporting the mask/unmask methods
outside of this file... Maybe it is not a big deal, but I see evil
everywhere.
> {
> u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -162,7 +162,7 @@ static void gic_mask_irq(struct irq_data *d)
> raw_spin_unlock(&irq_controller_lock);
> }
>
> -static void gic_unmask_irq(struct irq_data *d)
> +void gic_unmask_irq(struct irq_data *d)
> {
> u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -325,6 +325,15 @@ static struct irq_chip gic_chip = {
> .irq_set_wake = gic_set_wake,
> };
>
> +static struct irq_chip v2m_chip = {
> + .name = "GICv2m",
> + .irq_eoi = gic_eoi_irq,
> + .irq_set_type = gic_set_type,
> +#ifdef CONFIG_SMP
> + .irq_set_affinity = gic_set_affinity,
> +#endif
> +};
> +
> void __init gic_cascade_irq(unsigned int gic_nr, unsigned int irq)
> {
> if (gic_nr >= MAX_GIC_NR)
> @@ -767,6 +776,24 @@ void __init gic_init_physaddr(struct device_node *node)
> static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
> irq_hw_number_t hw)
> {
> + struct gic_chip_data *gic = d->host_data;
> + struct v2m_data *v2m;
> +
> + irq_set_chip_data(irq, gic);
>
> +#ifdef CONFIG_ARM_GIC_V2M
> + list_for_each_entry(v2m, &gic->v2m_list, list) {
> + if (hw >= v2m->spi_start &&
> + hw < v2m->spi_start + v2m->nr_spis) {
> +
> + irq_set_chip_and_handler(irq, &v2m_chip,
> + handle_fasteoi_irq);
> + set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
> + return 0;
> + }
> + }
> +#endif
> +
> if (hw < 32) {
> irq_set_percpu_devid(irq);
> irq_set_chip_and_handler(irq, &gic_chip,
> @@ -779,7 +806,6 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
>
> gic_routable_irq_domain_ops->map(d, irq, hw);
> }
> - irq_set_chip_data(irq, d->host_data);
These two hunks make the code a bit hard to read:
- First, message-based SPIs,
- Then PPIs
- and finally the rest of the SPIs.
Can you come up with a construct that keeps everything in a logical
order (PPIs, and then SPIs)? Not a big deal but it would make the code
nicer.
> return 0;
> }
>
> @@ -996,7 +1022,7 @@ gic_of_init(struct device_node *node, struct device_node *parent)
> void __iomem *cpu_base;
> void __iomem *dist_base;
> u32 percpu_offset;
> - int irq;
> + int irq, ret;
>
> if (WARN_ON(!node))
> return -ENODEV;
> @@ -1010,6 +1036,12 @@ gic_of_init(struct device_node *node, struct device_node *parent)
> if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
> percpu_offset = 0;
>
> + if (IS_ENABLED(CONFIG_ARM_GIC_V2M)) {
> + ret = gicv2m_of_init(node, &gic_data[gic_cnt], &v2m_chip);
> + if (ret)
> + pr_warn("Warning: Failed to enable GICv2m support.\n");
> + }
> +
> gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset, node);
> if (!gic_cnt)
> gic_init_physaddr(node);
> diff --git a/drivers/irqchip/irq-gic.h b/drivers/irqchip/irq-gic.h
> new file mode 100644
> index 0000000..1f4a2b0
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic.h
> @@ -0,0 +1,51 @@
> +#ifndef _IRQ_GIC_H_
> +#define _IRQ_GIC_H_
> +
> +#include <linux/msi.h>
> +#include <linux/pci.h>
> +
> +union gic_base {
> + void __iomem *common_base;
> + void __percpu * __iomem *percpu_base;
> +};
> +
> +struct gic_chip_data;
> +
> +struct v2m_data {
> +#ifdef CONFIG_ARM_GIC_V2M
> + struct list_head list;
> + spinlock_t msi_cnt_lock;
> + struct msi_chip msi_chip;
> + struct resource res; /* GICv2m resource */
> + void __iomem *base; /* GICv2m virt address */
> + unsigned int spi_start; /* The SPI number that MSIs start */
> + unsigned int nr_spis; /* The number of SPIs for MSIs */
> + unsigned long *bm; /* MSI vector bitmap */
> + struct gic_chip_data *gic;
> +#endif
> +};
> +
> +struct gic_chip_data {
> + union gic_base dist_base;
> + union gic_base cpu_base;
> +#ifdef CONFIG_CPU_PM
> + u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> + u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> + u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> + u32 __percpu *saved_ppi_enable;
> + u32 __percpu *saved_ppi_conf;
> +#endif
> + struct irq_domain *domain;
> + unsigned int gic_irqs;
> +#ifdef CONFIG_GIC_NON_BANKED
> + void __iomem *(*get_base)(union gic_base *);
> +#endif
> + struct list_head v2m_list;
> +};
Maybe wrap this list_head around #ifdef/#endif?
> +
> +void gic_mask_irq(struct irq_data *d);
> +void gic_unmask_irq(struct irq_data *d);
> +int gicv2m_of_init(struct device_node *node, struct gic_chip_data *gic,
> + struct irq_chip *v2m_chip) __init;
> +
> +#endif /* _IRQ_GIC_H_ */
> --
> 1.9.3
>
>
--
Jazz is not dead. It just smells funny.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH 1/2 V6] irqchip: gic: Add supports for ARM GICv2m MSI(-X)
@ 2014-09-15 19:26 ` Marc Zyngier
0 siblings, 0 replies; 17+ messages in thread
From: Marc Zyngier @ 2014-09-15 19:26 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Sep 14 2014 at 07:30:15 AM, "suravee.suthikulpanit at amd.com" <suravee.suthikulpanit@amd.com> wrote:
Hi Suravee,
> From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>
> ARM GICv2m specification extends GICv2 to support MSI(-X) with
> a new set of register frame. This patch introduces support for
> the non-secure GICv2m register frame. Currently, GICV2m is available
> in certain version of GIC-400.
>
> The patch introduces a new property in ARM gic binding, the v2m subnode.
> It is optional.
Overall, this looks much better than the previous versions I've
reviewed. There is still a number of points I'd like to see addressed,
see below.
>
> Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> Cc: Mark Rutland <Mark.Rutland@arm.com>
> Cc: Marc Zyngier <Marc.Zyngier@arm.com>
> Cc: Jason Cooper <jason@lakedaemon.net>
> Cc: Catalin Marinas <Catalin.Marinas@arm.com>
> Cc: Will Deacon <Will.Deacon@arm.com>
> ---
> Documentation/devicetree/bindings/arm/gic.txt | 39 ++++
> drivers/irqchip/Kconfig | 7 +
> drivers/irqchip/Makefile | 1 +
> drivers/irqchip/irq-gic-v2m.c | 261 ++++++++++++++++++++++++++
> drivers/irqchip/irq-gic.c | 88 ++++++---
> drivers/irqchip/irq-gic.h | 51 +++++
> 6 files changed, 419 insertions(+), 28 deletions(-)
> create mode 100644 drivers/irqchip/irq-gic-v2m.c
> create mode 100644 drivers/irqchip/irq-gic.h
>
> diff --git a/Documentation/devicetree/bindings/arm/gic.txt b/Documentation/devicetree/bindings/arm/gic.txt
> index c7d2fa1..a461591 100644
> --- a/Documentation/devicetree/bindings/arm/gic.txt
> +++ b/Documentation/devicetree/bindings/arm/gic.txt
> @@ -96,3 +96,42 @@ Example:
> <0x2c006000 0x2000>;
> interrupts = <1 9 0xf04>;
> };
> +
> +
> +* GICv2m extension for MSI/MSI-x support (Optional)
> +
> +Certain revision of GIC-400 supports MSI/MSI-x via V2M register frame.
> +This is enabled by specifying v2m sub-node.
> +
> +Required properties:
> +
> +- msi-controller : Identifies the node as an MSI controller.
> +
> +- reg : GICv2m MSI interface register base and size
> +
> +Example:
> +
> + interrupt-controller at e1101000 {
> + compatible = "arm,gic-400";
> + #interrupt-cells = <3>;
> + #address-cells = <2>;
> + #size-cells = <2>;
> + interrupt-controller;
> + interrupts = <1 8 0xf04>;
> + ranges = <0 0 0 0xe1100000 0 0x100000>;
> + reg = <0x0 0xe1110000 0 0x01000>,
> + <0x0 0xe112f000 0 0x02000>,
> + <0x0 0xe1140000 0 0x10000>,
> + <0x0 0xe1160000 0 0x10000>;
> + v2m0 {
I believe the convention is to have
v2m0 : v2m at 0x80000 {
> + msi-controller;
> + reg = <0x0 0x80000 0 0x1000>;
I think having a "compatible" property would be nice, and would help
handle variants if they ever happen.
> + };
> +
> + ....
> +
> + v2mN {
> + msi-controller;
> + reg = <0x0 0x90000 0 0x1000>;
> + };
> + };
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index b8632bf..ddb31ee 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -7,6 +7,13 @@ config ARM_GIC
> select IRQ_DOMAIN
> select MULTI_IRQ_HANDLER
>
> +config ARM_GIC_V2M
> + bool
> + select IRQ_DOMAIN
> + select MULTI_IRQ_HANDLER
> + depends on ARM_GIC
> + depends on PCI && PCI_MSI
> +
> config GIC_NON_BANKED
> bool
>
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index 73052ba..3bda951 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -17,6 +17,7 @@ obj-$(CONFIG_ARCH_SUNXI) += irq-sun4i.o
> obj-$(CONFIG_ARCH_SUNXI) += irq-sunxi-nmi.o
> obj-$(CONFIG_ARCH_SPEAR3XX) += spear-shirq.o
> obj-$(CONFIG_ARM_GIC) += irq-gic.o irq-gic-common.o
> +obj-$(CONFIG_ARM_GIC_V2M) += irq-gic-v2m.o
> obj-$(CONFIG_ARM_GIC_V3) += irq-gic-v3.o irq-gic-common.o
> obj-$(CONFIG_ARM_NVIC) += irq-nvic.o
> obj-$(CONFIG_ARM_VIC) += irq-vic.o
> diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
> new file mode 100644
> index 0000000..499bfb9
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic-v2m.c
> @@ -0,0 +1,261 @@
> +/*
> + * ARM GIC v2m MSI(-X) support
> + * Support for Message Signaled Interrupts for systems that
> + * implement ARM Generic Interrupt Controller: GICv2m.
> + *
> + * Copyright (C) 2014 Advanced Micro Devices, Inc.
> + * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> + * Harish Kasiviswanathan <harish.kasiviswanathan@amd.com>
> + * Brandon Anderson <brandon.anderson@amd.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published
> + * by the Free Software Foundation.
> + */
> +
> +#define pr_fmt(fmt) "GICv2m: " fmt
> +
> +#include <linux/bitmap.h>
> +#include <linux/irq.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/of_address.h>
> +#include <linux/of_pci.h>
> +#include <linux/pci.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +#include <linux/spinlock.h>
> +
> +#include "irqchip.h"
> +#include "irq-gic.h"
> +
> +/*
> +* MSI_TYPER:
> +* [31:26] Reserved
> +* [25:16] lowest SPI assigned to MSI
> +* [15:10] Reserved
> +* [9:0] Numer of SPIs assigned to MSI
> +*/
> +#define V2M_MSI_TYPER 0x008
> +#define V2M_MSI_TYPER_BASE_SHIFT 16
> +#define V2M_MSI_TYPER_BASE_MASK 0x3FF
> +#define V2M_MSI_TYPER_NUM_MASK 0x3FF
> +#define V2M_MSI_SETSPI_NS 0x040
> +#define V2M_MIN_SPI 32
> +#define V2M_MAX_SPI 1019
> +
> +/*
> + * alloc_msi_irq - Allocate MSIs from available MSI bitmap.
> + * @data: Pointer to v2m_data
> + * @nvec: Number of interrupts to allocate
> + * @irq: Pointer to the allocated irq
> + *
> + * Allocates interrupts only if the contiguous range of MSIs
> + * with specified nvec are available. Otherwise return the number
> + * of available interrupts. If none are available, then returns -ENOENT.
> + */
> +static int alloc_msi_irq(struct v2m_data *data, int nvec, int *irq)
> +{
> + int size = data->nr_spis;
> + int next = size, i = nvec, ret;
> +
> + /* We should never allocate more than available nr_spis */
> + if (i >= size)
> + i = size;
> +
> + spin_lock(&data->msi_cnt_lock);
> +
> + for (; i > 0; i--) {
> + next = bitmap_find_next_zero_area(data->bm,
> + size, 0, i, 0);
> + if (next < size)
> + break;
> + }
> +
> + if (i != nvec) {
> + ret = i ? : -ENOENT;
> + } else {
> + bitmap_set(data->bm, next, nvec);
> + *irq = data->spi_start + next;
> + ret = 0;
> + }
> +
> + spin_unlock(&data->msi_cnt_lock);
> +
> + return ret;
> +}
> +
> +static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
> +{
> + int pos;
> + struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> + spin_lock(&data->msi_cnt_lock);
> +
> + pos = irq - data->spi_start;
> + if (pos >= 0 && pos < data->nr_spis)
> + bitmap_clear(data->bm, pos, 1);
> +
> + spin_unlock(&data->msi_cnt_lock);
> +}
> +
> +static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> + struct pci_dev *pdev,
> + struct msi_desc *desc)
> +{
> + int irq, avail;
> + struct msi_msg msg;
> + phys_addr_t addr;
> + struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
> +
> + if (!desc) {
> + dev_err(&pdev->dev,
> + "GICv2m: MSI setup failed. Invalid msi descriptor\n");
Isn't the "GICv2m" header already provided by the pr_fmt at the
beginning of the file?
> + return -EINVAL;
> + }
> +
> + avail = alloc_msi_irq(data, 1, &irq);
> + if (avail != 0) {
> + dev_err(&pdev->dev,
> + "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> + return -ENOSPC;
> + }
> +
> + irq_set_chip_data(irq, chip);
> + irq_set_msi_desc(irq, desc);
> + irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> +
> + addr = data->res.start + V2M_MSI_SETSPI_NS;
> + msg.address_hi = (u32)(addr >> 32);
> + msg.address_lo = (u32)(addr);
> + msg.data = irq;
> + write_msi_msg(irq, &msg);
> +
> + return 0;
> +}
> +
> +static void gicv2m_mask_irq(struct irq_data *d)
> +{
> + gic_mask_irq(d);
> + if (d->msi_desc)
> + mask_msi_irq(d);
> +}
> +
> +static void gicv2m_unmask_irq(struct irq_data *d)
> +{
> + gic_unmask_irq(d);
> + if (d->msi_desc)
> + unmask_msi_irq(d);
> +}
> +
> +#ifdef CONFIG_OF
I think you can loose the CONFIG_OF here. There is no valid
configuration without OF on arm64.
> +
> +static int __init
> +v2m_init(struct device_node *node, struct v2m_data *v2m,
> + struct resource *res)
> +{
> + int ret;
> + unsigned int typer;
> +
> + v2m->msi_chip.owner = THIS_MODULE;
> + v2m->msi_chip.of_node = node;
> + v2m->msi_chip.setup_irq = gicv2m_setup_msi_irq;
> + v2m->msi_chip.teardown_irq = gicv2m_teardown_msi_irq;
> + ret = of_pci_msi_chip_add(&v2m->msi_chip);
> + if (ret) {
> + pr_info("Failed to add msi_chip.\n");
> + return ret;
> + }
> +
> + v2m->base = ioremap(res->start, resource_size(res));
> + if (!v2m->base) {
> + pr_err("Failed to map GICv2m resource\n");
> + return -EINVAL;
Beware, you're leaving with a registered msi_chip. You probably want to
remove it on the error path.
> + }
> +
> + memcpy(&v2m->res, res, sizeof(struct resource));
> +
> + typer = readl_relaxed(v2m->base + V2M_MSI_TYPER);
> + if (!typer) {
> + pr_warn("Failed to read V2M_MSI_TYPER register\n");
> + return -EINVAL;
And here leaving the v2m widget mapped.
> + }
> +
> + v2m->spi_start = (typer >> V2M_MSI_TYPER_BASE_SHIFT) &
> + V2M_MSI_TYPER_BASE_MASK;
> + v2m->nr_spis = typer & V2M_MSI_TYPER_NUM_MASK;
> + if ((v2m->spi_start < V2M_MIN_SPI) || (v2m->nr_spis >= V2M_MAX_SPI)) {
> + pr_err("Invalid MSI_TYPER (%#x)\n", typer);
> + return -EINVAL;
Blah. The whole error handling of this function is b0rked.
> + }
> +
> + v2m->bm = kzalloc(sizeof(long) * BITS_TO_LONGS(v2m->nr_spis),
> + GFP_KERNEL);
> + if (!v2m->bm) {
> + pr_err("Failed to allocate MSI bitmap\n");
> + return -ENOMEM;
> + }
> +
> + spin_lock_init(&v2m->msi_cnt_lock);
> +
> + pr_info("SPI range [%d:%d]\n",
> + v2m->spi_start, (v2m->spi_start + v2m->nr_spis));
> +
> + return 0;
> +}
> +
> +int __init gicv2m_of_init(struct device_node *node,
> + struct gic_chip_data *gic,
> + struct irq_chip *v2m_chip)
> +{
> + int ret = 0;
> + struct v2m_data *v2m;
> + struct resource res;
> + struct device_node *child = NULL;
> +
> + INIT_LIST_HEAD(&gic->v2m_list);
> +
> + for (;;) {
> + child = of_get_next_child(node, child);
I don't see any of_node_put() in this function. You'll need to fix
this...
> + if (!child)
> + break;
> +
> + if (!of_find_property(child, "msi-controller", NULL))
> + continue;
> +
> + ret = of_address_to_resource(child, 0, &res);
> + if (ret) {
> + pr_err("Failed to allocate v2m resource.\n");
> + break;
> + }
> +
> + v2m = kzalloc(sizeof(struct v2m_data), GFP_KERNEL);
> + if (!v2m) {
> + pr_err("Failed to allocate struct v2m_data.\n");
> + return -ENOMEM;
> + }
> +
> + ret = v2m_init(child, v2m, &res);
> + if (ret)
> + break;
> +
> + v2m->gic = gic;
> +
> + pr_info(" %s: [%lx : %lx]\n", child->name,
> + (unsigned long)res.start, (unsigned long)res.end);
> +
> + list_add_tail(&v2m->list, &gic->v2m_list);
> + }
> +
> + if (ret && list_empty(&gic->v2m_list)) {
> + pr_err("MSI resource not found\n");
> + return -EINVAL;
> + }
> +
> + v2m_chip->irq_mask = gicv2m_mask_irq;
> + v2m_chip->irq_unmask = gicv2m_unmask_irq;
> +
Again, the whole error handling is evil. You can end-up allocating a few
v2m structures, registering them, and yet have kzalloc failing at some
point. At that stage, you exit early, not setting the mas/unmask
methods. Boom.
> + return ret;
> +}
> +
> +#endif /* CONFIG_OF */
> diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c
> index 4b959e6..f490599 100644
> --- a/drivers/irqchip/irq-gic.c
> +++ b/drivers/irqchip/irq-gic.c
> @@ -46,30 +46,9 @@
> #include <asm/smp_plat.h>
>
> #include "irq-gic-common.h"
> +#include "irq-gic.h"
> #include "irqchip.h"
>
> -union gic_base {
> - void __iomem *common_base;
> - void __percpu * __iomem *percpu_base;
> -};
> -
> -struct gic_chip_data {
> - union gic_base dist_base;
> - union gic_base cpu_base;
> -#ifdef CONFIG_CPU_PM
> - u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> - u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> - u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> - u32 __percpu *saved_ppi_enable;
> - u32 __percpu *saved_ppi_conf;
> -#endif
> - struct irq_domain *domain;
> - unsigned int gic_irqs;
> -#ifdef CONFIG_GIC_NON_BANKED
> - void __iomem *(*get_base)(union gic_base *);
> -#endif
> -};
> -
> static DEFINE_RAW_SPINLOCK(irq_controller_lock);
>
> /*
> @@ -131,15 +110,36 @@ static inline void gic_set_base_accessor(struct gic_chip_data *data,
> #define gic_set_base_accessor(d, f)
> #endif
>
> +static inline
> +struct gic_chip_data *irq_data_get_gic_chip_data(struct irq_data *d)
> +{
> + struct gic_chip_data *gic_data;
> + struct msi_chip *mchip;
> + struct v2m_data *v2mdat;
> +
> + /*
> + * For MSI, irq_data.chip_data points to struct msi_chip.
> + * For non-MSI, irq_data.chip_data points to struct gic_chip_data.
> + */
> + if (d->msi_desc) {
> + mchip = irq_data_get_irq_chip_data(d);
> + v2mdat = container_of(mchip, struct v2m_data, msi_chip);
> + gic_data = v2mdat->gic;
> + } else {
> + gic_data = irq_data_get_irq_chip_data(d);
> + }
> + return gic_data;
> +}
> +
> static inline void __iomem *gic_dist_base(struct irq_data *d)
> {
> - struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> + struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
> return gic_data_dist_base(gic_data);
> }
>
> static inline void __iomem *gic_cpu_base(struct irq_data *d)
> {
> - struct gic_chip_data *gic_data = irq_data_get_irq_chip_data(d);
> + struct gic_chip_data *gic_data = irq_data_get_gic_chip_data(d);
> return gic_data_cpu_base(gic_data);
> }
>
> @@ -151,7 +151,7 @@ static inline unsigned int gic_irq(struct irq_data *d)
> /*
> * Routines to acknowledge, disable and enable interrupts
> */
> -static void gic_mask_irq(struct irq_data *d)
> +void gic_mask_irq(struct irq_data *d)
I'm still very uncomfortable with exporting the mask/unmask methods
outside of this file... Maybe it is not a big deal, but I see evil
everywhere.
> {
> u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -162,7 +162,7 @@ static void gic_mask_irq(struct irq_data *d)
> raw_spin_unlock(&irq_controller_lock);
> }
>
> -static void gic_unmask_irq(struct irq_data *d)
> +void gic_unmask_irq(struct irq_data *d)
> {
> u32 mask = 1 << (gic_irq(d) % 32);
>
> @@ -325,6 +325,15 @@ static struct irq_chip gic_chip = {
> .irq_set_wake = gic_set_wake,
> };
>
> +static struct irq_chip v2m_chip = {
> + .name = "GICv2m",
> + .irq_eoi = gic_eoi_irq,
> + .irq_set_type = gic_set_type,
> +#ifdef CONFIG_SMP
> + .irq_set_affinity = gic_set_affinity,
> +#endif
> +};
> +
> void __init gic_cascade_irq(unsigned int gic_nr, unsigned int irq)
> {
> if (gic_nr >= MAX_GIC_NR)
> @@ -767,6 +776,24 @@ void __init gic_init_physaddr(struct device_node *node)
> static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
> irq_hw_number_t hw)
> {
> + struct gic_chip_data *gic = d->host_data;
> + struct v2m_data *v2m;
> +
> + irq_set_chip_data(irq, gic);
>
> +#ifdef CONFIG_ARM_GIC_V2M
> + list_for_each_entry(v2m, &gic->v2m_list, list) {
> + if (hw >= v2m->spi_start &&
> + hw < v2m->spi_start + v2m->nr_spis) {
> +
> + irq_set_chip_and_handler(irq, &v2m_chip,
> + handle_fasteoi_irq);
> + set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
> + return 0;
> + }
> + }
> +#endif
> +
> if (hw < 32) {
> irq_set_percpu_devid(irq);
> irq_set_chip_and_handler(irq, &gic_chip,
> @@ -779,7 +806,6 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int irq,
>
> gic_routable_irq_domain_ops->map(d, irq, hw);
> }
> - irq_set_chip_data(irq, d->host_data);
These two hunks make the code a bit hard to read:
- First, message-based SPIs,
- Then PPIs
- and finally the rest of the SPIs.
Can you come up with a construct that keeps everything in a logical
order (PPIs, and then SPIs)? Not a big deal but it would make the code
nicer.
> return 0;
> }
>
> @@ -996,7 +1022,7 @@ gic_of_init(struct device_node *node, struct device_node *parent)
> void __iomem *cpu_base;
> void __iomem *dist_base;
> u32 percpu_offset;
> - int irq;
> + int irq, ret;
>
> if (WARN_ON(!node))
> return -ENODEV;
> @@ -1010,6 +1036,12 @@ gic_of_init(struct device_node *node, struct device_node *parent)
> if (of_property_read_u32(node, "cpu-offset", &percpu_offset))
> percpu_offset = 0;
>
> + if (IS_ENABLED(CONFIG_ARM_GIC_V2M)) {
> + ret = gicv2m_of_init(node, &gic_data[gic_cnt], &v2m_chip);
> + if (ret)
> + pr_warn("Warning: Failed to enable GICv2m support.\n");
> + }
> +
> gic_init_bases(gic_cnt, -1, dist_base, cpu_base, percpu_offset, node);
> if (!gic_cnt)
> gic_init_physaddr(node);
> diff --git a/drivers/irqchip/irq-gic.h b/drivers/irqchip/irq-gic.h
> new file mode 100644
> index 0000000..1f4a2b0
> --- /dev/null
> +++ b/drivers/irqchip/irq-gic.h
> @@ -0,0 +1,51 @@
> +#ifndef _IRQ_GIC_H_
> +#define _IRQ_GIC_H_
> +
> +#include <linux/msi.h>
> +#include <linux/pci.h>
> +
> +union gic_base {
> + void __iomem *common_base;
> + void __percpu * __iomem *percpu_base;
> +};
> +
> +struct gic_chip_data;
> +
> +struct v2m_data {
> +#ifdef CONFIG_ARM_GIC_V2M
> + struct list_head list;
> + spinlock_t msi_cnt_lock;
> + struct msi_chip msi_chip;
> + struct resource res; /* GICv2m resource */
> + void __iomem *base; /* GICv2m virt address */
> + unsigned int spi_start; /* The SPI number that MSIs start */
> + unsigned int nr_spis; /* The number of SPIs for MSIs */
> + unsigned long *bm; /* MSI vector bitmap */
> + struct gic_chip_data *gic;
> +#endif
> +};
> +
> +struct gic_chip_data {
> + union gic_base dist_base;
> + union gic_base cpu_base;
> +#ifdef CONFIG_CPU_PM
> + u32 saved_spi_enable[DIV_ROUND_UP(1020, 32)];
> + u32 saved_spi_conf[DIV_ROUND_UP(1020, 16)];
> + u32 saved_spi_target[DIV_ROUND_UP(1020, 4)];
> + u32 __percpu *saved_ppi_enable;
> + u32 __percpu *saved_ppi_conf;
> +#endif
> + struct irq_domain *domain;
> + unsigned int gic_irqs;
> +#ifdef CONFIG_GIC_NON_BANKED
> + void __iomem *(*get_base)(union gic_base *);
> +#endif
> + struct list_head v2m_list;
> +};
Maybe wrap this list_head around #ifdef/#endif?
> +
> +void gic_mask_irq(struct irq_data *d);
> +void gic_unmask_irq(struct irq_data *d);
> +int gicv2m_of_init(struct device_node *node, struct gic_chip_data *gic,
> + struct irq_chip *v2m_chip) __init;
> +
> +#endif /* _IRQ_GIC_H_ */
> --
> 1.9.3
>
>
--
Jazz is not dead. It just smells funny.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 2/2 V6] irqchip: gicv2m: Add support for multiple MSI for ARM64 GICv2m
2014-09-14 6:30 ` suravee.suthikulpanit
(?)
(?)
@ 2014-09-15 19:41 ` Marc Zyngier
-1 siblings, 0 replies; 17+ messages in thread
From: Marc Zyngier @ 2014-09-15 19:41 UTC (permalink / raw)
To: suravee.suthikulpanit
Cc: Mark Rutland, jason, Pawel Moll, Catalin Marinas, Will Deacon,
tglx, Harish.Kasiviswanathan, linux-arm-kernel, linux-pci,
linux-kernel, linux-doc, devicetree
On Sun, Sep 14 2014 at 07:30:16 AM, "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com> wrote:
> From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>
> This patch extend GICv2m MSI to support multiple MSI in ARM64.
>
> This requires the common arch_setup_msi_irqs() to be overwriten
> with ARM64 version which does not return 1 for PCI_CAP_ID_MSI and
> nvec > 1.
>
> Cc: Mark Rutland <Mark.Rutland@arm.com>
> Cc: Marc Zyngier <Marc.Zyngier@arm.com>
> Cc: Jason Cooper <jason@lakedaemon.net>
> Cc: Catalin Marinas <Catalin.Marinas@arm.com>
> Cc: Will Deacon <Will.Deacon@arm.com>
> Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> ---
> arch/arm64/kernel/Makefile | 1 +
> arch/arm64/kernel/msi.c | 41 ++++++++++++++++++++
> drivers/irqchip/irq-gic-v2m.c | 87 ++++++++++++++++++++++++++++++++++++++-----
> 3 files changed, 119 insertions(+), 10 deletions(-)
> create mode 100644 arch/arm64/kernel/msi.c
>
> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
> index df7ef87..a921c42 100644
> --- a/arch/arm64/kernel/Makefile
> +++ b/arch/arm64/kernel/Makefile
> @@ -29,6 +29,7 @@ arm64-obj-$(CONFIG_ARM64_CPU_SUSPEND) += sleep.o suspend.o
> arm64-obj-$(CONFIG_JUMP_LABEL) += jump_label.o
> arm64-obj-$(CONFIG_KGDB) += kgdb.o
> arm64-obj-$(CONFIG_EFI) += efi.o efi-stub.o efi-entry.o
> +arm64-obj-$(CONFIG_PCI_MSI) += msi.o
>
> obj-y += $(arm64-obj-y) vdso/
> obj-m += $(arm64-obj-m)
> diff --git a/arch/arm64/kernel/msi.c b/arch/arm64/kernel/msi.c
> new file mode 100644
> index 0000000..a295862
> --- /dev/null
> +++ b/arch/arm64/kernel/msi.c
> @@ -0,0 +1,41 @@
> +/*
> + * ARM64 architectural MSI implemention
> + *
> + * Support for Message Signalelled Interrupts for systems that
> + * implement ARM Generic Interrupt Controller: GICv2m.
> + *
> + * Copyright (C) 2014 Advanced Micro Devices, Inc.
> + * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published
> + * by the Free Software Foundation.
> + */
> +
> +#include <linux/irq.h>
> +#include <linux/msi.h>
> +#include <linux/pci.h>
> +
> +/*
> + * ARM64 function for seting up MSI irqs.
> + * Based on driver/pci/msi.c: arch_setup_msi_irqs().
> + *
> + * Note:
> + * Current implementation assumes that all interrupt controller used in
> + * ARM64 architecture _MUST_ supports multi-MSI.
> + */
> +int arm64_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
> +{
> + struct msi_desc *entry;
> + int ret;
> +
> + list_for_each_entry(entry, &dev->msi_list, list) {
> + ret = arch_setup_msi_irq(dev, entry);
> + if (ret < 0)
> + return ret;
> + if (ret > 0)
> + return -ENOSPC;
> + }
> +
> + return 0;
> +}
> diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
> index 499bfb9..4ddb47d 100644
> --- a/drivers/irqchip/irq-gic-v2m.c
> +++ b/drivers/irqchip/irq-gic-v2m.c
> @@ -99,13 +99,26 @@ static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
> spin_unlock(&data->msi_cnt_lock);
> }
>
> +static int gicv2m_msi_get_vec_count(struct pci_dev *pdev, struct msi_desc *desc)
> +{
> + int ret = -EINVAL;
> +#ifdef CONFIG_PCI_MSI
> + if (desc->msi_attrib.is_msix)
> + ret = pci_msix_vec_count(pdev);
> + else
> + ret = pci_msi_vec_count(pdev);
> +#endif
> + return ret;
> +}
> +
Since I also have:
https://git.kernel.org/cgit/linux/kernel/git/maz/arm-platforms.git/tree/drivers/irqchip/irq-gic-v3-its.c?h=gicv3/its-split#n1107
Maybe it would be a good thing to make this a core thing (not something
we have to do now though).
> static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> struct pci_dev *pdev,
> struct msi_desc *desc)
> {
> - int irq, avail;
> + int i, irq, nvec, avail;
> struct msi_msg msg;
> phys_addr_t addr;
> + struct msi_desc *entry;
> struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
>
> if (!desc) {
> @@ -114,16 +127,70 @@ static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> return -EINVAL;
> }
>
> - avail = alloc_msi_irq(data, 1, &irq);
> - if (avail != 0) {
> - dev_err(&pdev->dev,
> - "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> - return -ENOSPC;
> - }
> + if (desc->msi_attrib.is_msix) {
> + /**
> + * For MSIx:
> + * We allocate one irq at a time
> + */
> + avail = alloc_msi_irq(data, 1, &irq);
> + if (avail != 0) {
> + dev_err(&pdev->dev,
> + "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> + return -ENOSPC;
> + }
>
> - irq_set_chip_data(irq, chip);
> - irq_set_msi_desc(irq, desc);
> - irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> + irq_set_chip_data(irq, chip);
> + irq_set_msi_desc(irq, desc);
> + irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> + } else {
> + /**
> + * For MSI and Multi-MSI:
> + * All requested irqs are allocated and setup at
> + * once. Subsequent calls to this function would simply return
> + * success. This is to avoid having to implement a separate
> + * function for setting up multiple irqs.
> + */
> + BUG_ON(list_empty(&pdev->msi_list));
> + WARN_ON(!list_is_singular(&pdev->msi_list));
> +
> + nvec = gicv2m_msi_get_vec_count(pdev, desc);
> + if (WARN_ON(nvec <= 0))
> + return nvec;
> +
> + entry = list_first_entry(&pdev->msi_list,
> + struct msi_desc, list);
> +
> + if ((nvec > 1) && (entry->msi_attrib.multiple))
> + return 0;
> +
> + avail = alloc_msi_irq(data, nvec, &irq);
> + if (avail != 0) {
> + dev_err(&pdev->dev,
> + "GICv2m: Failed to allocate %d irqs.\n", nvec);
> + return avail;
> + }
> +
> + if (nvec > 1) {
> + /* Set lowest of the new interrupts assigned
> + * to the PCI device
> + */
> + entry->nvec_used = nvec;
> + entry->msi_attrib.multiple = ilog2(
> + __roundup_pow_of_two(nvec));
> + }
> +
> + for (i = 0; i < nvec; i++) {
> + irq_set_chip_data(irq+i, chip);
> + if (irq_set_msi_desc_off(irq, i, entry)) {
> + dev_err(&pdev->dev,
> + "GICv2m: Failed to set up MSI irq %d\n",
> + (irq+i));
> + return -EINVAL;
> + }
> +
> + irq_set_irq_type((irq+i), IRQ_TYPE_EDGE_RISING);
> + }
> + }
>
> addr = data->res.start + V2M_MSI_SETSPI_NS;
> msg.address_hi = (u32)(addr >> 32);
Overall, I think you should move this "msi > 1" code directly into the
first patch (not sure the intermediate implementation adds much to the
understanding of the whole driver).
You can then have the arm64-specific code as your second patch.
Thanks,
M.
--
Jazz is not dead. It just smells funny.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 2/2 V6] irqchip: gicv2m: Add support for multiple MSI for ARM64 GICv2m
@ 2014-09-15 19:41 ` Marc Zyngier
0 siblings, 0 replies; 17+ messages in thread
From: Marc Zyngier @ 2014-09-15 19:41 UTC (permalink / raw)
To: suravee.suthikulpanit
Cc: Mark Rutland, jason, Pawel Moll, Catalin Marinas, Will Deacon,
tglx, Harish.Kasiviswanathan, linux-arm-kernel, linux-pci,
linux-kernel, linux-doc, devicetree
On Sun, Sep 14 2014 at 07:30:16 AM, "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com> wrote:
> From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>
> This patch extend GICv2m MSI to support multiple MSI in ARM64.
>
> This requires the common arch_setup_msi_irqs() to be overwriten
> with ARM64 version which does not return 1 for PCI_CAP_ID_MSI and
> nvec > 1.
>
> Cc: Mark Rutland <Mark.Rutland@arm.com>
> Cc: Marc Zyngier <Marc.Zyngier@arm.com>
> Cc: Jason Cooper <jason@lakedaemon.net>
> Cc: Catalin Marinas <Catalin.Marinas@arm.com>
> Cc: Will Deacon <Will.Deacon@arm.com>
> Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> ---
> arch/arm64/kernel/Makefile | 1 +
> arch/arm64/kernel/msi.c | 41 ++++++++++++++++++++
> drivers/irqchip/irq-gic-v2m.c | 87 ++++++++++++++++++++++++++++++++++++++-----
> 3 files changed, 119 insertions(+), 10 deletions(-)
> create mode 100644 arch/arm64/kernel/msi.c
>
> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
> index df7ef87..a921c42 100644
> --- a/arch/arm64/kernel/Makefile
> +++ b/arch/arm64/kernel/Makefile
> @@ -29,6 +29,7 @@ arm64-obj-$(CONFIG_ARM64_CPU_SUSPEND) += sleep.o suspend.o
> arm64-obj-$(CONFIG_JUMP_LABEL) += jump_label.o
> arm64-obj-$(CONFIG_KGDB) += kgdb.o
> arm64-obj-$(CONFIG_EFI) += efi.o efi-stub.o efi-entry.o
> +arm64-obj-$(CONFIG_PCI_MSI) += msi.o
>
> obj-y += $(arm64-obj-y) vdso/
> obj-m += $(arm64-obj-m)
> diff --git a/arch/arm64/kernel/msi.c b/arch/arm64/kernel/msi.c
> new file mode 100644
> index 0000000..a295862
> --- /dev/null
> +++ b/arch/arm64/kernel/msi.c
> @@ -0,0 +1,41 @@
> +/*
> + * ARM64 architectural MSI implemention
> + *
> + * Support for Message Signalelled Interrupts for systems that
> + * implement ARM Generic Interrupt Controller: GICv2m.
> + *
> + * Copyright (C) 2014 Advanced Micro Devices, Inc.
> + * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published
> + * by the Free Software Foundation.
> + */
> +
> +#include <linux/irq.h>
> +#include <linux/msi.h>
> +#include <linux/pci.h>
> +
> +/*
> + * ARM64 function for seting up MSI irqs.
> + * Based on driver/pci/msi.c: arch_setup_msi_irqs().
> + *
> + * Note:
> + * Current implementation assumes that all interrupt controller used in
> + * ARM64 architecture _MUST_ supports multi-MSI.
> + */
> +int arm64_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
> +{
> + struct msi_desc *entry;
> + int ret;
> +
> + list_for_each_entry(entry, &dev->msi_list, list) {
> + ret = arch_setup_msi_irq(dev, entry);
> + if (ret < 0)
> + return ret;
> + if (ret > 0)
> + return -ENOSPC;
> + }
> +
> + return 0;
> +}
> diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
> index 499bfb9..4ddb47d 100644
> --- a/drivers/irqchip/irq-gic-v2m.c
> +++ b/drivers/irqchip/irq-gic-v2m.c
> @@ -99,13 +99,26 @@ static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
> spin_unlock(&data->msi_cnt_lock);
> }
>
> +static int gicv2m_msi_get_vec_count(struct pci_dev *pdev, struct msi_desc *desc)
> +{
> + int ret = -EINVAL;
> +#ifdef CONFIG_PCI_MSI
> + if (desc->msi_attrib.is_msix)
> + ret = pci_msix_vec_count(pdev);
> + else
> + ret = pci_msi_vec_count(pdev);
> +#endif
> + return ret;
> +}
> +
Since I also have:
https://git.kernel.org/cgit/linux/kernel/git/maz/arm-platforms.git/tree/drivers/irqchip/irq-gic-v3-its.c?h=gicv3/its-split#n1107
Maybe it would be a good thing to make this a core thing (not something
we have to do now though).
> static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> struct pci_dev *pdev,
> struct msi_desc *desc)
> {
> - int irq, avail;
> + int i, irq, nvec, avail;
> struct msi_msg msg;
> phys_addr_t addr;
> + struct msi_desc *entry;
> struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
>
> if (!desc) {
> @@ -114,16 +127,70 @@ static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> return -EINVAL;
> }
>
> - avail = alloc_msi_irq(data, 1, &irq);
> - if (avail != 0) {
> - dev_err(&pdev->dev,
> - "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> - return -ENOSPC;
> - }
> + if (desc->msi_attrib.is_msix) {
> + /**
> + * For MSIx:
> + * We allocate one irq at a time
> + */
> + avail = alloc_msi_irq(data, 1, &irq);
> + if (avail != 0) {
> + dev_err(&pdev->dev,
> + "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> + return -ENOSPC;
> + }
>
> - irq_set_chip_data(irq, chip);
> - irq_set_msi_desc(irq, desc);
> - irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> + irq_set_chip_data(irq, chip);
> + irq_set_msi_desc(irq, desc);
> + irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> + } else {
> + /**
> + * For MSI and Multi-MSI:
> + * All requested irqs are allocated and setup at
> + * once. Subsequent calls to this function would simply return
> + * success. This is to avoid having to implement a separate
> + * function for setting up multiple irqs.
> + */
> + BUG_ON(list_empty(&pdev->msi_list));
> + WARN_ON(!list_is_singular(&pdev->msi_list));
> +
> + nvec = gicv2m_msi_get_vec_count(pdev, desc);
> + if (WARN_ON(nvec <= 0))
> + return nvec;
> +
> + entry = list_first_entry(&pdev->msi_list,
> + struct msi_desc, list);
> +
> + if ((nvec > 1) && (entry->msi_attrib.multiple))
> + return 0;
> +
> + avail = alloc_msi_irq(data, nvec, &irq);
> + if (avail != 0) {
> + dev_err(&pdev->dev,
> + "GICv2m: Failed to allocate %d irqs.\n", nvec);
> + return avail;
> + }
> +
> + if (nvec > 1) {
> + /* Set lowest of the new interrupts assigned
> + * to the PCI device
> + */
> + entry->nvec_used = nvec;
> + entry->msi_attrib.multiple = ilog2(
> + __roundup_pow_of_two(nvec));
> + }
> +
> + for (i = 0; i < nvec; i++) {
> + irq_set_chip_data(irq+i, chip);
> + if (irq_set_msi_desc_off(irq, i, entry)) {
> + dev_err(&pdev->dev,
> + "GICv2m: Failed to set up MSI irq %d\n",
> + (irq+i));
> + return -EINVAL;
> + }
> +
> + irq_set_irq_type((irq+i), IRQ_TYPE_EDGE_RISING);
> + }
> + }
>
> addr = data->res.start + V2M_MSI_SETSPI_NS;
> msg.address_hi = (u32)(addr >> 32);
Overall, I think you should move this "msi > 1" code directly into the
first patch (not sure the intermediate implementation adds much to the
understanding of the whole driver).
You can then have the arm64-specific code as your second patch.
Thanks,
M.
--
Jazz is not dead. It just smells funny.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH 2/2 V6] irqchip: gicv2m: Add support for multiple MSI for ARM64 GICv2m
@ 2014-09-15 19:41 ` Marc Zyngier
0 siblings, 0 replies; 17+ messages in thread
From: Marc Zyngier @ 2014-09-15 19:41 UTC (permalink / raw)
To: suravee.suthikulpanit
Cc: Mark Rutland, jason, Pawel Moll, Catalin Marinas, Will Deacon,
tglx, Harish.Kasiviswanathan, linux-arm-kernel, linux-pci,
linux-kernel, linux-doc, devicetree
On Sun, Sep 14 2014 at 07:30:16 AM, "suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com> wrote:
> From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>
> This patch extend GICv2m MSI to support multiple MSI in ARM64.
>
> This requires the common arch_setup_msi_irqs() to be overwriten
> with ARM64 version which does not return 1 for PCI_CAP_ID_MSI and
> nvec > 1.
>
> Cc: Mark Rutland <Mark.Rutland@arm.com>
> Cc: Marc Zyngier <Marc.Zyngier@arm.com>
> Cc: Jason Cooper <jason@lakedaemon.net>
> Cc: Catalin Marinas <Catalin.Marinas@arm.com>
> Cc: Will Deacon <Will.Deacon@arm.com>
> Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> ---
> arch/arm64/kernel/Makefile | 1 +
> arch/arm64/kernel/msi.c | 41 ++++++++++++++++++++
> drivers/irqchip/irq-gic-v2m.c | 87 ++++++++++++++++++++++++++++++++++++++-----
> 3 files changed, 119 insertions(+), 10 deletions(-)
> create mode 100644 arch/arm64/kernel/msi.c
>
> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
> index df7ef87..a921c42 100644
> --- a/arch/arm64/kernel/Makefile
> +++ b/arch/arm64/kernel/Makefile
> @@ -29,6 +29,7 @@ arm64-obj-$(CONFIG_ARM64_CPU_SUSPEND) += sleep.o suspend.o
> arm64-obj-$(CONFIG_JUMP_LABEL) += jump_label.o
> arm64-obj-$(CONFIG_KGDB) += kgdb.o
> arm64-obj-$(CONFIG_EFI) += efi.o efi-stub.o efi-entry.o
> +arm64-obj-$(CONFIG_PCI_MSI) += msi.o
>
> obj-y += $(arm64-obj-y) vdso/
> obj-m += $(arm64-obj-m)
> diff --git a/arch/arm64/kernel/msi.c b/arch/arm64/kernel/msi.c
> new file mode 100644
> index 0000000..a295862
> --- /dev/null
> +++ b/arch/arm64/kernel/msi.c
> @@ -0,0 +1,41 @@
> +/*
> + * ARM64 architectural MSI implemention
> + *
> + * Support for Message Signalelled Interrupts for systems that
> + * implement ARM Generic Interrupt Controller: GICv2m.
> + *
> + * Copyright (C) 2014 Advanced Micro Devices, Inc.
> + * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published
> + * by the Free Software Foundation.
> + */
> +
> +#include <linux/irq.h>
> +#include <linux/msi.h>
> +#include <linux/pci.h>
> +
> +/*
> + * ARM64 function for seting up MSI irqs.
> + * Based on driver/pci/msi.c: arch_setup_msi_irqs().
> + *
> + * Note:
> + * Current implementation assumes that all interrupt controller used in
> + * ARM64 architecture _MUST_ supports multi-MSI.
> + */
> +int arm64_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
> +{
> + struct msi_desc *entry;
> + int ret;
> +
> + list_for_each_entry(entry, &dev->msi_list, list) {
> + ret = arch_setup_msi_irq(dev, entry);
> + if (ret < 0)
> + return ret;
> + if (ret > 0)
> + return -ENOSPC;
> + }
> +
> + return 0;
> +}
> diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
> index 499bfb9..4ddb47d 100644
> --- a/drivers/irqchip/irq-gic-v2m.c
> +++ b/drivers/irqchip/irq-gic-v2m.c
> @@ -99,13 +99,26 @@ static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
> spin_unlock(&data->msi_cnt_lock);
> }
>
> +static int gicv2m_msi_get_vec_count(struct pci_dev *pdev, struct msi_desc *desc)
> +{
> + int ret = -EINVAL;
> +#ifdef CONFIG_PCI_MSI
> + if (desc->msi_attrib.is_msix)
> + ret = pci_msix_vec_count(pdev);
> + else
> + ret = pci_msi_vec_count(pdev);
> +#endif
> + return ret;
> +}
> +
Since I also have:
https://git.kernel.org/cgit/linux/kernel/git/maz/arm-platforms.git/tree/drivers/irqchip/irq-gic-v3-its.c?h=gicv3/its-split#n1107
Maybe it would be a good thing to make this a core thing (not something
we have to do now though).
> static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> struct pci_dev *pdev,
> struct msi_desc *desc)
> {
> - int irq, avail;
> + int i, irq, nvec, avail;
> struct msi_msg msg;
> phys_addr_t addr;
> + struct msi_desc *entry;
> struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
>
> if (!desc) {
> @@ -114,16 +127,70 @@ static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> return -EINVAL;
> }
>
> - avail = alloc_msi_irq(data, 1, &irq);
> - if (avail != 0) {
> - dev_err(&pdev->dev,
> - "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> - return -ENOSPC;
> - }
> + if (desc->msi_attrib.is_msix) {
> + /**
> + * For MSIx:
> + * We allocate one irq at a time
> + */
> + avail = alloc_msi_irq(data, 1, &irq);
> + if (avail != 0) {
> + dev_err(&pdev->dev,
> + "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> + return -ENOSPC;
> + }
>
> - irq_set_chip_data(irq, chip);
> - irq_set_msi_desc(irq, desc);
> - irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> + irq_set_chip_data(irq, chip);
> + irq_set_msi_desc(irq, desc);
> + irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> + } else {
> + /**
> + * For MSI and Multi-MSI:
> + * All requested irqs are allocated and setup at
> + * once. Subsequent calls to this function would simply return
> + * success. This is to avoid having to implement a separate
> + * function for setting up multiple irqs.
> + */
> + BUG_ON(list_empty(&pdev->msi_list));
> + WARN_ON(!list_is_singular(&pdev->msi_list));
> +
> + nvec = gicv2m_msi_get_vec_count(pdev, desc);
> + if (WARN_ON(nvec <= 0))
> + return nvec;
> +
> + entry = list_first_entry(&pdev->msi_list,
> + struct msi_desc, list);
> +
> + if ((nvec > 1) && (entry->msi_attrib.multiple))
> + return 0;
> +
> + avail = alloc_msi_irq(data, nvec, &irq);
> + if (avail != 0) {
> + dev_err(&pdev->dev,
> + "GICv2m: Failed to allocate %d irqs.\n", nvec);
> + return avail;
> + }
> +
> + if (nvec > 1) {
> + /* Set lowest of the new interrupts assigned
> + * to the PCI device
> + */
> + entry->nvec_used = nvec;
> + entry->msi_attrib.multiple = ilog2(
> + __roundup_pow_of_two(nvec));
> + }
> +
> + for (i = 0; i < nvec; i++) {
> + irq_set_chip_data(irq+i, chip);
> + if (irq_set_msi_desc_off(irq, i, entry)) {
> + dev_err(&pdev->dev,
> + "GICv2m: Failed to set up MSI irq %d\n",
> + (irq+i));
> + return -EINVAL;
> + }
> +
> + irq_set_irq_type((irq+i), IRQ_TYPE_EDGE_RISING);
> + }
> + }
>
> addr = data->res.start + V2M_MSI_SETSPI_NS;
> msg.address_hi = (u32)(addr >> 32);
Overall, I think you should move this "msi > 1" code directly into the
first patch (not sure the intermediate implementation adds much to the
understanding of the whole driver).
You can then have the arm64-specific code as your second patch.
Thanks,
M.
--
Jazz is not dead. It just smells funny.
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH 2/2 V6] irqchip: gicv2m: Add support for multiple MSI for ARM64 GICv2m
@ 2014-09-15 19:41 ` Marc Zyngier
0 siblings, 0 replies; 17+ messages in thread
From: Marc Zyngier @ 2014-09-15 19:41 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Sep 14 2014 at 07:30:16 AM, "suravee.suthikulpanit at amd.com" <suravee.suthikulpanit@amd.com> wrote:
> From: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
>
> This patch extend GICv2m MSI to support multiple MSI in ARM64.
>
> This requires the common arch_setup_msi_irqs() to be overwriten
> with ARM64 version which does not return 1 for PCI_CAP_ID_MSI and
> nvec > 1.
>
> Cc: Mark Rutland <Mark.Rutland@arm.com>
> Cc: Marc Zyngier <Marc.Zyngier@arm.com>
> Cc: Jason Cooper <jason@lakedaemon.net>
> Cc: Catalin Marinas <Catalin.Marinas@arm.com>
> Cc: Will Deacon <Will.Deacon@arm.com>
> Signed-off-by: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
> ---
> arch/arm64/kernel/Makefile | 1 +
> arch/arm64/kernel/msi.c | 41 ++++++++++++++++++++
> drivers/irqchip/irq-gic-v2m.c | 87 ++++++++++++++++++++++++++++++++++++++-----
> 3 files changed, 119 insertions(+), 10 deletions(-)
> create mode 100644 arch/arm64/kernel/msi.c
>
> diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
> index df7ef87..a921c42 100644
> --- a/arch/arm64/kernel/Makefile
> +++ b/arch/arm64/kernel/Makefile
> @@ -29,6 +29,7 @@ arm64-obj-$(CONFIG_ARM64_CPU_SUSPEND) += sleep.o suspend.o
> arm64-obj-$(CONFIG_JUMP_LABEL) += jump_label.o
> arm64-obj-$(CONFIG_KGDB) += kgdb.o
> arm64-obj-$(CONFIG_EFI) += efi.o efi-stub.o efi-entry.o
> +arm64-obj-$(CONFIG_PCI_MSI) += msi.o
>
> obj-y += $(arm64-obj-y) vdso/
> obj-m += $(arm64-obj-m)
> diff --git a/arch/arm64/kernel/msi.c b/arch/arm64/kernel/msi.c
> new file mode 100644
> index 0000000..a295862
> --- /dev/null
> +++ b/arch/arm64/kernel/msi.c
> @@ -0,0 +1,41 @@
> +/*
> + * ARM64 architectural MSI implemention
> + *
> + * Support for Message Signalelled Interrupts for systems that
> + * implement ARM Generic Interrupt Controller: GICv2m.
> + *
> + * Copyright (C) 2014 Advanced Micro Devices, Inc.
> + * Authors: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2 as published
> + * by the Free Software Foundation.
> + */
> +
> +#include <linux/irq.h>
> +#include <linux/msi.h>
> +#include <linux/pci.h>
> +
> +/*
> + * ARM64 function for seting up MSI irqs.
> + * Based on driver/pci/msi.c: arch_setup_msi_irqs().
> + *
> + * Note:
> + * Current implementation assumes that all interrupt controller used in
> + * ARM64 architecture _MUST_ supports multi-MSI.
> + */
> +int arm64_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
> +{
> + struct msi_desc *entry;
> + int ret;
> +
> + list_for_each_entry(entry, &dev->msi_list, list) {
> + ret = arch_setup_msi_irq(dev, entry);
> + if (ret < 0)
> + return ret;
> + if (ret > 0)
> + return -ENOSPC;
> + }
> +
> + return 0;
> +}
> diff --git a/drivers/irqchip/irq-gic-v2m.c b/drivers/irqchip/irq-gic-v2m.c
> index 499bfb9..4ddb47d 100644
> --- a/drivers/irqchip/irq-gic-v2m.c
> +++ b/drivers/irqchip/irq-gic-v2m.c
> @@ -99,13 +99,26 @@ static void gicv2m_teardown_msi_irq(struct msi_chip *chip, unsigned int irq)
> spin_unlock(&data->msi_cnt_lock);
> }
>
> +static int gicv2m_msi_get_vec_count(struct pci_dev *pdev, struct msi_desc *desc)
> +{
> + int ret = -EINVAL;
> +#ifdef CONFIG_PCI_MSI
> + if (desc->msi_attrib.is_msix)
> + ret = pci_msix_vec_count(pdev);
> + else
> + ret = pci_msi_vec_count(pdev);
> +#endif
> + return ret;
> +}
> +
Since I also have:
https://git.kernel.org/cgit/linux/kernel/git/maz/arm-platforms.git/tree/drivers/irqchip/irq-gic-v3-its.c?h=gicv3/its-split#n1107
Maybe it would be a good thing to make this a core thing (not something
we have to do now though).
> static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> struct pci_dev *pdev,
> struct msi_desc *desc)
> {
> - int irq, avail;
> + int i, irq, nvec, avail;
> struct msi_msg msg;
> phys_addr_t addr;
> + struct msi_desc *entry;
> struct v2m_data *data = container_of(chip, struct v2m_data, msi_chip);
>
> if (!desc) {
> @@ -114,16 +127,70 @@ static int gicv2m_setup_msi_irq(struct msi_chip *chip,
> return -EINVAL;
> }
>
> - avail = alloc_msi_irq(data, 1, &irq);
> - if (avail != 0) {
> - dev_err(&pdev->dev,
> - "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> - return -ENOSPC;
> - }
> + if (desc->msi_attrib.is_msix) {
> + /**
> + * For MSIx:
> + * We allocate one irq at a time
> + */
> + avail = alloc_msi_irq(data, 1, &irq);
> + if (avail != 0) {
> + dev_err(&pdev->dev,
> + "GICv2m: MSI setup failed. Cannnot allocate IRQ\n");
> + return -ENOSPC;
> + }
>
> - irq_set_chip_data(irq, chip);
> - irq_set_msi_desc(irq, desc);
> - irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> + irq_set_chip_data(irq, chip);
> + irq_set_msi_desc(irq, desc);
> + irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
> + } else {
> + /**
> + * For MSI and Multi-MSI:
> + * All requested irqs are allocated and setup at
> + * once. Subsequent calls to this function would simply return
> + * success. This is to avoid having to implement a separate
> + * function for setting up multiple irqs.
> + */
> + BUG_ON(list_empty(&pdev->msi_list));
> + WARN_ON(!list_is_singular(&pdev->msi_list));
> +
> + nvec = gicv2m_msi_get_vec_count(pdev, desc);
> + if (WARN_ON(nvec <= 0))
> + return nvec;
> +
> + entry = list_first_entry(&pdev->msi_list,
> + struct msi_desc, list);
> +
> + if ((nvec > 1) && (entry->msi_attrib.multiple))
> + return 0;
> +
> + avail = alloc_msi_irq(data, nvec, &irq);
> + if (avail != 0) {
> + dev_err(&pdev->dev,
> + "GICv2m: Failed to allocate %d irqs.\n", nvec);
> + return avail;
> + }
> +
> + if (nvec > 1) {
> + /* Set lowest of the new interrupts assigned
> + * to the PCI device
> + */
> + entry->nvec_used = nvec;
> + entry->msi_attrib.multiple = ilog2(
> + __roundup_pow_of_two(nvec));
> + }
> +
> + for (i = 0; i < nvec; i++) {
> + irq_set_chip_data(irq+i, chip);
> + if (irq_set_msi_desc_off(irq, i, entry)) {
> + dev_err(&pdev->dev,
> + "GICv2m: Failed to set up MSI irq %d\n",
> + (irq+i));
> + return -EINVAL;
> + }
> +
> + irq_set_irq_type((irq+i), IRQ_TYPE_EDGE_RISING);
> + }
> + }
>
> addr = data->res.start + V2M_MSI_SETSPI_NS;
> msg.address_hi = (u32)(addr >> 32);
Overall, I think you should move this "msi > 1" code directly into the
first patch (not sure the intermediate implementation adds much to the
understanding of the whole driver).
You can then have the arm64-specific code as your second patch.
Thanks,
M.
--
Jazz is not dead. It just smells funny.
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2014-09-15 19:41 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-09-14 6:30 [PATCH 0/2 V6] irqchip: gic: Introduce ARM GICv2m MSI(-X) support suravee.suthikulpanit
2014-09-14 6:30 ` suravee.suthikulpanit at amd.com
2014-09-14 6:30 ` suravee.suthikulpanit
2014-09-14 6:30 ` [PATCH 1/2 V6] irqchip: gic: Add supports for ARM GICv2m MSI(-X) suravee.suthikulpanit
2014-09-14 6:30 ` suravee.suthikulpanit at amd.com
2014-09-14 6:30 ` suravee.suthikulpanit
2014-09-15 19:26 ` Marc Zyngier
2014-09-15 19:26 ` Marc Zyngier
2014-09-15 19:26 ` Marc Zyngier
2014-09-15 19:26 ` Marc Zyngier
2014-09-14 6:30 ` [PATCH 2/2 V6] irqchip: gicv2m: Add support for multiple MSI for ARM64 GICv2m suravee.suthikulpanit
2014-09-14 6:30 ` suravee.suthikulpanit at amd.com
2014-09-14 6:30 ` suravee.suthikulpanit
2014-09-15 19:41 ` Marc Zyngier
2014-09-15 19:41 ` Marc Zyngier
2014-09-15 19:41 ` Marc Zyngier
2014-09-15 19:41 ` Marc Zyngier
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.