[PATCHv3 0/6] Add TI PRUSS Local Interrupt Controller IRQChip driver

[PATCHv3 0/6] Add TI PRUSS Local Interrupt Controller IRQChip driver

[Grzegorz Jaszczyk]

Hi All,

The following is a v3 version of the series [1][2] that adds an IRQChip
driver for the local interrupt controller present within a Programmable
Real-Time Unit and Industrial Communication Subsystem (PRU-ICSS) present on a
number of TI SoCs including OMAP architecture based AM335x, AM437x, AM57xx SoCs,
Keystone 2 architecture based 66AK2G SoCs, Davinci architecture based
OMAP-L138/DA850 SoCs and the latest K3 architecture based AM65x and J721E SoCs.
Please see the v1 cover-letter [1] for details about the features of this
interrupt controller.  More details can be found in any of the supported SoC
TRMs.  Eg: Chapter 30.1.6 of AM5728 TRM [3]

Please see the individual patches for exact changes in each patch, following are
the main changes from v2:
 - Convert dt-binding to YAML (patch #1).
 - Address comments from Marc Zyngier regarding patch #2 and update following
   patches due to those changes.
 - Dropped the custom helper functions used for interrupt configuration outside
   of irq driver [4]. Introduce new patch (patch 6) which uses xlate and irq
   domain mapping functionality in order to map system event through 2 levels of
   many-to-one mapping i.e.  events to channel mapping and channels to host
   interrupts.

[1] https://patchwork.kernel.org/cover/11034561/
[2] https://patchwork.kernel.org/cover/11069749/
[3] http://www.ti.com/lit/pdf/spruhz6
[4] https://patchwork.kernel.org/patch/11069751/

Best regards
Grzegorz

David Lechner (1):
  irqchip/irq-pruss-intc: Implement irq_{get,set}_irqchip_state ops

Grzegorz Jaszczyk (1):
  irqchip/irq-pruss-intc: Add event mapping support

Suman Anna (4):
  dt-bindings: irqchip: Add PRU-ICSS interrupt controller bindings
  irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS
    interrupts
  irqchip/irq-pruss-intc: Add support for shared and invalid interrupts
  irqchip/irq-pruss-intc: Add support for ICSSG INTC on K3 SoCs

 .../interrupt-controller/ti,pruss-intc.yaml        | 135 ++++
 drivers/irqchip/Kconfig                            |  10 +
 drivers/irqchip/Makefile                           |   1 +
 drivers/irqchip/irq-pruss-intc.c                   | 717 +++++++++++++++++++++
 4 files changed, 863 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml
 create mode 100644 drivers/irqchip/irq-pruss-intc.c

-- 
2.7.4

[PATCHv3 1/6] dt-bindings: irqchip: Add PRU-ICSS interrupt controller bindings

[Grzegorz Jaszczyk]

From: Suman Anna <s-anna@ti.com>

The Programmable Real-Time Unit and Industrial Communication Subsystem
(PRU-ICSS or simply PRUSS) contains an interrupt controller (INTC) that
can handle various system input events and post interrupts back to the
device-level initiators. The INTC can support upto 64 input events on
most SoCs with individual control configuration and h/w prioritization.
These events are mapped onto 10 output interrupt lines through two levels
of many-to-one mapping support. Different interrupt lines are routed to
the individual PRU cores or to the host CPU or to other PRUSS instances.

The K3 AM65x and J721E SoCs have the next generation of the PRU-ICSS IP,
commonly called ICSSG. The ICSSG interrupt controller on K3 SoCs provide
a higher number of host interrupts (20 vs 10) and can handle an increased
number of input events (160 vs 64) from various SoC interrupt sources.

Add the bindings document for these interrupt controllers on all the
applicable SoCs. It covers the OMAP architecture SoCs - AM33xx, AM437x
and AM57xx; the Keystone 2 architecture based 66AK2G SoC; the Davinci
architecture based OMAPL138 SoCs, and the K3 architecture based AM65x
and J721E SoCs.

Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Andrew F. Davis <afd@ti.com>
Signed-off-by: Roger Quadros <rogerq@ti.com>
Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
Reviewed-by: Lee Jones <lee.jones@linaro.org>
---
v2->v3:
- Convert dt-binding to YAML
v1->v2:
- https://patchwork.kernel.org/patch/11069767/
---
 .../interrupt-controller/ti,pruss-intc.yaml        | 135 +++++++++++++++++++++
 1 file changed, 135 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml

diff --git a/Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml b/Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml
new file mode 100644
index 0000000..7fe4b95
--- /dev/null
+++ b/Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml
@@ -0,0 +1,135 @@
+# SPDX-License-Identifier: (GPL-2.0-only or BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/interrupt-controller/ti,pruss-intc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: TI PRU-ICSS Local Interrupt Controller
+
+maintainers:
+  - Suman Anna <s-anna@ti.com>
+
+description: |
+  Each PRU-ICSS has a single interrupt controller instance that is common
+  to all the PRU cores. Most interrupt controllers can route 64 input events
+  which are then mapped to 10 possible output interrupts through two levels
+  of mapping. The input events can be triggered by either the PRUs and/or
+  various other PRUSS internal and external peripherals. The first 2 output
+  interrupts (0, 1) are fed exclusively to the internal PRU cores, with the
+  remaining 8 (2 through 9) connected to external interrupt controllers
+  including the MPU and/or other PRUSS instances, DSPs or devices.
+
+  The properties "ti,irqs-reserved" and "ti,irqs-shared" are used for denoting
+  the connection differences on the output interrupts 2 through 9. If neither
+  of these properties are defined, it implies that all the PRUSS INTC output
+  interrupts 2 through 9 (host_intr0 through host_intr7) are connected
+  exclusively to the Arm interrupt controller.
+
+  The K3 family of SoCs can handle 160 input events that can be mapped to 20
+  different possible output interrupts. The additional output interrupts (10
+  through 19) are connected to new sub-modules within the ICSSG instances.
+
+  This interrupt-controller node should be defined as a child node of the
+  corresponding PRUSS node. The node should be named "interrupt-controller".
+
+allOf:
+  - $ref: /schemas/interrupt-controller.yaml#
+  - $ref: /schemas/interrupts.yaml#
+
+properties:
+  compatible:
+    enum:
+      - ti,pruss-intc
+      - ti,icssg-intc
+    description: |
+      Use "ti,pruss-intc" for OMAP-L13x/AM18x/DA850 SoCs,
+                              AM335x family of SoCs,
+                              AM437x family of SoCs,
+                              AM57xx family of SoCs
+                              66AK2G family of SoCs
+      Use "ti,icssg-intc" for K3 AM65x & J721E family of SoCs
+
+  reg:
+    items:
+      - description: base address and size for the PRUSS INTC sub-module
+
+  interrupts:
+    minItems: 1
+    maxItems: 8
+    description: |
+      all the interrupts generated towards the main host processor in the SoC.
+      The format depends on the interrupt specifier for the particular SoC's
+      Arm parent interrupt controller. A shared interrupt can be skipped if
+      the desired destination and usage is by a different processor/device.
+
+  interrupt-names:
+    minItems: 1
+    maxItems: 8
+    items:
+      pattern: host_intr[0-7]
+    description: |
+      should use one of the above names for each valid host event interrupt
+      connected to Arm interrupt controller, the name should match the
+      corresponding host event interrupt number
+
+  interrupt-controller: true
+
+  "#interrupt-cells":
+    const: 1
+    description: |
+      Client users shall use the PRU System event number (the interrupt source
+      that the client is interested in) as the value of the interrupts property
+      in their node
+
+  ti,irqs-reserved:
+    $ref: /schemas/types.yaml#definitions/uint8-array
+    description: |
+      an array of 8-bit elements of host interrupts between 0 and 7
+      (corresponding to PRUSS INTC output interrupts 2 through 9) that are
+      not connected to the Arm interrupt controller.
+        Eg: AM437x and 66AK2G SoCs do not have "host_intr5" interrupt connected
+            to MPU
+
+  ti,irqs-shared:
+    $ref: /schemas/types.yaml#definitions/uint8-array
+    description: |
+      an array of 8-bit elements of host interrupts between 0 and 7
+      (corresponding to PRUSS INTC output interrupts 2 through 9) that are
+      also connected to other devices or processors in the SoC.
+         Eg: AM65x and J721E SoCs have "host_intr5", "host_intr6" and
+             "host_intr7" interrupts connected to MPU, and other ICSSG
+             instances
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - interrupt-names
+ - interrupt-controller
+ - "#interrupt-cells"
+
+additionalProperties: false
+
+examples:
+  - |
+    /* AM33xx PRU-ICSS */
+    pruss: pruss@0 {
+        compatible = "ti,am3356-pruss";
+        reg = <0x0 0x80000>;
+        #address-cells = <1>;
+        #size-cells = <1>;
+        ranges;
+
+        pruss_intc: interrupt-controller@20000 {
+            compatible = "ti,pruss-intc";
+            reg = <0x20000 0x2000>;
+            interrupts = <20 21 22 23 24 25 26 27>;
+            interrupt-names = "host_intr0", "host_intr1",
+                              "host_intr2", "host_intr3",
+                              "host_intr4", "host_intr5",
+                              "host_intr6", "host_intr7";
+            interrupt-controller;
+            #interrupt-cells = <1>;
+            ti,irqs-shared = /bits/ 8 <0 6 7>;
+        };
+    };
-- 
2.7.4

[PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Grzegorz Jaszczyk]

From: Suman Anna <s-anna@ti.com>

The Programmable Real-Time Unit Subsystem (PRUSS) contains a local
interrupt controller (INTC) that can handle various system input events
and post interrupts back to the device-level initiators. The INTC can
support upto 64 input events with individual control configuration and
hardware prioritization. These events are mapped onto 10 output interrupt
lines through two levels of many-to-one mapping support. Different
interrupt lines are routed to the individual PRU cores or to the host
CPU, or to other devices on the SoC. Some of these events are sourced
from peripherals or other sub-modules within that PRUSS, while a few
others are sourced from SoC-level peripherals/devices.

The PRUSS INTC platform driver manages this PRUSS interrupt controller
and implements an irqchip driver to provide a Linux standard way for
the PRU client users to enable/disable/ack/re-trigger a PRUSS system
event. The system events to interrupt channels and output interrupts
relies on the mapping configuration provided either through the PRU
firmware blob or via the PRU application's device tree node. The
mappings will be programmed during the boot/shutdown of a PRU core.

The PRUSS INTC module is reference counted during the interrupt
setup phase through the irqchip's irq_request_resources() and
irq_release_resources() ops. This restricts the module from being
removed as long as there are active interrupt users.

The driver currently supports and can be built for OMAP architecture
based AM335x, AM437x and AM57xx SoCs; Keystone2 architecture based
66AK2G SoCs and Davinci architecture based OMAP-L13x/AM18x/DA850 SoCs.
All of these SoCs support 64 system events, 10 interrupt channels and
10 output interrupt lines per PRUSS INTC with a few SoC integration
differences.

NOTE:
Each PRU-ICSS's INTC on AM57xx SoCs is preceded by a Crossbar that
enables multiple external events to be routed to a specific number
of input interrupt events. Any non-default external interrupt event
directed towards PRUSS needs this crossbar to be setup properly.

Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Andrew F. Davis <afd@ti.com>
Signed-off-by: Roger Quadros <rogerq@ti.com>
Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
Reviewed-by: Lee Jones <lee.jones@linaro.org>
---
v2->v3:
- use single irqchip description instead of separately allocating it for
  each pruss_intc
- get rid of unused mutex
- improve error handling
v1->v2:
- https://patchwork.kernel.org/patch/11069771/
---
 drivers/irqchip/Kconfig          |  10 ++
 drivers/irqchip/Makefile         |   1 +
 drivers/irqchip/irq-pruss-intc.c | 307 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 318 insertions(+)
 create mode 100644 drivers/irqchip/irq-pruss-intc.c

diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 29fead2..733d7ec 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -493,6 +493,16 @@ config TI_SCI_INTA_IRQCHIP
 	  If you wish to use interrupt aggregator irq resources managed by the
 	  TI System Controller, say Y here. Otherwise, say N.
 
+config TI_PRUSS_INTC
+	tristate "TI PRU-ICSS Interrupt Controller"
+	depends on ARCH_DAVINCI || SOC_AM33XX || SOC_AM43XX || SOC_DRA7XX || ARCH_KEYSTONE
+	select IRQ_DOMAIN
+	help
+	   This enables support for the PRU-ICSS Local Interrupt Controller
+	   present within a PRU-ICSS subsystem present on various TI SoCs.
+	   The PRUSS INTC enables various interrupts to be routed to multiple
+	   different processors within the SoC.
+
 config RISCV_INTC
 	bool "RISC-V Local Interrupt Controller"
 	depends on RISCV
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 133f9c4..990a106 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -106,6 +106,7 @@ obj-$(CONFIG_MADERA_IRQ)		+= irq-madera.o
 obj-$(CONFIG_LS1X_IRQ)			+= irq-ls1x.o
 obj-$(CONFIG_TI_SCI_INTR_IRQCHIP)	+= irq-ti-sci-intr.o
 obj-$(CONFIG_TI_SCI_INTA_IRQCHIP)	+= irq-ti-sci-inta.o
+obj-$(CONFIG_TI_PRUSS_INTC)		+= irq-pruss-intc.o
 obj-$(CONFIG_LOONGSON_LIOINTC)		+= irq-loongson-liointc.o
 obj-$(CONFIG_LOONGSON_HTPIC)		+= irq-loongson-htpic.o
 obj-$(CONFIG_LOONGSON_HTVEC)		+= irq-loongson-htvec.o
diff --git a/drivers/irqchip/irq-pruss-intc.c b/drivers/irqchip/irq-pruss-intc.c
new file mode 100644
index 0000000..fb3dda3
--- /dev/null
+++ b/drivers/irqchip/irq-pruss-intc.c
@@ -0,0 +1,307 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * PRU-ICSS INTC IRQChip driver for various TI SoCs
+ *
+ * Copyright (C) 2016-2020 Texas Instruments Incorporated - http://www.ti.com/
+ *	Andrew F. Davis <afd@ti.com>
+ *	Suman Anna <s-anna@ti.com>
+ */
+
+#include <linux/irq.h>
+#include <linux/irqchip/chained_irq.h>
+#include <linux/irqdomain.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+
+/*
+ * Number of host interrupts reaching the main MPU sub-system. Note that this
+ * is not the same as the total number of host interrupts supported by the PRUSS
+ * INTC instance
+ */
+#define MAX_NUM_HOST_IRQS	8
+
+/* minimum starting host interrupt number for MPU */
+#define MIN_PRU_HOST_INT	2
+
+/* maximum number of system events */
+#define MAX_PRU_SYS_EVENTS	64
+
+/* PRU_ICSS_INTC registers */
+#define PRU_INTC_REVID		0x0000
+#define PRU_INTC_CR		0x0004
+#define PRU_INTC_GER		0x0010
+#define PRU_INTC_GNLR		0x001c
+#define PRU_INTC_SISR		0x0020
+#define PRU_INTC_SICR		0x0024
+#define PRU_INTC_EISR		0x0028
+#define PRU_INTC_EICR		0x002c
+#define PRU_INTC_HIEISR		0x0034
+#define PRU_INTC_HIDISR		0x0038
+#define PRU_INTC_GPIR		0x0080
+#define PRU_INTC_SRSR0		0x0200
+#define PRU_INTC_SRSR1		0x0204
+#define PRU_INTC_SECR0		0x0280
+#define PRU_INTC_SECR1		0x0284
+#define PRU_INTC_ESR0		0x0300
+#define PRU_INTC_ESR1		0x0304
+#define PRU_INTC_ECR0		0x0380
+#define PRU_INTC_ECR1		0x0384
+#define PRU_INTC_CMR(x)		(0x0400 + (x) * 4)
+#define PRU_INTC_HMR(x)		(0x0800 + (x) * 4)
+#define PRU_INTC_HIPIR(x)	(0x0900 + (x) * 4)
+#define PRU_INTC_SIPR0		0x0d00
+#define PRU_INTC_SIPR1		0x0d04
+#define PRU_INTC_SITR0		0x0d80
+#define PRU_INTC_SITR1		0x0d84
+#define PRU_INTC_HINLR(x)	(0x1100 + (x) * 4)
+#define PRU_INTC_HIER		0x1500
+
+/* HIPIR register bit-fields */
+#define INTC_HIPIR_NONE_HINT	0x80000000
+
+/**
+ * struct pruss_intc - PRUSS interrupt controller structure
+ * @irqs: kernel irq numbers corresponding to PRUSS host interrupts
+ * @base: base virtual address of INTC register space
+ * @domain: irq domain for this interrupt controller
+ */
+struct pruss_intc {
+	unsigned int irqs[MAX_NUM_HOST_IRQS];
+	void __iomem *base;
+	struct irq_domain *domain;
+};
+
+static inline u32 pruss_intc_read_reg(struct pruss_intc *intc, unsigned int reg)
+{
+	return readl_relaxed(intc->base + reg);
+}
+
+static inline void pruss_intc_write_reg(struct pruss_intc *intc,
+					unsigned int reg, u32 val)
+{
+	writel_relaxed(val, intc->base + reg);
+}
+
+static void pruss_intc_init(struct pruss_intc *intc)
+{
+	int i;
+
+	/* configure polarity to active high for all system interrupts */
+	pruss_intc_write_reg(intc, PRU_INTC_SIPR0, 0xffffffff);
+	pruss_intc_write_reg(intc, PRU_INTC_SIPR1, 0xffffffff);
+
+	/* configure type to pulse interrupt for all system interrupts */
+	pruss_intc_write_reg(intc, PRU_INTC_SITR0, 0);
+	pruss_intc_write_reg(intc, PRU_INTC_SITR1, 0);
+
+	/* clear all 16 interrupt channel map registers */
+	for (i = 0; i < 16; i++)
+		pruss_intc_write_reg(intc, PRU_INTC_CMR(i), 0);
+
+	/* clear all 3 host interrupt map registers */
+	for (i = 0; i < 3; i++)
+		pruss_intc_write_reg(intc, PRU_INTC_HMR(i), 0);
+}
+
+static void pruss_intc_irq_ack(struct irq_data *data)
+{
+	struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
+	unsigned int hwirq = data->hwirq;
+
+	pruss_intc_write_reg(intc, PRU_INTC_SICR, hwirq);
+}
+
+static void pruss_intc_irq_mask(struct irq_data *data)
+{
+	struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
+	unsigned int hwirq = data->hwirq;
+
+	pruss_intc_write_reg(intc, PRU_INTC_EICR, hwirq);
+}
+
+static void pruss_intc_irq_unmask(struct irq_data *data)
+{
+	struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
+	unsigned int hwirq = data->hwirq;
+
+	pruss_intc_write_reg(intc, PRU_INTC_EISR, hwirq);
+}
+
+static int pruss_intc_irq_reqres(struct irq_data *data)
+{
+	if (!try_module_get(THIS_MODULE))
+		return -ENODEV;
+
+	return 0;
+}
+
+static void pruss_intc_irq_relres(struct irq_data *data)
+{
+	module_put(THIS_MODULE);
+}
+
+static struct irq_chip pruss_irqchip = {
+	.name = "pruss-intc",
+	.irq_ack = pruss_intc_irq_ack,
+	.irq_mask = pruss_intc_irq_mask,
+	.irq_unmask = pruss_intc_irq_unmask,
+	.irq_request_resources = pruss_intc_irq_reqres,
+	.irq_release_resources = pruss_intc_irq_relres,
+};
+
+static int pruss_intc_irq_domain_map(struct irq_domain *d, unsigned int virq,
+				     irq_hw_number_t hw)
+{
+	struct pruss_intc *intc = d->host_data;
+
+	irq_set_chip_data(virq, intc);
+	irq_set_chip_and_handler(virq, &pruss_irqchip, handle_level_irq);
+
+	return 0;
+}
+
+static void pruss_intc_irq_domain_unmap(struct irq_domain *d, unsigned int virq)
+{
+	irq_set_chip_and_handler(virq, NULL, NULL);
+	irq_set_chip_data(virq, NULL);
+}
+
+static const struct irq_domain_ops pruss_intc_irq_domain_ops = {
+	.xlate	= irq_domain_xlate_onecell,
+	.map	= pruss_intc_irq_domain_map,
+	.unmap	= pruss_intc_irq_domain_unmap,
+};
+
+static void pruss_intc_irq_handler(struct irq_desc *desc)
+{
+	unsigned int irq = irq_desc_get_irq(desc);
+	struct irq_chip *chip = irq_desc_get_chip(desc);
+	struct pruss_intc *intc = irq_get_handler_data(irq);
+	u32 hipir;
+	unsigned int virq;
+	int i, hwirq;
+
+	chained_irq_enter(chip, desc);
+
+	/* find our host irq number */
+	for (i = 0; i < MAX_NUM_HOST_IRQS; i++)
+		if (intc->irqs[i] == irq)
+			break;
+	if (i == MAX_NUM_HOST_IRQS)
+		goto err;
+
+	i += MIN_PRU_HOST_INT;
+
+	/* get highest priority pending PRUSS system event */
+	hipir = pruss_intc_read_reg(intc, PRU_INTC_HIPIR(i));
+	while (!(hipir & INTC_HIPIR_NONE_HINT)) {
+		hwirq = hipir & GENMASK(9, 0);
+		virq = irq_linear_revmap(intc->domain, hwirq);
+
+		/*
+		 * NOTE: manually ACK any system events that do not have a
+		 * handler mapped yet
+		 */
+		if (WARN_ON(!virq))
+			pruss_intc_write_reg(intc, PRU_INTC_SICR, hwirq);
+		else
+			generic_handle_irq(virq);
+
+		/* get next system event */
+		hipir = pruss_intc_read_reg(intc, PRU_INTC_HIPIR(i));
+	}
+err:
+	chained_irq_exit(chip, desc);
+}
+
+static int pruss_intc_probe(struct platform_device *pdev)
+{
+	static const char * const irq_names[MAX_NUM_HOST_IRQS] = {
+		"host_intr0", "host_intr1", "host_intr2", "host_intr3",
+		"host_intr4", "host_intr5", "host_intr6", "host_intr7", };
+	struct device *dev = &pdev->dev;
+	struct pruss_intc *intc;
+	int i, irq;
+
+	intc = devm_kzalloc(dev, sizeof(*intc), GFP_KERNEL);
+	if (!intc)
+		return -ENOMEM;
+	platform_set_drvdata(pdev, intc);
+
+	intc->base = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(intc->base)) {
+		dev_err(dev, "failed to parse and map intc memory resource\n");
+		return PTR_ERR(intc->base);
+	}
+
+	pruss_intc_init(intc);
+
+	/* always 64 events */
+	intc->domain = irq_domain_add_linear(dev->of_node, MAX_PRU_SYS_EVENTS,
+					     &pruss_intc_irq_domain_ops, intc);
+	if (!intc->domain)
+		return -ENOMEM;
+
+	for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
+		irq = platform_get_irq_byname(pdev, irq_names[i]);
+		if (irq <= 0) {
+			dev_err(dev, "platform_get_irq_byname failed for %s : %d\n",
+				irq_names[i], irq);
+			goto fail_irq;
+		}
+
+		intc->irqs[i] = irq;
+		irq_set_handler_data(irq, intc);
+		irq_set_chained_handler(irq, pruss_intc_irq_handler);
+	}
+
+	return 0;
+
+fail_irq:
+	while (--i >= 0)
+		irq_set_chained_handler_and_data(intc->irqs[i], NULL, NULL);
+
+	irq_domain_remove(intc->domain);
+
+	return irq;
+}
+
+static int pruss_intc_remove(struct platform_device *pdev)
+{
+	struct pruss_intc *intc = platform_get_drvdata(pdev);
+	unsigned int hwirq;
+	int i;
+
+	for (i = 0; i < MAX_NUM_HOST_IRQS; i++)
+		irq_set_chained_handler_and_data(intc->irqs[i], NULL, NULL);
+
+	for (hwirq = 0; hwirq < MAX_PRU_SYS_EVENTS; hwirq++)
+		irq_dispose_mapping(irq_find_mapping(intc->domain, hwirq));
+
+	irq_domain_remove(intc->domain);
+
+	return 0;
+}
+
+static const struct of_device_id pruss_intc_of_match[] = {
+	{ .compatible = "ti,pruss-intc", },
+	{ /* sentinel */ },
+};
+MODULE_DEVICE_TABLE(of, pruss_intc_of_match);
+
+static struct platform_driver pruss_intc_driver = {
+	.driver = {
+		.name = "pruss-intc",
+		.of_match_table = pruss_intc_of_match,
+		.suppress_bind_attrs = true,
+	},
+	.probe  = pruss_intc_probe,
+	.remove = pruss_intc_remove,
+};
+module_platform_driver(pruss_intc_driver);
+
+MODULE_AUTHOR("Andrew F. Davis <afd@ti.com>");
+MODULE_AUTHOR("Suman Anna <s-anna@ti.com>");
+MODULE_DESCRIPTION("TI PRU-ICSS INTC Driver");
+MODULE_LICENSE("GPL v2");
-- 
2.7.4

[PATCHv3 3/6] irqchip/irq-pruss-intc: Add support for shared and invalid interrupts

[Grzegorz Jaszczyk]

From: Suman Anna <s-anna@ti.com>

The PRUSS INTC has a fixed number of output interrupt lines that are
connected to a number of processors or other PRUSS instances or other
devices (like DMA) on the SoC. The output interrupt lines 2 through 9
are usually connected to the main Arm host processor and are referred
to as host interrupts 0 through 7 from ARM/MPU perspective.

All of these 8 host interrupts are not always exclusively connected
to the Arm interrupt controller. Some SoCs have some interrupt lines
not connected to the Arm interrupt controller at all, while a few others
have the interrupt lines connected to multiple processors in which they
need to be partitioned as per SoC integration needs. For example, AM437x
and 66AK2G SoCs have 2 PRUSS instances each and have the host interrupt 5
connected to the other PRUSS, while AM335x has host interrupt 0 shared
between MPU and TSC_ADC and host interrupts 6 & 7 shared between MPU and
a DMA controller.

Add support to the PRUSS INTC driver to allow both these shared and
invalid interrupts by not returning a failure if any of these interrupts
are skipped from the corresponding INTC DT node.

Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
---
v2->v3:
- Extra checks for (intc->irqs[i]) in error/remove path was moved from
  "irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS
  interrupts" to this patch
v1->v2:
- https://patchwork.kernel.org/patch/11069757/
---
 drivers/irqchip/irq-pruss-intc.c | 73 +++++++++++++++++++++++++++++++++++++---
 1 file changed, 68 insertions(+), 5 deletions(-)

diff --git a/drivers/irqchip/irq-pruss-intc.c b/drivers/irqchip/irq-pruss-intc.c
index fb3dda3..49c936f 100644
--- a/drivers/irqchip/irq-pruss-intc.c
+++ b/drivers/irqchip/irq-pruss-intc.c
@@ -65,11 +65,15 @@
  * @irqs: kernel irq numbers corresponding to PRUSS host interrupts
  * @base: base virtual address of INTC register space
  * @domain: irq domain for this interrupt controller
+ * @shared_intr: bit-map denoting if the MPU host interrupt is shared
+ * @invalid_intr: bit-map denoting if host interrupt is not connected to MPU
  */
 struct pruss_intc {
 	unsigned int irqs[MAX_NUM_HOST_IRQS];
 	void __iomem *base;
 	struct irq_domain *domain;
+	u16 shared_intr;
+	u16 invalid_intr;
 };
 
 static inline u32 pruss_intc_read_reg(struct pruss_intc *intc, unsigned int reg)
@@ -222,7 +226,8 @@ static int pruss_intc_probe(struct platform_device *pdev)
 		"host_intr4", "host_intr5", "host_intr6", "host_intr7", };
 	struct device *dev = &pdev->dev;
 	struct pruss_intc *intc;
-	int i, irq;
+	int i, irq, count;
+	u8 temp_intr[MAX_NUM_HOST_IRQS] = { 0 };
 
 	intc = devm_kzalloc(dev, sizeof(*intc), GFP_KERNEL);
 	if (!intc)
@@ -235,6 +240,52 @@ static int pruss_intc_probe(struct platform_device *pdev)
 		return PTR_ERR(intc->base);
 	}
 
+	count = of_property_read_variable_u8_array(dev->of_node,
+						   "ti,irqs-reserved",
+						   temp_intr, 0,
+						   MAX_NUM_HOST_IRQS);
+	/*
+	 * The irqs-reserved is used only for some SoC's therefore not having
+	 * this property is still valid
+	 */
+	if (count == -EINVAL)
+		count = 0;
+	if (count < 0)
+		return count;
+
+	for (i = 0; i < count; i++) {
+		if (temp_intr[i] >= MAX_NUM_HOST_IRQS) {
+			dev_warn(dev, "ignoring invalid reserved irq %d\n",
+				 temp_intr[i]);
+			continue;
+		}
+
+		intc->invalid_intr |= BIT(temp_intr[i]);
+	}
+
+	count = of_property_read_variable_u8_array(dev->of_node,
+						   "ti,irqs-shared",
+						   temp_intr, 0,
+						   MAX_NUM_HOST_IRQS);
+	/*
+	 * The irqs-shared is used only for some SoC's therefore not having
+	 * this property is still valid
+	 */
+	if (count == -EINVAL)
+		count = 0;
+	if (count < 0)
+		return count;
+
+	for (i = 0; i < count; i++) {
+		if (temp_intr[i] >= MAX_NUM_HOST_IRQS) {
+			dev_warn(dev, "ignoring invalid shared irq %d\n",
+				 temp_intr[i]);
+			continue;
+		}
+
+		intc->shared_intr |= BIT(temp_intr[i]);
+	}
+
 	pruss_intc_init(intc);
 
 	/* always 64 events */
@@ -244,8 +295,14 @@ static int pruss_intc_probe(struct platform_device *pdev)
 		return -ENOMEM;
 
 	for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
+		if (intc->invalid_intr & BIT(i))
+			continue;
+
 		irq = platform_get_irq_byname(pdev, irq_names[i]);
 		if (irq <= 0) {
+			if (intc->shared_intr & BIT(i))
+				continue;
+
 			dev_err(dev, "platform_get_irq_byname failed for %s : %d\n",
 				irq_names[i], irq);
 			goto fail_irq;
@@ -259,8 +316,11 @@ static int pruss_intc_probe(struct platform_device *pdev)
 	return 0;
 
 fail_irq:
-	while (--i >= 0)
-		irq_set_chained_handler_and_data(intc->irqs[i], NULL, NULL);
+	while (--i >= 0) {
+		if (intc->irqs[i])
+			irq_set_chained_handler_and_data(intc->irqs[i], NULL,
+							 NULL);
+	}
 
 	irq_domain_remove(intc->domain);
 
@@ -273,8 +333,11 @@ static int pruss_intc_remove(struct platform_device *pdev)
 	unsigned int hwirq;
 	int i;
 
-	for (i = 0; i < MAX_NUM_HOST_IRQS; i++)
-		irq_set_chained_handler_and_data(intc->irqs[i], NULL, NULL);
+	for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
+		if (intc->irqs[i])
+			irq_set_chained_handler_and_data(intc->irqs[i], NULL,
+							 NULL);
+	}
 
 	for (hwirq = 0; hwirq < MAX_PRU_SYS_EVENTS; hwirq++)
 		irq_dispose_mapping(irq_find_mapping(intc->domain, hwirq));
-- 
2.7.4

[PATCHv3 4/6] irqchip/irq-pruss-intc: Implement irq_{get,set}_irqchip_state ops

[Grzegorz Jaszczyk]

From: David Lechner <david@lechnology.com>

This implements the irq_get_irqchip_state and irq_set_irqchip_state
callbacks for the TI PRUSS INTC driver. The set callback can be used
by drivers to "kick" a PRU by enabling a PRU system event.

Example:
     irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, true);

Signed-off-by: David Lechner <david@lechnology.com>
Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
Reviewed-by: Lee Jones <lee.jones@linaro.org>
---
v2->v3:
- Get rid of unnecessary pruss_intc_check_write() and use
  pruss_intc_write_reg directly.
v1->v2:
- https://patchwork.kernel.org/patch/11069769/
---
 drivers/irqchip/irq-pruss-intc.c | 43 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

diff --git a/drivers/irqchip/irq-pruss-intc.c b/drivers/irqchip/irq-pruss-intc.c
index 49c936f..19b3d38 100644
--- a/drivers/irqchip/irq-pruss-intc.c
+++ b/drivers/irqchip/irq-pruss-intc.c
@@ -7,6 +7,7 @@
  *	Suman Anna <s-anna@ti.com>
  */
 
+#include <linux/interrupt.h>
 #include <linux/irq.h>
 #include <linux/irqchip/chained_irq.h>
 #include <linux/irqdomain.h>
@@ -39,8 +40,7 @@
 #define PRU_INTC_HIEISR		0x0034
 #define PRU_INTC_HIDISR		0x0038
 #define PRU_INTC_GPIR		0x0080
-#define PRU_INTC_SRSR0		0x0200
-#define PRU_INTC_SRSR1		0x0204
+#define PRU_INTC_SRSR(x)	(0x0200 + (x) * 4)
 #define PRU_INTC_SECR0		0x0280
 #define PRU_INTC_SECR1		0x0284
 #define PRU_INTC_ESR0		0x0300
@@ -145,6 +145,43 @@ static void pruss_intc_irq_relres(struct irq_data *data)
 	module_put(THIS_MODULE);
 }
 
+static int pruss_intc_irq_get_irqchip_state(struct irq_data *data,
+					    enum irqchip_irq_state which,
+					    bool *state)
+{
+	struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
+	u32 reg, mask, srsr;
+
+	if (which != IRQCHIP_STATE_PENDING)
+		return -EINVAL;
+
+	reg = PRU_INTC_SRSR(data->hwirq / 32);
+	mask = BIT(data->hwirq % 32);
+
+	srsr = pruss_intc_read_reg(intc, reg);
+
+	*state = !!(srsr & mask);
+
+	return 0;
+}
+
+static int pruss_intc_irq_set_irqchip_state(struct irq_data *data,
+					    enum irqchip_irq_state which,
+					    bool state)
+{
+	struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
+
+	if (which != IRQCHIP_STATE_PENDING)
+		return -EINVAL;
+
+	if (state)
+		pruss_intc_write_reg(intc, PRU_INTC_SISR, data->hwirq);
+	else
+		pruss_intc_write_reg(intc, PRU_INTC_SICR, data->hwirq);
+
+	return 0;
+}
+
 static struct irq_chip pruss_irqchip = {
 	.name = "pruss-intc",
 	.irq_ack = pruss_intc_irq_ack,
@@ -152,6 +189,8 @@ static struct irq_chip pruss_irqchip = {
 	.irq_unmask = pruss_intc_irq_unmask,
 	.irq_request_resources = pruss_intc_irq_reqres,
 	.irq_release_resources = pruss_intc_irq_relres,
+	.irq_get_irqchip_state = pruss_intc_irq_get_irqchip_state,
+	.irq_set_irqchip_state = pruss_intc_irq_set_irqchip_state,
 };
 
 static int pruss_intc_irq_domain_map(struct irq_domain *d, unsigned int virq,
-- 
2.7.4

[PATCHv3 5/6] irqchip/irq-pruss-intc: Add support for ICSSG INTC on K3 SoCs

[Grzegorz Jaszczyk]

From: Suman Anna <s-anna@ti.com>

The K3 AM65x and J721E SoCs have the next generation of the PRU-ICSS IP,
commonly called ICSSG. The PRUSS INTC present within the ICSSG supports
more System Events (160 vs 64), more Interrupt Channels and Host Interrupts
(20 vs 10) compared to the previous generation PRUSS INTC instances. The
first 2 and the last 10 of these host interrupt lines are used by the
PRU and other auxiliary cores and sub-modules within the ICSSG, with 8
host interrupts connected to MPU. The host interrupts 5, 6, 7 are also
connected to the other ICSSG instances within the SoC and can be
partitioned as per system integration through the board dts files.

Enhance the PRUSS INTC driver to add support for this ICSSG INTC
instance. This support is added using specific compatible and match
data and updating the code to use this data instead of the current
hard-coded macros. The INTC config structure is updated to use the
higher events and channels on all SoCs, while limiting the actual
processing to only the relevant number of events/channels/interrupts.

Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
---
v2->v3:
- Change patch order: use it directly after "irqchip/irq-pruss-intc:
  Implement irq_{get,set}_irqchip_state ops" and before new
  "irqchip/irq-pruss-intc: Add event mapping support" in order to reduce
  diff.

v1->v2:
- https://patchwork.kernel.org/patch/11069773/
---
 drivers/irqchip/Kconfig          |   2 +-
 drivers/irqchip/irq-pruss-intc.c | 101 ++++++++++++++++++++++++++++-----------
 2 files changed, 74 insertions(+), 29 deletions(-)

diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 733d7ec..9abee84 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -495,7 +495,7 @@ config TI_SCI_INTA_IRQCHIP
 
 config TI_PRUSS_INTC
 	tristate "TI PRU-ICSS Interrupt Controller"
-	depends on ARCH_DAVINCI || SOC_AM33XX || SOC_AM43XX || SOC_DRA7XX || ARCH_KEYSTONE
+	depends on ARCH_DAVINCI || SOC_AM33XX || SOC_AM43XX || SOC_DRA7XX || ARCH_KEYSTONE || ARCH_K3
 	select IRQ_DOMAIN
 	help
 	   This enables support for the PRU-ICSS Local Interrupt Controller
diff --git a/drivers/irqchip/irq-pruss-intc.c b/drivers/irqchip/irq-pruss-intc.c
index 19b3d38..362aa01 100644
--- a/drivers/irqchip/irq-pruss-intc.c
+++ b/drivers/irqchip/irq-pruss-intc.c
@@ -7,6 +7,7 @@
  *	Suman Anna <s-anna@ti.com>
  */
 
+#include <linux/bitmap.h>
 #include <linux/interrupt.h>
 #include <linux/irq.h>
 #include <linux/irqchip/chained_irq.h>
@@ -25,9 +26,6 @@
 /* minimum starting host interrupt number for MPU */
 #define MIN_PRU_HOST_INT	2
 
-/* maximum number of system events */
-#define MAX_PRU_SYS_EVENTS	64
-
 /* PRU_ICSS_INTC registers */
 #define PRU_INTC_REVID		0x0000
 #define PRU_INTC_CR		0x0004
@@ -41,30 +39,42 @@
 #define PRU_INTC_HIDISR		0x0038
 #define PRU_INTC_GPIR		0x0080
 #define PRU_INTC_SRSR(x)	(0x0200 + (x) * 4)
-#define PRU_INTC_SECR0		0x0280
-#define PRU_INTC_SECR1		0x0284
-#define PRU_INTC_ESR0		0x0300
-#define PRU_INTC_ESR1		0x0304
-#define PRU_INTC_ECR0		0x0380
-#define PRU_INTC_ECR1		0x0384
+#define PRU_INTC_SECR(x)	(0x0280 + (x) * 4)
+#define PRU_INTC_ESR(x)		(0x0300 + (x) * 4)
+#define PRU_INTC_ECR(x)		(0x0380 + (x) * 4)
 #define PRU_INTC_CMR(x)		(0x0400 + (x) * 4)
 #define PRU_INTC_HMR(x)		(0x0800 + (x) * 4)
 #define PRU_INTC_HIPIR(x)	(0x0900 + (x) * 4)
-#define PRU_INTC_SIPR0		0x0d00
-#define PRU_INTC_SIPR1		0x0d04
-#define PRU_INTC_SITR0		0x0d80
-#define PRU_INTC_SITR1		0x0d84
+#define PRU_INTC_SIPR(x)	(0x0d00 + (x) * 4)
+#define PRU_INTC_SITR(x)	(0x0d80 + (x) * 4)
 #define PRU_INTC_HINLR(x)	(0x1100 + (x) * 4)
 #define PRU_INTC_HIER		0x1500
 
+/* CMR register bit-field macros */
+#define CMR_EVT_PER_REG		4
+
+/* HMR register bit-field macros */
+#define HMR_CH_PER_REG		4
+
 /* HIPIR register bit-fields */
 #define INTC_HIPIR_NONE_HINT	0x80000000
 
 /**
+ * struct pruss_intc_match_data - match data to handle SoC variations
+ * @num_system_events: number of input system events handled by the PRUSS INTC
+ * @num_host_intrs: number of host interrupts supported by the PRUSS INTC
+ */
+struct pruss_intc_match_data {
+	u8 num_system_events;
+	u8 num_host_intrs;
+};
+
+/**
  * struct pruss_intc - PRUSS interrupt controller structure
  * @irqs: kernel irq numbers corresponding to PRUSS host interrupts
  * @base: base virtual address of INTC register space
  * @domain: irq domain for this interrupt controller
+ * @soc_config: cached PRUSS INTC IP configuration data
  * @shared_intr: bit-map denoting if the MPU host interrupt is shared
  * @invalid_intr: bit-map denoting if host interrupt is not connected to MPU
  */
@@ -72,6 +82,7 @@ struct pruss_intc {
 	unsigned int irqs[MAX_NUM_HOST_IRQS];
 	void __iomem *base;
 	struct irq_domain *domain;
+	const struct pruss_intc_match_data *soc_config;
 	u16 shared_intr;
 	u16 invalid_intr;
 };
@@ -89,22 +100,30 @@ static inline void pruss_intc_write_reg(struct pruss_intc *intc,
 
 static void pruss_intc_init(struct pruss_intc *intc)
 {
+	const struct pruss_intc_match_data *soc_config = intc->soc_config;
 	int i;
+	int num_chnl_map_regs = DIV_ROUND_UP(soc_config->num_system_events,
+					     CMR_EVT_PER_REG);
+	int num_host_intr_regs = DIV_ROUND_UP(soc_config->num_host_intrs,
+					      HMR_CH_PER_REG);
+	int num_event_type_regs =
+			DIV_ROUND_UP(soc_config->num_system_events, 32);
 
-	/* configure polarity to active high for all system interrupts */
-	pruss_intc_write_reg(intc, PRU_INTC_SIPR0, 0xffffffff);
-	pruss_intc_write_reg(intc, PRU_INTC_SIPR1, 0xffffffff);
-
-	/* configure type to pulse interrupt for all system interrupts */
-	pruss_intc_write_reg(intc, PRU_INTC_SITR0, 0);
-	pruss_intc_write_reg(intc, PRU_INTC_SITR1, 0);
+	/*
+	 * configure polarity (SIPR register) to active high and
+	 * type (SITR register) to pulse interrupt for all system events
+	 */
+	for (i = 0; i < num_event_type_regs; i++) {
+		pruss_intc_write_reg(intc, PRU_INTC_SIPR(i), 0xffffffff);
+		pruss_intc_write_reg(intc, PRU_INTC_SITR(i), 0);
+	}
 
-	/* clear all 16 interrupt channel map registers */
-	for (i = 0; i < 16; i++)
+	/* clear all interrupt channel map registers, 4 events per register */
+	for (i = 0; i < num_chnl_map_regs; i++)
 		pruss_intc_write_reg(intc, PRU_INTC_CMR(i), 0);
 
-	/* clear all 3 host interrupt map registers */
-	for (i = 0; i < 3; i++)
+	/* clear all host interrupt map registers, 4 channels per register */
+	for (i = 0; i < num_host_intr_regs; i++)
 		pruss_intc_write_reg(intc, PRU_INTC_HMR(i), 0);
 }
 
@@ -266,11 +285,20 @@ static int pruss_intc_probe(struct platform_device *pdev)
 	struct device *dev = &pdev->dev;
 	struct pruss_intc *intc;
 	int i, irq, count;
+	const struct pruss_intc_match_data *data;
 	u8 temp_intr[MAX_NUM_HOST_IRQS] = { 0 };
+	u8 max_system_events;
+
+	data = of_device_get_match_data(dev);
+	if (!data)
+		return -ENODEV;
+
+	max_system_events = data->num_system_events;
 
 	intc = devm_kzalloc(dev, sizeof(*intc), GFP_KERNEL);
 	if (!intc)
 		return -ENOMEM;
+	intc->soc_config = data;
 	platform_set_drvdata(pdev, intc);
 
 	intc->base = devm_platform_ioremap_resource(pdev, 0);
@@ -327,8 +355,7 @@ static int pruss_intc_probe(struct platform_device *pdev)
 
 	pruss_intc_init(intc);
 
-	/* always 64 events */
-	intc->domain = irq_domain_add_linear(dev->of_node, MAX_PRU_SYS_EVENTS,
+	intc->domain = irq_domain_add_linear(dev->of_node, max_system_events,
 					     &pruss_intc_irq_domain_ops, intc);
 	if (!intc->domain)
 		return -ENOMEM;
@@ -369,6 +396,7 @@ static int pruss_intc_probe(struct platform_device *pdev)
 static int pruss_intc_remove(struct platform_device *pdev)
 {
 	struct pruss_intc *intc = platform_get_drvdata(pdev);
+	u8 max_system_events = intc->soc_config->num_system_events;
 	unsigned int hwirq;
 	int i;
 
@@ -378,7 +406,7 @@ static int pruss_intc_remove(struct platform_device *pdev)
 							 NULL);
 	}
 
-	for (hwirq = 0; hwirq < MAX_PRU_SYS_EVENTS; hwirq++)
+	for (hwirq = 0; hwirq < max_system_events; hwirq++)
 		irq_dispose_mapping(irq_find_mapping(intc->domain, hwirq));
 
 	irq_domain_remove(intc->domain);
@@ -386,8 +414,25 @@ static int pruss_intc_remove(struct platform_device *pdev)
 	return 0;
 }
 
+static const struct pruss_intc_match_data pruss_intc_data = {
+	.num_system_events = 64,
+	.num_host_intrs = 10,
+};
+
+static const struct pruss_intc_match_data icssg_intc_data = {
+	.num_system_events = 160,
+	.num_host_intrs = 20,
+};
+
 static const struct of_device_id pruss_intc_of_match[] = {
-	{ .compatible = "ti,pruss-intc", },
+	{
+		.compatible = "ti,pruss-intc",
+		.data = &pruss_intc_data,
+	},
+	{
+		.compatible = "ti,icssg-intc",
+		.data = &icssg_intc_data,
+	},
 	{ /* sentinel */ },
 };
 MODULE_DEVICE_TABLE(of, pruss_intc_of_match);
-- 
2.7.4

[PATCHv3 6/6] irqchip/irq-pruss-intc: Add event mapping support

[Grzegorz Jaszczyk]

The PRUSS INTC receives a number of system input interrupt source events
and supports individual control configuration and hardware prioritization.
These input events can be mapped to some output host interrupts through 2
levels of many-to-one mapping i.e. events to channel mapping and channels
to host interrupts.

This mapping information is provided through the PRU firmware that is
loaded onto a PRU core/s or through the device tree node of the PRU
application. The mapping configuration is triggered by the PRU
remoteproc driver, and is setup before the PRU core is started and
cleaned up after the PRU core is stopped. This event mapping
configuration logic programs the Channel Map Registers (CMRx) and
Host-Interrupt Map Registers (HMRx) only when a new program is being
loaded/started and the same events and interrupt channels are reset to
zero when stopping a PRU.

Reference counting is used to allow multiple system events to share a
single channel and to allow multiple channels to share a single host
event.

The remoteproc driver can register mappings read from a firmware blob
as shown below.

	struct irq_fwspec fwspec;
	int irq;

	fwspec.fwnode = of_node_to_fwnode(dev->of_node);
	fwspec.param_count = 3;
	fwspec.param[0] = 63; // system event
	fwspec.param[1] = 5;  // channel
	fwspec.param[2] = 6;  // host event

	irq = irq_create_fwspec_mapping(&fwspec);
	if (irq < 0) {
		dev_err(dev, "failed to get irq\n");
		return irq;
	}

Suggested-by: David Lechner <david@lechnology.com>
Signed-off-by: Suman Anna <s-anna@ti.com>
Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
---
v3:
- This patch replaces https://patchwork.kernel.org/patch/11069753/
  according to received feedback. Instead of exporting custom functions
  from interrupt driver, the xlate and irq domain map is used for
  interrupt parsing and mapping.
---
 drivers/irqchip/irq-pruss-intc.c | 265 ++++++++++++++++++++++++++++++++++++++-
 1 file changed, 264 insertions(+), 1 deletion(-)

diff --git a/drivers/irqchip/irq-pruss-intc.c b/drivers/irqchip/irq-pruss-intc.c
index 362aa01..cf40a97 100644
--- a/drivers/irqchip/irq-pruss-intc.c
+++ b/drivers/irqchip/irq-pruss-intc.c
@@ -51,14 +51,42 @@
 #define PRU_INTC_HIER		0x1500
 
 /* CMR register bit-field macros */
+#define CMR_EVT_MAP_MASK	0xf
+#define CMR_EVT_MAP_BITS	8
 #define CMR_EVT_PER_REG		4
 
 /* HMR register bit-field macros */
+#define HMR_CH_MAP_MASK		0xf
+#define HMR_CH_MAP_BITS		8
 #define HMR_CH_PER_REG		4
 
 /* HIPIR register bit-fields */
 #define INTC_HIPIR_NONE_HINT	0x80000000
 
+#define MAX_PRU_SYS_EVENTS 160
+#define MAX_PRU_CHANNELS 20
+
+/**
+ * struct pruss_intc_hwirq_data - additional metadata associated with a PRU
+ * system event
+ * @channel: The PRU INTC channel that the system event should be mapped to
+ * @host: The PRU INTC host that the channel should be mapped to
+ */
+struct pruss_intc_hwirq_data {
+	u8 channel;
+	u8 host;
+};
+
+/**
+ * struct pruss_intc_map_record - keeps track of actual mapping state
+ * @value: The currently mapped value (channel or host)
+ * @ref_count: Keeps track of number of current users of this resource
+ */
+struct pruss_intc_map_record {
+	u8 value;
+	u8 ref_count;
+};
+
 /**
  * struct pruss_intc_match_data - match data to handle SoC variations
  * @num_system_events: number of input system events handled by the PRUSS INTC
@@ -71,18 +99,29 @@ struct pruss_intc_match_data {
 
 /**
  * struct pruss_intc - PRUSS interrupt controller structure
+ * @hwirq_data: table of additional mapping data received from device tree
+ *	or PRU firmware
+ * @event_channel: current state of system event to channel mappings
+ * @channel_host: current state of channel to host mappings
  * @irqs: kernel irq numbers corresponding to PRUSS host interrupts
  * @base: base virtual address of INTC register space
  * @domain: irq domain for this interrupt controller
  * @soc_config: cached PRUSS INTC IP configuration data
+ * @lock: mutex to serialize access to INTC
+ * @dev: PRUSS INTC device pointer
  * @shared_intr: bit-map denoting if the MPU host interrupt is shared
  * @invalid_intr: bit-map denoting if host interrupt is not connected to MPU
  */
 struct pruss_intc {
+	struct pruss_intc_hwirq_data hwirq_data[MAX_PRU_SYS_EVENTS];
+	struct pruss_intc_map_record event_channel[MAX_PRU_SYS_EVENTS];
+	struct pruss_intc_map_record channel_host[MAX_PRU_CHANNELS];
 	unsigned int irqs[MAX_NUM_HOST_IRQS];
 	void __iomem *base;
 	struct irq_domain *domain;
 	const struct pruss_intc_match_data *soc_config;
+	struct mutex lock; /* PRUSS INTC lock */
+	struct device *dev;
 	u16 shared_intr;
 	u16 invalid_intr;
 };
@@ -98,6 +137,165 @@ static inline void pruss_intc_write_reg(struct pruss_intc *intc,
 	writel_relaxed(val, intc->base + reg);
 }
 
+static void pruss_intc_update_cmr(struct pruss_intc *intc, int evt, s8 ch)
+{
+	u32 idx, offset, val;
+
+	idx = evt / CMR_EVT_PER_REG;
+	offset = (evt % CMR_EVT_PER_REG) * CMR_EVT_MAP_BITS;
+
+	val = pruss_intc_read_reg(intc, PRU_INTC_CMR(idx));
+	val &= ~(CMR_EVT_MAP_MASK << offset);
+	val |= ch << offset;
+	pruss_intc_write_reg(intc, PRU_INTC_CMR(idx), val);
+
+	dev_dbg(intc->dev, "SYSEV%u -> CH%d (CMR%d 0x%08x)\n", evt, ch,
+		idx, pruss_intc_read_reg(intc, PRU_INTC_CMR(idx)));
+}
+
+static void pruss_intc_update_hmr(struct pruss_intc *intc, int ch, s8 host)
+{
+	u32 idx, offset, val;
+
+	idx = ch / HMR_CH_PER_REG;
+	offset = (ch % HMR_CH_PER_REG) * HMR_CH_MAP_BITS;
+
+	val = pruss_intc_read_reg(intc, PRU_INTC_HMR(idx));
+	val &= ~(HMR_CH_MAP_MASK << offset);
+	val |= host << offset;
+	pruss_intc_write_reg(intc, PRU_INTC_HMR(idx), val);
+
+	dev_dbg(intc->dev, "CH%d -> HOST%d (HMR%d 0x%08x)\n", ch, host, idx,
+		pruss_intc_read_reg(intc, PRU_INTC_HMR(idx)));
+}
+
+/**
+ * pruss_intc_map() - configure the PRUSS INTC
+ * @intc: PRUSS interrupt controller pointer
+ * @hwirq: the system event number
+ *
+ * Configures the PRUSS INTC with the provided configuration from the one
+ * parsed in the xlate function. Any existing event to channel mappings or
+ * channel to host interrupt mappings are checked to make sure there are no
+ * conflicting configuration between both the PRU cores.
+ *
+ * Returns 0 on success, or a suitable error code otherwise
+ */
+static int pruss_intc_map(struct pruss_intc *intc, unsigned long hwirq)
+{
+	struct device *dev = intc->dev;
+	int ret = 0;
+	u8 ch, host, reg_idx;
+	u32 val;
+
+	if (hwirq >= intc->soc_config->num_system_events)
+		return -EINVAL;
+
+	mutex_lock(&intc->lock);
+
+	ch = intc->hwirq_data[hwirq].channel;
+	host = intc->hwirq_data[hwirq].host;
+
+	/* check if sysevent already assigned */
+	if (intc->event_channel[hwirq].ref_count > 0 &&
+	    intc->event_channel[hwirq].value != ch) {
+		dev_err(dev, "event %lu (req. channel %d) already assigned to channel %d\n",
+			hwirq, ch, intc->event_channel[hwirq].value);
+		ret = -EBUSY;
+		goto unlock;
+	}
+
+	/* check if channel already assigned */
+	if (intc->channel_host[ch].ref_count > 0 &&
+	    intc->channel_host[ch].value != host) {
+		dev_err(dev, "channel %d (req. host %d) already assigned to host %d\n",
+			ch, host, intc->channel_host[ch].value);
+		ret = -EBUSY;
+		goto unlock;
+	}
+
+	if (++intc->event_channel[hwirq].ref_count == 1) {
+		intc->event_channel[hwirq].value = ch;
+
+		pruss_intc_update_cmr(intc, hwirq, ch);
+
+		reg_idx = hwirq / 32;
+		val = BIT(hwirq  % 32);
+
+		/* clear and enable system event */
+		pruss_intc_write_reg(intc, PRU_INTC_ESR(reg_idx), val);
+		pruss_intc_write_reg(intc, PRU_INTC_SECR(reg_idx), val);
+	}
+
+	if (++intc->channel_host[ch].ref_count == 1) {
+		intc->channel_host[ch].value = host;
+
+		pruss_intc_update_hmr(intc, ch, host);
+
+		/* enable host interrupts */
+		pruss_intc_write_reg(intc, PRU_INTC_HIEISR, host);
+	}
+
+	dev_dbg(dev, "mapped system_event = %lu channel = %d host = %d",
+		 hwirq, ch, host);
+
+	/* global interrupt enable */
+	pruss_intc_write_reg(intc, PRU_INTC_GER, 1);
+
+unlock:
+	mutex_unlock(&intc->lock);
+	return ret;
+}
+
+/**
+ * pruss_intc_unmap() - unconfigure the PRUSS INTC
+ * @intc: PRUSS interrupt controller pointer
+ * @hwirq: the system event number
+ *
+ * Undo whatever was done in pruss_intc_map() for a PRU core.
+ * Mappings are reference counted, so resources are only disabled when there
+ * are no longer any users.
+ */
+static void pruss_intc_unmap(struct pruss_intc *intc, unsigned long hwirq)
+{
+	u8 ch, host, reg_idx;
+	u32 val;
+
+	if (hwirq >= intc->soc_config->num_system_events)
+		return;
+
+	mutex_lock(&intc->lock);
+
+	ch = intc->event_channel[hwirq].value;
+	host = intc->channel_host[ch].value;
+
+	if (--intc->channel_host[ch].ref_count == 0) {
+		/* disable host interrupts */
+		pruss_intc_write_reg(intc, PRU_INTC_HIDISR, host);
+
+		/* clear the map using reset value 0 */
+		pruss_intc_update_hmr(intc, ch, 0);
+	}
+
+	if (--intc->event_channel[hwirq].ref_count == 0) {
+		reg_idx = hwirq / 32;
+		val = BIT(hwirq  % 32);
+
+		/* disable system events */
+		pruss_intc_write_reg(intc, PRU_INTC_ECR(reg_idx), val);
+		/* clear any pending status */
+		pruss_intc_write_reg(intc, PRU_INTC_SECR(reg_idx), val);
+
+		/* clear the map using reset value 0 */
+		pruss_intc_update_cmr(intc, hwirq, 0);
+	}
+
+	dev_dbg(intc->dev, "unmapped system_event = %lu channel = %d host = %d\n",
+		hwirq, ch, host);
+
+	mutex_unlock(&intc->lock);
+}
+
 static void pruss_intc_init(struct pruss_intc *intc)
 {
 	const struct pruss_intc_match_data *soc_config = intc->soc_config;
@@ -212,10 +410,67 @@ static struct irq_chip pruss_irqchip = {
 	.irq_set_irqchip_state = pruss_intc_irq_set_irqchip_state,
 };
 
+static int
+pruss_intc_irq_domain_xlate(struct irq_domain *d, struct device_node *node,
+			    const u32 *intspec, unsigned int intsize,
+			    unsigned long *out_hwirq, unsigned int *out_type)
+{
+	struct pruss_intc *intc = d->host_data;
+	struct device *dev = intc->dev;
+	int sys_event, channel, host;
+
+	if (intsize == 1) {
+		/*
+		 * In case of short version (intsize == 1) verify if sysevent
+		 * already mapped to channel/host irq if not return error
+		 */
+		sys_event = intspec[0];
+		if (intc->event_channel[sys_event].ref_count)
+			goto already_mapped;
+		else
+			return -EINVAL;
+	}
+
+	if (intsize < 3)
+		return -EINVAL;
+
+	sys_event = intspec[0];
+	if (sys_event < 0 || sys_event >= intc->soc_config->num_system_events) {
+		dev_err(dev, "not valid event number\n");
+		return -EINVAL;
+	}
+
+	channel = intspec[1];
+	if (channel < 0 || channel >= intc->soc_config->num_host_intrs) {
+		dev_err(dev, "not valid channel number");
+		return -EINVAL;
+	}
+
+	host = intspec[2];
+	if (host < 0 || host >= intc->soc_config->num_host_intrs) {
+		dev_err(dev, "not valid host irq number\n");
+		return -EINVAL;
+	}
+
+	intc->hwirq_data[sys_event].channel = channel;
+	intc->hwirq_data[sys_event].host = host;
+
+already_mapped:
+	*out_hwirq = sys_event;
+	*out_type = IRQ_TYPE_NONE;
+
+	return 0;
+}
+
 static int pruss_intc_irq_domain_map(struct irq_domain *d, unsigned int virq,
 				     irq_hw_number_t hw)
 {
 	struct pruss_intc *intc = d->host_data;
+	int err;
+
+	err = pruss_intc_map(intc, hw);
+	if (err < 0)
+		return err;
 
 	irq_set_chip_data(virq, intc);
 	irq_set_chip_and_handler(virq, &pruss_irqchip, handle_level_irq);
@@ -225,12 +480,16 @@ static int pruss_intc_irq_domain_map(struct irq_domain *d, unsigned int virq,
 
 static void pruss_intc_irq_domain_unmap(struct irq_domain *d, unsigned int virq)
 {
+	struct pruss_intc *intc = d->host_data;
+	unsigned long hwirq = irqd_to_hwirq(irq_get_irq_data(virq));
+
 	irq_set_chip_and_handler(virq, NULL, NULL);
 	irq_set_chip_data(virq, NULL);
+	pruss_intc_unmap(intc, hwirq);
 }
 
 static const struct irq_domain_ops pruss_intc_irq_domain_ops = {
-	.xlate	= irq_domain_xlate_onecell,
+	.xlate	= pruss_intc_irq_domain_xlate,
 	.map	= pruss_intc_irq_domain_map,
 	.unmap	= pruss_intc_irq_domain_unmap,
 };
@@ -298,6 +557,8 @@ static int pruss_intc_probe(struct platform_device *pdev)
 	intc = devm_kzalloc(dev, sizeof(*intc), GFP_KERNEL);
 	if (!intc)
 		return -ENOMEM;
+
+	intc->dev = dev;
 	intc->soc_config = data;
 	platform_set_drvdata(pdev, intc);
 
@@ -355,6 +616,8 @@ static int pruss_intc_probe(struct platform_device *pdev)
 
 	pruss_intc_init(intc);
 
+	mutex_init(&intc->lock);
+
 	intc->domain = irq_domain_add_linear(dev->of_node, max_system_events,
 					     &pruss_intc_irq_domain_ops, intc);
 	if (!intc->domain)
-- 
2.7.4

Re: [PATCHv3 6/6] irqchip/irq-pruss-intc: Add event mapping support

[Suman Anna]

Hi Greg,

On 7/2/20 9:17 AM, Grzegorz Jaszczyk wrote:
> The PRUSS INTC receives a number of system input interrupt source events
> and supports individual control configuration and hardware prioritization.
> These input events can be mapped to some output host interrupts through 2
> levels of many-to-one mapping i.e. events to channel mapping and channels
> to host interrupts.
> 
> This mapping information is provided through the PRU firmware that is
> loaded onto a PRU core/s or through the device tree node of the PRU
> application. The mapping configuration is triggered by the PRU
> remoteproc driver, and is setup before the PRU core is started and
> cleaned up after the PRU core is stopped. This event mapping
> configuration logic programs the Channel Map Registers (CMRx) and
> Host-Interrupt Map Registers (HMRx) only when a new program is being
> loaded/started and the same events and interrupt channels are reset to
> zero when stopping a PRU.
> 
> Reference counting is used to allow multiple system events to share a
> single channel and to allow multiple channels to share a single host
> event.
> 
> The remoteproc driver can register mappings read from a firmware blob
> as shown below.
> 
> 	struct irq_fwspec fwspec;
> 	int irq;
> 
> 	fwspec.fwnode = of_node_to_fwnode(dev->of_node);
> 	fwspec.param_count = 3;
> 	fwspec.param[0] = 63; // system event
> 	fwspec.param[1] = 5;  // channel
> 	fwspec.param[2] = 6;  // host event
> 
> 	irq = irq_create_fwspec_mapping(&fwspec);
> 	if (irq < 0) {
> 		dev_err(dev, "failed to get irq\n");
> 		return irq;
> 	}
> 
> Suggested-by: David Lechner <david@lechnology.com>
> Signed-off-by: Suman Anna <s-anna@ti.com>
> Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> ---
> v3:
> - This patch replaces https://patchwork.kernel.org/patch/11069753/
>    according to received feedback. Instead of exporting custom functions
>    from interrupt driver, the xlate and irq domain map is used for
>    interrupt parsing and mapping.

Thanks for reworking this. Only have couple of very minor comments below.

> ---
>   drivers/irqchip/irq-pruss-intc.c | 265 ++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 264 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/irqchip/irq-pruss-intc.c b/drivers/irqchip/irq-pruss-intc.c
> index 362aa01..cf40a97 100644
> --- a/drivers/irqchip/irq-pruss-intc.c
> +++ b/drivers/irqchip/irq-pruss-intc.c
> @@ -51,14 +51,42 @@
>   #define PRU_INTC_HIER		0x1500
>   
>   /* CMR register bit-field macros */
> +#define CMR_EVT_MAP_MASK	0xf
> +#define CMR_EVT_MAP_BITS	8
>   #define CMR_EVT_PER_REG		4
>   
>   /* HMR register bit-field macros */
> +#define HMR_CH_MAP_MASK		0xf
> +#define HMR_CH_MAP_BITS		8
>   #define HMR_CH_PER_REG		4
>   
>   /* HIPIR register bit-fields */
>   #define INTC_HIPIR_NONE_HINT	0x80000000
>   
> +#define MAX_PRU_SYS_EVENTS 160
> +#define MAX_PRU_CHANNELS 20
> +
> +/**
> + * struct pruss_intc_hwirq_data - additional metadata associated with a PRU
> + * system event
> + * @channel: The PRU INTC channel that the system event should be mapped to
> + * @host: The PRU INTC host that the channel should be mapped to
> + */
> +struct pruss_intc_hwirq_data {
> +	u8 channel;
> +	u8 host;
> +};
> +
> +/**
> + * struct pruss_intc_map_record - keeps track of actual mapping state
> + * @value: The currently mapped value (channel or host)
> + * @ref_count: Keeps track of number of current users of this resource
> + */
> +struct pruss_intc_map_record {
> +	u8 value;
> +	u8 ref_count;
> +};
> +
>   /**
>    * struct pruss_intc_match_data - match data to handle SoC variations
>    * @num_system_events: number of input system events handled by the PRUSS INTC
> @@ -71,18 +99,29 @@ struct pruss_intc_match_data {
>   
>   /**
>    * struct pruss_intc - PRUSS interrupt controller structure
> + * @hwirq_data: table of additional mapping data received from device tree
> + *	or PRU firmware
> + * @event_channel: current state of system event to channel mappings
> + * @channel_host: current state of channel to host mappings
>    * @irqs: kernel irq numbers corresponding to PRUSS host interrupts
>    * @base: base virtual address of INTC register space
>    * @domain: irq domain for this interrupt controller
>    * @soc_config: cached PRUSS INTC IP configuration data
> + * @lock: mutex to serialize access to INTC
> + * @dev: PRUSS INTC device pointer
>    * @shared_intr: bit-map denoting if the MPU host interrupt is shared
>    * @invalid_intr: bit-map denoting if host interrupt is not connected to MPU
>    */
>   struct pruss_intc {
> +	struct pruss_intc_hwirq_data hwirq_data[MAX_PRU_SYS_EVENTS];
> +	struct pruss_intc_map_record event_channel[MAX_PRU_SYS_EVENTS];
> +	struct pruss_intc_map_record channel_host[MAX_PRU_CHANNELS];
>   	unsigned int irqs[MAX_NUM_HOST_IRQS];
>   	void __iomem *base;
>   	struct irq_domain *domain;
>   	const struct pruss_intc_match_data *soc_config;
> +	struct mutex lock; /* PRUSS INTC lock */
> +	struct device *dev;
>   	u16 shared_intr;
>   	u16 invalid_intr;
>   };
> @@ -98,6 +137,165 @@ static inline void pruss_intc_write_reg(struct pruss_intc *intc,
>   	writel_relaxed(val, intc->base + reg);
>   }
>   
> +static void pruss_intc_update_cmr(struct pruss_intc *intc, int evt, s8 ch)
> +{
> +	u32 idx, offset, val;
> +
> +	idx = evt / CMR_EVT_PER_REG;
> +	offset = (evt % CMR_EVT_PER_REG) * CMR_EVT_MAP_BITS;
> +
> +	val = pruss_intc_read_reg(intc, PRU_INTC_CMR(idx));
> +	val &= ~(CMR_EVT_MAP_MASK << offset);
> +	val |= ch << offset;
> +	pruss_intc_write_reg(intc, PRU_INTC_CMR(idx), val);
> +
> +	dev_dbg(intc->dev, "SYSEV%u -> CH%d (CMR%d 0x%08x)\n", evt, ch,
> +		idx, pruss_intc_read_reg(intc, PRU_INTC_CMR(idx)));
> +}
> +
> +static void pruss_intc_update_hmr(struct pruss_intc *intc, int ch, s8 host)
> +{
> +	u32 idx, offset, val;
> +
> +	idx = ch / HMR_CH_PER_REG;
> +	offset = (ch % HMR_CH_PER_REG) * HMR_CH_MAP_BITS;
> +
> +	val = pruss_intc_read_reg(intc, PRU_INTC_HMR(idx));
> +	val &= ~(HMR_CH_MAP_MASK << offset);
> +	val |= host << offset;
> +	pruss_intc_write_reg(intc, PRU_INTC_HMR(idx), val);
> +
> +	dev_dbg(intc->dev, "CH%d -> HOST%d (HMR%d 0x%08x)\n", ch, host, idx,
> +		pruss_intc_read_reg(intc, PRU_INTC_HMR(idx)));
> +}
> +
> +/**
> + * pruss_intc_map() - configure the PRUSS INTC
> + * @intc: PRUSS interrupt controller pointer
> + * @hwirq: the system event number
> + *
> + * Configures the PRUSS INTC with the provided configuration from the one
> + * parsed in the xlate function. Any existing event to channel mappings or
> + * channel to host interrupt mappings are checked to make sure there are no
> + * conflicting configuration between both the PRU cores.
> + *
> + * Returns 0 on success, or a suitable error code otherwise
> + */
> +static int pruss_intc_map(struct pruss_intc *intc, unsigned long hwirq)
> +{
> +	struct device *dev = intc->dev;
> +	int ret = 0;
> +	u8 ch, host, reg_idx;
> +	u32 val;
> +
> +	if (hwirq >= intc->soc_config->num_system_events)
> +		return -EINVAL;
> +
> +	mutex_lock(&intc->lock);
> +
> +	ch = intc->hwirq_data[hwirq].channel;
> +	host = intc->hwirq_data[hwirq].host;
> +
> +	/* check if sysevent already assigned */
> +	if (intc->event_channel[hwirq].ref_count > 0 &&
> +	    intc->event_channel[hwirq].value != ch) {
> +		dev_err(dev, "event %lu (req. channel %d) already assigned to channel %d\n",
> +			hwirq, ch, intc->event_channel[hwirq].value);
> +		ret = -EBUSY;
> +		goto unlock;
> +	}
> +
> +	/* check if channel already assigned */
> +	if (intc->channel_host[ch].ref_count > 0 &&
> +	    intc->channel_host[ch].value != host) {
> +		dev_err(dev, "channel %d (req. host %d) already assigned to host %d\n",
> +			ch, host, intc->channel_host[ch].value);
> +		ret = -EBUSY;
> +		goto unlock;
> +	}
> +
> +	if (++intc->event_channel[hwirq].ref_count == 1) {
> +		intc->event_channel[hwirq].value = ch;
> +
> +		pruss_intc_update_cmr(intc, hwirq, ch);
> +
> +		reg_idx = hwirq / 32;
> +		val = BIT(hwirq  % 32);
> +
> +		/* clear and enable system event */
> +		pruss_intc_write_reg(intc, PRU_INTC_ESR(reg_idx), val);
> +		pruss_intc_write_reg(intc, PRU_INTC_SECR(reg_idx), val);
> +	}
> +
> +	if (++intc->channel_host[ch].ref_count == 1) {
> +		intc->channel_host[ch].value = host;
> +
> +		pruss_intc_update_hmr(intc, ch, host);
> +
> +		/* enable host interrupts */
> +		pruss_intc_write_reg(intc, PRU_INTC_HIEISR, host);
> +	}
> +
> +	dev_dbg(dev, "mapped system_event = %lu channel = %d host = %d",
> +		 hwirq, ch, host);
> +
> +	/* global interrupt enable */
> +	pruss_intc_write_reg(intc, PRU_INTC_GER, 1);
> +
> +unlock:
> +	mutex_unlock(&intc->lock);
> +	return ret;
> +}
> +
> +/**
> + * pruss_intc_unmap() - unconfigure the PRUSS INTC
> + * @intc: PRUSS interrupt controller pointer
> + * @hwirq: the system event number
> + *
> + * Undo whatever was done in pruss_intc_map() for a PRU core.
> + * Mappings are reference counted, so resources are only disabled when there
> + * are no longer any users.
> + */
> +static void pruss_intc_unmap(struct pruss_intc *intc, unsigned long hwirq)
> +{
> +	u8 ch, host, reg_idx;
> +	u32 val;
> +
> +	if (hwirq >= intc->soc_config->num_system_events)
> +		return;
> +
> +	mutex_lock(&intc->lock);
> +
> +	ch = intc->event_channel[hwirq].value;
> +	host = intc->channel_host[ch].value;
> +
> +	if (--intc->channel_host[ch].ref_count == 0) {
> +		/* disable host interrupts */
> +		pruss_intc_write_reg(intc, PRU_INTC_HIDISR, host);
> +
> +		/* clear the map using reset value 0 */
> +		pruss_intc_update_hmr(intc, ch, 0);
> +	}
> +
> +	if (--intc->event_channel[hwirq].ref_count == 0) {
> +		reg_idx = hwirq / 32;
> +		val = BIT(hwirq  % 32);
> +
> +		/* disable system events */
> +		pruss_intc_write_reg(intc, PRU_INTC_ECR(reg_idx), val);
> +		/* clear any pending status */
> +		pruss_intc_write_reg(intc, PRU_INTC_SECR(reg_idx), val);
> +
> +		/* clear the map using reset value 0 */
> +		pruss_intc_update_cmr(intc, hwirq, 0);
> +	}
> +
> +	dev_dbg(intc->dev, "unmapped system_event = %lu channel = %d host = %d\n",
> +		hwirq, ch, host);
> +
> +	mutex_unlock(&intc->lock);
> +}
> +
>   static void pruss_intc_init(struct pruss_intc *intc)
>   {
>   	const struct pruss_intc_match_data *soc_config = intc->soc_config;
> @@ -212,10 +410,67 @@ static struct irq_chip pruss_irqchip = {
>   	.irq_set_irqchip_state = pruss_intc_irq_set_irqchip_state,
>   };
>   
> +static int
> +pruss_intc_irq_domain_xlate(struct irq_domain *d, struct device_node *node,
> +			    const u32 *intspec, unsigned int intsize,
> +			    unsigned long *out_hwirq, unsigned int *out_type)
> +{
> +	struct pruss_intc *intc = d->host_data;
> +	struct device *dev = intc->dev;
> +	int sys_event, channel, host;
> +
> +	if (intsize == 1) {
> +		/*
> +		 * In case of short version (intsize == 1) verify if sysevent
> +		 * already mapped to channel/host irq if not return error
> +		 */
> +		sys_event = intspec[0];
> +		if (intc->event_channel[sys_event].ref_count)
> +			goto already_mapped;
> +		else
> +			return -EINVAL;

Perhaps revise this to check the error condition first and skipping the 
else, similar to the change you have done in Patch 3.

> +	}
> +
> +	if (intsize < 3)
> +		return -EINVAL;
> +
> +	sys_event = intspec[0];
> +	if (sys_event < 0 || sys_event >= intc->soc_config->num_system_events) {
> +		dev_err(dev, "not valid event number\n");

Would be useful to print the invalid event numbers here, and each of 
channel and host below.

regards
Suman

> +		return -EINVAL;
> +	}
> +
> +	channel = intspec[1];
> +	if (channel < 0 || channel >= intc->soc_config->num_host_intrs) {
> +		dev_err(dev, "not valid channel number");
> +		return -EINVAL;
> +	}
> +
> +	host = intspec[2];
> +	if (host < 0 || host >= intc->soc_config->num_host_intrs) {
> +		dev_err(dev, "not valid host irq number\n");
> +		return -EINVAL;
> +	}
> +
> +	intc->hwirq_data[sys_event].channel = channel;
> +	intc->hwirq_data[sys_event].host = host;
> +
> +already_mapped:
> +	*out_hwirq = sys_event;
> +	*out_type = IRQ_TYPE_NONE;
> +
> +	return 0;
> +}
> +
>   static int pruss_intc_irq_domain_map(struct irq_domain *d, unsigned int virq,
>   				     irq_hw_number_t hw)
>   {
>   	struct pruss_intc *intc = d->host_data;
> +	int err;
> +
> +	err = pruss_intc_map(intc, hw);
> +	if (err < 0)
> +		return err;
>   
>   	irq_set_chip_data(virq, intc);
>   	irq_set_chip_and_handler(virq, &pruss_irqchip, handle_level_irq);
> @@ -225,12 +480,16 @@ static int pruss_intc_irq_domain_map(struct irq_domain *d, unsigned int virq,
>   
>   static void pruss_intc_irq_domain_unmap(struct irq_domain *d, unsigned int virq)
>   {
> +	struct pruss_intc *intc = d->host_data;
> +	unsigned long hwirq = irqd_to_hwirq(irq_get_irq_data(virq));
> +
>   	irq_set_chip_and_handler(virq, NULL, NULL);
>   	irq_set_chip_data(virq, NULL);
> +	pruss_intc_unmap(intc, hwirq);
>   }
>   
>   static const struct irq_domain_ops pruss_intc_irq_domain_ops = {
> -	.xlate	= irq_domain_xlate_onecell,
> +	.xlate	= pruss_intc_irq_domain_xlate,
>   	.map	= pruss_intc_irq_domain_map,
>   	.unmap	= pruss_intc_irq_domain_unmap,
>   };
> @@ -298,6 +557,8 @@ static int pruss_intc_probe(struct platform_device *pdev)
>   	intc = devm_kzalloc(dev, sizeof(*intc), GFP_KERNEL);
>   	if (!intc)
>   		return -ENOMEM;
> +
> +	intc->dev = dev;
>   	intc->soc_config = data;
>   	platform_set_drvdata(pdev, intc);
>   
> @@ -355,6 +616,8 @@ static int pruss_intc_probe(struct platform_device *pdev)
>   
>   	pruss_intc_init(intc);
>   
> +	mutex_init(&intc->lock);
> +
>   	intc->domain = irq_domain_add_linear(dev->of_node, max_system_events,
>   					     &pruss_intc_irq_domain_ops, intc);
>   	if (!intc->domain)
> 

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Marc Zyngier]

On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
> From: Suman Anna <s-anna@ti.com>
> 
> The Programmable Real-Time Unit Subsystem (PRUSS) contains a local
> interrupt controller (INTC) that can handle various system input events
> and post interrupts back to the device-level initiators. The INTC can
> support upto 64 input events with individual control configuration and
> hardware prioritization. These events are mapped onto 10 output 
> interrupt
> lines through two levels of many-to-one mapping support. Different
> interrupt lines are routed to the individual PRU cores or to the host
> CPU, or to other devices on the SoC. Some of these events are sourced
> from peripherals or other sub-modules within that PRUSS, while a few
> others are sourced from SoC-level peripherals/devices.
> 
> The PRUSS INTC platform driver manages this PRUSS interrupt controller
> and implements an irqchip driver to provide a Linux standard way for
> the PRU client users to enable/disable/ack/re-trigger a PRUSS system
> event. The system events to interrupt channels and output interrupts
> relies on the mapping configuration provided either through the PRU
> firmware blob or via the PRU application's device tree node. The
> mappings will be programmed during the boot/shutdown of a PRU core.
> 
> The PRUSS INTC module is reference counted during the interrupt
> setup phase through the irqchip's irq_request_resources() and
> irq_release_resources() ops. This restricts the module from being
> removed as long as there are active interrupt users.
> 
> The driver currently supports and can be built for OMAP architecture
> based AM335x, AM437x and AM57xx SoCs; Keystone2 architecture based
> 66AK2G SoCs and Davinci architecture based OMAP-L13x/AM18x/DA850 SoCs.
> All of these SoCs support 64 system events, 10 interrupt channels and
> 10 output interrupt lines per PRUSS INTC with a few SoC integration
> differences.
> 
> NOTE:
> Each PRU-ICSS's INTC on AM57xx SoCs is preceded by a Crossbar that
> enables multiple external events to be routed to a specific number
> of input interrupt events. Any non-default external interrupt event
> directed towards PRUSS needs this crossbar to be setup properly.
> 
> Signed-off-by: Suman Anna <s-anna@ti.com>
> Signed-off-by: Andrew F. Davis <afd@ti.com>
> Signed-off-by: Roger Quadros <rogerq@ti.com>
> Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> Reviewed-by: Lee Jones <lee.jones@linaro.org>
> ---
> v2->v3:
> - use single irqchip description instead of separately allocating it 
> for
>   each pruss_intc
> - get rid of unused mutex
> - improve error handling
> v1->v2:
> - https://patchwork.kernel.org/patch/11069771/
> ---
>  drivers/irqchip/Kconfig          |  10 ++
>  drivers/irqchip/Makefile         |   1 +
>  drivers/irqchip/irq-pruss-intc.c | 307 
> +++++++++++++++++++++++++++++++++++++++
>  3 files changed, 318 insertions(+)
>  create mode 100644 drivers/irqchip/irq-pruss-intc.c
> 
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index 29fead2..733d7ec 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -493,6 +493,16 @@ config TI_SCI_INTA_IRQCHIP
>  	  If you wish to use interrupt aggregator irq resources managed by 
> the
>  	  TI System Controller, say Y here. Otherwise, say N.
> 
> +config TI_PRUSS_INTC
> +	tristate "TI PRU-ICSS Interrupt Controller"
> +	depends on ARCH_DAVINCI || SOC_AM33XX || SOC_AM43XX || SOC_DRA7XX ||
> ARCH_KEYSTONE
> +	select IRQ_DOMAIN
> +	help
> +	   This enables support for the PRU-ICSS Local Interrupt Controller
> +	   present within a PRU-ICSS subsystem present on various TI SoCs.
> +	   The PRUSS INTC enables various interrupts to be routed to multiple
> +	   different processors within the SoC.
> +
>  config RISCV_INTC
>  	bool "RISC-V Local Interrupt Controller"
>  	depends on RISCV
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index 133f9c4..990a106 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -106,6 +106,7 @@ obj-$(CONFIG_MADERA_IRQ)		+= irq-madera.o
>  obj-$(CONFIG_LS1X_IRQ)			+= irq-ls1x.o
>  obj-$(CONFIG_TI_SCI_INTR_IRQCHIP)	+= irq-ti-sci-intr.o
>  obj-$(CONFIG_TI_SCI_INTA_IRQCHIP)	+= irq-ti-sci-inta.o
> +obj-$(CONFIG_TI_PRUSS_INTC)		+= irq-pruss-intc.o
>  obj-$(CONFIG_LOONGSON_LIOINTC)		+= irq-loongson-liointc.o
>  obj-$(CONFIG_LOONGSON_HTPIC)		+= irq-loongson-htpic.o
>  obj-$(CONFIG_LOONGSON_HTVEC)		+= irq-loongson-htvec.o
> diff --git a/drivers/irqchip/irq-pruss-intc.c 
> b/drivers/irqchip/irq-pruss-intc.c
> new file mode 100644
> index 0000000..fb3dda3
> --- /dev/null
> +++ b/drivers/irqchip/irq-pruss-intc.c
> @@ -0,0 +1,307 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * PRU-ICSS INTC IRQChip driver for various TI SoCs
> + *
> + * Copyright (C) 2016-2020 Texas Instruments Incorporated - 
> http://www.ti.com/
> + *	Andrew F. Davis <afd@ti.com>
> + *	Suman Anna <s-anna@ti.com>
> + */
> +
> +#include <linux/irq.h>
> +#include <linux/irqchip/chained_irq.h>
> +#include <linux/irqdomain.h>
> +#include <linux/module.h>
> +#include <linux/of_device.h>
> +#include <linux/platform_device.h>
> +
> +/*
> + * Number of host interrupts reaching the main MPU sub-system. Note 
> that this
> + * is not the same as the total number of host interrupts supported
> by the PRUSS
> + * INTC instance
> + */
> +#define MAX_NUM_HOST_IRQS	8
> +
> +/* minimum starting host interrupt number for MPU */
> +#define MIN_PRU_HOST_INT	2
> +
> +/* maximum number of system events */
> +#define MAX_PRU_SYS_EVENTS	64
> +
> +/* PRU_ICSS_INTC registers */
> +#define PRU_INTC_REVID		0x0000
> +#define PRU_INTC_CR		0x0004
> +#define PRU_INTC_GER		0x0010
> +#define PRU_INTC_GNLR		0x001c
> +#define PRU_INTC_SISR		0x0020
> +#define PRU_INTC_SICR		0x0024
> +#define PRU_INTC_EISR		0x0028
> +#define PRU_INTC_EICR		0x002c
> +#define PRU_INTC_HIEISR		0x0034
> +#define PRU_INTC_HIDISR		0x0038
> +#define PRU_INTC_GPIR		0x0080
> +#define PRU_INTC_SRSR0		0x0200
> +#define PRU_INTC_SRSR1		0x0204
> +#define PRU_INTC_SECR0		0x0280
> +#define PRU_INTC_SECR1		0x0284
> +#define PRU_INTC_ESR0		0x0300
> +#define PRU_INTC_ESR1		0x0304
> +#define PRU_INTC_ECR0		0x0380
> +#define PRU_INTC_ECR1		0x0384
> +#define PRU_INTC_CMR(x)		(0x0400 + (x) * 4)
> +#define PRU_INTC_HMR(x)		(0x0800 + (x) * 4)
> +#define PRU_INTC_HIPIR(x)	(0x0900 + (x) * 4)
> +#define PRU_INTC_SIPR0		0x0d00
> +#define PRU_INTC_SIPR1		0x0d04
> +#define PRU_INTC_SITR0		0x0d80
> +#define PRU_INTC_SITR1		0x0d84
> +#define PRU_INTC_HINLR(x)	(0x1100 + (x) * 4)
> +#define PRU_INTC_HIER		0x1500
> +
> +/* HIPIR register bit-fields */
> +#define INTC_HIPIR_NONE_HINT	0x80000000
> +
> +/**
> + * struct pruss_intc - PRUSS interrupt controller structure
> + * @irqs: kernel irq numbers corresponding to PRUSS host interrupts
> + * @base: base virtual address of INTC register space
> + * @domain: irq domain for this interrupt controller
> + */
> +struct pruss_intc {
> +	unsigned int irqs[MAX_NUM_HOST_IRQS];
> +	void __iomem *base;
> +	struct irq_domain *domain;
> +};
> +
> +static inline u32 pruss_intc_read_reg(struct pruss_intc *intc,
> unsigned int reg)
> +{
> +	return readl_relaxed(intc->base + reg);
> +}
> +
> +static inline void pruss_intc_write_reg(struct pruss_intc *intc,
> +					unsigned int reg, u32 val)
> +{
> +	writel_relaxed(val, intc->base + reg);
> +}
> +
> +static void pruss_intc_init(struct pruss_intc *intc)
> +{
> +	int i;
> +
> +	/* configure polarity to active high for all system interrupts */
> +	pruss_intc_write_reg(intc, PRU_INTC_SIPR0, 0xffffffff);
> +	pruss_intc_write_reg(intc, PRU_INTC_SIPR1, 0xffffffff);
> +
> +	/* configure type to pulse interrupt for all system interrupts */
> +	pruss_intc_write_reg(intc, PRU_INTC_SITR0, 0);
> +	pruss_intc_write_reg(intc, PRU_INTC_SITR1, 0);

So the default is to configure everything as edge...

> +
> +	/* clear all 16 interrupt channel map registers */
> +	for (i = 0; i < 16; i++)
> +		pruss_intc_write_reg(intc, PRU_INTC_CMR(i), 0);
> +
> +	/* clear all 3 host interrupt map registers */
> +	for (i = 0; i < 3; i++)
> +		pruss_intc_write_reg(intc, PRU_INTC_HMR(i), 0);
> +}
> +
> +static void pruss_intc_irq_ack(struct irq_data *data)
> +{
> +	struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
> +	unsigned int hwirq = data->hwirq;
> +
> +	pruss_intc_write_reg(intc, PRU_INTC_SICR, hwirq);
> +}
> +
> +static void pruss_intc_irq_mask(struct irq_data *data)
> +{
> +	struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
> +	unsigned int hwirq = data->hwirq;
> +
> +	pruss_intc_write_reg(intc, PRU_INTC_EICR, hwirq);
> +}
> +
> +static void pruss_intc_irq_unmask(struct irq_data *data)
> +{
> +	struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
> +	unsigned int hwirq = data->hwirq;
> +
> +	pruss_intc_write_reg(intc, PRU_INTC_EISR, hwirq);
> +}
> +
> +static int pruss_intc_irq_reqres(struct irq_data *data)
> +{
> +	if (!try_module_get(THIS_MODULE))
> +		return -ENODEV;
> +
> +	return 0;
> +}
> +
> +static void pruss_intc_irq_relres(struct irq_data *data)
> +{
> +	module_put(THIS_MODULE);
> +}
> +
> +static struct irq_chip pruss_irqchip = {
> +	.name = "pruss-intc",
> +	.irq_ack = pruss_intc_irq_ack,
> +	.irq_mask = pruss_intc_irq_mask,
> +	.irq_unmask = pruss_intc_irq_unmask,
> +	.irq_request_resources = pruss_intc_irq_reqres,
> +	.irq_release_resources = pruss_intc_irq_relres,
> +};
> +
> +static int pruss_intc_irq_domain_map(struct irq_domain *d, unsigned 
> int virq,
> +				     irq_hw_number_t hw)
> +{
> +	struct pruss_intc *intc = d->host_data;
> +
> +	irq_set_chip_data(virq, intc);
> +	irq_set_chip_and_handler(virq, &pruss_irqchip, handle_level_irq);

... and despite this edge-triggered default, you handle things as level.
This doesn't seem quite right.

> +
> +	return 0;
> +}
> +
> +static void pruss_intc_irq_domain_unmap(struct irq_domain *d,
> unsigned int virq)
> +{
> +	irq_set_chip_and_handler(virq, NULL, NULL);
> +	irq_set_chip_data(virq, NULL);
> +}
> +
> +static const struct irq_domain_ops pruss_intc_irq_domain_ops = {
> +	.xlate	= irq_domain_xlate_onecell,
> +	.map	= pruss_intc_irq_domain_map,
> +	.unmap	= pruss_intc_irq_domain_unmap,
> +};
> +
> +static void pruss_intc_irq_handler(struct irq_desc *desc)
> +{
> +	unsigned int irq = irq_desc_get_irq(desc);
> +	struct irq_chip *chip = irq_desc_get_chip(desc);
> +	struct pruss_intc *intc = irq_get_handler_data(irq);
> +	u32 hipir;
> +	unsigned int virq;
> +	int i, hwirq;
> +
> +	chained_irq_enter(chip, desc);
> +
> +	/* find our host irq number */
> +	for (i = 0; i < MAX_NUM_HOST_IRQS; i++)
> +		if (intc->irqs[i] == irq)
> +			break;

This loop is pretty ugly. The way to do it would normally to
associate the right data structure to the chained interrupt,
and only that one, directly associating the input signal
with the correct mux. Using the Linux irq as a discriminant is
at best clumsy.

But it feels to me that the base data structure is not
exactly the right one here, see below.

> +	if (i == MAX_NUM_HOST_IRQS)
> +		goto err;
> +
> +	i += MIN_PRU_HOST_INT;
> +
> +	/* get highest priority pending PRUSS system event */
> +	hipir = pruss_intc_read_reg(intc, PRU_INTC_HIPIR(i));
> +	while (!(hipir & INTC_HIPIR_NONE_HINT)) {

Please write this as a do { } while() loop, with a single instance
of the HW register read inside the loop (instead of one outside
and one inside.

> +		hwirq = hipir & GENMASK(9, 0);
> +		virq = irq_linear_revmap(intc->domain, hwirq);

And this is where I worry. You seems to have a single irqdomain
for all the muxes. Are you guaranteed that you will have no
overlap between muxes? And please use irq_find_mapping(), as
I have top-secret plans to kill irq_linear_revmap().

> +
> +		/*
> +		 * NOTE: manually ACK any system events that do not have a
> +		 * handler mapped yet
> +		 */
> +		if (WARN_ON(!virq))
> +			pruss_intc_write_reg(intc, PRU_INTC_SICR, hwirq);

How can this happen? If you really need it, you probable want to
warn once only.

> +		else
> +			generic_handle_irq(virq);
> +
> +		/* get next system event */
> +		hipir = pruss_intc_read_reg(intc, PRU_INTC_HIPIR(i));
> +	}
> +err:
> +	chained_irq_exit(chip, desc);
> +}
> +
> +static int pruss_intc_probe(struct platform_device *pdev)
> +{
> +	static const char * const irq_names[MAX_NUM_HOST_IRQS] = {
> +		"host_intr0", "host_intr1", "host_intr2", "host_intr3",
> +		"host_intr4", "host_intr5", "host_intr6", "host_intr7", };
> +	struct device *dev = &pdev->dev;
> +	struct pruss_intc *intc;
> +	int i, irq;
> +
> +	intc = devm_kzalloc(dev, sizeof(*intc), GFP_KERNEL);
> +	if (!intc)
> +		return -ENOMEM;
> +	platform_set_drvdata(pdev, intc);
> +
> +	intc->base = devm_platform_ioremap_resource(pdev, 0);
> +	if (IS_ERR(intc->base)) {
> +		dev_err(dev, "failed to parse and map intc memory resource\n");
> +		return PTR_ERR(intc->base);
> +	}
> +
> +	pruss_intc_init(intc);
> +
> +	/* always 64 events */
> +	intc->domain = irq_domain_add_linear(dev->of_node, 
> MAX_PRU_SYS_EVENTS,
> +					     &pruss_intc_irq_domain_ops, intc);
> +	if (!intc->domain)
> +		return -ENOMEM;
> +
> +	for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
> +		irq = platform_get_irq_byname(pdev, irq_names[i]);
> +		if (irq <= 0) {
> +			dev_err(dev, "platform_get_irq_byname failed for %s : %d\n",
> +				irq_names[i], irq);
> +			goto fail_irq;
> +		}
> +
> +		intc->irqs[i] = irq;

Are the output IRQs guaranteed to be contiguous? If so, that'd be
a much nicer way to work out which mux has fired (just store the
base, and use it as an offset on handling the chained interrupt).

> +		irq_set_handler_data(irq, intc);
> +		irq_set_chained_handler(irq, pruss_intc_irq_handler);
> +	}
> +
> +	return 0;
> +
> +fail_irq:
> +	while (--i >= 0)
> +		irq_set_chained_handler_and_data(intc->irqs[i], NULL, NULL);
> +
> +	irq_domain_remove(intc->domain);
> +
> +	return irq;
> +}
> +
> +static int pruss_intc_remove(struct platform_device *pdev)
> +{
> +	struct pruss_intc *intc = platform_get_drvdata(pdev);
> +	unsigned int hwirq;
> +	int i;
> +
> +	for (i = 0; i < MAX_NUM_HOST_IRQS; i++)
> +		irq_set_chained_handler_and_data(intc->irqs[i], NULL, NULL);
> +
> +	for (hwirq = 0; hwirq < MAX_PRU_SYS_EVENTS; hwirq++)
> +		irq_dispose_mapping(irq_find_mapping(intc->domain, hwirq));
> +
> +	irq_domain_remove(intc->domain);
> +
> +	return 0;
> +}
> +
> +static const struct of_device_id pruss_intc_of_match[] = {
> +	{ .compatible = "ti,pruss-intc", },
> +	{ /* sentinel */ },
> +};
> +MODULE_DEVICE_TABLE(of, pruss_intc_of_match);
> +
> +static struct platform_driver pruss_intc_driver = {
> +	.driver = {
> +		.name = "pruss-intc",
> +		.of_match_table = pruss_intc_of_match,
> +		.suppress_bind_attrs = true,
> +	},
> +	.probe  = pruss_intc_probe,
> +	.remove = pruss_intc_remove,
> +};
> +module_platform_driver(pruss_intc_driver);
> +
> +MODULE_AUTHOR("Andrew F. Davis <afd@ti.com>");
> +MODULE_AUTHOR("Suman Anna <s-anna@ti.com>");
> +MODULE_DESCRIPTION("TI PRU-ICSS INTC Driver");
> +MODULE_LICENSE("GPL v2");

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCHv3 3/6] irqchip/irq-pruss-intc: Add support for shared and invalid interrupts

[Marc Zyngier]

On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
> From: Suman Anna <s-anna@ti.com>
> 
> The PRUSS INTC has a fixed number of output interrupt lines that are
> connected to a number of processors or other PRUSS instances or other
> devices (like DMA) on the SoC. The output interrupt lines 2 through 9
> are usually connected to the main Arm host processor and are referred
> to as host interrupts 0 through 7 from ARM/MPU perspective.
> 
> All of these 8 host interrupts are not always exclusively connected
> to the Arm interrupt controller. Some SoCs have some interrupt lines
> not connected to the Arm interrupt controller at all, while a few 
> others
> have the interrupt lines connected to multiple processors in which they
> need to be partitioned as per SoC integration needs. For example, 
> AM437x
> and 66AK2G SoCs have 2 PRUSS instances each and have the host interrupt 
> 5
> connected to the other PRUSS, while AM335x has host interrupt 0 shared
> between MPU and TSC_ADC and host interrupts 6 & 7 shared between MPU 
> and
> a DMA controller.
> 
> Add support to the PRUSS INTC driver to allow both these shared and
> invalid interrupts by not returning a failure if any of these 
> interrupts
> are skipped from the corresponding INTC DT node.

That's not exactly "adding support", is it? It really is "ignore these
interrupts because they are useless from the main CPU's perspective",
right?

> 
> Signed-off-by: Suman Anna <s-anna@ti.com>
> Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> ---
> v2->v3:
> - Extra checks for (intc->irqs[i]) in error/remove path was moved from
>   "irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS
>   interrupts" to this patch
> v1->v2:
> - https://patchwork.kernel.org/patch/11069757/
> ---
>  drivers/irqchip/irq-pruss-intc.c | 73 
> +++++++++++++++++++++++++++++++++++++---
>  1 file changed, 68 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/irqchip/irq-pruss-intc.c 
> b/drivers/irqchip/irq-pruss-intc.c
> index fb3dda3..49c936f 100644
> --- a/drivers/irqchip/irq-pruss-intc.c
> +++ b/drivers/irqchip/irq-pruss-intc.c
> @@ -65,11 +65,15 @@
>   * @irqs: kernel irq numbers corresponding to PRUSS host interrupts
>   * @base: base virtual address of INTC register space
>   * @domain: irq domain for this interrupt controller
> + * @shared_intr: bit-map denoting if the MPU host interrupt is shared

nit: bitmap

> + * @invalid_intr: bit-map denoting if host interrupt is not connected 
> to MPU
>   */
>  struct pruss_intc {
>  	unsigned int irqs[MAX_NUM_HOST_IRQS];
>  	void __iomem *base;
>  	struct irq_domain *domain;
> +	u16 shared_intr;
> +	u16 invalid_intr;

Please represent bitmaps as an unsigned long.

>  };
> 
>  static inline u32 pruss_intc_read_reg(struct pruss_intc *intc,
> unsigned int reg)
> @@ -222,7 +226,8 @@ static int pruss_intc_probe(struct platform_device 
> *pdev)
>  		"host_intr4", "host_intr5", "host_intr6", "host_intr7", };
>  	struct device *dev = &pdev->dev;
>  	struct pruss_intc *intc;
> -	int i, irq;
> +	int i, irq, count;
> +	u8 temp_intr[MAX_NUM_HOST_IRQS] = { 0 };
> 
>  	intc = devm_kzalloc(dev, sizeof(*intc), GFP_KERNEL);
>  	if (!intc)
> @@ -235,6 +240,52 @@ static int pruss_intc_probe(struct platform_device 
> *pdev)
>  		return PTR_ERR(intc->base);
>  	}
> 
> +	count = of_property_read_variable_u8_array(dev->of_node,
> +						   "ti,irqs-reserved",
> +						   temp_intr, 0,
> +						   MAX_NUM_HOST_IRQS);
> +	/*
> +	 * The irqs-reserved is used only for some SoC's therefore not having
> +	 * this property is still valid
> +	 */
> +	if (count == -EINVAL)
> +		count = 0;
> +	if (count < 0)
> +		return count;
> +
> +	for (i = 0; i < count; i++) {
> +		if (temp_intr[i] >= MAX_NUM_HOST_IRQS) {
> +			dev_warn(dev, "ignoring invalid reserved irq %d\n",
> +				 temp_intr[i]);
> +			continue;
> +		}
> +
> +		intc->invalid_intr |= BIT(temp_intr[i]);
> +	}
> +
> +	count = of_property_read_variable_u8_array(dev->of_node,
> +						   "ti,irqs-shared",
> +						   temp_intr, 0,
> +						   MAX_NUM_HOST_IRQS);
> +	/*
> +	 * The irqs-shared is used only for some SoC's therefore not having
> +	 * this property is still valid
> +	 */
> +	if (count == -EINVAL)
> +		count = 0;
> +	if (count < 0)
> +		return count;
> +
> +	for (i = 0; i < count; i++) {
> +		if (temp_intr[i] >= MAX_NUM_HOST_IRQS) {
> +			dev_warn(dev, "ignoring invalid shared irq %d\n",
> +				 temp_intr[i]);
> +			continue;
> +		}
> +
> +		intc->shared_intr |= BIT(temp_intr[i]);
> +	}
> +

You probably want to move this in a separate function, since you 
populate a
common structure.

>  	pruss_intc_init(intc);
> 
>  	/* always 64 events */
> @@ -244,8 +295,14 @@ static int pruss_intc_probe(struct platform_device 
> *pdev)
>  		return -ENOMEM;
> 
>  	for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
> +		if (intc->invalid_intr & BIT(i))
> +			continue;
> +
>  		irq = platform_get_irq_byname(pdev, irq_names[i]);
>  		if (irq <= 0) {
> +			if (intc->shared_intr & BIT(i))
> +				continue;

I don't really understand why you are treating these "shared" interrupts
differently from the invalid ones. In all cases, they shouldn't be used.

> +
>  			dev_err(dev, "platform_get_irq_byname failed for %s : %d\n",
>  				irq_names[i], irq);
>  			goto fail_irq;
> @@ -259,8 +316,11 @@ static int pruss_intc_probe(struct platform_device 
> *pdev)
>  	return 0;
> 
>  fail_irq:
> -	while (--i >= 0)
> -		irq_set_chained_handler_and_data(intc->irqs[i], NULL, NULL);
> +	while (--i >= 0) {
> +		if (intc->irqs[i])
> +			irq_set_chained_handler_and_data(intc->irqs[i], NULL,
> +							 NULL);
> +	}
> 
>  	irq_domain_remove(intc->domain);
> 
> @@ -273,8 +333,11 @@ static int pruss_intc_remove(struct 
> platform_device *pdev)
>  	unsigned int hwirq;
>  	int i;
> 
> -	for (i = 0; i < MAX_NUM_HOST_IRQS; i++)
> -		irq_set_chained_handler_and_data(intc->irqs[i], NULL, NULL);
> +	for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
> +		if (intc->irqs[i])
> +			irq_set_chained_handler_and_data(intc->irqs[i], NULL,
> +							 NULL);
> +	}
> 
>  	for (hwirq = 0; hwirq < MAX_PRU_SYS_EVENTS; hwirq++)
>  		irq_dispose_mapping(irq_find_mapping(intc->domain, hwirq));

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCHv3 4/6] irqchip/irq-pruss-intc: Implement irq_{get,set}_irqchip_state ops

[Marc Zyngier]

On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
> From: David Lechner <david@lechnology.com>
> 
> This implements the irq_get_irqchip_state and irq_set_irqchip_state
> callbacks for the TI PRUSS INTC driver. The set callback can be used
> by drivers to "kick" a PRU by enabling a PRU system event.

"enabling"? That'd be unmasking an interrupt, which isn't what this
does. "injecting", maybe?

> 
> Example:
>      irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, true);

Nice example.

What this example does explain is how you are actually going to kick
a PRU via this interface. For that to happen, you'd have to have on
the Linux side an interrupt that is actually routed to a PRU.
And from what I have understood of the previous patches, this can't
be the case. What didi I miss?

> 
> Signed-off-by: David Lechner <david@lechnology.com>
> Signed-off-by: Suman Anna <s-anna@ti.com>
> Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> Reviewed-by: Lee Jones <lee.jones@linaro.org>
> ---
> v2->v3:
> - Get rid of unnecessary pruss_intc_check_write() and use
>   pruss_intc_write_reg directly.
> v1->v2:
> - https://patchwork.kernel.org/patch/11069769/
> ---
>  drivers/irqchip/irq-pruss-intc.c | 43 
> ++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 41 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/irqchip/irq-pruss-intc.c 
> b/drivers/irqchip/irq-pruss-intc.c
> index 49c936f..19b3d38 100644
> --- a/drivers/irqchip/irq-pruss-intc.c
> +++ b/drivers/irqchip/irq-pruss-intc.c
> @@ -7,6 +7,7 @@
>   *	Suman Anna <s-anna@ti.com>
>   */
> 
> +#include <linux/interrupt.h>
>  #include <linux/irq.h>
>  #include <linux/irqchip/chained_irq.h>
>  #include <linux/irqdomain.h>
> @@ -39,8 +40,7 @@
>  #define PRU_INTC_HIEISR		0x0034
>  #define PRU_INTC_HIDISR		0x0038
>  #define PRU_INTC_GPIR		0x0080
> -#define PRU_INTC_SRSR0		0x0200
> -#define PRU_INTC_SRSR1		0x0204
> +#define PRU_INTC_SRSR(x)	(0x0200 + (x) * 4)
>  #define PRU_INTC_SECR0		0x0280
>  #define PRU_INTC_SECR1		0x0284
>  #define PRU_INTC_ESR0		0x0300
> @@ -145,6 +145,43 @@ static void pruss_intc_irq_relres(struct irq_data 
> *data)
>  	module_put(THIS_MODULE);
>  }
> 
> +static int pruss_intc_irq_get_irqchip_state(struct irq_data *data,
> +					    enum irqchip_irq_state which,
> +					    bool *state)
> +{
> +	struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
> +	u32 reg, mask, srsr;
> +
> +	if (which != IRQCHIP_STATE_PENDING)
> +		return -EINVAL;
> +
> +	reg = PRU_INTC_SRSR(data->hwirq / 32);

I assume the register file scales as more interrupts are added in the
subsequent patch?

> +	mask = BIT(data->hwirq % 32);
> +
> +	srsr = pruss_intc_read_reg(intc, reg);
> +
> +	*state = !!(srsr & mask);
> +
> +	return 0;
> +}
> +
> +static int pruss_intc_irq_set_irqchip_state(struct irq_data *data,
> +					    enum irqchip_irq_state which,
> +					    bool state)
> +{
> +	struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
> +
> +	if (which != IRQCHIP_STATE_PENDING)
> +		return -EINVAL;
> +
> +	if (state)
> +		pruss_intc_write_reg(intc, PRU_INTC_SISR, data->hwirq);
> +	else
> +		pruss_intc_write_reg(intc, PRU_INTC_SICR, data->hwirq);
> +
> +	return 0;
> +}
> +
>  static struct irq_chip pruss_irqchip = {
>  	.name = "pruss-intc",
>  	.irq_ack = pruss_intc_irq_ack,
> @@ -152,6 +189,8 @@ static struct irq_chip pruss_irqchip = {
>  	.irq_unmask = pruss_intc_irq_unmask,
>  	.irq_request_resources = pruss_intc_irq_reqres,
>  	.irq_release_resources = pruss_intc_irq_relres,
> +	.irq_get_irqchip_state = pruss_intc_irq_get_irqchip_state,
> +	.irq_set_irqchip_state = pruss_intc_irq_set_irqchip_state,
>  };
> 
>  static int pruss_intc_irq_domain_map(struct irq_domain *d, unsigned 
> int virq,

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCHv3 5/6] irqchip/irq-pruss-intc: Add support for ICSSG INTC on K3 SoCs

[Marc Zyngier]

On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
> From: Suman Anna <s-anna@ti.com>
> 
> The K3 AM65x and J721E SoCs have the next generation of the PRU-ICSS 
> IP,
> commonly called ICSSG. The PRUSS INTC present within the ICSSG supports
> more System Events (160 vs 64), more Interrupt Channels and Host 
> Interrupts
> (20 vs 10) compared to the previous generation PRUSS INTC instances. 
> The
> first 2 and the last 10 of these host interrupt lines are used by the
> PRU and other auxiliary cores and sub-modules within the ICSSG, with 8
> host interrupts connected to MPU. The host interrupts 5, 6, 7 are also
> connected to the other ICSSG instances within the SoC and can be
> partitioned as per system integration through the board dts files.
> 
> Enhance the PRUSS INTC driver to add support for this ICSSG INTC
> instance. This support is added using specific compatible and match
> data and updating the code to use this data instead of the current
> hard-coded macros. The INTC config structure is updated to use the
> higher events and channels on all SoCs, while limiting the actual
> processing to only the relevant number of events/channels/interrupts.
> 
> Signed-off-by: Suman Anna <s-anna@ti.com>
> Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> ---
> v2->v3:
> - Change patch order: use it directly after "irqchip/irq-pruss-intc:
>   Implement irq_{get,set}_irqchip_state ops" and before new
>   "irqchip/irq-pruss-intc: Add event mapping support" in order to 
> reduce
>   diff.

The diff would be even smaller if you introduced a variable number of
inputs the first place, i.e. in patch #2. Most if this patch just
retrofits it. Please squash these changes into that initial patch,
and only add the platform stuff here.

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Grzegorz Jaszczyk]

On Thu, 2 Jul 2020 at 19:24, Marc Zyngier <maz@kernel.org> wrote:
>
> On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
> > From: Suman Anna <s-anna@ti.com>
> >
> > The Programmable Real-Time Unit Subsystem (PRUSS) contains a local
> > interrupt controller (INTC) that can handle various system input events
> > and post interrupts back to the device-level initiators. The INTC can
> > support upto 64 input events with individual control configuration and
> > hardware prioritization. These events are mapped onto 10 output
> > interrupt
> > lines through two levels of many-to-one mapping support. Different
> > interrupt lines are routed to the individual PRU cores or to the host
> > CPU, or to other devices on the SoC. Some of these events are sourced
> > from peripherals or other sub-modules within that PRUSS, while a few
> > others are sourced from SoC-level peripherals/devices.
> >
> > The PRUSS INTC platform driver manages this PRUSS interrupt controller
> > and implements an irqchip driver to provide a Linux standard way for
> > the PRU client users to enable/disable/ack/re-trigger a PRUSS system
> > event. The system events to interrupt channels and output interrupts
> > relies on the mapping configuration provided either through the PRU
> > firmware blob or via the PRU application's device tree node. The
> > mappings will be programmed during the boot/shutdown of a PRU core.
> >
> > The PRUSS INTC module is reference counted during the interrupt
> > setup phase through the irqchip's irq_request_resources() and
> > irq_release_resources() ops. This restricts the module from being
> > removed as long as there are active interrupt users.
> >
> > The driver currently supports and can be built for OMAP architecture
> > based AM335x, AM437x and AM57xx SoCs; Keystone2 architecture based
> > 66AK2G SoCs and Davinci architecture based OMAP-L13x/AM18x/DA850 SoCs.
> > All of these SoCs support 64 system events, 10 interrupt channels and
> > 10 output interrupt lines per PRUSS INTC with a few SoC integration
> > differences.
> >
> > NOTE:
> > Each PRU-ICSS's INTC on AM57xx SoCs is preceded by a Crossbar that
> > enables multiple external events to be routed to a specific number
> > of input interrupt events. Any non-default external interrupt event
> > directed towards PRUSS needs this crossbar to be setup properly.
> >
> > Signed-off-by: Suman Anna <s-anna@ti.com>
> > Signed-off-by: Andrew F. Davis <afd@ti.com>
> > Signed-off-by: Roger Quadros <rogerq@ti.com>
> > Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> > Reviewed-by: Lee Jones <lee.jones@linaro.org>
> > ---
> > v2->v3:
> > - use single irqchip description instead of separately allocating it
> > for
> >   each pruss_intc
> > - get rid of unused mutex
> > - improve error handling
> > v1->v2:
> > - https://patchwork.kernel.org/patch/11069771/
<snip>
> > +static void pruss_intc_init(struct pruss_intc *intc)
> > +{
> > +     int i;
> > +
> > +     /* configure polarity to active high for all system interrupts */
> > +     pruss_intc_write_reg(intc, PRU_INTC_SIPR0, 0xffffffff);
> > +     pruss_intc_write_reg(intc, PRU_INTC_SIPR1, 0xffffffff);
> > +
> > +     /* configure type to pulse interrupt for all system interrupts */
> > +     pruss_intc_write_reg(intc, PRU_INTC_SITR0, 0);
> > +     pruss_intc_write_reg(intc, PRU_INTC_SITR1, 0);
>
> So the default is to configure everything as edge...

Sorry, the description is wrong - '0' indicates level and '1' edge. So
the default configuration is level - I will fix the comment.

>
> > +
> > +     /* clear all 16 interrupt channel map registers */
> > +     for (i = 0; i < 16; i++)
> > +             pruss_intc_write_reg(intc, PRU_INTC_CMR(i), 0);
> > +
> > +     /* clear all 3 host interrupt map registers */
> > +     for (i = 0; i < 3; i++)
> > +             pruss_intc_write_reg(intc, PRU_INTC_HMR(i), 0);
> > +}
> > +
> > +static void pruss_intc_irq_ack(struct irq_data *data)
> > +{
> > +     struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
> > +     unsigned int hwirq = data->hwirq;
> > +
> > +     pruss_intc_write_reg(intc, PRU_INTC_SICR, hwirq);
> > +}
> > +
> > +static void pruss_intc_irq_mask(struct irq_data *data)
> > +{
> > +     struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
> > +     unsigned int hwirq = data->hwirq;
> > +
> > +     pruss_intc_write_reg(intc, PRU_INTC_EICR, hwirq);
> > +}
> > +
> > +static void pruss_intc_irq_unmask(struct irq_data *data)
> > +{
> > +     struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
> > +     unsigned int hwirq = data->hwirq;
> > +
> > +     pruss_intc_write_reg(intc, PRU_INTC_EISR, hwirq);
> > +}
> > +
> > +static int pruss_intc_irq_reqres(struct irq_data *data)
> > +{
> > +     if (!try_module_get(THIS_MODULE))
> > +             return -ENODEV;
> > +
> > +     return 0;
> > +}
> > +
> > +static void pruss_intc_irq_relres(struct irq_data *data)
> > +{
> > +     module_put(THIS_MODULE);
> > +}
> > +
> > +static struct irq_chip pruss_irqchip = {
> > +     .name = "pruss-intc",
> > +     .irq_ack = pruss_intc_irq_ack,
> > +     .irq_mask = pruss_intc_irq_mask,
> > +     .irq_unmask = pruss_intc_irq_unmask,
> > +     .irq_request_resources = pruss_intc_irq_reqres,
> > +     .irq_release_resources = pruss_intc_irq_relres,
> > +};
> > +
> > +static int pruss_intc_irq_domain_map(struct irq_domain *d, unsigned
> > int virq,
> > +                                  irq_hw_number_t hw)
> > +{
> > +     struct pruss_intc *intc = d->host_data;
> > +
> > +     irq_set_chip_data(virq, intc);
> > +     irq_set_chip_and_handler(virq, &pruss_irqchip, handle_level_irq);
>
> ... and despite this edge-triggered default, you handle things as level.
> This doesn't seem quite right.

As above it is level. I will fix the comment

>
> > +
> > +     return 0;
> > +}
> > +
> > +static void pruss_intc_irq_domain_unmap(struct irq_domain *d,
> > unsigned int virq)
> > +{
> > +     irq_set_chip_and_handler(virq, NULL, NULL);
> > +     irq_set_chip_data(virq, NULL);
> > +}
> > +
> > +static const struct irq_domain_ops pruss_intc_irq_domain_ops = {
> > +     .xlate  = irq_domain_xlate_onecell,
> > +     .map    = pruss_intc_irq_domain_map,
> > +     .unmap  = pruss_intc_irq_domain_unmap,
> > +};
> > +
> > +static void pruss_intc_irq_handler(struct irq_desc *desc)
> > +{
> > +     unsigned int irq = irq_desc_get_irq(desc);
> > +     struct irq_chip *chip = irq_desc_get_chip(desc);
> > +     struct pruss_intc *intc = irq_get_handler_data(irq);
> > +     u32 hipir;
> > +     unsigned int virq;
> > +     int i, hwirq;
> > +
> > +     chained_irq_enter(chip, desc);
> > +
> > +     /* find our host irq number */
> > +     for (i = 0; i < MAX_NUM_HOST_IRQS; i++)
> > +             if (intc->irqs[i] == irq)
> > +                     break;
>
> This loop is pretty ugly. The way to do it would normally to
> associate the right data structure to the chained interrupt,
> and only that one, directly associating the input signal
> with the correct mux. Using the Linux irq as a discriminant is
> at best clumsy.
>
> But it feels to me that the base data structure is not
> exactly the right one here, see below.
>

Ok, you are right. I will introduce a new structure for host_irq data
which will be associated with chained interrupt and get rid of this
loop.

> > +     if (i == MAX_NUM_HOST_IRQS)
> > +             goto err;
> > +
> > +     i += MIN_PRU_HOST_INT;
> > +
> > +     /* get highest priority pending PRUSS system event */
> > +     hipir = pruss_intc_read_reg(intc, PRU_INTC_HIPIR(i));
> > +     while (!(hipir & INTC_HIPIR_NONE_HINT)) {
>
> Please write this as a do { } while() loop, with a single instance
> of the HW register read inside the loop (instead of one outside
> and one inside.

Ok, I will get rid of the outside HW register read, but I think it is
better to use bellow instead of do {} while () loop:
while (1) {
  /* get highest priority pending PRUSS system event */
  hipir = pruss_intc_read_reg(intc, PRU_INTC_HIPIR(host_irq));
  if (hipir & INTC_HIPIR_NONE_HINT)
    break;
...

Hope it works for you.

>
> > +             hwirq = hipir & GENMASK(9, 0);
> > +             virq = irq_linear_revmap(intc->domain, hwirq);
>
> And this is where I worry. You seems to have a single irqdomain
> for all the muxes. Are you guaranteed that you will have no
> overlap between muxes? And please use irq_find_mapping(), as
> I have top-secret plans to kill irq_linear_revmap().

Regarding irq_find_mapping - sure.

Regarding irqdomains:
It is a single irqdomain since the hwirq (system event) can be mapped
to different irq_host (muxes). Patch #6
https://lkml.org/lkml/2020/7/2/616 implements and describes how input
events can be mapped to some output host interrupts through 2 levels
of many-to-one mapping i.e. events to channel mapping and channels to
host interrupts. Mentioned implementation ensures that specific system
event (hwirq) can be mapped through PRUSS specific channel into a
single host interrupt.

>
> > +
> > +             /*
> > +              * NOTE: manually ACK any system events that do not have a
> > +              * handler mapped yet
> > +              */
> > +             if (WARN_ON(!virq))
> > +                     pruss_intc_write_reg(intc, PRU_INTC_SICR, hwirq);
>
> How can this happen? If you really need it, you probable want to
> warn once only.

Ideally it shouldn't happen but I prefer to keep it to catch any
misuse. It is because the PRUSS INTC unit can be also accessed by PRU
cores which use the same registers to ack the internal events. The
current design is limited to only acking and triggering the interrupts
from PRU firmwares while the entire mapping is done by Linux (patch #6
https://lkml.org/lkml/2020/7/2/612).

I will convert it to WARN_ON_ONCE.

>
> > +             else
> > +                     generic_handle_irq(virq);
> > +
> > +             /* get next system event */
> > +             hipir = pruss_intc_read_reg(intc, PRU_INTC_HIPIR(i));
> > +     }
> > +err:
> > +     chained_irq_exit(chip, desc);
> > +}
> > +
> > +static int pruss_intc_probe(struct platform_device *pdev)
> > +{
> > +     static const char * const irq_names[MAX_NUM_HOST_IRQS] = {
> > +             "host_intr0", "host_intr1", "host_intr2", "host_intr3",
> > +             "host_intr4", "host_intr5", "host_intr6", "host_intr7", };
> > +     struct device *dev = &pdev->dev;
> > +     struct pruss_intc *intc;
> > +     int i, irq;
> > +
> > +     intc = devm_kzalloc(dev, sizeof(*intc), GFP_KERNEL);
> > +     if (!intc)
> > +             return -ENOMEM;
> > +     platform_set_drvdata(pdev, intc);
> > +
> > +     intc->base = devm_platform_ioremap_resource(pdev, 0);
> > +     if (IS_ERR(intc->base)) {
> > +             dev_err(dev, "failed to parse and map intc memory resource\n");
> > +             return PTR_ERR(intc->base);
> > +     }
> > +
> > +     pruss_intc_init(intc);
> > +
> > +     /* always 64 events */
> > +     intc->domain = irq_domain_add_linear(dev->of_node,
> > MAX_PRU_SYS_EVENTS,
> > +                                          &pruss_intc_irq_domain_ops, intc);
> > +     if (!intc->domain)
> > +             return -ENOMEM;
> > +
> > +     for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
> > +             irq = platform_get_irq_byname(pdev, irq_names[i]);
> > +             if (irq <= 0) {
> > +                     dev_err(dev, "platform_get_irq_byname failed for %s : %d\n",
> > +                             irq_names[i], irq);
> > +                     goto fail_irq;
> > +             }
> > +
> > +             intc->irqs[i] = irq;
>
> Are the output IRQs guaranteed to be contiguous? If so, that'd be
> a much nicer way to work out which mux has fired (just store the
> base, and use it as an offset on handling the chained interrupt).

How about doing something like below in order to get rid of the for()
loop from the handler:

struct pruss_host_irq_data *host_data[]
...

host_data[i]->intc = intc;
host_data[i]->host_irq = i;

irq_set_handler_data(irq, host_data[i]);
irq_set_chained_handler(irq, pruss_intc_irq_handler);

>
> > +             irq_set_handler_data(irq, intc);
> > +             irq_set_chained_handler(irq, pruss_intc_irq_handler);
> > +     }
> > +
> > +     return 0;
> > +
> > +fail_irq:
> > +     while (--i >= 0)
> > +             irq_set_chained_handler_and_data(intc->irqs[i], NULL, NULL);
> > +
> > +     irq_domain_remove(intc->domain);
> > +
> > +     return irq;
> > +}
> > +
> > +static int pruss_intc_remove(struct platform_device *pdev)
> > +{
> > +     struct pruss_intc *intc = platform_get_drvdata(pdev);
> > +     unsigned int hwirq;
> > +     int i;
> > +
> > +     for (i = 0; i < MAX_NUM_HOST_IRQS; i++)
> > +             irq_set_chained_handler_and_data(intc->irqs[i], NULL, NULL);
> > +
> > +     for (hwirq = 0; hwirq < MAX_PRU_SYS_EVENTS; hwirq++)
> > +             irq_dispose_mapping(irq_find_mapping(intc->domain, hwirq));
> > +
> > +     irq_domain_remove(intc->domain);
> > +
> > +     return 0;
> > +}
> > +
> > +static const struct of_device_id pruss_intc_of_match[] = {
> > +     { .compatible = "ti,pruss-intc", },
> > +     { /* sentinel */ },
> > +};
> > +MODULE_DEVICE_TABLE(of, pruss_intc_of_match);
> > +
> > +static struct platform_driver pruss_intc_driver = {
> > +     .driver = {
> > +             .name = "pruss-intc",
> > +             .of_match_table = pruss_intc_of_match,
> > +             .suppress_bind_attrs = true,
> > +     },
> > +     .probe  = pruss_intc_probe,
> > +     .remove = pruss_intc_remove,
> > +};
> > +module_platform_driver(pruss_intc_driver);
> > +
> > +MODULE_AUTHOR("Andrew F. Davis <afd@ti.com>");
> > +MODULE_AUTHOR("Suman Anna <s-anna@ti.com>");
> > +MODULE_DESCRIPTION("TI PRU-ICSS INTC Driver");
> > +MODULE_LICENSE("GPL v2");
>
> Thanks,
>
>          M.

Thank you for your feedback and suggestions,
Grzegorz

Re: [PATCHv3 4/6] irqchip/irq-pruss-intc: Implement irq_{get,set}_irqchip_state ops

[Grzegorz Jaszczyk]

On Thu, 2 Jul 2020 at 19:54, Marc Zyngier <maz@kernel.org> wrote:
>
> On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
> > From: David Lechner <david@lechnology.com>
> >
> > This implements the irq_get_irqchip_state and irq_set_irqchip_state
> > callbacks for the TI PRUSS INTC driver. The set callback can be used
> > by drivers to "kick" a PRU by enabling a PRU system event.
>
> "enabling"? That'd be unmasking an interrupt, which isn't what this
> does. "injecting", maybe?

Yes "injecting" is much better.

>
> >
> > Example:
> >      irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, true);
>
> Nice example.
>
> What this example does explain is how you are actually going to kick
> a PRU via this interface. For that to happen, you'd have to have on
> the Linux side an interrupt that is actually routed to a PRU.

Correct.

> And from what I have understood of the previous patches, this can't
> be the case. What didi I miss?

The hwirq's handled by this driver are so called system events in
PRUSS nomenclature. This driver is responsible for the entire mapping
of those system events to PRUSS specific channels which are next
mapped to host_irq (patch #6 https://lkml.org/lkml/2020/7/2/612).
There are 8 host_irqs that are routed to the main cpu (running Linux)
and they are called host_intr0..host_intr7 (were seen in previous
patches of this series). But there are other "host_interrupts" that
are routed not to the main CPU but to PRU cores and this driver is
responsible for creating proper mapping (system
event/channel/host_irq) for them, and allowing to kick PRU via the
introduced interface.

It is worth noting that the PRUSS is quite flexible and allows various
things e.g.:
- map any of 160/64 internal system events to any of the 20/10 channels
- map any of the 20/10 channels to any of the 20/10 host interrupts.

So e.g. it is possible to map e.g. system event 17 to the main CPU
(through e.g. channel 1 which is the next map to e.g. host_intr0). Or
(exclusively) map the same system event 17 to PRU core (through e.g.
channel 1 which is the next map to PRU0).

>
> >
> > Signed-off-by: David Lechner <david@lechnology.com>
> > Signed-off-by: Suman Anna <s-anna@ti.com>
> > Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> > Reviewed-by: Lee Jones <lee.jones@linaro.org>
> > ---
> > v2->v3:
> > - Get rid of unnecessary pruss_intc_check_write() and use
> >   pruss_intc_write_reg directly.
> > v1->v2:
> > - https://patchwork.kernel.org/patch/11069769/
> > ---
> >  drivers/irqchip/irq-pruss-intc.c | 43
> > ++++++++++++++++++++++++++++++++++++++--
> >  1 file changed, 41 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/irqchip/irq-pruss-intc.c
> > b/drivers/irqchip/irq-pruss-intc.c
> > index 49c936f..19b3d38 100644
> > --- a/drivers/irqchip/irq-pruss-intc.c
> > +++ b/drivers/irqchip/irq-pruss-intc.c
> > @@ -7,6 +7,7 @@
> >   *   Suman Anna <s-anna@ti.com>
> >   */
> >
> > +#include <linux/interrupt.h>
> >  #include <linux/irq.h>
> >  #include <linux/irqchip/chained_irq.h>
> >  #include <linux/irqdomain.h>
> > @@ -39,8 +40,7 @@
> >  #define PRU_INTC_HIEISR              0x0034
> >  #define PRU_INTC_HIDISR              0x0038
> >  #define PRU_INTC_GPIR                0x0080
> > -#define PRU_INTC_SRSR0               0x0200
> > -#define PRU_INTC_SRSR1               0x0204
> > +#define PRU_INTC_SRSR(x)     (0x0200 + (x) * 4)
> >  #define PRU_INTC_SECR0               0x0280
> >  #define PRU_INTC_SECR1               0x0284
> >  #define PRU_INTC_ESR0                0x0300
> > @@ -145,6 +145,43 @@ static void pruss_intc_irq_relres(struct irq_data
> > *data)
> >       module_put(THIS_MODULE);
> >  }
> >
> > +static int pruss_intc_irq_get_irqchip_state(struct irq_data *data,
> > +                                         enum irqchip_irq_state which,
> > +                                         bool *state)
> > +{
> > +     struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
> > +     u32 reg, mask, srsr;
> > +
> > +     if (which != IRQCHIP_STATE_PENDING)
> > +             return -EINVAL;
> > +
> > +     reg = PRU_INTC_SRSR(data->hwirq / 32);
>
> I assume the register file scales as more interrupts are added in the
> subsequent patch?
>
Yes, after I will move part of the next patch to patch #2 as you
suggested it will stop being confusing.

Thank you,
Grzegorz

Re: [PATCHv3 5/6] irqchip/irq-pruss-intc: Add support for ICSSG INTC on K3 SoCs

[Grzegorz Jaszczyk]

On Thu, 2 Jul 2020 at 19:59, Marc Zyngier <maz@kernel.org> wrote:
>
> On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
> > From: Suman Anna <s-anna@ti.com>
> >
> > The K3 AM65x and J721E SoCs have the next generation of the PRU-ICSS
> > IP,
> > commonly called ICSSG. The PRUSS INTC present within the ICSSG supports
> > more System Events (160 vs 64), more Interrupt Channels and Host
> > Interrupts
> > (20 vs 10) compared to the previous generation PRUSS INTC instances.
> > The
> > first 2 and the last 10 of these host interrupt lines are used by the
> > PRU and other auxiliary cores and sub-modules within the ICSSG, with 8
> > host interrupts connected to MPU. The host interrupts 5, 6, 7 are also
> > connected to the other ICSSG instances within the SoC and can be
> > partitioned as per system integration through the board dts files.
> >
> > Enhance the PRUSS INTC driver to add support for this ICSSG INTC
> > instance. This support is added using specific compatible and match
> > data and updating the code to use this data instead of the current
> > hard-coded macros. The INTC config structure is updated to use the
> > higher events and channels on all SoCs, while limiting the actual
> > processing to only the relevant number of events/channels/interrupts.
> >
> > Signed-off-by: Suman Anna <s-anna@ti.com>
> > Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> > ---
> > v2->v3:
> > - Change patch order: use it directly after "irqchip/irq-pruss-intc:
> >   Implement irq_{get,set}_irqchip_state ops" and before new
> >   "irqchip/irq-pruss-intc: Add event mapping support" in order to
> > reduce
> >   diff.
>
> The diff would be even smaller if you introduced a variable number of
> inputs the first place, i.e. in patch #2. Most if this patch just
> retrofits it. Please squash these changes into that initial patch,
> and only add the platform stuff here.

Sure I will do that.

Thank you,
Grzegorz

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Marc Zyngier]

On 2020-07-03 15:28, Grzegorz Jaszczyk wrote:
> On Thu, 2 Jul 2020 at 19:24, Marc Zyngier <maz@kernel.org> wrote:
>> 
>> On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
>> > From: Suman Anna <s-anna@ti.com>
>> >
>> > The Programmable Real-Time Unit Subsystem (PRUSS) contains a local
>> > interrupt controller (INTC) that can handle various system input events
>> > and post interrupts back to the device-level initiators. The INTC can
>> > support upto 64 input events with individual control configuration and
>> > hardware prioritization. These events are mapped onto 10 output
>> > interrupt
>> > lines through two levels of many-to-one mapping support. Different
>> > interrupt lines are routed to the individual PRU cores or to the host
>> > CPU, or to other devices on the SoC. Some of these events are sourced
>> > from peripherals or other sub-modules within that PRUSS, while a few
>> > others are sourced from SoC-level peripherals/devices.
>> >
>> > The PRUSS INTC platform driver manages this PRUSS interrupt controller
>> > and implements an irqchip driver to provide a Linux standard way for
>> > the PRU client users to enable/disable/ack/re-trigger a PRUSS system
>> > event. The system events to interrupt channels and output interrupts
>> > relies on the mapping configuration provided either through the PRU
>> > firmware blob or via the PRU application's device tree node. The
>> > mappings will be programmed during the boot/shutdown of a PRU core.
>> >
>> > The PRUSS INTC module is reference counted during the interrupt
>> > setup phase through the irqchip's irq_request_resources() and
>> > irq_release_resources() ops. This restricts the module from being
>> > removed as long as there are active interrupt users.
>> >
>> > The driver currently supports and can be built for OMAP architecture
>> > based AM335x, AM437x and AM57xx SoCs; Keystone2 architecture based
>> > 66AK2G SoCs and Davinci architecture based OMAP-L13x/AM18x/DA850 SoCs.
>> > All of these SoCs support 64 system events, 10 interrupt channels and
>> > 10 output interrupt lines per PRUSS INTC with a few SoC integration
>> > differences.
>> >
>> > NOTE:
>> > Each PRU-ICSS's INTC on AM57xx SoCs is preceded by a Crossbar that
>> > enables multiple external events to be routed to a specific number
>> > of input interrupt events. Any non-default external interrupt event
>> > directed towards PRUSS needs this crossbar to be setup properly.
>> >
>> > Signed-off-by: Suman Anna <s-anna@ti.com>
>> > Signed-off-by: Andrew F. Davis <afd@ti.com>
>> > Signed-off-by: Roger Quadros <rogerq@ti.com>
>> > Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
>> > Reviewed-by: Lee Jones <lee.jones@linaro.org>
>> > ---
>> > v2->v3:
>> > - use single irqchip description instead of separately allocating it
>> > for
>> >   each pruss_intc
>> > - get rid of unused mutex
>> > - improve error handling
>> > v1->v2:
>> > - https://patchwork.kernel.org/patch/11069771/
> <snip>
>> > +static void pruss_intc_init(struct pruss_intc *intc)
>> > +{
>> > +     int i;
>> > +
>> > +     /* configure polarity to active high for all system interrupts */
>> > +     pruss_intc_write_reg(intc, PRU_INTC_SIPR0, 0xffffffff);
>> > +     pruss_intc_write_reg(intc, PRU_INTC_SIPR1, 0xffffffff);
>> > +
>> > +     /* configure type to pulse interrupt for all system interrupts */
>> > +     pruss_intc_write_reg(intc, PRU_INTC_SITR0, 0);
>> > +     pruss_intc_write_reg(intc, PRU_INTC_SITR1, 0);
>> 
>> So the default is to configure everything as edge...
> 
> Sorry, the description is wrong - '0' indicates level and '1' edge. So
> the default configuration is level - I will fix the comment.
> 
>> 
>> > +
>> > +     /* clear all 16 interrupt channel map registers */
>> > +     for (i = 0; i < 16; i++)
>> > +             pruss_intc_write_reg(intc, PRU_INTC_CMR(i), 0);
>> > +
>> > +     /* clear all 3 host interrupt map registers */
>> > +     for (i = 0; i < 3; i++)
>> > +             pruss_intc_write_reg(intc, PRU_INTC_HMR(i), 0);
>> > +}
>> > +
>> > +static void pruss_intc_irq_ack(struct irq_data *data)
>> > +{
>> > +     struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
>> > +     unsigned int hwirq = data->hwirq;
>> > +
>> > +     pruss_intc_write_reg(intc, PRU_INTC_SICR, hwirq);
>> > +}
>> > +
>> > +static void pruss_intc_irq_mask(struct irq_data *data)
>> > +{
>> > +     struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
>> > +     unsigned int hwirq = data->hwirq;
>> > +
>> > +     pruss_intc_write_reg(intc, PRU_INTC_EICR, hwirq);
>> > +}
>> > +
>> > +static void pruss_intc_irq_unmask(struct irq_data *data)
>> > +{
>> > +     struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
>> > +     unsigned int hwirq = data->hwirq;
>> > +
>> > +     pruss_intc_write_reg(intc, PRU_INTC_EISR, hwirq);
>> > +}
>> > +
>> > +static int pruss_intc_irq_reqres(struct irq_data *data)
>> > +{
>> > +     if (!try_module_get(THIS_MODULE))
>> > +             return -ENODEV;
>> > +
>> > +     return 0;
>> > +}
>> > +
>> > +static void pruss_intc_irq_relres(struct irq_data *data)
>> > +{
>> > +     module_put(THIS_MODULE);
>> > +}
>> > +
>> > +static struct irq_chip pruss_irqchip = {
>> > +     .name = "pruss-intc",
>> > +     .irq_ack = pruss_intc_irq_ack,
>> > +     .irq_mask = pruss_intc_irq_mask,
>> > +     .irq_unmask = pruss_intc_irq_unmask,
>> > +     .irq_request_resources = pruss_intc_irq_reqres,
>> > +     .irq_release_resources = pruss_intc_irq_relres,
>> > +};
>> > +
>> > +static int pruss_intc_irq_domain_map(struct irq_domain *d, unsigned
>> > int virq,
>> > +                                  irq_hw_number_t hw)
>> > +{
>> > +     struct pruss_intc *intc = d->host_data;
>> > +
>> > +     irq_set_chip_data(virq, intc);
>> > +     irq_set_chip_and_handler(virq, &pruss_irqchip, handle_level_irq);
>> 
>> ... and despite this edge-triggered default, you handle things as 
>> level.
>> This doesn't seem quite right.
> 
> As above it is level. I will fix the comment

It still begs the question: if the HW can support both edge and level
triggered interrupts, why isn't the driver supporting this diversity?
I appreciate that your HW may only have level interrupts so far, but
what guarantees that this will forever be true? It would imply a change
in the DT binding, which isn't desirable.

> 
>> 
>> > +
>> > +     return 0;
>> > +}
>> > +
>> > +static void pruss_intc_irq_domain_unmap(struct irq_domain *d,
>> > unsigned int virq)
>> > +{
>> > +     irq_set_chip_and_handler(virq, NULL, NULL);
>> > +     irq_set_chip_data(virq, NULL);
>> > +}
>> > +
>> > +static const struct irq_domain_ops pruss_intc_irq_domain_ops = {
>> > +     .xlate  = irq_domain_xlate_onecell,
>> > +     .map    = pruss_intc_irq_domain_map,
>> > +     .unmap  = pruss_intc_irq_domain_unmap,
>> > +};
>> > +
>> > +static void pruss_intc_irq_handler(struct irq_desc *desc)
>> > +{
>> > +     unsigned int irq = irq_desc_get_irq(desc);
>> > +     struct irq_chip *chip = irq_desc_get_chip(desc);
>> > +     struct pruss_intc *intc = irq_get_handler_data(irq);
>> > +     u32 hipir;
>> > +     unsigned int virq;
>> > +     int i, hwirq;
>> > +
>> > +     chained_irq_enter(chip, desc);
>> > +
>> > +     /* find our host irq number */
>> > +     for (i = 0; i < MAX_NUM_HOST_IRQS; i++)
>> > +             if (intc->irqs[i] == irq)
>> > +                     break;
>> 
>> This loop is pretty ugly. The way to do it would normally to
>> associate the right data structure to the chained interrupt,
>> and only that one, directly associating the input signal
>> with the correct mux. Using the Linux irq as a discriminant is
>> at best clumsy.
>> 
>> But it feels to me that the base data structure is not
>> exactly the right one here, see below.
>> 
> 
> Ok, you are right. I will introduce a new structure for host_irq data
> which will be associated with chained interrupt and get rid of this
> loop.
> 
>> > +     if (i == MAX_NUM_HOST_IRQS)
>> > +             goto err;
>> > +
>> > +     i += MIN_PRU_HOST_INT;
>> > +
>> > +     /* get highest priority pending PRUSS system event */
>> > +     hipir = pruss_intc_read_reg(intc, PRU_INTC_HIPIR(i));
>> > +     while (!(hipir & INTC_HIPIR_NONE_HINT)) {
>> 
>> Please write this as a do { } while() loop, with a single instance
>> of the HW register read inside the loop (instead of one outside
>> and one inside.
> 
> Ok, I will get rid of the outside HW register read, but I think it is
> better to use bellow instead of do {} while () loop:
> while (1) {
>   /* get highest priority pending PRUSS system event */
>   hipir = pruss_intc_read_reg(intc, PRU_INTC_HIPIR(host_irq));
>   if (hipir & INTC_HIPIR_NONE_HINT)
>     break;
> ...
> 
> Hope it works for you.

Up to you. I don't understand your allergy to do {} while(), but
as long as there is only a single read of the register to deal
with, I'm fine with it.

> 
>> 
>> > +             hwirq = hipir & GENMASK(9, 0);
>> > +             virq = irq_linear_revmap(intc->domain, hwirq);
>> 
>> And this is where I worry. You seems to have a single irqdomain
>> for all the muxes. Are you guaranteed that you will have no
>> overlap between muxes? And please use irq_find_mapping(), as
>> I have top-secret plans to kill irq_linear_revmap().
> 
> Regarding irq_find_mapping - sure.
> 
> Regarding irqdomains:
> It is a single irqdomain since the hwirq (system event) can be mapped
> to different irq_host (muxes). Patch #6
> https://lkml.org/lkml/2020/7/2/616 implements and describes how input
> events can be mapped to some output host interrupts through 2 levels
> of many-to-one mapping i.e. events to channel mapping and channels to
> host interrupts. Mentioned implementation ensures that specific system
> event (hwirq) can be mapped through PRUSS specific channel into a
> single host interrupt.

Patch #6 is a nightmare of its own, and I haven't fully groked it yet.
Also, this driver seems to totally ignore the 2-level routing. Where
is it set up? map/unmap in this driver do exactly *nothing*, so
something somewhere must set it up.

>> 
>> > +
>> > +             /*
>> > +              * NOTE: manually ACK any system events that do not have a
>> > +              * handler mapped yet
>> > +              */
>> > +             if (WARN_ON(!virq))
>> > +                     pruss_intc_write_reg(intc, PRU_INTC_SICR, hwirq);
>> 
>> How can this happen? If you really need it, you probable want to
>> warn once only.
> 
> Ideally it shouldn't happen but I prefer to keep it to catch any
> misuse. It is because the PRUSS INTC unit can be also accessed by PRU
> cores which use the same registers to ack the internal events. The
> current design is limited to only acking and triggering the interrupts
> from PRU firmwares while the entire mapping is done by Linux (patch #6
> https://lkml.org/lkml/2020/7/2/612).

So patch #6 deals with routing of interrupts that are not aimed at Linux
(humf...), but nothing deals with the routing of interrupts that Linux
must handle. Why?

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Grzegorz Jaszczyk]

On Sat, 4 Jul 2020 at 11:39, Marc Zyngier <maz@kernel.org> wrote:
>
> On 2020-07-03 15:28, Grzegorz Jaszczyk wrote:
> > On Thu, 2 Jul 2020 at 19:24, Marc Zyngier <maz@kernel.org> wrote:
> >>
> >> On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
> >> > From: Suman Anna <s-anna@ti.com>
> >> >
> >> > The Programmable Real-Time Unit Subsystem (PRUSS) contains a local
> >> > interrupt controller (INTC) that can handle various system input events
> >> > and post interrupts back to the device-level initiators. The INTC can
> >> > support upto 64 input events with individual control configuration and
> >> > hardware prioritization. These events are mapped onto 10 output
> >> > interrupt
> >> > lines through two levels of many-to-one mapping support. Different
> >> > interrupt lines are routed to the individual PRU cores or to the host
> >> > CPU, or to other devices on the SoC. Some of these events are sourced
> >> > from peripherals or other sub-modules within that PRUSS, while a few
> >> > others are sourced from SoC-level peripherals/devices.
> >> >
> >> > The PRUSS INTC platform driver manages this PRUSS interrupt controller
> >> > and implements an irqchip driver to provide a Linux standard way for
> >> > the PRU client users to enable/disable/ack/re-trigger a PRUSS system
> >> > event. The system events to interrupt channels and output interrupts
> >> > relies on the mapping configuration provided either through the PRU
> >> > firmware blob or via the PRU application's device tree node. The
> >> > mappings will be programmed during the boot/shutdown of a PRU core.
> >> >
> >> > The PRUSS INTC module is reference counted during the interrupt
> >> > setup phase through the irqchip's irq_request_resources() and
> >> > irq_release_resources() ops. This restricts the module from being
> >> > removed as long as there are active interrupt users.
> >> >
> >> > The driver currently supports and can be built for OMAP architecture
> >> > based AM335x, AM437x and AM57xx SoCs; Keystone2 architecture based
> >> > 66AK2G SoCs and Davinci architecture based OMAP-L13x/AM18x/DA850 SoCs.
> >> > All of these SoCs support 64 system events, 10 interrupt channels and
> >> > 10 output interrupt lines per PRUSS INTC with a few SoC integration
> >> > differences.
> >> >
> >> > NOTE:
> >> > Each PRU-ICSS's INTC on AM57xx SoCs is preceded by a Crossbar that
> >> > enables multiple external events to be routed to a specific number
> >> > of input interrupt events. Any non-default external interrupt event
> >> > directed towards PRUSS needs this crossbar to be setup properly.
> >> >
> >> > Signed-off-by: Suman Anna <s-anna@ti.com>
> >> > Signed-off-by: Andrew F. Davis <afd@ti.com>
> >> > Signed-off-by: Roger Quadros <rogerq@ti.com>
> >> > Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> >> > Reviewed-by: Lee Jones <lee.jones@linaro.org>
> >> > ---
> >> > v2->v3:
> >> > - use single irqchip description instead of separately allocating it
> >> > for
> >> >   each pruss_intc
> >> > - get rid of unused mutex
> >> > - improve error handling
> >> > v1->v2:
> >> > - https://patchwork.kernel.org/patch/11069771/
> > <snip>
> >> > +static void pruss_intc_init(struct pruss_intc *intc)
> >> > +{
> >> > +     int i;
> >> > +
> >> > +     /* configure polarity to active high for all system interrupts */
> >> > +     pruss_intc_write_reg(intc, PRU_INTC_SIPR0, 0xffffffff);
> >> > +     pruss_intc_write_reg(intc, PRU_INTC_SIPR1, 0xffffffff);
> >> > +
> >> > +     /* configure type to pulse interrupt for all system interrupts */
> >> > +     pruss_intc_write_reg(intc, PRU_INTC_SITR0, 0);
> >> > +     pruss_intc_write_reg(intc, PRU_INTC_SITR1, 0);
> >>
> >> So the default is to configure everything as edge...
> >
> > Sorry, the description is wrong - '0' indicates level and '1' edge. So
> > the default configuration is level - I will fix the comment.
> >
> >>
> >> > +
> >> > +     /* clear all 16 interrupt channel map registers */
> >> > +     for (i = 0; i < 16; i++)
> >> > +             pruss_intc_write_reg(intc, PRU_INTC_CMR(i), 0);
> >> > +
> >> > +     /* clear all 3 host interrupt map registers */
> >> > +     for (i = 0; i < 3; i++)
> >> > +             pruss_intc_write_reg(intc, PRU_INTC_HMR(i), 0);
> >> > +}
> >> > +
> >> > +static void pruss_intc_irq_ack(struct irq_data *data)
> >> > +{
> >> > +     struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
> >> > +     unsigned int hwirq = data->hwirq;
> >> > +
> >> > +     pruss_intc_write_reg(intc, PRU_INTC_SICR, hwirq);
> >> > +}
> >> > +
> >> > +static void pruss_intc_irq_mask(struct irq_data *data)
> >> > +{
> >> > +     struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
> >> > +     unsigned int hwirq = data->hwirq;
> >> > +
> >> > +     pruss_intc_write_reg(intc, PRU_INTC_EICR, hwirq);
> >> > +}
> >> > +
> >> > +static void pruss_intc_irq_unmask(struct irq_data *data)
> >> > +{
> >> > +     struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
> >> > +     unsigned int hwirq = data->hwirq;
> >> > +
> >> > +     pruss_intc_write_reg(intc, PRU_INTC_EISR, hwirq);
> >> > +}
> >> > +
> >> > +static int pruss_intc_irq_reqres(struct irq_data *data)
> >> > +{
> >> > +     if (!try_module_get(THIS_MODULE))
> >> > +             return -ENODEV;
> >> > +
> >> > +     return 0;
> >> > +}
> >> > +
> >> > +static void pruss_intc_irq_relres(struct irq_data *data)
> >> > +{
> >> > +     module_put(THIS_MODULE);
> >> > +}
> >> > +
> >> > +static struct irq_chip pruss_irqchip = {
> >> > +     .name = "pruss-intc",
> >> > +     .irq_ack = pruss_intc_irq_ack,
> >> > +     .irq_mask = pruss_intc_irq_mask,
> >> > +     .irq_unmask = pruss_intc_irq_unmask,
> >> > +     .irq_request_resources = pruss_intc_irq_reqres,
> >> > +     .irq_release_resources = pruss_intc_irq_relres,
> >> > +};
> >> > +
> >> > +static int pruss_intc_irq_domain_map(struct irq_domain *d, unsigned
> >> > int virq,
> >> > +                                  irq_hw_number_t hw)
> >> > +{
> >> > +     struct pruss_intc *intc = d->host_data;
> >> > +
> >> > +     irq_set_chip_data(virq, intc);
> >> > +     irq_set_chip_and_handler(virq, &pruss_irqchip, handle_level_irq);
> >>
> >> ... and despite this edge-triggered default, you handle things as
> >> level.
> >> This doesn't seem quite right.
> >
> > As above it is level. I will fix the comment
>
> It still begs the question: if the HW can support both edge and level
> triggered interrupts, why isn't the driver supporting this diversity?
> I appreciate that your HW may only have level interrupts so far, but
> what guarantees that this will forever be true? It would imply a change
> in the DT binding, which isn't desirable.

Ok, I've got your point. I will try to come up with something later
on. Probably extending interrupt-cells by one and passing interrupt
type will be enough for now. Extending this driver to actually support
it can be handled later if needed. Hope it works for you.

>
> >
> >>
> >> > +
> >> > +     return 0;
> >> > +}
> >> > +
> >> > +static void pruss_intc_irq_domain_unmap(struct irq_domain *d,
> >> > unsigned int virq)
> >> > +{
> >> > +     irq_set_chip_and_handler(virq, NULL, NULL);
> >> > +     irq_set_chip_data(virq, NULL);
> >> > +}
> >> > +
> >> > +static const struct irq_domain_ops pruss_intc_irq_domain_ops = {
> >> > +     .xlate  = irq_domain_xlate_onecell,
> >> > +     .map    = pruss_intc_irq_domain_map,
> >> > +     .unmap  = pruss_intc_irq_domain_unmap,
> >> > +};
> >> > +
> >> > +static void pruss_intc_irq_handler(struct irq_desc *desc)
> >> > +{
> >> > +     unsigned int irq = irq_desc_get_irq(desc);
> >> > +     struct irq_chip *chip = irq_desc_get_chip(desc);
> >> > +     struct pruss_intc *intc = irq_get_handler_data(irq);
> >> > +     u32 hipir;
> >> > +     unsigned int virq;
> >> > +     int i, hwirq;
> >> > +
> >> > +     chained_irq_enter(chip, desc);
> >> > +
> >> > +     /* find our host irq number */
> >> > +     for (i = 0; i < MAX_NUM_HOST_IRQS; i++)
> >> > +             if (intc->irqs[i] == irq)
> >> > +                     break;
> >>
> >> This loop is pretty ugly. The way to do it would normally to
> >> associate the right data structure to the chained interrupt,
> >> and only that one, directly associating the input signal
> >> with the correct mux. Using the Linux irq as a discriminant is
> >> at best clumsy.
> >>
> >> But it feels to me that the base data structure is not
> >> exactly the right one here, see below.
> >>
> >
> > Ok, you are right. I will introduce a new structure for host_irq data
> > which will be associated with chained interrupt and get rid of this
> > loop.
> >
> >> > +     if (i == MAX_NUM_HOST_IRQS)
> >> > +             goto err;
> >> > +
> >> > +     i += MIN_PRU_HOST_INT;
> >> > +
> >> > +     /* get highest priority pending PRUSS system event */
> >> > +     hipir = pruss_intc_read_reg(intc, PRU_INTC_HIPIR(i));
> >> > +     while (!(hipir & INTC_HIPIR_NONE_HINT)) {
> >>
> >> Please write this as a do { } while() loop, with a single instance
> >> of the HW register read inside the loop (instead of one outside
> >> and one inside.
> >
> > Ok, I will get rid of the outside HW register read, but I think it is
> > better to use bellow instead of do {} while () loop:
> > while (1) {
> >   /* get highest priority pending PRUSS system event */
> >   hipir = pruss_intc_read_reg(intc, PRU_INTC_HIPIR(host_irq));
> >   if (hipir & INTC_HIPIR_NONE_HINT)
> >     break;
> > ...
> >
> > Hope it works for you.
>
> Up to you. I don't understand your allergy to do {} while(), but
> as long as there is only a single read of the register to deal
> with, I'm fine with it.

Ok.

>
> >
> >>
> >> > +             hwirq = hipir & GENMASK(9, 0);
> >> > +             virq = irq_linear_revmap(intc->domain, hwirq);
> >>
> >> And this is where I worry. You seems to have a single irqdomain
> >> for all the muxes. Are you guaranteed that you will have no
> >> overlap between muxes? And please use irq_find_mapping(), as
> >> I have top-secret plans to kill irq_linear_revmap().
> >
> > Regarding irq_find_mapping - sure.
> >
> > Regarding irqdomains:
> > It is a single irqdomain since the hwirq (system event) can be mapped
> > to different irq_host (muxes). Patch #6
> > https://lkml.org/lkml/2020/7/2/616 implements and describes how input
> > events can be mapped to some output host interrupts through 2 levels
> > of many-to-one mapping i.e. events to channel mapping and channels to
> > host interrupts. Mentioned implementation ensures that specific system
> > event (hwirq) can be mapped through PRUSS specific channel into a
> > single host interrupt.
>
> Patch #6 is a nightmare of its own, and I haven't fully groked it yet.
> Also, this driver seems to totally ignore the 2-level routing. Where
> is it set up? map/unmap in this driver do exactly *nothing*, so
> something somewhere must set it up.

The map/unmap is updated in patch #6 and it deals with those 2-level
routing setup. Map is responsible for programming the Channel Map
Registers (CMRx) and Host-Interrupt Map Registers (HMRx) basing on
provided configuration from the one parsed in the xlate function.
Unmap undo whatever was done on the map. More details can be found in
patch #6.

Maybe it would be better to squash patch #6 with this one so it would
be less confusing. What is your advice?

>
> >>
> >> > +
> >> > +             /*
> >> > +              * NOTE: manually ACK any system events that do not have a
> >> > +              * handler mapped yet
> >> > +              */
> >> > +             if (WARN_ON(!virq))
> >> > +                     pruss_intc_write_reg(intc, PRU_INTC_SICR, hwirq);
> >>
> >> How can this happen? If you really need it, you probable want to
> >> warn once only.
> >
> > Ideally it shouldn't happen but I prefer to keep it to catch any
> > misuse. It is because the PRUSS INTC unit can be also accessed by PRU
> > cores which use the same registers to ack the internal events. The
> > current design is limited to only acking and triggering the interrupts
> > from PRU firmwares while the entire mapping is done by Linux (patch #6
> > https://lkml.org/lkml/2020/7/2/612).
>
> So patch #6 deals with routing of interrupts that are not aimed at Linux
> (humf...), but nothing deals with the routing of interrupts that Linux
> must handle. Why?

Actually patch #6 deals with the entire routing of all PRUSS related
interrupts: the ones that *are* aimed at Linux and ones that are not.

The PRU core is responsible only for acking interrupts that are routed
to the PRU core and triggering interrupts from PRU core (e.g. from PRU
to main CPU). All other actions related to PRUSS INTC, including the
entire 2-level routing setup (patch #6), are handled by this Linux
driver.

Best regards,
Grzegorz

Re: [PATCHv3 6/6] irqchip/irq-pruss-intc: Add event mapping support

[Grzegorz Jaszczyk]

Hi Suman,

On Thu, 2 Jul 2020 at 18:25, Suman Anna <s-anna@ti.com> wrote:
>
> Hi Greg,
>
> On 7/2/20 9:17 AM, Grzegorz Jaszczyk wrote:
> > The PRUSS INTC receives a number of system input interrupt source events
> > and supports individual control configuration and hardware prioritization.
> > These input events can be mapped to some output host interrupts through 2
> > levels of many-to-one mapping i.e. events to channel mapping and channels
> > to host interrupts.
> >
> > This mapping information is provided through the PRU firmware that is
> > loaded onto a PRU core/s or through the device tree node of the PRU
> > application. The mapping configuration is triggered by the PRU
> > remoteproc driver, and is setup before the PRU core is started and
> > cleaned up after the PRU core is stopped. This event mapping
> > configuration logic programs the Channel Map Registers (CMRx) and
> > Host-Interrupt Map Registers (HMRx) only when a new program is being
> > loaded/started and the same events and interrupt channels are reset to
> > zero when stopping a PRU.
> >
> > Reference counting is used to allow multiple system events to share a
> > single channel and to allow multiple channels to share a single host
> > event.
> >
> > The remoteproc driver can register mappings read from a firmware blob
> > as shown below.
> >
> >       struct irq_fwspec fwspec;
> >       int irq;
> >
> >       fwspec.fwnode = of_node_to_fwnode(dev->of_node);
> >       fwspec.param_count = 3;
> >       fwspec.param[0] = 63; // system event
> >       fwspec.param[1] = 5;  // channel
> >       fwspec.param[2] = 6;  // host event
> >
> >       irq = irq_create_fwspec_mapping(&fwspec);
> >       if (irq < 0) {
> >               dev_err(dev, "failed to get irq\n");
> >               return irq;
> >       }
> >
> > Suggested-by: David Lechner <david@lechnology.com>
> > Signed-off-by: Suman Anna <s-anna@ti.com>
> > Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> > ---
> > v3:
> > - This patch replaces https://patchwork.kernel.org/patch/11069753/
> >    according to received feedback. Instead of exporting custom functions
> >    from interrupt driver, the xlate and irq domain map is used for
> >    interrupt parsing and mapping.
>
> Thanks for reworking this. Only have couple of very minor comments below.
>
> > ---
> >   drivers/irqchip/irq-pruss-intc.c | 265 ++++++++++++++++++++++++++++++++++++++-
> >   1 file changed, 264 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/irqchip/irq-pruss-intc.c b/drivers/irqchip/irq-pruss-intc.c
> > index 362aa01..cf40a97 100644
> > --- a/drivers/irqchip/irq-pruss-intc.c
> > +++ b/drivers/irqchip/irq-pruss-intc.c
> > @@ -51,14 +51,42 @@
> >   #define PRU_INTC_HIER               0x1500
> >
> >   /* CMR register bit-field macros */
> > +#define CMR_EVT_MAP_MASK     0xf
> > +#define CMR_EVT_MAP_BITS     8
> >   #define CMR_EVT_PER_REG             4
> >
> >   /* HMR register bit-field macros */
> > +#define HMR_CH_MAP_MASK              0xf
> > +#define HMR_CH_MAP_BITS              8
> >   #define HMR_CH_PER_REG              4
> >
> >   /* HIPIR register bit-fields */
> >   #define INTC_HIPIR_NONE_HINT        0x80000000
> >
> > +#define MAX_PRU_SYS_EVENTS 160
> > +#define MAX_PRU_CHANNELS 20
> > +
> > +/**
> > + * struct pruss_intc_hwirq_data - additional metadata associated with a PRU
> > + * system event
> > + * @channel: The PRU INTC channel that the system event should be mapped to
> > + * @host: The PRU INTC host that the channel should be mapped to
> > + */
> > +struct pruss_intc_hwirq_data {
> > +     u8 channel;
> > +     u8 host;
> > +};
> > +
> > +/**
> > + * struct pruss_intc_map_record - keeps track of actual mapping state
> > + * @value: The currently mapped value (channel or host)
> > + * @ref_count: Keeps track of number of current users of this resource
> > + */
> > +struct pruss_intc_map_record {
> > +     u8 value;
> > +     u8 ref_count;
> > +};
> > +
> >   /**
> >    * struct pruss_intc_match_data - match data to handle SoC variations
> >    * @num_system_events: number of input system events handled by the PRUSS INTC
> > @@ -71,18 +99,29 @@ struct pruss_intc_match_data {
> >
> >   /**
> >    * struct pruss_intc - PRUSS interrupt controller structure
> > + * @hwirq_data: table of additional mapping data received from device tree
> > + *   or PRU firmware
> > + * @event_channel: current state of system event to channel mappings
> > + * @channel_host: current state of channel to host mappings
> >    * @irqs: kernel irq numbers corresponding to PRUSS host interrupts
> >    * @base: base virtual address of INTC register space
> >    * @domain: irq domain for this interrupt controller
> >    * @soc_config: cached PRUSS INTC IP configuration data
> > + * @lock: mutex to serialize access to INTC
> > + * @dev: PRUSS INTC device pointer
> >    * @shared_intr: bit-map denoting if the MPU host interrupt is shared
> >    * @invalid_intr: bit-map denoting if host interrupt is not connected to MPU
> >    */
> >   struct pruss_intc {
> > +     struct pruss_intc_hwirq_data hwirq_data[MAX_PRU_SYS_EVENTS];
> > +     struct pruss_intc_map_record event_channel[MAX_PRU_SYS_EVENTS];
> > +     struct pruss_intc_map_record channel_host[MAX_PRU_CHANNELS];
> >       unsigned int irqs[MAX_NUM_HOST_IRQS];
> >       void __iomem *base;
> >       struct irq_domain *domain;
> >       const struct pruss_intc_match_data *soc_config;
> > +     struct mutex lock; /* PRUSS INTC lock */
> > +     struct device *dev;
> >       u16 shared_intr;
> >       u16 invalid_intr;
> >   };
> > @@ -98,6 +137,165 @@ static inline void pruss_intc_write_reg(struct pruss_intc *intc,
> >       writel_relaxed(val, intc->base + reg);
> >   }
> >
> > +static void pruss_intc_update_cmr(struct pruss_intc *intc, int evt, s8 ch)
> > +{
> > +     u32 idx, offset, val;
> > +
> > +     idx = evt / CMR_EVT_PER_REG;
> > +     offset = (evt % CMR_EVT_PER_REG) * CMR_EVT_MAP_BITS;
> > +
> > +     val = pruss_intc_read_reg(intc, PRU_INTC_CMR(idx));
> > +     val &= ~(CMR_EVT_MAP_MASK << offset);
> > +     val |= ch << offset;
> > +     pruss_intc_write_reg(intc, PRU_INTC_CMR(idx), val);
> > +
> > +     dev_dbg(intc->dev, "SYSEV%u -> CH%d (CMR%d 0x%08x)\n", evt, ch,
> > +             idx, pruss_intc_read_reg(intc, PRU_INTC_CMR(idx)));
> > +}
> > +
> > +static void pruss_intc_update_hmr(struct pruss_intc *intc, int ch, s8 host)
> > +{
> > +     u32 idx, offset, val;
> > +
> > +     idx = ch / HMR_CH_PER_REG;
> > +     offset = (ch % HMR_CH_PER_REG) * HMR_CH_MAP_BITS;
> > +
> > +     val = pruss_intc_read_reg(intc, PRU_INTC_HMR(idx));
> > +     val &= ~(HMR_CH_MAP_MASK << offset);
> > +     val |= host << offset;
> > +     pruss_intc_write_reg(intc, PRU_INTC_HMR(idx), val);
> > +
> > +     dev_dbg(intc->dev, "CH%d -> HOST%d (HMR%d 0x%08x)\n", ch, host, idx,
> > +             pruss_intc_read_reg(intc, PRU_INTC_HMR(idx)));
> > +}
> > +
> > +/**
> > + * pruss_intc_map() - configure the PRUSS INTC
> > + * @intc: PRUSS interrupt controller pointer
> > + * @hwirq: the system event number
> > + *
> > + * Configures the PRUSS INTC with the provided configuration from the one
> > + * parsed in the xlate function. Any existing event to channel mappings or
> > + * channel to host interrupt mappings are checked to make sure there are no
> > + * conflicting configuration between both the PRU cores.
> > + *
> > + * Returns 0 on success, or a suitable error code otherwise
> > + */
> > +static int pruss_intc_map(struct pruss_intc *intc, unsigned long hwirq)
> > +{
> > +     struct device *dev = intc->dev;
> > +     int ret = 0;
> > +     u8 ch, host, reg_idx;
> > +     u32 val;
> > +
> > +     if (hwirq >= intc->soc_config->num_system_events)
> > +             return -EINVAL;
> > +
> > +     mutex_lock(&intc->lock);
> > +
> > +     ch = intc->hwirq_data[hwirq].channel;
> > +     host = intc->hwirq_data[hwirq].host;
> > +
> > +     /* check if sysevent already assigned */
> > +     if (intc->event_channel[hwirq].ref_count > 0 &&
> > +         intc->event_channel[hwirq].value != ch) {
> > +             dev_err(dev, "event %lu (req. channel %d) already assigned to channel %d\n",
> > +                     hwirq, ch, intc->event_channel[hwirq].value);
> > +             ret = -EBUSY;
> > +             goto unlock;
> > +     }
> > +
> > +     /* check if channel already assigned */
> > +     if (intc->channel_host[ch].ref_count > 0 &&
> > +         intc->channel_host[ch].value != host) {
> > +             dev_err(dev, "channel %d (req. host %d) already assigned to host %d\n",
> > +                     ch, host, intc->channel_host[ch].value);
> > +             ret = -EBUSY;
> > +             goto unlock;
> > +     }
> > +
> > +     if (++intc->event_channel[hwirq].ref_count == 1) {
> > +             intc->event_channel[hwirq].value = ch;
> > +
> > +             pruss_intc_update_cmr(intc, hwirq, ch);
> > +
> > +             reg_idx = hwirq / 32;
> > +             val = BIT(hwirq  % 32);
> > +
> > +             /* clear and enable system event */
> > +             pruss_intc_write_reg(intc, PRU_INTC_ESR(reg_idx), val);
> > +             pruss_intc_write_reg(intc, PRU_INTC_SECR(reg_idx), val);
> > +     }
> > +
> > +     if (++intc->channel_host[ch].ref_count == 1) {
> > +             intc->channel_host[ch].value = host;
> > +
> > +             pruss_intc_update_hmr(intc, ch, host);
> > +
> > +             /* enable host interrupts */
> > +             pruss_intc_write_reg(intc, PRU_INTC_HIEISR, host);
> > +     }
> > +
> > +     dev_dbg(dev, "mapped system_event = %lu channel = %d host = %d",
> > +              hwirq, ch, host);
> > +
> > +     /* global interrupt enable */
> > +     pruss_intc_write_reg(intc, PRU_INTC_GER, 1);
> > +
> > +unlock:
> > +     mutex_unlock(&intc->lock);
> > +     return ret;
> > +}
> > +
> > +/**
> > + * pruss_intc_unmap() - unconfigure the PRUSS INTC
> > + * @intc: PRUSS interrupt controller pointer
> > + * @hwirq: the system event number
> > + *
> > + * Undo whatever was done in pruss_intc_map() for a PRU core.
> > + * Mappings are reference counted, so resources are only disabled when there
> > + * are no longer any users.
> > + */
> > +static void pruss_intc_unmap(struct pruss_intc *intc, unsigned long hwirq)
> > +{
> > +     u8 ch, host, reg_idx;
> > +     u32 val;
> > +
> > +     if (hwirq >= intc->soc_config->num_system_events)
> > +             return;
> > +
> > +     mutex_lock(&intc->lock);
> > +
> > +     ch = intc->event_channel[hwirq].value;
> > +     host = intc->channel_host[ch].value;
> > +
> > +     if (--intc->channel_host[ch].ref_count == 0) {
> > +             /* disable host interrupts */
> > +             pruss_intc_write_reg(intc, PRU_INTC_HIDISR, host);
> > +
> > +             /* clear the map using reset value 0 */
> > +             pruss_intc_update_hmr(intc, ch, 0);
> > +     }
> > +
> > +     if (--intc->event_channel[hwirq].ref_count == 0) {
> > +             reg_idx = hwirq / 32;
> > +             val = BIT(hwirq  % 32);
> > +
> > +             /* disable system events */
> > +             pruss_intc_write_reg(intc, PRU_INTC_ECR(reg_idx), val);
> > +             /* clear any pending status */
> > +             pruss_intc_write_reg(intc, PRU_INTC_SECR(reg_idx), val);
> > +
> > +             /* clear the map using reset value 0 */
> > +             pruss_intc_update_cmr(intc, hwirq, 0);
> > +     }
> > +
> > +     dev_dbg(intc->dev, "unmapped system_event = %lu channel = %d host = %d\n",
> > +             hwirq, ch, host);
> > +
> > +     mutex_unlock(&intc->lock);
> > +}
> > +
> >   static void pruss_intc_init(struct pruss_intc *intc)
> >   {
> >       const struct pruss_intc_match_data *soc_config = intc->soc_config;
> > @@ -212,10 +410,67 @@ static struct irq_chip pruss_irqchip = {
> >       .irq_set_irqchip_state = pruss_intc_irq_set_irqchip_state,
> >   };
> >
> > +static int
> > +pruss_intc_irq_domain_xlate(struct irq_domain *d, struct device_node *node,
> > +                         const u32 *intspec, unsigned int intsize,
> > +                         unsigned long *out_hwirq, unsigned int *out_type)
> > +{
> > +     struct pruss_intc *intc = d->host_data;
> > +     struct device *dev = intc->dev;
> > +     int sys_event, channel, host;
> > +
> > +     if (intsize == 1) {
> > +             /*
> > +              * In case of short version (intsize == 1) verify if sysevent
> > +              * already mapped to channel/host irq if not return error
> > +              */
> > +             sys_event = intspec[0];
> > +             if (intc->event_channel[sys_event].ref_count)
> > +                     goto already_mapped;
> > +             else
> > +                     return -EINVAL;
>
> Perhaps revise this to check the error condition first and skipping the
> else, similar to the change you have done in Patch 3.

Ok. I will do that for v4.

>
> > +     }
> > +
> > +     if (intsize < 3)
> > +             return -EINVAL;
> > +
> > +     sys_event = intspec[0];
> > +     if (sys_event < 0 || sys_event >= intc->soc_config->num_system_events) {
> > +             dev_err(dev, "not valid event number\n");
>
> Would be useful to print the invalid event numbers here, and each of
> channel and host below.

Yes, good point.

Thank you,
Grzegorz

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Marc Zyngier]

On 2020-07-05 14:26, Grzegorz Jaszczyk wrote:
> On Sat, 4 Jul 2020 at 11:39, Marc Zyngier <maz@kernel.org> wrote:
>> 
>> On 2020-07-03 15:28, Grzegorz Jaszczyk wrote:

[...]

>> It still begs the question: if the HW can support both edge and level
>> triggered interrupts, why isn't the driver supporting this diversity?
>> I appreciate that your HW may only have level interrupts so far, but
>> what guarantees that this will forever be true? It would imply a 
>> change
>> in the DT binding, which isn't desirable.
> 
> Ok, I've got your point. I will try to come up with something later
> on. Probably extending interrupt-cells by one and passing interrupt
> type will be enough for now. Extending this driver to actually support
> it can be handled later if needed. Hope it works for you.

Writing a set_type callback to deal with this should be pretty easy.
Don't delay doing the right thing.

[...]

>> >> > +             hwirq = hipir & GENMASK(9, 0);
>> >> > +             virq = irq_linear_revmap(intc->domain, hwirq);
>> >>
>> >> And this is where I worry. You seems to have a single irqdomain
>> >> for all the muxes. Are you guaranteed that you will have no
>> >> overlap between muxes? And please use irq_find_mapping(), as
>> >> I have top-secret plans to kill irq_linear_revmap().
>> >
>> > Regarding irq_find_mapping - sure.
>> >
>> > Regarding irqdomains:
>> > It is a single irqdomain since the hwirq (system event) can be mapped
>> > to different irq_host (muxes). Patch #6
>> > https://lkml.org/lkml/2020/7/2/616 implements and describes how input
>> > events can be mapped to some output host interrupts through 2 levels
>> > of many-to-one mapping i.e. events to channel mapping and channels to
>> > host interrupts. Mentioned implementation ensures that specific system
>> > event (hwirq) can be mapped through PRUSS specific channel into a
>> > single host interrupt.
>> 
>> Patch #6 is a nightmare of its own, and I haven't fully groked it yet.
>> Also, this driver seems to totally ignore the 2-level routing. Where
>> is it set up? map/unmap in this driver do exactly *nothing*, so
>> something somewhere must set it up.
> 
> The map/unmap is updated in patch #6 and it deals with those 2-level
> routing setup. Map is responsible for programming the Channel Map
> Registers (CMRx) and Host-Interrupt Map Registers (HMRx) basing on
> provided configuration from the one parsed in the xlate function.
> Unmap undo whatever was done on the map. More details can be found in
> patch #6.
> 
> Maybe it would be better to squash patch #6 with this one so it would
> be less confusing. What is your advice?

So am I right in understanding that without patch #6, this driver does
exactly nothing? If so, it has been a waste of review time.

Please split patch #6 so that this driver does something useful
for Linux, without any of the PRU interrupt routing stuff. I want
to see a Linux-only driver that works and doesn't rely on any other
exotic feature.

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Grzegorz Jaszczyk]

On Sun, 5 Jul 2020 at 22:45, Marc Zyngier <maz@kernel.org> wrote:
>
> On 2020-07-05 14:26, Grzegorz Jaszczyk wrote:
> > On Sat, 4 Jul 2020 at 11:39, Marc Zyngier <maz@kernel.org> wrote:
> >>
> >> On 2020-07-03 15:28, Grzegorz Jaszczyk wrote:
>
> [...]
>
> >> It still begs the question: if the HW can support both edge and level
> >> triggered interrupts, why isn't the driver supporting this diversity?
> >> I appreciate that your HW may only have level interrupts so far, but
> >> what guarantees that this will forever be true? It would imply a
> >> change
> >> in the DT binding, which isn't desirable.
> >
> > Ok, I've got your point. I will try to come up with something later
> > on. Probably extending interrupt-cells by one and passing interrupt
> > type will be enough for now. Extending this driver to actually support
> > it can be handled later if needed. Hope it works for you.
>
> Writing a set_type callback to deal with this should be pretty easy.
> Don't delay doing the right thing.

Ok.

>
> [...]
>
> >> >> > +             hwirq = hipir & GENMASK(9, 0);
> >> >> > +             virq = irq_linear_revmap(intc->domain, hwirq);
> >> >>
> >> >> And this is where I worry. You seems to have a single irqdomain
> >> >> for all the muxes. Are you guaranteed that you will have no
> >> >> overlap between muxes? And please use irq_find_mapping(), as
> >> >> I have top-secret plans to kill irq_linear_revmap().
> >> >
> >> > Regarding irq_find_mapping - sure.
> >> >
> >> > Regarding irqdomains:
> >> > It is a single irqdomain since the hwirq (system event) can be mapped
> >> > to different irq_host (muxes). Patch #6
> >> > https://lkml.org/lkml/2020/7/2/616 implements and describes how input
> >> > events can be mapped to some output host interrupts through 2 levels
> >> > of many-to-one mapping i.e. events to channel mapping and channels to
> >> > host interrupts. Mentioned implementation ensures that specific system
> >> > event (hwirq) can be mapped through PRUSS specific channel into a
> >> > single host interrupt.
> >>
> >> Patch #6 is a nightmare of its own, and I haven't fully groked it yet.
> >> Also, this driver seems to totally ignore the 2-level routing. Where
> >> is it set up? map/unmap in this driver do exactly *nothing*, so
> >> something somewhere must set it up.
> >
> > The map/unmap is updated in patch #6 and it deals with those 2-level
> > routing setup. Map is responsible for programming the Channel Map
> > Registers (CMRx) and Host-Interrupt Map Registers (HMRx) basing on
> > provided configuration from the one parsed in the xlate function.
> > Unmap undo whatever was done on the map. More details can be found in
> > patch #6.
> >
> > Maybe it would be better to squash patch #6 with this one so it would
> > be less confusing. What is your advice?
>
> So am I right in understanding that without patch #6, this driver does
> exactly nothing? If so, it has been a waste of review time.
>
> Please split patch #6 so that this driver does something useful
> for Linux, without any of the PRU interrupt routing stuff. I want
> to see a Linux-only driver that works and doesn't rely on any other
> exotic feature.
>

Patch #6 provides PRU specific 2-level routing setup. This step is
required and it is part of the entire patch-set. Theoretically routing
setup could be done by other platform driver (not irq one) or e.g. by
PRU firmware. In such case this driver would be functional without
patch #6 but I do not think it would be proper. All this routing setup
is done via PRUSS INTC unit and uses PRUSS INTC register set,
therefore delegating it to another driver doesn't seem to be the best
option. Furthermore delegating this step to PRU firmware is also
problematic. First of all the PRU core tiny Instruction RAM space
makes it difficult to fit it together with the code that is required
for running a PRU specific application. Another issue that I see is
splitting management of the PRUSS INTC unit to Linux (main CPU) and
PRU firmware (PRU core).

I am also not sure if splitting patch #6 makes sense. Mentioned patch
allows to perform the entire 2-level routing setup. There is no
distinction between routing setup for main CPU and PRU core, both use
the same logic.The only difference between setting up the routing for
main CPU (Linux) and PRU core is choosing different, so called, "host
interrupt" in final level mapping.

Discussion about previous approach of handling this 2-level routing
setup can be found in v2 of this patch-set
(https://patchwork.kernel.org/patch/11069751/) - mentioned approach
wasn't good and was dropped but problem description made by Suman may
be useful.

I am open to any suggestion if there is a better way of handling
2-level routing. I will also appreciate if you could elaborate about
issues that you see with patch #6.

Best regards,
Grzegorz

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Marc Zyngier]

On 2020-07-08 08:04, Grzegorz Jaszczyk wrote:
> On Sun, 5 Jul 2020 at 22:45, Marc Zyngier <maz@kernel.org> wrote:
>> 
>> On 2020-07-05 14:26, Grzegorz Jaszczyk wrote:
>> > On Sat, 4 Jul 2020 at 11:39, Marc Zyngier <maz@kernel.org> wrote:
>> >>
>> >> On 2020-07-03 15:28, Grzegorz Jaszczyk wrote:
>> 
>> [...]
>> 
>> >> It still begs the question: if the HW can support both edge and level
>> >> triggered interrupts, why isn't the driver supporting this diversity?
>> >> I appreciate that your HW may only have level interrupts so far, but
>> >> what guarantees that this will forever be true? It would imply a
>> >> change
>> >> in the DT binding, which isn't desirable.
>> >
>> > Ok, I've got your point. I will try to come up with something later
>> > on. Probably extending interrupt-cells by one and passing interrupt
>> > type will be enough for now. Extending this driver to actually support
>> > it can be handled later if needed. Hope it works for you.
>> 
>> Writing a set_type callback to deal with this should be pretty easy.
>> Don't delay doing the right thing.
> 
> Ok.
> 
>> 
>> [...]
>> 
>> >> >> > +             hwirq = hipir & GENMASK(9, 0);
>> >> >> > +             virq = irq_linear_revmap(intc->domain, hwirq);
>> >> >>
>> >> >> And this is where I worry. You seems to have a single irqdomain
>> >> >> for all the muxes. Are you guaranteed that you will have no
>> >> >> overlap between muxes? And please use irq_find_mapping(), as
>> >> >> I have top-secret plans to kill irq_linear_revmap().
>> >> >
>> >> > Regarding irq_find_mapping - sure.
>> >> >
>> >> > Regarding irqdomains:
>> >> > It is a single irqdomain since the hwirq (system event) can be mapped
>> >> > to different irq_host (muxes). Patch #6
>> >> > https://lkml.org/lkml/2020/7/2/616 implements and describes how input
>> >> > events can be mapped to some output host interrupts through 2 levels
>> >> > of many-to-one mapping i.e. events to channel mapping and channels to
>> >> > host interrupts. Mentioned implementation ensures that specific system
>> >> > event (hwirq) can be mapped through PRUSS specific channel into a
>> >> > single host interrupt.
>> >>
>> >> Patch #6 is a nightmare of its own, and I haven't fully groked it yet.
>> >> Also, this driver seems to totally ignore the 2-level routing. Where
>> >> is it set up? map/unmap in this driver do exactly *nothing*, so
>> >> something somewhere must set it up.
>> >
>> > The map/unmap is updated in patch #6 and it deals with those 2-level
>> > routing setup. Map is responsible for programming the Channel Map
>> > Registers (CMRx) and Host-Interrupt Map Registers (HMRx) basing on
>> > provided configuration from the one parsed in the xlate function.
>> > Unmap undo whatever was done on the map. More details can be found in
>> > patch #6.
>> >
>> > Maybe it would be better to squash patch #6 with this one so it would
>> > be less confusing. What is your advice?
>> 
>> So am I right in understanding that without patch #6, this driver does
>> exactly nothing? If so, it has been a waste of review time.
>> 
>> Please split patch #6 so that this driver does something useful
>> for Linux, without any of the PRU interrupt routing stuff. I want
>> to see a Linux-only driver that works and doesn't rely on any other
>> exotic feature.
>> 
> 
> Patch #6 provides PRU specific 2-level routing setup. This step is
> required and it is part of the entire patch-set. Theoretically routing
> setup could be done by other platform driver (not irq one) or e.g. by
> PRU firmware. In such case this driver would be functional without
> patch #6 but I do not think it would be proper.

Then this whole driver is non-functional until the last patch that
comes with the PRU-specific "value-add".

[...]

> I am open to any suggestion if there is a better way of handling
> 2-level routing. I will also appreciate if you could elaborate about
> issues that you see with patch #6.

The two level routing has to be part of this (or another) irqchip
driver (specially given that it appears to me like another set of
crossbar). There should only be a *single* binding for all interrupts,
including those targeting the PRU (you seem to have two).

And the non-CPU interrupt code has to be in its own patch, because
it is pretty borderline anyway (I'm still not completely convinced
this is Linux's job).

         N,
-- 
Jazz is not dead. It just smells funny...

Re: [PATCHv3 3/6] irqchip/irq-pruss-intc: Add support for shared and invalid interrupts

[Suman Anna]

Hi Marc,

On 7/2/20 12:44 PM, Marc Zyngier wrote:
> On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
>> From: Suman Anna <s-anna@ti.com>
>>
>> The PRUSS INTC has a fixed number of output interrupt lines that are
>> connected to a number of processors or other PRUSS instances or other
>> devices (like DMA) on the SoC. The output interrupt lines 2 through 9
>> are usually connected to the main Arm host processor and are referred
>> to as host interrupts 0 through 7 from ARM/MPU perspective.
>>
>> All of these 8 host interrupts are not always exclusively connected
>> to the Arm interrupt controller. Some SoCs have some interrupt lines
>> not connected to the Arm interrupt controller at all, while a few others
>> have the interrupt lines connected to multiple processors in which they
>> need to be partitioned as per SoC integration needs. For example, AM437x
>> and 66AK2G SoCs have 2 PRUSS instances each and have the host interrupt 5
>> connected to the other PRUSS, while AM335x has host interrupt 0 shared
>> between MPU and TSC_ADC and host interrupts 6 & 7 shared between MPU and
>> a DMA controller.
>>
>> Add support to the PRUSS INTC driver to allow both these shared and
>> invalid interrupts by not returning a failure if any of these interrupts
>> are skipped from the corresponding INTC DT node.
> 
> That's not exactly "adding support", is it? It really is "ignore these
> interrupts because they are useless from the main CPU's perspective",
> right?

Correct. We can rephrase this to something like
"Add logic to the PRUSS INTC driver to ignore.."

> 
>>
>> Signed-off-by: Suman Anna <s-anna@ti.com>
>> Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
>> ---
>> v2->v3:
>> - Extra checks for (intc->irqs[i]) in error/remove path was moved from
>>   "irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS
>>   interrupts" to this patch
>> v1->v2:
>> - https://patchwork.kernel.org/patch/11069757/
>> ---
>>  drivers/irqchip/irq-pruss-intc.c | 73 
>> +++++++++++++++++++++++++++++++++++++---
>>  1 file changed, 68 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/irqchip/irq-pruss-intc.c 
>> b/drivers/irqchip/irq-pruss-intc.c
>> index fb3dda3..49c936f 100644
>> --- a/drivers/irqchip/irq-pruss-intc.c
>> +++ b/drivers/irqchip/irq-pruss-intc.c
>> @@ -65,11 +65,15 @@
>>   * @irqs: kernel irq numbers corresponding to PRUSS host interrupts
>>   * @base: base virtual address of INTC register space
>>   * @domain: irq domain for this interrupt controller
>> + * @shared_intr: bit-map denoting if the MPU host interrupt is shared
> 
> nit: bitmap

ok

> 
>> + * @invalid_intr: bit-map denoting if host interrupt is not connected 
>> to MPU
>>   */
>>  struct pruss_intc {
>>      unsigned int irqs[MAX_NUM_HOST_IRQS];
>>      void __iomem *base;
>>      struct irq_domain *domain;
>> +    u16 shared_intr;
>> +    u16 invalid_intr;
> 
> Please represent bitmaps as an unsigned long.

ok. We have atmost 8 interrupts coming in, but agree on the change since 
we are using the BIT() macro below.

> 
>>  };
>>
>>  static inline u32 pruss_intc_read_reg(struct pruss_intc *intc,
>> unsigned int reg)
>> @@ -222,7 +226,8 @@ static int pruss_intc_probe(struct platform_device 
>> *pdev)
>>          "host_intr4", "host_intr5", "host_intr6", "host_intr7", };
>>      struct device *dev = &pdev->dev;
>>      struct pruss_intc *intc;
>> -    int i, irq;
>> +    int i, irq, count;
>> +    u8 temp_intr[MAX_NUM_HOST_IRQS] = { 0 };
>>
>>      intc = devm_kzalloc(dev, sizeof(*intc), GFP_KERNEL);
>>      if (!intc)
>> @@ -235,6 +240,52 @@ static int pruss_intc_probe(struct 
>> platform_device *pdev)
>>          return PTR_ERR(intc->base);
>>      }
>>
>> +    count = of_property_read_variable_u8_array(dev->of_node,
>> +                           "ti,irqs-reserved",
>> +                           temp_intr, 0,
>> +                           MAX_NUM_HOST_IRQS);
>> +    /*
>> +     * The irqs-reserved is used only for some SoC's therefore not 
>> having
>> +     * this property is still valid
>> +     */
>> +    if (count == -EINVAL)
>> +        count = 0;
>> +    if (count < 0)
>> +        return count;
>> +
>> +    for (i = 0; i < count; i++) {
>> +        if (temp_intr[i] >= MAX_NUM_HOST_IRQS) {
>> +            dev_warn(dev, "ignoring invalid reserved irq %d\n",
>> +                 temp_intr[i]);
>> +            continue;
>> +        }
>> +
>> +        intc->invalid_intr |= BIT(temp_intr[i]);
>> +    }
>> +
>> +    count = of_property_read_variable_u8_array(dev->of_node,
>> +                           "ti,irqs-shared",
>> +                           temp_intr, 0,
>> +                           MAX_NUM_HOST_IRQS);
>> +    /*
>> +     * The irqs-shared is used only for some SoC's therefore not having
>> +     * this property is still valid
>> +     */
>> +    if (count == -EINVAL)
>> +        count = 0;
>> +    if (count < 0)
>> +        return count;
>> +
>> +    for (i = 0; i < count; i++) {
>> +        if (temp_intr[i] >= MAX_NUM_HOST_IRQS) {
>> +            dev_warn(dev, "ignoring invalid shared irq %d\n",
>> +                 temp_intr[i]);
>> +            continue;
>> +        }
>> +
>> +        intc->shared_intr |= BIT(temp_intr[i]);
>> +    }
>> +
> 
> You probably want to move this in a separate function, since you populate a
> common structure.
> 
>>      pruss_intc_init(intc);
>>
>>      /* always 64 events */
>> @@ -244,8 +295,14 @@ static int pruss_intc_probe(struct 
>> platform_device *pdev)
>>          return -ENOMEM;
>>
>>      for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
>> +        if (intc->invalid_intr & BIT(i))
>> +            continue;
>> +
>>          irq = platform_get_irq_byname(pdev, irq_names[i]);
>>          if (irq <= 0) {
>> +            if (intc->shared_intr & BIT(i))
>> +                continue;
> 
> I don't really understand why you are treating these "shared" interrupts
> differently from the invalid ones. In all cases, they shouldn't be used.

The behavior is the same in how we handle it, but the difference is that 
an "invalid" one is never even connected to the ARM interrupt 
controller, while the "shared" one is a choice. So, unless this 
interrupt is being used/handled by a different processor/entity, you 
would not see this skipped from the dts node.

regards
Suman


> 
>> +
>>              dev_err(dev, "platform_get_irq_byname failed for %s : %d\n",
>>                  irq_names[i], irq);
>>              goto fail_irq;
>> @@ -259,8 +316,11 @@ static int pruss_intc_probe(struct 
>> platform_device *pdev)
>>      return 0;
>>
>>  fail_irq:
>> -    while (--i >= 0)
>> -        irq_set_chained_handler_and_data(intc->irqs[i], NULL, NULL);
>> +    while (--i >= 0) {
>> +        if (intc->irqs[i])
>> +            irq_set_chained_handler_and_data(intc->irqs[i], NULL,
>> +                             NULL);
>> +    }
>>
>>      irq_domain_remove(intc->domain);
>>
>> @@ -273,8 +333,11 @@ static int pruss_intc_remove(struct 
>> platform_device *pdev)
>>      unsigned int hwirq;
>>      int i;
>>
>> -    for (i = 0; i < MAX_NUM_HOST_IRQS; i++)
>> -        irq_set_chained_handler_and_data(intc->irqs[i], NULL, NULL);
>> +    for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
>> +        if (intc->irqs[i])
>> +            irq_set_chained_handler_and_data(intc->irqs[i], NULL,
>> +                             NULL);
>> +    }
>>
>>      for (hwirq = 0; hwirq < MAX_PRU_SYS_EVENTS; hwirq++)
>>          irq_dispose_mapping(irq_find_mapping(intc->domain, hwirq));
> 
> Thanks,
> 
>          M.

Re: [PATCHv3 4/6] irqchip/irq-pruss-intc: Implement irq_{get,set}_irqchip_state ops

[Suman Anna]

On 7/3/20 12:04 PM, Grzegorz Jaszczyk wrote:
> On Thu, 2 Jul 2020 at 19:54, Marc Zyngier <maz@kernel.org> wrote:
>>
>> On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
>>> From: David Lechner <david@lechnology.com>
>>>
>>> This implements the irq_get_irqchip_state and irq_set_irqchip_state
>>> callbacks for the TI PRUSS INTC driver. The set callback can be used
>>> by drivers to "kick" a PRU by enabling a PRU system event.
>>
>> "enabling"? That'd be unmasking an interrupt, which isn't what this
>> does. "injecting", maybe?
> 
> Yes "injecting" is much better.
> 
>>
>>>
>>> Example:
>>>       irq_set_irqchip_state(irq, IRQCHIP_STATE_PENDING, true);
>>
>> Nice example.
>>
>> What this example does explain is how you are actually going to kick
>> a PRU via this interface. For that to happen, you'd have to have on
>> the Linux side an interrupt that is actually routed to a PRU.
> 
> Correct.
> 
>> And from what I have understood of the previous patches, this can't
>> be the case. What didi I miss?
> 
> The hwirq's handled by this driver are so called system events in
> PRUSS nomenclature. This driver is responsible for the entire mapping
> of those system events to PRUSS specific channels which are next
> mapped to host_irq (patch #6 https://lkml.org/lkml/2020/7/2/612).
> There are 8 host_irqs that are routed to the main cpu (running Linux)
> and they are called host_intr0..host_intr7 (were seen in previous
> patches of this series). But there are other "host_interrupts" that
> are routed not to the main CPU but to PRU cores and this driver is
> responsible for creating proper mapping (system
> event/channel/host_irq) for them, and allowing to kick PRU via the
> introduced interface.
> 
> It is worth noting that the PRUSS is quite flexible and allows various
> things e.g.:
> - map any of 160/64 internal system events to any of the 20/10 channels
> - map any of the 20/10 channels to any of the 20/10 host interrupts.
> 
> So e.g. it is possible to map e.g. system event 17 to the main CPU
> (through e.g. channel 1 which is the next map to e.g. host_intr0). Or
> (exclusively) map the same system event 17 to PRU core (through e.g.
> channel 1 which is the next map to PRU0).

Grzegorz has explained in detail, the short summary is we trigger a 
system event, and the mapping for that event ensures the interrupt is 
routed at the desired processor.

regards
Suman

> 
>>
>>>
>>> Signed-off-by: David Lechner <david@lechnology.com>
>>> Signed-off-by: Suman Anna <s-anna@ti.com>
>>> Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
>>> Reviewed-by: Lee Jones <lee.jones@linaro.org>
>>> ---
>>> v2->v3:
>>> - Get rid of unnecessary pruss_intc_check_write() and use
>>>    pruss_intc_write_reg directly.
>>> v1->v2:
>>> - https://patchwork.kernel.org/patch/11069769/
>>> ---
>>>   drivers/irqchip/irq-pruss-intc.c | 43
>>> ++++++++++++++++++++++++++++++++++++++--
>>>   1 file changed, 41 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/irqchip/irq-pruss-intc.c
>>> b/drivers/irqchip/irq-pruss-intc.c
>>> index 49c936f..19b3d38 100644
>>> --- a/drivers/irqchip/irq-pruss-intc.c
>>> +++ b/drivers/irqchip/irq-pruss-intc.c
>>> @@ -7,6 +7,7 @@
>>>    *   Suman Anna <s-anna@ti.com>
>>>    */
>>>
>>> +#include <linux/interrupt.h>
>>>   #include <linux/irq.h>
>>>   #include <linux/irqchip/chained_irq.h>
>>>   #include <linux/irqdomain.h>
>>> @@ -39,8 +40,7 @@
>>>   #define PRU_INTC_HIEISR              0x0034
>>>   #define PRU_INTC_HIDISR              0x0038
>>>   #define PRU_INTC_GPIR                0x0080
>>> -#define PRU_INTC_SRSR0               0x0200
>>> -#define PRU_INTC_SRSR1               0x0204
>>> +#define PRU_INTC_SRSR(x)     (0x0200 + (x) * 4)
>>>   #define PRU_INTC_SECR0               0x0280
>>>   #define PRU_INTC_SECR1               0x0284
>>>   #define PRU_INTC_ESR0                0x0300
>>> @@ -145,6 +145,43 @@ static void pruss_intc_irq_relres(struct irq_data
>>> *data)
>>>        module_put(THIS_MODULE);
>>>   }
>>>
>>> +static int pruss_intc_irq_get_irqchip_state(struct irq_data *data,
>>> +                                         enum irqchip_irq_state which,
>>> +                                         bool *state)
>>> +{
>>> +     struct pruss_intc *intc = irq_data_get_irq_chip_data(data);
>>> +     u32 reg, mask, srsr;
>>> +
>>> +     if (which != IRQCHIP_STATE_PENDING)
>>> +             return -EINVAL;
>>> +
>>> +     reg = PRU_INTC_SRSR(data->hwirq / 32);
>>
>> I assume the register file scales as more interrupts are added in the
>> subsequent patch?
>>
> Yes, after I will move part of the next patch to patch #2 as you
> suggested it will stop being confusing.
> 
> Thank you,
> Grzegorz
> 

Re: [PATCHv3 5/6] irqchip/irq-pruss-intc: Add support for ICSSG INTC on K3 SoCs

[Suman Anna]

Hi Marc,

On 7/3/20 12:05 PM, Grzegorz Jaszczyk wrote:
> On Thu, 2 Jul 2020 at 19:59, Marc Zyngier <maz@kernel.org> wrote:
>>
>> On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
>>> From: Suman Anna <s-anna@ti.com>
>>>
>>> The K3 AM65x and J721E SoCs have the next generation of the PRU-ICSS
>>> IP,
>>> commonly called ICSSG. The PRUSS INTC present within the ICSSG supports
>>> more System Events (160 vs 64), more Interrupt Channels and Host
>>> Interrupts
>>> (20 vs 10) compared to the previous generation PRUSS INTC instances.
>>> The
>>> first 2 and the last 10 of these host interrupt lines are used by the
>>> PRU and other auxiliary cores and sub-modules within the ICSSG, with 8
>>> host interrupts connected to MPU. The host interrupts 5, 6, 7 are also
>>> connected to the other ICSSG instances within the SoC and can be
>>> partitioned as per system integration through the board dts files.
>>>
>>> Enhance the PRUSS INTC driver to add support for this ICSSG INTC
>>> instance. This support is added using specific compatible and match
>>> data and updating the code to use this data instead of the current
>>> hard-coded macros. The INTC config structure is updated to use the
>>> higher events and channels on all SoCs, while limiting the actual
>>> processing to only the relevant number of events/channels/interrupts.
>>>
>>> Signed-off-by: Suman Anna <s-anna@ti.com>
>>> Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
>>> ---
>>> v2->v3:
>>> - Change patch order: use it directly after "irqchip/irq-pruss-intc:
>>>    Implement irq_{get,set}_irqchip_state ops" and before new
>>>    "irqchip/irq-pruss-intc: Add event mapping support" in order to
>>> reduce
>>>    diff.
>>
>> The diff would be even smaller if you introduced a variable number of
>> inputs the first place, i.e. in patch #2. Most if this patch just
>> retrofits it. Please squash these changes into that initial patch,
>> and only add the platform stuff here.

Yeah, all the variables were introduced in this patch based on patch 
history. Until this new IP came, we haven't had a need to use variables 
in the original driver code. That's also the reason for this being a 
separate patch rather than squashed into the original patch.

regards
Suman

>
> Sure I will do that.
> 
> Thank you,
> Grzegorz
> 

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Suman Anna]

Hi Marc,

On 7/8/20 5:47 AM, Marc Zyngier wrote:
> On 2020-07-08 08:04, Grzegorz Jaszczyk wrote:
>> On Sun, 5 Jul 2020 at 22:45, Marc Zyngier <maz@kernel.org> wrote:
>>>
>>> On 2020-07-05 14:26, Grzegorz Jaszczyk wrote:
>>> > On Sat, 4 Jul 2020 at 11:39, Marc Zyngier <maz@kernel.org> wrote:
>>> >>
>>> >> On 2020-07-03 15:28, Grzegorz Jaszczyk wrote:
>>>
>>> [...]
>>>
>>> >> It still begs the question: if the HW can support both edge and level
>>> >> triggered interrupts, why isn't the driver supporting this diversity?
>>> >> I appreciate that your HW may only have level interrupts so far, but
>>> >> what guarantees that this will forever be true? It would imply a
>>> >> change
>>> >> in the DT binding, which isn't desirable.
>>> >
>>> > Ok, I've got your point. I will try to come up with something later
>>> > on. Probably extending interrupt-cells by one and passing interrupt
>>> > type will be enough for now. Extending this driver to actually support
>>> > it can be handled later if needed. Hope it works for you.
>>>
>>> Writing a set_type callback to deal with this should be pretty easy.
>>> Don't delay doing the right thing.
>>
>> Ok.

Sorry for the typo in my comment causing this confusion.

The h/w actually doesn't support the edge-interrupts. Likewise, the 
polarity is always high. The individual register bit descriptions 
mention what the bit values 0 and 1 mean, but there is additional 
description in the TRMs on all the SoCs that says
"always write 1 to the bits of this register" for PRUSS_INTC_SIPR(x) and
"always write 0 to the bits of this register" for PRUSS_INTC_SITR(x).
FWIW, these are also the reset values.

Eg: AM335x TRM - https://www.ti.com/lit/pdf/spruh73
Please see Section 4.4.2.5 and the register descriptions in 4.5.3.49, 
4.5.3.51. Please also see Section 4.4.2.3 that explains the PRUSS INTC 
methodology.

>>
>>>
>>> [...]
>>>
>>> >> >> > +             hwirq = hipir & GENMASK(9, 0);
>>> >> >> > +             virq = irq_linear_revmap(intc->domain, hwirq);
>>> >> >>
>>> >> >> And this is where I worry. You seems to have a single irqdomain
>>> >> >> for all the muxes. Are you guaranteed that you will have no
>>> >> >> overlap between muxes? And please use irq_find_mapping(), as
>>> >> >> I have top-secret plans to kill irq_linear_revmap().
>>> >> >
>>> >> > Regarding irq_find_mapping - sure.
>>> >> >
>>> >> > Regarding irqdomains:
>>> >> > It is a single irqdomain since the hwirq (system event) can be 
>>> mapped
>>> >> > to different irq_host (muxes). Patch #6
>>> >> > https://lkml.org/lkml/2020/7/2/616 implements and describes how 
>>> input
>>> >> > events can be mapped to some output host interrupts through 2 
>>> levels
>>> >> > of many-to-one mapping i.e. events to channel mapping and 
>>> channels to
>>> >> > host interrupts. Mentioned implementation ensures that specific 
>>> system
>>> >> > event (hwirq) can be mapped through PRUSS specific channel into a
>>> >> > single host interrupt.
>>> >>
>>> >> Patch #6 is a nightmare of its own, and I haven't fully groked it 
>>> yet.
>>> >> Also, this driver seems to totally ignore the 2-level routing. Where
>>> >> is it set up? map/unmap in this driver do exactly *nothing*, so
>>> >> something somewhere must set it up.
>>> >
>>> > The map/unmap is updated in patch #6 and it deals with those 2-level
>>> > routing setup. Map is responsible for programming the Channel Map
>>> > Registers (CMRx) and Host-Interrupt Map Registers (HMRx) basing on
>>> > provided configuration from the one parsed in the xlate function.
>>> > Unmap undo whatever was done on the map. More details can be found in
>>> > patch #6.
>>> >
>>> > Maybe it would be better to squash patch #6 with this one so it would
>>> > be less confusing. What is your advice?
>>>
>>> So am I right in understanding that without patch #6, this driver does
>>> exactly nothing? If so, it has been a waste of review time.
>>>
>>> Please split patch #6 so that this driver does something useful
>>> for Linux, without any of the PRU interrupt routing stuff. I want
>>> to see a Linux-only driver that works and doesn't rely on any other
>>> exotic feature.
>>>
>>
>> Patch #6 provides PRU specific 2-level routing setup. This step is
>> required and it is part of the entire patch-set. Theoretically routing
>> setup could be done by other platform driver (not irq one) or e.g. by
>> PRU firmware. In such case this driver would be functional without
>> patch #6 but I do not think it would be proper.
> 
> Then this whole driver is non-functional until the last patch that
> comes with the PRU-specific "value-add".

It is all moot actually and the interrupts work only when the PRU 
remoteproc/clients have invoked the irq_create_fwspec_mapping()
for all of the desired system events. It does not make much difference 
if it was a separate patch or squashed in, patch #6 is a replacement for 
the previous logic, and since it was complex, it was done in a separate 
patch to better explain the usage (same reason on v1 and v2 as well).

> 
> [...]
> 
>> I am open to any suggestion if there is a better way of handling
>> 2-level routing. I will also appreciate if you could elaborate about
>> issues that you see with patch #6.
> 
> The two level routing has to be part of this (or another) irqchip
> driver (specially given that it appears to me like another set of
> crossbar). There should only be a *single* binding for all interrupts,
> including those targeting the PRU (you seem to have two).
> 

Yeah, there hasn't been a clean way of doing this. Our previous attempt 
was to do this through custom exported functions so that the PRU 
remoteproc driver can set these up correctly, but that was shot down and 
this is the direction we are pointed to.

We do want to leverage the "interrupts" property in the PRU user nodes 
instead of inventing our own paradigm through a non-irqchip driver, and 
at the same time, be able to configure this at the run time only when 
that PRU driver is running, and remove the mappings once that driver is 
removed allowing another PRU application/driver. We treat PRUs as an 
exclusive resource, so everything needs to go along with an appropriate 
client user.

> And the non-CPU interrupt code has to be in its own patch, because
> it is pretty borderline anyway (I'm still not completely convinced
> this is Linux's job).

The logic for non-CPU interrupt code is exactly the same as the CPU 
interrupt code, as they are all setup through the 
irq_create_fwspec_mapping(). The CPU-specific pieces are primarily the 
chained interrupt handling.

We have already argued internally about the last part, but our firmware 
developers literally don't have any IRAM space (we have a lot of 
Industrial protocols working out of 4K/8K memory), and have pushed all 
one-time setup to the OS running (Linux or otherwise) on the main ARM 
core, and INTC is one among the other many such settings. Every word in 
Instruction RAM was crucial for them.

So, we are all ears if there is still an elegant way of doing this. Look 
forward to any suggestions you may have.

And thank you for all your review comments.

regards
Suman

Re: [PATCHv3 1/6] dt-bindings: irqchip: Add PRU-ICSS interrupt controller bindings

[Rob Herring]

On Thu, Jul 02, 2020 at 04:17:54PM +0200, Grzegorz Jaszczyk wrote:
> From: Suman Anna <s-anna@ti.com>
> 
> The Programmable Real-Time Unit and Industrial Communication Subsystem
> (PRU-ICSS or simply PRUSS) contains an interrupt controller (INTC) that
> can handle various system input events and post interrupts back to the
> device-level initiators. The INTC can support upto 64 input events on
> most SoCs with individual control configuration and h/w prioritization.
> These events are mapped onto 10 output interrupt lines through two levels
> of many-to-one mapping support. Different interrupt lines are routed to
> the individual PRU cores or to the host CPU or to other PRUSS instances.
> 
> The K3 AM65x and J721E SoCs have the next generation of the PRU-ICSS IP,
> commonly called ICSSG. The ICSSG interrupt controller on K3 SoCs provide
> a higher number of host interrupts (20 vs 10) and can handle an increased
> number of input events (160 vs 64) from various SoC interrupt sources.
> 
> Add the bindings document for these interrupt controllers on all the
> applicable SoCs. It covers the OMAP architecture SoCs - AM33xx, AM437x
> and AM57xx; the Keystone 2 architecture based 66AK2G SoC; the Davinci
> architecture based OMAPL138 SoCs, and the K3 architecture based AM65x
> and J721E SoCs.
> 
> Signed-off-by: Suman Anna <s-anna@ti.com>
> Signed-off-by: Andrew F. Davis <afd@ti.com>
> Signed-off-by: Roger Quadros <rogerq@ti.com>
> Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> Reviewed-by: Lee Jones <lee.jones@linaro.org>
> ---
> v2->v3:
> - Convert dt-binding to YAML
> v1->v2:
> - https://patchwork.kernel.org/patch/11069767/
> ---
>  .../interrupt-controller/ti,pruss-intc.yaml        | 135 +++++++++++++++++++++
>  1 file changed, 135 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml
> 
> diff --git a/Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml b/Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml
> new file mode 100644
> index 0000000..7fe4b95
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml
> @@ -0,0 +1,135 @@
> +# SPDX-License-Identifier: (GPL-2.0-only or BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/interrupt-controller/ti,pruss-intc.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: TI PRU-ICSS Local Interrupt Controller
> +
> +maintainers:
> +  - Suman Anna <s-anna@ti.com>
> +
> +description: |
> +  Each PRU-ICSS has a single interrupt controller instance that is common
> +  to all the PRU cores. Most interrupt controllers can route 64 input events
> +  which are then mapped to 10 possible output interrupts through two levels
> +  of mapping. The input events can be triggered by either the PRUs and/or
> +  various other PRUSS internal and external peripherals. The first 2 output
> +  interrupts (0, 1) are fed exclusively to the internal PRU cores, with the
> +  remaining 8 (2 through 9) connected to external interrupt controllers
> +  including the MPU and/or other PRUSS instances, DSPs or devices.
> +
> +  The properties "ti,irqs-reserved" and "ti,irqs-shared" are used for denoting
> +  the connection differences on the output interrupts 2 through 9. If neither
> +  of these properties are defined, it implies that all the PRUSS INTC output
> +  interrupts 2 through 9 (host_intr0 through host_intr7) are connected
> +  exclusively to the Arm interrupt controller.
> +
> +  The K3 family of SoCs can handle 160 input events that can be mapped to 20
> +  different possible output interrupts. The additional output interrupts (10
> +  through 19) are connected to new sub-modules within the ICSSG instances.
> +
> +  This interrupt-controller node should be defined as a child node of the
> +  corresponding PRUSS node. The node should be named "interrupt-controller".
> +
> +allOf:
> +  - $ref: /schemas/interrupt-controller.yaml#
> +  - $ref: /schemas/interrupts.yaml#

Drop these. They already get applied.

> +
> +properties:
> +  compatible:
> +    enum:
> +      - ti,pruss-intc
> +      - ti,icssg-intc
> +    description: |
> +      Use "ti,pruss-intc" for OMAP-L13x/AM18x/DA850 SoCs,
> +                              AM335x family of SoCs,
> +                              AM437x family of SoCs,
> +                              AM57xx family of SoCs
> +                              66AK2G family of SoCs

I guess you can look at the parent if there's differences in features 
and bugs.

> +      Use "ti,icssg-intc" for K3 AM65x & J721E family of SoCs
> +
> +  reg:
> +    items:
> +      - description: base address and size for the PRUSS INTC sub-module

Just: maxItems: 1

Don't need genericish descriptions.

> +
> +  interrupts:
> +    minItems: 1
> +    maxItems: 8
> +    description: |
> +      all the interrupts generated towards the main host processor in the SoC.

> +      The format depends on the interrupt specifier for the particular SoC's
> +      Arm parent interrupt controller. A shared interrupt can be skipped if

That's true for all 'interrupts' properties, so not needed here.

> +      the desired destination and usage is by a different processor/device.
> +
> +  interrupt-names:
> +    minItems: 1
> +    maxItems: 8
> +    items:
> +      pattern: host_intr[0-7]

'intr' is redundant.

So we could have?:  host0, host3, host4

> +    description: |
> +      should use one of the above names for each valid host event interrupt
> +      connected to Arm interrupt controller, the name should match the
> +      corresponding host event interrupt number
> +
> +  interrupt-controller: true
> +
> +  "#interrupt-cells":
> +    const: 1
> +    description: |
> +      Client users shall use the PRU System event number (the interrupt source
> +      that the client is interested in) as the value of the interrupts property
> +      in their node
> +
> +  ti,irqs-reserved:
> +    $ref: /schemas/types.yaml#definitions/uint8-array
> +    description: |
> +      an array of 8-bit elements of host interrupts between 0 and 7
> +      (corresponding to PRUSS INTC output interrupts 2 through 9) that are
> +      not connected to the Arm interrupt controller.
> +        Eg: AM437x and 66AK2G SoCs do not have "host_intr5" interrupt connected
> +            to MPU

Seems redundant with interrupt-names.

> +
> +  ti,irqs-shared:
> +    $ref: /schemas/types.yaml#definitions/uint8-array
> +    description: |
> +      an array of 8-bit elements of host interrupts between 0 and 7
> +      (corresponding to PRUSS INTC output interrupts 2 through 9) that are
> +      also connected to other devices or processors in the SoC.
> +         Eg: AM65x and J721E SoCs have "host_intr5", "host_intr6" and
> +             "host_intr7" interrupts connected to MPU, and other ICSSG
> +             instances

Wouldn't a bit mask work for both of these?

> +
> +required:
> + - compatible
> + - reg
> + - interrupts
> + - interrupt-names
> + - interrupt-controller
> + - "#interrupt-cells"
> +
> +additionalProperties: false
> +
> +examples:
> +  - |
> +    /* AM33xx PRU-ICSS */
> +    pruss: pruss@0 {
> +        compatible = "ti,am3356-pruss";
> +        reg = <0x0 0x80000>;
> +        #address-cells = <1>;
> +        #size-cells = <1>;
> +        ranges;
> +
> +        pruss_intc: interrupt-controller@20000 {
> +            compatible = "ti,pruss-intc";
> +            reg = <0x20000 0x2000>;
> +            interrupts = <20 21 22 23 24 25 26 27>;
> +            interrupt-names = "host_intr0", "host_intr1",
> +                              "host_intr2", "host_intr3",
> +                              "host_intr4", "host_intr5",
> +                              "host_intr6", "host_intr7";
> +            interrupt-controller;
> +            #interrupt-cells = <1>;
> +            ti,irqs-shared = /bits/ 8 <0 6 7>;
> +        };
> +    };
> -- 
> 2.7.4
> 

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Grzegorz Jaszczyk]

Hi Marc,

> On 7/8/20 5:47 AM, Marc Zyngier wrote:
> > On 2020-07-08 08:04, Grzegorz Jaszczyk wrote:
> >> On Sun, 5 Jul 2020 at 22:45, Marc Zyngier <maz@kernel.org> wrote:
> >>>
> >>> On 2020-07-05 14:26, Grzegorz Jaszczyk wrote:
> >>> > On Sat, 4 Jul 2020 at 11:39, Marc Zyngier <maz@kernel.org> wrote:
> >>> >>
> >>> >> On 2020-07-03 15:28, Grzegorz Jaszczyk wrote:
> >>>
> >>> [...]
> >>>
> >>> >> It still begs the question: if the HW can support both edge and level
> >>> >> triggered interrupts, why isn't the driver supporting this diversity?
> >>> >> I appreciate that your HW may only have level interrupts so far, but
> >>> >> what guarantees that this will forever be true? It would imply a
> >>> >> change
> >>> >> in the DT binding, which isn't desirable.
> >>> >
> >>> > Ok, I've got your point. I will try to come up with something later
> >>> > on. Probably extending interrupt-cells by one and passing interrupt
> >>> > type will be enough for now. Extending this driver to actually support
> >>> > it can be handled later if needed. Hope it works for you.
> >>>
> >>> Writing a set_type callback to deal with this should be pretty easy.
> >>> Don't delay doing the right thing.
> >>
> >> Ok.
>
> Sorry for the typo in my comment causing this confusion.
>
> The h/w actually doesn't support the edge-interrupts. Likewise, the
> polarity is always high. The individual register bit descriptions
> mention what the bit values 0 and 1 mean, but there is additional
> description in the TRMs on all the SoCs that says
> "always write 1 to the bits of this register" for PRUSS_INTC_SIPR(x) and
> "always write 0 to the bits of this register" for PRUSS_INTC_SITR(x).
> FWIW, these are also the reset values.
>
> Eg: AM335x TRM - https://www.ti.com/lit/pdf/spruh73
> Please see Section 4.4.2.5 and the register descriptions in 4.5.3.49,
> 4.5.3.51. Please also see Section 4.4.2.3 that explains the PRUSS INTC
> methodology.
>
> >>
> >>>
> >>> [...]
> >>>
> >>> >> >> > +             hwirq = hipir & GENMASK(9, 0);
> >>> >> >> > +             virq = irq_linear_revmap(intc->domain, hwirq);
> >>> >> >>
> >>> >> >> And this is where I worry. You seems to have a single irqdomain
> >>> >> >> for all the muxes. Are you guaranteed that you will have no
> >>> >> >> overlap between muxes? And please use irq_find_mapping(), as
> >>> >> >> I have top-secret plans to kill irq_linear_revmap().
> >>> >> >
> >>> >> > Regarding irq_find_mapping - sure.
> >>> >> >
> >>> >> > Regarding irqdomains:
> >>> >> > It is a single irqdomain since the hwirq (system event) can be
> >>> mapped
> >>> >> > to different irq_host (muxes). Patch #6
> >>> >> > https://lkml.org/lkml/2020/7/2/616 implements and describes how
> >>> input
> >>> >> > events can be mapped to some output host interrupts through 2
> >>> levels
> >>> >> > of many-to-one mapping i.e. events to channel mapping and
> >>> channels to
> >>> >> > host interrupts. Mentioned implementation ensures that specific
> >>> system
> >>> >> > event (hwirq) can be mapped through PRUSS specific channel into a
> >>> >> > single host interrupt.
> >>> >>
> >>> >> Patch #6 is a nightmare of its own, and I haven't fully groked it
> >>> yet.
> >>> >> Also, this driver seems to totally ignore the 2-level routing. Where
> >>> >> is it set up? map/unmap in this driver do exactly *nothing*, so
> >>> >> something somewhere must set it up.
> >>> >
> >>> > The map/unmap is updated in patch #6 and it deals with those 2-level
> >>> > routing setup. Map is responsible for programming the Channel Map
> >>> > Registers (CMRx) and Host-Interrupt Map Registers (HMRx) basing on
> >>> > provided configuration from the one parsed in the xlate function.
> >>> > Unmap undo whatever was done on the map. More details can be found in
> >>> > patch #6.
> >>> >
> >>> > Maybe it would be better to squash patch #6 with this one so it would
> >>> > be less confusing. What is your advice?
> >>>
> >>> So am I right in understanding that without patch #6, this driver does
> >>> exactly nothing? If so, it has been a waste of review time.
> >>>
> >>> Please split patch #6 so that this driver does something useful
> >>> for Linux, without any of the PRU interrupt routing stuff. I want
> >>> to see a Linux-only driver that works and doesn't rely on any other
> >>> exotic feature.
> >>>
> >>
> >> Patch #6 provides PRU specific 2-level routing setup. This step is
> >> required and it is part of the entire patch-set. Theoretically routing
> >> setup could be done by other platform driver (not irq one) or e.g. by
> >> PRU firmware. In such case this driver would be functional without
> >> patch #6 but I do not think it would be proper.
> >
> > Then this whole driver is non-functional until the last patch that
> > comes with the PRU-specific "value-add".
>
> It is all moot actually and the interrupts work only when the PRU
> remoteproc/clients have invoked the irq_create_fwspec_mapping()
> for all of the desired system events. It does not make much difference
> if it was a separate patch or squashed in, patch #6 is a replacement for
> the previous logic, and since it was complex, it was done in a separate
> patch to better explain the usage (same reason on v1 and v2 as well).
>
> >
> > [...]
> >
> >> I am open to any suggestion if there is a better way of handling
> >> 2-level routing. I will also appreciate if you could elaborate about
> >> issues that you see with patch #6.
> >
> > The two level routing has to be part of this (or another) irqchip
> > driver (specially given that it appears to me like another set of
> > crossbar). There should only be a *single* binding for all interrupts,
> > including those targeting the PRU (you seem to have two).
> >
>
> Yeah, there hasn't been a clean way of doing this. Our previous attempt
> was to do this through custom exported functions so that the PRU
> remoteproc driver can set these up correctly, but that was shot down and
> this is the direction we are pointed to.
>
> We do want to leverage the "interrupts" property in the PRU user nodes
> instead of inventing our own paradigm through a non-irqchip driver, and
> at the same time, be able to configure this at the run time only when
> that PRU driver is running, and remove the mappings once that driver is
> removed allowing another PRU application/driver. We treat PRUs as an
> exclusive resource, so everything needs to go along with an appropriate
> client user.

I will just add an explanation about interrupt binding. So actually
there is one dt-binding defined in yaml (interrupt-cells = 1). The
reason why you see xlate allowing to proceed with 1 or 3 parameters is
because linux can change the PRU firmware at run-time (thorough linux
remoteproc framework) and different firmware may require different
kinds of interrupt mapping. Therefore during firmware load, the new
mapping is created through irq_create_fwspec_mapping() and in this
case 3 parameters are passed: system event, channel and host irq.
Similarly the mapping is disposed during remoteproc stop by invoking
irq_dispose_mapping. This allows to create new mapping, in the same
way, for next firmware loaded through Linux remote-proc at runtime
(depending on the needs of new remoteproc firmware).

On the other hand dt-bindings defines interrupt-cells = 1, so when the
interrupt is registered the xlate function (proceed with 1 parameter)
checks if this event already has valid mapping - if yes we are fine,
if not we return -EINVAL.

>
> > And the non-CPU interrupt code has to be in its own patch, because
> > it is pretty borderline anyway (I'm still not completely convinced
> > this is Linux's job).
>
> The logic for non-CPU interrupt code is exactly the same as the CPU
> interrupt code, as they are all setup through the
> irq_create_fwspec_mapping(). The CPU-specific pieces are primarily the
> chained interrupt handling.
>
> We have already argued internally about the last part, but our firmware
> developers literally don't have any IRAM space (we have a lot of
> Industrial protocols working out of 4K/8K memory), and have pushed all
> one-time setup to the OS running (Linux or otherwise) on the main ARM
> core, and INTC is one among the other many such settings. Every word in
> Instruction RAM was crucial for them.
>
> So, we are all ears if there is still an elegant way of doing this. Look
> forward to any suggestions you may have.

Yes, the non-CPU logic is exactly the same as the CPU interrupt code
as Suman described. There is no distinction between routing setup for
main CPU and PRU core, both use exactly the same logic, just different
numbers are passed through  irq_create_fwspec_mapping.

Looking forward to your feedback.

Best regards,
Grzegorz

Re: [PATCHv3 1/6] dt-bindings: irqchip: Add PRU-ICSS interrupt controller bindings

[Grzegorz Jaszczyk]

On Mon, 13 Jul 2020 at 23:25, Rob Herring <robh@kernel.org> wrote:
>
> On Thu, Jul 02, 2020 at 04:17:54PM +0200, Grzegorz Jaszczyk wrote:
> > From: Suman Anna <s-anna@ti.com>
> >
> > The Programmable Real-Time Unit and Industrial Communication Subsystem
> > (PRU-ICSS or simply PRUSS) contains an interrupt controller (INTC) that
> > can handle various system input events and post interrupts back to the
> > device-level initiators. The INTC can support upto 64 input events on
> > most SoCs with individual control configuration and h/w prioritization.
> > These events are mapped onto 10 output interrupt lines through two levels
> > of many-to-one mapping support. Different interrupt lines are routed to
> > the individual PRU cores or to the host CPU or to other PRUSS instances.
> >
> > The K3 AM65x and J721E SoCs have the next generation of the PRU-ICSS IP,
> > commonly called ICSSG. The ICSSG interrupt controller on K3 SoCs provide
> > a higher number of host interrupts (20 vs 10) and can handle an increased
> > number of input events (160 vs 64) from various SoC interrupt sources.
> >
> > Add the bindings document for these interrupt controllers on all the
> > applicable SoCs. It covers the OMAP architecture SoCs - AM33xx, AM437x
> > and AM57xx; the Keystone 2 architecture based 66AK2G SoC; the Davinci
> > architecture based OMAPL138 SoCs, and the K3 architecture based AM65x
> > and J721E SoCs.
> >
> > Signed-off-by: Suman Anna <s-anna@ti.com>
> > Signed-off-by: Andrew F. Davis <afd@ti.com>
> > Signed-off-by: Roger Quadros <rogerq@ti.com>
> > Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> > Reviewed-by: Lee Jones <lee.jones@linaro.org>
> > ---
> > v2->v3:
> > - Convert dt-binding to YAML
> > v1->v2:
> > - https://patchwork.kernel.org/patch/11069767/
> > ---
> >  .../interrupt-controller/ti,pruss-intc.yaml        | 135 +++++++++++++++++++++
> >  1 file changed, 135 insertions(+)
> >  create mode 100644 Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml
> >
> > diff --git a/Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml b/Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml
> > new file mode 100644
> > index 0000000..7fe4b95
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/interrupt-controller/ti,pruss-intc.yaml
> > @@ -0,0 +1,135 @@
> > +# SPDX-License-Identifier: (GPL-2.0-only or BSD-2-Clause)
> > +%YAML 1.2
> > +---
> > +$id: http://devicetree.org/schemas/interrupt-controller/ti,pruss-intc.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: TI PRU-ICSS Local Interrupt Controller
> > +
> > +maintainers:
> > +  - Suman Anna <s-anna@ti.com>
> > +
> > +description: |
> > +  Each PRU-ICSS has a single interrupt controller instance that is common
> > +  to all the PRU cores. Most interrupt controllers can route 64 input events
> > +  which are then mapped to 10 possible output interrupts through two levels
> > +  of mapping. The input events can be triggered by either the PRUs and/or
> > +  various other PRUSS internal and external peripherals. The first 2 output
> > +  interrupts (0, 1) are fed exclusively to the internal PRU cores, with the
> > +  remaining 8 (2 through 9) connected to external interrupt controllers
> > +  including the MPU and/or other PRUSS instances, DSPs or devices.
> > +
> > +  The properties "ti,irqs-reserved" and "ti,irqs-shared" are used for denoting
> > +  the connection differences on the output interrupts 2 through 9. If neither
> > +  of these properties are defined, it implies that all the PRUSS INTC output
> > +  interrupts 2 through 9 (host_intr0 through host_intr7) are connected
> > +  exclusively to the Arm interrupt controller.
> > +
> > +  The K3 family of SoCs can handle 160 input events that can be mapped to 20
> > +  different possible output interrupts. The additional output interrupts (10
> > +  through 19) are connected to new sub-modules within the ICSSG instances.
> > +
> > +  This interrupt-controller node should be defined as a child node of the
> > +  corresponding PRUSS node. The node should be named "interrupt-controller".
> > +
> > +allOf:
> > +  - $ref: /schemas/interrupt-controller.yaml#
> > +  - $ref: /schemas/interrupts.yaml#
>
> Drop these. They already get applied.

Ok.

>
> > +
> > +properties:
> > +  compatible:
> > +    enum:
> > +      - ti,pruss-intc
> > +      - ti,icssg-intc
> > +    description: |
> > +      Use "ti,pruss-intc" for OMAP-L13x/AM18x/DA850 SoCs,
> > +                              AM335x family of SoCs,
> > +                              AM437x family of SoCs,
> > +                              AM57xx family of SoCs
> > +                              66AK2G family of SoCs
>
> I guess you can look at the parent if there's differences in features
> and bugs.

Currently we are using of_device_id data in the driver to store the
number of system events and number of host interrupts which are
different for pruss-intc and icssg-intc. I prefer to keep it for
clarity if you don't mind.

>
> > +      Use "ti,icssg-intc" for K3 AM65x & J721E family of SoCs
> > +
> > +  reg:
> > +    items:
> > +      - description: base address and size for the PRUSS INTC sub-module
>
> Just: maxItems: 1
>
> Don't need genericish descriptions.

Ok.

>
> > +
> > +  interrupts:
> > +    minItems: 1
> > +    maxItems: 8
> > +    description: |
> > +      all the interrupts generated towards the main host processor in the SoC.
>
> > +      The format depends on the interrupt specifier for the particular SoC's
> > +      Arm parent interrupt controller. A shared interrupt can be skipped if
>
> That's true for all 'interrupts' properties, so not needed here.

Ok.

>
> > +      the desired destination and usage is by a different processor/device.
> > +
> > +  interrupt-names:
> > +    minItems: 1
> > +    maxItems: 8
> > +    items:
> > +      pattern: host_intr[0-7]
>
> 'intr' is redundant.

Those names are also used in TRM so I prefer to keep it as is. IMO it
will be less confusing.

>
> So we could have?:  host0, host3, host4

Currently most of the SoCs have 8 valid host interrupts wired to the
main CPU. But there are some exceptions e.g. in AM437x the host_intr5
is not wired to the main CPU. Additionally e.g. AM335x has host
interrupt 0 shared between MPU and TSC_ADC and host interrupts 6 & 7
shared between MPU and  a DMA controller. Therefore valid example are
e.g.:
1) host_intr[0-4], host_intr[6-7] (without host_intr5)
2) host_intr[1-5] (without host_intr0 and hostr_intr[6-7].

>
> > +    description: |
> > +      should use one of the above names for each valid host event interrupt
> > +      connected to Arm interrupt controller, the name should match the
> > +      corresponding host event interrupt number
> > +
> > +  interrupt-controller: true
> > +
> > +  "#interrupt-cells":
> > +    const: 1
> > +    description: |
> > +      Client users shall use the PRU System event number (the interrupt source
> > +      that the client is interested in) as the value of the interrupts property
> > +      in their node
> > +
> > +  ti,irqs-reserved:
> > +    $ref: /schemas/types.yaml#definitions/uint8-array
> > +    description: |
> > +      an array of 8-bit elements of host interrupts between 0 and 7
> > +      (corresponding to PRUSS INTC output interrupts 2 through 9) that are
> > +      not connected to the Arm interrupt controller.
> > +        Eg: AM437x and 66AK2G SoCs do not have "host_intr5" interrupt connected
> > +            to MPU
>
> Seems redundant with interrupt-names.

It can be seen as redundant but it describes the hardware more
precisely and emphasizes that some irqs are not available for the main
CPU in some SoC's, even if they are mentioned in several places in
TRM. If you don't mind I will prefer to keep it as is.

>
> > +
> > +  ti,irqs-shared:
> > +    $ref: /schemas/types.yaml#definitions/uint8-array
> > +    description: |
> > +      an array of 8-bit elements of host interrupts between 0 and 7
> > +      (corresponding to PRUSS INTC output interrupts 2 through 9) that are
> > +      also connected to other devices or processors in the SoC.
> > +         Eg: AM65x and J721E SoCs have "host_intr5", "host_intr6" and
> > +             "host_intr7" interrupts connected to MPU, and other ICSSG
> > +             instances
>
> Wouldn't a bit mask work for both of these?

Yes - I will convert it as you suggested.

Thank you for your review.
Grzegorz

Re: [PATCHv3 3/6] irqchip/irq-pruss-intc: Add support for shared and invalid interrupts

[Marc Zyngier]

On Fri, 10 Jul 2020 21:59:17 +0100,
Suman Anna <s-anna@ti.com> wrote:

Hi Suman,

[...]

>
> Hi Marc,
> 
> On 7/2/20 12:44 PM, Marc Zyngier wrote:
> > On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
> >> From: Suman Anna <s-anna@ti.com>
> >> 
> >> The PRUSS INTC has a fixed number of output interrupt lines that are
> >> connected to a number of processors or other PRUSS instances or other
> >> devices (like DMA) on the SoC. The output interrupt lines 2 through 9
> >> are usually connected to the main Arm host processor and are referred
> >> to as host interrupts 0 through 7 from ARM/MPU perspective.
> >> 
> >> All of these 8 host interrupts are not always exclusively connected
> >> to the Arm interrupt controller. Some SoCs have some interrupt lines
> >> not connected to the Arm interrupt controller at all, while a few others
> >> have the interrupt lines connected to multiple processors in which they
> >> need to be partitioned as per SoC integration needs. For example, AM437x
> >> and 66AK2G SoCs have 2 PRUSS instances each and have the host interrupt 5
> >> connected to the other PRUSS, while AM335x has host interrupt 0 shared
> >> between MPU and TSC_ADC and host interrupts 6 & 7 shared between MPU and
> >> a DMA controller.
> >> 
> >> Add support to the PRUSS INTC driver to allow both these shared and
> >> invalid interrupts by not returning a failure if any of these interrupts
> >> are skipped from the corresponding INTC DT node.
> > 
> > That's not exactly "adding support", is it? It really is "ignore these
> > interrupts because they are useless from the main CPU's perspective",
> > right?
> 
> Correct. We can rephrase this to something like
> "Add logic to the PRUSS INTC driver to ignore.."
> 
> > 
> >> 
> >> Signed-off-by: Suman Anna <s-anna@ti.com>
> >> Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
> >> ---
> >> v2->v3:
> >> - Extra checks for (intc->irqs[i]) in error/remove path was moved from
> >>   "irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS
> >>   interrupts" to this patch
> >> v1->v2:
> >> - https://patchwork.kernel.org/patch/11069757/
> >> ---
> >>  drivers/irqchip/irq-pruss-intc.c | 73
> >> +++++++++++++++++++++++++++++++++++++---
> >>  1 file changed, 68 insertions(+), 5 deletions(-)
> >> 
> >> diff --git a/drivers/irqchip/irq-pruss-intc.c
> >> b/drivers/irqchip/irq-pruss-intc.c
> >> index fb3dda3..49c936f 100644
> >> --- a/drivers/irqchip/irq-pruss-intc.c
> >> +++ b/drivers/irqchip/irq-pruss-intc.c
> >> @@ -65,11 +65,15 @@
> >>   * @irqs: kernel irq numbers corresponding to PRUSS host interrupts
> >>   * @base: base virtual address of INTC register space
> >>   * @domain: irq domain for this interrupt controller
> >> + * @shared_intr: bit-map denoting if the MPU host interrupt is shared
> > 
> > nit: bitmap
> 
> ok
> 
> > 
> >> + * @invalid_intr: bit-map denoting if host interrupt is not
> >> connected to MPU
> >>   */
> >>  struct pruss_intc {
> >>      unsigned int irqs[MAX_NUM_HOST_IRQS];
> >>      void __iomem *base;
> >>      struct irq_domain *domain;
> >> +    u16 shared_intr;
> >> +    u16 invalid_intr;
> > 
> > Please represent bitmaps as an unsigned long.
> 
> ok. We have atmost 8 interrupts coming in, but agree on the change
> since we are using the BIT() macro below.
> 
> > 
> >>  };
> >> 
> >>  static inline u32 pruss_intc_read_reg(struct pruss_intc *intc,
> >> unsigned int reg)
> >> @@ -222,7 +226,8 @@ static int pruss_intc_probe(struct
> >> platform_device *pdev)
> >>          "host_intr4", "host_intr5", "host_intr6", "host_intr7", };
> >>      struct device *dev = &pdev->dev;
> >>      struct pruss_intc *intc;
> >> -    int i, irq;
> >> +    int i, irq, count;
> >> +    u8 temp_intr[MAX_NUM_HOST_IRQS] = { 0 };
> >> 
> >>      intc = devm_kzalloc(dev, sizeof(*intc), GFP_KERNEL);
> >>      if (!intc)
> >> @@ -235,6 +240,52 @@ static int pruss_intc_probe(struct
> >> platform_device *pdev)
> >>          return PTR_ERR(intc->base);
> >>      }
> >> 
> >> +    count = of_property_read_variable_u8_array(dev->of_node,
> >> +                           "ti,irqs-reserved",
> >> +                           temp_intr, 0,
> >> +                           MAX_NUM_HOST_IRQS);
> >> +    /*
> >> +     * The irqs-reserved is used only for some SoC's therefore not
> >> having
> >> +     * this property is still valid
> >> +     */
> >> +    if (count == -EINVAL)
> >> +        count = 0;
> >> +    if (count < 0)
> >> +        return count;
> >> +
> >> +    for (i = 0; i < count; i++) {
> >> +        if (temp_intr[i] >= MAX_NUM_HOST_IRQS) {
> >> +            dev_warn(dev, "ignoring invalid reserved irq %d\n",
> >> +                 temp_intr[i]);
> >> +            continue;
> >> +        }
> >> +
> >> +        intc->invalid_intr |= BIT(temp_intr[i]);
> >> +    }
> >> +
> >> +    count = of_property_read_variable_u8_array(dev->of_node,
> >> +                           "ti,irqs-shared",
> >> +                           temp_intr, 0,
> >> +                           MAX_NUM_HOST_IRQS);
> >> +    /*
> >> +     * The irqs-shared is used only for some SoC's therefore not having
> >> +     * this property is still valid
> >> +     */
> >> +    if (count == -EINVAL)
> >> +        count = 0;
> >> +    if (count < 0)
> >> +        return count;
> >> +
> >> +    for (i = 0; i < count; i++) {
> >> +        if (temp_intr[i] >= MAX_NUM_HOST_IRQS) {
> >> +            dev_warn(dev, "ignoring invalid shared irq %d\n",
> >> +                 temp_intr[i]);
> >> +            continue;
> >> +        }
> >> +
> >> +        intc->shared_intr |= BIT(temp_intr[i]);
> >> +    }
> >> +
> > 
> > You probably want to move this in a separate function, since you populate a
> > common structure.
> > 
> >>      pruss_intc_init(intc);
> >> 
> >>      /* always 64 events */
> >> @@ -244,8 +295,14 @@ static int pruss_intc_probe(struct
> >> platform_device *pdev)
> >>          return -ENOMEM;
> >> 
> >>      for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
> >> +        if (intc->invalid_intr & BIT(i))
> >> +            continue;
> >> +
> >>          irq = platform_get_irq_byname(pdev, irq_names[i]);
> >>          if (irq <= 0) {
> >> +            if (intc->shared_intr & BIT(i))
> >> +                continue;
> > 
> > I don't really understand why you are treating these "shared" interrupts
> > differently from the invalid ones. In all cases, they shouldn't be used.
> 
> The behavior is the same in how we handle it, but the difference is
> that an "invalid" one is never even connected to the ARM interrupt
> controller, while the "shared" one is a choice. So, unless this
> interrupt is being used/handled by a different processor/entity, you
> would not see this skipped from the dts node.

And I'm saying that all that matters is that you are discarding these
interrupts. Whether they are flagged invalid or shared, they are not
available to Linux. So the difference in handling is pointless and
only makes it harder to understand what you are doing.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Marc Zyngier]

Suman, Grzegorz,

On Wed, 15 Jul 2020 14:38:05 +0100,
Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org> wrote:
> 
> Hi Marc,
> 
> > On 7/8/20 5:47 AM, Marc Zyngier wrote:
> > > On 2020-07-08 08:04, Grzegorz Jaszczyk wrote:
> > >> On Sun, 5 Jul 2020 at 22:45, Marc Zyngier <maz@kernel.org> wrote:
> > >>>
> > >>> On 2020-07-05 14:26, Grzegorz Jaszczyk wrote:
> > >>> > On Sat, 4 Jul 2020 at 11:39, Marc Zyngier <maz@kernel.org> wrote:
> > >>> >>
> > >>> >> On 2020-07-03 15:28, Grzegorz Jaszczyk wrote:
> > >>>
> > >>> [...]
> > >>>
> > >>> >> It still begs the question: if the HW can support both edge and level
> > >>> >> triggered interrupts, why isn't the driver supporting this diversity?
> > >>> >> I appreciate that your HW may only have level interrupts so far, but
> > >>> >> what guarantees that this will forever be true? It would imply a
> > >>> >> change
> > >>> >> in the DT binding, which isn't desirable.
> > >>> >
> > >>> > Ok, I've got your point. I will try to come up with something later
> > >>> > on. Probably extending interrupt-cells by one and passing interrupt
> > >>> > type will be enough for now. Extending this driver to actually support
> > >>> > it can be handled later if needed. Hope it works for you.
> > >>>
> > >>> Writing a set_type callback to deal with this should be pretty easy.
> > >>> Don't delay doing the right thing.
> > >>
> > >> Ok.
> >
> > Sorry for the typo in my comment causing this confusion.
> >
> > The h/w actually doesn't support the edge-interrupts. Likewise, the
> > polarity is always high. The individual register bit descriptions
> > mention what the bit values 0 and 1 mean, but there is additional
> > description in the TRMs on all the SoCs that says
> > "always write 1 to the bits of this register" for PRUSS_INTC_SIPR(x) and
> > "always write 0 to the bits of this register" for PRUSS_INTC_SITR(x).
> > FWIW, these are also the reset values.
> >
> > Eg: AM335x TRM - https://www.ti.com/lit/pdf/spruh73
> > Please see Section 4.4.2.5 and the register descriptions in 4.5.3.49,
> > 4.5.3.51. Please also see Section 4.4.2.3 that explains the PRUSS INTC
> > methodology.
> >
> > >>
> > >>>
> > >>> [...]
> > >>>
> > >>> >> >> > +             hwirq = hipir & GENMASK(9, 0);
> > >>> >> >> > +             virq = irq_linear_revmap(intc->domain, hwirq);
> > >>> >> >>
> > >>> >> >> And this is where I worry. You seems to have a single irqdomain
> > >>> >> >> for all the muxes. Are you guaranteed that you will have no
> > >>> >> >> overlap between muxes? And please use irq_find_mapping(), as
> > >>> >> >> I have top-secret plans to kill irq_linear_revmap().
> > >>> >> >
> > >>> >> > Regarding irq_find_mapping - sure.
> > >>> >> >
> > >>> >> > Regarding irqdomains:
> > >>> >> > It is a single irqdomain since the hwirq (system event) can be
> > >>> mapped
> > >>> >> > to different irq_host (muxes). Patch #6
> > >>> >> > https://lkml.org/lkml/2020/7/2/616 implements and describes how
> > >>> input
> > >>> >> > events can be mapped to some output host interrupts through 2
> > >>> levels
> > >>> >> > of many-to-one mapping i.e. events to channel mapping and
> > >>> channels to
> > >>> >> > host interrupts. Mentioned implementation ensures that specific
> > >>> system
> > >>> >> > event (hwirq) can be mapped through PRUSS specific channel into a
> > >>> >> > single host interrupt.
> > >>> >>
> > >>> >> Patch #6 is a nightmare of its own, and I haven't fully groked it
> > >>> yet.
> > >>> >> Also, this driver seems to totally ignore the 2-level routing. Where
> > >>> >> is it set up? map/unmap in this driver do exactly *nothing*, so
> > >>> >> something somewhere must set it up.
> > >>> >
> > >>> > The map/unmap is updated in patch #6 and it deals with those 2-level
> > >>> > routing setup. Map is responsible for programming the Channel Map
> > >>> > Registers (CMRx) and Host-Interrupt Map Registers (HMRx) basing on
> > >>> > provided configuration from the one parsed in the xlate function.
> > >>> > Unmap undo whatever was done on the map. More details can be found in
> > >>> > patch #6.
> > >>> >
> > >>> > Maybe it would be better to squash patch #6 with this one so it would
> > >>> > be less confusing. What is your advice?
> > >>>
> > >>> So am I right in understanding that without patch #6, this driver does
> > >>> exactly nothing? If so, it has been a waste of review time.
> > >>>
> > >>> Please split patch #6 so that this driver does something useful
> > >>> for Linux, without any of the PRU interrupt routing stuff. I want
> > >>> to see a Linux-only driver that works and doesn't rely on any other
> > >>> exotic feature.
> > >>>
> > >>
> > >> Patch #6 provides PRU specific 2-level routing setup. This step is
> > >> required and it is part of the entire patch-set. Theoretically routing
> > >> setup could be done by other platform driver (not irq one) or e.g. by
> > >> PRU firmware. In such case this driver would be functional without
> > >> patch #6 but I do not think it would be proper.
> > >
> > > Then this whole driver is non-functional until the last patch that
> > > comes with the PRU-specific "value-add".
> >
> > It is all moot actually and the interrupts work only when the PRU
> > remoteproc/clients have invoked the irq_create_fwspec_mapping()
> > for all of the desired system events. It does not make much difference
> > if it was a separate patch or squashed in, patch #6 is a replacement for
> > the previous logic, and since it was complex, it was done in a separate
> > patch to better explain the usage (same reason on v1 and v2 as
> > well).

It may make no difference to you, but it does for me, as I'm the lucky
idiot reviewing this code. So I am going to say it again: please keep
anything that only exists for the PRU subsystem benefit out of the
initial patches.

I want to see something that works for Linux, and only for Linux. Once
we have that working, we'll see to add more stuff. But stop throwing
the PRU business into the early patches, as all you are achieving is
to delay the whole thing.

> >
> > >
> > > [...]
> > >
> > >> I am open to any suggestion if there is a better way of handling
> > >> 2-level routing. I will also appreciate if you could elaborate about
> > >> issues that you see with patch #6.
> > >
> > > The two level routing has to be part of this (or another) irqchip
> > > driver (specially given that it appears to me like another set of
> > > crossbar). There should only be a *single* binding for all interrupts,
> > > including those targeting the PRU (you seem to have two).
> > >
> >
> > Yeah, there hasn't been a clean way of doing this. Our previous attempt
> > was to do this through custom exported functions so that the PRU
> > remoteproc driver can set these up correctly, but that was shot down and
> > this is the direction we are pointed to.
> >
> > We do want to leverage the "interrupts" property in the PRU user nodes
> > instead of inventing our own paradigm through a non-irqchip driver, and
> > at the same time, be able to configure this at the run time only when
> > that PRU driver is running, and remove the mappings once that driver is
> > removed allowing another PRU application/driver. We treat PRUs as an
> > exclusive resource, so everything needs to go along with an appropriate
> > client user.
> 
> I will just add an explanation about interrupt binding. So actually
> there is one dt-binding defined in yaml (interrupt-cells = 1). The
> reason why you see xlate allowing to proceed with 1 or 3 parameters is
> because linux can change the PRU firmware at run-time (thorough linux
> remoteproc framework) and different firmware may require different
> kinds of interrupt mapping. Therefore during firmware load, the new
> mapping is created through irq_create_fwspec_mapping() and in this
> case 3 parameters are passed: system event, channel and host irq.
> Similarly the mapping is disposed during remoteproc stop by invoking
> irq_dispose_mapping. This allows to create new mapping, in the same
> way, for next firmware loaded through Linux remote-proc at runtime
> (depending on the needs of new remoteproc firmware).
> 
> On the other hand dt-bindings defines interrupt-cells = 1, so when the
> interrupt is registered the xlate function (proceed with 1 parameter)
> checks if this event already has valid mapping - if yes we are fine,
> if not we return -EINVAL.

It means that interrupts declared in DT get their two-level routing
via the kernel driver, while PRU interrupts get their routing via some
external blob that Linux is not in control of?

If so, this looks broken. What if you get a resource allocation
conflict because the kernel and the blob are stepping into each
other's toes? Why should an end-point client decide on the routing of
the interrupt?

All the end-point should provide is the ID of the input signal, and to
which PRU this is routed. Interrupts described in DT should have the
exact same model (input signal, target). All the intermediate routing
logic should be handled by the Linux driver for *all* interrupts in
the system.

> 
> >
> > > And the non-CPU interrupt code has to be in its own patch, because
> > > it is pretty borderline anyway (I'm still not completely convinced
> > > this is Linux's job).
> >
> > The logic for non-CPU interrupt code is exactly the same as the CPU
> > interrupt code, as they are all setup through the
> > irq_create_fwspec_mapping(). The CPU-specific pieces are primarily the
> > chained interrupt handling.
> >
> > We have already argued internally about the last part, but our firmware
> > developers literally don't have any IRAM space (we have a lot of
> > Industrial protocols working out of 4K/8K memory), and have pushed all
> > one-time setup to the OS running (Linux or otherwise) on the main ARM
> > core, and INTC is one among the other many such settings. Every word in
> > Instruction RAM was crucial for them.

And that's fine. Just push *all* of it into Linux, and not just the
programming of the registers.

> >
> > So, we are all ears if there is still an elegant way of doing this. Look
> > forward to any suggestions you may have.
> 
> Yes, the non-CPU logic is exactly the same as the CPU interrupt code
> as Suman described. There is no distinction between routing setup for
> main CPU and PRU core, both use exactly the same logic, just different
> numbers are passed through  irq_create_fwspec_mapping.

It obviously isn't the same at the moment. You have two distinct code
paths, two ways to describe a mapping, and a potential resource
allocation issue.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Grzegorz Jaszczyk]

Hi Marc,

First of all thank you very much for your review. I apologize in
advance if the description below is too verbose or not detailed
enough.

On Fri, 17 Jul 2020 at 14:36, Marc Zyngier <maz@kernel.org> wrote:
>
> Suman, Grzegorz,
>
> On Wed, 15 Jul 2020 14:38:05 +0100,
> Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org> wrote:
> >
> > Hi Marc,
> >
> > > On 7/8/20 5:47 AM, Marc Zyngier wrote:
> > > > On 2020-07-08 08:04, Grzegorz Jaszczyk wrote:
> > > >> On Sun, 5 Jul 2020 at 22:45, Marc Zyngier <maz@kernel.org> wrote:
> > > >>>
> > > >>> On 2020-07-05 14:26, Grzegorz Jaszczyk wrote:
> > > >>> > On Sat, 4 Jul 2020 at 11:39, Marc Zyngier <maz@kernel.org> wrote:
> > > >>> >>
> > > >>> >> On 2020-07-03 15:28, Grzegorz Jaszczyk wrote:
> > > >>>
> > > >>> [...]
> > > >>>
> > > >>> >> It still begs the question: if the HW can support both edge and level
> > > >>> >> triggered interrupts, why isn't the driver supporting this diversity?
> > > >>> >> I appreciate that your HW may only have level interrupts so far, but
> > > >>> >> what guarantees that this will forever be true? It would imply a
> > > >>> >> change
> > > >>> >> in the DT binding, which isn't desirable.
> > > >>> >
> > > >>> > Ok, I've got your point. I will try to come up with something later
> > > >>> > on. Probably extending interrupt-cells by one and passing interrupt
> > > >>> > type will be enough for now. Extending this driver to actually support
> > > >>> > it can be handled later if needed. Hope it works for you.
> > > >>>
> > > >>> Writing a set_type callback to deal with this should be pretty easy.
> > > >>> Don't delay doing the right thing.
> > > >>
> > > >> Ok.
> > >
> > > Sorry for the typo in my comment causing this confusion.
> > >
> > > The h/w actually doesn't support the edge-interrupts. Likewise, the
> > > polarity is always high. The individual register bit descriptions
> > > mention what the bit values 0 and 1 mean, but there is additional
> > > description in the TRMs on all the SoCs that says
> > > "always write 1 to the bits of this register" for PRUSS_INTC_SIPR(x) and
> > > "always write 0 to the bits of this register" for PRUSS_INTC_SITR(x).
> > > FWIW, these are also the reset values.
> > >
> > > Eg: AM335x TRM - https://www.ti.com/lit/pdf/spruh73
> > > Please see Section 4.4.2.5 and the register descriptions in 4.5.3.49,
> > > 4.5.3.51. Please also see Section 4.4.2.3 that explains the PRUSS INTC
> > > methodology.
> > >
> > > >>
> > > >>>
> > > >>> [...]
> > > >>>
> > > >>> >> >> > +             hwirq = hipir & GENMASK(9, 0);
> > > >>> >> >> > +             virq = irq_linear_revmap(intc->domain, hwirq);
> > > >>> >> >>
> > > >>> >> >> And this is where I worry. You seems to have a single irqdomain
> > > >>> >> >> for all the muxes. Are you guaranteed that you will have no
> > > >>> >> >> overlap between muxes? And please use irq_find_mapping(), as
> > > >>> >> >> I have top-secret plans to kill irq_linear_revmap().
> > > >>> >> >
> > > >>> >> > Regarding irq_find_mapping - sure.
> > > >>> >> >
> > > >>> >> > Regarding irqdomains:
> > > >>> >> > It is a single irqdomain since the hwirq (system event) can be
> > > >>> mapped
> > > >>> >> > to different irq_host (muxes). Patch #6
> > > >>> >> > https://lkml.org/lkml/2020/7/2/616 implements and describes how
> > > >>> input
> > > >>> >> > events can be mapped to some output host interrupts through 2
> > > >>> levels
> > > >>> >> > of many-to-one mapping i.e. events to channel mapping and
> > > >>> channels to
> > > >>> >> > host interrupts. Mentioned implementation ensures that specific
> > > >>> system
> > > >>> >> > event (hwirq) can be mapped through PRUSS specific channel into a
> > > >>> >> > single host interrupt.
> > > >>> >>
> > > >>> >> Patch #6 is a nightmare of its own, and I haven't fully groked it
> > > >>> yet.
> > > >>> >> Also, this driver seems to totally ignore the 2-level routing. Where
> > > >>> >> is it set up? map/unmap in this driver do exactly *nothing*, so
> > > >>> >> something somewhere must set it up.
> > > >>> >
> > > >>> > The map/unmap is updated in patch #6 and it deals with those 2-level
> > > >>> > routing setup. Map is responsible for programming the Channel Map
> > > >>> > Registers (CMRx) and Host-Interrupt Map Registers (HMRx) basing on
> > > >>> > provided configuration from the one parsed in the xlate function.
> > > >>> > Unmap undo whatever was done on the map. More details can be found in
> > > >>> > patch #6.
> > > >>> >
> > > >>> > Maybe it would be better to squash patch #6 with this one so it would
> > > >>> > be less confusing. What is your advice?
> > > >>>
> > > >>> So am I right in understanding that without patch #6, this driver does
> > > >>> exactly nothing? If so, it has been a waste of review time.
> > > >>>
> > > >>> Please split patch #6 so that this driver does something useful
> > > >>> for Linux, without any of the PRU interrupt routing stuff. I want
> > > >>> to see a Linux-only driver that works and doesn't rely on any other
> > > >>> exotic feature.
> > > >>>
> > > >>
> > > >> Patch #6 provides PRU specific 2-level routing setup. This step is
> > > >> required and it is part of the entire patch-set. Theoretically routing
> > > >> setup could be done by other platform driver (not irq one) or e.g. by
> > > >> PRU firmware. In such case this driver would be functional without
> > > >> patch #6 but I do not think it would be proper.
> > > >
> > > > Then this whole driver is non-functional until the last patch that
> > > > comes with the PRU-specific "value-add".
> > >
> > > It is all moot actually and the interrupts work only when the PRU
> > > remoteproc/clients have invoked the irq_create_fwspec_mapping()
> > > for all of the desired system events. It does not make much difference
> > > if it was a separate patch or squashed in, patch #6 is a replacement for
> > > the previous logic, and since it was complex, it was done in a separate
> > > patch to better explain the usage (same reason on v1 and v2 as
> > > well).
>
> It may make no difference to you, but it does for me, as I'm the lucky
> idiot reviewing this code. So I am going to say it again: please keep
> anything that only exists for the PRU subsystem benefit out of the
> initial patches.
>
> I want to see something that works for Linux, and only for Linux. Once
> we have that working, we'll see to add more stuff. But stop throwing
> the PRU business into the early patches, as all you are achieving is
> to delay the whole thing.
>
> > >
> > > >
> > > > [...]
> > > >
> > > >> I am open to any suggestion if there is a better way of handling
> > > >> 2-level routing. I will also appreciate if you could elaborate about
> > > >> issues that you see with patch #6.
> > > >
> > > > The two level routing has to be part of this (or another) irqchip
> > > > driver (specially given that it appears to me like another set of
> > > > crossbar). There should only be a *single* binding for all interrupts,
> > > > including those targeting the PRU (you seem to have two).
> > > >
> > >
> > > Yeah, there hasn't been a clean way of doing this. Our previous attempt
> > > was to do this through custom exported functions so that the PRU
> > > remoteproc driver can set these up correctly, but that was shot down and
> > > this is the direction we are pointed to.
> > >
> > > We do want to leverage the "interrupts" property in the PRU user nodes
> > > instead of inventing our own paradigm through a non-irqchip driver, and
> > > at the same time, be able to configure this at the run time only when
> > > that PRU driver is running, and remove the mappings once that driver is
> > > removed allowing another PRU application/driver. We treat PRUs as an
> > > exclusive resource, so everything needs to go along with an appropriate
> > > client user.
> >
> > I will just add an explanation about interrupt binding. So actually
> > there is one dt-binding defined in yaml (interrupt-cells = 1). The
> > reason why you see xlate allowing to proceed with 1 or 3 parameters is
> > because linux can change the PRU firmware at run-time (thorough linux
> > remoteproc framework) and different firmware may require different
> > kinds of interrupt mapping. Therefore during firmware load, the new
> > mapping is created through irq_create_fwspec_mapping() and in this
> > case 3 parameters are passed: system event, channel and host irq.
> > Similarly the mapping is disposed during remoteproc stop by invoking
> > irq_dispose_mapping. This allows to create new mapping, in the same
> > way, for next firmware loaded through Linux remote-proc at runtime
> > (depending on the needs of new remoteproc firmware).
> >
> > On the other hand dt-bindings defines interrupt-cells = 1, so when the
> > interrupt is registered the xlate function (proceed with 1 parameter)
> > checks if this event already has valid mapping - if yes we are fine,
> > if not we return -EINVAL.
>
> It means that interrupts declared in DT get their two-level routing
> via the kernel driver, while PRU interrupts get their routing via some
> external blob that Linux is not in control of?

Actually with the current approach all two-level routing goes through
this linux driver. The interrupts that should be routed to PRU are
described in remoteproc firmware resource table [1] and it is under
Linux remoteproc driver control. In general, the resource table
contains system resources that the remote processor requires before it
should be powered on. We treat the interrupt mapping (described in the
resource table, which is a dedicated elf section defined in [1]) as
one of system resources that linux has to provide before we power on
the PRU core. Therefore the remoteproce driver will parse the resource
table and trigger irq_create_fwspec_mapping() after validating
resource table content.

[1] https://www.kernel.org/doc/Documentation/remoteproc.txt (Binary
Firmware Structure)

>
> If so, this looks broken. What if you get a resource allocation
> conflict because the kernel and the blob are stepping into each
> other's toes? Why should an end-point client decide on the routing of
> the interrupt?

The code in the pruss_intc_map function checks if there are no
allocation conflicts: e.g. if the sysevent is already assigned it will
throw -EBUSY. Similarly when some channel was already assigned to
host_irq and a different assignment is requested it will again throw
-EBUSY.

>
> All the end-point should provide is the ID of the input signal, and to
> which PRU this is routed. Interrupts described in DT should have the
> exact same model (input signal, target). All the intermediate routing
> logic should be handled by the Linux driver for *all* interrupts in
> the system.

There is one issue with this approach: the channel number corresponds
to the priority as described in TRM and PRU core firmware relies on
those priorities. Because the interrupt routing for the PRU core will
also go through this linux interrupt driver I think we have to stick
with 3 parameter descriptions.

>
> >
> > >
> > > > And the non-CPU interrupt code has to be in its own patch, because
> > > > it is pretty borderline anyway (I'm still not completely convinced
> > > > this is Linux's job).
> > >
> > > The logic for non-CPU interrupt code is exactly the same as the CPU
> > > interrupt code, as they are all setup through the
> > > irq_create_fwspec_mapping(). The CPU-specific pieces are primarily the
> > > chained interrupt handling.
> > >
> > > We have already argued internally about the last part, but our firmware
> > > developers literally don't have any IRAM space (we have a lot of
> > > Industrial protocols working out of 4K/8K memory), and have pushed all
> > > one-time setup to the OS running (Linux or otherwise) on the main ARM
> > > core, and INTC is one among the other many such settings. Every word in
> > > Instruction RAM was crucial for them.
>
> And that's fine. Just push *all* of it into Linux, and not just the
> programming of the registers.
>
> > >
> > > So, we are all ears if there is still an elegant way of doing this. Look
> > > forward to any suggestions you may have.
> >
> > Yes, the non-CPU logic is exactly the same as the CPU interrupt code
> > as Suman described. There is no distinction between routing setup for
> > main CPU and PRU core, both use exactly the same logic, just different
> > numbers are passed through  irq_create_fwspec_mapping.
>
> It obviously isn't the same at the moment. You have two distinct code
> paths, two ways to describe a mapping, and a potential resource
> allocation issue.
>

Ok, I will get rid of the two distinct code paths in the xlate
function (in patch#6) and change the #interrupt-cells to 3 which and
describe the entire interrupt routing in DT for interrupts targeted to
the main CPU. Please let me know if you have any further comments.

Thank you,
Grzegorz

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Marc Zyngier]

On 2020-07-21 10:27, Grzegorz Jaszczyk wrote:
> Hi Marc,
> 
> First of all thank you very much for your review. I apologize in
> advance if the description below is too verbose or not detailed
> enough.
> 
> On Fri, 17 Jul 2020 at 14:36, Marc Zyngier <maz@kernel.org> wrote:
>> 
>> Suman, Grzegorz,
>> 
>> On Wed, 15 Jul 2020 14:38:05 +0100,
>> Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org> wrote:
>> >
>> > Hi Marc,
>> >
>> > > On 7/8/20 5:47 AM, Marc Zyngier wrote:
>> > > > On 2020-07-08 08:04, Grzegorz Jaszczyk wrote:
>> > > >> On Sun, 5 Jul 2020 at 22:45, Marc Zyngier <maz@kernel.org> wrote:
>> > > >>>
>> > > >>> On 2020-07-05 14:26, Grzegorz Jaszczyk wrote:
>> > > >>> > On Sat, 4 Jul 2020 at 11:39, Marc Zyngier <maz@kernel.org> wrote:
>> > > >>> >>
>> > > >>> >> On 2020-07-03 15:28, Grzegorz Jaszczyk wrote:
>> > > >>>
>> > > >>> [...]
>> > > >>>
>> > > >>> >> It still begs the question: if the HW can support both edge and level
>> > > >>> >> triggered interrupts, why isn't the driver supporting this diversity?
>> > > >>> >> I appreciate that your HW may only have level interrupts so far, but
>> > > >>> >> what guarantees that this will forever be true? It would imply a
>> > > >>> >> change
>> > > >>> >> in the DT binding, which isn't desirable.
>> > > >>> >
>> > > >>> > Ok, I've got your point. I will try to come up with something later
>> > > >>> > on. Probably extending interrupt-cells by one and passing interrupt
>> > > >>> > type will be enough for now. Extending this driver to actually support
>> > > >>> > it can be handled later if needed. Hope it works for you.
>> > > >>>
>> > > >>> Writing a set_type callback to deal with this should be pretty easy.
>> > > >>> Don't delay doing the right thing.
>> > > >>
>> > > >> Ok.
>> > >
>> > > Sorry for the typo in my comment causing this confusion.
>> > >
>> > > The h/w actually doesn't support the edge-interrupts. Likewise, the
>> > > polarity is always high. The individual register bit descriptions
>> > > mention what the bit values 0 and 1 mean, but there is additional
>> > > description in the TRMs on all the SoCs that says
>> > > "always write 1 to the bits of this register" for PRUSS_INTC_SIPR(x) and
>> > > "always write 0 to the bits of this register" for PRUSS_INTC_SITR(x).
>> > > FWIW, these are also the reset values.
>> > >
>> > > Eg: AM335x TRM - https://www.ti.com/lit/pdf/spruh73
>> > > Please see Section 4.4.2.5 and the register descriptions in 4.5.3.49,
>> > > 4.5.3.51. Please also see Section 4.4.2.3 that explains the PRUSS INTC
>> > > methodology.
>> > >
>> > > >>
>> > > >>>
>> > > >>> [...]
>> > > >>>
>> > > >>> >> >> > +             hwirq = hipir & GENMASK(9, 0);
>> > > >>> >> >> > +             virq = irq_linear_revmap(intc->domain, hwirq);
>> > > >>> >> >>
>> > > >>> >> >> And this is where I worry. You seems to have a single irqdomain
>> > > >>> >> >> for all the muxes. Are you guaranteed that you will have no
>> > > >>> >> >> overlap between muxes? And please use irq_find_mapping(), as
>> > > >>> >> >> I have top-secret plans to kill irq_linear_revmap().
>> > > >>> >> >
>> > > >>> >> > Regarding irq_find_mapping - sure.
>> > > >>> >> >
>> > > >>> >> > Regarding irqdomains:
>> > > >>> >> > It is a single irqdomain since the hwirq (system event) can be
>> > > >>> mapped
>> > > >>> >> > to different irq_host (muxes). Patch #6
>> > > >>> >> > https://lkml.org/lkml/2020/7/2/616 implements and describes how
>> > > >>> input
>> > > >>> >> > events can be mapped to some output host interrupts through 2
>> > > >>> levels
>> > > >>> >> > of many-to-one mapping i.e. events to channel mapping and
>> > > >>> channels to
>> > > >>> >> > host interrupts. Mentioned implementation ensures that specific
>> > > >>> system
>> > > >>> >> > event (hwirq) can be mapped through PRUSS specific channel into a
>> > > >>> >> > single host interrupt.
>> > > >>> >>
>> > > >>> >> Patch #6 is a nightmare of its own, and I haven't fully groked it
>> > > >>> yet.
>> > > >>> >> Also, this driver seems to totally ignore the 2-level routing. Where
>> > > >>> >> is it set up? map/unmap in this driver do exactly *nothing*, so
>> > > >>> >> something somewhere must set it up.
>> > > >>> >
>> > > >>> > The map/unmap is updated in patch #6 and it deals with those 2-level
>> > > >>> > routing setup. Map is responsible for programming the Channel Map
>> > > >>> > Registers (CMRx) and Host-Interrupt Map Registers (HMRx) basing on
>> > > >>> > provided configuration from the one parsed in the xlate function.
>> > > >>> > Unmap undo whatever was done on the map. More details can be found in
>> > > >>> > patch #6.
>> > > >>> >
>> > > >>> > Maybe it would be better to squash patch #6 with this one so it would
>> > > >>> > be less confusing. What is your advice?
>> > > >>>
>> > > >>> So am I right in understanding that without patch #6, this driver does
>> > > >>> exactly nothing? If so, it has been a waste of review time.
>> > > >>>
>> > > >>> Please split patch #6 so that this driver does something useful
>> > > >>> for Linux, without any of the PRU interrupt routing stuff. I want
>> > > >>> to see a Linux-only driver that works and doesn't rely on any other
>> > > >>> exotic feature.
>> > > >>>
>> > > >>
>> > > >> Patch #6 provides PRU specific 2-level routing setup. This step is
>> > > >> required and it is part of the entire patch-set. Theoretically routing
>> > > >> setup could be done by other platform driver (not irq one) or e.g. by
>> > > >> PRU firmware. In such case this driver would be functional without
>> > > >> patch #6 but I do not think it would be proper.
>> > > >
>> > > > Then this whole driver is non-functional until the last patch that
>> > > > comes with the PRU-specific "value-add".
>> > >
>> > > It is all moot actually and the interrupts work only when the PRU
>> > > remoteproc/clients have invoked the irq_create_fwspec_mapping()
>> > > for all of the desired system events. It does not make much difference
>> > > if it was a separate patch or squashed in, patch #6 is a replacement for
>> > > the previous logic, and since it was complex, it was done in a separate
>> > > patch to better explain the usage (same reason on v1 and v2 as
>> > > well).
>> 
>> It may make no difference to you, but it does for me, as I'm the lucky
>> idiot reviewing this code. So I am going to say it again: please keep
>> anything that only exists for the PRU subsystem benefit out of the
>> initial patches.
>> 
>> I want to see something that works for Linux, and only for Linux. Once
>> we have that working, we'll see to add more stuff. But stop throwing
>> the PRU business into the early patches, as all you are achieving is
>> to delay the whole thing.
>> 
>> > >
>> > > >
>> > > > [...]
>> > > >
>> > > >> I am open to any suggestion if there is a better way of handling
>> > > >> 2-level routing. I will also appreciate if you could elaborate about
>> > > >> issues that you see with patch #6.
>> > > >
>> > > > The two level routing has to be part of this (or another) irqchip
>> > > > driver (specially given that it appears to me like another set of
>> > > > crossbar). There should only be a *single* binding for all interrupts,
>> > > > including those targeting the PRU (you seem to have two).
>> > > >
>> > >
>> > > Yeah, there hasn't been a clean way of doing this. Our previous attempt
>> > > was to do this through custom exported functions so that the PRU
>> > > remoteproc driver can set these up correctly, but that was shot down and
>> > > this is the direction we are pointed to.
>> > >
>> > > We do want to leverage the "interrupts" property in the PRU user nodes
>> > > instead of inventing our own paradigm through a non-irqchip driver, and
>> > > at the same time, be able to configure this at the run time only when
>> > > that PRU driver is running, and remove the mappings once that driver is
>> > > removed allowing another PRU application/driver. We treat PRUs as an
>> > > exclusive resource, so everything needs to go along with an appropriate
>> > > client user.
>> >
>> > I will just add an explanation about interrupt binding. So actually
>> > there is one dt-binding defined in yaml (interrupt-cells = 1). The
>> > reason why you see xlate allowing to proceed with 1 or 3 parameters is
>> > because linux can change the PRU firmware at run-time (thorough linux
>> > remoteproc framework) and different firmware may require different
>> > kinds of interrupt mapping. Therefore during firmware load, the new
>> > mapping is created through irq_create_fwspec_mapping() and in this
>> > case 3 parameters are passed: system event, channel and host irq.
>> > Similarly the mapping is disposed during remoteproc stop by invoking
>> > irq_dispose_mapping. This allows to create new mapping, in the same
>> > way, for next firmware loaded through Linux remote-proc at runtime
>> > (depending on the needs of new remoteproc firmware).
>> >
>> > On the other hand dt-bindings defines interrupt-cells = 1, so when the
>> > interrupt is registered the xlate function (proceed with 1 parameter)
>> > checks if this event already has valid mapping - if yes we are fine,
>> > if not we return -EINVAL.
>> 
>> It means that interrupts declared in DT get their two-level routing
>> via the kernel driver, while PRU interrupts get their routing via some
>> external blob that Linux is not in control of?
> 
> Actually with the current approach all two-level routing goes through
> this linux driver. The interrupts that should be routed to PRU are
> described in remoteproc firmware resource table [1] and it is under
> Linux remoteproc driver control. In general, the resource table
> contains system resources that the remote processor requires before it
> should be powered on. We treat the interrupt mapping (described in the
> resource table, which is a dedicated elf section defined in [1]) as
> one of system resources that linux has to provide before we power on
> the PRU core. Therefore the remoteproce driver will parse the resource
> table and trigger irq_create_fwspec_mapping() after validating
> resource table content.

Validating the resource table says nothing of a potential conflict
with previous configured interrupts.

> 
> [1] https://www.kernel.org/doc/Documentation/remoteproc.txt (Binary
> Firmware Structure)
> 
>> 
>> If so, this looks broken. What if you get a resource allocation
>> conflict because the kernel and the blob are stepping into each
>> other's toes? Why should an end-point client decide on the routing of
>> the interrupt?
> 
> The code in the pruss_intc_map function checks if there are no
> allocation conflicts: e.g. if the sysevent is already assigned it will
> throw -EBUSY. Similarly when some channel was already assigned to
> host_irq and a different assignment is requested it will again throw
> -EBUSY.

But why should it? The allocation should take place based on constraints
(source, target, and as you mentioned below, priority). Instead, you
seem to be relying on static allocation coming from binary blobs,
standardized or not.

I claim that this static allocation is madness and should be eliminated.
Instead, the Linux driver should perform the routing based on allocation
requirements (the above constraints), and only fail if it cannot satisfy
these constraints.

> 
>> 
>> All the end-point should provide is the ID of the input signal, and to
>> which PRU this is routed. Interrupts described in DT should have the
>> exact same model (input signal, target). All the intermediate routing
>> logic should be handled by the Linux driver for *all* interrupts in
>> the system.
> 
> There is one issue with this approach: the channel number corresponds
> to the priority as described in TRM and PRU core firmware relies on
> those priorities. Because the interrupt routing for the PRU core will
> also go through this linux interrupt driver I think we have to stick
> with 3 parameter descriptions.

Sure, that's fine. All I want to see is a single way to route an
interrupt from source to destination, and stop relying on static
allocations coming from binary blobs.

>> > > > And the non-CPU interrupt code has to be in its own patch, because
>> > > > it is pretty borderline anyway (I'm still not completely convinced
>> > > > this is Linux's job).
>> > >
>> > > The logic for non-CPU interrupt code is exactly the same as the CPU
>> > > interrupt code, as they are all setup through the
>> > > irq_create_fwspec_mapping(). The CPU-specific pieces are primarily the
>> > > chained interrupt handling.
>> > >
>> > > We have already argued internally about the last part, but our firmware
>> > > developers literally don't have any IRAM space (we have a lot of
>> > > Industrial protocols working out of 4K/8K memory), and have pushed all
>> > > one-time setup to the OS running (Linux or otherwise) on the main ARM
>> > > core, and INTC is one among the other many such settings. Every word in
>> > > Instruction RAM was crucial for them.
>> 
>> And that's fine. Just push *all* of it into Linux, and not just the
>> programming of the registers.
>> 
>> > >
>> > > So, we are all ears if there is still an elegant way of doing this. Look
>> > > forward to any suggestions you may have.
>> >
>> > Yes, the non-CPU logic is exactly the same as the CPU interrupt code
>> > as Suman described. There is no distinction between routing setup for
>> > main CPU and PRU core, both use exactly the same logic, just different
>> > numbers are passed through  irq_create_fwspec_mapping.
>> 
>> It obviously isn't the same at the moment. You have two distinct code
>> paths, two ways to describe a mapping, and a potential resource
>> allocation issue.
>> 
> 
> Ok, I will get rid of the two distinct code paths in the xlate
> function (in patch#6) and change the #interrupt-cells to 3 which and
> describe the entire interrupt routing in DT for interrupts targeted to
> the main CPU. Please let me know if you have any further comments.

None for now, as I think I have made my point clear enough.

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts

[Grzegorz Jaszczyk]

Hi Marc,

Thank you again.

On Tue, 21 Jul 2020 at 12:10, Marc Zyngier <maz@kernel.org> wrote:
>
> On 2020-07-21 10:27, Grzegorz Jaszczyk wrote:
> > Hi Marc,
> >
> > First of all thank you very much for your review. I apologize in
> > advance if the description below is too verbose or not detailed
> > enough.
> >
> > On Fri, 17 Jul 2020 at 14:36, Marc Zyngier <maz@kernel.org> wrote:
> >>
> >> Suman, Grzegorz,
> >>
> >> On Wed, 15 Jul 2020 14:38:05 +0100,
> >> Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org> wrote:
> >> >
> >> > Hi Marc,
> >> >
> >> > > On 7/8/20 5:47 AM, Marc Zyngier wrote:
> >> > > > On 2020-07-08 08:04, Grzegorz Jaszczyk wrote:
> >> > > >> On Sun, 5 Jul 2020 at 22:45, Marc Zyngier <maz@kernel.org> wrote:
> >> > > >>>
> >> > > >>> On 2020-07-05 14:26, Grzegorz Jaszczyk wrote:
> >> > > >>> > On Sat, 4 Jul 2020 at 11:39, Marc Zyngier <maz@kernel.org> wrote:
> >> > > >>> >>
> >> > > >>> >> On 2020-07-03 15:28, Grzegorz Jaszczyk wrote:
> >> > > >>>
> >> > > >>> [...]
> >> > > >>>
> >> > > >>> >> It still begs the question: if the HW can support both edge and level
> >> > > >>> >> triggered interrupts, why isn't the driver supporting this diversity?
> >> > > >>> >> I appreciate that your HW may only have level interrupts so far, but
> >> > > >>> >> what guarantees that this will forever be true? It would imply a
> >> > > >>> >> change
> >> > > >>> >> in the DT binding, which isn't desirable.
> >> > > >>> >
> >> > > >>> > Ok, I've got your point. I will try to come up with something later
> >> > > >>> > on. Probably extending interrupt-cells by one and passing interrupt
> >> > > >>> > type will be enough for now. Extending this driver to actually support
> >> > > >>> > it can be handled later if needed. Hope it works for you.
> >> > > >>>
> >> > > >>> Writing a set_type callback to deal with this should be pretty easy.
> >> > > >>> Don't delay doing the right thing.
> >> > > >>
> >> > > >> Ok.
> >> > >
> >> > > Sorry for the typo in my comment causing this confusion.
> >> > >
> >> > > The h/w actually doesn't support the edge-interrupts. Likewise, the
> >> > > polarity is always high. The individual register bit descriptions
> >> > > mention what the bit values 0 and 1 mean, but there is additional
> >> > > description in the TRMs on all the SoCs that says
> >> > > "always write 1 to the bits of this register" for PRUSS_INTC_SIPR(x) and
> >> > > "always write 0 to the bits of this register" for PRUSS_INTC_SITR(x).
> >> > > FWIW, these are also the reset values.
> >> > >
> >> > > Eg: AM335x TRM - https://www.ti.com/lit/pdf/spruh73
> >> > > Please see Section 4.4.2.5 and the register descriptions in 4.5.3.49,
> >> > > 4.5.3.51. Please also see Section 4.4.2.3 that explains the PRUSS INTC
> >> > > methodology.
> >> > >
> >> > > >>
> >> > > >>>
> >> > > >>> [...]
> >> > > >>>
> >> > > >>> >> >> > +             hwirq = hipir & GENMASK(9, 0);
> >> > > >>> >> >> > +             virq = irq_linear_revmap(intc->domain, hwirq);
> >> > > >>> >> >>
> >> > > >>> >> >> And this is where I worry. You seems to have a single irqdomain
> >> > > >>> >> >> for all the muxes. Are you guaranteed that you will have no
> >> > > >>> >> >> overlap between muxes? And please use irq_find_mapping(), as
> >> > > >>> >> >> I have top-secret plans to kill irq_linear_revmap().
> >> > > >>> >> >
> >> > > >>> >> > Regarding irq_find_mapping - sure.
> >> > > >>> >> >
> >> > > >>> >> > Regarding irqdomains:
> >> > > >>> >> > It is a single irqdomain since the hwirq (system event) can be
> >> > > >>> mapped
> >> > > >>> >> > to different irq_host (muxes). Patch #6
> >> > > >>> >> > https://lkml.org/lkml/2020/7/2/616 implements and describes how
> >> > > >>> input
> >> > > >>> >> > events can be mapped to some output host interrupts through 2
> >> > > >>> levels
> >> > > >>> >> > of many-to-one mapping i.e. events to channel mapping and
> >> > > >>> channels to
> >> > > >>> >> > host interrupts. Mentioned implementation ensures that specific
> >> > > >>> system
> >> > > >>> >> > event (hwirq) can be mapped through PRUSS specific channel into a
> >> > > >>> >> > single host interrupt.
> >> > > >>> >>
> >> > > >>> >> Patch #6 is a nightmare of its own, and I haven't fully groked it
> >> > > >>> yet.
> >> > > >>> >> Also, this driver seems to totally ignore the 2-level routing. Where
> >> > > >>> >> is it set up? map/unmap in this driver do exactly *nothing*, so
> >> > > >>> >> something somewhere must set it up.
> >> > > >>> >
> >> > > >>> > The map/unmap is updated in patch #6 and it deals with those 2-level
> >> > > >>> > routing setup. Map is responsible for programming the Channel Map
> >> > > >>> > Registers (CMRx) and Host-Interrupt Map Registers (HMRx) basing on
> >> > > >>> > provided configuration from the one parsed in the xlate function.
> >> > > >>> > Unmap undo whatever was done on the map. More details can be found in
> >> > > >>> > patch #6.
> >> > > >>> >
> >> > > >>> > Maybe it would be better to squash patch #6 with this one so it would
> >> > > >>> > be less confusing. What is your advice?
> >> > > >>>
> >> > > >>> So am I right in understanding that without patch #6, this driver does
> >> > > >>> exactly nothing? If so, it has been a waste of review time.
> >> > > >>>
> >> > > >>> Please split patch #6 so that this driver does something useful
> >> > > >>> for Linux, without any of the PRU interrupt routing stuff. I want
> >> > > >>> to see a Linux-only driver that works and doesn't rely on any other
> >> > > >>> exotic feature.
> >> > > >>>
> >> > > >>
> >> > > >> Patch #6 provides PRU specific 2-level routing setup. This step is
> >> > > >> required and it is part of the entire patch-set. Theoretically routing
> >> > > >> setup could be done by other platform driver (not irq one) or e.g. by
> >> > > >> PRU firmware. In such case this driver would be functional without
> >> > > >> patch #6 but I do not think it would be proper.
> >> > > >
> >> > > > Then this whole driver is non-functional until the last patch that
> >> > > > comes with the PRU-specific "value-add".
> >> > >
> >> > > It is all moot actually and the interrupts work only when the PRU
> >> > > remoteproc/clients have invoked the irq_create_fwspec_mapping()
> >> > > for all of the desired system events. It does not make much difference
> >> > > if it was a separate patch or squashed in, patch #6 is a replacement for
> >> > > the previous logic, and since it was complex, it was done in a separate
> >> > > patch to better explain the usage (same reason on v1 and v2 as
> >> > > well).
> >>
> >> It may make no difference to you, but it does for me, as I'm the lucky
> >> idiot reviewing this code. So I am going to say it again: please keep
> >> anything that only exists for the PRU subsystem benefit out of the
> >> initial patches.
> >>
> >> I want to see something that works for Linux, and only for Linux. Once
> >> we have that working, we'll see to add more stuff. But stop throwing
> >> the PRU business into the early patches, as all you are achieving is
> >> to delay the whole thing.
> >>
> >> > >
> >> > > >
> >> > > > [...]
> >> > > >
> >> > > >> I am open to any suggestion if there is a better way of handling
> >> > > >> 2-level routing. I will also appreciate if you could elaborate about
> >> > > >> issues that you see with patch #6.
> >> > > >
> >> > > > The two level routing has to be part of this (or another) irqchip
> >> > > > driver (specially given that it appears to me like another set of
> >> > > > crossbar). There should only be a *single* binding for all interrupts,
> >> > > > including those targeting the PRU (you seem to have two).
> >> > > >
> >> > >
> >> > > Yeah, there hasn't been a clean way of doing this. Our previous attempt
> >> > > was to do this through custom exported functions so that the PRU
> >> > > remoteproc driver can set these up correctly, but that was shot down and
> >> > > this is the direction we are pointed to.
> >> > >
> >> > > We do want to leverage the "interrupts" property in the PRU user nodes
> >> > > instead of inventing our own paradigm through a non-irqchip driver, and
> >> > > at the same time, be able to configure this at the run time only when
> >> > > that PRU driver is running, and remove the mappings once that driver is
> >> > > removed allowing another PRU application/driver. We treat PRUs as an
> >> > > exclusive resource, so everything needs to go along with an appropriate
> >> > > client user.
> >> >
> >> > I will just add an explanation about interrupt binding. So actually
> >> > there is one dt-binding defined in yaml (interrupt-cells = 1). The
> >> > reason why you see xlate allowing to proceed with 1 or 3 parameters is
> >> > because linux can change the PRU firmware at run-time (thorough linux
> >> > remoteproc framework) and different firmware may require different
> >> > kinds of interrupt mapping. Therefore during firmware load, the new
> >> > mapping is created through irq_create_fwspec_mapping() and in this
> >> > case 3 parameters are passed: system event, channel and host irq.
> >> > Similarly the mapping is disposed during remoteproc stop by invoking
> >> > irq_dispose_mapping. This allows to create new mapping, in the same
> >> > way, for next firmware loaded through Linux remote-proc at runtime
> >> > (depending on the needs of new remoteproc firmware).
> >> >
> >> > On the other hand dt-bindings defines interrupt-cells = 1, so when the
> >> > interrupt is registered the xlate function (proceed with 1 parameter)
> >> > checks if this event already has valid mapping - if yes we are fine,
> >> > if not we return -EINVAL.
> >>
> >> It means that interrupts declared in DT get their two-level routing
> >> via the kernel driver, while PRU interrupts get their routing via some
> >> external blob that Linux is not in control of?
> >
> > Actually with the current approach all two-level routing goes through
> > this linux driver. The interrupts that should be routed to PRU are
> > described in remoteproc firmware resource table [1] and it is under
> > Linux remoteproc driver control. In general, the resource table
> > contains system resources that the remote processor requires before it
> > should be powered on. We treat the interrupt mapping (described in the
> > resource table, which is a dedicated elf section defined in [1]) as
> > one of system resources that linux has to provide before we power on
> > the PRU core. Therefore the remoteproce driver will parse the resource
> > table and trigger irq_create_fwspec_mapping() after validating
> > resource table content.
>
> Validating the resource table says nothing of a potential conflict
> with previous configured interrupts.

Yes, that's why we introduced the logic in pruss_intc_irq_domain_xlate
and pruss_intc_map triggered by irq_create_fwspec_mapping, which will
check potential conflicts with previous configured interrupts. I
understand that you do not like how it is done but I do not know how
to do it in a different way so it will cover all caveats, please see
below.

>
> >
> > [1] https://www.kernel.org/doc/Documentation/remoteproc.txt (Binary
> > Firmware Structure)
> >
> >>
> >> If so, this looks broken. What if you get a resource allocation
> >> conflict because the kernel and the blob are stepping into each
> >> other's toes? Why should an end-point client decide on the routing of
> >> the interrupt?
> >
> > The code in the pruss_intc_map function checks if there are no
> > allocation conflicts: e.g. if the sysevent is already assigned it will
> > throw -EBUSY. Similarly when some channel was already assigned to
> > host_irq and a different assignment is requested it will again throw
> > -EBUSY.
>
> But why should it? The allocation should take place based on constraints
> (source, target, and as you mentioned below, priority). Instead, you
> seem to be relying on static allocation coming from binary blobs,
> standardized or not.
>
> I claim that this static allocation is madness and should be eliminated.
> Instead, the Linux driver should perform the routing based on allocation
> requirements (the above constraints), and only fail if it cannot satisfy
> these constraints.

I am not sure if I understood. The allocation requirements are as
you've described: source (system event), target (host interrupt) and
priority (channel).
E.g.:
- routing system event 3 with priority (chanell) 2 to PRU core 0 will
be described as bellow: (3, 2, 0) (0 corresponds to PRU0)
- routing system event 10 with priority (chanell) 3 to PRU core 1: (10, 3, 1)
- routing system event 15 with priority (5) to MCPU interrupt 0*: (15, 5, 2)
* interrupts 2 through 9 (host_intr0 through host_intr7)

I am not sure but you probably refer to changing it to loosely dynamic
allocation but this will not work for any of them since:
- different system event is just a different sources (e.g. some of
them are tightly coupled with PRUSS industrial ethernet peripheral,
other with PRUSS UART, other are general purpose ones and so on).
- lower number channels have higher priority (10 different channels,
each with different priority).
- host interrupt 0 is for PRU core 0; host interrupt 1 is for PRU core
1; host interrupts 2 through 9 are for main CPU.

So the logic in patch #6 prevents mapping system events if it was
already assigned to a different channel or target (host interrupt) and
only fails if it cannot be satisfied. Moreover I do not see a way to
relax the static description since picking different numbers for each
individual: system event, channel and host interrupt will result with
something unintentional and wrong.

Sorry if I misunderstood you, if so could you please elaborate?

Thank you,
Grzegorz

Re: [PATCHv3 3/6] irqchip/irq-pruss-intc: Add support for shared and invalid interrupts

[Suman Anna]

Hi Marc,

On 7/17/20 6:02 AM, Marc Zyngier wrote:
> On Fri, 10 Jul 2020 21:59:17 +0100,
> Suman Anna <s-anna@ti.com> wrote:
> 
> Hi Suman,
> 
> [...]
> 
>>
>> Hi Marc,
>>
>> On 7/2/20 12:44 PM, Marc Zyngier wrote:
>>> On 2020-07-02 15:17, Grzegorz Jaszczyk wrote:
>>>> From: Suman Anna <s-anna@ti.com>
>>>>
>>>> The PRUSS INTC has a fixed number of output interrupt lines that are
>>>> connected to a number of processors or other PRUSS instances or other
>>>> devices (like DMA) on the SoC. The output interrupt lines 2 through 9
>>>> are usually connected to the main Arm host processor and are referred
>>>> to as host interrupts 0 through 7 from ARM/MPU perspective.
>>>>
>>>> All of these 8 host interrupts are not always exclusively connected
>>>> to the Arm interrupt controller. Some SoCs have some interrupt lines
>>>> not connected to the Arm interrupt controller at all, while a few others
>>>> have the interrupt lines connected to multiple processors in which they
>>>> need to be partitioned as per SoC integration needs. For example, AM437x
>>>> and 66AK2G SoCs have 2 PRUSS instances each and have the host interrupt 5
>>>> connected to the other PRUSS, while AM335x has host interrupt 0 shared
>>>> between MPU and TSC_ADC and host interrupts 6 & 7 shared between MPU and
>>>> a DMA controller.
>>>>
>>>> Add support to the PRUSS INTC driver to allow both these shared and
>>>> invalid interrupts by not returning a failure if any of these interrupts
>>>> are skipped from the corresponding INTC DT node.
>>>
>>> That's not exactly "adding support", is it? It really is "ignore these
>>> interrupts because they are useless from the main CPU's perspective",
>>> right?
>>
>> Correct. We can rephrase this to something like
>> "Add logic to the PRUSS INTC driver to ignore.."
>>
>>>
>>>>
>>>> Signed-off-by: Suman Anna <s-anna@ti.com>
>>>> Signed-off-by: Grzegorz Jaszczyk <grzegorz.jaszczyk@linaro.org>
>>>> ---
>>>> v2->v3:
>>>> - Extra checks for (intc->irqs[i]) in error/remove path was moved from
>>>>    "irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS
>>>>    interrupts" to this patch
>>>> v1->v2:
>>>> - https://patchwork.kernel.org/patch/11069757/
>>>> ---
>>>>   drivers/irqchip/irq-pruss-intc.c | 73
>>>> +++++++++++++++++++++++++++++++++++++---
>>>>   1 file changed, 68 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/drivers/irqchip/irq-pruss-intc.c
>>>> b/drivers/irqchip/irq-pruss-intc.c
>>>> index fb3dda3..49c936f 100644
>>>> --- a/drivers/irqchip/irq-pruss-intc.c
>>>> +++ b/drivers/irqchip/irq-pruss-intc.c
>>>> @@ -65,11 +65,15 @@
>>>>    * @irqs: kernel irq numbers corresponding to PRUSS host interrupts
>>>>    * @base: base virtual address of INTC register space
>>>>    * @domain: irq domain for this interrupt controller
>>>> + * @shared_intr: bit-map denoting if the MPU host interrupt is shared
>>>
>>> nit: bitmap
>>
>> ok
>>
>>>
>>>> + * @invalid_intr: bit-map denoting if host interrupt is not
>>>> connected to MPU
>>>>    */
>>>>   struct pruss_intc {
>>>>       unsigned int irqs[MAX_NUM_HOST_IRQS];
>>>>       void __iomem *base;
>>>>       struct irq_domain *domain;
>>>> +    u16 shared_intr;
>>>> +    u16 invalid_intr;
>>>
>>> Please represent bitmaps as an unsigned long.
>>
>> ok. We have atmost 8 interrupts coming in, but agree on the change
>> since we are using the BIT() macro below.
>>
>>>
>>>>   };
>>>>
>>>>   static inline u32 pruss_intc_read_reg(struct pruss_intc *intc,
>>>> unsigned int reg)
>>>> @@ -222,7 +226,8 @@ static int pruss_intc_probe(struct
>>>> platform_device *pdev)
>>>>           "host_intr4", "host_intr5", "host_intr6", "host_intr7", };
>>>>       struct device *dev = &pdev->dev;
>>>>       struct pruss_intc *intc;
>>>> -    int i, irq;
>>>> +    int i, irq, count;
>>>> +    u8 temp_intr[MAX_NUM_HOST_IRQS] = { 0 };
>>>>
>>>>       intc = devm_kzalloc(dev, sizeof(*intc), GFP_KERNEL);
>>>>       if (!intc)
>>>> @@ -235,6 +240,52 @@ static int pruss_intc_probe(struct
>>>> platform_device *pdev)
>>>>           return PTR_ERR(intc->base);
>>>>       }
>>>>
>>>> +    count = of_property_read_variable_u8_array(dev->of_node,
>>>> +                           "ti,irqs-reserved",
>>>> +                           temp_intr, 0,
>>>> +                           MAX_NUM_HOST_IRQS);
>>>> +    /*
>>>> +     * The irqs-reserved is used only for some SoC's therefore not
>>>> having
>>>> +     * this property is still valid
>>>> +     */
>>>> +    if (count == -EINVAL)
>>>> +        count = 0;
>>>> +    if (count < 0)
>>>> +        return count;
>>>> +
>>>> +    for (i = 0; i < count; i++) {
>>>> +        if (temp_intr[i] >= MAX_NUM_HOST_IRQS) {
>>>> +            dev_warn(dev, "ignoring invalid reserved irq %d\n",
>>>> +                 temp_intr[i]);
>>>> +            continue;
>>>> +        }
>>>> +
>>>> +        intc->invalid_intr |= BIT(temp_intr[i]);
>>>> +    }
>>>> +
>>>> +    count = of_property_read_variable_u8_array(dev->of_node,
>>>> +                           "ti,irqs-shared",
>>>> +                           temp_intr, 0,
>>>> +                           MAX_NUM_HOST_IRQS);
>>>> +    /*
>>>> +     * The irqs-shared is used only for some SoC's therefore not having
>>>> +     * this property is still valid
>>>> +     */
>>>> +    if (count == -EINVAL)
>>>> +        count = 0;
>>>> +    if (count < 0)
>>>> +        return count;
>>>> +
>>>> +    for (i = 0; i < count; i++) {
>>>> +        if (temp_intr[i] >= MAX_NUM_HOST_IRQS) {
>>>> +            dev_warn(dev, "ignoring invalid shared irq %d\n",
>>>> +                 temp_intr[i]);
>>>> +            continue;
>>>> +        }
>>>> +
>>>> +        intc->shared_intr |= BIT(temp_intr[i]);
>>>> +    }
>>>> +
>>>
>>> You probably want to move this in a separate function, since you populate a
>>> common structure.
>>>
>>>>       pruss_intc_init(intc);
>>>>
>>>>       /* always 64 events */
>>>> @@ -244,8 +295,14 @@ static int pruss_intc_probe(struct
>>>> platform_device *pdev)
>>>>           return -ENOMEM;
>>>>
>>>>       for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
>>>> +        if (intc->invalid_intr & BIT(i))
>>>> +            continue;
>>>> +
>>>>           irq = platform_get_irq_byname(pdev, irq_names[i]);
>>>>           if (irq <= 0) {
>>>> +            if (intc->shared_intr & BIT(i))
>>>> +                continue;
>>>
>>> I don't really understand why you are treating these "shared" interrupts
>>> differently from the invalid ones. In all cases, they shouldn't be used.
>>
>> The behavior is the same in how we handle it, but the difference is
>> that an "invalid" one is never even connected to the ARM interrupt
>> controller, while the "shared" one is a choice. So, unless this
>> interrupt is being used/handled by a different processor/entity, you
>> would not see this skipped from the dts node.
> 
> And I'm saying that all that matters is that you are discarding these
> interrupts. Whether they are flagged invalid or shared, they are not
> available to Linux. So the difference in handling is pointless and
> only makes it harder to understand what you are doing.

The primary reason for using two properties and this logic was to 
accurately describe the h/w and usage of these in the DT bindings to 
distinguish the "never connected" vs the "optionally can be skipped" 
interrupts rather than go by how these are handled in the driver. I feel 
we will loose this description and make it confusing for SoC product 
integration developers.

Greg is planning to consolidate these for the next version. It would 
have been nice if we could have retained them.

regards
Suman

Re: [PATCHv3 3/6] irqchip/irq-pruss-intc: Add support for shared and invalid interrupts

[Marc Zyngier]

On 2020-07-25 16:57, Suman Anna wrote:

Suman,

> Hi Marc,

[...]

>>>>> @@ -244,8 +295,14 @@ static int pruss_intc_probe(struct
>>>>> platform_device *pdev)
>>>>>           return -ENOMEM;
>>>>> 
>>>>>       for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
>>>>> +        if (intc->invalid_intr & BIT(i))
>>>>> +            continue;
>>>>> +
>>>>>           irq = platform_get_irq_byname(pdev, irq_names[i]);
>>>>>           if (irq <= 0) {
>>>>> +            if (intc->shared_intr & BIT(i))
>>>>> +                continue;
>>>> 
>>>> I don't really understand why you are treating these "shared" 
>>>> interrupts
>>>> differently from the invalid ones. In all cases, they shouldn't be 
>>>> used.
>>> 
>>> The behavior is the same in how we handle it, but the difference is
>>> that an "invalid" one is never even connected to the ARM interrupt
>>> controller, while the "shared" one is a choice. So, unless this
>>> interrupt is being used/handled by a different processor/entity, you
>>> would not see this skipped from the dts node.
>> 
>> And I'm saying that all that matters is that you are discarding these
>> interrupts. Whether they are flagged invalid or shared, they are not
>> available to Linux. So the difference in handling is pointless and
>> only makes it harder to understand what you are doing.
> 
> The primary reason for using two properties and this logic was to
> accurately describe the h/w and usage of these in the DT bindings to
> distinguish the "never connected" vs the "optionally can be skipped"
> interrupts rather than go by how these are handled in the driver. I
> feel we will loose this description and make it confusing for SoC
> product integration developers.

This logic makes zero difference to Linux, and I do not see what
you gain by having two code paths with separate list of unusable
interrupts. And why on Earth would a "Soc product integration
developer" have any business to mess with this driver code?
They should very much stay away from it and deal with their
precious value add.

If you want two properties or even twenty, go for it, and have fun.
Just don't make this driver even more unreadable than it already is.
Merge all these interrupts in *one* list of unusable interrupts,
and be done with it.

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCHv3 3/6] irqchip/irq-pruss-intc: Add support for shared and invalid interrupts

[Suman Anna]

Hi Marc,

On 7/25/20 11:27 AM, Marc Zyngier wrote:
> On 2020-07-25 16:57, Suman Anna wrote:
> 
> Suman,
> 
>> Hi Marc,
> 
> [...]
> 
>>>>>> @@ -244,8 +295,14 @@ static int pruss_intc_probe(struct
>>>>>> platform_device *pdev)
>>>>>>           return -ENOMEM;
>>>>>>
>>>>>>       for (i = 0; i < MAX_NUM_HOST_IRQS; i++) {
>>>>>> +        if (intc->invalid_intr & BIT(i))
>>>>>> +            continue;
>>>>>> +
>>>>>>           irq = platform_get_irq_byname(pdev, irq_names[i]);
>>>>>>           if (irq <= 0) {
>>>>>> +            if (intc->shared_intr & BIT(i))
>>>>>> +                continue;
>>>>>
>>>>> I don't really understand why you are treating these "shared" 
>>>>> interrupts
>>>>> differently from the invalid ones. In all cases, they shouldn't be 
>>>>> used.
>>>>
>>>> The behavior is the same in how we handle it, but the difference is
>>>> that an "invalid" one is never even connected to the ARM interrupt
>>>> controller, while the "shared" one is a choice. So, unless this
>>>> interrupt is being used/handled by a different processor/entity, you
>>>> would not see this skipped from the dts node.
>>>
>>> And I'm saying that all that matters is that you are discarding these
>>> interrupts. Whether they are flagged invalid or shared, they are not
>>> available to Linux. So the difference in handling is pointless and
>>> only makes it harder to understand what you are doing.
>>
>> The primary reason for using two properties and this logic was to
>> accurately describe the h/w and usage of these in the DT bindings to
>> distinguish the "never connected" vs the "optionally can be skipped"
>> interrupts rather than go by how these are handled in the driver. I
>> feel we will loose this description and make it confusing for SoC
>> product integration developers.
> 
> This logic makes zero difference to Linux, and I do not see what
> you gain by having two code paths with separate list of unusable
> interrupts. 

OK, I understand your stance on this.

And why on Earth would a "Soc product integration
> developer" have any business to mess with this driver code?
> They should very much stay away from it and deal with their
> precious value add.

It really depends on how they put together the system and exercise the 
PRUs and the number of processors interacting with them. We have had 
customers put together usecases where both the ARM core running Linux 
and a remote processor like an M4 or R5 talk to the PRU at the same 
time, or even inter PRUSS instances. They would have to adjust the DT in 
their board dts files in general.

> 
> If you want two properties or even twenty, go for it, and have fun.
> Just don't make this driver even more unreadable than it already is.
> Merge all these interrupts in *one* list of unusable interrupts,
> and be done with it.

Yes, we are merging this for the next version.

regards
Suman