[PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[Anup Patel]

This patchset provides a new RISC-V Local Interrupt Controller Driver
for managing per-CPU local interrupts. The overall approach is inspired
from the way per-CPU local interrupts are handled by Linux ARM64 and
ARM GICv3 driver.

Few advantages of this new driver over previous one are:
1. All local interrupts are registered as per-CPU interrupts
2. The RISC-V timer driver can register timer interrupt handler
   using kernel irq subsystem without relying on arch/riscv to
   explicitly call it's interrupt handler
3. The KVM RISC-V can use this driver to implement interrupt
   handler for per-HART guest external interrupt defined by
   the RISC-V H-Extension
4. In future, we can develop drivers for devices with per-HART
   interrupts without changing arch code or this driver (example,
   CLINT timer driver for RISC-V M-mode kernel)

With this patchset, output of "cat /proc/interrupts" looks as follows:
           CPU0       CPU1       CPU2       CPU3       
  2:        379          0          0          0  SiFive PLIC  10  ttyS0
  3:        591          0          0          0  SiFive PLIC   8  virtio0
  5:       5079      10821       8435      12984  RISC-V INTC   5  riscv-timer
IPI0:      2045       2537        891        870  Rescheduling interrupts
IPI1:         9        269         91        168  Function call interrupts
IPI2:         0          0          0          0  CPU stop interrupts

The patchset is based up Linux-5.7-rc6 and can be found at riscv_intc_v5
branch of: https://github.com/avpatel/linux.git

This series is tested on:
 1. QEMU RV64 virt machine using Linux RISC-V S-mode
 2. QEMU RV32 virt machine using Linux RISC-V S-mode
 3. QEMU RV64 virt machine using Linux RISC-V M-mode (i.e. NoMMU)

Changes since v4:
 - Rebased to Linux-5.7-rc6 and multi-PLIC improvement patches
 - Added separate patch to force select RISCV_INTC for CONFIG_RISCV
 - Fixed the driver for Linux RISC-V NoMMU

Changes since v3:
 - Rebased to Linux-5.6-rc5 and Atish's PLIC patches
 - Added separate patch to rename and move plic_find_hart_id()
   to arch directory
 - Use riscv_of_parent_hartid() in riscv_intc_init() instead of
   atomic counter

Changes since v2:
 - Dropped PATCH2 since it was merged long-time back
 - Rebased series from Linux-4.19-rc2 to Linux-5.6-rc2

Changes since v1:
 - Removed changes related to puggable IPI triggering
 - Separate patch for self-contained IPI handling routine
 - Removed patch for GENERIC_IRQ kconfig options
 - Added patch to remove do_IRQ() function
 - Rebased upon Atish's SMP patches

Anup Patel (6):
  RISC-V: self-contained IPI handling routine
  RISC-V: Rename and move plic_find_hart_id() to arch directory
  irqchip: RISC-V per-HART local interrupt controller driver
  clocksource/drivers/timer-riscv: Use per-CPU timer interrupt
  RISC-V: Remove do_IRQ() function
  RISC-V: Force select RISCV_INTC for CONFIG_RISCV

 arch/riscv/Kconfig                     |   2 +
 arch/riscv/include/asm/irq.h           |   5 -
 arch/riscv/include/asm/processor.h     |   1 +
 arch/riscv/include/asm/smp.h           |   3 +
 arch/riscv/kernel/cpu.c                |  16 +++
 arch/riscv/kernel/entry.S              |   4 +-
 arch/riscv/kernel/irq.c                |  33 +-----
 arch/riscv/kernel/smp.c                |  11 +-
 arch/riscv/kernel/traps.c              |   2 -
 drivers/clocksource/timer-riscv.c      |  30 ++++-
 drivers/irqchip/Kconfig                |  13 +++
 drivers/irqchip/Makefile               |   1 +
 drivers/irqchip/irq-riscv-intc.c       | 150 +++++++++++++++++++++++++
 drivers/irqchip/irq-sifive-plic.c      |  52 +++++----
 include/linux/cpuhotplug.h             |   1 +
 include/linux/irqchip/irq-riscv-intc.h |  20 ++++
 16 files changed, 280 insertions(+), 64 deletions(-)
 create mode 100644 drivers/irqchip/irq-riscv-intc.c
 create mode 100644 include/linux/irqchip/irq-riscv-intc.h

-- 
2.25.1

[PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[Anup Patel]

This patchset provides a new RISC-V Local Interrupt Controller Driver
for managing per-CPU local interrupts. The overall approach is inspired
from the way per-CPU local interrupts are handled by Linux ARM64 and
ARM GICv3 driver.

Few advantages of this new driver over previous one are:
1. All local interrupts are registered as per-CPU interrupts
2. The RISC-V timer driver can register timer interrupt handler
   using kernel irq subsystem without relying on arch/riscv to
   explicitly call it's interrupt handler
3. The KVM RISC-V can use this driver to implement interrupt
   handler for per-HART guest external interrupt defined by
   the RISC-V H-Extension
4. In future, we can develop drivers for devices with per-HART
   interrupts without changing arch code or this driver (example,
   CLINT timer driver for RISC-V M-mode kernel)

With this patchset, output of "cat /proc/interrupts" looks as follows:
           CPU0       CPU1       CPU2       CPU3       
  2:        379          0          0          0  SiFive PLIC  10  ttyS0
  3:        591          0          0          0  SiFive PLIC   8  virtio0
  5:       5079      10821       8435      12984  RISC-V INTC   5  riscv-timer
IPI0:      2045       2537        891        870  Rescheduling interrupts
IPI1:         9        269         91        168  Function call interrupts
IPI2:         0          0          0          0  CPU stop interrupts

The patchset is based up Linux-5.7-rc6 and can be found at riscv_intc_v5
branch of: https://github.com/avpatel/linux.git

This series is tested on:
 1. QEMU RV64 virt machine using Linux RISC-V S-mode
 2. QEMU RV32 virt machine using Linux RISC-V S-mode
 3. QEMU RV64 virt machine using Linux RISC-V M-mode (i.e. NoMMU)

Changes since v4:
 - Rebased to Linux-5.7-rc6 and multi-PLIC improvement patches
 - Added separate patch to force select RISCV_INTC for CONFIG_RISCV
 - Fixed the driver for Linux RISC-V NoMMU

Changes since v3:
 - Rebased to Linux-5.6-rc5 and Atish's PLIC patches
 - Added separate patch to rename and move plic_find_hart_id()
   to arch directory
 - Use riscv_of_parent_hartid() in riscv_intc_init() instead of
   atomic counter

Changes since v2:
 - Dropped PATCH2 since it was merged long-time back
 - Rebased series from Linux-4.19-rc2 to Linux-5.6-rc2

Changes since v1:
 - Removed changes related to puggable IPI triggering
 - Separate patch for self-contained IPI handling routine
 - Removed patch for GENERIC_IRQ kconfig options
 - Added patch to remove do_IRQ() function
 - Rebased upon Atish's SMP patches

Anup Patel (6):
  RISC-V: self-contained IPI handling routine
  RISC-V: Rename and move plic_find_hart_id() to arch directory
  irqchip: RISC-V per-HART local interrupt controller driver
  clocksource/drivers/timer-riscv: Use per-CPU timer interrupt
  RISC-V: Remove do_IRQ() function
  RISC-V: Force select RISCV_INTC for CONFIG_RISCV

 arch/riscv/Kconfig                     |   2 +
 arch/riscv/include/asm/irq.h           |   5 -
 arch/riscv/include/asm/processor.h     |   1 +
 arch/riscv/include/asm/smp.h           |   3 +
 arch/riscv/kernel/cpu.c                |  16 +++
 arch/riscv/kernel/entry.S              |   4 +-
 arch/riscv/kernel/irq.c                |  33 +-----
 arch/riscv/kernel/smp.c                |  11 +-
 arch/riscv/kernel/traps.c              |   2 -
 drivers/clocksource/timer-riscv.c      |  30 ++++-
 drivers/irqchip/Kconfig                |  13 +++
 drivers/irqchip/Makefile               |   1 +
 drivers/irqchip/irq-riscv-intc.c       | 150 +++++++++++++++++++++++++
 drivers/irqchip/irq-sifive-plic.c      |  52 +++++----
 include/linux/cpuhotplug.h             |   1 +
 include/linux/irqchip/irq-riscv-intc.h |  20 ++++
 16 files changed, 280 insertions(+), 64 deletions(-)
 create mode 100644 drivers/irqchip/irq-riscv-intc.c
 create mode 100644 include/linux/irqchip/irq-riscv-intc.h

-- 
2.25.1

[PATCH v5 1/6] RISC-V: self-contained IPI handling routine

[Anup Patel]

Currently, the IPI handling routine riscv_software_interrupt() does
not take any argument and also does not perform irq_enter()/irq_exit().

This patch makes IPI handling routine more self-contained by:
1. Passing "pt_regs *" argument
2. Explicitly doing irq_enter()/irq_exit()
3. Explicitly save/restore "pt_regs *" using set_irq_regs()

With above changes, IPI handling routine does not depend on caller
function to perform irq_enter()/irq_exit() and save/restore of
"pt_regs *" hence its more self-contained. This also enables us
to call IPI handling routine from IRQCHIP drivers.

Signed-off-by: Anup Patel <anup.patel@wdc.com>
---
 arch/riscv/include/asm/irq.h |  1 -
 arch/riscv/include/asm/smp.h |  3 +++
 arch/riscv/kernel/irq.c      | 16 ++++++++++------
 arch/riscv/kernel/smp.c      | 11 +++++++++--
 4 files changed, 22 insertions(+), 9 deletions(-)

diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
index 6e1b0e0325eb..0183e15ace66 100644
--- a/arch/riscv/include/asm/irq.h
+++ b/arch/riscv/include/asm/irq.h
@@ -13,7 +13,6 @@
 #define NR_IRQS         0
 
 void riscv_timer_interrupt(void);
-void riscv_software_interrupt(void);
 
 #include <asm-generic/irq.h>
 
diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h
index f4c7cfda6b7f..40bb1c15a731 100644
--- a/arch/riscv/include/asm/smp.h
+++ b/arch/riscv/include/asm/smp.h
@@ -28,6 +28,9 @@ void show_ipi_stats(struct seq_file *p, int prec);
 /* SMP initialization hook for setup_arch */
 void __init setup_smp(void);
 
+/* Called from C code, this handles an IPI. */
+void handle_IPI(struct pt_regs *regs);
+
 /* Hook for the generic smp_call_function_many() routine. */
 void arch_send_call_function_ipi_mask(struct cpumask *mask);
 
diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
index 345c4f2eba13..bb0bfcd537e7 100644
--- a/arch/riscv/kernel/irq.c
+++ b/arch/riscv/kernel/irq.c
@@ -19,12 +19,15 @@ int arch_show_interrupts(struct seq_file *p, int prec)
 
 asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
 {
-	struct pt_regs *old_regs = set_irq_regs(regs);
+	struct pt_regs *old_regs;
 
-	irq_enter();
 	switch (regs->cause & ~CAUSE_IRQ_FLAG) {
 	case RV_IRQ_TIMER:
+		old_regs = set_irq_regs(regs);
+		irq_enter();
 		riscv_timer_interrupt();
+		irq_exit();
+		set_irq_regs(old_regs);
 		break;
 #ifdef CONFIG_SMP
 	case RV_IRQ_SOFT:
@@ -32,19 +35,20 @@ asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
 		 * We only use software interrupts to pass IPIs, so if a non-SMP
 		 * system gets one, then we don't know what to do.
 		 */
-		riscv_software_interrupt();
+		handle_IPI(regs);
 		break;
 #endif
 	case RV_IRQ_EXT:
+		old_regs = set_irq_regs(regs);
+		irq_enter();
 		handle_arch_irq(regs);
+		irq_exit();
+		set_irq_regs(old_regs);
 		break;
 	default:
 		pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
 		BUG();
 	}
-	irq_exit();
-
-	set_irq_regs(old_regs);
 }
 
 void __init init_IRQ(void)
diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
index a65a8fa0c22d..b1d4f452f843 100644
--- a/arch/riscv/kernel/smp.c
+++ b/arch/riscv/kernel/smp.c
@@ -123,11 +123,14 @@ static inline void clear_ipi(void)
 		clint_clear_ipi(cpuid_to_hartid_map(smp_processor_id()));
 }
 
-void riscv_software_interrupt(void)
+void handle_IPI(struct pt_regs *regs)
 {
+	struct pt_regs *old_regs = set_irq_regs(regs);
 	unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits;
 	unsigned long *stats = ipi_data[smp_processor_id()].stats;
 
+	irq_enter();
+
 	clear_ipi();
 
 	while (true) {
@@ -138,7 +141,7 @@ void riscv_software_interrupt(void)
 
 		ops = xchg(pending_ipis, 0);
 		if (ops == 0)
-			return;
+			goto done;
 
 		if (ops & (1 << IPI_RESCHEDULE)) {
 			stats[IPI_RESCHEDULE]++;
@@ -160,6 +163,10 @@ void riscv_software_interrupt(void)
 		/* Order data access and bit testing. */
 		mb();
 	}
+
+done:
+	irq_exit();
+	set_irq_regs(old_regs);
 }
 
 static const char * const ipi_names[] = {
-- 
2.25.1

[PATCH v5 1/6] RISC-V: self-contained IPI handling routine

[Anup Patel]

Currently, the IPI handling routine riscv_software_interrupt() does
not take any argument and also does not perform irq_enter()/irq_exit().

This patch makes IPI handling routine more self-contained by:
1. Passing "pt_regs *" argument
2. Explicitly doing irq_enter()/irq_exit()
3. Explicitly save/restore "pt_regs *" using set_irq_regs()

With above changes, IPI handling routine does not depend on caller
function to perform irq_enter()/irq_exit() and save/restore of
"pt_regs *" hence its more self-contained. This also enables us
to call IPI handling routine from IRQCHIP drivers.

Signed-off-by: Anup Patel <anup.patel@wdc.com>
---
 arch/riscv/include/asm/irq.h |  1 -
 arch/riscv/include/asm/smp.h |  3 +++
 arch/riscv/kernel/irq.c      | 16 ++++++++++------
 arch/riscv/kernel/smp.c      | 11 +++++++++--
 4 files changed, 22 insertions(+), 9 deletions(-)

diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
index 6e1b0e0325eb..0183e15ace66 100644
--- a/arch/riscv/include/asm/irq.h
+++ b/arch/riscv/include/asm/irq.h
@@ -13,7 +13,6 @@
 #define NR_IRQS         0
 
 void riscv_timer_interrupt(void);
-void riscv_software_interrupt(void);
 
 #include <asm-generic/irq.h>
 
diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h
index f4c7cfda6b7f..40bb1c15a731 100644
--- a/arch/riscv/include/asm/smp.h
+++ b/arch/riscv/include/asm/smp.h
@@ -28,6 +28,9 @@ void show_ipi_stats(struct seq_file *p, int prec);
 /* SMP initialization hook for setup_arch */
 void __init setup_smp(void);
 
+/* Called from C code, this handles an IPI. */
+void handle_IPI(struct pt_regs *regs);
+
 /* Hook for the generic smp_call_function_many() routine. */
 void arch_send_call_function_ipi_mask(struct cpumask *mask);
 
diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
index 345c4f2eba13..bb0bfcd537e7 100644
--- a/arch/riscv/kernel/irq.c
+++ b/arch/riscv/kernel/irq.c
@@ -19,12 +19,15 @@ int arch_show_interrupts(struct seq_file *p, int prec)
 
 asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
 {
-	struct pt_regs *old_regs = set_irq_regs(regs);
+	struct pt_regs *old_regs;
 
-	irq_enter();
 	switch (regs->cause & ~CAUSE_IRQ_FLAG) {
 	case RV_IRQ_TIMER:
+		old_regs = set_irq_regs(regs);
+		irq_enter();
 		riscv_timer_interrupt();
+		irq_exit();
+		set_irq_regs(old_regs);
 		break;
 #ifdef CONFIG_SMP
 	case RV_IRQ_SOFT:
@@ -32,19 +35,20 @@ asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
 		 * We only use software interrupts to pass IPIs, so if a non-SMP
 		 * system gets one, then we don't know what to do.
 		 */
-		riscv_software_interrupt();
+		handle_IPI(regs);
 		break;
 #endif
 	case RV_IRQ_EXT:
+		old_regs = set_irq_regs(regs);
+		irq_enter();
 		handle_arch_irq(regs);
+		irq_exit();
+		set_irq_regs(old_regs);
 		break;
 	default:
 		pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
 		BUG();
 	}
-	irq_exit();
-
-	set_irq_regs(old_regs);
 }
 
 void __init init_IRQ(void)
diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
index a65a8fa0c22d..b1d4f452f843 100644
--- a/arch/riscv/kernel/smp.c
+++ b/arch/riscv/kernel/smp.c
@@ -123,11 +123,14 @@ static inline void clear_ipi(void)
 		clint_clear_ipi(cpuid_to_hartid_map(smp_processor_id()));
 }
 
-void riscv_software_interrupt(void)
+void handle_IPI(struct pt_regs *regs)
 {
+	struct pt_regs *old_regs = set_irq_regs(regs);
 	unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits;
 	unsigned long *stats = ipi_data[smp_processor_id()].stats;
 
+	irq_enter();
+
 	clear_ipi();
 
 	while (true) {
@@ -138,7 +141,7 @@ void riscv_software_interrupt(void)
 
 		ops = xchg(pending_ipis, 0);
 		if (ops == 0)
-			return;
+			goto done;
 
 		if (ops & (1 << IPI_RESCHEDULE)) {
 			stats[IPI_RESCHEDULE]++;
@@ -160,6 +163,10 @@ void riscv_software_interrupt(void)
 		/* Order data access and bit testing. */
 		mb();
 	}
+
+done:
+	irq_exit();
+	set_irq_regs(old_regs);
 }
 
 static const char * const ipi_names[] = {
-- 
2.25.1

Re: [PATCH v5 1/6] RISC-V: self-contained IPI handling routine

[Atish Patra]

On Thu, May 21, 2020 at 6:34 AM Anup Patel <anup.patel@wdc.com> wrote:
>
> Currently, the IPI handling routine riscv_software_interrupt() does
> not take any argument and also does not perform irq_enter()/irq_exit().
>
> This patch makes IPI handling routine more self-contained by:
> 1. Passing "pt_regs *" argument
> 2. Explicitly doing irq_enter()/irq_exit()
> 3. Explicitly save/restore "pt_regs *" using set_irq_regs()
>
> With above changes, IPI handling routine does not depend on caller
> function to perform irq_enter()/irq_exit() and save/restore of
> "pt_regs *" hence its more self-contained. This also enables us
> to call IPI handling routine from IRQCHIP drivers.
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/include/asm/irq.h |  1 -
>  arch/riscv/include/asm/smp.h |  3 +++
>  arch/riscv/kernel/irq.c      | 16 ++++++++++------
>  arch/riscv/kernel/smp.c      | 11 +++++++++--
>  4 files changed, 22 insertions(+), 9 deletions(-)
>
> diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
> index 6e1b0e0325eb..0183e15ace66 100644
> --- a/arch/riscv/include/asm/irq.h
> +++ b/arch/riscv/include/asm/irq.h
> @@ -13,7 +13,6 @@
>  #define NR_IRQS         0
>
>  void riscv_timer_interrupt(void);
> -void riscv_software_interrupt(void);
>
>  #include <asm-generic/irq.h>
>
> diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h
> index f4c7cfda6b7f..40bb1c15a731 100644
> --- a/arch/riscv/include/asm/smp.h
> +++ b/arch/riscv/include/asm/smp.h
> @@ -28,6 +28,9 @@ void show_ipi_stats(struct seq_file *p, int prec);
>  /* SMP initialization hook for setup_arch */
>  void __init setup_smp(void);
>
> +/* Called from C code, this handles an IPI. */
> +void handle_IPI(struct pt_regs *regs);
> +
>  /* Hook for the generic smp_call_function_many() routine. */
>  void arch_send_call_function_ipi_mask(struct cpumask *mask);
>
> diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> index 345c4f2eba13..bb0bfcd537e7 100644
> --- a/arch/riscv/kernel/irq.c
> +++ b/arch/riscv/kernel/irq.c
> @@ -19,12 +19,15 @@ int arch_show_interrupts(struct seq_file *p, int prec)
>
>  asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
>  {
> -       struct pt_regs *old_regs = set_irq_regs(regs);
> +       struct pt_regs *old_regs;
>
> -       irq_enter();
>         switch (regs->cause & ~CAUSE_IRQ_FLAG) {
>         case RV_IRQ_TIMER:
> +               old_regs = set_irq_regs(regs);
> +               irq_enter();
>                 riscv_timer_interrupt();
> +               irq_exit();
> +               set_irq_regs(old_regs);
>                 break;
>  #ifdef CONFIG_SMP
>         case RV_IRQ_SOFT:
> @@ -32,19 +35,20 @@ asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
>                  * We only use software interrupts to pass IPIs, so if a non-SMP
>                  * system gets one, then we don't know what to do.
>                  */
> -               riscv_software_interrupt();
> +               handle_IPI(regs);
>                 break;
>  #endif
>         case RV_IRQ_EXT:
> +               old_regs = set_irq_regs(regs);
> +               irq_enter();
>                 handle_arch_irq(regs);
> +               irq_exit();
> +               set_irq_regs(old_regs);
>                 break;
>         default:
>                 pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
>                 BUG();
>         }
> -       irq_exit();
> -
> -       set_irq_regs(old_regs);
>  }
>
>  void __init init_IRQ(void)
> diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
> index a65a8fa0c22d..b1d4f452f843 100644
> --- a/arch/riscv/kernel/smp.c
> +++ b/arch/riscv/kernel/smp.c
> @@ -123,11 +123,14 @@ static inline void clear_ipi(void)
>                 clint_clear_ipi(cpuid_to_hartid_map(smp_processor_id()));
>  }
>
> -void riscv_software_interrupt(void)
> +void handle_IPI(struct pt_regs *regs)
>  {
> +       struct pt_regs *old_regs = set_irq_regs(regs);
>         unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits;
>         unsigned long *stats = ipi_data[smp_processor_id()].stats;
>
> +       irq_enter();
> +
>         clear_ipi();
>
>         while (true) {
> @@ -138,7 +141,7 @@ void riscv_software_interrupt(void)
>
>                 ops = xchg(pending_ipis, 0);
>                 if (ops == 0)
> -                       return;
> +                       goto done;
>
>                 if (ops & (1 << IPI_RESCHEDULE)) {
>                         stats[IPI_RESCHEDULE]++;
> @@ -160,6 +163,10 @@ void riscv_software_interrupt(void)
>                 /* Order data access and bit testing. */
>                 mb();
>         }
> +
> +done:
> +       irq_exit();
> +       set_irq_regs(old_regs);
>  }
>
>  static const char * const ipi_names[] = {
> --
> 2.25.1
>
>

Reviewed-by: Atish Patra <atish.patra@wdc.com>

-- 
Regards,
Atish

Re: [PATCH v5 1/6] RISC-V: self-contained IPI handling routine

[Atish Patra]

On Thu, May 21, 2020 at 6:34 AM Anup Patel <anup.patel@wdc.com> wrote:
>
> Currently, the IPI handling routine riscv_software_interrupt() does
> not take any argument and also does not perform irq_enter()/irq_exit().
>
> This patch makes IPI handling routine more self-contained by:
> 1. Passing "pt_regs *" argument
> 2. Explicitly doing irq_enter()/irq_exit()
> 3. Explicitly save/restore "pt_regs *" using set_irq_regs()
>
> With above changes, IPI handling routine does not depend on caller
> function to perform irq_enter()/irq_exit() and save/restore of
> "pt_regs *" hence its more self-contained. This also enables us
> to call IPI handling routine from IRQCHIP drivers.
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/include/asm/irq.h |  1 -
>  arch/riscv/include/asm/smp.h |  3 +++
>  arch/riscv/kernel/irq.c      | 16 ++++++++++------
>  arch/riscv/kernel/smp.c      | 11 +++++++++--
>  4 files changed, 22 insertions(+), 9 deletions(-)
>
> diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
> index 6e1b0e0325eb..0183e15ace66 100644
> --- a/arch/riscv/include/asm/irq.h
> +++ b/arch/riscv/include/asm/irq.h
> @@ -13,7 +13,6 @@
>  #define NR_IRQS         0
>
>  void riscv_timer_interrupt(void);
> -void riscv_software_interrupt(void);
>
>  #include <asm-generic/irq.h>
>
> diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h
> index f4c7cfda6b7f..40bb1c15a731 100644
> --- a/arch/riscv/include/asm/smp.h
> +++ b/arch/riscv/include/asm/smp.h
> @@ -28,6 +28,9 @@ void show_ipi_stats(struct seq_file *p, int prec);
>  /* SMP initialization hook for setup_arch */
>  void __init setup_smp(void);
>
> +/* Called from C code, this handles an IPI. */
> +void handle_IPI(struct pt_regs *regs);
> +
>  /* Hook for the generic smp_call_function_many() routine. */
>  void arch_send_call_function_ipi_mask(struct cpumask *mask);
>
> diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> index 345c4f2eba13..bb0bfcd537e7 100644
> --- a/arch/riscv/kernel/irq.c
> +++ b/arch/riscv/kernel/irq.c
> @@ -19,12 +19,15 @@ int arch_show_interrupts(struct seq_file *p, int prec)
>
>  asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
>  {
> -       struct pt_regs *old_regs = set_irq_regs(regs);
> +       struct pt_regs *old_regs;
>
> -       irq_enter();
>         switch (regs->cause & ~CAUSE_IRQ_FLAG) {
>         case RV_IRQ_TIMER:
> +               old_regs = set_irq_regs(regs);
> +               irq_enter();
>                 riscv_timer_interrupt();
> +               irq_exit();
> +               set_irq_regs(old_regs);
>                 break;
>  #ifdef CONFIG_SMP
>         case RV_IRQ_SOFT:
> @@ -32,19 +35,20 @@ asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
>                  * We only use software interrupts to pass IPIs, so if a non-SMP
>                  * system gets one, then we don't know what to do.
>                  */
> -               riscv_software_interrupt();
> +               handle_IPI(regs);
>                 break;
>  #endif
>         case RV_IRQ_EXT:
> +               old_regs = set_irq_regs(regs);
> +               irq_enter();
>                 handle_arch_irq(regs);
> +               irq_exit();
> +               set_irq_regs(old_regs);
>                 break;
>         default:
>                 pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
>                 BUG();
>         }
> -       irq_exit();
> -
> -       set_irq_regs(old_regs);
>  }
>
>  void __init init_IRQ(void)
> diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
> index a65a8fa0c22d..b1d4f452f843 100644
> --- a/arch/riscv/kernel/smp.c
> +++ b/arch/riscv/kernel/smp.c
> @@ -123,11 +123,14 @@ static inline void clear_ipi(void)
>                 clint_clear_ipi(cpuid_to_hartid_map(smp_processor_id()));
>  }
>
> -void riscv_software_interrupt(void)
> +void handle_IPI(struct pt_regs *regs)
>  {
> +       struct pt_regs *old_regs = set_irq_regs(regs);
>         unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits;
>         unsigned long *stats = ipi_data[smp_processor_id()].stats;
>
> +       irq_enter();
> +
>         clear_ipi();
>
>         while (true) {
> @@ -138,7 +141,7 @@ void riscv_software_interrupt(void)
>
>                 ops = xchg(pending_ipis, 0);
>                 if (ops == 0)
> -                       return;
> +                       goto done;
>
>                 if (ops & (1 << IPI_RESCHEDULE)) {
>                         stats[IPI_RESCHEDULE]++;
> @@ -160,6 +163,10 @@ void riscv_software_interrupt(void)
>                 /* Order data access and bit testing. */
>                 mb();
>         }
> +
> +done:
> +       irq_exit();
> +       set_irq_regs(old_regs);
>  }
>
>  static const char * const ipi_names[] = {
> --
> 2.25.1
>
>

Reviewed-by: Atish Patra <atish.patra@wdc.com>

-- 
Regards,
Atish

Re: [PATCH v5 1/6] RISC-V: self-contained IPI handling routine

[Palmer Dabbelt]

On Thu, 21 May 2020 06:32:56 PDT (-0700), Anup Patel wrote:
> Currently, the IPI handling routine riscv_software_interrupt() does
> not take any argument and also does not perform irq_enter()/irq_exit().
>
> This patch makes IPI handling routine more self-contained by:
> 1. Passing "pt_regs *" argument
> 2. Explicitly doing irq_enter()/irq_exit()
> 3. Explicitly save/restore "pt_regs *" using set_irq_regs()
>
> With above changes, IPI handling routine does not depend on caller
> function to perform irq_enter()/irq_exit() and save/restore of
> "pt_regs *" hence its more self-contained. This also enables us
> to call IPI handling routine from IRQCHIP drivers.
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/include/asm/irq.h |  1 -
>  arch/riscv/include/asm/smp.h |  3 +++
>  arch/riscv/kernel/irq.c      | 16 ++++++++++------
>  arch/riscv/kernel/smp.c      | 11 +++++++++--
>  4 files changed, 22 insertions(+), 9 deletions(-)
>
> diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
> index 6e1b0e0325eb..0183e15ace66 100644
> --- a/arch/riscv/include/asm/irq.h
> +++ b/arch/riscv/include/asm/irq.h
> @@ -13,7 +13,6 @@
>  #define NR_IRQS         0
>
>  void riscv_timer_interrupt(void);
> -void riscv_software_interrupt(void);
>
>  #include <asm-generic/irq.h>
>
> diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h
> index f4c7cfda6b7f..40bb1c15a731 100644
> --- a/arch/riscv/include/asm/smp.h
> +++ b/arch/riscv/include/asm/smp.h
> @@ -28,6 +28,9 @@ void show_ipi_stats(struct seq_file *p, int prec);
>  /* SMP initialization hook for setup_arch */
>  void __init setup_smp(void);
>
> +/* Called from C code, this handles an IPI. */
> +void handle_IPI(struct pt_regs *regs);
> +
>  /* Hook for the generic smp_call_function_many() routine. */
>  void arch_send_call_function_ipi_mask(struct cpumask *mask);
>
> diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> index 345c4f2eba13..bb0bfcd537e7 100644
> --- a/arch/riscv/kernel/irq.c
> +++ b/arch/riscv/kernel/irq.c
> @@ -19,12 +19,15 @@ int arch_show_interrupts(struct seq_file *p, int prec)
>
>  asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
>  {
> -	struct pt_regs *old_regs = set_irq_regs(regs);
> +	struct pt_regs *old_regs;
>
> -	irq_enter();
>  	switch (regs->cause & ~CAUSE_IRQ_FLAG) {
>  	case RV_IRQ_TIMER:
> +		old_regs = set_irq_regs(regs);
> +		irq_enter();
>  		riscv_timer_interrupt();
> +		irq_exit();
> +		set_irq_regs(old_regs);
>  		break;
>  #ifdef CONFIG_SMP
>  	case RV_IRQ_SOFT:
> @@ -32,19 +35,20 @@ asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
>  		 * We only use software interrupts to pass IPIs, so if a non-SMP
>  		 * system gets one, then we don't know what to do.
>  		 */
> -		riscv_software_interrupt();
> +		handle_IPI(regs);
>  		break;
>  #endif
>  	case RV_IRQ_EXT:
> +		old_regs = set_irq_regs(regs);
> +		irq_enter();
>  		handle_arch_irq(regs);
> +		irq_exit();
> +		set_irq_regs(old_regs);
>  		break;
>  	default:
>  		pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
>  		BUG();
>  	}
> -	irq_exit();
> -
> -	set_irq_regs(old_regs);
>  }
>
>  void __init init_IRQ(void)
> diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
> index a65a8fa0c22d..b1d4f452f843 100644
> --- a/arch/riscv/kernel/smp.c
> +++ b/arch/riscv/kernel/smp.c
> @@ -123,11 +123,14 @@ static inline void clear_ipi(void)
>  		clint_clear_ipi(cpuid_to_hartid_map(smp_processor_id()));
>  }
>
> -void riscv_software_interrupt(void)
> +void handle_IPI(struct pt_regs *regs)
>  {
> +	struct pt_regs *old_regs = set_irq_regs(regs);
>  	unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits;
>  	unsigned long *stats = ipi_data[smp_processor_id()].stats;
>
> +	irq_enter();
> +
>  	clear_ipi();
>
>  	while (true) {
> @@ -138,7 +141,7 @@ void riscv_software_interrupt(void)
>
>  		ops = xchg(pending_ipis, 0);
>  		if (ops == 0)
> -			return;
> +			goto done;
>
>  		if (ops & (1 << IPI_RESCHEDULE)) {
>  			stats[IPI_RESCHEDULE]++;
> @@ -160,6 +163,10 @@ void riscv_software_interrupt(void)
>  		/* Order data access and bit testing. */
>  		mb();
>  	}
> +
> +done:
> +	irq_exit();
> +	set_irq_regs(old_regs);
>  }
>
>  static const char * const ipi_names[] = {

I guess we still have to figure out which tree these are going in through, but
I'm going to start acking them as I review them just so it all stays together.

Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com>
Acked-by: Palmer Dabbelt <palmerdabbelt@google.com>

Re: [PATCH v5 1/6] RISC-V: self-contained IPI handling routine

[Palmer Dabbelt]

On Thu, 21 May 2020 06:32:56 PDT (-0700), Anup Patel wrote:
> Currently, the IPI handling routine riscv_software_interrupt() does
> not take any argument and also does not perform irq_enter()/irq_exit().
>
> This patch makes IPI handling routine more self-contained by:
> 1. Passing "pt_regs *" argument
> 2. Explicitly doing irq_enter()/irq_exit()
> 3. Explicitly save/restore "pt_regs *" using set_irq_regs()
>
> With above changes, IPI handling routine does not depend on caller
> function to perform irq_enter()/irq_exit() and save/restore of
> "pt_regs *" hence its more self-contained. This also enables us
> to call IPI handling routine from IRQCHIP drivers.
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/include/asm/irq.h |  1 -
>  arch/riscv/include/asm/smp.h |  3 +++
>  arch/riscv/kernel/irq.c      | 16 ++++++++++------
>  arch/riscv/kernel/smp.c      | 11 +++++++++--
>  4 files changed, 22 insertions(+), 9 deletions(-)
>
> diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
> index 6e1b0e0325eb..0183e15ace66 100644
> --- a/arch/riscv/include/asm/irq.h
> +++ b/arch/riscv/include/asm/irq.h
> @@ -13,7 +13,6 @@
>  #define NR_IRQS         0
>
>  void riscv_timer_interrupt(void);
> -void riscv_software_interrupt(void);
>
>  #include <asm-generic/irq.h>
>
> diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h
> index f4c7cfda6b7f..40bb1c15a731 100644
> --- a/arch/riscv/include/asm/smp.h
> +++ b/arch/riscv/include/asm/smp.h
> @@ -28,6 +28,9 @@ void show_ipi_stats(struct seq_file *p, int prec);
>  /* SMP initialization hook for setup_arch */
>  void __init setup_smp(void);
>
> +/* Called from C code, this handles an IPI. */
> +void handle_IPI(struct pt_regs *regs);
> +
>  /* Hook for the generic smp_call_function_many() routine. */
>  void arch_send_call_function_ipi_mask(struct cpumask *mask);
>
> diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> index 345c4f2eba13..bb0bfcd537e7 100644
> --- a/arch/riscv/kernel/irq.c
> +++ b/arch/riscv/kernel/irq.c
> @@ -19,12 +19,15 @@ int arch_show_interrupts(struct seq_file *p, int prec)
>
>  asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
>  {
> -	struct pt_regs *old_regs = set_irq_regs(regs);
> +	struct pt_regs *old_regs;
>
> -	irq_enter();
>  	switch (regs->cause & ~CAUSE_IRQ_FLAG) {
>  	case RV_IRQ_TIMER:
> +		old_regs = set_irq_regs(regs);
> +		irq_enter();
>  		riscv_timer_interrupt();
> +		irq_exit();
> +		set_irq_regs(old_regs);
>  		break;
>  #ifdef CONFIG_SMP
>  	case RV_IRQ_SOFT:
> @@ -32,19 +35,20 @@ asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
>  		 * We only use software interrupts to pass IPIs, so if a non-SMP
>  		 * system gets one, then we don't know what to do.
>  		 */
> -		riscv_software_interrupt();
> +		handle_IPI(regs);
>  		break;
>  #endif
>  	case RV_IRQ_EXT:
> +		old_regs = set_irq_regs(regs);
> +		irq_enter();
>  		handle_arch_irq(regs);
> +		irq_exit();
> +		set_irq_regs(old_regs);
>  		break;
>  	default:
>  		pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
>  		BUG();
>  	}
> -	irq_exit();
> -
> -	set_irq_regs(old_regs);
>  }
>
>  void __init init_IRQ(void)
> diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
> index a65a8fa0c22d..b1d4f452f843 100644
> --- a/arch/riscv/kernel/smp.c
> +++ b/arch/riscv/kernel/smp.c
> @@ -123,11 +123,14 @@ static inline void clear_ipi(void)
>  		clint_clear_ipi(cpuid_to_hartid_map(smp_processor_id()));
>  }
>
> -void riscv_software_interrupt(void)
> +void handle_IPI(struct pt_regs *regs)
>  {
> +	struct pt_regs *old_regs = set_irq_regs(regs);
>  	unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits;
>  	unsigned long *stats = ipi_data[smp_processor_id()].stats;
>
> +	irq_enter();
> +
>  	clear_ipi();
>
>  	while (true) {
> @@ -138,7 +141,7 @@ void riscv_software_interrupt(void)
>
>  		ops = xchg(pending_ipis, 0);
>  		if (ops == 0)
> -			return;
> +			goto done;
>
>  		if (ops & (1 << IPI_RESCHEDULE)) {
>  			stats[IPI_RESCHEDULE]++;
> @@ -160,6 +163,10 @@ void riscv_software_interrupt(void)
>  		/* Order data access and bit testing. */
>  		mb();
>  	}
> +
> +done:
> +	irq_exit();
> +	set_irq_regs(old_regs);
>  }
>
>  static const char * const ipi_names[] = {

I guess we still have to figure out which tree these are going in through, but
I'm going to start acking them as I review them just so it all stays together.

Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com>
Acked-by: Palmer Dabbelt <palmerdabbelt@google.com>

[PATCH v5 2/6] RISC-V: Rename and move plic_find_hart_id() to arch directory

[Anup Patel]

The plic_find_hart_id() can be useful to other interrupt controller
drivers (such as RISC-V local interrupt driver) so we rename this
function to riscv_of_parent_hartid() and place it in arch directory
along with riscv_of_processor_hartid().

Signed-off-by: Anup Patel <anup.patel@wdc.com>
---
 arch/riscv/include/asm/processor.h |  1 +
 arch/riscv/kernel/cpu.c            | 16 ++++++++++++++++
 drivers/irqchip/irq-sifive-plic.c  | 16 +---------------
 3 files changed, 18 insertions(+), 15 deletions(-)

diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index 3ddb798264f1..b1efd840003c 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -75,6 +75,7 @@ static inline void wait_for_interrupt(void)
 
 struct device_node;
 int riscv_of_processor_hartid(struct device_node *node);
+int riscv_of_parent_hartid(struct device_node *node);
 
 extern void riscv_fill_hwcap(void);
 
diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c
index 40a3c442ac5f..6d59e6906fdd 100644
--- a/arch/riscv/kernel/cpu.c
+++ b/arch/riscv/kernel/cpu.c
@@ -44,6 +44,22 @@ int riscv_of_processor_hartid(struct device_node *node)
 	return hart;
 }
 
+/*
+ * Find hart ID of the CPU DT node under which given DT node falls.
+ *
+ * To achieve this, we walk up the DT tree until we find an active
+ * RISC-V core (HART) node and extract the cpuid from it.
+ */
+int riscv_of_parent_hartid(struct device_node *node)
+{
+	for (; node; node = node->parent) {
+		if (of_device_is_compatible(node, "riscv"))
+			return riscv_of_processor_hartid(node);
+	}
+
+	return -1;
+}
+
 #ifdef CONFIG_PROC_FS
 
 static void print_isa(struct seq_file *f, const char *isa)
diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
index d9c53f85a68e..16d31d114c30 100644
--- a/drivers/irqchip/irq-sifive-plic.c
+++ b/drivers/irqchip/irq-sifive-plic.c
@@ -240,20 +240,6 @@ static void plic_handle_irq(struct pt_regs *regs)
 	csr_set(CSR_IE, IE_EIE);
 }
 
-/*
- * Walk up the DT tree until we find an active RISC-V core (HART) node and
- * extract the cpuid from it.
- */
-static int plic_find_hart_id(struct device_node *node)
-{
-	for (; node; node = node->parent) {
-		if (of_device_is_compatible(node, "riscv"))
-			return riscv_of_processor_hartid(node);
-	}
-
-	return -1;
-}
-
 static void plic_set_threshold(struct plic_handler *handler, u32 threshold)
 {
 	/* priority must be > threshold to trigger an interrupt */
@@ -330,7 +316,7 @@ static int __init plic_init(struct device_node *node,
 		if (parent.args[0] != RV_IRQ_EXT)
 			continue;
 
-		hartid = plic_find_hart_id(parent.np);
+		hartid = riscv_of_parent_hartid(parent.np);
 		if (hartid < 0) {
 			pr_warn("failed to parse hart ID for context %d.\n", i);
 			continue;
-- 
2.25.1

[PATCH v5 2/6] RISC-V: Rename and move plic_find_hart_id() to arch directory

[Anup Patel]

The plic_find_hart_id() can be useful to other interrupt controller
drivers (such as RISC-V local interrupt driver) so we rename this
function to riscv_of_parent_hartid() and place it in arch directory
along with riscv_of_processor_hartid().

Signed-off-by: Anup Patel <anup.patel@wdc.com>
---
 arch/riscv/include/asm/processor.h |  1 +
 arch/riscv/kernel/cpu.c            | 16 ++++++++++++++++
 drivers/irqchip/irq-sifive-plic.c  | 16 +---------------
 3 files changed, 18 insertions(+), 15 deletions(-)

diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index 3ddb798264f1..b1efd840003c 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -75,6 +75,7 @@ static inline void wait_for_interrupt(void)
 
 struct device_node;
 int riscv_of_processor_hartid(struct device_node *node);
+int riscv_of_parent_hartid(struct device_node *node);
 
 extern void riscv_fill_hwcap(void);
 
diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c
index 40a3c442ac5f..6d59e6906fdd 100644
--- a/arch/riscv/kernel/cpu.c
+++ b/arch/riscv/kernel/cpu.c
@@ -44,6 +44,22 @@ int riscv_of_processor_hartid(struct device_node *node)
 	return hart;
 }
 
+/*
+ * Find hart ID of the CPU DT node under which given DT node falls.
+ *
+ * To achieve this, we walk up the DT tree until we find an active
+ * RISC-V core (HART) node and extract the cpuid from it.
+ */
+int riscv_of_parent_hartid(struct device_node *node)
+{
+	for (; node; node = node->parent) {
+		if (of_device_is_compatible(node, "riscv"))
+			return riscv_of_processor_hartid(node);
+	}
+
+	return -1;
+}
+
 #ifdef CONFIG_PROC_FS
 
 static void print_isa(struct seq_file *f, const char *isa)
diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
index d9c53f85a68e..16d31d114c30 100644
--- a/drivers/irqchip/irq-sifive-plic.c
+++ b/drivers/irqchip/irq-sifive-plic.c
@@ -240,20 +240,6 @@ static void plic_handle_irq(struct pt_regs *regs)
 	csr_set(CSR_IE, IE_EIE);
 }
 
-/*
- * Walk up the DT tree until we find an active RISC-V core (HART) node and
- * extract the cpuid from it.
- */
-static int plic_find_hart_id(struct device_node *node)
-{
-	for (; node; node = node->parent) {
-		if (of_device_is_compatible(node, "riscv"))
-			return riscv_of_processor_hartid(node);
-	}
-
-	return -1;
-}
-
 static void plic_set_threshold(struct plic_handler *handler, u32 threshold)
 {
 	/* priority must be > threshold to trigger an interrupt */
@@ -330,7 +316,7 @@ static int __init plic_init(struct device_node *node,
 		if (parent.args[0] != RV_IRQ_EXT)
 			continue;
 
-		hartid = plic_find_hart_id(parent.np);
+		hartid = riscv_of_parent_hartid(parent.np);
 		if (hartid < 0) {
 			pr_warn("failed to parse hart ID for context %d.\n", i);
 			continue;
-- 
2.25.1

Re: [PATCH v5 2/6] RISC-V: Rename and move plic_find_hart_id() to arch directory

[Atish Patra]

On Thu, May 21, 2020 at 6:34 AM Anup Patel <anup.patel@wdc.com> wrote:
>
> The plic_find_hart_id() can be useful to other interrupt controller
> drivers (such as RISC-V local interrupt driver) so we rename this
> function to riscv_of_parent_hartid() and place it in arch directory
> along with riscv_of_processor_hartid().
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/include/asm/processor.h |  1 +
>  arch/riscv/kernel/cpu.c            | 16 ++++++++++++++++
>  drivers/irqchip/irq-sifive-plic.c  | 16 +---------------
>  3 files changed, 18 insertions(+), 15 deletions(-)
>
> diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> index 3ddb798264f1..b1efd840003c 100644
> --- a/arch/riscv/include/asm/processor.h
> +++ b/arch/riscv/include/asm/processor.h
> @@ -75,6 +75,7 @@ static inline void wait_for_interrupt(void)
>
>  struct device_node;
>  int riscv_of_processor_hartid(struct device_node *node);
> +int riscv_of_parent_hartid(struct device_node *node);
>
>  extern void riscv_fill_hwcap(void);
>
> diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c
> index 40a3c442ac5f..6d59e6906fdd 100644
> --- a/arch/riscv/kernel/cpu.c
> +++ b/arch/riscv/kernel/cpu.c
> @@ -44,6 +44,22 @@ int riscv_of_processor_hartid(struct device_node *node)
>         return hart;
>  }
>
> +/*
> + * Find hart ID of the CPU DT node under which given DT node falls.
> + *
> + * To achieve this, we walk up the DT tree until we find an active
> + * RISC-V core (HART) node and extract the cpuid from it.
> + */
> +int riscv_of_parent_hartid(struct device_node *node)
> +{
> +       for (; node; node = node->parent) {
> +               if (of_device_is_compatible(node, "riscv"))
> +                       return riscv_of_processor_hartid(node);
> +       }
> +
> +       return -1;
> +}
> +
>  #ifdef CONFIG_PROC_FS
>
>  static void print_isa(struct seq_file *f, const char *isa)
> diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
> index d9c53f85a68e..16d31d114c30 100644
> --- a/drivers/irqchip/irq-sifive-plic.c
> +++ b/drivers/irqchip/irq-sifive-plic.c
> @@ -240,20 +240,6 @@ static void plic_handle_irq(struct pt_regs *regs)
>         csr_set(CSR_IE, IE_EIE);
>  }
>
> -/*
> - * Walk up the DT tree until we find an active RISC-V core (HART) node and
> - * extract the cpuid from it.
> - */
> -static int plic_find_hart_id(struct device_node *node)
> -{
> -       for (; node; node = node->parent) {
> -               if (of_device_is_compatible(node, "riscv"))
> -                       return riscv_of_processor_hartid(node);
> -       }
> -
> -       return -1;
> -}
> -
>  static void plic_set_threshold(struct plic_handler *handler, u32 threshold)
>  {
>         /* priority must be > threshold to trigger an interrupt */
> @@ -330,7 +316,7 @@ static int __init plic_init(struct device_node *node,
>                 if (parent.args[0] != RV_IRQ_EXT)
>                         continue;
>
> -               hartid = plic_find_hart_id(parent.np);
> +               hartid = riscv_of_parent_hartid(parent.np);
>                 if (hartid < 0) {
>                         pr_warn("failed to parse hart ID for context %d.\n", i);
>                         continue;
> --
> 2.25.1
>
>

Reviewed-by: Atish Patra <atish.patra@wdc.com>

-- 
Regards,
Atish

Re: [PATCH v5 2/6] RISC-V: Rename and move plic_find_hart_id() to arch directory

[Atish Patra]

On Thu, May 21, 2020 at 6:34 AM Anup Patel <anup.patel@wdc.com> wrote:
>
> The plic_find_hart_id() can be useful to other interrupt controller
> drivers (such as RISC-V local interrupt driver) so we rename this
> function to riscv_of_parent_hartid() and place it in arch directory
> along with riscv_of_processor_hartid().
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/include/asm/processor.h |  1 +
>  arch/riscv/kernel/cpu.c            | 16 ++++++++++++++++
>  drivers/irqchip/irq-sifive-plic.c  | 16 +---------------
>  3 files changed, 18 insertions(+), 15 deletions(-)
>
> diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> index 3ddb798264f1..b1efd840003c 100644
> --- a/arch/riscv/include/asm/processor.h
> +++ b/arch/riscv/include/asm/processor.h
> @@ -75,6 +75,7 @@ static inline void wait_for_interrupt(void)
>
>  struct device_node;
>  int riscv_of_processor_hartid(struct device_node *node);
> +int riscv_of_parent_hartid(struct device_node *node);
>
>  extern void riscv_fill_hwcap(void);
>
> diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c
> index 40a3c442ac5f..6d59e6906fdd 100644
> --- a/arch/riscv/kernel/cpu.c
> +++ b/arch/riscv/kernel/cpu.c
> @@ -44,6 +44,22 @@ int riscv_of_processor_hartid(struct device_node *node)
>         return hart;
>  }
>
> +/*
> + * Find hart ID of the CPU DT node under which given DT node falls.
> + *
> + * To achieve this, we walk up the DT tree until we find an active
> + * RISC-V core (HART) node and extract the cpuid from it.
> + */
> +int riscv_of_parent_hartid(struct device_node *node)
> +{
> +       for (; node; node = node->parent) {
> +               if (of_device_is_compatible(node, "riscv"))
> +                       return riscv_of_processor_hartid(node);
> +       }
> +
> +       return -1;
> +}
> +
>  #ifdef CONFIG_PROC_FS
>
>  static void print_isa(struct seq_file *f, const char *isa)
> diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
> index d9c53f85a68e..16d31d114c30 100644
> --- a/drivers/irqchip/irq-sifive-plic.c
> +++ b/drivers/irqchip/irq-sifive-plic.c
> @@ -240,20 +240,6 @@ static void plic_handle_irq(struct pt_regs *regs)
>         csr_set(CSR_IE, IE_EIE);
>  }
>
> -/*
> - * Walk up the DT tree until we find an active RISC-V core (HART) node and
> - * extract the cpuid from it.
> - */
> -static int plic_find_hart_id(struct device_node *node)
> -{
> -       for (; node; node = node->parent) {
> -               if (of_device_is_compatible(node, "riscv"))
> -                       return riscv_of_processor_hartid(node);
> -       }
> -
> -       return -1;
> -}
> -
>  static void plic_set_threshold(struct plic_handler *handler, u32 threshold)
>  {
>         /* priority must be > threshold to trigger an interrupt */
> @@ -330,7 +316,7 @@ static int __init plic_init(struct device_node *node,
>                 if (parent.args[0] != RV_IRQ_EXT)
>                         continue;
>
> -               hartid = plic_find_hart_id(parent.np);
> +               hartid = riscv_of_parent_hartid(parent.np);
>                 if (hartid < 0) {
>                         pr_warn("failed to parse hart ID for context %d.\n", i);
>                         continue;
> --
> 2.25.1
>
>

Reviewed-by: Atish Patra <atish.patra@wdc.com>

-- 
Regards,
Atish

Re: [PATCH v5 2/6] RISC-V: Rename and move plic_find_hart_id() to arch directory

[Palmer Dabbelt]

On Thu, 21 May 2020 06:32:57 PDT (-0700), Anup Patel wrote:
> The plic_find_hart_id() can be useful to other interrupt controller
> drivers (such as RISC-V local interrupt driver) so we rename this
> function to riscv_of_parent_hartid() and place it in arch directory
> along with riscv_of_processor_hartid().
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/include/asm/processor.h |  1 +
>  arch/riscv/kernel/cpu.c            | 16 ++++++++++++++++
>  drivers/irqchip/irq-sifive-plic.c  | 16 +---------------
>  3 files changed, 18 insertions(+), 15 deletions(-)
>
> diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> index 3ddb798264f1..b1efd840003c 100644
> --- a/arch/riscv/include/asm/processor.h
> +++ b/arch/riscv/include/asm/processor.h
> @@ -75,6 +75,7 @@ static inline void wait_for_interrupt(void)
>
>  struct device_node;
>  int riscv_of_processor_hartid(struct device_node *node);
> +int riscv_of_parent_hartid(struct device_node *node);
>
>  extern void riscv_fill_hwcap(void);
>
> diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c
> index 40a3c442ac5f..6d59e6906fdd 100644
> --- a/arch/riscv/kernel/cpu.c
> +++ b/arch/riscv/kernel/cpu.c
> @@ -44,6 +44,22 @@ int riscv_of_processor_hartid(struct device_node *node)
>  	return hart;
>  }
>
> +/*
> + * Find hart ID of the CPU DT node under which given DT node falls.
> + *
> + * To achieve this, we walk up the DT tree until we find an active
> + * RISC-V core (HART) node and extract the cpuid from it.
> + */
> +int riscv_of_parent_hartid(struct device_node *node)
> +{
> +	for (; node; node = node->parent) {
> +		if (of_device_is_compatible(node, "riscv"))
> +			return riscv_of_processor_hartid(node);
> +	}
> +
> +	return -1;
> +}
> +
>  #ifdef CONFIG_PROC_FS
>
>  static void print_isa(struct seq_file *f, const char *isa)
> diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
> index d9c53f85a68e..16d31d114c30 100644
> --- a/drivers/irqchip/irq-sifive-plic.c
> +++ b/drivers/irqchip/irq-sifive-plic.c
> @@ -240,20 +240,6 @@ static void plic_handle_irq(struct pt_regs *regs)
>  	csr_set(CSR_IE, IE_EIE);
>  }
>
> -/*
> - * Walk up the DT tree until we find an active RISC-V core (HART) node and
> - * extract the cpuid from it.
> - */
> -static int plic_find_hart_id(struct device_node *node)
> -{
> -	for (; node; node = node->parent) {
> -		if (of_device_is_compatible(node, "riscv"))
> -			return riscv_of_processor_hartid(node);
> -	}
> -
> -	return -1;
> -}
> -
>  static void plic_set_threshold(struct plic_handler *handler, u32 threshold)
>  {
>  	/* priority must be > threshold to trigger an interrupt */
> @@ -330,7 +316,7 @@ static int __init plic_init(struct device_node *node,
>  		if (parent.args[0] != RV_IRQ_EXT)
>  			continue;
>
> -		hartid = plic_find_hart_id(parent.np);
> +		hartid = riscv_of_parent_hartid(parent.np);
>  		if (hartid < 0) {
>  			pr_warn("failed to parse hart ID for context %d.\n", i);
>  			continue;

Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com>
Acked-by: Palmer Dabbelt <palmerdabbelt@google.com>

Re: [PATCH v5 2/6] RISC-V: Rename and move plic_find_hart_id() to arch directory

[Palmer Dabbelt]

On Thu, 21 May 2020 06:32:57 PDT (-0700), Anup Patel wrote:
> The plic_find_hart_id() can be useful to other interrupt controller
> drivers (such as RISC-V local interrupt driver) so we rename this
> function to riscv_of_parent_hartid() and place it in arch directory
> along with riscv_of_processor_hartid().
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/include/asm/processor.h |  1 +
>  arch/riscv/kernel/cpu.c            | 16 ++++++++++++++++
>  drivers/irqchip/irq-sifive-plic.c  | 16 +---------------
>  3 files changed, 18 insertions(+), 15 deletions(-)
>
> diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
> index 3ddb798264f1..b1efd840003c 100644
> --- a/arch/riscv/include/asm/processor.h
> +++ b/arch/riscv/include/asm/processor.h
> @@ -75,6 +75,7 @@ static inline void wait_for_interrupt(void)
>
>  struct device_node;
>  int riscv_of_processor_hartid(struct device_node *node);
> +int riscv_of_parent_hartid(struct device_node *node);
>
>  extern void riscv_fill_hwcap(void);
>
> diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c
> index 40a3c442ac5f..6d59e6906fdd 100644
> --- a/arch/riscv/kernel/cpu.c
> +++ b/arch/riscv/kernel/cpu.c
> @@ -44,6 +44,22 @@ int riscv_of_processor_hartid(struct device_node *node)
>  	return hart;
>  }
>
> +/*
> + * Find hart ID of the CPU DT node under which given DT node falls.
> + *
> + * To achieve this, we walk up the DT tree until we find an active
> + * RISC-V core (HART) node and extract the cpuid from it.
> + */
> +int riscv_of_parent_hartid(struct device_node *node)
> +{
> +	for (; node; node = node->parent) {
> +		if (of_device_is_compatible(node, "riscv"))
> +			return riscv_of_processor_hartid(node);
> +	}
> +
> +	return -1;
> +}
> +
>  #ifdef CONFIG_PROC_FS
>
>  static void print_isa(struct seq_file *f, const char *isa)
> diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
> index d9c53f85a68e..16d31d114c30 100644
> --- a/drivers/irqchip/irq-sifive-plic.c
> +++ b/drivers/irqchip/irq-sifive-plic.c
> @@ -240,20 +240,6 @@ static void plic_handle_irq(struct pt_regs *regs)
>  	csr_set(CSR_IE, IE_EIE);
>  }
>
> -/*
> - * Walk up the DT tree until we find an active RISC-V core (HART) node and
> - * extract the cpuid from it.
> - */
> -static int plic_find_hart_id(struct device_node *node)
> -{
> -	for (; node; node = node->parent) {
> -		if (of_device_is_compatible(node, "riscv"))
> -			return riscv_of_processor_hartid(node);
> -	}
> -
> -	return -1;
> -}
> -
>  static void plic_set_threshold(struct plic_handler *handler, u32 threshold)
>  {
>  	/* priority must be > threshold to trigger an interrupt */
> @@ -330,7 +316,7 @@ static int __init plic_init(struct device_node *node,
>  		if (parent.args[0] != RV_IRQ_EXT)
>  			continue;
>
> -		hartid = plic_find_hart_id(parent.np);
> +		hartid = riscv_of_parent_hartid(parent.np);
>  		if (hartid < 0) {
>  			pr_warn("failed to parse hart ID for context %d.\n", i);
>  			continue;

Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com>
Acked-by: Palmer Dabbelt <palmerdabbelt@google.com>

[PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Anup Patel]

The RISC-V per-HART local interrupt controller manages software
interrupts, timer interrupts, external interrupts (which are routed
via the platform level interrupt controller) and other per-HART
local interrupts.

This patch adds a driver for the RISC-V local interrupt controller.
It is a major re-write over perviously submitted version.
(Refer, https://www.spinics.net/lists/devicetree/msg241230.html)

Few advantages of this new driver over previous one are:
1. All local interrupts are registered as per-CPU interrupts
2. The RISC-V timer driver can register timer interrupt handler
   using kernel irq subsystem without relying on arch/riscv to
   explicitly call it's interrupt handler
3. The KVM RISC-V can use this driver to implement interrupt
   handler for per-HART guest external interrupt defined by
   the RISC-V H-Extension
4. In future, we can develop drivers for devices with per-HART
   interrupts without changing arch code or this driver (example,
   CLINT timer driver for RISC-V M-mode kernel)

The RISC-V INTC driver is compliant with RISC-V Hart-Level Interrupt
Controller DT bindings located at:
Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
Signed-off-by: Anup Patel <anup.patel@wdc.com>
---
 arch/riscv/Kconfig                     |   1 +
 arch/riscv/include/asm/irq.h           |   2 -
 arch/riscv/kernel/irq.c                |  33 +-----
 arch/riscv/kernel/traps.c              |   2 -
 drivers/irqchip/Kconfig                |  13 ++
 drivers/irqchip/Makefile               |   1 +
 drivers/irqchip/irq-riscv-intc.c       | 158 +++++++++++++++++++++++++
 drivers/irqchip/irq-sifive-plic.c      |  38 +++++-
 include/linux/cpuhotplug.h             |   1 +
 include/linux/irqchip/irq-riscv-intc.h |  20 ++++
 10 files changed, 229 insertions(+), 40 deletions(-)
 create mode 100644 drivers/irqchip/irq-riscv-intc.c
 create mode 100644 include/linux/irqchip/irq-riscv-intc.h

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 90a008e28f7e..822cb0e1a380 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -40,6 +40,7 @@ config RISCV
 	select HAVE_PERF_REGS
 	select HAVE_PERF_USER_STACK_DUMP
 	select HAVE_SYSCALL_TRACEPOINTS
+	select HANDLE_DOMAIN_IRQ
 	select IRQ_DOMAIN
 	select SPARSE_IRQ
 	select SYSCTL_EXCEPTION_TRACE
diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
index 0183e15ace66..a9e5f07a7e9c 100644
--- a/arch/riscv/include/asm/irq.h
+++ b/arch/riscv/include/asm/irq.h
@@ -10,8 +10,6 @@
 #include <linux/interrupt.h>
 #include <linux/linkage.h>
 
-#define NR_IRQS         0
-
 void riscv_timer_interrupt(void);
 
 #include <asm-generic/irq.h>
diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
index bb0bfcd537e7..eb8777642ce6 100644
--- a/arch/riscv/kernel/irq.c
+++ b/arch/riscv/kernel/irq.c
@@ -7,7 +7,6 @@
 
 #include <linux/interrupt.h>
 #include <linux/irqchip.h>
-#include <linux/irqdomain.h>
 #include <linux/seq_file.h>
 #include <asm/smp.h>
 
@@ -19,39 +18,13 @@ int arch_show_interrupts(struct seq_file *p, int prec)
 
 asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
 {
-	struct pt_regs *old_regs;
-
-	switch (regs->cause & ~CAUSE_IRQ_FLAG) {
-	case RV_IRQ_TIMER:
-		old_regs = set_irq_regs(regs);
-		irq_enter();
-		riscv_timer_interrupt();
-		irq_exit();
-		set_irq_regs(old_regs);
-		break;
-#ifdef CONFIG_SMP
-	case RV_IRQ_SOFT:
-		/*
-		 * We only use software interrupts to pass IPIs, so if a non-SMP
-		 * system gets one, then we don't know what to do.
-		 */
-		handle_IPI(regs);
-		break;
-#endif
-	case RV_IRQ_EXT:
-		old_regs = set_irq_regs(regs);
-		irq_enter();
+	if (handle_arch_irq)
 		handle_arch_irq(regs);
-		irq_exit();
-		set_irq_regs(old_regs);
-		break;
-	default:
-		pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
-		BUG();
-	}
 }
 
 void __init init_IRQ(void)
 {
 	irqchip_init();
+	if (!handle_arch_irq)
+		panic("No interrupt controller found.");
 }
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index 7f58fa53033f..f48c76aadbf3 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -178,6 +178,4 @@ void trap_init(void)
 	csr_write(CSR_SCRATCH, 0);
 	/* Set the exception vector address */
 	csr_write(CSR_TVEC, &handle_exception);
-	/* Enable interrupts */
-	csr_write(CSR_IE, IE_SIE);
 }
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index a85aada04a64..95d6137a8117 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -493,6 +493,19 @@ config TI_SCI_INTA_IRQCHIP
 	  If you wish to use interrupt aggregator irq resources managed by the
 	  TI System Controller, say Y here. Otherwise, say N.
 
+config RISCV_INTC
+	bool "RISC-V Local Interrupt Controller"
+	depends on RISCV
+	default y
+	help
+	   This enables support for the per-HART local interrupt controller
+	   found in standard RISC-V systems.  The per-HART local interrupt
+	   controller handles timer interrupts, software interrupts, and
+	   hardware interrupts. Without a per-HART local interrupt controller,
+	   a RISC-V system will be unable to handle any interrupts.
+
+	   If you don't know what to do here, say Y.
+
 config SIFIVE_PLIC
 	bool "SiFive Platform-Level Interrupt Controller"
 	depends on RISCV
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 37bbe39bf909..b8319f045472 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -98,6 +98,7 @@ obj-$(CONFIG_NDS32)			+= irq-ativic32.o
 obj-$(CONFIG_QCOM_PDC)			+= qcom-pdc.o
 obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
 obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
+obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
 obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
 obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
 obj-$(CONFIG_IMX_INTMUX)		+= irq-imx-intmux.o
diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
new file mode 100644
index 000000000000..2f364e6a87f9
--- /dev/null
+++ b/drivers/irqchip/irq-riscv-intc.c
@@ -0,0 +1,158 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012 Regents of the University of California
+ * Copyright (C) 2017-2018 SiFive
+ * Copyright (C) 2020 Western Digital Corporation or its affiliates.
+ */
+
+#define pr_fmt(fmt) "riscv-intc: " fmt
+#include <linux/atomic.h>
+#include <linux/bits.h>
+#include <linux/cpu.h>
+#include <linux/irq.h>
+#include <linux/irqchip.h>
+#include <linux/irqchip/irq-riscv-intc.h>
+#include <linux/irqdomain.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/smp.h>
+
+static struct device_node *intc_domain_node;
+static struct irq_domain *intc_domain;
+
+static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
+{
+	struct pt_regs *old_regs;
+	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
+
+	if (unlikely(cause >= BITS_PER_LONG))
+		panic("unexpected interrupt cause");
+
+	switch (cause) {
+	case RV_IRQ_TIMER:
+		old_regs = set_irq_regs(regs);
+		irq_enter();
+		riscv_timer_interrupt();
+		irq_exit();
+		set_irq_regs(old_regs);
+		break;
+#ifdef CONFIG_SMP
+	case RV_IRQ_SOFT:
+		/*
+		 * We only use software interrupts to pass IPIs, so if a
+		 * non-SMP system gets one, then we don't know what to do.
+		 */
+		handle_IPI(regs);
+		break;
+#endif
+	default:
+		handle_domain_irq(intc_domain, cause, regs);
+		break;
+	}
+}
+
+/*
+ * On RISC-V systems local interrupts are masked or unmasked by writing
+ * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be written
+ * on the local hart, these functions can only be called on the hart that
+ * corresponds to the IRQ chip.
+ */
+
+static void riscv_intc_irq_mask(struct irq_data *d)
+{
+	csr_clear(CSR_IE, 1 << (long)d->hwirq);
+}
+
+static void riscv_intc_irq_unmask(struct irq_data *d)
+{
+	csr_set(CSR_IE, 1 << (long)d->hwirq);
+}
+
+static int riscv_intc_cpu_starting(unsigned int cpu)
+{
+	csr_write(CSR_IE, 1UL << RV_IRQ_SOFT);
+	csr_write(CSR_IP, 0);
+	return 0;
+}
+
+static int riscv_intc_cpu_dying(unsigned int cpu)
+{
+	csr_clear(CSR_IE, 1UL << RV_IRQ_SOFT);
+	return 0;
+}
+
+static struct irq_chip riscv_intc_chip = {
+	.name = "RISC-V INTC",
+	.irq_mask = riscv_intc_irq_mask,
+	.irq_unmask = riscv_intc_irq_unmask,
+};
+
+static int riscv_intc_domain_map(struct irq_domain *d, unsigned int irq,
+				 irq_hw_number_t hwirq)
+{
+	irq_set_percpu_devid(irq);
+	irq_domain_set_info(d, irq, hwirq, &riscv_intc_chip, d->host_data,
+			    handle_percpu_devid_irq, NULL, NULL);
+	irq_set_status_flags(irq, IRQ_NOAUTOEN);
+
+	return 0;
+}
+
+static const struct irq_domain_ops riscv_intc_domain_ops = {
+	.map	= riscv_intc_domain_map,
+	.xlate	= irq_domain_xlate_onecell,
+};
+
+/* Get the OF device node used by INTC irq domain */
+struct device_node *riscv_of_intc_domain_node(void)
+{
+	return intc_domain_node;
+}
+EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);
+
+static int __init riscv_intc_init(struct device_node *node,
+				  struct device_node *parent)
+{
+	int hartid;
+
+	/*
+	 * The DT will have one INTC DT node under each CPU (or HART)
+	 * DT node so riscv_intc_init() function will be called once
+	 * for each INTC DT node. We only need INTC initialization for
+	 * the INTC DT node belonging to boot CPU (or boot HART).
+	 */
+	hartid = riscv_of_parent_hartid(node);
+	if (hartid < 0)
+		return 0;
+	if (riscv_hartid_to_cpuid(hartid) != smp_processor_id())
+		return 0;
+
+	intc_domain = irq_domain_add_linear(node, BITS_PER_LONG,
+					    &riscv_intc_domain_ops, NULL);
+	if (!intc_domain)
+		goto error_add_linear;
+
+	/*
+	 * We save the DT node used for creating irq domain and provide
+	 * it to other drivers using iscv_of_intc_domain_node() function.
+	 */
+	intc_domain_node = of_node_get(node);
+
+	set_handle_irq(&riscv_intc_irq);
+
+	cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_STARTING,
+			  "irqchip/riscv/intc:starting",
+			  riscv_intc_cpu_starting,
+			  riscv_intc_cpu_dying);
+
+	pr_info("%lu local interrupts mapped\n", (long)BITS_PER_LONG);
+
+	return 0;
+
+error_add_linear:
+	pr_warn("unable to add IRQ domain\n");
+	return -ENXIO;
+}
+
+IRQCHIP_DECLARE(riscv, "riscv,cpu-intc", riscv_intc_init);
diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
index 16d31d114c30..4f9b2877aa9d 100644
--- a/drivers/irqchip/irq-sifive-plic.c
+++ b/drivers/irqchip/irq-sifive-plic.c
@@ -9,6 +9,8 @@
 #include <linux/io.h>
 #include <linux/irq.h>
 #include <linux/irqchip.h>
+#include <linux/irqchip/chained_irq.h>
+#include <linux/irqchip/irq-riscv-intc.h>
 #include <linux/irqdomain.h>
 #include <linux/module.h>
 #include <linux/of.h>
@@ -60,6 +62,7 @@
 #define	PLIC_ENABLE_THRESHOLD		0
 
 struct plic_priv {
+	int parent_irq;
 	struct cpumask lmask;
 	struct irq_domain *irqdomain;
 	void __iomem *regs;
@@ -219,15 +222,17 @@ static const struct irq_domain_ops plic_irqdomain_ops = {
  * that source ID back to the same claim register.  This automatically enables
  * and disables the interrupt, so there's nothing else to do.
  */
-static void plic_handle_irq(struct pt_regs *regs)
+static void plic_handle_irq(struct irq_desc *desc)
 {
 	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
+	struct irq_chip *chip = irq_desc_get_chip(desc);
 	void __iomem *claim = handler->hart_base + CONTEXT_CLAIM;
 	irq_hw_number_t hwirq;
 
 	WARN_ON_ONCE(!handler->present);
 
-	csr_clear(CSR_IE, IE_EIE);
+	chained_irq_enter(chip, desc);
+
 	while ((hwirq = readl(claim))) {
 		int irq = irq_find_mapping(handler->priv->irqdomain, hwirq);
 
@@ -237,7 +242,8 @@ static void plic_handle_irq(struct pt_regs *regs)
 		else
 			generic_handle_irq(irq);
 	}
-	csr_set(CSR_IE, IE_EIE);
+
+	chained_irq_exit(chip, desc);
 }
 
 static void plic_set_threshold(struct plic_handler *handler, u32 threshold)
@@ -250,7 +256,8 @@ static int plic_dying_cpu(unsigned int cpu)
 {
 	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
 
-	csr_clear(CSR_IE, IE_EIE);
+	if (handler->priv->parent_irq)
+		disable_percpu_irq(handler->priv->parent_irq);
 	plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
 
 	return 0;
@@ -259,8 +266,10 @@ static int plic_dying_cpu(unsigned int cpu)
 static int plic_starting_cpu(unsigned int cpu)
 {
 	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
+	int pirq = handler->priv->parent_irq;
 
-	csr_set(CSR_IE, IE_EIE);
+	if (pirq)
+		enable_percpu_irq(pirq, irq_get_trigger_type(pirq));
 	plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
 
 	return 0;
@@ -273,6 +282,7 @@ static int __init plic_init(struct device_node *node,
 	u32 nr_irqs;
 	struct plic_priv *priv;
 	struct plic_handler *handler;
+	struct of_phandle_args intc_args;
 
 	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
 	if (!priv)
@@ -354,6 +364,23 @@ static int __init plic_init(struct device_node *node,
 		nr_handlers++;
 	}
 
+	/*
+	 * We can have multiple PLIC instances so setup chained handler
+	 * for all PLIC instances.
+	 */
+	intc_args.np = riscv_of_intc_domain_node();
+	intc_args.args_count = 1;
+	intc_args.args[0] = RV_IRQ_EXT;
+	priv->parent_irq = irq_create_of_mapping(&intc_args);
+	if (priv->parent_irq)
+		irq_set_chained_handler(priv->parent_irq,
+					 plic_handle_irq);
+	else {
+		pr_err("%pOFP: intc mapping failed\n", node);
+		error = -ENODEV;
+		goto out_iounmap;
+	}
+
 	/*
 	 * We can have multiple PLIC instances so setup cpuhp state only
 	 * when context handler for current/boot CPU is present.
@@ -368,7 +395,6 @@ static int __init plic_init(struct device_node *node,
 
 	pr_info("%pOFP: mapped %d interrupts with %d handlers for"
 		" %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
-	set_handle_irq(plic_handle_irq);
 	return 0;
 
 out_iounmap:
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 77d70b633531..57b1f8f777d9 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -102,6 +102,7 @@ enum cpuhp_state {
 	CPUHP_AP_IRQ_ARMADA_XP_STARTING,
 	CPUHP_AP_IRQ_BCM2836_STARTING,
 	CPUHP_AP_IRQ_MIPS_GIC_STARTING,
+	CPUHP_AP_IRQ_RISCV_STARTING,
 	CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
 	CPUHP_AP_ARM_MVEBU_COHERENCY,
 	CPUHP_AP_MICROCODE_LOADER,
diff --git a/include/linux/irqchip/irq-riscv-intc.h b/include/linux/irqchip/irq-riscv-intc.h
new file mode 100644
index 000000000000..b11d38353099
--- /dev/null
+++ b/include/linux/irqchip/irq-riscv-intc.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2020 Western Digital Corporation or its affiliates.
+ */
+
+#ifndef __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
+#define __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
+
+struct device_node;
+
+#ifdef CONFIG_RISCV_INTC
+struct device_node *riscv_of_intc_domain_node(void);
+#else
+static inline struct device_node *riscv_of_intc_domain_node(void)
+{
+	return NULL;
+}
+#endif
+
+#endif
-- 
2.25.1

[PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Anup Patel]

The RISC-V per-HART local interrupt controller manages software
interrupts, timer interrupts, external interrupts (which are routed
via the platform level interrupt controller) and other per-HART
local interrupts.

This patch adds a driver for the RISC-V local interrupt controller.
It is a major re-write over perviously submitted version.
(Refer, https://www.spinics.net/lists/devicetree/msg241230.html)

Few advantages of this new driver over previous one are:
1. All local interrupts are registered as per-CPU interrupts
2. The RISC-V timer driver can register timer interrupt handler
   using kernel irq subsystem without relying on arch/riscv to
   explicitly call it's interrupt handler
3. The KVM RISC-V can use this driver to implement interrupt
   handler for per-HART guest external interrupt defined by
   the RISC-V H-Extension
4. In future, we can develop drivers for devices with per-HART
   interrupts without changing arch code or this driver (example,
   CLINT timer driver for RISC-V M-mode kernel)

The RISC-V INTC driver is compliant with RISC-V Hart-Level Interrupt
Controller DT bindings located at:
Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt

Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
Signed-off-by: Anup Patel <anup.patel@wdc.com>
---
 arch/riscv/Kconfig                     |   1 +
 arch/riscv/include/asm/irq.h           |   2 -
 arch/riscv/kernel/irq.c                |  33 +-----
 arch/riscv/kernel/traps.c              |   2 -
 drivers/irqchip/Kconfig                |  13 ++
 drivers/irqchip/Makefile               |   1 +
 drivers/irqchip/irq-riscv-intc.c       | 158 +++++++++++++++++++++++++
 drivers/irqchip/irq-sifive-plic.c      |  38 +++++-
 include/linux/cpuhotplug.h             |   1 +
 include/linux/irqchip/irq-riscv-intc.h |  20 ++++
 10 files changed, 229 insertions(+), 40 deletions(-)
 create mode 100644 drivers/irqchip/irq-riscv-intc.c
 create mode 100644 include/linux/irqchip/irq-riscv-intc.h

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 90a008e28f7e..822cb0e1a380 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -40,6 +40,7 @@ config RISCV
 	select HAVE_PERF_REGS
 	select HAVE_PERF_USER_STACK_DUMP
 	select HAVE_SYSCALL_TRACEPOINTS
+	select HANDLE_DOMAIN_IRQ
 	select IRQ_DOMAIN
 	select SPARSE_IRQ
 	select SYSCTL_EXCEPTION_TRACE
diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
index 0183e15ace66..a9e5f07a7e9c 100644
--- a/arch/riscv/include/asm/irq.h
+++ b/arch/riscv/include/asm/irq.h
@@ -10,8 +10,6 @@
 #include <linux/interrupt.h>
 #include <linux/linkage.h>
 
-#define NR_IRQS         0
-
 void riscv_timer_interrupt(void);
 
 #include <asm-generic/irq.h>
diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
index bb0bfcd537e7..eb8777642ce6 100644
--- a/arch/riscv/kernel/irq.c
+++ b/arch/riscv/kernel/irq.c
@@ -7,7 +7,6 @@
 
 #include <linux/interrupt.h>
 #include <linux/irqchip.h>
-#include <linux/irqdomain.h>
 #include <linux/seq_file.h>
 #include <asm/smp.h>
 
@@ -19,39 +18,13 @@ int arch_show_interrupts(struct seq_file *p, int prec)
 
 asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
 {
-	struct pt_regs *old_regs;
-
-	switch (regs->cause & ~CAUSE_IRQ_FLAG) {
-	case RV_IRQ_TIMER:
-		old_regs = set_irq_regs(regs);
-		irq_enter();
-		riscv_timer_interrupt();
-		irq_exit();
-		set_irq_regs(old_regs);
-		break;
-#ifdef CONFIG_SMP
-	case RV_IRQ_SOFT:
-		/*
-		 * We only use software interrupts to pass IPIs, so if a non-SMP
-		 * system gets one, then we don't know what to do.
-		 */
-		handle_IPI(regs);
-		break;
-#endif
-	case RV_IRQ_EXT:
-		old_regs = set_irq_regs(regs);
-		irq_enter();
+	if (handle_arch_irq)
 		handle_arch_irq(regs);
-		irq_exit();
-		set_irq_regs(old_regs);
-		break;
-	default:
-		pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
-		BUG();
-	}
 }
 
 void __init init_IRQ(void)
 {
 	irqchip_init();
+	if (!handle_arch_irq)
+		panic("No interrupt controller found.");
 }
diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
index 7f58fa53033f..f48c76aadbf3 100644
--- a/arch/riscv/kernel/traps.c
+++ b/arch/riscv/kernel/traps.c
@@ -178,6 +178,4 @@ void trap_init(void)
 	csr_write(CSR_SCRATCH, 0);
 	/* Set the exception vector address */
 	csr_write(CSR_TVEC, &handle_exception);
-	/* Enable interrupts */
-	csr_write(CSR_IE, IE_SIE);
 }
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index a85aada04a64..95d6137a8117 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -493,6 +493,19 @@ config TI_SCI_INTA_IRQCHIP
 	  If you wish to use interrupt aggregator irq resources managed by the
 	  TI System Controller, say Y here. Otherwise, say N.
 
+config RISCV_INTC
+	bool "RISC-V Local Interrupt Controller"
+	depends on RISCV
+	default y
+	help
+	   This enables support for the per-HART local interrupt controller
+	   found in standard RISC-V systems.  The per-HART local interrupt
+	   controller handles timer interrupts, software interrupts, and
+	   hardware interrupts. Without a per-HART local interrupt controller,
+	   a RISC-V system will be unable to handle any interrupts.
+
+	   If you don't know what to do here, say Y.
+
 config SIFIVE_PLIC
 	bool "SiFive Platform-Level Interrupt Controller"
 	depends on RISCV
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index 37bbe39bf909..b8319f045472 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -98,6 +98,7 @@ obj-$(CONFIG_NDS32)			+= irq-ativic32.o
 obj-$(CONFIG_QCOM_PDC)			+= qcom-pdc.o
 obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
 obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
+obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
 obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
 obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
 obj-$(CONFIG_IMX_INTMUX)		+= irq-imx-intmux.o
diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
new file mode 100644
index 000000000000..2f364e6a87f9
--- /dev/null
+++ b/drivers/irqchip/irq-riscv-intc.c
@@ -0,0 +1,158 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012 Regents of the University of California
+ * Copyright (C) 2017-2018 SiFive
+ * Copyright (C) 2020 Western Digital Corporation or its affiliates.
+ */
+
+#define pr_fmt(fmt) "riscv-intc: " fmt
+#include <linux/atomic.h>
+#include <linux/bits.h>
+#include <linux/cpu.h>
+#include <linux/irq.h>
+#include <linux/irqchip.h>
+#include <linux/irqchip/irq-riscv-intc.h>
+#include <linux/irqdomain.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/smp.h>
+
+static struct device_node *intc_domain_node;
+static struct irq_domain *intc_domain;
+
+static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
+{
+	struct pt_regs *old_regs;
+	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
+
+	if (unlikely(cause >= BITS_PER_LONG))
+		panic("unexpected interrupt cause");
+
+	switch (cause) {
+	case RV_IRQ_TIMER:
+		old_regs = set_irq_regs(regs);
+		irq_enter();
+		riscv_timer_interrupt();
+		irq_exit();
+		set_irq_regs(old_regs);
+		break;
+#ifdef CONFIG_SMP
+	case RV_IRQ_SOFT:
+		/*
+		 * We only use software interrupts to pass IPIs, so if a
+		 * non-SMP system gets one, then we don't know what to do.
+		 */
+		handle_IPI(regs);
+		break;
+#endif
+	default:
+		handle_domain_irq(intc_domain, cause, regs);
+		break;
+	}
+}
+
+/*
+ * On RISC-V systems local interrupts are masked or unmasked by writing
+ * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be written
+ * on the local hart, these functions can only be called on the hart that
+ * corresponds to the IRQ chip.
+ */
+
+static void riscv_intc_irq_mask(struct irq_data *d)
+{
+	csr_clear(CSR_IE, 1 << (long)d->hwirq);
+}
+
+static void riscv_intc_irq_unmask(struct irq_data *d)
+{
+	csr_set(CSR_IE, 1 << (long)d->hwirq);
+}
+
+static int riscv_intc_cpu_starting(unsigned int cpu)
+{
+	csr_write(CSR_IE, 1UL << RV_IRQ_SOFT);
+	csr_write(CSR_IP, 0);
+	return 0;
+}
+
+static int riscv_intc_cpu_dying(unsigned int cpu)
+{
+	csr_clear(CSR_IE, 1UL << RV_IRQ_SOFT);
+	return 0;
+}
+
+static struct irq_chip riscv_intc_chip = {
+	.name = "RISC-V INTC",
+	.irq_mask = riscv_intc_irq_mask,
+	.irq_unmask = riscv_intc_irq_unmask,
+};
+
+static int riscv_intc_domain_map(struct irq_domain *d, unsigned int irq,
+				 irq_hw_number_t hwirq)
+{
+	irq_set_percpu_devid(irq);
+	irq_domain_set_info(d, irq, hwirq, &riscv_intc_chip, d->host_data,
+			    handle_percpu_devid_irq, NULL, NULL);
+	irq_set_status_flags(irq, IRQ_NOAUTOEN);
+
+	return 0;
+}
+
+static const struct irq_domain_ops riscv_intc_domain_ops = {
+	.map	= riscv_intc_domain_map,
+	.xlate	= irq_domain_xlate_onecell,
+};
+
+/* Get the OF device node used by INTC irq domain */
+struct device_node *riscv_of_intc_domain_node(void)
+{
+	return intc_domain_node;
+}
+EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);
+
+static int __init riscv_intc_init(struct device_node *node,
+				  struct device_node *parent)
+{
+	int hartid;
+
+	/*
+	 * The DT will have one INTC DT node under each CPU (or HART)
+	 * DT node so riscv_intc_init() function will be called once
+	 * for each INTC DT node. We only need INTC initialization for
+	 * the INTC DT node belonging to boot CPU (or boot HART).
+	 */
+	hartid = riscv_of_parent_hartid(node);
+	if (hartid < 0)
+		return 0;
+	if (riscv_hartid_to_cpuid(hartid) != smp_processor_id())
+		return 0;
+
+	intc_domain = irq_domain_add_linear(node, BITS_PER_LONG,
+					    &riscv_intc_domain_ops, NULL);
+	if (!intc_domain)
+		goto error_add_linear;
+
+	/*
+	 * We save the DT node used for creating irq domain and provide
+	 * it to other drivers using iscv_of_intc_domain_node() function.
+	 */
+	intc_domain_node = of_node_get(node);
+
+	set_handle_irq(&riscv_intc_irq);
+
+	cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_STARTING,
+			  "irqchip/riscv/intc:starting",
+			  riscv_intc_cpu_starting,
+			  riscv_intc_cpu_dying);
+
+	pr_info("%lu local interrupts mapped\n", (long)BITS_PER_LONG);
+
+	return 0;
+
+error_add_linear:
+	pr_warn("unable to add IRQ domain\n");
+	return -ENXIO;
+}
+
+IRQCHIP_DECLARE(riscv, "riscv,cpu-intc", riscv_intc_init);
diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
index 16d31d114c30..4f9b2877aa9d 100644
--- a/drivers/irqchip/irq-sifive-plic.c
+++ b/drivers/irqchip/irq-sifive-plic.c
@@ -9,6 +9,8 @@
 #include <linux/io.h>
 #include <linux/irq.h>
 #include <linux/irqchip.h>
+#include <linux/irqchip/chained_irq.h>
+#include <linux/irqchip/irq-riscv-intc.h>
 #include <linux/irqdomain.h>
 #include <linux/module.h>
 #include <linux/of.h>
@@ -60,6 +62,7 @@
 #define	PLIC_ENABLE_THRESHOLD		0
 
 struct plic_priv {
+	int parent_irq;
 	struct cpumask lmask;
 	struct irq_domain *irqdomain;
 	void __iomem *regs;
@@ -219,15 +222,17 @@ static const struct irq_domain_ops plic_irqdomain_ops = {
  * that source ID back to the same claim register.  This automatically enables
  * and disables the interrupt, so there's nothing else to do.
  */
-static void plic_handle_irq(struct pt_regs *regs)
+static void plic_handle_irq(struct irq_desc *desc)
 {
 	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
+	struct irq_chip *chip = irq_desc_get_chip(desc);
 	void __iomem *claim = handler->hart_base + CONTEXT_CLAIM;
 	irq_hw_number_t hwirq;
 
 	WARN_ON_ONCE(!handler->present);
 
-	csr_clear(CSR_IE, IE_EIE);
+	chained_irq_enter(chip, desc);
+
 	while ((hwirq = readl(claim))) {
 		int irq = irq_find_mapping(handler->priv->irqdomain, hwirq);
 
@@ -237,7 +242,8 @@ static void plic_handle_irq(struct pt_regs *regs)
 		else
 			generic_handle_irq(irq);
 	}
-	csr_set(CSR_IE, IE_EIE);
+
+	chained_irq_exit(chip, desc);
 }
 
 static void plic_set_threshold(struct plic_handler *handler, u32 threshold)
@@ -250,7 +256,8 @@ static int plic_dying_cpu(unsigned int cpu)
 {
 	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
 
-	csr_clear(CSR_IE, IE_EIE);
+	if (handler->priv->parent_irq)
+		disable_percpu_irq(handler->priv->parent_irq);
 	plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
 
 	return 0;
@@ -259,8 +266,10 @@ static int plic_dying_cpu(unsigned int cpu)
 static int plic_starting_cpu(unsigned int cpu)
 {
 	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
+	int pirq = handler->priv->parent_irq;
 
-	csr_set(CSR_IE, IE_EIE);
+	if (pirq)
+		enable_percpu_irq(pirq, irq_get_trigger_type(pirq));
 	plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
 
 	return 0;
@@ -273,6 +282,7 @@ static int __init plic_init(struct device_node *node,
 	u32 nr_irqs;
 	struct plic_priv *priv;
 	struct plic_handler *handler;
+	struct of_phandle_args intc_args;
 
 	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
 	if (!priv)
@@ -354,6 +364,23 @@ static int __init plic_init(struct device_node *node,
 		nr_handlers++;
 	}
 
+	/*
+	 * We can have multiple PLIC instances so setup chained handler
+	 * for all PLIC instances.
+	 */
+	intc_args.np = riscv_of_intc_domain_node();
+	intc_args.args_count = 1;
+	intc_args.args[0] = RV_IRQ_EXT;
+	priv->parent_irq = irq_create_of_mapping(&intc_args);
+	if (priv->parent_irq)
+		irq_set_chained_handler(priv->parent_irq,
+					 plic_handle_irq);
+	else {
+		pr_err("%pOFP: intc mapping failed\n", node);
+		error = -ENODEV;
+		goto out_iounmap;
+	}
+
 	/*
 	 * We can have multiple PLIC instances so setup cpuhp state only
 	 * when context handler for current/boot CPU is present.
@@ -368,7 +395,6 @@ static int __init plic_init(struct device_node *node,
 
 	pr_info("%pOFP: mapped %d interrupts with %d handlers for"
 		" %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
-	set_handle_irq(plic_handle_irq);
 	return 0;
 
 out_iounmap:
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 77d70b633531..57b1f8f777d9 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -102,6 +102,7 @@ enum cpuhp_state {
 	CPUHP_AP_IRQ_ARMADA_XP_STARTING,
 	CPUHP_AP_IRQ_BCM2836_STARTING,
 	CPUHP_AP_IRQ_MIPS_GIC_STARTING,
+	CPUHP_AP_IRQ_RISCV_STARTING,
 	CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
 	CPUHP_AP_ARM_MVEBU_COHERENCY,
 	CPUHP_AP_MICROCODE_LOADER,
diff --git a/include/linux/irqchip/irq-riscv-intc.h b/include/linux/irqchip/irq-riscv-intc.h
new file mode 100644
index 000000000000..b11d38353099
--- /dev/null
+++ b/include/linux/irqchip/irq-riscv-intc.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2020 Western Digital Corporation or its affiliates.
+ */
+
+#ifndef __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
+#define __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
+
+struct device_node;
+
+#ifdef CONFIG_RISCV_INTC
+struct device_node *riscv_of_intc_domain_node(void);
+#else
+static inline struct device_node *riscv_of_intc_domain_node(void)
+{
+	return NULL;
+}
+#endif
+
+#endif
-- 
2.25.1

Re: [PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Palmer Dabbelt]

On Thu, 21 May 2020 06:32:58 PDT (-0700), Anup Patel wrote:
> The RISC-V per-HART local interrupt controller manages software
> interrupts, timer interrupts, external interrupts (which are routed
> via the platform level interrupt controller) and other per-HART
> local interrupts.
>
> This patch adds a driver for the RISC-V local interrupt controller.
> It is a major re-write over perviously submitted version.
> (Refer, https://www.spinics.net/lists/devicetree/msg241230.html)
>
> Few advantages of this new driver over previous one are:
> 1. All local interrupts are registered as per-CPU interrupts
> 2. The RISC-V timer driver can register timer interrupt handler
>    using kernel irq subsystem without relying on arch/riscv to
>    explicitly call it's interrupt handler
> 3. The KVM RISC-V can use this driver to implement interrupt
>    handler for per-HART guest external interrupt defined by
>    the RISC-V H-Extension
> 4. In future, we can develop drivers for devices with per-HART
>    interrupts without changing arch code or this driver (example,
>    CLINT timer driver for RISC-V M-mode kernel)
>
> The RISC-V INTC driver is compliant with RISC-V Hart-Level Interrupt
> Controller DT bindings located at:
> Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
>
> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/Kconfig                     |   1 +
>  arch/riscv/include/asm/irq.h           |   2 -
>  arch/riscv/kernel/irq.c                |  33 +-----
>  arch/riscv/kernel/traps.c              |   2 -
>  drivers/irqchip/Kconfig                |  13 ++
>  drivers/irqchip/Makefile               |   1 +
>  drivers/irqchip/irq-riscv-intc.c       | 158 +++++++++++++++++++++++++
>  drivers/irqchip/irq-sifive-plic.c      |  38 +++++-
>  include/linux/cpuhotplug.h             |   1 +
>  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
>  10 files changed, 229 insertions(+), 40 deletions(-)
>  create mode 100644 drivers/irqchip/irq-riscv-intc.c
>  create mode 100644 include/linux/irqchip/irq-riscv-intc.h
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 90a008e28f7e..822cb0e1a380 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -40,6 +40,7 @@ config RISCV
>  	select HAVE_PERF_REGS
>  	select HAVE_PERF_USER_STACK_DUMP
>  	select HAVE_SYSCALL_TRACEPOINTS
> +	select HANDLE_DOMAIN_IRQ
>  	select IRQ_DOMAIN
>  	select SPARSE_IRQ
>  	select SYSCTL_EXCEPTION_TRACE
> diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
> index 0183e15ace66..a9e5f07a7e9c 100644
> --- a/arch/riscv/include/asm/irq.h
> +++ b/arch/riscv/include/asm/irq.h
> @@ -10,8 +10,6 @@
>  #include <linux/interrupt.h>
>  #include <linux/linkage.h>
>
> -#define NR_IRQS         0
> -
>  void riscv_timer_interrupt(void);
>
>  #include <asm-generic/irq.h>
> diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> index bb0bfcd537e7..eb8777642ce6 100644
> --- a/arch/riscv/kernel/irq.c
> +++ b/arch/riscv/kernel/irq.c
> @@ -7,7 +7,6 @@
>
>  #include <linux/interrupt.h>
>  #include <linux/irqchip.h>
> -#include <linux/irqdomain.h>
>  #include <linux/seq_file.h>
>  #include <asm/smp.h>
>
> @@ -19,39 +18,13 @@ int arch_show_interrupts(struct seq_file *p, int prec)
>
>  asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
>  {
> -	struct pt_regs *old_regs;
> -
> -	switch (regs->cause & ~CAUSE_IRQ_FLAG) {
> -	case RV_IRQ_TIMER:
> -		old_regs = set_irq_regs(regs);
> -		irq_enter();
> -		riscv_timer_interrupt();
> -		irq_exit();
> -		set_irq_regs(old_regs);
> -		break;
> -#ifdef CONFIG_SMP
> -	case RV_IRQ_SOFT:
> -		/*
> -		 * We only use software interrupts to pass IPIs, so if a non-SMP
> -		 * system gets one, then we don't know what to do.
> -		 */
> -		handle_IPI(regs);
> -		break;
> -#endif
> -	case RV_IRQ_EXT:
> -		old_regs = set_irq_regs(regs);
> -		irq_enter();
> +	if (handle_arch_irq)
>  		handle_arch_irq(regs);
> -		irq_exit();
> -		set_irq_regs(old_regs);
> -		break;
> -	default:
> -		pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
> -		BUG();
> -	}
>  }
>
>  void __init init_IRQ(void)
>  {
>  	irqchip_init();
> +	if (!handle_arch_irq)
> +		panic("No interrupt controller found.");
>  }
> diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> index 7f58fa53033f..f48c76aadbf3 100644
> --- a/arch/riscv/kernel/traps.c
> +++ b/arch/riscv/kernel/traps.c
> @@ -178,6 +178,4 @@ void trap_init(void)
>  	csr_write(CSR_SCRATCH, 0);
>  	/* Set the exception vector address */
>  	csr_write(CSR_TVEC, &handle_exception);
> -	/* Enable interrupts */
> -	csr_write(CSR_IE, IE_SIE);
>  }
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index a85aada04a64..95d6137a8117 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -493,6 +493,19 @@ config TI_SCI_INTA_IRQCHIP
>  	  If you wish to use interrupt aggregator irq resources managed by the
>  	  TI System Controller, say Y here. Otherwise, say N.
>
> +config RISCV_INTC
> +	bool "RISC-V Local Interrupt Controller"
> +	depends on RISCV
> +	default y
> +	help
> +	   This enables support for the per-HART local interrupt controller
> +	   found in standard RISC-V systems.  The per-HART local interrupt
> +	   controller handles timer interrupts, software interrupts, and
> +	   hardware interrupts. Without a per-HART local interrupt controller,
> +	   a RISC-V system will be unable to handle any interrupts.
> +
> +	   If you don't know what to do here, say Y.
> +
>  config SIFIVE_PLIC
>  	bool "SiFive Platform-Level Interrupt Controller"
>  	depends on RISCV
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index 37bbe39bf909..b8319f045472 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -98,6 +98,7 @@ obj-$(CONFIG_NDS32)			+= irq-ativic32.o
>  obj-$(CONFIG_QCOM_PDC)			+= qcom-pdc.o
>  obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
>  obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
> +obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
>  obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
>  obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
>  obj-$(CONFIG_IMX_INTMUX)		+= irq-imx-intmux.o
> diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> new file mode 100644
> index 000000000000..2f364e6a87f9
> --- /dev/null
> +++ b/drivers/irqchip/irq-riscv-intc.c
> @@ -0,0 +1,158 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2012 Regents of the University of California
> + * Copyright (C) 2017-2018 SiFive
> + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> + */
> +
> +#define pr_fmt(fmt) "riscv-intc: " fmt
> +#include <linux/atomic.h>
> +#include <linux/bits.h>
> +#include <linux/cpu.h>
> +#include <linux/irq.h>
> +#include <linux/irqchip.h>
> +#include <linux/irqchip/irq-riscv-intc.h>
> +#include <linux/irqdomain.h>
> +#include <linux/interrupt.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/smp.h>
> +
> +static struct device_node *intc_domain_node;
> +static struct irq_domain *intc_domain;
> +
> +static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
> +{
> +	struct pt_regs *old_regs;
> +	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
> +
> +	if (unlikely(cause >= BITS_PER_LONG))
> +		panic("unexpected interrupt cause");
> +
> +	switch (cause) {
> +	case RV_IRQ_TIMER:
> +		old_regs = set_irq_regs(regs);
> +		irq_enter();
> +		riscv_timer_interrupt();
> +		irq_exit();
> +		set_irq_regs(old_regs);
> +		break;
> +#ifdef CONFIG_SMP
> +	case RV_IRQ_SOFT:
> +		/*
> +		 * We only use software interrupts to pass IPIs, so if a
> +		 * non-SMP system gets one, then we don't know what to do.
> +		 */
> +		handle_IPI(regs);
> +		break;
> +#endif
> +	default:
> +		handle_domain_irq(intc_domain, cause, regs);
> +		break;
> +	}
> +}
> +
> +/*
> + * On RISC-V systems local interrupts are masked or unmasked by writing
> + * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be written
> + * on the local hart, these functions can only be called on the hart that
> + * corresponds to the IRQ chip.
> + */
> +
> +static void riscv_intc_irq_mask(struct irq_data *d)
> +{
> +	csr_clear(CSR_IE, 1 << (long)d->hwirq);
> +}
> +
> +static void riscv_intc_irq_unmask(struct irq_data *d)
> +{
> +	csr_set(CSR_IE, 1 << (long)d->hwirq);
> +}
> +
> +static int riscv_intc_cpu_starting(unsigned int cpu)
> +{
> +	csr_write(CSR_IE, 1UL << RV_IRQ_SOFT);
> +	csr_write(CSR_IP, 0);
> +	return 0;
> +}
> +
> +static int riscv_intc_cpu_dying(unsigned int cpu)
> +{
> +	csr_clear(CSR_IE, 1UL << RV_IRQ_SOFT);
> +	return 0;
> +}
> +
> +static struct irq_chip riscv_intc_chip = {
> +	.name = "RISC-V INTC",
> +	.irq_mask = riscv_intc_irq_mask,
> +	.irq_unmask = riscv_intc_irq_unmask,
> +};
> +
> +static int riscv_intc_domain_map(struct irq_domain *d, unsigned int irq,
> +				 irq_hw_number_t hwirq)
> +{
> +	irq_set_percpu_devid(irq);
> +	irq_domain_set_info(d, irq, hwirq, &riscv_intc_chip, d->host_data,
> +			    handle_percpu_devid_irq, NULL, NULL);
> +	irq_set_status_flags(irq, IRQ_NOAUTOEN);
> +
> +	return 0;
> +}
> +
> +static const struct irq_domain_ops riscv_intc_domain_ops = {
> +	.map	= riscv_intc_domain_map,
> +	.xlate	= irq_domain_xlate_onecell,
> +};
> +
> +/* Get the OF device node used by INTC irq domain */
> +struct device_node *riscv_of_intc_domain_node(void)
> +{
> +	return intc_domain_node;
> +}
> +EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);
> +
> +static int __init riscv_intc_init(struct device_node *node,
> +				  struct device_node *parent)
> +{
> +	int hartid;
> +
> +	/*
> +	 * The DT will have one INTC DT node under each CPU (or HART)
> +	 * DT node so riscv_intc_init() function will be called once
> +	 * for each INTC DT node. We only need INTC initialization for
> +	 * the INTC DT node belonging to boot CPU (or boot HART).
> +	 */
> +	hartid = riscv_of_parent_hartid(node);
> +	if (hartid < 0)
> +		return 0;

This should at least be a warning, as it's an invalid device tree.

> +	if (riscv_hartid_to_cpuid(hartid) != smp_processor_id())
> +		return 0;

There should be a comment about this one, as it only works becuase there's one
CPU up and running while interrupts are being enabled.

> +
> +	intc_domain = irq_domain_add_linear(node, BITS_PER_LONG,
> +					    &riscv_intc_domain_ops, NULL);
> +	if (!intc_domain)
> +		goto error_add_linear;
> +
> +	/*
> +	 * We save the DT node used for creating irq domain and provide
> +	 * it to other drivers using iscv_of_intc_domain_node() function.

Missing the "r" in iscv_of_intc_domain_node

> +	 */
> +	intc_domain_node = of_node_get(node);
> +
> +	set_handle_irq(&riscv_intc_irq);

So there's now this implicit assumption that every CPU node has a
riscv,cpu-intc.  That's probably fine, as we don't have any other first-level
interrupt controllers, but at least whacking a warning if set_handle_irq()
fails seems prudent.  It'd be best to actually check that assumption (maybe
before quitting above?) and actually installing the percpu OF nodes, but that
would take a performance hit so I'm not sure it's worth it.

> +
> +	cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_STARTING,
> +			  "irqchip/riscv/intc:starting",
> +			  riscv_intc_cpu_starting,
> +			  riscv_intc_cpu_dying);
> +
> +	pr_info("%lu local interrupts mapped\n", (long)BITS_PER_LONG);
> +
> +	return 0;
> +
> +error_add_linear:
> +	pr_warn("unable to add IRQ domain\n");
> +	return -ENXIO;
> +}
> +
> +IRQCHIP_DECLARE(riscv, "riscv,cpu-intc", riscv_intc_init);
> diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
> index 16d31d114c30..4f9b2877aa9d 100644
> --- a/drivers/irqchip/irq-sifive-plic.c
> +++ b/drivers/irqchip/irq-sifive-plic.c
> @@ -9,6 +9,8 @@
>  #include <linux/io.h>
>  #include <linux/irq.h>
>  #include <linux/irqchip.h>
> +#include <linux/irqchip/chained_irq.h>
> +#include <linux/irqchip/irq-riscv-intc.h>
>  #include <linux/irqdomain.h>
>  #include <linux/module.h>
>  #include <linux/of.h>
> @@ -60,6 +62,7 @@
>  #define	PLIC_ENABLE_THRESHOLD		0
>
>  struct plic_priv {
> +	int parent_irq;
>  	struct cpumask lmask;
>  	struct irq_domain *irqdomain;
>  	void __iomem *regs;
> @@ -219,15 +222,17 @@ static const struct irq_domain_ops plic_irqdomain_ops = {
>   * that source ID back to the same claim register.  This automatically enables
>   * and disables the interrupt, so there's nothing else to do.
>   */
> -static void plic_handle_irq(struct pt_regs *regs)
> +static void plic_handle_irq(struct irq_desc *desc)
>  {
>  	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> +	struct irq_chip *chip = irq_desc_get_chip(desc);
>  	void __iomem *claim = handler->hart_base + CONTEXT_CLAIM;
>  	irq_hw_number_t hwirq;
>
>  	WARN_ON_ONCE(!handler->present);
>
> -	csr_clear(CSR_IE, IE_EIE);
> +	chained_irq_enter(chip, desc);
> +

Ya, it's pretty clear putting the local interrupt controller in irqchip is the
right way to go :)

>  	while ((hwirq = readl(claim))) {
>  		int irq = irq_find_mapping(handler->priv->irqdomain, hwirq);
>
> @@ -237,7 +242,8 @@ static void plic_handle_irq(struct pt_regs *regs)
>  		else
>  			generic_handle_irq(irq);
>  	}
> -	csr_set(CSR_IE, IE_EIE);
> +
> +	chained_irq_exit(chip, desc);
>  }
>
>  static void plic_set_threshold(struct plic_handler *handler, u32 threshold)
> @@ -250,7 +256,8 @@ static int plic_dying_cpu(unsigned int cpu)
>  {
>  	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
>
> -	csr_clear(CSR_IE, IE_EIE);
> +	if (handler->priv->parent_irq)
> +		disable_percpu_irq(handler->priv->parent_irq);
>  	plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
>
>  	return 0;
> @@ -259,8 +266,10 @@ static int plic_dying_cpu(unsigned int cpu)
>  static int plic_starting_cpu(unsigned int cpu)
>  {
>  	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> +	int pirq = handler->priv->parent_irq;
>
> -	csr_set(CSR_IE, IE_EIE);
> +	if (pirq)
> +		enable_percpu_irq(pirq, irq_get_trigger_type(pirq));
>  	plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
>
>  	return 0;
> @@ -273,6 +282,7 @@ static int __init plic_init(struct device_node *node,
>  	u32 nr_irqs;
>  	struct plic_priv *priv;
>  	struct plic_handler *handler;
> +	struct of_phandle_args intc_args;
>
>  	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
>  	if (!priv)
> @@ -354,6 +364,23 @@ static int __init plic_init(struct device_node *node,
>  		nr_handlers++;
>  	}
>
> +	/*
> +	 * We can have multiple PLIC instances so setup chained handler
> +	 * for all PLIC instances.
> +	 */
> +	intc_args.np = riscv_of_intc_domain_node();
> +	intc_args.args_count = 1;
> +	intc_args.args[0] = RV_IRQ_EXT;

You could just use the value from the device tree rather than deciding on the
value a-priori.  It's a bit pedantic to worry about that now, as the PLIC
assumes that there's a RISC-V system and its parent is an cpu-intc, but it
always seems ugly that the PLIC driver is so RISC-V specific.

> +	priv->parent_irq = irq_create_of_mapping(&intc_args);
> +	if (priv->parent_irq)
> +		irq_set_chained_handler(priv->parent_irq,
> +					 plic_handle_irq);
> +	else {
> +		pr_err("%pOFP: intc mapping failed\n", node);
> +		error = -ENODEV;
> +		goto out_iounmap;
> +	}
> +
>  	/*
>  	 * We can have multiple PLIC instances so setup cpuhp state only
>  	 * when context handler for current/boot CPU is present.
> @@ -368,7 +395,6 @@ static int __init plic_init(struct device_node *node,
>
>  	pr_info("%pOFP: mapped %d interrupts with %d handlers for"
>  		" %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
> -	set_handle_irq(plic_handle_irq);
>  	return 0;
>
>  out_iounmap:
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index 77d70b633531..57b1f8f777d9 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -102,6 +102,7 @@ enum cpuhp_state {
>  	CPUHP_AP_IRQ_ARMADA_XP_STARTING,
>  	CPUHP_AP_IRQ_BCM2836_STARTING,
>  	CPUHP_AP_IRQ_MIPS_GIC_STARTING,
> +	CPUHP_AP_IRQ_RISCV_STARTING,
>  	CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
>  	CPUHP_AP_ARM_MVEBU_COHERENCY,
>  	CPUHP_AP_MICROCODE_LOADER,
> diff --git a/include/linux/irqchip/irq-riscv-intc.h b/include/linux/irqchip/irq-riscv-intc.h
> new file mode 100644
> index 000000000000..b11d38353099
> --- /dev/null
> +++ b/include/linux/irqchip/irq-riscv-intc.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> + */
> +
> +#ifndef __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> +#define __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> +
> +struct device_node;
> +
> +#ifdef CONFIG_RISCV_INTC
> +struct device_node *riscv_of_intc_domain_node(void);
> +#else
> +static inline struct device_node *riscv_of_intc_domain_node(void)
> +{
> +	return NULL;
> +}
> +#endif
> +
> +#endif

These issues are all pretty minor, so I'm OK with this getting merged as-is.  I
fixed up some minor stuff and have it on my intc branch as 2bd957e2b0b6
("irqchip: RISC-V per-HART local interrupt controller driver"), but if the
irqchip folks want to take the patches then I'm not opposed to either version
getting merged.

Acked-by: Palmer Dabbelt <palmerdabbelt@google.com>

Re: [PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Palmer Dabbelt]

On Thu, 21 May 2020 06:32:58 PDT (-0700), Anup Patel wrote:
> The RISC-V per-HART local interrupt controller manages software
> interrupts, timer interrupts, external interrupts (which are routed
> via the platform level interrupt controller) and other per-HART
> local interrupts.
>
> This patch adds a driver for the RISC-V local interrupt controller.
> It is a major re-write over perviously submitted version.
> (Refer, https://www.spinics.net/lists/devicetree/msg241230.html)
>
> Few advantages of this new driver over previous one are:
> 1. All local interrupts are registered as per-CPU interrupts
> 2. The RISC-V timer driver can register timer interrupt handler
>    using kernel irq subsystem without relying on arch/riscv to
>    explicitly call it's interrupt handler
> 3. The KVM RISC-V can use this driver to implement interrupt
>    handler for per-HART guest external interrupt defined by
>    the RISC-V H-Extension
> 4. In future, we can develop drivers for devices with per-HART
>    interrupts without changing arch code or this driver (example,
>    CLINT timer driver for RISC-V M-mode kernel)
>
> The RISC-V INTC driver is compliant with RISC-V Hart-Level Interrupt
> Controller DT bindings located at:
> Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
>
> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/Kconfig                     |   1 +
>  arch/riscv/include/asm/irq.h           |   2 -
>  arch/riscv/kernel/irq.c                |  33 +-----
>  arch/riscv/kernel/traps.c              |   2 -
>  drivers/irqchip/Kconfig                |  13 ++
>  drivers/irqchip/Makefile               |   1 +
>  drivers/irqchip/irq-riscv-intc.c       | 158 +++++++++++++++++++++++++
>  drivers/irqchip/irq-sifive-plic.c      |  38 +++++-
>  include/linux/cpuhotplug.h             |   1 +
>  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
>  10 files changed, 229 insertions(+), 40 deletions(-)
>  create mode 100644 drivers/irqchip/irq-riscv-intc.c
>  create mode 100644 include/linux/irqchip/irq-riscv-intc.h
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 90a008e28f7e..822cb0e1a380 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -40,6 +40,7 @@ config RISCV
>  	select HAVE_PERF_REGS
>  	select HAVE_PERF_USER_STACK_DUMP
>  	select HAVE_SYSCALL_TRACEPOINTS
> +	select HANDLE_DOMAIN_IRQ
>  	select IRQ_DOMAIN
>  	select SPARSE_IRQ
>  	select SYSCTL_EXCEPTION_TRACE
> diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
> index 0183e15ace66..a9e5f07a7e9c 100644
> --- a/arch/riscv/include/asm/irq.h
> +++ b/arch/riscv/include/asm/irq.h
> @@ -10,8 +10,6 @@
>  #include <linux/interrupt.h>
>  #include <linux/linkage.h>
>
> -#define NR_IRQS         0
> -
>  void riscv_timer_interrupt(void);
>
>  #include <asm-generic/irq.h>
> diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> index bb0bfcd537e7..eb8777642ce6 100644
> --- a/arch/riscv/kernel/irq.c
> +++ b/arch/riscv/kernel/irq.c
> @@ -7,7 +7,6 @@
>
>  #include <linux/interrupt.h>
>  #include <linux/irqchip.h>
> -#include <linux/irqdomain.h>
>  #include <linux/seq_file.h>
>  #include <asm/smp.h>
>
> @@ -19,39 +18,13 @@ int arch_show_interrupts(struct seq_file *p, int prec)
>
>  asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
>  {
> -	struct pt_regs *old_regs;
> -
> -	switch (regs->cause & ~CAUSE_IRQ_FLAG) {
> -	case RV_IRQ_TIMER:
> -		old_regs = set_irq_regs(regs);
> -		irq_enter();
> -		riscv_timer_interrupt();
> -		irq_exit();
> -		set_irq_regs(old_regs);
> -		break;
> -#ifdef CONFIG_SMP
> -	case RV_IRQ_SOFT:
> -		/*
> -		 * We only use software interrupts to pass IPIs, so if a non-SMP
> -		 * system gets one, then we don't know what to do.
> -		 */
> -		handle_IPI(regs);
> -		break;
> -#endif
> -	case RV_IRQ_EXT:
> -		old_regs = set_irq_regs(regs);
> -		irq_enter();
> +	if (handle_arch_irq)
>  		handle_arch_irq(regs);
> -		irq_exit();
> -		set_irq_regs(old_regs);
> -		break;
> -	default:
> -		pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
> -		BUG();
> -	}
>  }
>
>  void __init init_IRQ(void)
>  {
>  	irqchip_init();
> +	if (!handle_arch_irq)
> +		panic("No interrupt controller found.");
>  }
> diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> index 7f58fa53033f..f48c76aadbf3 100644
> --- a/arch/riscv/kernel/traps.c
> +++ b/arch/riscv/kernel/traps.c
> @@ -178,6 +178,4 @@ void trap_init(void)
>  	csr_write(CSR_SCRATCH, 0);
>  	/* Set the exception vector address */
>  	csr_write(CSR_TVEC, &handle_exception);
> -	/* Enable interrupts */
> -	csr_write(CSR_IE, IE_SIE);
>  }
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index a85aada04a64..95d6137a8117 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -493,6 +493,19 @@ config TI_SCI_INTA_IRQCHIP
>  	  If you wish to use interrupt aggregator irq resources managed by the
>  	  TI System Controller, say Y here. Otherwise, say N.
>
> +config RISCV_INTC
> +	bool "RISC-V Local Interrupt Controller"
> +	depends on RISCV
> +	default y
> +	help
> +	   This enables support for the per-HART local interrupt controller
> +	   found in standard RISC-V systems.  The per-HART local interrupt
> +	   controller handles timer interrupts, software interrupts, and
> +	   hardware interrupts. Without a per-HART local interrupt controller,
> +	   a RISC-V system will be unable to handle any interrupts.
> +
> +	   If you don't know what to do here, say Y.
> +
>  config SIFIVE_PLIC
>  	bool "SiFive Platform-Level Interrupt Controller"
>  	depends on RISCV
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index 37bbe39bf909..b8319f045472 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -98,6 +98,7 @@ obj-$(CONFIG_NDS32)			+= irq-ativic32.o
>  obj-$(CONFIG_QCOM_PDC)			+= qcom-pdc.o
>  obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
>  obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
> +obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
>  obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
>  obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
>  obj-$(CONFIG_IMX_INTMUX)		+= irq-imx-intmux.o
> diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> new file mode 100644
> index 000000000000..2f364e6a87f9
> --- /dev/null
> +++ b/drivers/irqchip/irq-riscv-intc.c
> @@ -0,0 +1,158 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2012 Regents of the University of California
> + * Copyright (C) 2017-2018 SiFive
> + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> + */
> +
> +#define pr_fmt(fmt) "riscv-intc: " fmt
> +#include <linux/atomic.h>
> +#include <linux/bits.h>
> +#include <linux/cpu.h>
> +#include <linux/irq.h>
> +#include <linux/irqchip.h>
> +#include <linux/irqchip/irq-riscv-intc.h>
> +#include <linux/irqdomain.h>
> +#include <linux/interrupt.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/smp.h>
> +
> +static struct device_node *intc_domain_node;
> +static struct irq_domain *intc_domain;
> +
> +static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
> +{
> +	struct pt_regs *old_regs;
> +	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
> +
> +	if (unlikely(cause >= BITS_PER_LONG))
> +		panic("unexpected interrupt cause");
> +
> +	switch (cause) {
> +	case RV_IRQ_TIMER:
> +		old_regs = set_irq_regs(regs);
> +		irq_enter();
> +		riscv_timer_interrupt();
> +		irq_exit();
> +		set_irq_regs(old_regs);
> +		break;
> +#ifdef CONFIG_SMP
> +	case RV_IRQ_SOFT:
> +		/*
> +		 * We only use software interrupts to pass IPIs, so if a
> +		 * non-SMP system gets one, then we don't know what to do.
> +		 */
> +		handle_IPI(regs);
> +		break;
> +#endif
> +	default:
> +		handle_domain_irq(intc_domain, cause, regs);
> +		break;
> +	}
> +}
> +
> +/*
> + * On RISC-V systems local interrupts are masked or unmasked by writing
> + * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be written
> + * on the local hart, these functions can only be called on the hart that
> + * corresponds to the IRQ chip.
> + */
> +
> +static void riscv_intc_irq_mask(struct irq_data *d)
> +{
> +	csr_clear(CSR_IE, 1 << (long)d->hwirq);
> +}
> +
> +static void riscv_intc_irq_unmask(struct irq_data *d)
> +{
> +	csr_set(CSR_IE, 1 << (long)d->hwirq);
> +}
> +
> +static int riscv_intc_cpu_starting(unsigned int cpu)
> +{
> +	csr_write(CSR_IE, 1UL << RV_IRQ_SOFT);
> +	csr_write(CSR_IP, 0);
> +	return 0;
> +}
> +
> +static int riscv_intc_cpu_dying(unsigned int cpu)
> +{
> +	csr_clear(CSR_IE, 1UL << RV_IRQ_SOFT);
> +	return 0;
> +}
> +
> +static struct irq_chip riscv_intc_chip = {
> +	.name = "RISC-V INTC",
> +	.irq_mask = riscv_intc_irq_mask,
> +	.irq_unmask = riscv_intc_irq_unmask,
> +};
> +
> +static int riscv_intc_domain_map(struct irq_domain *d, unsigned int irq,
> +				 irq_hw_number_t hwirq)
> +{
> +	irq_set_percpu_devid(irq);
> +	irq_domain_set_info(d, irq, hwirq, &riscv_intc_chip, d->host_data,
> +			    handle_percpu_devid_irq, NULL, NULL);
> +	irq_set_status_flags(irq, IRQ_NOAUTOEN);
> +
> +	return 0;
> +}
> +
> +static const struct irq_domain_ops riscv_intc_domain_ops = {
> +	.map	= riscv_intc_domain_map,
> +	.xlate	= irq_domain_xlate_onecell,
> +};
> +
> +/* Get the OF device node used by INTC irq domain */
> +struct device_node *riscv_of_intc_domain_node(void)
> +{
> +	return intc_domain_node;
> +}
> +EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);
> +
> +static int __init riscv_intc_init(struct device_node *node,
> +				  struct device_node *parent)
> +{
> +	int hartid;
> +
> +	/*
> +	 * The DT will have one INTC DT node under each CPU (or HART)
> +	 * DT node so riscv_intc_init() function will be called once
> +	 * for each INTC DT node. We only need INTC initialization for
> +	 * the INTC DT node belonging to boot CPU (or boot HART).
> +	 */
> +	hartid = riscv_of_parent_hartid(node);
> +	if (hartid < 0)
> +		return 0;

This should at least be a warning, as it's an invalid device tree.

> +	if (riscv_hartid_to_cpuid(hartid) != smp_processor_id())
> +		return 0;

There should be a comment about this one, as it only works becuase there's one
CPU up and running while interrupts are being enabled.

> +
> +	intc_domain = irq_domain_add_linear(node, BITS_PER_LONG,
> +					    &riscv_intc_domain_ops, NULL);
> +	if (!intc_domain)
> +		goto error_add_linear;
> +
> +	/*
> +	 * We save the DT node used for creating irq domain and provide
> +	 * it to other drivers using iscv_of_intc_domain_node() function.

Missing the "r" in iscv_of_intc_domain_node

> +	 */
> +	intc_domain_node = of_node_get(node);
> +
> +	set_handle_irq(&riscv_intc_irq);

So there's now this implicit assumption that every CPU node has a
riscv,cpu-intc.  That's probably fine, as we don't have any other first-level
interrupt controllers, but at least whacking a warning if set_handle_irq()
fails seems prudent.  It'd be best to actually check that assumption (maybe
before quitting above?) and actually installing the percpu OF nodes, but that
would take a performance hit so I'm not sure it's worth it.

> +
> +	cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_STARTING,
> +			  "irqchip/riscv/intc:starting",
> +			  riscv_intc_cpu_starting,
> +			  riscv_intc_cpu_dying);
> +
> +	pr_info("%lu local interrupts mapped\n", (long)BITS_PER_LONG);
> +
> +	return 0;
> +
> +error_add_linear:
> +	pr_warn("unable to add IRQ domain\n");
> +	return -ENXIO;
> +}
> +
> +IRQCHIP_DECLARE(riscv, "riscv,cpu-intc", riscv_intc_init);
> diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
> index 16d31d114c30..4f9b2877aa9d 100644
> --- a/drivers/irqchip/irq-sifive-plic.c
> +++ b/drivers/irqchip/irq-sifive-plic.c
> @@ -9,6 +9,8 @@
>  #include <linux/io.h>
>  #include <linux/irq.h>
>  #include <linux/irqchip.h>
> +#include <linux/irqchip/chained_irq.h>
> +#include <linux/irqchip/irq-riscv-intc.h>
>  #include <linux/irqdomain.h>
>  #include <linux/module.h>
>  #include <linux/of.h>
> @@ -60,6 +62,7 @@
>  #define	PLIC_ENABLE_THRESHOLD		0
>
>  struct plic_priv {
> +	int parent_irq;
>  	struct cpumask lmask;
>  	struct irq_domain *irqdomain;
>  	void __iomem *regs;
> @@ -219,15 +222,17 @@ static const struct irq_domain_ops plic_irqdomain_ops = {
>   * that source ID back to the same claim register.  This automatically enables
>   * and disables the interrupt, so there's nothing else to do.
>   */
> -static void plic_handle_irq(struct pt_regs *regs)
> +static void plic_handle_irq(struct irq_desc *desc)
>  {
>  	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> +	struct irq_chip *chip = irq_desc_get_chip(desc);
>  	void __iomem *claim = handler->hart_base + CONTEXT_CLAIM;
>  	irq_hw_number_t hwirq;
>
>  	WARN_ON_ONCE(!handler->present);
>
> -	csr_clear(CSR_IE, IE_EIE);
> +	chained_irq_enter(chip, desc);
> +

Ya, it's pretty clear putting the local interrupt controller in irqchip is the
right way to go :)

>  	while ((hwirq = readl(claim))) {
>  		int irq = irq_find_mapping(handler->priv->irqdomain, hwirq);
>
> @@ -237,7 +242,8 @@ static void plic_handle_irq(struct pt_regs *regs)
>  		else
>  			generic_handle_irq(irq);
>  	}
> -	csr_set(CSR_IE, IE_EIE);
> +
> +	chained_irq_exit(chip, desc);
>  }
>
>  static void plic_set_threshold(struct plic_handler *handler, u32 threshold)
> @@ -250,7 +256,8 @@ static int plic_dying_cpu(unsigned int cpu)
>  {
>  	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
>
> -	csr_clear(CSR_IE, IE_EIE);
> +	if (handler->priv->parent_irq)
> +		disable_percpu_irq(handler->priv->parent_irq);
>  	plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
>
>  	return 0;
> @@ -259,8 +266,10 @@ static int plic_dying_cpu(unsigned int cpu)
>  static int plic_starting_cpu(unsigned int cpu)
>  {
>  	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> +	int pirq = handler->priv->parent_irq;
>
> -	csr_set(CSR_IE, IE_EIE);
> +	if (pirq)
> +		enable_percpu_irq(pirq, irq_get_trigger_type(pirq));
>  	plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
>
>  	return 0;
> @@ -273,6 +282,7 @@ static int __init plic_init(struct device_node *node,
>  	u32 nr_irqs;
>  	struct plic_priv *priv;
>  	struct plic_handler *handler;
> +	struct of_phandle_args intc_args;
>
>  	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
>  	if (!priv)
> @@ -354,6 +364,23 @@ static int __init plic_init(struct device_node *node,
>  		nr_handlers++;
>  	}
>
> +	/*
> +	 * We can have multiple PLIC instances so setup chained handler
> +	 * for all PLIC instances.
> +	 */
> +	intc_args.np = riscv_of_intc_domain_node();
> +	intc_args.args_count = 1;
> +	intc_args.args[0] = RV_IRQ_EXT;

You could just use the value from the device tree rather than deciding on the
value a-priori.  It's a bit pedantic to worry about that now, as the PLIC
assumes that there's a RISC-V system and its parent is an cpu-intc, but it
always seems ugly that the PLIC driver is so RISC-V specific.

> +	priv->parent_irq = irq_create_of_mapping(&intc_args);
> +	if (priv->parent_irq)
> +		irq_set_chained_handler(priv->parent_irq,
> +					 plic_handle_irq);
> +	else {
> +		pr_err("%pOFP: intc mapping failed\n", node);
> +		error = -ENODEV;
> +		goto out_iounmap;
> +	}
> +
>  	/*
>  	 * We can have multiple PLIC instances so setup cpuhp state only
>  	 * when context handler for current/boot CPU is present.
> @@ -368,7 +395,6 @@ static int __init plic_init(struct device_node *node,
>
>  	pr_info("%pOFP: mapped %d interrupts with %d handlers for"
>  		" %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
> -	set_handle_irq(plic_handle_irq);
>  	return 0;
>
>  out_iounmap:
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index 77d70b633531..57b1f8f777d9 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -102,6 +102,7 @@ enum cpuhp_state {
>  	CPUHP_AP_IRQ_ARMADA_XP_STARTING,
>  	CPUHP_AP_IRQ_BCM2836_STARTING,
>  	CPUHP_AP_IRQ_MIPS_GIC_STARTING,
> +	CPUHP_AP_IRQ_RISCV_STARTING,
>  	CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
>  	CPUHP_AP_ARM_MVEBU_COHERENCY,
>  	CPUHP_AP_MICROCODE_LOADER,
> diff --git a/include/linux/irqchip/irq-riscv-intc.h b/include/linux/irqchip/irq-riscv-intc.h
> new file mode 100644
> index 000000000000..b11d38353099
> --- /dev/null
> +++ b/include/linux/irqchip/irq-riscv-intc.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> + */
> +
> +#ifndef __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> +#define __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> +
> +struct device_node;
> +
> +#ifdef CONFIG_RISCV_INTC
> +struct device_node *riscv_of_intc_domain_node(void);
> +#else
> +static inline struct device_node *riscv_of_intc_domain_node(void)
> +{
> +	return NULL;
> +}
> +#endif
> +
> +#endif

These issues are all pretty minor, so I'm OK with this getting merged as-is.  I
fixed up some minor stuff and have it on my intc branch as 2bd957e2b0b6
("irqchip: RISC-V per-HART local interrupt controller driver"), but if the
irqchip folks want to take the patches then I'm not opposed to either version
getting merged.

Acked-by: Palmer Dabbelt <palmerdabbelt@google.com>

Re: [PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Anup Patel]

On Fri, May 29, 2020 at 8:10 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>
> On Thu, 21 May 2020 06:32:58 PDT (-0700), Anup Patel wrote:
> > The RISC-V per-HART local interrupt controller manages software
> > interrupts, timer interrupts, external interrupts (which are routed
> > via the platform level interrupt controller) and other per-HART
> > local interrupts.
> >
> > This patch adds a driver for the RISC-V local interrupt controller.
> > It is a major re-write over perviously submitted version.
> > (Refer, https://www.spinics.net/lists/devicetree/msg241230.html)
> >
> > Few advantages of this new driver over previous one are:
> > 1. All local interrupts are registered as per-CPU interrupts
> > 2. The RISC-V timer driver can register timer interrupt handler
> >    using kernel irq subsystem without relying on arch/riscv to
> >    explicitly call it's interrupt handler
> > 3. The KVM RISC-V can use this driver to implement interrupt
> >    handler for per-HART guest external interrupt defined by
> >    the RISC-V H-Extension
> > 4. In future, we can develop drivers for devices with per-HART
> >    interrupts without changing arch code or this driver (example,
> >    CLINT timer driver for RISC-V M-mode kernel)
> >
> > The RISC-V INTC driver is compliant with RISC-V Hart-Level Interrupt
> > Controller DT bindings located at:
> > Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
> >
> > Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
> > Signed-off-by: Anup Patel <anup.patel@wdc.com>
> > ---
> >  arch/riscv/Kconfig                     |   1 +
> >  arch/riscv/include/asm/irq.h           |   2 -
> >  arch/riscv/kernel/irq.c                |  33 +-----
> >  arch/riscv/kernel/traps.c              |   2 -
> >  drivers/irqchip/Kconfig                |  13 ++
> >  drivers/irqchip/Makefile               |   1 +
> >  drivers/irqchip/irq-riscv-intc.c       | 158 +++++++++++++++++++++++++
> >  drivers/irqchip/irq-sifive-plic.c      |  38 +++++-
> >  include/linux/cpuhotplug.h             |   1 +
> >  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
> >  10 files changed, 229 insertions(+), 40 deletions(-)
> >  create mode 100644 drivers/irqchip/irq-riscv-intc.c
> >  create mode 100644 include/linux/irqchip/irq-riscv-intc.h
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 90a008e28f7e..822cb0e1a380 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -40,6 +40,7 @@ config RISCV
> >       select HAVE_PERF_REGS
> >       select HAVE_PERF_USER_STACK_DUMP
> >       select HAVE_SYSCALL_TRACEPOINTS
> > +     select HANDLE_DOMAIN_IRQ
> >       select IRQ_DOMAIN
> >       select SPARSE_IRQ
> >       select SYSCTL_EXCEPTION_TRACE
> > diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
> > index 0183e15ace66..a9e5f07a7e9c 100644
> > --- a/arch/riscv/include/asm/irq.h
> > +++ b/arch/riscv/include/asm/irq.h
> > @@ -10,8 +10,6 @@
> >  #include <linux/interrupt.h>
> >  #include <linux/linkage.h>
> >
> > -#define NR_IRQS         0
> > -
> >  void riscv_timer_interrupt(void);
> >
> >  #include <asm-generic/irq.h>
> > diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> > index bb0bfcd537e7..eb8777642ce6 100644
> > --- a/arch/riscv/kernel/irq.c
> > +++ b/arch/riscv/kernel/irq.c
> > @@ -7,7 +7,6 @@
> >
> >  #include <linux/interrupt.h>
> >  #include <linux/irqchip.h>
> > -#include <linux/irqdomain.h>
> >  #include <linux/seq_file.h>
> >  #include <asm/smp.h>
> >
> > @@ -19,39 +18,13 @@ int arch_show_interrupts(struct seq_file *p, int prec)
> >
> >  asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
> >  {
> > -     struct pt_regs *old_regs;
> > -
> > -     switch (regs->cause & ~CAUSE_IRQ_FLAG) {
> > -     case RV_IRQ_TIMER:
> > -             old_regs = set_irq_regs(regs);
> > -             irq_enter();
> > -             riscv_timer_interrupt();
> > -             irq_exit();
> > -             set_irq_regs(old_regs);
> > -             break;
> > -#ifdef CONFIG_SMP
> > -     case RV_IRQ_SOFT:
> > -             /*
> > -              * We only use software interrupts to pass IPIs, so if a non-SMP
> > -              * system gets one, then we don't know what to do.
> > -              */
> > -             handle_IPI(regs);
> > -             break;
> > -#endif
> > -     case RV_IRQ_EXT:
> > -             old_regs = set_irq_regs(regs);
> > -             irq_enter();
> > +     if (handle_arch_irq)
> >               handle_arch_irq(regs);
> > -             irq_exit();
> > -             set_irq_regs(old_regs);
> > -             break;
> > -     default:
> > -             pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
> > -             BUG();
> > -     }
> >  }
> >
> >  void __init init_IRQ(void)
> >  {
> >       irqchip_init();
> > +     if (!handle_arch_irq)
> > +             panic("No interrupt controller found.");
> >  }
> > diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> > index 7f58fa53033f..f48c76aadbf3 100644
> > --- a/arch/riscv/kernel/traps.c
> > +++ b/arch/riscv/kernel/traps.c
> > @@ -178,6 +178,4 @@ void trap_init(void)
> >       csr_write(CSR_SCRATCH, 0);
> >       /* Set the exception vector address */
> >       csr_write(CSR_TVEC, &handle_exception);
> > -     /* Enable interrupts */
> > -     csr_write(CSR_IE, IE_SIE);
> >  }
> > diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> > index a85aada04a64..95d6137a8117 100644
> > --- a/drivers/irqchip/Kconfig
> > +++ b/drivers/irqchip/Kconfig
> > @@ -493,6 +493,19 @@ config TI_SCI_INTA_IRQCHIP
> >         If you wish to use interrupt aggregator irq resources managed by the
> >         TI System Controller, say Y here. Otherwise, say N.
> >
> > +config RISCV_INTC
> > +     bool "RISC-V Local Interrupt Controller"
> > +     depends on RISCV
> > +     default y
> > +     help
> > +        This enables support for the per-HART local interrupt controller
> > +        found in standard RISC-V systems.  The per-HART local interrupt
> > +        controller handles timer interrupts, software interrupts, and
> > +        hardware interrupts. Without a per-HART local interrupt controller,
> > +        a RISC-V system will be unable to handle any interrupts.
> > +
> > +        If you don't know what to do here, say Y.
> > +
> >  config SIFIVE_PLIC
> >       bool "SiFive Platform-Level Interrupt Controller"
> >       depends on RISCV
> > diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> > index 37bbe39bf909..b8319f045472 100644
> > --- a/drivers/irqchip/Makefile
> > +++ b/drivers/irqchip/Makefile
> > @@ -98,6 +98,7 @@ obj-$(CONFIG_NDS32)                 += irq-ativic32.o
> >  obj-$(CONFIG_QCOM_PDC)                       += qcom-pdc.o
> >  obj-$(CONFIG_CSKY_MPINTC)            += irq-csky-mpintc.o
> >  obj-$(CONFIG_CSKY_APB_INTC)          += irq-csky-apb-intc.o
> > +obj-$(CONFIG_RISCV_INTC)             += irq-riscv-intc.o
> >  obj-$(CONFIG_SIFIVE_PLIC)            += irq-sifive-plic.o
> >  obj-$(CONFIG_IMX_IRQSTEER)           += irq-imx-irqsteer.o
> >  obj-$(CONFIG_IMX_INTMUX)             += irq-imx-intmux.o
> > diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> > new file mode 100644
> > index 000000000000..2f364e6a87f9
> > --- /dev/null
> > +++ b/drivers/irqchip/irq-riscv-intc.c
> > @@ -0,0 +1,158 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (C) 2012 Regents of the University of California
> > + * Copyright (C) 2017-2018 SiFive
> > + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> > + */
> > +
> > +#define pr_fmt(fmt) "riscv-intc: " fmt
> > +#include <linux/atomic.h>
> > +#include <linux/bits.h>
> > +#include <linux/cpu.h>
> > +#include <linux/irq.h>
> > +#include <linux/irqchip.h>
> > +#include <linux/irqchip/irq-riscv-intc.h>
> > +#include <linux/irqdomain.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/module.h>
> > +#include <linux/of.h>
> > +#include <linux/smp.h>
> > +
> > +static struct device_node *intc_domain_node;
> > +static struct irq_domain *intc_domain;
> > +
> > +static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
> > +{
> > +     struct pt_regs *old_regs;
> > +     unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
> > +
> > +     if (unlikely(cause >= BITS_PER_LONG))
> > +             panic("unexpected interrupt cause");
> > +
> > +     switch (cause) {
> > +     case RV_IRQ_TIMER:
> > +             old_regs = set_irq_regs(regs);
> > +             irq_enter();
> > +             riscv_timer_interrupt();
> > +             irq_exit();
> > +             set_irq_regs(old_regs);
> > +             break;
> > +#ifdef CONFIG_SMP
> > +     case RV_IRQ_SOFT:
> > +             /*
> > +              * We only use software interrupts to pass IPIs, so if a
> > +              * non-SMP system gets one, then we don't know what to do.
> > +              */
> > +             handle_IPI(regs);
> > +             break;
> > +#endif
> > +     default:
> > +             handle_domain_irq(intc_domain, cause, regs);
> > +             break;
> > +     }
> > +}
> > +
> > +/*
> > + * On RISC-V systems local interrupts are masked or unmasked by writing
> > + * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be written
> > + * on the local hart, these functions can only be called on the hart that
> > + * corresponds to the IRQ chip.
> > + */
> > +
> > +static void riscv_intc_irq_mask(struct irq_data *d)
> > +{
> > +     csr_clear(CSR_IE, 1 << (long)d->hwirq);
> > +}
> > +
> > +static void riscv_intc_irq_unmask(struct irq_data *d)
> > +{
> > +     csr_set(CSR_IE, 1 << (long)d->hwirq);
> > +}
> > +
> > +static int riscv_intc_cpu_starting(unsigned int cpu)
> > +{
> > +     csr_write(CSR_IE, 1UL << RV_IRQ_SOFT);
> > +     csr_write(CSR_IP, 0);
> > +     return 0;
> > +}
> > +
> > +static int riscv_intc_cpu_dying(unsigned int cpu)
> > +{
> > +     csr_clear(CSR_IE, 1UL << RV_IRQ_SOFT);
> > +     return 0;
> > +}
> > +
> > +static struct irq_chip riscv_intc_chip = {
> > +     .name = "RISC-V INTC",
> > +     .irq_mask = riscv_intc_irq_mask,
> > +     .irq_unmask = riscv_intc_irq_unmask,
> > +};
> > +
> > +static int riscv_intc_domain_map(struct irq_domain *d, unsigned int irq,
> > +                              irq_hw_number_t hwirq)
> > +{
> > +     irq_set_percpu_devid(irq);
> > +     irq_domain_set_info(d, irq, hwirq, &riscv_intc_chip, d->host_data,
> > +                         handle_percpu_devid_irq, NULL, NULL);
> > +     irq_set_status_flags(irq, IRQ_NOAUTOEN);
> > +
> > +     return 0;
> > +}
> > +
> > +static const struct irq_domain_ops riscv_intc_domain_ops = {
> > +     .map    = riscv_intc_domain_map,
> > +     .xlate  = irq_domain_xlate_onecell,
> > +};
> > +
> > +/* Get the OF device node used by INTC irq domain */
> > +struct device_node *riscv_of_intc_domain_node(void)
> > +{
> > +     return intc_domain_node;
> > +}
> > +EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);
> > +
> > +static int __init riscv_intc_init(struct device_node *node,
> > +                               struct device_node *parent)
> > +{
> > +     int hartid;
> > +
> > +     /*
> > +      * The DT will have one INTC DT node under each CPU (or HART)
> > +      * DT node so riscv_intc_init() function will be called once
> > +      * for each INTC DT node. We only need INTC initialization for
> > +      * the INTC DT node belonging to boot CPU (or boot HART).
> > +      */
> > +     hartid = riscv_of_parent_hartid(node);
> > +     if (hartid < 0)
> > +             return 0;
>
> This should at least be a warning, as it's an invalid device tree.

Okay, I will add a pr_warn() here.

>
> > +     if (riscv_hartid_to_cpuid(hartid) != smp_processor_id())
> > +             return 0;
>
> There should be a comment about this one, as it only works becuase there's one
> CPU up and running while interrupts are being enabled.

The first comment block in riscv_intc_init() tries to explain it. If you
want I can move the comment block just above this "if-statement"

>
> > +
> > +     intc_domain = irq_domain_add_linear(node, BITS_PER_LONG,
> > +                                         &riscv_intc_domain_ops, NULL);
> > +     if (!intc_domain)
> > +             goto error_add_linear;
> > +
> > +     /*
> > +      * We save the DT node used for creating irq domain and provide
> > +      * it to other drivers using iscv_of_intc_domain_node() function.
>
> Missing the "r" in iscv_of_intc_domain_node

Okay, will update.

>
> > +      */
> > +     intc_domain_node = of_node_get(node);
> > +
> > +     set_handle_irq(&riscv_intc_irq);
>
> So there's now this implicit assumption that every CPU node has a
> riscv,cpu-intc.  That's probably fine, as we don't have any other first-level
> interrupt controllers, but at least whacking a warning if set_handle_irq()
> fails seems prudent.  It'd be best to actually check that assumption (maybe
> before quitting above?) and actually installing the percpu OF nodes, but that
> would take a performance hit so I'm not sure it's worth it.

Yes, we should have a warning if set_handle_irq() fails. I will update.

>
> > +
> > +     cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_STARTING,
> > +                       "irqchip/riscv/intc:starting",
> > +                       riscv_intc_cpu_starting,
> > +                       riscv_intc_cpu_dying);
> > +
> > +     pr_info("%lu local interrupts mapped\n", (long)BITS_PER_LONG);
> > +
> > +     return 0;
> > +
> > +error_add_linear:
> > +     pr_warn("unable to add IRQ domain\n");
> > +     return -ENXIO;
> > +}
> > +
> > +IRQCHIP_DECLARE(riscv, "riscv,cpu-intc", riscv_intc_init);
> > diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
> > index 16d31d114c30..4f9b2877aa9d 100644
> > --- a/drivers/irqchip/irq-sifive-plic.c
> > +++ b/drivers/irqchip/irq-sifive-plic.c
> > @@ -9,6 +9,8 @@
> >  #include <linux/io.h>
> >  #include <linux/irq.h>
> >  #include <linux/irqchip.h>
> > +#include <linux/irqchip/chained_irq.h>
> > +#include <linux/irqchip/irq-riscv-intc.h>
> >  #include <linux/irqdomain.h>
> >  #include <linux/module.h>
> >  #include <linux/of.h>
> > @@ -60,6 +62,7 @@
> >  #define      PLIC_ENABLE_THRESHOLD           0
> >
> >  struct plic_priv {
> > +     int parent_irq;
> >       struct cpumask lmask;
> >       struct irq_domain *irqdomain;
> >       void __iomem *regs;
> > @@ -219,15 +222,17 @@ static const struct irq_domain_ops plic_irqdomain_ops = {
> >   * that source ID back to the same claim register.  This automatically enables
> >   * and disables the interrupt, so there's nothing else to do.
> >   */
> > -static void plic_handle_irq(struct pt_regs *regs)
> > +static void plic_handle_irq(struct irq_desc *desc)
> >  {
> >       struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> > +     struct irq_chip *chip = irq_desc_get_chip(desc);
> >       void __iomem *claim = handler->hart_base + CONTEXT_CLAIM;
> >       irq_hw_number_t hwirq;
> >
> >       WARN_ON_ONCE(!handler->present);
> >
> > -     csr_clear(CSR_IE, IE_EIE);
> > +     chained_irq_enter(chip, desc);
> > +
>
> Ya, it's pretty clear putting the local interrupt controller in irqchip is the
> right way to go :)
>
> >       while ((hwirq = readl(claim))) {
> >               int irq = irq_find_mapping(handler->priv->irqdomain, hwirq);
> >
> > @@ -237,7 +242,8 @@ static void plic_handle_irq(struct pt_regs *regs)
> >               else
> >                       generic_handle_irq(irq);
> >       }
> > -     csr_set(CSR_IE, IE_EIE);
> > +
> > +     chained_irq_exit(chip, desc);
> >  }
> >
> >  static void plic_set_threshold(struct plic_handler *handler, u32 threshold)
> > @@ -250,7 +256,8 @@ static int plic_dying_cpu(unsigned int cpu)
> >  {
> >       struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> >
> > -     csr_clear(CSR_IE, IE_EIE);
> > +     if (handler->priv->parent_irq)
> > +             disable_percpu_irq(handler->priv->parent_irq);
> >       plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
> >
> >       return 0;
> > @@ -259,8 +266,10 @@ static int plic_dying_cpu(unsigned int cpu)
> >  static int plic_starting_cpu(unsigned int cpu)
> >  {
> >       struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> > +     int pirq = handler->priv->parent_irq;
> >
> > -     csr_set(CSR_IE, IE_EIE);
> > +     if (pirq)
> > +             enable_percpu_irq(pirq, irq_get_trigger_type(pirq));
> >       plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
> >
> >       return 0;
> > @@ -273,6 +282,7 @@ static int __init plic_init(struct device_node *node,
> >       u32 nr_irqs;
> >       struct plic_priv *priv;
> >       struct plic_handler *handler;
> > +     struct of_phandle_args intc_args;
> >
> >       priv = kzalloc(sizeof(*priv), GFP_KERNEL);
> >       if (!priv)
> > @@ -354,6 +364,23 @@ static int __init plic_init(struct device_node *node,
> >               nr_handlers++;
> >       }
> >
> > +     /*
> > +      * We can have multiple PLIC instances so setup chained handler
> > +      * for all PLIC instances.
> > +      */
> > +     intc_args.np = riscv_of_intc_domain_node();
> > +     intc_args.args_count = 1;
> > +     intc_args.args[0] = RV_IRQ_EXT;
>
> You could just use the value from the device tree rather than deciding on the
> value a-priori.  It's a bit pedantic to worry about that now, as the PLIC
> assumes that there's a RISC-V system and its parent is an cpu-intc, but it
> always seems ugly that the PLIC driver is so RISC-V specific.

We are creating only one RISC-V INTC irq_domain for the DT node
belonging to the boot CPU.

This becomes a problem for multiple PLIC instances because the
interrupts-extended property of the PLIC DT node only points to a
subset of HARTs.

Let me know if you have any other suggestion.


>
> > +     priv->parent_irq = irq_create_of_mapping(&intc_args);
> > +     if (priv->parent_irq)
> > +             irq_set_chained_handler(priv->parent_irq,
> > +                                      plic_handle_irq);
> > +     else {
> > +             pr_err("%pOFP: intc mapping failed\n", node);
> > +             error = -ENODEV;
> > +             goto out_iounmap;
> > +     }
> > +
> >       /*
> >        * We can have multiple PLIC instances so setup cpuhp state only
> >        * when context handler for current/boot CPU is present.
> > @@ -368,7 +395,6 @@ static int __init plic_init(struct device_node *node,
> >
> >       pr_info("%pOFP: mapped %d interrupts with %d handlers for"
> >               " %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
> > -     set_handle_irq(plic_handle_irq);
> >       return 0;
> >
> >  out_iounmap:
> > diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> > index 77d70b633531..57b1f8f777d9 100644
> > --- a/include/linux/cpuhotplug.h
> > +++ b/include/linux/cpuhotplug.h
> > @@ -102,6 +102,7 @@ enum cpuhp_state {
> >       CPUHP_AP_IRQ_ARMADA_XP_STARTING,
> >       CPUHP_AP_IRQ_BCM2836_STARTING,
> >       CPUHP_AP_IRQ_MIPS_GIC_STARTING,
> > +     CPUHP_AP_IRQ_RISCV_STARTING,
> >       CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
> >       CPUHP_AP_ARM_MVEBU_COHERENCY,
> >       CPUHP_AP_MICROCODE_LOADER,
> > diff --git a/include/linux/irqchip/irq-riscv-intc.h b/include/linux/irqchip/irq-riscv-intc.h
> > new file mode 100644
> > index 000000000000..b11d38353099
> > --- /dev/null
> > +++ b/include/linux/irqchip/irq-riscv-intc.h
> > @@ -0,0 +1,20 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> > + */
> > +
> > +#ifndef __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> > +#define __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> > +
> > +struct device_node;
> > +
> > +#ifdef CONFIG_RISCV_INTC
> > +struct device_node *riscv_of_intc_domain_node(void);
> > +#else
> > +static inline struct device_node *riscv_of_intc_domain_node(void)
> > +{
> > +     return NULL;
> > +}
> > +#endif
> > +
> > +#endif
>
> These issues are all pretty minor, so I'm OK with this getting merged as-is.  I
> fixed up some minor stuff and have it on my intc branch as 2bd957e2b0b6
> ("irqchip: RISC-V per-HART local interrupt controller driver"), but if the
> irqchip folks want to take the patches then I'm not opposed to either version
> getting merged.
>
> Acked-by: Palmer Dabbelt <palmerdabbelt@google.com>

Thanks, I can quickly send v6 with your comments addressed in a few hours.

Do you want me to send v6 ?

Regards,
Anup

Re: [PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Anup Patel]

On Fri, May 29, 2020 at 8:10 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>
> On Thu, 21 May 2020 06:32:58 PDT (-0700), Anup Patel wrote:
> > The RISC-V per-HART local interrupt controller manages software
> > interrupts, timer interrupts, external interrupts (which are routed
> > via the platform level interrupt controller) and other per-HART
> > local interrupts.
> >
> > This patch adds a driver for the RISC-V local interrupt controller.
> > It is a major re-write over perviously submitted version.
> > (Refer, https://www.spinics.net/lists/devicetree/msg241230.html)
> >
> > Few advantages of this new driver over previous one are:
> > 1. All local interrupts are registered as per-CPU interrupts
> > 2. The RISC-V timer driver can register timer interrupt handler
> >    using kernel irq subsystem without relying on arch/riscv to
> >    explicitly call it's interrupt handler
> > 3. The KVM RISC-V can use this driver to implement interrupt
> >    handler for per-HART guest external interrupt defined by
> >    the RISC-V H-Extension
> > 4. In future, we can develop drivers for devices with per-HART
> >    interrupts without changing arch code or this driver (example,
> >    CLINT timer driver for RISC-V M-mode kernel)
> >
> > The RISC-V INTC driver is compliant with RISC-V Hart-Level Interrupt
> > Controller DT bindings located at:
> > Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
> >
> > Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
> > Signed-off-by: Anup Patel <anup.patel@wdc.com>
> > ---
> >  arch/riscv/Kconfig                     |   1 +
> >  arch/riscv/include/asm/irq.h           |   2 -
> >  arch/riscv/kernel/irq.c                |  33 +-----
> >  arch/riscv/kernel/traps.c              |   2 -
> >  drivers/irqchip/Kconfig                |  13 ++
> >  drivers/irqchip/Makefile               |   1 +
> >  drivers/irqchip/irq-riscv-intc.c       | 158 +++++++++++++++++++++++++
> >  drivers/irqchip/irq-sifive-plic.c      |  38 +++++-
> >  include/linux/cpuhotplug.h             |   1 +
> >  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
> >  10 files changed, 229 insertions(+), 40 deletions(-)
> >  create mode 100644 drivers/irqchip/irq-riscv-intc.c
> >  create mode 100644 include/linux/irqchip/irq-riscv-intc.h
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 90a008e28f7e..822cb0e1a380 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -40,6 +40,7 @@ config RISCV
> >       select HAVE_PERF_REGS
> >       select HAVE_PERF_USER_STACK_DUMP
> >       select HAVE_SYSCALL_TRACEPOINTS
> > +     select HANDLE_DOMAIN_IRQ
> >       select IRQ_DOMAIN
> >       select SPARSE_IRQ
> >       select SYSCTL_EXCEPTION_TRACE
> > diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
> > index 0183e15ace66..a9e5f07a7e9c 100644
> > --- a/arch/riscv/include/asm/irq.h
> > +++ b/arch/riscv/include/asm/irq.h
> > @@ -10,8 +10,6 @@
> >  #include <linux/interrupt.h>
> >  #include <linux/linkage.h>
> >
> > -#define NR_IRQS         0
> > -
> >  void riscv_timer_interrupt(void);
> >
> >  #include <asm-generic/irq.h>
> > diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> > index bb0bfcd537e7..eb8777642ce6 100644
> > --- a/arch/riscv/kernel/irq.c
> > +++ b/arch/riscv/kernel/irq.c
> > @@ -7,7 +7,6 @@
> >
> >  #include <linux/interrupt.h>
> >  #include <linux/irqchip.h>
> > -#include <linux/irqdomain.h>
> >  #include <linux/seq_file.h>
> >  #include <asm/smp.h>
> >
> > @@ -19,39 +18,13 @@ int arch_show_interrupts(struct seq_file *p, int prec)
> >
> >  asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
> >  {
> > -     struct pt_regs *old_regs;
> > -
> > -     switch (regs->cause & ~CAUSE_IRQ_FLAG) {
> > -     case RV_IRQ_TIMER:
> > -             old_regs = set_irq_regs(regs);
> > -             irq_enter();
> > -             riscv_timer_interrupt();
> > -             irq_exit();
> > -             set_irq_regs(old_regs);
> > -             break;
> > -#ifdef CONFIG_SMP
> > -     case RV_IRQ_SOFT:
> > -             /*
> > -              * We only use software interrupts to pass IPIs, so if a non-SMP
> > -              * system gets one, then we don't know what to do.
> > -              */
> > -             handle_IPI(regs);
> > -             break;
> > -#endif
> > -     case RV_IRQ_EXT:
> > -             old_regs = set_irq_regs(regs);
> > -             irq_enter();
> > +     if (handle_arch_irq)
> >               handle_arch_irq(regs);
> > -             irq_exit();
> > -             set_irq_regs(old_regs);
> > -             break;
> > -     default:
> > -             pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
> > -             BUG();
> > -     }
> >  }
> >
> >  void __init init_IRQ(void)
> >  {
> >       irqchip_init();
> > +     if (!handle_arch_irq)
> > +             panic("No interrupt controller found.");
> >  }
> > diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> > index 7f58fa53033f..f48c76aadbf3 100644
> > --- a/arch/riscv/kernel/traps.c
> > +++ b/arch/riscv/kernel/traps.c
> > @@ -178,6 +178,4 @@ void trap_init(void)
> >       csr_write(CSR_SCRATCH, 0);
> >       /* Set the exception vector address */
> >       csr_write(CSR_TVEC, &handle_exception);
> > -     /* Enable interrupts */
> > -     csr_write(CSR_IE, IE_SIE);
> >  }
> > diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> > index a85aada04a64..95d6137a8117 100644
> > --- a/drivers/irqchip/Kconfig
> > +++ b/drivers/irqchip/Kconfig
> > @@ -493,6 +493,19 @@ config TI_SCI_INTA_IRQCHIP
> >         If you wish to use interrupt aggregator irq resources managed by the
> >         TI System Controller, say Y here. Otherwise, say N.
> >
> > +config RISCV_INTC
> > +     bool "RISC-V Local Interrupt Controller"
> > +     depends on RISCV
> > +     default y
> > +     help
> > +        This enables support for the per-HART local interrupt controller
> > +        found in standard RISC-V systems.  The per-HART local interrupt
> > +        controller handles timer interrupts, software interrupts, and
> > +        hardware interrupts. Without a per-HART local interrupt controller,
> > +        a RISC-V system will be unable to handle any interrupts.
> > +
> > +        If you don't know what to do here, say Y.
> > +
> >  config SIFIVE_PLIC
> >       bool "SiFive Platform-Level Interrupt Controller"
> >       depends on RISCV
> > diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> > index 37bbe39bf909..b8319f045472 100644
> > --- a/drivers/irqchip/Makefile
> > +++ b/drivers/irqchip/Makefile
> > @@ -98,6 +98,7 @@ obj-$(CONFIG_NDS32)                 += irq-ativic32.o
> >  obj-$(CONFIG_QCOM_PDC)                       += qcom-pdc.o
> >  obj-$(CONFIG_CSKY_MPINTC)            += irq-csky-mpintc.o
> >  obj-$(CONFIG_CSKY_APB_INTC)          += irq-csky-apb-intc.o
> > +obj-$(CONFIG_RISCV_INTC)             += irq-riscv-intc.o
> >  obj-$(CONFIG_SIFIVE_PLIC)            += irq-sifive-plic.o
> >  obj-$(CONFIG_IMX_IRQSTEER)           += irq-imx-irqsteer.o
> >  obj-$(CONFIG_IMX_INTMUX)             += irq-imx-intmux.o
> > diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> > new file mode 100644
> > index 000000000000..2f364e6a87f9
> > --- /dev/null
> > +++ b/drivers/irqchip/irq-riscv-intc.c
> > @@ -0,0 +1,158 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (C) 2012 Regents of the University of California
> > + * Copyright (C) 2017-2018 SiFive
> > + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> > + */
> > +
> > +#define pr_fmt(fmt) "riscv-intc: " fmt
> > +#include <linux/atomic.h>
> > +#include <linux/bits.h>
> > +#include <linux/cpu.h>
> > +#include <linux/irq.h>
> > +#include <linux/irqchip.h>
> > +#include <linux/irqchip/irq-riscv-intc.h>
> > +#include <linux/irqdomain.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/module.h>
> > +#include <linux/of.h>
> > +#include <linux/smp.h>
> > +
> > +static struct device_node *intc_domain_node;
> > +static struct irq_domain *intc_domain;
> > +
> > +static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
> > +{
> > +     struct pt_regs *old_regs;
> > +     unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
> > +
> > +     if (unlikely(cause >= BITS_PER_LONG))
> > +             panic("unexpected interrupt cause");
> > +
> > +     switch (cause) {
> > +     case RV_IRQ_TIMER:
> > +             old_regs = set_irq_regs(regs);
> > +             irq_enter();
> > +             riscv_timer_interrupt();
> > +             irq_exit();
> > +             set_irq_regs(old_regs);
> > +             break;
> > +#ifdef CONFIG_SMP
> > +     case RV_IRQ_SOFT:
> > +             /*
> > +              * We only use software interrupts to pass IPIs, so if a
> > +              * non-SMP system gets one, then we don't know what to do.
> > +              */
> > +             handle_IPI(regs);
> > +             break;
> > +#endif
> > +     default:
> > +             handle_domain_irq(intc_domain, cause, regs);
> > +             break;
> > +     }
> > +}
> > +
> > +/*
> > + * On RISC-V systems local interrupts are masked or unmasked by writing
> > + * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be written
> > + * on the local hart, these functions can only be called on the hart that
> > + * corresponds to the IRQ chip.
> > + */
> > +
> > +static void riscv_intc_irq_mask(struct irq_data *d)
> > +{
> > +     csr_clear(CSR_IE, 1 << (long)d->hwirq);
> > +}
> > +
> > +static void riscv_intc_irq_unmask(struct irq_data *d)
> > +{
> > +     csr_set(CSR_IE, 1 << (long)d->hwirq);
> > +}
> > +
> > +static int riscv_intc_cpu_starting(unsigned int cpu)
> > +{
> > +     csr_write(CSR_IE, 1UL << RV_IRQ_SOFT);
> > +     csr_write(CSR_IP, 0);
> > +     return 0;
> > +}
> > +
> > +static int riscv_intc_cpu_dying(unsigned int cpu)
> > +{
> > +     csr_clear(CSR_IE, 1UL << RV_IRQ_SOFT);
> > +     return 0;
> > +}
> > +
> > +static struct irq_chip riscv_intc_chip = {
> > +     .name = "RISC-V INTC",
> > +     .irq_mask = riscv_intc_irq_mask,
> > +     .irq_unmask = riscv_intc_irq_unmask,
> > +};
> > +
> > +static int riscv_intc_domain_map(struct irq_domain *d, unsigned int irq,
> > +                              irq_hw_number_t hwirq)
> > +{
> > +     irq_set_percpu_devid(irq);
> > +     irq_domain_set_info(d, irq, hwirq, &riscv_intc_chip, d->host_data,
> > +                         handle_percpu_devid_irq, NULL, NULL);
> > +     irq_set_status_flags(irq, IRQ_NOAUTOEN);
> > +
> > +     return 0;
> > +}
> > +
> > +static const struct irq_domain_ops riscv_intc_domain_ops = {
> > +     .map    = riscv_intc_domain_map,
> > +     .xlate  = irq_domain_xlate_onecell,
> > +};
> > +
> > +/* Get the OF device node used by INTC irq domain */
> > +struct device_node *riscv_of_intc_domain_node(void)
> > +{
> > +     return intc_domain_node;
> > +}
> > +EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);
> > +
> > +static int __init riscv_intc_init(struct device_node *node,
> > +                               struct device_node *parent)
> > +{
> > +     int hartid;
> > +
> > +     /*
> > +      * The DT will have one INTC DT node under each CPU (or HART)
> > +      * DT node so riscv_intc_init() function will be called once
> > +      * for each INTC DT node. We only need INTC initialization for
> > +      * the INTC DT node belonging to boot CPU (or boot HART).
> > +      */
> > +     hartid = riscv_of_parent_hartid(node);
> > +     if (hartid < 0)
> > +             return 0;
>
> This should at least be a warning, as it's an invalid device tree.

Okay, I will add a pr_warn() here.

>
> > +     if (riscv_hartid_to_cpuid(hartid) != smp_processor_id())
> > +             return 0;
>
> There should be a comment about this one, as it only works becuase there's one
> CPU up and running while interrupts are being enabled.

The first comment block in riscv_intc_init() tries to explain it. If you
want I can move the comment block just above this "if-statement"

>
> > +
> > +     intc_domain = irq_domain_add_linear(node, BITS_PER_LONG,
> > +                                         &riscv_intc_domain_ops, NULL);
> > +     if (!intc_domain)
> > +             goto error_add_linear;
> > +
> > +     /*
> > +      * We save the DT node used for creating irq domain and provide
> > +      * it to other drivers using iscv_of_intc_domain_node() function.
>
> Missing the "r" in iscv_of_intc_domain_node

Okay, will update.

>
> > +      */
> > +     intc_domain_node = of_node_get(node);
> > +
> > +     set_handle_irq(&riscv_intc_irq);
>
> So there's now this implicit assumption that every CPU node has a
> riscv,cpu-intc.  That's probably fine, as we don't have any other first-level
> interrupt controllers, but at least whacking a warning if set_handle_irq()
> fails seems prudent.  It'd be best to actually check that assumption (maybe
> before quitting above?) and actually installing the percpu OF nodes, but that
> would take a performance hit so I'm not sure it's worth it.

Yes, we should have a warning if set_handle_irq() fails. I will update.

>
> > +
> > +     cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_STARTING,
> > +                       "irqchip/riscv/intc:starting",
> > +                       riscv_intc_cpu_starting,
> > +                       riscv_intc_cpu_dying);
> > +
> > +     pr_info("%lu local interrupts mapped\n", (long)BITS_PER_LONG);
> > +
> > +     return 0;
> > +
> > +error_add_linear:
> > +     pr_warn("unable to add IRQ domain\n");
> > +     return -ENXIO;
> > +}
> > +
> > +IRQCHIP_DECLARE(riscv, "riscv,cpu-intc", riscv_intc_init);
> > diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
> > index 16d31d114c30..4f9b2877aa9d 100644
> > --- a/drivers/irqchip/irq-sifive-plic.c
> > +++ b/drivers/irqchip/irq-sifive-plic.c
> > @@ -9,6 +9,8 @@
> >  #include <linux/io.h>
> >  #include <linux/irq.h>
> >  #include <linux/irqchip.h>
> > +#include <linux/irqchip/chained_irq.h>
> > +#include <linux/irqchip/irq-riscv-intc.h>
> >  #include <linux/irqdomain.h>
> >  #include <linux/module.h>
> >  #include <linux/of.h>
> > @@ -60,6 +62,7 @@
> >  #define      PLIC_ENABLE_THRESHOLD           0
> >
> >  struct plic_priv {
> > +     int parent_irq;
> >       struct cpumask lmask;
> >       struct irq_domain *irqdomain;
> >       void __iomem *regs;
> > @@ -219,15 +222,17 @@ static const struct irq_domain_ops plic_irqdomain_ops = {
> >   * that source ID back to the same claim register.  This automatically enables
> >   * and disables the interrupt, so there's nothing else to do.
> >   */
> > -static void plic_handle_irq(struct pt_regs *regs)
> > +static void plic_handle_irq(struct irq_desc *desc)
> >  {
> >       struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> > +     struct irq_chip *chip = irq_desc_get_chip(desc);
> >       void __iomem *claim = handler->hart_base + CONTEXT_CLAIM;
> >       irq_hw_number_t hwirq;
> >
> >       WARN_ON_ONCE(!handler->present);
> >
> > -     csr_clear(CSR_IE, IE_EIE);
> > +     chained_irq_enter(chip, desc);
> > +
>
> Ya, it's pretty clear putting the local interrupt controller in irqchip is the
> right way to go :)
>
> >       while ((hwirq = readl(claim))) {
> >               int irq = irq_find_mapping(handler->priv->irqdomain, hwirq);
> >
> > @@ -237,7 +242,8 @@ static void plic_handle_irq(struct pt_regs *regs)
> >               else
> >                       generic_handle_irq(irq);
> >       }
> > -     csr_set(CSR_IE, IE_EIE);
> > +
> > +     chained_irq_exit(chip, desc);
> >  }
> >
> >  static void plic_set_threshold(struct plic_handler *handler, u32 threshold)
> > @@ -250,7 +256,8 @@ static int plic_dying_cpu(unsigned int cpu)
> >  {
> >       struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> >
> > -     csr_clear(CSR_IE, IE_EIE);
> > +     if (handler->priv->parent_irq)
> > +             disable_percpu_irq(handler->priv->parent_irq);
> >       plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
> >
> >       return 0;
> > @@ -259,8 +266,10 @@ static int plic_dying_cpu(unsigned int cpu)
> >  static int plic_starting_cpu(unsigned int cpu)
> >  {
> >       struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> > +     int pirq = handler->priv->parent_irq;
> >
> > -     csr_set(CSR_IE, IE_EIE);
> > +     if (pirq)
> > +             enable_percpu_irq(pirq, irq_get_trigger_type(pirq));
> >       plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
> >
> >       return 0;
> > @@ -273,6 +282,7 @@ static int __init plic_init(struct device_node *node,
> >       u32 nr_irqs;
> >       struct plic_priv *priv;
> >       struct plic_handler *handler;
> > +     struct of_phandle_args intc_args;
> >
> >       priv = kzalloc(sizeof(*priv), GFP_KERNEL);
> >       if (!priv)
> > @@ -354,6 +364,23 @@ static int __init plic_init(struct device_node *node,
> >               nr_handlers++;
> >       }
> >
> > +     /*
> > +      * We can have multiple PLIC instances so setup chained handler
> > +      * for all PLIC instances.
> > +      */
> > +     intc_args.np = riscv_of_intc_domain_node();
> > +     intc_args.args_count = 1;
> > +     intc_args.args[0] = RV_IRQ_EXT;
>
> You could just use the value from the device tree rather than deciding on the
> value a-priori.  It's a bit pedantic to worry about that now, as the PLIC
> assumes that there's a RISC-V system and its parent is an cpu-intc, but it
> always seems ugly that the PLIC driver is so RISC-V specific.

We are creating only one RISC-V INTC irq_domain for the DT node
belonging to the boot CPU.

This becomes a problem for multiple PLIC instances because the
interrupts-extended property of the PLIC DT node only points to a
subset of HARTs.

Let me know if you have any other suggestion.


>
> > +     priv->parent_irq = irq_create_of_mapping(&intc_args);
> > +     if (priv->parent_irq)
> > +             irq_set_chained_handler(priv->parent_irq,
> > +                                      plic_handle_irq);
> > +     else {
> > +             pr_err("%pOFP: intc mapping failed\n", node);
> > +             error = -ENODEV;
> > +             goto out_iounmap;
> > +     }
> > +
> >       /*
> >        * We can have multiple PLIC instances so setup cpuhp state only
> >        * when context handler for current/boot CPU is present.
> > @@ -368,7 +395,6 @@ static int __init plic_init(struct device_node *node,
> >
> >       pr_info("%pOFP: mapped %d interrupts with %d handlers for"
> >               " %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
> > -     set_handle_irq(plic_handle_irq);
> >       return 0;
> >
> >  out_iounmap:
> > diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> > index 77d70b633531..57b1f8f777d9 100644
> > --- a/include/linux/cpuhotplug.h
> > +++ b/include/linux/cpuhotplug.h
> > @@ -102,6 +102,7 @@ enum cpuhp_state {
> >       CPUHP_AP_IRQ_ARMADA_XP_STARTING,
> >       CPUHP_AP_IRQ_BCM2836_STARTING,
> >       CPUHP_AP_IRQ_MIPS_GIC_STARTING,
> > +     CPUHP_AP_IRQ_RISCV_STARTING,
> >       CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
> >       CPUHP_AP_ARM_MVEBU_COHERENCY,
> >       CPUHP_AP_MICROCODE_LOADER,
> > diff --git a/include/linux/irqchip/irq-riscv-intc.h b/include/linux/irqchip/irq-riscv-intc.h
> > new file mode 100644
> > index 000000000000..b11d38353099
> > --- /dev/null
> > +++ b/include/linux/irqchip/irq-riscv-intc.h
> > @@ -0,0 +1,20 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> > + */
> > +
> > +#ifndef __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> > +#define __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> > +
> > +struct device_node;
> > +
> > +#ifdef CONFIG_RISCV_INTC
> > +struct device_node *riscv_of_intc_domain_node(void);
> > +#else
> > +static inline struct device_node *riscv_of_intc_domain_node(void)
> > +{
> > +     return NULL;
> > +}
> > +#endif
> > +
> > +#endif
>
> These issues are all pretty minor, so I'm OK with this getting merged as-is.  I
> fixed up some minor stuff and have it on my intc branch as 2bd957e2b0b6
> ("irqchip: RISC-V per-HART local interrupt controller driver"), but if the
> irqchip folks want to take the patches then I'm not opposed to either version
> getting merged.
>
> Acked-by: Palmer Dabbelt <palmerdabbelt@google.com>

Thanks, I can quickly send v6 with your comments addressed in a few hours.

Do you want me to send v6 ?

Regards,
Anup

Re: [PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Marc Zyngier]

On 2020-05-21 14:32, Anup Patel wrote:
> The RISC-V per-HART local interrupt controller manages software
> interrupts, timer interrupts, external interrupts (which are routed
> via the platform level interrupt controller) and other per-HART
> local interrupts.
> 
> This patch adds a driver for the RISC-V local interrupt controller.
> It is a major re-write over perviously submitted version.
> (Refer, https://www.spinics.net/lists/devicetree/msg241230.html)
> 
> Few advantages of this new driver over previous one are:
> 1. All local interrupts are registered as per-CPU interrupts
> 2. The RISC-V timer driver can register timer interrupt handler
>    using kernel irq subsystem without relying on arch/riscv to
>    explicitly call it's interrupt handler
> 3. The KVM RISC-V can use this driver to implement interrupt
>    handler for per-HART guest external interrupt defined by
>    the RISC-V H-Extension
> 4. In future, we can develop drivers for devices with per-HART
>    interrupts without changing arch code or this driver (example,
>    CLINT timer driver for RISC-V M-mode kernel)

Most of this is cover letter material, and doesn't belong to the 
changelog.

> 
> The RISC-V INTC driver is compliant with RISC-V Hart-Level Interrupt
> Controller DT bindings located at:
> Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
> 
> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

Is Palmer the author? If so, you're missing the 'From' line at the top.
Otherwise, you're missing a Co-developed-by tag.

> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/Kconfig                     |   1 +
>  arch/riscv/include/asm/irq.h           |   2 -
>  arch/riscv/kernel/irq.c                |  33 +-----
>  arch/riscv/kernel/traps.c              |   2 -
>  drivers/irqchip/Kconfig                |  13 ++
>  drivers/irqchip/Makefile               |   1 +
>  drivers/irqchip/irq-riscv-intc.c       | 158 +++++++++++++++++++++++++
>  drivers/irqchip/irq-sifive-plic.c      |  38 +++++-
>  include/linux/cpuhotplug.h             |   1 +
>  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
>  10 files changed, 229 insertions(+), 40 deletions(-)
>  create mode 100644 drivers/irqchip/irq-riscv-intc.c
>  create mode 100644 include/linux/irqchip/irq-riscv-intc.h
> 
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 90a008e28f7e..822cb0e1a380 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -40,6 +40,7 @@ config RISCV
>  	select HAVE_PERF_REGS
>  	select HAVE_PERF_USER_STACK_DUMP
>  	select HAVE_SYSCALL_TRACEPOINTS
> +	select HANDLE_DOMAIN_IRQ
>  	select IRQ_DOMAIN
>  	select SPARSE_IRQ
>  	select SYSCTL_EXCEPTION_TRACE
> diff --git a/arch/riscv/include/asm/irq.h 
> b/arch/riscv/include/asm/irq.h
> index 0183e15ace66..a9e5f07a7e9c 100644
> --- a/arch/riscv/include/asm/irq.h
> +++ b/arch/riscv/include/asm/irq.h
> @@ -10,8 +10,6 @@
>  #include <linux/interrupt.h>
>  #include <linux/linkage.h>
> 
> -#define NR_IRQS         0
> -
>  void riscv_timer_interrupt(void);
> 
>  #include <asm-generic/irq.h>
> diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> index bb0bfcd537e7..eb8777642ce6 100644
> --- a/arch/riscv/kernel/irq.c
> +++ b/arch/riscv/kernel/irq.c
> @@ -7,7 +7,6 @@
> 
>  #include <linux/interrupt.h>
>  #include <linux/irqchip.h>
> -#include <linux/irqdomain.h>
>  #include <linux/seq_file.h>
>  #include <asm/smp.h>
> 
> @@ -19,39 +18,13 @@ int arch_show_interrupts(struct seq_file *p, int 
> prec)
> 
>  asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
>  {
> -	struct pt_regs *old_regs;
> -
> -	switch (regs->cause & ~CAUSE_IRQ_FLAG) {
> -	case RV_IRQ_TIMER:
> -		old_regs = set_irq_regs(regs);
> -		irq_enter();
> -		riscv_timer_interrupt();
> -		irq_exit();
> -		set_irq_regs(old_regs);
> -		break;
> -#ifdef CONFIG_SMP
> -	case RV_IRQ_SOFT:
> -		/*
> -		 * We only use software interrupts to pass IPIs, so if a non-SMP
> -		 * system gets one, then we don't know what to do.
> -		 */
> -		handle_IPI(regs);
> -		break;
> -#endif
> -	case RV_IRQ_EXT:
> -		old_regs = set_irq_regs(regs);
> -		irq_enter();
> +	if (handle_arch_irq)
>  		handle_arch_irq(regs);
> -		irq_exit();
> -		set_irq_regs(old_regs);
> -		break;
> -	default:
> -		pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
> -		BUG();
> -	}
>  }
> 
>  void __init init_IRQ(void)
>  {
>  	irqchip_init();
> +	if (!handle_arch_irq)
> +		panic("No interrupt controller found.");
>  }
> diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> index 7f58fa53033f..f48c76aadbf3 100644
> --- a/arch/riscv/kernel/traps.c
> +++ b/arch/riscv/kernel/traps.c
> @@ -178,6 +178,4 @@ void trap_init(void)
>  	csr_write(CSR_SCRATCH, 0);
>  	/* Set the exception vector address */
>  	csr_write(CSR_TVEC, &handle_exception);
> -	/* Enable interrupts */
> -	csr_write(CSR_IE, IE_SIE);
>  }
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index a85aada04a64..95d6137a8117 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -493,6 +493,19 @@ config TI_SCI_INTA_IRQCHIP
>  	  If you wish to use interrupt aggregator irq resources managed by 
> the
>  	  TI System Controller, say Y here. Otherwise, say N.
> 
> +config RISCV_INTC
> +	bool "RISC-V Local Interrupt Controller"
> +	depends on RISCV
> +	default y
> +	help
> +	   This enables support for the per-HART local interrupt controller
> +	   found in standard RISC-V systems.  The per-HART local interrupt
> +	   controller handles timer interrupts, software interrupts, and
> +	   hardware interrupts. Without a per-HART local interrupt 
> controller,
> +	   a RISC-V system will be unable to handle any interrupts.
> +
> +	   If you don't know what to do here, say Y.
> +
>  config SIFIVE_PLIC
>  	bool "SiFive Platform-Level Interrupt Controller"
>  	depends on RISCV
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index 37bbe39bf909..b8319f045472 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -98,6 +98,7 @@ obj-$(CONFIG_NDS32)			+= irq-ativic32.o
>  obj-$(CONFIG_QCOM_PDC)			+= qcom-pdc.o
>  obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
>  obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
> +obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
>  obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
>  obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
>  obj-$(CONFIG_IMX_INTMUX)		+= irq-imx-intmux.o
> diff --git a/drivers/irqchip/irq-riscv-intc.c 
> b/drivers/irqchip/irq-riscv-intc.c
> new file mode 100644
> index 000000000000..2f364e6a87f9
> --- /dev/null
> +++ b/drivers/irqchip/irq-riscv-intc.c
> @@ -0,0 +1,158 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2012 Regents of the University of California
> + * Copyright (C) 2017-2018 SiFive
> + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> + */
> +
> +#define pr_fmt(fmt) "riscv-intc: " fmt
> +#include <linux/atomic.h>
> +#include <linux/bits.h>
> +#include <linux/cpu.h>
> +#include <linux/irq.h>
> +#include <linux/irqchip.h>
> +#include <linux/irqchip/irq-riscv-intc.h>
> +#include <linux/irqdomain.h>
> +#include <linux/interrupt.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/smp.h>
> +
> +static struct device_node *intc_domain_node;
> +static struct irq_domain *intc_domain;
> +
> +static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
> +{
> +	struct pt_regs *old_regs;
> +	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
> +
> +	if (unlikely(cause >= BITS_PER_LONG))
> +		panic("unexpected interrupt cause");
> +
> +	switch (cause) {
> +	case RV_IRQ_TIMER:
> +		old_regs = set_irq_regs(regs);
> +		irq_enter();
> +		riscv_timer_interrupt();
> +		irq_exit();
> +		set_irq_regs(old_regs);
> +		break;
> +#ifdef CONFIG_SMP
> +	case RV_IRQ_SOFT:
> +		/*
> +		 * We only use software interrupts to pass IPIs, so if a
> +		 * non-SMP system gets one, then we don't know what to do.
> +		 */
> +		handle_IPI(regs);
> +		break;
> +#endif
> +	default:
> +		handle_domain_irq(intc_domain, cause, regs);
> +		break;
> +	}
> +}
> +
> +/*
> + * On RISC-V systems local interrupts are masked or unmasked by 
> writing
> + * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be 
> written
> + * on the local hart, these functions can only be called on the hart 
> that
> + * corresponds to the IRQ chip.
> + */
> +
> +static void riscv_intc_irq_mask(struct irq_data *d)
> +{
> +	csr_clear(CSR_IE, 1 << (long)d->hwirq);

It feels pretty odd to cast a shift to a signed value. In any case, this
would look better as BIT(d->hwirq), all over the place.

> +}
> +
> +static void riscv_intc_irq_unmask(struct irq_data *d)
> +{
> +	csr_set(CSR_IE, 1 << (long)d->hwirq);
> +}
> +
> +static int riscv_intc_cpu_starting(unsigned int cpu)
> +{
> +	csr_write(CSR_IE, 1UL << RV_IRQ_SOFT);
> +	csr_write(CSR_IP, 0);
> +	return 0;
> +}
> +
> +static int riscv_intc_cpu_dying(unsigned int cpu)
> +{
> +	csr_clear(CSR_IE, 1UL << RV_IRQ_SOFT);
> +	return 0;
> +}
> +
> +static struct irq_chip riscv_intc_chip = {
> +	.name = "RISC-V INTC",
> +	.irq_mask = riscv_intc_irq_mask,
> +	.irq_unmask = riscv_intc_irq_unmask,
> +};
> +
> +static int riscv_intc_domain_map(struct irq_domain *d, unsigned int 
> irq,
> +				 irq_hw_number_t hwirq)
> +{
> +	irq_set_percpu_devid(irq);
> +	irq_domain_set_info(d, irq, hwirq, &riscv_intc_chip, d->host_data,
> +			    handle_percpu_devid_irq, NULL, NULL);
> +	irq_set_status_flags(irq, IRQ_NOAUTOEN);

As I was recently reminded, percpu_devid implies NO_AUTOEN.
This is thus supperfluous.

> +
> +	return 0;
> +}
> +
> +static const struct irq_domain_ops riscv_intc_domain_ops = {
> +	.map	= riscv_intc_domain_map,
> +	.xlate	= irq_domain_xlate_onecell,
> +};
> +
> +/* Get the OF device node used by INTC irq domain */
> +struct device_node *riscv_of_intc_domain_node(void)
> +{
> +	return intc_domain_node;
> +}
> +EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);

Why do you need this? Why can't the timer node refer to its
interrupt-parent? The irqchip shouldn't be in the business of
working around DT issues.

At worse, use the default irqdomain if you must, but please
avoid this kind of construct.

> +
> +static int __init riscv_intc_init(struct device_node *node,
> +				  struct device_node *parent)
> +{
> +	int hartid;
> +
> +	/*
> +	 * The DT will have one INTC DT node under each CPU (or HART)
> +	 * DT node so riscv_intc_init() function will be called once
> +	 * for each INTC DT node. We only need INTC initialization for
> +	 * the INTC DT node belonging to boot CPU (or boot HART).
> +	 */
> +	hartid = riscv_of_parent_hartid(node);
> +	if (hartid < 0)
> +		return 0;
> +	if (riscv_hartid_to_cpuid(hartid) != smp_processor_id())
> +		return 0;
> +
> +	intc_domain = irq_domain_add_linear(node, BITS_PER_LONG,
> +					    &riscv_intc_domain_ops, NULL);
> +	if (!intc_domain)
> +		goto error_add_linear;
> +
> +	/*
> +	 * We save the DT node used for creating irq domain and provide
> +	 * it to other drivers using iscv_of_intc_domain_node() function.
> +	 */
> +	intc_domain_node = of_node_get(node);
> +
> +	set_handle_irq(&riscv_intc_irq);
> +
> +	cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_STARTING,
> +			  "irqchip/riscv/intc:starting",
> +			  riscv_intc_cpu_starting,
> +			  riscv_intc_cpu_dying);
> +
> +	pr_info("%lu local interrupts mapped\n", (long)BITS_PER_LONG);

Signed or unsigned? Pick one.

> +	return 0;
> +
> +error_add_linear:
> +	pr_warn("unable to add IRQ domain\n");
> +	return -ENXIO;
> +}
> +
> +IRQCHIP_DECLARE(riscv, "riscv,cpu-intc", riscv_intc_init);
> diff --git a/drivers/irqchip/irq-sifive-plic.c
> b/drivers/irqchip/irq-sifive-plic.c
> index 16d31d114c30..4f9b2877aa9d 100644
> --- a/drivers/irqchip/irq-sifive-plic.c
> +++ b/drivers/irqchip/irq-sifive-plic.c
> @@ -9,6 +9,8 @@
>  #include <linux/io.h>
>  #include <linux/irq.h>
>  #include <linux/irqchip.h>
> +#include <linux/irqchip/chained_irq.h>
> +#include <linux/irqchip/irq-riscv-intc.h>
>  #include <linux/irqdomain.h>
>  #include <linux/module.h>
>  #include <linux/of.h>
> @@ -60,6 +62,7 @@
>  #define	PLIC_ENABLE_THRESHOLD		0
> 
>  struct plic_priv {
> +	int parent_irq;
>  	struct cpumask lmask;
>  	struct irq_domain *irqdomain;
>  	void __iomem *regs;
> @@ -219,15 +222,17 @@ static const struct irq_domain_ops 
> plic_irqdomain_ops = {
>   * that source ID back to the same claim register.  This automatically 
> enables
>   * and disables the interrupt, so there's nothing else to do.
>   */
> -static void plic_handle_irq(struct pt_regs *regs)
> +static void plic_handle_irq(struct irq_desc *desc)
>  {
>  	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> +	struct irq_chip *chip = irq_desc_get_chip(desc);
>  	void __iomem *claim = handler->hart_base + CONTEXT_CLAIM;
>  	irq_hw_number_t hwirq;
> 
>  	WARN_ON_ONCE(!handler->present);
> 
> -	csr_clear(CSR_IE, IE_EIE);
> +	chained_irq_enter(chip, desc);
> +
>  	while ((hwirq = readl(claim))) {
>  		int irq = irq_find_mapping(handler->priv->irqdomain, hwirq);
> 
> @@ -237,7 +242,8 @@ static void plic_handle_irq(struct pt_regs *regs)
>  		else
>  			generic_handle_irq(irq);
>  	}
> -	csr_set(CSR_IE, IE_EIE);
> +
> +	chained_irq_exit(chip, desc);
>  }
> 
>  static void plic_set_threshold(struct plic_handler *handler, u32 
> threshold)
> @@ -250,7 +256,8 @@ static int plic_dying_cpu(unsigned int cpu)
>  {
>  	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> 
> -	csr_clear(CSR_IE, IE_EIE);
> +	if (handler->priv->parent_irq)
> +		disable_percpu_irq(handler->priv->parent_irq);
>  	plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
> 
>  	return 0;
> @@ -259,8 +266,10 @@ static int plic_dying_cpu(unsigned int cpu)
>  static int plic_starting_cpu(unsigned int cpu)
>  {
>  	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> +	int pirq = handler->priv->parent_irq;
> 
> -	csr_set(CSR_IE, IE_EIE);
> +	if (pirq)
> +		enable_percpu_irq(pirq, irq_get_trigger_type(pirq));
>  	plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
> 
>  	return 0;
> @@ -273,6 +282,7 @@ static int __init plic_init(struct device_node 
> *node,
>  	u32 nr_irqs;
>  	struct plic_priv *priv;
>  	struct plic_handler *handler;
> +	struct of_phandle_args intc_args;
> 
>  	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
>  	if (!priv)
> @@ -354,6 +364,23 @@ static int __init plic_init(struct device_node 
> *node,
>  		nr_handlers++;
>  	}
> 
> +	/*
> +	 * We can have multiple PLIC instances so setup chained handler
> +	 * for all PLIC instances.
> +	 */
> +	intc_args.np = riscv_of_intc_domain_node();
> +	intc_args.args_count = 1;
> +	intc_args.args[0] = RV_IRQ_EXT;
> +	priv->parent_irq = irq_create_of_mapping(&intc_args);
> +	if (priv->parent_irq)
> +		irq_set_chained_handler(priv->parent_irq,
> +					 plic_handle_irq);
> +	else {

Please use braces on both sides of the 'else'.

> +		pr_err("%pOFP: intc mapping failed\n", node);
> +		error = -ENODEV;
> +		goto out_iounmap;
> +	}
> +
>  	/*
>  	 * We can have multiple PLIC instances so setup cpuhp state only
>  	 * when context handler for current/boot CPU is present.
> @@ -368,7 +395,6 @@ static int __init plic_init(struct device_node 
> *node,
> 
>  	pr_info("%pOFP: mapped %d interrupts with %d handlers for"
>  		" %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
> -	set_handle_irq(plic_handle_irq);
>  	return 0;
> 
>  out_iounmap:
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index 77d70b633531..57b1f8f777d9 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -102,6 +102,7 @@ enum cpuhp_state {
>  	CPUHP_AP_IRQ_ARMADA_XP_STARTING,
>  	CPUHP_AP_IRQ_BCM2836_STARTING,
>  	CPUHP_AP_IRQ_MIPS_GIC_STARTING,
> +	CPUHP_AP_IRQ_RISCV_STARTING,
>  	CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
>  	CPUHP_AP_ARM_MVEBU_COHERENCY,
>  	CPUHP_AP_MICROCODE_LOADER,
> diff --git a/include/linux/irqchip/irq-riscv-intc.h
> b/include/linux/irqchip/irq-riscv-intc.h
> new file mode 100644
> index 000000000000..b11d38353099
> --- /dev/null
> +++ b/include/linux/irqchip/irq-riscv-intc.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> + */
> +
> +#ifndef __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> +#define __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> +
> +struct device_node;
> +
> +#ifdef CONFIG_RISCV_INTC
> +struct device_node *riscv_of_intc_domain_node(void);
> +#else
> +static inline struct device_node *riscv_of_intc_domain_node(void)
> +{
> +	return NULL;
> +}
> +#endif
> +
> +#endif

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Marc Zyngier]

On 2020-05-21 14:32, Anup Patel wrote:
> The RISC-V per-HART local interrupt controller manages software
> interrupts, timer interrupts, external interrupts (which are routed
> via the platform level interrupt controller) and other per-HART
> local interrupts.
> 
> This patch adds a driver for the RISC-V local interrupt controller.
> It is a major re-write over perviously submitted version.
> (Refer, https://www.spinics.net/lists/devicetree/msg241230.html)
> 
> Few advantages of this new driver over previous one are:
> 1. All local interrupts are registered as per-CPU interrupts
> 2. The RISC-V timer driver can register timer interrupt handler
>    using kernel irq subsystem without relying on arch/riscv to
>    explicitly call it's interrupt handler
> 3. The KVM RISC-V can use this driver to implement interrupt
>    handler for per-HART guest external interrupt defined by
>    the RISC-V H-Extension
> 4. In future, we can develop drivers for devices with per-HART
>    interrupts without changing arch code or this driver (example,
>    CLINT timer driver for RISC-V M-mode kernel)

Most of this is cover letter material, and doesn't belong to the 
changelog.

> 
> The RISC-V INTC driver is compliant with RISC-V Hart-Level Interrupt
> Controller DT bindings located at:
> Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
> 
> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>

Is Palmer the author? If so, you're missing the 'From' line at the top.
Otherwise, you're missing a Co-developed-by tag.

> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/Kconfig                     |   1 +
>  arch/riscv/include/asm/irq.h           |   2 -
>  arch/riscv/kernel/irq.c                |  33 +-----
>  arch/riscv/kernel/traps.c              |   2 -
>  drivers/irqchip/Kconfig                |  13 ++
>  drivers/irqchip/Makefile               |   1 +
>  drivers/irqchip/irq-riscv-intc.c       | 158 +++++++++++++++++++++++++
>  drivers/irqchip/irq-sifive-plic.c      |  38 +++++-
>  include/linux/cpuhotplug.h             |   1 +
>  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
>  10 files changed, 229 insertions(+), 40 deletions(-)
>  create mode 100644 drivers/irqchip/irq-riscv-intc.c
>  create mode 100644 include/linux/irqchip/irq-riscv-intc.h
> 
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 90a008e28f7e..822cb0e1a380 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -40,6 +40,7 @@ config RISCV
>  	select HAVE_PERF_REGS
>  	select HAVE_PERF_USER_STACK_DUMP
>  	select HAVE_SYSCALL_TRACEPOINTS
> +	select HANDLE_DOMAIN_IRQ
>  	select IRQ_DOMAIN
>  	select SPARSE_IRQ
>  	select SYSCTL_EXCEPTION_TRACE
> diff --git a/arch/riscv/include/asm/irq.h 
> b/arch/riscv/include/asm/irq.h
> index 0183e15ace66..a9e5f07a7e9c 100644
> --- a/arch/riscv/include/asm/irq.h
> +++ b/arch/riscv/include/asm/irq.h
> @@ -10,8 +10,6 @@
>  #include <linux/interrupt.h>
>  #include <linux/linkage.h>
> 
> -#define NR_IRQS         0
> -
>  void riscv_timer_interrupt(void);
> 
>  #include <asm-generic/irq.h>
> diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> index bb0bfcd537e7..eb8777642ce6 100644
> --- a/arch/riscv/kernel/irq.c
> +++ b/arch/riscv/kernel/irq.c
> @@ -7,7 +7,6 @@
> 
>  #include <linux/interrupt.h>
>  #include <linux/irqchip.h>
> -#include <linux/irqdomain.h>
>  #include <linux/seq_file.h>
>  #include <asm/smp.h>
> 
> @@ -19,39 +18,13 @@ int arch_show_interrupts(struct seq_file *p, int 
> prec)
> 
>  asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
>  {
> -	struct pt_regs *old_regs;
> -
> -	switch (regs->cause & ~CAUSE_IRQ_FLAG) {
> -	case RV_IRQ_TIMER:
> -		old_regs = set_irq_regs(regs);
> -		irq_enter();
> -		riscv_timer_interrupt();
> -		irq_exit();
> -		set_irq_regs(old_regs);
> -		break;
> -#ifdef CONFIG_SMP
> -	case RV_IRQ_SOFT:
> -		/*
> -		 * We only use software interrupts to pass IPIs, so if a non-SMP
> -		 * system gets one, then we don't know what to do.
> -		 */
> -		handle_IPI(regs);
> -		break;
> -#endif
> -	case RV_IRQ_EXT:
> -		old_regs = set_irq_regs(regs);
> -		irq_enter();
> +	if (handle_arch_irq)
>  		handle_arch_irq(regs);
> -		irq_exit();
> -		set_irq_regs(old_regs);
> -		break;
> -	default:
> -		pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
> -		BUG();
> -	}
>  }
> 
>  void __init init_IRQ(void)
>  {
>  	irqchip_init();
> +	if (!handle_arch_irq)
> +		panic("No interrupt controller found.");
>  }
> diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> index 7f58fa53033f..f48c76aadbf3 100644
> --- a/arch/riscv/kernel/traps.c
> +++ b/arch/riscv/kernel/traps.c
> @@ -178,6 +178,4 @@ void trap_init(void)
>  	csr_write(CSR_SCRATCH, 0);
>  	/* Set the exception vector address */
>  	csr_write(CSR_TVEC, &handle_exception);
> -	/* Enable interrupts */
> -	csr_write(CSR_IE, IE_SIE);
>  }
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index a85aada04a64..95d6137a8117 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -493,6 +493,19 @@ config TI_SCI_INTA_IRQCHIP
>  	  If you wish to use interrupt aggregator irq resources managed by 
> the
>  	  TI System Controller, say Y here. Otherwise, say N.
> 
> +config RISCV_INTC
> +	bool "RISC-V Local Interrupt Controller"
> +	depends on RISCV
> +	default y
> +	help
> +	   This enables support for the per-HART local interrupt controller
> +	   found in standard RISC-V systems.  The per-HART local interrupt
> +	   controller handles timer interrupts, software interrupts, and
> +	   hardware interrupts. Without a per-HART local interrupt 
> controller,
> +	   a RISC-V system will be unable to handle any interrupts.
> +
> +	   If you don't know what to do here, say Y.
> +
>  config SIFIVE_PLIC
>  	bool "SiFive Platform-Level Interrupt Controller"
>  	depends on RISCV
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index 37bbe39bf909..b8319f045472 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -98,6 +98,7 @@ obj-$(CONFIG_NDS32)			+= irq-ativic32.o
>  obj-$(CONFIG_QCOM_PDC)			+= qcom-pdc.o
>  obj-$(CONFIG_CSKY_MPINTC)		+= irq-csky-mpintc.o
>  obj-$(CONFIG_CSKY_APB_INTC)		+= irq-csky-apb-intc.o
> +obj-$(CONFIG_RISCV_INTC)		+= irq-riscv-intc.o
>  obj-$(CONFIG_SIFIVE_PLIC)		+= irq-sifive-plic.o
>  obj-$(CONFIG_IMX_IRQSTEER)		+= irq-imx-irqsteer.o
>  obj-$(CONFIG_IMX_INTMUX)		+= irq-imx-intmux.o
> diff --git a/drivers/irqchip/irq-riscv-intc.c 
> b/drivers/irqchip/irq-riscv-intc.c
> new file mode 100644
> index 000000000000..2f364e6a87f9
> --- /dev/null
> +++ b/drivers/irqchip/irq-riscv-intc.c
> @@ -0,0 +1,158 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (C) 2012 Regents of the University of California
> + * Copyright (C) 2017-2018 SiFive
> + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> + */
> +
> +#define pr_fmt(fmt) "riscv-intc: " fmt
> +#include <linux/atomic.h>
> +#include <linux/bits.h>
> +#include <linux/cpu.h>
> +#include <linux/irq.h>
> +#include <linux/irqchip.h>
> +#include <linux/irqchip/irq-riscv-intc.h>
> +#include <linux/irqdomain.h>
> +#include <linux/interrupt.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/smp.h>
> +
> +static struct device_node *intc_domain_node;
> +static struct irq_domain *intc_domain;
> +
> +static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
> +{
> +	struct pt_regs *old_regs;
> +	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
> +
> +	if (unlikely(cause >= BITS_PER_LONG))
> +		panic("unexpected interrupt cause");
> +
> +	switch (cause) {
> +	case RV_IRQ_TIMER:
> +		old_regs = set_irq_regs(regs);
> +		irq_enter();
> +		riscv_timer_interrupt();
> +		irq_exit();
> +		set_irq_regs(old_regs);
> +		break;
> +#ifdef CONFIG_SMP
> +	case RV_IRQ_SOFT:
> +		/*
> +		 * We only use software interrupts to pass IPIs, so if a
> +		 * non-SMP system gets one, then we don't know what to do.
> +		 */
> +		handle_IPI(regs);
> +		break;
> +#endif
> +	default:
> +		handle_domain_irq(intc_domain, cause, regs);
> +		break;
> +	}
> +}
> +
> +/*
> + * On RISC-V systems local interrupts are masked or unmasked by 
> writing
> + * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be 
> written
> + * on the local hart, these functions can only be called on the hart 
> that
> + * corresponds to the IRQ chip.
> + */
> +
> +static void riscv_intc_irq_mask(struct irq_data *d)
> +{
> +	csr_clear(CSR_IE, 1 << (long)d->hwirq);

It feels pretty odd to cast a shift to a signed value. In any case, this
would look better as BIT(d->hwirq), all over the place.

> +}
> +
> +static void riscv_intc_irq_unmask(struct irq_data *d)
> +{
> +	csr_set(CSR_IE, 1 << (long)d->hwirq);
> +}
> +
> +static int riscv_intc_cpu_starting(unsigned int cpu)
> +{
> +	csr_write(CSR_IE, 1UL << RV_IRQ_SOFT);
> +	csr_write(CSR_IP, 0);
> +	return 0;
> +}
> +
> +static int riscv_intc_cpu_dying(unsigned int cpu)
> +{
> +	csr_clear(CSR_IE, 1UL << RV_IRQ_SOFT);
> +	return 0;
> +}
> +
> +static struct irq_chip riscv_intc_chip = {
> +	.name = "RISC-V INTC",
> +	.irq_mask = riscv_intc_irq_mask,
> +	.irq_unmask = riscv_intc_irq_unmask,
> +};
> +
> +static int riscv_intc_domain_map(struct irq_domain *d, unsigned int 
> irq,
> +				 irq_hw_number_t hwirq)
> +{
> +	irq_set_percpu_devid(irq);
> +	irq_domain_set_info(d, irq, hwirq, &riscv_intc_chip, d->host_data,
> +			    handle_percpu_devid_irq, NULL, NULL);
> +	irq_set_status_flags(irq, IRQ_NOAUTOEN);

As I was recently reminded, percpu_devid implies NO_AUTOEN.
This is thus supperfluous.

> +
> +	return 0;
> +}
> +
> +static const struct irq_domain_ops riscv_intc_domain_ops = {
> +	.map	= riscv_intc_domain_map,
> +	.xlate	= irq_domain_xlate_onecell,
> +};
> +
> +/* Get the OF device node used by INTC irq domain */
> +struct device_node *riscv_of_intc_domain_node(void)
> +{
> +	return intc_domain_node;
> +}
> +EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);

Why do you need this? Why can't the timer node refer to its
interrupt-parent? The irqchip shouldn't be in the business of
working around DT issues.

At worse, use the default irqdomain if you must, but please
avoid this kind of construct.

> +
> +static int __init riscv_intc_init(struct device_node *node,
> +				  struct device_node *parent)
> +{
> +	int hartid;
> +
> +	/*
> +	 * The DT will have one INTC DT node under each CPU (or HART)
> +	 * DT node so riscv_intc_init() function will be called once
> +	 * for each INTC DT node. We only need INTC initialization for
> +	 * the INTC DT node belonging to boot CPU (or boot HART).
> +	 */
> +	hartid = riscv_of_parent_hartid(node);
> +	if (hartid < 0)
> +		return 0;
> +	if (riscv_hartid_to_cpuid(hartid) != smp_processor_id())
> +		return 0;
> +
> +	intc_domain = irq_domain_add_linear(node, BITS_PER_LONG,
> +					    &riscv_intc_domain_ops, NULL);
> +	if (!intc_domain)
> +		goto error_add_linear;
> +
> +	/*
> +	 * We save the DT node used for creating irq domain and provide
> +	 * it to other drivers using iscv_of_intc_domain_node() function.
> +	 */
> +	intc_domain_node = of_node_get(node);
> +
> +	set_handle_irq(&riscv_intc_irq);
> +
> +	cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_STARTING,
> +			  "irqchip/riscv/intc:starting",
> +			  riscv_intc_cpu_starting,
> +			  riscv_intc_cpu_dying);
> +
> +	pr_info("%lu local interrupts mapped\n", (long)BITS_PER_LONG);

Signed or unsigned? Pick one.

> +	return 0;
> +
> +error_add_linear:
> +	pr_warn("unable to add IRQ domain\n");
> +	return -ENXIO;
> +}
> +
> +IRQCHIP_DECLARE(riscv, "riscv,cpu-intc", riscv_intc_init);
> diff --git a/drivers/irqchip/irq-sifive-plic.c
> b/drivers/irqchip/irq-sifive-plic.c
> index 16d31d114c30..4f9b2877aa9d 100644
> --- a/drivers/irqchip/irq-sifive-plic.c
> +++ b/drivers/irqchip/irq-sifive-plic.c
> @@ -9,6 +9,8 @@
>  #include <linux/io.h>
>  #include <linux/irq.h>
>  #include <linux/irqchip.h>
> +#include <linux/irqchip/chained_irq.h>
> +#include <linux/irqchip/irq-riscv-intc.h>
>  #include <linux/irqdomain.h>
>  #include <linux/module.h>
>  #include <linux/of.h>
> @@ -60,6 +62,7 @@
>  #define	PLIC_ENABLE_THRESHOLD		0
> 
>  struct plic_priv {
> +	int parent_irq;
>  	struct cpumask lmask;
>  	struct irq_domain *irqdomain;
>  	void __iomem *regs;
> @@ -219,15 +222,17 @@ static const struct irq_domain_ops 
> plic_irqdomain_ops = {
>   * that source ID back to the same claim register.  This automatically 
> enables
>   * and disables the interrupt, so there's nothing else to do.
>   */
> -static void plic_handle_irq(struct pt_regs *regs)
> +static void plic_handle_irq(struct irq_desc *desc)
>  {
>  	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> +	struct irq_chip *chip = irq_desc_get_chip(desc);
>  	void __iomem *claim = handler->hart_base + CONTEXT_CLAIM;
>  	irq_hw_number_t hwirq;
> 
>  	WARN_ON_ONCE(!handler->present);
> 
> -	csr_clear(CSR_IE, IE_EIE);
> +	chained_irq_enter(chip, desc);
> +
>  	while ((hwirq = readl(claim))) {
>  		int irq = irq_find_mapping(handler->priv->irqdomain, hwirq);
> 
> @@ -237,7 +242,8 @@ static void plic_handle_irq(struct pt_regs *regs)
>  		else
>  			generic_handle_irq(irq);
>  	}
> -	csr_set(CSR_IE, IE_EIE);
> +
> +	chained_irq_exit(chip, desc);
>  }
> 
>  static void plic_set_threshold(struct plic_handler *handler, u32 
> threshold)
> @@ -250,7 +256,8 @@ static int plic_dying_cpu(unsigned int cpu)
>  {
>  	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> 
> -	csr_clear(CSR_IE, IE_EIE);
> +	if (handler->priv->parent_irq)
> +		disable_percpu_irq(handler->priv->parent_irq);
>  	plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
> 
>  	return 0;
> @@ -259,8 +266,10 @@ static int plic_dying_cpu(unsigned int cpu)
>  static int plic_starting_cpu(unsigned int cpu)
>  {
>  	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> +	int pirq = handler->priv->parent_irq;
> 
> -	csr_set(CSR_IE, IE_EIE);
> +	if (pirq)
> +		enable_percpu_irq(pirq, irq_get_trigger_type(pirq));
>  	plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
> 
>  	return 0;
> @@ -273,6 +282,7 @@ static int __init plic_init(struct device_node 
> *node,
>  	u32 nr_irqs;
>  	struct plic_priv *priv;
>  	struct plic_handler *handler;
> +	struct of_phandle_args intc_args;
> 
>  	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
>  	if (!priv)
> @@ -354,6 +364,23 @@ static int __init plic_init(struct device_node 
> *node,
>  		nr_handlers++;
>  	}
> 
> +	/*
> +	 * We can have multiple PLIC instances so setup chained handler
> +	 * for all PLIC instances.
> +	 */
> +	intc_args.np = riscv_of_intc_domain_node();
> +	intc_args.args_count = 1;
> +	intc_args.args[0] = RV_IRQ_EXT;
> +	priv->parent_irq = irq_create_of_mapping(&intc_args);
> +	if (priv->parent_irq)
> +		irq_set_chained_handler(priv->parent_irq,
> +					 plic_handle_irq);
> +	else {

Please use braces on both sides of the 'else'.

> +		pr_err("%pOFP: intc mapping failed\n", node);
> +		error = -ENODEV;
> +		goto out_iounmap;
> +	}
> +
>  	/*
>  	 * We can have multiple PLIC instances so setup cpuhp state only
>  	 * when context handler for current/boot CPU is present.
> @@ -368,7 +395,6 @@ static int __init plic_init(struct device_node 
> *node,
> 
>  	pr_info("%pOFP: mapped %d interrupts with %d handlers for"
>  		" %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
> -	set_handle_irq(plic_handle_irq);
>  	return 0;
> 
>  out_iounmap:
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index 77d70b633531..57b1f8f777d9 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -102,6 +102,7 @@ enum cpuhp_state {
>  	CPUHP_AP_IRQ_ARMADA_XP_STARTING,
>  	CPUHP_AP_IRQ_BCM2836_STARTING,
>  	CPUHP_AP_IRQ_MIPS_GIC_STARTING,
> +	CPUHP_AP_IRQ_RISCV_STARTING,
>  	CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
>  	CPUHP_AP_ARM_MVEBU_COHERENCY,
>  	CPUHP_AP_MICROCODE_LOADER,
> diff --git a/include/linux/irqchip/irq-riscv-intc.h
> b/include/linux/irqchip/irq-riscv-intc.h
> new file mode 100644
> index 000000000000..b11d38353099
> --- /dev/null
> +++ b/include/linux/irqchip/irq-riscv-intc.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> + */
> +
> +#ifndef __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> +#define __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> +
> +struct device_node;
> +
> +#ifdef CONFIG_RISCV_INTC
> +struct device_node *riscv_of_intc_domain_node(void);
> +#else
> +static inline struct device_node *riscv_of_intc_domain_node(void)
> +{
> +	return NULL;
> +}
> +#endif
> +
> +#endif

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Anup Patel]

On Fri, May 29, 2020 at 3:39 PM Marc Zyngier <maz@kernel.org> wrote:
>
> On 2020-05-21 14:32, Anup Patel wrote:
> > The RISC-V per-HART local interrupt controller manages software
> > interrupts, timer interrupts, external interrupts (which are routed
> > via the platform level interrupt controller) and other per-HART
> > local interrupts.
> >
> > This patch adds a driver for the RISC-V local interrupt controller.
> > It is a major re-write over perviously submitted version.
> > (Refer, https://www.spinics.net/lists/devicetree/msg241230.html)
> >
> > Few advantages of this new driver over previous one are:
> > 1. All local interrupts are registered as per-CPU interrupts
> > 2. The RISC-V timer driver can register timer interrupt handler
> >    using kernel irq subsystem without relying on arch/riscv to
> >    explicitly call it's interrupt handler
> > 3. The KVM RISC-V can use this driver to implement interrupt
> >    handler for per-HART guest external interrupt defined by
> >    the RISC-V H-Extension
> > 4. In future, we can develop drivers for devices with per-HART
> >    interrupts without changing arch code or this driver (example,
> >    CLINT timer driver for RISC-V M-mode kernel)
>
> Most of this is cover letter material, and doesn't belong to the
> changelog.

Okay, I will drop this from commit description.

>
> >
> > The RISC-V INTC driver is compliant with RISC-V Hart-Level Interrupt
> > Controller DT bindings located at:
> > Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
> >
> > Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
>
> Is Palmer the author? If so, you're missing the 'From' line at the top.
> Otherwise, you're missing a Co-developed-by tag.

Palmer had submitted the first implementation of RISC-V INTC driver
more than 2 years back. That driver had separate IRQ domain for
each CPU and did not use per-CPU interrupts.

The driver added by this patch is re-write to have only one IRQ
domain with all per-CPU interrupts.

I kept Palmer's SoB to acknowledge his initial efforts. I will add
Co-developed-by tag like you suggested.

>
> > Signed-off-by: Anup Patel <anup.patel@wdc.com>
> > ---
> >  arch/riscv/Kconfig                     |   1 +
> >  arch/riscv/include/asm/irq.h           |   2 -
> >  arch/riscv/kernel/irq.c                |  33 +-----
> >  arch/riscv/kernel/traps.c              |   2 -
> >  drivers/irqchip/Kconfig                |  13 ++
> >  drivers/irqchip/Makefile               |   1 +
> >  drivers/irqchip/irq-riscv-intc.c       | 158 +++++++++++++++++++++++++
> >  drivers/irqchip/irq-sifive-plic.c      |  38 +++++-
> >  include/linux/cpuhotplug.h             |   1 +
> >  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
> >  10 files changed, 229 insertions(+), 40 deletions(-)
> >  create mode 100644 drivers/irqchip/irq-riscv-intc.c
> >  create mode 100644 include/linux/irqchip/irq-riscv-intc.h
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 90a008e28f7e..822cb0e1a380 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -40,6 +40,7 @@ config RISCV
> >       select HAVE_PERF_REGS
> >       select HAVE_PERF_USER_STACK_DUMP
> >       select HAVE_SYSCALL_TRACEPOINTS
> > +     select HANDLE_DOMAIN_IRQ
> >       select IRQ_DOMAIN
> >       select SPARSE_IRQ
> >       select SYSCTL_EXCEPTION_TRACE
> > diff --git a/arch/riscv/include/asm/irq.h
> > b/arch/riscv/include/asm/irq.h
> > index 0183e15ace66..a9e5f07a7e9c 100644
> > --- a/arch/riscv/include/asm/irq.h
> > +++ b/arch/riscv/include/asm/irq.h
> > @@ -10,8 +10,6 @@
> >  #include <linux/interrupt.h>
> >  #include <linux/linkage.h>
> >
> > -#define NR_IRQS         0
> > -
> >  void riscv_timer_interrupt(void);
> >
> >  #include <asm-generic/irq.h>
> > diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> > index bb0bfcd537e7..eb8777642ce6 100644
> > --- a/arch/riscv/kernel/irq.c
> > +++ b/arch/riscv/kernel/irq.c
> > @@ -7,7 +7,6 @@
> >
> >  #include <linux/interrupt.h>
> >  #include <linux/irqchip.h>
> > -#include <linux/irqdomain.h>
> >  #include <linux/seq_file.h>
> >  #include <asm/smp.h>
> >
> > @@ -19,39 +18,13 @@ int arch_show_interrupts(struct seq_file *p, int
> > prec)
> >
> >  asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
> >  {
> > -     struct pt_regs *old_regs;
> > -
> > -     switch (regs->cause & ~CAUSE_IRQ_FLAG) {
> > -     case RV_IRQ_TIMER:
> > -             old_regs = set_irq_regs(regs);
> > -             irq_enter();
> > -             riscv_timer_interrupt();
> > -             irq_exit();
> > -             set_irq_regs(old_regs);
> > -             break;
> > -#ifdef CONFIG_SMP
> > -     case RV_IRQ_SOFT:
> > -             /*
> > -              * We only use software interrupts to pass IPIs, so if a non-SMP
> > -              * system gets one, then we don't know what to do.
> > -              */
> > -             handle_IPI(regs);
> > -             break;
> > -#endif
> > -     case RV_IRQ_EXT:
> > -             old_regs = set_irq_regs(regs);
> > -             irq_enter();
> > +     if (handle_arch_irq)
> >               handle_arch_irq(regs);
> > -             irq_exit();
> > -             set_irq_regs(old_regs);
> > -             break;
> > -     default:
> > -             pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
> > -             BUG();
> > -     }
> >  }
> >
> >  void __init init_IRQ(void)
> >  {
> >       irqchip_init();
> > +     if (!handle_arch_irq)
> > +             panic("No interrupt controller found.");
> >  }
> > diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> > index 7f58fa53033f..f48c76aadbf3 100644
> > --- a/arch/riscv/kernel/traps.c
> > +++ b/arch/riscv/kernel/traps.c
> > @@ -178,6 +178,4 @@ void trap_init(void)
> >       csr_write(CSR_SCRATCH, 0);
> >       /* Set the exception vector address */
> >       csr_write(CSR_TVEC, &handle_exception);
> > -     /* Enable interrupts */
> > -     csr_write(CSR_IE, IE_SIE);
> >  }
> > diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> > index a85aada04a64..95d6137a8117 100644
> > --- a/drivers/irqchip/Kconfig
> > +++ b/drivers/irqchip/Kconfig
> > @@ -493,6 +493,19 @@ config TI_SCI_INTA_IRQCHIP
> >         If you wish to use interrupt aggregator irq resources managed by
> > the
> >         TI System Controller, say Y here. Otherwise, say N.
> >
> > +config RISCV_INTC
> > +     bool "RISC-V Local Interrupt Controller"
> > +     depends on RISCV
> > +     default y
> > +     help
> > +        This enables support for the per-HART local interrupt controller
> > +        found in standard RISC-V systems.  The per-HART local interrupt
> > +        controller handles timer interrupts, software interrupts, and
> > +        hardware interrupts. Without a per-HART local interrupt
> > controller,
> > +        a RISC-V system will be unable to handle any interrupts.
> > +
> > +        If you don't know what to do here, say Y.
> > +
> >  config SIFIVE_PLIC
> >       bool "SiFive Platform-Level Interrupt Controller"
> >       depends on RISCV
> > diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> > index 37bbe39bf909..b8319f045472 100644
> > --- a/drivers/irqchip/Makefile
> > +++ b/drivers/irqchip/Makefile
> > @@ -98,6 +98,7 @@ obj-$(CONFIG_NDS32)                 += irq-ativic32.o
> >  obj-$(CONFIG_QCOM_PDC)                       += qcom-pdc.o
> >  obj-$(CONFIG_CSKY_MPINTC)            += irq-csky-mpintc.o
> >  obj-$(CONFIG_CSKY_APB_INTC)          += irq-csky-apb-intc.o
> > +obj-$(CONFIG_RISCV_INTC)             += irq-riscv-intc.o
> >  obj-$(CONFIG_SIFIVE_PLIC)            += irq-sifive-plic.o
> >  obj-$(CONFIG_IMX_IRQSTEER)           += irq-imx-irqsteer.o
> >  obj-$(CONFIG_IMX_INTMUX)             += irq-imx-intmux.o
> > diff --git a/drivers/irqchip/irq-riscv-intc.c
> > b/drivers/irqchip/irq-riscv-intc.c
> > new file mode 100644
> > index 000000000000..2f364e6a87f9
> > --- /dev/null
> > +++ b/drivers/irqchip/irq-riscv-intc.c
> > @@ -0,0 +1,158 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (C) 2012 Regents of the University of California
> > + * Copyright (C) 2017-2018 SiFive
> > + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> > + */
> > +
> > +#define pr_fmt(fmt) "riscv-intc: " fmt
> > +#include <linux/atomic.h>
> > +#include <linux/bits.h>
> > +#include <linux/cpu.h>
> > +#include <linux/irq.h>
> > +#include <linux/irqchip.h>
> > +#include <linux/irqchip/irq-riscv-intc.h>
> > +#include <linux/irqdomain.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/module.h>
> > +#include <linux/of.h>
> > +#include <linux/smp.h>
> > +
> > +static struct device_node *intc_domain_node;
> > +static struct irq_domain *intc_domain;
> > +
> > +static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
> > +{
> > +     struct pt_regs *old_regs;
> > +     unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
> > +
> > +     if (unlikely(cause >= BITS_PER_LONG))
> > +             panic("unexpected interrupt cause");
> > +
> > +     switch (cause) {
> > +     case RV_IRQ_TIMER:
> > +             old_regs = set_irq_regs(regs);
> > +             irq_enter();
> > +             riscv_timer_interrupt();
> > +             irq_exit();
> > +             set_irq_regs(old_regs);
> > +             break;
> > +#ifdef CONFIG_SMP
> > +     case RV_IRQ_SOFT:
> > +             /*
> > +              * We only use software interrupts to pass IPIs, so if a
> > +              * non-SMP system gets one, then we don't know what to do.
> > +              */
> > +             handle_IPI(regs);
> > +             break;
> > +#endif
> > +     default:
> > +             handle_domain_irq(intc_domain, cause, regs);
> > +             break;
> > +     }
> > +}
> > +
> > +/*
> > + * On RISC-V systems local interrupts are masked or unmasked by
> > writing
> > + * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be
> > written
> > + * on the local hart, these functions can only be called on the hart
> > that
> > + * corresponds to the IRQ chip.
> > + */
> > +
> > +static void riscv_intc_irq_mask(struct irq_data *d)
> > +{
> > +     csr_clear(CSR_IE, 1 << (long)d->hwirq);
>
> It feels pretty odd to cast a shift to a signed value. In any case, this
> would look better as BIT(d->hwirq), all over the place.
>
> > +}
> > +
> > +static void riscv_intc_irq_unmask(struct irq_data *d)
> > +{
> > +     csr_set(CSR_IE, 1 << (long)d->hwirq);
> > +}
> > +
> > +static int riscv_intc_cpu_starting(unsigned int cpu)
> > +{
> > +     csr_write(CSR_IE, 1UL << RV_IRQ_SOFT);
> > +     csr_write(CSR_IP, 0);
> > +     return 0;
> > +}
> > +
> > +static int riscv_intc_cpu_dying(unsigned int cpu)
> > +{
> > +     csr_clear(CSR_IE, 1UL << RV_IRQ_SOFT);
> > +     return 0;
> > +}
> > +
> > +static struct irq_chip riscv_intc_chip = {
> > +     .name = "RISC-V INTC",
> > +     .irq_mask = riscv_intc_irq_mask,
> > +     .irq_unmask = riscv_intc_irq_unmask,
> > +};
> > +
> > +static int riscv_intc_domain_map(struct irq_domain *d, unsigned int
> > irq,
> > +                              irq_hw_number_t hwirq)
> > +{
> > +     irq_set_percpu_devid(irq);
> > +     irq_domain_set_info(d, irq, hwirq, &riscv_intc_chip, d->host_data,
> > +                         handle_percpu_devid_irq, NULL, NULL);
> > +     irq_set_status_flags(irq, IRQ_NOAUTOEN);
>
> As I was recently reminded, percpu_devid implies NO_AUTOEN.
> This is thus supperfluous.

Okay, I will drop this line.

>
> > +
> > +     return 0;
> > +}
> > +
> > +static const struct irq_domain_ops riscv_intc_domain_ops = {
> > +     .map    = riscv_intc_domain_map,
> > +     .xlate  = irq_domain_xlate_onecell,
> > +};
> > +
> > +/* Get the OF device node used by INTC irq domain */
> > +struct device_node *riscv_of_intc_domain_node(void)
> > +{
> > +     return intc_domain_node;
> > +}
> > +EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);
>
> Why do you need this? Why can't the timer node refer to its
> interrupt-parent? The irqchip shouldn't be in the business of
> working around DT issues.
>
> At worse, use the default irqdomain if you must, but please
> avoid this kind of construct.

Even, I don't like exporting riscv_of_intc_domain_node().

Thanks for your suggestion, I will certainly use the default irqdomain.

>
> > +
> > +static int __init riscv_intc_init(struct device_node *node,
> > +                               struct device_node *parent)
> > +{
> > +     int hartid;
> > +
> > +     /*
> > +      * The DT will have one INTC DT node under each CPU (or HART)
> > +      * DT node so riscv_intc_init() function will be called once
> > +      * for each INTC DT node. We only need INTC initialization for
> > +      * the INTC DT node belonging to boot CPU (or boot HART).
> > +      */
> > +     hartid = riscv_of_parent_hartid(node);
> > +     if (hartid < 0)
> > +             return 0;
> > +     if (riscv_hartid_to_cpuid(hartid) != smp_processor_id())
> > +             return 0;
> > +
> > +     intc_domain = irq_domain_add_linear(node, BITS_PER_LONG,
> > +                                         &riscv_intc_domain_ops, NULL);
> > +     if (!intc_domain)
> > +             goto error_add_linear;
> > +
> > +     /*
> > +      * We save the DT node used for creating irq domain and provide
> > +      * it to other drivers using iscv_of_intc_domain_node() function.
> > +      */
> > +     intc_domain_node = of_node_get(node);
> > +
> > +     set_handle_irq(&riscv_intc_irq);
> > +
> > +     cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_STARTING,
> > +                       "irqchip/riscv/intc:starting",
> > +                       riscv_intc_cpu_starting,
> > +                       riscv_intc_cpu_dying);
> > +
> > +     pr_info("%lu local interrupts mapped\n", (long)BITS_PER_LONG);
>
> Signed or unsigned? Pick one.

Okay, will fix it.

>
> > +     return 0;
> > +
> > +error_add_linear:
> > +     pr_warn("unable to add IRQ domain\n");
> > +     return -ENXIO;
> > +}
> > +
> > +IRQCHIP_DECLARE(riscv, "riscv,cpu-intc", riscv_intc_init);
> > diff --git a/drivers/irqchip/irq-sifive-plic.c
> > b/drivers/irqchip/irq-sifive-plic.c
> > index 16d31d114c30..4f9b2877aa9d 100644
> > --- a/drivers/irqchip/irq-sifive-plic.c
> > +++ b/drivers/irqchip/irq-sifive-plic.c
> > @@ -9,6 +9,8 @@
> >  #include <linux/io.h>
> >  #include <linux/irq.h>
> >  #include <linux/irqchip.h>
> > +#include <linux/irqchip/chained_irq.h>
> > +#include <linux/irqchip/irq-riscv-intc.h>
> >  #include <linux/irqdomain.h>
> >  #include <linux/module.h>
> >  #include <linux/of.h>
> > @@ -60,6 +62,7 @@
> >  #define      PLIC_ENABLE_THRESHOLD           0
> >
> >  struct plic_priv {
> > +     int parent_irq;
> >       struct cpumask lmask;
> >       struct irq_domain *irqdomain;
> >       void __iomem *regs;
> > @@ -219,15 +222,17 @@ static const struct irq_domain_ops
> > plic_irqdomain_ops = {
> >   * that source ID back to the same claim register.  This automatically
> > enables
> >   * and disables the interrupt, so there's nothing else to do.
> >   */
> > -static void plic_handle_irq(struct pt_regs *regs)
> > +static void plic_handle_irq(struct irq_desc *desc)
> >  {
> >       struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> > +     struct irq_chip *chip = irq_desc_get_chip(desc);
> >       void __iomem *claim = handler->hart_base + CONTEXT_CLAIM;
> >       irq_hw_number_t hwirq;
> >
> >       WARN_ON_ONCE(!handler->present);
> >
> > -     csr_clear(CSR_IE, IE_EIE);
> > +     chained_irq_enter(chip, desc);
> > +
> >       while ((hwirq = readl(claim))) {
> >               int irq = irq_find_mapping(handler->priv->irqdomain, hwirq);
> >
> > @@ -237,7 +242,8 @@ static void plic_handle_irq(struct pt_regs *regs)
> >               else
> >                       generic_handle_irq(irq);
> >       }
> > -     csr_set(CSR_IE, IE_EIE);
> > +
> > +     chained_irq_exit(chip, desc);
> >  }
> >
> >  static void plic_set_threshold(struct plic_handler *handler, u32
> > threshold)
> > @@ -250,7 +256,8 @@ static int plic_dying_cpu(unsigned int cpu)
> >  {
> >       struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> >
> > -     csr_clear(CSR_IE, IE_EIE);
> > +     if (handler->priv->parent_irq)
> > +             disable_percpu_irq(handler->priv->parent_irq);
> >       plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
> >
> >       return 0;
> > @@ -259,8 +266,10 @@ static int plic_dying_cpu(unsigned int cpu)
> >  static int plic_starting_cpu(unsigned int cpu)
> >  {
> >       struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> > +     int pirq = handler->priv->parent_irq;
> >
> > -     csr_set(CSR_IE, IE_EIE);
> > +     if (pirq)
> > +             enable_percpu_irq(pirq, irq_get_trigger_type(pirq));
> >       plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
> >
> >       return 0;
> > @@ -273,6 +282,7 @@ static int __init plic_init(struct device_node
> > *node,
> >       u32 nr_irqs;
> >       struct plic_priv *priv;
> >       struct plic_handler *handler;
> > +     struct of_phandle_args intc_args;
> >
> >       priv = kzalloc(sizeof(*priv), GFP_KERNEL);
> >       if (!priv)
> > @@ -354,6 +364,23 @@ static int __init plic_init(struct device_node
> > *node,
> >               nr_handlers++;
> >       }
> >
> > +     /*
> > +      * We can have multiple PLIC instances so setup chained handler
> > +      * for all PLIC instances.
> > +      */
> > +     intc_args.np = riscv_of_intc_domain_node();
> > +     intc_args.args_count = 1;
> > +     intc_args.args[0] = RV_IRQ_EXT;
> > +     priv->parent_irq = irq_create_of_mapping(&intc_args);
> > +     if (priv->parent_irq)
> > +             irq_set_chained_handler(priv->parent_irq,
> > +                                      plic_handle_irq);
> > +     else {
>
> Please use braces on both sides of the 'else'.

Okay, will update.

>
> > +             pr_err("%pOFP: intc mapping failed\n", node);
> > +             error = -ENODEV;
> > +             goto out_iounmap;
> > +     }
> > +
> >       /*
> >        * We can have multiple PLIC instances so setup cpuhp state only
> >        * when context handler for current/boot CPU is present.
> > @@ -368,7 +395,6 @@ static int __init plic_init(struct device_node
> > *node,
> >
> >       pr_info("%pOFP: mapped %d interrupts with %d handlers for"
> >               " %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
> > -     set_handle_irq(plic_handle_irq);
> >       return 0;
> >
> >  out_iounmap:
> > diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> > index 77d70b633531..57b1f8f777d9 100644
> > --- a/include/linux/cpuhotplug.h
> > +++ b/include/linux/cpuhotplug.h
> > @@ -102,6 +102,7 @@ enum cpuhp_state {
> >       CPUHP_AP_IRQ_ARMADA_XP_STARTING,
> >       CPUHP_AP_IRQ_BCM2836_STARTING,
> >       CPUHP_AP_IRQ_MIPS_GIC_STARTING,
> > +     CPUHP_AP_IRQ_RISCV_STARTING,
> >       CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
> >       CPUHP_AP_ARM_MVEBU_COHERENCY,
> >       CPUHP_AP_MICROCODE_LOADER,
> > diff --git a/include/linux/irqchip/irq-riscv-intc.h
> > b/include/linux/irqchip/irq-riscv-intc.h
> > new file mode 100644
> > index 000000000000..b11d38353099
> > --- /dev/null
> > +++ b/include/linux/irqchip/irq-riscv-intc.h
> > @@ -0,0 +1,20 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> > + */
> > +
> > +#ifndef __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> > +#define __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> > +
> > +struct device_node;
> > +
> > +#ifdef CONFIG_RISCV_INTC
> > +struct device_node *riscv_of_intc_domain_node(void);
> > +#else
> > +static inline struct device_node *riscv_of_intc_domain_node(void)
> > +{
> > +     return NULL;
> > +}
> > +#endif
> > +
> > +#endif
>
> Thanks,
>
>          M.
> --
> Jazz is not dead. It just smells funny...

Regards,
Anup

Re: [PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Anup Patel]

On Fri, May 29, 2020 at 3:39 PM Marc Zyngier <maz@kernel.org> wrote:
>
> On 2020-05-21 14:32, Anup Patel wrote:
> > The RISC-V per-HART local interrupt controller manages software
> > interrupts, timer interrupts, external interrupts (which are routed
> > via the platform level interrupt controller) and other per-HART
> > local interrupts.
> >
> > This patch adds a driver for the RISC-V local interrupt controller.
> > It is a major re-write over perviously submitted version.
> > (Refer, https://www.spinics.net/lists/devicetree/msg241230.html)
> >
> > Few advantages of this new driver over previous one are:
> > 1. All local interrupts are registered as per-CPU interrupts
> > 2. The RISC-V timer driver can register timer interrupt handler
> >    using kernel irq subsystem without relying on arch/riscv to
> >    explicitly call it's interrupt handler
> > 3. The KVM RISC-V can use this driver to implement interrupt
> >    handler for per-HART guest external interrupt defined by
> >    the RISC-V H-Extension
> > 4. In future, we can develop drivers for devices with per-HART
> >    interrupts without changing arch code or this driver (example,
> >    CLINT timer driver for RISC-V M-mode kernel)
>
> Most of this is cover letter material, and doesn't belong to the
> changelog.

Okay, I will drop this from commit description.

>
> >
> > The RISC-V INTC driver is compliant with RISC-V Hart-Level Interrupt
> > Controller DT bindings located at:
> > Documentation/devicetree/bindings/interrupt-controller/riscv,cpu-intc.txt
> >
> > Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
>
> Is Palmer the author? If so, you're missing the 'From' line at the top.
> Otherwise, you're missing a Co-developed-by tag.

Palmer had submitted the first implementation of RISC-V INTC driver
more than 2 years back. That driver had separate IRQ domain for
each CPU and did not use per-CPU interrupts.

The driver added by this patch is re-write to have only one IRQ
domain with all per-CPU interrupts.

I kept Palmer's SoB to acknowledge his initial efforts. I will add
Co-developed-by tag like you suggested.

>
> > Signed-off-by: Anup Patel <anup.patel@wdc.com>
> > ---
> >  arch/riscv/Kconfig                     |   1 +
> >  arch/riscv/include/asm/irq.h           |   2 -
> >  arch/riscv/kernel/irq.c                |  33 +-----
> >  arch/riscv/kernel/traps.c              |   2 -
> >  drivers/irqchip/Kconfig                |  13 ++
> >  drivers/irqchip/Makefile               |   1 +
> >  drivers/irqchip/irq-riscv-intc.c       | 158 +++++++++++++++++++++++++
> >  drivers/irqchip/irq-sifive-plic.c      |  38 +++++-
> >  include/linux/cpuhotplug.h             |   1 +
> >  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
> >  10 files changed, 229 insertions(+), 40 deletions(-)
> >  create mode 100644 drivers/irqchip/irq-riscv-intc.c
> >  create mode 100644 include/linux/irqchip/irq-riscv-intc.h
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index 90a008e28f7e..822cb0e1a380 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -40,6 +40,7 @@ config RISCV
> >       select HAVE_PERF_REGS
> >       select HAVE_PERF_USER_STACK_DUMP
> >       select HAVE_SYSCALL_TRACEPOINTS
> > +     select HANDLE_DOMAIN_IRQ
> >       select IRQ_DOMAIN
> >       select SPARSE_IRQ
> >       select SYSCTL_EXCEPTION_TRACE
> > diff --git a/arch/riscv/include/asm/irq.h
> > b/arch/riscv/include/asm/irq.h
> > index 0183e15ace66..a9e5f07a7e9c 100644
> > --- a/arch/riscv/include/asm/irq.h
> > +++ b/arch/riscv/include/asm/irq.h
> > @@ -10,8 +10,6 @@
> >  #include <linux/interrupt.h>
> >  #include <linux/linkage.h>
> >
> > -#define NR_IRQS         0
> > -
> >  void riscv_timer_interrupt(void);
> >
> >  #include <asm-generic/irq.h>
> > diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> > index bb0bfcd537e7..eb8777642ce6 100644
> > --- a/arch/riscv/kernel/irq.c
> > +++ b/arch/riscv/kernel/irq.c
> > @@ -7,7 +7,6 @@
> >
> >  #include <linux/interrupt.h>
> >  #include <linux/irqchip.h>
> > -#include <linux/irqdomain.h>
> >  #include <linux/seq_file.h>
> >  #include <asm/smp.h>
> >
> > @@ -19,39 +18,13 @@ int arch_show_interrupts(struct seq_file *p, int
> > prec)
> >
> >  asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
> >  {
> > -     struct pt_regs *old_regs;
> > -
> > -     switch (regs->cause & ~CAUSE_IRQ_FLAG) {
> > -     case RV_IRQ_TIMER:
> > -             old_regs = set_irq_regs(regs);
> > -             irq_enter();
> > -             riscv_timer_interrupt();
> > -             irq_exit();
> > -             set_irq_regs(old_regs);
> > -             break;
> > -#ifdef CONFIG_SMP
> > -     case RV_IRQ_SOFT:
> > -             /*
> > -              * We only use software interrupts to pass IPIs, so if a non-SMP
> > -              * system gets one, then we don't know what to do.
> > -              */
> > -             handle_IPI(regs);
> > -             break;
> > -#endif
> > -     case RV_IRQ_EXT:
> > -             old_regs = set_irq_regs(regs);
> > -             irq_enter();
> > +     if (handle_arch_irq)
> >               handle_arch_irq(regs);
> > -             irq_exit();
> > -             set_irq_regs(old_regs);
> > -             break;
> > -     default:
> > -             pr_alert("unexpected interrupt cause 0x%lx", regs->cause);
> > -             BUG();
> > -     }
> >  }
> >
> >  void __init init_IRQ(void)
> >  {
> >       irqchip_init();
> > +     if (!handle_arch_irq)
> > +             panic("No interrupt controller found.");
> >  }
> > diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c
> > index 7f58fa53033f..f48c76aadbf3 100644
> > --- a/arch/riscv/kernel/traps.c
> > +++ b/arch/riscv/kernel/traps.c
> > @@ -178,6 +178,4 @@ void trap_init(void)
> >       csr_write(CSR_SCRATCH, 0);
> >       /* Set the exception vector address */
> >       csr_write(CSR_TVEC, &handle_exception);
> > -     /* Enable interrupts */
> > -     csr_write(CSR_IE, IE_SIE);
> >  }
> > diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> > index a85aada04a64..95d6137a8117 100644
> > --- a/drivers/irqchip/Kconfig
> > +++ b/drivers/irqchip/Kconfig
> > @@ -493,6 +493,19 @@ config TI_SCI_INTA_IRQCHIP
> >         If you wish to use interrupt aggregator irq resources managed by
> > the
> >         TI System Controller, say Y here. Otherwise, say N.
> >
> > +config RISCV_INTC
> > +     bool "RISC-V Local Interrupt Controller"
> > +     depends on RISCV
> > +     default y
> > +     help
> > +        This enables support for the per-HART local interrupt controller
> > +        found in standard RISC-V systems.  The per-HART local interrupt
> > +        controller handles timer interrupts, software interrupts, and
> > +        hardware interrupts. Without a per-HART local interrupt
> > controller,
> > +        a RISC-V system will be unable to handle any interrupts.
> > +
> > +        If you don't know what to do here, say Y.
> > +
> >  config SIFIVE_PLIC
> >       bool "SiFive Platform-Level Interrupt Controller"
> >       depends on RISCV
> > diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> > index 37bbe39bf909..b8319f045472 100644
> > --- a/drivers/irqchip/Makefile
> > +++ b/drivers/irqchip/Makefile
> > @@ -98,6 +98,7 @@ obj-$(CONFIG_NDS32)                 += irq-ativic32.o
> >  obj-$(CONFIG_QCOM_PDC)                       += qcom-pdc.o
> >  obj-$(CONFIG_CSKY_MPINTC)            += irq-csky-mpintc.o
> >  obj-$(CONFIG_CSKY_APB_INTC)          += irq-csky-apb-intc.o
> > +obj-$(CONFIG_RISCV_INTC)             += irq-riscv-intc.o
> >  obj-$(CONFIG_SIFIVE_PLIC)            += irq-sifive-plic.o
> >  obj-$(CONFIG_IMX_IRQSTEER)           += irq-imx-irqsteer.o
> >  obj-$(CONFIG_IMX_INTMUX)             += irq-imx-intmux.o
> > diff --git a/drivers/irqchip/irq-riscv-intc.c
> > b/drivers/irqchip/irq-riscv-intc.c
> > new file mode 100644
> > index 000000000000..2f364e6a87f9
> > --- /dev/null
> > +++ b/drivers/irqchip/irq-riscv-intc.c
> > @@ -0,0 +1,158 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (C) 2012 Regents of the University of California
> > + * Copyright (C) 2017-2018 SiFive
> > + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> > + */
> > +
> > +#define pr_fmt(fmt) "riscv-intc: " fmt
> > +#include <linux/atomic.h>
> > +#include <linux/bits.h>
> > +#include <linux/cpu.h>
> > +#include <linux/irq.h>
> > +#include <linux/irqchip.h>
> > +#include <linux/irqchip/irq-riscv-intc.h>
> > +#include <linux/irqdomain.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/module.h>
> > +#include <linux/of.h>
> > +#include <linux/smp.h>
> > +
> > +static struct device_node *intc_domain_node;
> > +static struct irq_domain *intc_domain;
> > +
> > +static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
> > +{
> > +     struct pt_regs *old_regs;
> > +     unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
> > +
> > +     if (unlikely(cause >= BITS_PER_LONG))
> > +             panic("unexpected interrupt cause");
> > +
> > +     switch (cause) {
> > +     case RV_IRQ_TIMER:
> > +             old_regs = set_irq_regs(regs);
> > +             irq_enter();
> > +             riscv_timer_interrupt();
> > +             irq_exit();
> > +             set_irq_regs(old_regs);
> > +             break;
> > +#ifdef CONFIG_SMP
> > +     case RV_IRQ_SOFT:
> > +             /*
> > +              * We only use software interrupts to pass IPIs, so if a
> > +              * non-SMP system gets one, then we don't know what to do.
> > +              */
> > +             handle_IPI(regs);
> > +             break;
> > +#endif
> > +     default:
> > +             handle_domain_irq(intc_domain, cause, regs);
> > +             break;
> > +     }
> > +}
> > +
> > +/*
> > + * On RISC-V systems local interrupts are masked or unmasked by
> > writing
> > + * the SIE (Supervisor Interrupt Enable) CSR.  As CSRs can only be
> > written
> > + * on the local hart, these functions can only be called on the hart
> > that
> > + * corresponds to the IRQ chip.
> > + */
> > +
> > +static void riscv_intc_irq_mask(struct irq_data *d)
> > +{
> > +     csr_clear(CSR_IE, 1 << (long)d->hwirq);
>
> It feels pretty odd to cast a shift to a signed value. In any case, this
> would look better as BIT(d->hwirq), all over the place.
>
> > +}
> > +
> > +static void riscv_intc_irq_unmask(struct irq_data *d)
> > +{
> > +     csr_set(CSR_IE, 1 << (long)d->hwirq);
> > +}
> > +
> > +static int riscv_intc_cpu_starting(unsigned int cpu)
> > +{
> > +     csr_write(CSR_IE, 1UL << RV_IRQ_SOFT);
> > +     csr_write(CSR_IP, 0);
> > +     return 0;
> > +}
> > +
> > +static int riscv_intc_cpu_dying(unsigned int cpu)
> > +{
> > +     csr_clear(CSR_IE, 1UL << RV_IRQ_SOFT);
> > +     return 0;
> > +}
> > +
> > +static struct irq_chip riscv_intc_chip = {
> > +     .name = "RISC-V INTC",
> > +     .irq_mask = riscv_intc_irq_mask,
> > +     .irq_unmask = riscv_intc_irq_unmask,
> > +};
> > +
> > +static int riscv_intc_domain_map(struct irq_domain *d, unsigned int
> > irq,
> > +                              irq_hw_number_t hwirq)
> > +{
> > +     irq_set_percpu_devid(irq);
> > +     irq_domain_set_info(d, irq, hwirq, &riscv_intc_chip, d->host_data,
> > +                         handle_percpu_devid_irq, NULL, NULL);
> > +     irq_set_status_flags(irq, IRQ_NOAUTOEN);
>
> As I was recently reminded, percpu_devid implies NO_AUTOEN.
> This is thus supperfluous.

Okay, I will drop this line.

>
> > +
> > +     return 0;
> > +}
> > +
> > +static const struct irq_domain_ops riscv_intc_domain_ops = {
> > +     .map    = riscv_intc_domain_map,
> > +     .xlate  = irq_domain_xlate_onecell,
> > +};
> > +
> > +/* Get the OF device node used by INTC irq domain */
> > +struct device_node *riscv_of_intc_domain_node(void)
> > +{
> > +     return intc_domain_node;
> > +}
> > +EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);
>
> Why do you need this? Why can't the timer node refer to its
> interrupt-parent? The irqchip shouldn't be in the business of
> working around DT issues.
>
> At worse, use the default irqdomain if you must, but please
> avoid this kind of construct.

Even, I don't like exporting riscv_of_intc_domain_node().

Thanks for your suggestion, I will certainly use the default irqdomain.

>
> > +
> > +static int __init riscv_intc_init(struct device_node *node,
> > +                               struct device_node *parent)
> > +{
> > +     int hartid;
> > +
> > +     /*
> > +      * The DT will have one INTC DT node under each CPU (or HART)
> > +      * DT node so riscv_intc_init() function will be called once
> > +      * for each INTC DT node. We only need INTC initialization for
> > +      * the INTC DT node belonging to boot CPU (or boot HART).
> > +      */
> > +     hartid = riscv_of_parent_hartid(node);
> > +     if (hartid < 0)
> > +             return 0;
> > +     if (riscv_hartid_to_cpuid(hartid) != smp_processor_id())
> > +             return 0;
> > +
> > +     intc_domain = irq_domain_add_linear(node, BITS_PER_LONG,
> > +                                         &riscv_intc_domain_ops, NULL);
> > +     if (!intc_domain)
> > +             goto error_add_linear;
> > +
> > +     /*
> > +      * We save the DT node used for creating irq domain and provide
> > +      * it to other drivers using iscv_of_intc_domain_node() function.
> > +      */
> > +     intc_domain_node = of_node_get(node);
> > +
> > +     set_handle_irq(&riscv_intc_irq);
> > +
> > +     cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_STARTING,
> > +                       "irqchip/riscv/intc:starting",
> > +                       riscv_intc_cpu_starting,
> > +                       riscv_intc_cpu_dying);
> > +
> > +     pr_info("%lu local interrupts mapped\n", (long)BITS_PER_LONG);
>
> Signed or unsigned? Pick one.

Okay, will fix it.

>
> > +     return 0;
> > +
> > +error_add_linear:
> > +     pr_warn("unable to add IRQ domain\n");
> > +     return -ENXIO;
> > +}
> > +
> > +IRQCHIP_DECLARE(riscv, "riscv,cpu-intc", riscv_intc_init);
> > diff --git a/drivers/irqchip/irq-sifive-plic.c
> > b/drivers/irqchip/irq-sifive-plic.c
> > index 16d31d114c30..4f9b2877aa9d 100644
> > --- a/drivers/irqchip/irq-sifive-plic.c
> > +++ b/drivers/irqchip/irq-sifive-plic.c
> > @@ -9,6 +9,8 @@
> >  #include <linux/io.h>
> >  #include <linux/irq.h>
> >  #include <linux/irqchip.h>
> > +#include <linux/irqchip/chained_irq.h>
> > +#include <linux/irqchip/irq-riscv-intc.h>
> >  #include <linux/irqdomain.h>
> >  #include <linux/module.h>
> >  #include <linux/of.h>
> > @@ -60,6 +62,7 @@
> >  #define      PLIC_ENABLE_THRESHOLD           0
> >
> >  struct plic_priv {
> > +     int parent_irq;
> >       struct cpumask lmask;
> >       struct irq_domain *irqdomain;
> >       void __iomem *regs;
> > @@ -219,15 +222,17 @@ static const struct irq_domain_ops
> > plic_irqdomain_ops = {
> >   * that source ID back to the same claim register.  This automatically
> > enables
> >   * and disables the interrupt, so there's nothing else to do.
> >   */
> > -static void plic_handle_irq(struct pt_regs *regs)
> > +static void plic_handle_irq(struct irq_desc *desc)
> >  {
> >       struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> > +     struct irq_chip *chip = irq_desc_get_chip(desc);
> >       void __iomem *claim = handler->hart_base + CONTEXT_CLAIM;
> >       irq_hw_number_t hwirq;
> >
> >       WARN_ON_ONCE(!handler->present);
> >
> > -     csr_clear(CSR_IE, IE_EIE);
> > +     chained_irq_enter(chip, desc);
> > +
> >       while ((hwirq = readl(claim))) {
> >               int irq = irq_find_mapping(handler->priv->irqdomain, hwirq);
> >
> > @@ -237,7 +242,8 @@ static void plic_handle_irq(struct pt_regs *regs)
> >               else
> >                       generic_handle_irq(irq);
> >       }
> > -     csr_set(CSR_IE, IE_EIE);
> > +
> > +     chained_irq_exit(chip, desc);
> >  }
> >
> >  static void plic_set_threshold(struct plic_handler *handler, u32
> > threshold)
> > @@ -250,7 +256,8 @@ static int plic_dying_cpu(unsigned int cpu)
> >  {
> >       struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> >
> > -     csr_clear(CSR_IE, IE_EIE);
> > +     if (handler->priv->parent_irq)
> > +             disable_percpu_irq(handler->priv->parent_irq);
> >       plic_set_threshold(handler, PLIC_DISABLE_THRESHOLD);
> >
> >       return 0;
> > @@ -259,8 +266,10 @@ static int plic_dying_cpu(unsigned int cpu)
> >  static int plic_starting_cpu(unsigned int cpu)
> >  {
> >       struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> > +     int pirq = handler->priv->parent_irq;
> >
> > -     csr_set(CSR_IE, IE_EIE);
> > +     if (pirq)
> > +             enable_percpu_irq(pirq, irq_get_trigger_type(pirq));
> >       plic_set_threshold(handler, PLIC_ENABLE_THRESHOLD);
> >
> >       return 0;
> > @@ -273,6 +282,7 @@ static int __init plic_init(struct device_node
> > *node,
> >       u32 nr_irqs;
> >       struct plic_priv *priv;
> >       struct plic_handler *handler;
> > +     struct of_phandle_args intc_args;
> >
> >       priv = kzalloc(sizeof(*priv), GFP_KERNEL);
> >       if (!priv)
> > @@ -354,6 +364,23 @@ static int __init plic_init(struct device_node
> > *node,
> >               nr_handlers++;
> >       }
> >
> > +     /*
> > +      * We can have multiple PLIC instances so setup chained handler
> > +      * for all PLIC instances.
> > +      */
> > +     intc_args.np = riscv_of_intc_domain_node();
> > +     intc_args.args_count = 1;
> > +     intc_args.args[0] = RV_IRQ_EXT;
> > +     priv->parent_irq = irq_create_of_mapping(&intc_args);
> > +     if (priv->parent_irq)
> > +             irq_set_chained_handler(priv->parent_irq,
> > +                                      plic_handle_irq);
> > +     else {
>
> Please use braces on both sides of the 'else'.

Okay, will update.

>
> > +             pr_err("%pOFP: intc mapping failed\n", node);
> > +             error = -ENODEV;
> > +             goto out_iounmap;
> > +     }
> > +
> >       /*
> >        * We can have multiple PLIC instances so setup cpuhp state only
> >        * when context handler for current/boot CPU is present.
> > @@ -368,7 +395,6 @@ static int __init plic_init(struct device_node
> > *node,
> >
> >       pr_info("%pOFP: mapped %d interrupts with %d handlers for"
> >               " %d contexts.\n", node, nr_irqs, nr_handlers, nr_contexts);
> > -     set_handle_irq(plic_handle_irq);
> >       return 0;
> >
> >  out_iounmap:
> > diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> > index 77d70b633531..57b1f8f777d9 100644
> > --- a/include/linux/cpuhotplug.h
> > +++ b/include/linux/cpuhotplug.h
> > @@ -102,6 +102,7 @@ enum cpuhp_state {
> >       CPUHP_AP_IRQ_ARMADA_XP_STARTING,
> >       CPUHP_AP_IRQ_BCM2836_STARTING,
> >       CPUHP_AP_IRQ_MIPS_GIC_STARTING,
> > +     CPUHP_AP_IRQ_RISCV_STARTING,
> >       CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING,
> >       CPUHP_AP_ARM_MVEBU_COHERENCY,
> >       CPUHP_AP_MICROCODE_LOADER,
> > diff --git a/include/linux/irqchip/irq-riscv-intc.h
> > b/include/linux/irqchip/irq-riscv-intc.h
> > new file mode 100644
> > index 000000000000..b11d38353099
> > --- /dev/null
> > +++ b/include/linux/irqchip/irq-riscv-intc.h
> > @@ -0,0 +1,20 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Copyright (C) 2020 Western Digital Corporation or its affiliates.
> > + */
> > +
> > +#ifndef __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> > +#define __INCLUDE_LINUX_IRQCHIP_IRQ_RISCV_INTC_H
> > +
> > +struct device_node;
> > +
> > +#ifdef CONFIG_RISCV_INTC
> > +struct device_node *riscv_of_intc_domain_node(void);
> > +#else
> > +static inline struct device_node *riscv_of_intc_domain_node(void)
> > +{
> > +     return NULL;
> > +}
> > +#endif
> > +
> > +#endif
>
> Thanks,
>
>          M.
> --
> Jazz is not dead. It just smells funny...

Regards,
Anup

Re: [PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Marc Zyngier]

On 2020-05-29 11:45, Anup Patel wrote:
> On Fri, May 29, 2020 at 3:39 PM Marc Zyngier <maz@kernel.org> wrote:
>> 
>> On 2020-05-21 14:32, Anup Patel wrote:

[...]

>> > +/* Get the OF device node used by INTC irq domain */
>> > +struct device_node *riscv_of_intc_domain_node(void)
>> > +{
>> > +     return intc_domain_node;
>> > +}
>> > +EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);
>> 
>> Why do you need this? Why can't the timer node refer to its
>> interrupt-parent? The irqchip shouldn't be in the business of
>> working around DT issues.
>> 
>> At worse, use the default irqdomain if you must, but please
>> avoid this kind of construct.
> 
> Even, I don't like exporting riscv_of_intc_domain_node().
> 
> Thanks for your suggestion, I will certainly use the default irqdomain.

This should be a last resort solution. The irqdomain should
naturally come from the parent interrupt controller, accessible
from the device (the timer in this case) node.

Use it to for backward compatibility if you *really* must,
but this is generally a very bad idea as it allows all kind
of bizarre fallbacks and hides bugs.

Thanks,

          M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Marc Zyngier]

On 2020-05-29 11:45, Anup Patel wrote:
> On Fri, May 29, 2020 at 3:39 PM Marc Zyngier <maz@kernel.org> wrote:
>> 
>> On 2020-05-21 14:32, Anup Patel wrote:

[...]

>> > +/* Get the OF device node used by INTC irq domain */
>> > +struct device_node *riscv_of_intc_domain_node(void)
>> > +{
>> > +     return intc_domain_node;
>> > +}
>> > +EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);
>> 
>> Why do you need this? Why can't the timer node refer to its
>> interrupt-parent? The irqchip shouldn't be in the business of
>> working around DT issues.
>> 
>> At worse, use the default irqdomain if you must, but please
>> avoid this kind of construct.
> 
> Even, I don't like exporting riscv_of_intc_domain_node().
> 
> Thanks for your suggestion, I will certainly use the default irqdomain.

This should be a last resort solution. The irqdomain should
naturally come from the parent interrupt controller, accessible
from the device (the timer in this case) node.

Use it to for backward compatibility if you *really* must,
but this is generally a very bad idea as it allows all kind
of bizarre fallbacks and hides bugs.

Thanks,

          M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Anup Patel]

On Fri, May 29, 2020 at 4:40 PM Marc Zyngier <maz@kernel.org> wrote:
>
> On 2020-05-29 11:45, Anup Patel wrote:
> > On Fri, May 29, 2020 at 3:39 PM Marc Zyngier <maz@kernel.org> wrote:
> >>
> >> On 2020-05-21 14:32, Anup Patel wrote:
>
> [...]
>
> >> > +/* Get the OF device node used by INTC irq domain */
> >> > +struct device_node *riscv_of_intc_domain_node(void)
> >> > +{
> >> > +     return intc_domain_node;
> >> > +}
> >> > +EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);
> >>
> >> Why do you need this? Why can't the timer node refer to its
> >> interrupt-parent? The irqchip shouldn't be in the business of
> >> working around DT issues.
> >>
> >> At worse, use the default irqdomain if you must, but please
> >> avoid this kind of construct.
> >
> > Even, I don't like exporting riscv_of_intc_domain_node().
> >
> > Thanks for your suggestion, I will certainly use the default irqdomain.
>
> This should be a last resort solution. The irqdomain should
> naturally come from the parent interrupt controller, accessible
> from the device (the timer in this case) node.
>
> Use it to for backward compatibility if you *really* must,
> but this is generally a very bad idea as it allows all kind
> of bizarre fallbacks and hides bugs.

Okay, I will explore other approaches and keep default irqdomain
as a last resort solution.

Regards,
Anup

Re: [PATCH v5 3/6] irqchip: RISC-V per-HART local interrupt controller driver

[Anup Patel]

On Fri, May 29, 2020 at 4:40 PM Marc Zyngier <maz@kernel.org> wrote:
>
> On 2020-05-29 11:45, Anup Patel wrote:
> > On Fri, May 29, 2020 at 3:39 PM Marc Zyngier <maz@kernel.org> wrote:
> >>
> >> On 2020-05-21 14:32, Anup Patel wrote:
>
> [...]
>
> >> > +/* Get the OF device node used by INTC irq domain */
> >> > +struct device_node *riscv_of_intc_domain_node(void)
> >> > +{
> >> > +     return intc_domain_node;
> >> > +}
> >> > +EXPORT_SYMBOL_GPL(riscv_of_intc_domain_node);
> >>
> >> Why do you need this? Why can't the timer node refer to its
> >> interrupt-parent? The irqchip shouldn't be in the business of
> >> working around DT issues.
> >>
> >> At worse, use the default irqdomain if you must, but please
> >> avoid this kind of construct.
> >
> > Even, I don't like exporting riscv_of_intc_domain_node().
> >
> > Thanks for your suggestion, I will certainly use the default irqdomain.
>
> This should be a last resort solution. The irqdomain should
> naturally come from the parent interrupt controller, accessible
> from the device (the timer in this case) node.
>
> Use it to for backward compatibility if you *really* must,
> but this is generally a very bad idea as it allows all kind
> of bizarre fallbacks and hides bugs.

Okay, I will explore other approaches and keep default irqdomain
as a last resort solution.

Regards,
Anup

[PATCH v5 4/6] clocksource/drivers/timer-riscv: Use per-CPU timer interrupt

[Anup Patel]

Instead of directly calling RISC-V timer interrupt handler from
RISC-V local interrupt conntroller driver, this patch implements
RISC-V timer interrupt as a per-CPU interrupt using per-CPU APIs
of Linux IRQ subsystem.

Signed-off-by: Anup Patel <anup.patel@wdc.com>
---
 arch/riscv/include/asm/irq.h      |  2 --
 drivers/clocksource/timer-riscv.c | 30 +++++++++++++++++++++++++++---
 drivers/irqchip/irq-riscv-intc.c  |  8 --------
 3 files changed, 27 insertions(+), 13 deletions(-)

diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
index a9e5f07a7e9c..9807ad164015 100644
--- a/arch/riscv/include/asm/irq.h
+++ b/arch/riscv/include/asm/irq.h
@@ -10,8 +10,6 @@
 #include <linux/interrupt.h>
 #include <linux/linkage.h>
 
-void riscv_timer_interrupt(void);
-
 #include <asm-generic/irq.h>
 
 #endif /* _ASM_RISCV_IRQ_H */
diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c
index c4f15c4068c0..5fb7c5ba5c91 100644
--- a/drivers/clocksource/timer-riscv.c
+++ b/drivers/clocksource/timer-riscv.c
@@ -14,6 +14,9 @@
 #include <linux/irq.h>
 #include <linux/sched_clock.h>
 #include <linux/io-64-nonatomic-lo-hi.h>
+#include <linux/irqchip/irq-riscv-intc.h>
+#include <linux/interrupt.h>
+#include <linux/of_irq.h>
 #include <asm/smp.h>
 #include <asm/sbi.h>
 
@@ -39,6 +42,7 @@ static int riscv_clock_next_event(unsigned long delta,
 	return 0;
 }
 
+static unsigned int riscv_clock_event_irq;
 static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
 	.name			= "riscv_timer_clockevent",
 	.features		= CLOCK_EVT_FEAT_ONESHOT,
@@ -74,30 +78,35 @@ static int riscv_timer_starting_cpu(unsigned int cpu)
 	struct clock_event_device *ce = per_cpu_ptr(&riscv_clock_event, cpu);
 
 	ce->cpumask = cpumask_of(cpu);
+	ce->irq = riscv_clock_event_irq;
 	clockevents_config_and_register(ce, riscv_timebase, 100, 0x7fffffff);
 
-	csr_set(CSR_IE, IE_TIE);
+	enable_percpu_irq(riscv_clock_event_irq,
+			  irq_get_trigger_type(riscv_clock_event_irq));
 	return 0;
 }
 
 static int riscv_timer_dying_cpu(unsigned int cpu)
 {
-	csr_clear(CSR_IE, IE_TIE);
+	disable_percpu_irq(riscv_clock_event_irq);
 	return 0;
 }
 
 /* called directly from the low-level interrupt handler */
-void riscv_timer_interrupt(void)
+static irqreturn_t riscv_timer_interrupt(int irq, void *dev_id)
 {
 	struct clock_event_device *evdev = this_cpu_ptr(&riscv_clock_event);
 
 	csr_clear(CSR_IE, IE_TIE);
 	evdev->event_handler(evdev);
+
+	return IRQ_HANDLED;
 }
 
 static int __init riscv_timer_init_dt(struct device_node *n)
 {
 	int cpuid, hartid, error;
+	struct of_phandle_args oirq;
 
 	hartid = riscv_of_processor_hartid(n);
 	if (hartid < 0) {
@@ -115,6 +124,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)
 	if (cpuid != smp_processor_id())
 		return 0;
 
+	oirq.np = riscv_of_intc_domain_node();
+	oirq.args_count = 1;
+	oirq.args[0] = RV_IRQ_TIMER;
+	riscv_clock_event_irq = irq_create_of_mapping(&oirq);
+	if (!riscv_clock_event_irq)
+		return -ENODEV;
+
 	pr_info("%s: Registering clocksource cpuid [%d] hartid [%d]\n",
 	       __func__, cpuid, hartid);
 	error = clocksource_register_hz(&riscv_clocksource, riscv_timebase);
@@ -126,6 +142,14 @@ static int __init riscv_timer_init_dt(struct device_node *n)
 
 	sched_clock_register(riscv_sched_clock, 64, riscv_timebase);
 
+	error = request_percpu_irq(riscv_clock_event_irq,
+				    riscv_timer_interrupt,
+				    "riscv-timer", &riscv_clock_event);
+	if (error) {
+		pr_err("registering percpu irq failed [%d]\n", error);
+		return error;
+	}
+
 	error = cpuhp_setup_state(CPUHP_AP_RISCV_TIMER_STARTING,
 			 "clockevents/riscv/timer:starting",
 			 riscv_timer_starting_cpu, riscv_timer_dying_cpu);
diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
index 2f364e6a87f9..d4fbc3543459 100644
--- a/drivers/irqchip/irq-riscv-intc.c
+++ b/drivers/irqchip/irq-riscv-intc.c
@@ -23,20 +23,12 @@ static struct irq_domain *intc_domain;
 
 static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
 {
-	struct pt_regs *old_regs;
 	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
 
 	if (unlikely(cause >= BITS_PER_LONG))
 		panic("unexpected interrupt cause");
 
 	switch (cause) {
-	case RV_IRQ_TIMER:
-		old_regs = set_irq_regs(regs);
-		irq_enter();
-		riscv_timer_interrupt();
-		irq_exit();
-		set_irq_regs(old_regs);
-		break;
 #ifdef CONFIG_SMP
 	case RV_IRQ_SOFT:
 		/*
-- 
2.25.1

[PATCH v5 4/6] clocksource/drivers/timer-riscv: Use per-CPU timer interrupt

[Anup Patel]

Instead of directly calling RISC-V timer interrupt handler from
RISC-V local interrupt conntroller driver, this patch implements
RISC-V timer interrupt as a per-CPU interrupt using per-CPU APIs
of Linux IRQ subsystem.

Signed-off-by: Anup Patel <anup.patel@wdc.com>
---
 arch/riscv/include/asm/irq.h      |  2 --
 drivers/clocksource/timer-riscv.c | 30 +++++++++++++++++++++++++++---
 drivers/irqchip/irq-riscv-intc.c  |  8 --------
 3 files changed, 27 insertions(+), 13 deletions(-)

diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
index a9e5f07a7e9c..9807ad164015 100644
--- a/arch/riscv/include/asm/irq.h
+++ b/arch/riscv/include/asm/irq.h
@@ -10,8 +10,6 @@
 #include <linux/interrupt.h>
 #include <linux/linkage.h>
 
-void riscv_timer_interrupt(void);
-
 #include <asm-generic/irq.h>
 
 #endif /* _ASM_RISCV_IRQ_H */
diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c
index c4f15c4068c0..5fb7c5ba5c91 100644
--- a/drivers/clocksource/timer-riscv.c
+++ b/drivers/clocksource/timer-riscv.c
@@ -14,6 +14,9 @@
 #include <linux/irq.h>
 #include <linux/sched_clock.h>
 #include <linux/io-64-nonatomic-lo-hi.h>
+#include <linux/irqchip/irq-riscv-intc.h>
+#include <linux/interrupt.h>
+#include <linux/of_irq.h>
 #include <asm/smp.h>
 #include <asm/sbi.h>
 
@@ -39,6 +42,7 @@ static int riscv_clock_next_event(unsigned long delta,
 	return 0;
 }
 
+static unsigned int riscv_clock_event_irq;
 static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
 	.name			= "riscv_timer_clockevent",
 	.features		= CLOCK_EVT_FEAT_ONESHOT,
@@ -74,30 +78,35 @@ static int riscv_timer_starting_cpu(unsigned int cpu)
 	struct clock_event_device *ce = per_cpu_ptr(&riscv_clock_event, cpu);
 
 	ce->cpumask = cpumask_of(cpu);
+	ce->irq = riscv_clock_event_irq;
 	clockevents_config_and_register(ce, riscv_timebase, 100, 0x7fffffff);
 
-	csr_set(CSR_IE, IE_TIE);
+	enable_percpu_irq(riscv_clock_event_irq,
+			  irq_get_trigger_type(riscv_clock_event_irq));
 	return 0;
 }
 
 static int riscv_timer_dying_cpu(unsigned int cpu)
 {
-	csr_clear(CSR_IE, IE_TIE);
+	disable_percpu_irq(riscv_clock_event_irq);
 	return 0;
 }
 
 /* called directly from the low-level interrupt handler */
-void riscv_timer_interrupt(void)
+static irqreturn_t riscv_timer_interrupt(int irq, void *dev_id)
 {
 	struct clock_event_device *evdev = this_cpu_ptr(&riscv_clock_event);
 
 	csr_clear(CSR_IE, IE_TIE);
 	evdev->event_handler(evdev);
+
+	return IRQ_HANDLED;
 }
 
 static int __init riscv_timer_init_dt(struct device_node *n)
 {
 	int cpuid, hartid, error;
+	struct of_phandle_args oirq;
 
 	hartid = riscv_of_processor_hartid(n);
 	if (hartid < 0) {
@@ -115,6 +124,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)
 	if (cpuid != smp_processor_id())
 		return 0;
 
+	oirq.np = riscv_of_intc_domain_node();
+	oirq.args_count = 1;
+	oirq.args[0] = RV_IRQ_TIMER;
+	riscv_clock_event_irq = irq_create_of_mapping(&oirq);
+	if (!riscv_clock_event_irq)
+		return -ENODEV;
+
 	pr_info("%s: Registering clocksource cpuid [%d] hartid [%d]\n",
 	       __func__, cpuid, hartid);
 	error = clocksource_register_hz(&riscv_clocksource, riscv_timebase);
@@ -126,6 +142,14 @@ static int __init riscv_timer_init_dt(struct device_node *n)
 
 	sched_clock_register(riscv_sched_clock, 64, riscv_timebase);
 
+	error = request_percpu_irq(riscv_clock_event_irq,
+				    riscv_timer_interrupt,
+				    "riscv-timer", &riscv_clock_event);
+	if (error) {
+		pr_err("registering percpu irq failed [%d]\n", error);
+		return error;
+	}
+
 	error = cpuhp_setup_state(CPUHP_AP_RISCV_TIMER_STARTING,
 			 "clockevents/riscv/timer:starting",
 			 riscv_timer_starting_cpu, riscv_timer_dying_cpu);
diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
index 2f364e6a87f9..d4fbc3543459 100644
--- a/drivers/irqchip/irq-riscv-intc.c
+++ b/drivers/irqchip/irq-riscv-intc.c
@@ -23,20 +23,12 @@ static struct irq_domain *intc_domain;
 
 static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
 {
-	struct pt_regs *old_regs;
 	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
 
 	if (unlikely(cause >= BITS_PER_LONG))
 		panic("unexpected interrupt cause");
 
 	switch (cause) {
-	case RV_IRQ_TIMER:
-		old_regs = set_irq_regs(regs);
-		irq_enter();
-		riscv_timer_interrupt();
-		irq_exit();
-		set_irq_regs(old_regs);
-		break;
 #ifdef CONFIG_SMP
 	case RV_IRQ_SOFT:
 		/*
-- 
2.25.1

Re: [PATCH v5 4/6] clocksource/drivers/timer-riscv: Use per-CPU timer interrupt

[Daniel Lezcano]

On 21/05/2020 15:32, Anup Patel wrote:
> Instead of directly calling RISC-V timer interrupt handler from
> RISC-V local interrupt conntroller driver, this patch implements
> RISC-V timer interrupt as a per-CPU interrupt using per-CPU APIs
> of Linux IRQ subsystem.
> 
> Signed-off-by: Anup Patel <anup.patel@wdc.com>

via which tree do you want this patch to be merged ?

> ---
>  arch/riscv/include/asm/irq.h      |  2 --
>  drivers/clocksource/timer-riscv.c | 30 +++++++++++++++++++++++++++---
>  drivers/irqchip/irq-riscv-intc.c  |  8 --------
>  3 files changed, 27 insertions(+), 13 deletions(-)
> 
> diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
> index a9e5f07a7e9c..9807ad164015 100644
> --- a/arch/riscv/include/asm/irq.h
> +++ b/arch/riscv/include/asm/irq.h
> @@ -10,8 +10,6 @@
>  #include <linux/interrupt.h>
>  #include <linux/linkage.h>
>  
> -void riscv_timer_interrupt(void);
> -
>  #include <asm-generic/irq.h>
>  
>  #endif /* _ASM_RISCV_IRQ_H */
> diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c
> index c4f15c4068c0..5fb7c5ba5c91 100644
> --- a/drivers/clocksource/timer-riscv.c
> +++ b/drivers/clocksource/timer-riscv.c
> @@ -14,6 +14,9 @@
>  #include <linux/irq.h>
>  #include <linux/sched_clock.h>
>  #include <linux/io-64-nonatomic-lo-hi.h>
> +#include <linux/irqchip/irq-riscv-intc.h>
> +#include <linux/interrupt.h>
> +#include <linux/of_irq.h>
>  #include <asm/smp.h>
>  #include <asm/sbi.h>
>  
> @@ -39,6 +42,7 @@ static int riscv_clock_next_event(unsigned long delta,
>  	return 0;
>  }
>  
> +static unsigned int riscv_clock_event_irq;
>  static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
>  	.name			= "riscv_timer_clockevent",
>  	.features		= CLOCK_EVT_FEAT_ONESHOT,
> @@ -74,30 +78,35 @@ static int riscv_timer_starting_cpu(unsigned int cpu)
>  	struct clock_event_device *ce = per_cpu_ptr(&riscv_clock_event, cpu);
>  
>  	ce->cpumask = cpumask_of(cpu);
> +	ce->irq = riscv_clock_event_irq;
>  	clockevents_config_and_register(ce, riscv_timebase, 100, 0x7fffffff);
>  
> -	csr_set(CSR_IE, IE_TIE);
> +	enable_percpu_irq(riscv_clock_event_irq,
> +			  irq_get_trigger_type(riscv_clock_event_irq));
>  	return 0;
>  }
>  
>  static int riscv_timer_dying_cpu(unsigned int cpu)
>  {
> -	csr_clear(CSR_IE, IE_TIE);
> +	disable_percpu_irq(riscv_clock_event_irq);
>  	return 0;
>  }
>  
>  /* called directly from the low-level interrupt handler */
> -void riscv_timer_interrupt(void)
> +static irqreturn_t riscv_timer_interrupt(int irq, void *dev_id)
>  {
>  	struct clock_event_device *evdev = this_cpu_ptr(&riscv_clock_event);
>  
>  	csr_clear(CSR_IE, IE_TIE);
>  	evdev->event_handler(evdev);
> +
> +	return IRQ_HANDLED;
>  }
>  
>  static int __init riscv_timer_init_dt(struct device_node *n)
>  {
>  	int cpuid, hartid, error;
> +	struct of_phandle_args oirq;
>  
>  	hartid = riscv_of_processor_hartid(n);
>  	if (hartid < 0) {
> @@ -115,6 +124,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)
>  	if (cpuid != smp_processor_id())
>  		return 0;
>  
> +	oirq.np = riscv_of_intc_domain_node();
> +	oirq.args_count = 1;
> +	oirq.args[0] = RV_IRQ_TIMER;
> +	riscv_clock_event_irq = irq_create_of_mapping(&oirq);
> +	if (!riscv_clock_event_irq)
> +		return -ENODEV;
> +
>  	pr_info("%s: Registering clocksource cpuid [%d] hartid [%d]\n",
>  	       __func__, cpuid, hartid);
>  	error = clocksource_register_hz(&riscv_clocksource, riscv_timebase);
> @@ -126,6 +142,14 @@ static int __init riscv_timer_init_dt(struct device_node *n)
>  
>  	sched_clock_register(riscv_sched_clock, 64, riscv_timebase);
>  
> +	error = request_percpu_irq(riscv_clock_event_irq,
> +				    riscv_timer_interrupt,
> +				    "riscv-timer", &riscv_clock_event);
> +	if (error) {
> +		pr_err("registering percpu irq failed [%d]\n", error);
> +		return error;
> +	}
> +
>  	error = cpuhp_setup_state(CPUHP_AP_RISCV_TIMER_STARTING,
>  			 "clockevents/riscv/timer:starting",
>  			 riscv_timer_starting_cpu, riscv_timer_dying_cpu);
> diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> index 2f364e6a87f9..d4fbc3543459 100644
> --- a/drivers/irqchip/irq-riscv-intc.c
> +++ b/drivers/irqchip/irq-riscv-intc.c
> @@ -23,20 +23,12 @@ static struct irq_domain *intc_domain;
>  
>  static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
>  {
> -	struct pt_regs *old_regs;
>  	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
>  
>  	if (unlikely(cause >= BITS_PER_LONG))
>  		panic("unexpected interrupt cause");
>  
>  	switch (cause) {
> -	case RV_IRQ_TIMER:
> -		old_regs = set_irq_regs(regs);
> -		irq_enter();
> -		riscv_timer_interrupt();
> -		irq_exit();
> -		set_irq_regs(old_regs);
> -		break;
>  #ifdef CONFIG_SMP
>  	case RV_IRQ_SOFT:
>  		/*
> 


-- 
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog

Re: [PATCH v5 4/6] clocksource/drivers/timer-riscv: Use per-CPU timer interrupt

[Daniel Lezcano]

On 21/05/2020 15:32, Anup Patel wrote:
> Instead of directly calling RISC-V timer interrupt handler from
> RISC-V local interrupt conntroller driver, this patch implements
> RISC-V timer interrupt as a per-CPU interrupt using per-CPU APIs
> of Linux IRQ subsystem.
> 
> Signed-off-by: Anup Patel <anup.patel@wdc.com>

via which tree do you want this patch to be merged ?

> ---
>  arch/riscv/include/asm/irq.h      |  2 --
>  drivers/clocksource/timer-riscv.c | 30 +++++++++++++++++++++++++++---
>  drivers/irqchip/irq-riscv-intc.c  |  8 --------
>  3 files changed, 27 insertions(+), 13 deletions(-)
> 
> diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
> index a9e5f07a7e9c..9807ad164015 100644
> --- a/arch/riscv/include/asm/irq.h
> +++ b/arch/riscv/include/asm/irq.h
> @@ -10,8 +10,6 @@
>  #include <linux/interrupt.h>
>  #include <linux/linkage.h>
>  
> -void riscv_timer_interrupt(void);
> -
>  #include <asm-generic/irq.h>
>  
>  #endif /* _ASM_RISCV_IRQ_H */
> diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c
> index c4f15c4068c0..5fb7c5ba5c91 100644
> --- a/drivers/clocksource/timer-riscv.c
> +++ b/drivers/clocksource/timer-riscv.c
> @@ -14,6 +14,9 @@
>  #include <linux/irq.h>
>  #include <linux/sched_clock.h>
>  #include <linux/io-64-nonatomic-lo-hi.h>
> +#include <linux/irqchip/irq-riscv-intc.h>
> +#include <linux/interrupt.h>
> +#include <linux/of_irq.h>
>  #include <asm/smp.h>
>  #include <asm/sbi.h>
>  
> @@ -39,6 +42,7 @@ static int riscv_clock_next_event(unsigned long delta,
>  	return 0;
>  }
>  
> +static unsigned int riscv_clock_event_irq;
>  static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
>  	.name			= "riscv_timer_clockevent",
>  	.features		= CLOCK_EVT_FEAT_ONESHOT,
> @@ -74,30 +78,35 @@ static int riscv_timer_starting_cpu(unsigned int cpu)
>  	struct clock_event_device *ce = per_cpu_ptr(&riscv_clock_event, cpu);
>  
>  	ce->cpumask = cpumask_of(cpu);
> +	ce->irq = riscv_clock_event_irq;
>  	clockevents_config_and_register(ce, riscv_timebase, 100, 0x7fffffff);
>  
> -	csr_set(CSR_IE, IE_TIE);
> +	enable_percpu_irq(riscv_clock_event_irq,
> +			  irq_get_trigger_type(riscv_clock_event_irq));
>  	return 0;
>  }
>  
>  static int riscv_timer_dying_cpu(unsigned int cpu)
>  {
> -	csr_clear(CSR_IE, IE_TIE);
> +	disable_percpu_irq(riscv_clock_event_irq);
>  	return 0;
>  }
>  
>  /* called directly from the low-level interrupt handler */
> -void riscv_timer_interrupt(void)
> +static irqreturn_t riscv_timer_interrupt(int irq, void *dev_id)
>  {
>  	struct clock_event_device *evdev = this_cpu_ptr(&riscv_clock_event);
>  
>  	csr_clear(CSR_IE, IE_TIE);
>  	evdev->event_handler(evdev);
> +
> +	return IRQ_HANDLED;
>  }
>  
>  static int __init riscv_timer_init_dt(struct device_node *n)
>  {
>  	int cpuid, hartid, error;
> +	struct of_phandle_args oirq;
>  
>  	hartid = riscv_of_processor_hartid(n);
>  	if (hartid < 0) {
> @@ -115,6 +124,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)
>  	if (cpuid != smp_processor_id())
>  		return 0;
>  
> +	oirq.np = riscv_of_intc_domain_node();
> +	oirq.args_count = 1;
> +	oirq.args[0] = RV_IRQ_TIMER;
> +	riscv_clock_event_irq = irq_create_of_mapping(&oirq);
> +	if (!riscv_clock_event_irq)
> +		return -ENODEV;
> +
>  	pr_info("%s: Registering clocksource cpuid [%d] hartid [%d]\n",
>  	       __func__, cpuid, hartid);
>  	error = clocksource_register_hz(&riscv_clocksource, riscv_timebase);
> @@ -126,6 +142,14 @@ static int __init riscv_timer_init_dt(struct device_node *n)
>  
>  	sched_clock_register(riscv_sched_clock, 64, riscv_timebase);
>  
> +	error = request_percpu_irq(riscv_clock_event_irq,
> +				    riscv_timer_interrupt,
> +				    "riscv-timer", &riscv_clock_event);
> +	if (error) {
> +		pr_err("registering percpu irq failed [%d]\n", error);
> +		return error;
> +	}
> +
>  	error = cpuhp_setup_state(CPUHP_AP_RISCV_TIMER_STARTING,
>  			 "clockevents/riscv/timer:starting",
>  			 riscv_timer_starting_cpu, riscv_timer_dying_cpu);
> diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> index 2f364e6a87f9..d4fbc3543459 100644
> --- a/drivers/irqchip/irq-riscv-intc.c
> +++ b/drivers/irqchip/irq-riscv-intc.c
> @@ -23,20 +23,12 @@ static struct irq_domain *intc_domain;
>  
>  static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
>  {
> -	struct pt_regs *old_regs;
>  	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
>  
>  	if (unlikely(cause >= BITS_PER_LONG))
>  		panic("unexpected interrupt cause");
>  
>  	switch (cause) {
> -	case RV_IRQ_TIMER:
> -		old_regs = set_irq_regs(regs);
> -		irq_enter();
> -		riscv_timer_interrupt();
> -		irq_exit();
> -		set_irq_regs(old_regs);
> -		break;
>  #ifdef CONFIG_SMP
>  	case RV_IRQ_SOFT:
>  		/*
> 


-- 
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog

RE: [PATCH v5 4/6] clocksource/drivers/timer-riscv: Use per-CPU timer interrupt

[Anup Patel]

> -----Original Message-----
> From: Daniel Lezcano <daniel.lezcano@linaro.org>
> Sent: 22 May 2020 18:38
> To: Anup Patel <Anup.Patel@wdc.com>; Palmer Dabbelt
> <palmer@dabbelt.com>; Paul Walmsley <paul.walmsley@sifive.com>; Albert
> Ou <aou@eecs.berkeley.edu>; Thomas Gleixner <tglx@linutronix.de>; Jason
> Cooper <jason@lakedaemon.net>; Marc Zyngier <maz@kernel.org>
> Cc: Atish Patra <Atish.Patra@wdc.com>; Alistair Francis
> <Alistair.Francis@wdc.com>; Anup Patel <anup@brainfault.org>; linux-
> riscv@lists.infradead.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v5 4/6] clocksource/drivers/timer-riscv: Use per-CPU timer
> interrupt
> 
> On 21/05/2020 15:32, Anup Patel wrote:
> > Instead of directly calling RISC-V timer interrupt handler from RISC-V
> > local interrupt conntroller driver, this patch implements RISC-V timer
> > interrupt as a per-CPU interrupt using per-CPU APIs of Linux IRQ
> > subsystem.
> >
> > Signed-off-by: Anup Patel <anup.patel@wdc.com>
> 
> via which tree do you want this patch to be merged ?

What is your suggestion ?

This series is primarily change in arch/riscv and drivers/irqchip.

Regards,
Anup

> 
> > ---
> >  arch/riscv/include/asm/irq.h      |  2 --
> >  drivers/clocksource/timer-riscv.c | 30 +++++++++++++++++++++++++++---
> > drivers/irqchip/irq-riscv-intc.c  |  8 --------
> >  3 files changed, 27 insertions(+), 13 deletions(-)
> >
> > diff --git a/arch/riscv/include/asm/irq.h
> > b/arch/riscv/include/asm/irq.h index a9e5f07a7e9c..9807ad164015 100644
> > --- a/arch/riscv/include/asm/irq.h
> > +++ b/arch/riscv/include/asm/irq.h
> > @@ -10,8 +10,6 @@
> >  #include <linux/interrupt.h>
> >  #include <linux/linkage.h>
> >
> > -void riscv_timer_interrupt(void);
> > -
> >  #include <asm-generic/irq.h>
> >
> >  #endif /* _ASM_RISCV_IRQ_H */
> > diff --git a/drivers/clocksource/timer-riscv.c
> > b/drivers/clocksource/timer-riscv.c
> > index c4f15c4068c0..5fb7c5ba5c91 100644
> > --- a/drivers/clocksource/timer-riscv.c
> > +++ b/drivers/clocksource/timer-riscv.c
> > @@ -14,6 +14,9 @@
> >  #include <linux/irq.h>
> >  #include <linux/sched_clock.h>
> >  #include <linux/io-64-nonatomic-lo-hi.h>
> > +#include <linux/irqchip/irq-riscv-intc.h> #include
> > +<linux/interrupt.h> #include <linux/of_irq.h>
> >  #include <asm/smp.h>
> >  #include <asm/sbi.h>
> >
> > @@ -39,6 +42,7 @@ static int riscv_clock_next_event(unsigned long delta,
> >  	return 0;
> >  }
> >
> > +static unsigned int riscv_clock_event_irq;
> >  static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
> >  	.name			= "riscv_timer_clockevent",
> >  	.features		= CLOCK_EVT_FEAT_ONESHOT,
> > @@ -74,30 +78,35 @@ static int riscv_timer_starting_cpu(unsigned int cpu)
> >  	struct clock_event_device *ce = per_cpu_ptr(&riscv_clock_event,
> > cpu);
> >
> >  	ce->cpumask = cpumask_of(cpu);
> > +	ce->irq = riscv_clock_event_irq;
> >  	clockevents_config_and_register(ce, riscv_timebase, 100,
> > 0x7fffffff);
> >
> > -	csr_set(CSR_IE, IE_TIE);
> > +	enable_percpu_irq(riscv_clock_event_irq,
> > +			  irq_get_trigger_type(riscv_clock_event_irq));
> >  	return 0;
> >  }
> >
> >  static int riscv_timer_dying_cpu(unsigned int cpu)  {
> > -	csr_clear(CSR_IE, IE_TIE);
> > +	disable_percpu_irq(riscv_clock_event_irq);
> >  	return 0;
> >  }
> >
> >  /* called directly from the low-level interrupt handler */ -void
> > riscv_timer_interrupt(void)
> > +static irqreturn_t riscv_timer_interrupt(int irq, void *dev_id)
> >  {
> >  	struct clock_event_device *evdev = this_cpu_ptr(&riscv_clock_event);
> >
> >  	csr_clear(CSR_IE, IE_TIE);
> >  	evdev->event_handler(evdev);
> > +
> > +	return IRQ_HANDLED;
> >  }
> >
> >  static int __init riscv_timer_init_dt(struct device_node *n)  {
> >  	int cpuid, hartid, error;
> > +	struct of_phandle_args oirq;
> >
> >  	hartid = riscv_of_processor_hartid(n);
> >  	if (hartid < 0) {
> > @@ -115,6 +124,13 @@ static int __init riscv_timer_init_dt(struct
> device_node *n)
> >  	if (cpuid != smp_processor_id())
> >  		return 0;
> >
> > +	oirq.np = riscv_of_intc_domain_node();
> > +	oirq.args_count = 1;
> > +	oirq.args[0] = RV_IRQ_TIMER;
> > +	riscv_clock_event_irq = irq_create_of_mapping(&oirq);
> > +	if (!riscv_clock_event_irq)
> > +		return -ENODEV;
> > +
> >  	pr_info("%s: Registering clocksource cpuid [%d] hartid [%d]\n",
> >  	       __func__, cpuid, hartid);
> >  	error = clocksource_register_hz(&riscv_clocksource, riscv_timebase);
> > @@ -126,6 +142,14 @@ static int __init riscv_timer_init_dt(struct
> > device_node *n)
> >
> >  	sched_clock_register(riscv_sched_clock, 64, riscv_timebase);
> >
> > +	error = request_percpu_irq(riscv_clock_event_irq,
> > +				    riscv_timer_interrupt,
> > +				    "riscv-timer", &riscv_clock_event);
> > +	if (error) {
> > +		pr_err("registering percpu irq failed [%d]\n", error);
> > +		return error;
> > +	}
> > +
> >  	error = cpuhp_setup_state(CPUHP_AP_RISCV_TIMER_STARTING,
> >  			 "clockevents/riscv/timer:starting",
> >  			 riscv_timer_starting_cpu, riscv_timer_dying_cpu); diff
> --git
> > a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> > index 2f364e6a87f9..d4fbc3543459 100644
> > --- a/drivers/irqchip/irq-riscv-intc.c
> > +++ b/drivers/irqchip/irq-riscv-intc.c
> > @@ -23,20 +23,12 @@ static struct irq_domain *intc_domain;
> >
> >  static asmlinkage void riscv_intc_irq(struct pt_regs *regs)  {
> > -	struct pt_regs *old_regs;
> >  	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
> >
> >  	if (unlikely(cause >= BITS_PER_LONG))
> >  		panic("unexpected interrupt cause");
> >
> >  	switch (cause) {
> > -	case RV_IRQ_TIMER:
> > -		old_regs = set_irq_regs(regs);
> > -		irq_enter();
> > -		riscv_timer_interrupt();
> > -		irq_exit();
> > -		set_irq_regs(old_regs);
> > -		break;
> >  #ifdef CONFIG_SMP
> >  	case RV_IRQ_SOFT:
> >  		/*
> >
> 
> 
> --
> <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs
> 
> Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
> <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-
> blog/> Blog

RE: [PATCH v5 4/6] clocksource/drivers/timer-riscv: Use per-CPU timer interrupt

[Anup Patel]

> -----Original Message-----
> From: Daniel Lezcano <daniel.lezcano@linaro.org>
> Sent: 22 May 2020 18:38
> To: Anup Patel <Anup.Patel@wdc.com>; Palmer Dabbelt
> <palmer@dabbelt.com>; Paul Walmsley <paul.walmsley@sifive.com>; Albert
> Ou <aou@eecs.berkeley.edu>; Thomas Gleixner <tglx@linutronix.de>; Jason
> Cooper <jason@lakedaemon.net>; Marc Zyngier <maz@kernel.org>
> Cc: Atish Patra <Atish.Patra@wdc.com>; Alistair Francis
> <Alistair.Francis@wdc.com>; Anup Patel <anup@brainfault.org>; linux-
> riscv@lists.infradead.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v5 4/6] clocksource/drivers/timer-riscv: Use per-CPU timer
> interrupt
> 
> On 21/05/2020 15:32, Anup Patel wrote:
> > Instead of directly calling RISC-V timer interrupt handler from RISC-V
> > local interrupt conntroller driver, this patch implements RISC-V timer
> > interrupt as a per-CPU interrupt using per-CPU APIs of Linux IRQ
> > subsystem.
> >
> > Signed-off-by: Anup Patel <anup.patel@wdc.com>
> 
> via which tree do you want this patch to be merged ?

What is your suggestion ?

This series is primarily change in arch/riscv and drivers/irqchip.

Regards,
Anup

> 
> > ---
> >  arch/riscv/include/asm/irq.h      |  2 --
> >  drivers/clocksource/timer-riscv.c | 30 +++++++++++++++++++++++++++---
> > drivers/irqchip/irq-riscv-intc.c  |  8 --------
> >  3 files changed, 27 insertions(+), 13 deletions(-)
> >
> > diff --git a/arch/riscv/include/asm/irq.h
> > b/arch/riscv/include/asm/irq.h index a9e5f07a7e9c..9807ad164015 100644
> > --- a/arch/riscv/include/asm/irq.h
> > +++ b/arch/riscv/include/asm/irq.h
> > @@ -10,8 +10,6 @@
> >  #include <linux/interrupt.h>
> >  #include <linux/linkage.h>
> >
> > -void riscv_timer_interrupt(void);
> > -
> >  #include <asm-generic/irq.h>
> >
> >  #endif /* _ASM_RISCV_IRQ_H */
> > diff --git a/drivers/clocksource/timer-riscv.c
> > b/drivers/clocksource/timer-riscv.c
> > index c4f15c4068c0..5fb7c5ba5c91 100644
> > --- a/drivers/clocksource/timer-riscv.c
> > +++ b/drivers/clocksource/timer-riscv.c
> > @@ -14,6 +14,9 @@
> >  #include <linux/irq.h>
> >  #include <linux/sched_clock.h>
> >  #include <linux/io-64-nonatomic-lo-hi.h>
> > +#include <linux/irqchip/irq-riscv-intc.h> #include
> > +<linux/interrupt.h> #include <linux/of_irq.h>
> >  #include <asm/smp.h>
> >  #include <asm/sbi.h>
> >
> > @@ -39,6 +42,7 @@ static int riscv_clock_next_event(unsigned long delta,
> >  	return 0;
> >  }
> >
> > +static unsigned int riscv_clock_event_irq;
> >  static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
> >  	.name			= "riscv_timer_clockevent",
> >  	.features		= CLOCK_EVT_FEAT_ONESHOT,
> > @@ -74,30 +78,35 @@ static int riscv_timer_starting_cpu(unsigned int cpu)
> >  	struct clock_event_device *ce = per_cpu_ptr(&riscv_clock_event,
> > cpu);
> >
> >  	ce->cpumask = cpumask_of(cpu);
> > +	ce->irq = riscv_clock_event_irq;
> >  	clockevents_config_and_register(ce, riscv_timebase, 100,
> > 0x7fffffff);
> >
> > -	csr_set(CSR_IE, IE_TIE);
> > +	enable_percpu_irq(riscv_clock_event_irq,
> > +			  irq_get_trigger_type(riscv_clock_event_irq));
> >  	return 0;
> >  }
> >
> >  static int riscv_timer_dying_cpu(unsigned int cpu)  {
> > -	csr_clear(CSR_IE, IE_TIE);
> > +	disable_percpu_irq(riscv_clock_event_irq);
> >  	return 0;
> >  }
> >
> >  /* called directly from the low-level interrupt handler */ -void
> > riscv_timer_interrupt(void)
> > +static irqreturn_t riscv_timer_interrupt(int irq, void *dev_id)
> >  {
> >  	struct clock_event_device *evdev = this_cpu_ptr(&riscv_clock_event);
> >
> >  	csr_clear(CSR_IE, IE_TIE);
> >  	evdev->event_handler(evdev);
> > +
> > +	return IRQ_HANDLED;
> >  }
> >
> >  static int __init riscv_timer_init_dt(struct device_node *n)  {
> >  	int cpuid, hartid, error;
> > +	struct of_phandle_args oirq;
> >
> >  	hartid = riscv_of_processor_hartid(n);
> >  	if (hartid < 0) {
> > @@ -115,6 +124,13 @@ static int __init riscv_timer_init_dt(struct
> device_node *n)
> >  	if (cpuid != smp_processor_id())
> >  		return 0;
> >
> > +	oirq.np = riscv_of_intc_domain_node();
> > +	oirq.args_count = 1;
> > +	oirq.args[0] = RV_IRQ_TIMER;
> > +	riscv_clock_event_irq = irq_create_of_mapping(&oirq);
> > +	if (!riscv_clock_event_irq)
> > +		return -ENODEV;
> > +
> >  	pr_info("%s: Registering clocksource cpuid [%d] hartid [%d]\n",
> >  	       __func__, cpuid, hartid);
> >  	error = clocksource_register_hz(&riscv_clocksource, riscv_timebase);
> > @@ -126,6 +142,14 @@ static int __init riscv_timer_init_dt(struct
> > device_node *n)
> >
> >  	sched_clock_register(riscv_sched_clock, 64, riscv_timebase);
> >
> > +	error = request_percpu_irq(riscv_clock_event_irq,
> > +				    riscv_timer_interrupt,
> > +				    "riscv-timer", &riscv_clock_event);
> > +	if (error) {
> > +		pr_err("registering percpu irq failed [%d]\n", error);
> > +		return error;
> > +	}
> > +
> >  	error = cpuhp_setup_state(CPUHP_AP_RISCV_TIMER_STARTING,
> >  			 "clockevents/riscv/timer:starting",
> >  			 riscv_timer_starting_cpu, riscv_timer_dying_cpu); diff
> --git
> > a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> > index 2f364e6a87f9..d4fbc3543459 100644
> > --- a/drivers/irqchip/irq-riscv-intc.c
> > +++ b/drivers/irqchip/irq-riscv-intc.c
> > @@ -23,20 +23,12 @@ static struct irq_domain *intc_domain;
> >
> >  static asmlinkage void riscv_intc_irq(struct pt_regs *regs)  {
> > -	struct pt_regs *old_regs;
> >  	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
> >
> >  	if (unlikely(cause >= BITS_PER_LONG))
> >  		panic("unexpected interrupt cause");
> >
> >  	switch (cause) {
> > -	case RV_IRQ_TIMER:
> > -		old_regs = set_irq_regs(regs);
> > -		irq_enter();
> > -		riscv_timer_interrupt();
> > -		irq_exit();
> > -		set_irq_regs(old_regs);
> > -		break;
> >  #ifdef CONFIG_SMP
> >  	case RV_IRQ_SOFT:
> >  		/*
> >
> 
> 
> --
> <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs
> 
> Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
> <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-
> blog/> Blog

Re: [PATCH v5 4/6] clocksource/drivers/timer-riscv: Use per-CPU timer interrupt

[Atish Patra]

On Thu, May 21, 2020 at 6:34 AM Anup Patel <anup.patel@wdc.com> wrote:
>
> Instead of directly calling RISC-V timer interrupt handler from
> RISC-V local interrupt conntroller driver, this patch implements
> RISC-V timer interrupt as a per-CPU interrupt using per-CPU APIs
> of Linux IRQ subsystem.
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/include/asm/irq.h      |  2 --
>  drivers/clocksource/timer-riscv.c | 30 +++++++++++++++++++++++++++---
>  drivers/irqchip/irq-riscv-intc.c  |  8 --------
>  3 files changed, 27 insertions(+), 13 deletions(-)
>
> diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
> index a9e5f07a7e9c..9807ad164015 100644
> --- a/arch/riscv/include/asm/irq.h
> +++ b/arch/riscv/include/asm/irq.h
> @@ -10,8 +10,6 @@
>  #include <linux/interrupt.h>
>  #include <linux/linkage.h>
>
> -void riscv_timer_interrupt(void);
> -
>  #include <asm-generic/irq.h>
>
>  #endif /* _ASM_RISCV_IRQ_H */
> diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c
> index c4f15c4068c0..5fb7c5ba5c91 100644
> --- a/drivers/clocksource/timer-riscv.c
> +++ b/drivers/clocksource/timer-riscv.c
> @@ -14,6 +14,9 @@
>  #include <linux/irq.h>
>  #include <linux/sched_clock.h>
>  #include <linux/io-64-nonatomic-lo-hi.h>
> +#include <linux/irqchip/irq-riscv-intc.h>
> +#include <linux/interrupt.h>
> +#include <linux/of_irq.h>
>  #include <asm/smp.h>
>  #include <asm/sbi.h>
>
> @@ -39,6 +42,7 @@ static int riscv_clock_next_event(unsigned long delta,
>         return 0;
>  }
>
> +static unsigned int riscv_clock_event_irq;
>  static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
>         .name                   = "riscv_timer_clockevent",
>         .features               = CLOCK_EVT_FEAT_ONESHOT,
> @@ -74,30 +78,35 @@ static int riscv_timer_starting_cpu(unsigned int cpu)
>         struct clock_event_device *ce = per_cpu_ptr(&riscv_clock_event, cpu);
>
>         ce->cpumask = cpumask_of(cpu);
> +       ce->irq = riscv_clock_event_irq;
>         clockevents_config_and_register(ce, riscv_timebase, 100, 0x7fffffff);
>
> -       csr_set(CSR_IE, IE_TIE);
> +       enable_percpu_irq(riscv_clock_event_irq,
> +                         irq_get_trigger_type(riscv_clock_event_irq));
>         return 0;
>  }
>
>  static int riscv_timer_dying_cpu(unsigned int cpu)
>  {
> -       csr_clear(CSR_IE, IE_TIE);
> +       disable_percpu_irq(riscv_clock_event_irq);
>         return 0;
>  }
>
>  /* called directly from the low-level interrupt handler */
> -void riscv_timer_interrupt(void)
> +static irqreturn_t riscv_timer_interrupt(int irq, void *dev_id)
>  {
>         struct clock_event_device *evdev = this_cpu_ptr(&riscv_clock_event);
>
>         csr_clear(CSR_IE, IE_TIE);
>         evdev->event_handler(evdev);
> +
> +       return IRQ_HANDLED;
>  }
>
>  static int __init riscv_timer_init_dt(struct device_node *n)
>  {
>         int cpuid, hartid, error;
> +       struct of_phandle_args oirq;
>
>         hartid = riscv_of_processor_hartid(n);
>         if (hartid < 0) {
> @@ -115,6 +124,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)
>         if (cpuid != smp_processor_id())
>                 return 0;
>
> +       oirq.np = riscv_of_intc_domain_node();
> +       oirq.args_count = 1;
> +       oirq.args[0] = RV_IRQ_TIMER;
> +       riscv_clock_event_irq = irq_create_of_mapping(&oirq);
> +       if (!riscv_clock_event_irq)
> +               return -ENODEV;
> +

As the error case will result in a system without a clock, a warning message
may be a good option.

>         pr_info("%s: Registering clocksource cpuid [%d] hartid [%d]\n",
>                __func__, cpuid, hartid);
>         error = clocksource_register_hz(&riscv_clocksource, riscv_timebase);
> @@ -126,6 +142,14 @@ static int __init riscv_timer_init_dt(struct device_node *n)
>
>         sched_clock_register(riscv_sched_clock, 64, riscv_timebase);
>
> +       error = request_percpu_irq(riscv_clock_event_irq,
> +                                   riscv_timer_interrupt,
> +                                   "riscv-timer", &riscv_clock_event);
> +       if (error) {
> +               pr_err("registering percpu irq failed [%d]\n", error);
> +               return error;
> +       }
> +
>         error = cpuhp_setup_state(CPUHP_AP_RISCV_TIMER_STARTING,
>                          "clockevents/riscv/timer:starting",
>                          riscv_timer_starting_cpu, riscv_timer_dying_cpu);
> diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> index 2f364e6a87f9..d4fbc3543459 100644
> --- a/drivers/irqchip/irq-riscv-intc.c
> +++ b/drivers/irqchip/irq-riscv-intc.c
> @@ -23,20 +23,12 @@ static struct irq_domain *intc_domain;
>
>  static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
>  {
> -       struct pt_regs *old_regs;
>         unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
>
>         if (unlikely(cause >= BITS_PER_LONG))
>                 panic("unexpected interrupt cause");
>
>         switch (cause) {
> -       case RV_IRQ_TIMER:
> -               old_regs = set_irq_regs(regs);
> -               irq_enter();
> -               riscv_timer_interrupt();
> -               irq_exit();
> -               set_irq_regs(old_regs);
> -               break;
>  #ifdef CONFIG_SMP
>         case RV_IRQ_SOFT:
>                 /*
> --
> 2.25.1
>
>
Otherwise, LGTM.

Reviewed-by: Atish Patra <atish.patra@wdc.com>

-- 
Regards,
Atish

Re: [PATCH v5 4/6] clocksource/drivers/timer-riscv: Use per-CPU timer interrupt

[Atish Patra]

On Thu, May 21, 2020 at 6:34 AM Anup Patel <anup.patel@wdc.com> wrote:
>
> Instead of directly calling RISC-V timer interrupt handler from
> RISC-V local interrupt conntroller driver, this patch implements
> RISC-V timer interrupt as a per-CPU interrupt using per-CPU APIs
> of Linux IRQ subsystem.
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/include/asm/irq.h      |  2 --
>  drivers/clocksource/timer-riscv.c | 30 +++++++++++++++++++++++++++---
>  drivers/irqchip/irq-riscv-intc.c  |  8 --------
>  3 files changed, 27 insertions(+), 13 deletions(-)
>
> diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
> index a9e5f07a7e9c..9807ad164015 100644
> --- a/arch/riscv/include/asm/irq.h
> +++ b/arch/riscv/include/asm/irq.h
> @@ -10,8 +10,6 @@
>  #include <linux/interrupt.h>
>  #include <linux/linkage.h>
>
> -void riscv_timer_interrupt(void);
> -
>  #include <asm-generic/irq.h>
>
>  #endif /* _ASM_RISCV_IRQ_H */
> diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c
> index c4f15c4068c0..5fb7c5ba5c91 100644
> --- a/drivers/clocksource/timer-riscv.c
> +++ b/drivers/clocksource/timer-riscv.c
> @@ -14,6 +14,9 @@
>  #include <linux/irq.h>
>  #include <linux/sched_clock.h>
>  #include <linux/io-64-nonatomic-lo-hi.h>
> +#include <linux/irqchip/irq-riscv-intc.h>
> +#include <linux/interrupt.h>
> +#include <linux/of_irq.h>
>  #include <asm/smp.h>
>  #include <asm/sbi.h>
>
> @@ -39,6 +42,7 @@ static int riscv_clock_next_event(unsigned long delta,
>         return 0;
>  }
>
> +static unsigned int riscv_clock_event_irq;
>  static DEFINE_PER_CPU(struct clock_event_device, riscv_clock_event) = {
>         .name                   = "riscv_timer_clockevent",
>         .features               = CLOCK_EVT_FEAT_ONESHOT,
> @@ -74,30 +78,35 @@ static int riscv_timer_starting_cpu(unsigned int cpu)
>         struct clock_event_device *ce = per_cpu_ptr(&riscv_clock_event, cpu);
>
>         ce->cpumask = cpumask_of(cpu);
> +       ce->irq = riscv_clock_event_irq;
>         clockevents_config_and_register(ce, riscv_timebase, 100, 0x7fffffff);
>
> -       csr_set(CSR_IE, IE_TIE);
> +       enable_percpu_irq(riscv_clock_event_irq,
> +                         irq_get_trigger_type(riscv_clock_event_irq));
>         return 0;
>  }
>
>  static int riscv_timer_dying_cpu(unsigned int cpu)
>  {
> -       csr_clear(CSR_IE, IE_TIE);
> +       disable_percpu_irq(riscv_clock_event_irq);
>         return 0;
>  }
>
>  /* called directly from the low-level interrupt handler */
> -void riscv_timer_interrupt(void)
> +static irqreturn_t riscv_timer_interrupt(int irq, void *dev_id)
>  {
>         struct clock_event_device *evdev = this_cpu_ptr(&riscv_clock_event);
>
>         csr_clear(CSR_IE, IE_TIE);
>         evdev->event_handler(evdev);
> +
> +       return IRQ_HANDLED;
>  }
>
>  static int __init riscv_timer_init_dt(struct device_node *n)
>  {
>         int cpuid, hartid, error;
> +       struct of_phandle_args oirq;
>
>         hartid = riscv_of_processor_hartid(n);
>         if (hartid < 0) {
> @@ -115,6 +124,13 @@ static int __init riscv_timer_init_dt(struct device_node *n)
>         if (cpuid != smp_processor_id())
>                 return 0;
>
> +       oirq.np = riscv_of_intc_domain_node();
> +       oirq.args_count = 1;
> +       oirq.args[0] = RV_IRQ_TIMER;
> +       riscv_clock_event_irq = irq_create_of_mapping(&oirq);
> +       if (!riscv_clock_event_irq)
> +               return -ENODEV;
> +

As the error case will result in a system without a clock, a warning message
may be a good option.

>         pr_info("%s: Registering clocksource cpuid [%d] hartid [%d]\n",
>                __func__, cpuid, hartid);
>         error = clocksource_register_hz(&riscv_clocksource, riscv_timebase);
> @@ -126,6 +142,14 @@ static int __init riscv_timer_init_dt(struct device_node *n)
>
>         sched_clock_register(riscv_sched_clock, 64, riscv_timebase);
>
> +       error = request_percpu_irq(riscv_clock_event_irq,
> +                                   riscv_timer_interrupt,
> +                                   "riscv-timer", &riscv_clock_event);
> +       if (error) {
> +               pr_err("registering percpu irq failed [%d]\n", error);
> +               return error;
> +       }
> +
>         error = cpuhp_setup_state(CPUHP_AP_RISCV_TIMER_STARTING,
>                          "clockevents/riscv/timer:starting",
>                          riscv_timer_starting_cpu, riscv_timer_dying_cpu);
> diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> index 2f364e6a87f9..d4fbc3543459 100644
> --- a/drivers/irqchip/irq-riscv-intc.c
> +++ b/drivers/irqchip/irq-riscv-intc.c
> @@ -23,20 +23,12 @@ static struct irq_domain *intc_domain;
>
>  static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
>  {
> -       struct pt_regs *old_regs;
>         unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
>
>         if (unlikely(cause >= BITS_PER_LONG))
>                 panic("unexpected interrupt cause");
>
>         switch (cause) {
> -       case RV_IRQ_TIMER:
> -               old_regs = set_irq_regs(regs);
> -               irq_enter();
> -               riscv_timer_interrupt();
> -               irq_exit();
> -               set_irq_regs(old_regs);
> -               break;
>  #ifdef CONFIG_SMP
>         case RV_IRQ_SOFT:
>                 /*
> --
> 2.25.1
>
>
Otherwise, LGTM.

Reviewed-by: Atish Patra <atish.patra@wdc.com>

-- 
Regards,
Atish

[PATCH v5 5/6] RISC-V: Remove do_IRQ() function

[Anup Patel]

The only thing do_IRQ() does is call handle_arch_irq function
pointer. We can very well call handle_arch_irq function pointer
directly from assembly and remove do_IRQ() function hence this
patch.

Signed-off-by: Anup Patel <anup.patel@wdc.com>
---
 arch/riscv/kernel/entry.S | 4 +++-
 arch/riscv/kernel/irq.c   | 6 ------
 2 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 56d071b2c0a1..cae7e6d4c7ef 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -106,7 +106,9 @@ _save_context:
 
 	/* Handle interrupts */
 	move a0, sp /* pt_regs */
-	tail do_IRQ
+	la a1, handle_arch_irq
+	REG_L a1, (a1)
+	jr a1
 1:
 	/*
 	 * Exceptions run with interrupts enabled or disabled depending on the
diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
index eb8777642ce6..7207fa08d78f 100644
--- a/arch/riscv/kernel/irq.c
+++ b/arch/riscv/kernel/irq.c
@@ -16,12 +16,6 @@ int arch_show_interrupts(struct seq_file *p, int prec)
 	return 0;
 }
 
-asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
-{
-	if (handle_arch_irq)
-		handle_arch_irq(regs);
-}
-
 void __init init_IRQ(void)
 {
 	irqchip_init();
-- 
2.25.1

[PATCH v5 5/6] RISC-V: Remove do_IRQ() function

[Anup Patel]

The only thing do_IRQ() does is call handle_arch_irq function
pointer. We can very well call handle_arch_irq function pointer
directly from assembly and remove do_IRQ() function hence this
patch.

Signed-off-by: Anup Patel <anup.patel@wdc.com>
---
 arch/riscv/kernel/entry.S | 4 +++-
 arch/riscv/kernel/irq.c   | 6 ------
 2 files changed, 3 insertions(+), 7 deletions(-)

diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 56d071b2c0a1..cae7e6d4c7ef 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -106,7 +106,9 @@ _save_context:
 
 	/* Handle interrupts */
 	move a0, sp /* pt_regs */
-	tail do_IRQ
+	la a1, handle_arch_irq
+	REG_L a1, (a1)
+	jr a1
 1:
 	/*
 	 * Exceptions run with interrupts enabled or disabled depending on the
diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
index eb8777642ce6..7207fa08d78f 100644
--- a/arch/riscv/kernel/irq.c
+++ b/arch/riscv/kernel/irq.c
@@ -16,12 +16,6 @@ int arch_show_interrupts(struct seq_file *p, int prec)
 	return 0;
 }
 
-asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
-{
-	if (handle_arch_irq)
-		handle_arch_irq(regs);
-}
-
 void __init init_IRQ(void)
 {
 	irqchip_init();
-- 
2.25.1

Re: [PATCH v5 5/6] RISC-V: Remove do_IRQ() function

[Atish Patra]

On Thu, May 21, 2020 at 6:34 AM Anup Patel <anup.patel@wdc.com> wrote:
>
> The only thing do_IRQ() does is call handle_arch_irq function
> pointer. We can very well call handle_arch_irq function pointer
> directly from assembly and remove do_IRQ() function hence this
> patch.
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/kernel/entry.S | 4 +++-
>  arch/riscv/kernel/irq.c   | 6 ------
>  2 files changed, 3 insertions(+), 7 deletions(-)
>
> diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
> index 56d071b2c0a1..cae7e6d4c7ef 100644
> --- a/arch/riscv/kernel/entry.S
> +++ b/arch/riscv/kernel/entry.S
> @@ -106,7 +106,9 @@ _save_context:
>
>         /* Handle interrupts */
>         move a0, sp /* pt_regs */
> -       tail do_IRQ
> +       la a1, handle_arch_irq
> +       REG_L a1, (a1)
> +       jr a1
>  1:
>         /*
>          * Exceptions run with interrupts enabled or disabled depending on the
> diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> index eb8777642ce6..7207fa08d78f 100644
> --- a/arch/riscv/kernel/irq.c
> +++ b/arch/riscv/kernel/irq.c
> @@ -16,12 +16,6 @@ int arch_show_interrupts(struct seq_file *p, int prec)
>         return 0;
>  }
>
> -asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
> -{
> -       if (handle_arch_irq)
> -               handle_arch_irq(regs);
> -}
> -
>  void __init init_IRQ(void)
>  {
>         irqchip_init();
> --
> 2.25.1
>
>


Reviewed-by: Atish Patra <atish.patra@wdc.com>
-- 
Regards,
Atish

Re: [PATCH v5 5/6] RISC-V: Remove do_IRQ() function

[Atish Patra]

On Thu, May 21, 2020 at 6:34 AM Anup Patel <anup.patel@wdc.com> wrote:
>
> The only thing do_IRQ() does is call handle_arch_irq function
> pointer. We can very well call handle_arch_irq function pointer
> directly from assembly and remove do_IRQ() function hence this
> patch.
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/kernel/entry.S | 4 +++-
>  arch/riscv/kernel/irq.c   | 6 ------
>  2 files changed, 3 insertions(+), 7 deletions(-)
>
> diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
> index 56d071b2c0a1..cae7e6d4c7ef 100644
> --- a/arch/riscv/kernel/entry.S
> +++ b/arch/riscv/kernel/entry.S
> @@ -106,7 +106,9 @@ _save_context:
>
>         /* Handle interrupts */
>         move a0, sp /* pt_regs */
> -       tail do_IRQ
> +       la a1, handle_arch_irq
> +       REG_L a1, (a1)
> +       jr a1
>  1:
>         /*
>          * Exceptions run with interrupts enabled or disabled depending on the
> diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c
> index eb8777642ce6..7207fa08d78f 100644
> --- a/arch/riscv/kernel/irq.c
> +++ b/arch/riscv/kernel/irq.c
> @@ -16,12 +16,6 @@ int arch_show_interrupts(struct seq_file *p, int prec)
>         return 0;
>  }
>
> -asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs)
> -{
> -       if (handle_arch_irq)
> -               handle_arch_irq(regs);
> -}
> -
>  void __init init_IRQ(void)
>  {
>         irqchip_init();
> --
> 2.25.1
>
>


Reviewed-by: Atish Patra <atish.patra@wdc.com>
-- 
Regards,
Atish

[PATCH v5 6/6] RISC-V: Force select RISCV_INTC for CONFIG_RISCV

[Anup Patel]

The RISC-V per-HART local interrupt controller driver is mandatory
for all RISC-V system (with/without MMU) hence we force select it
for CONFIG_RISCV (just like RISCV_TIMER).

Signed-off-by: Anup Patel <anup.patel@wdc.com>
---
 arch/riscv/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 822cb0e1a380..2cf0c83c1a47 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -51,6 +51,7 @@ config RISCV
 	select THREAD_INFO_IN_TASK
 	select PCI_DOMAINS_GENERIC if PCI
 	select PCI_MSI if PCI
+	select RISCV_INTC
 	select RISCV_TIMER
 	select GENERIC_IRQ_MULTI_HANDLER
 	select GENERIC_ARCH_TOPOLOGY if SMP
-- 
2.25.1

[PATCH v5 6/6] RISC-V: Force select RISCV_INTC for CONFIG_RISCV

[Anup Patel]

The RISC-V per-HART local interrupt controller driver is mandatory
for all RISC-V system (with/without MMU) hence we force select it
for CONFIG_RISCV (just like RISCV_TIMER).

Signed-off-by: Anup Patel <anup.patel@wdc.com>
---
 arch/riscv/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 822cb0e1a380..2cf0c83c1a47 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -51,6 +51,7 @@ config RISCV
 	select THREAD_INFO_IN_TASK
 	select PCI_DOMAINS_GENERIC if PCI
 	select PCI_MSI if PCI
+	select RISCV_INTC
 	select RISCV_TIMER
 	select GENERIC_IRQ_MULTI_HANDLER
 	select GENERIC_ARCH_TOPOLOGY if SMP
-- 
2.25.1

Re: [PATCH v5 6/6] RISC-V: Force select RISCV_INTC for CONFIG_RISCV

[Atish Patra]

On Thu, May 21, 2020 at 6:34 AM Anup Patel <anup.patel@wdc.com> wrote:
>
> The RISC-V per-HART local interrupt controller driver is mandatory
> for all RISC-V system (with/without MMU) hence we force select it
> for CONFIG_RISCV (just like RISCV_TIMER).
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/Kconfig | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 822cb0e1a380..2cf0c83c1a47 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -51,6 +51,7 @@ config RISCV
>         select THREAD_INFO_IN_TASK
>         select PCI_DOMAINS_GENERIC if PCI
>         select PCI_MSI if PCI
> +       select RISCV_INTC
>         select RISCV_TIMER
>         select GENERIC_IRQ_MULTI_HANDLER
>         select GENERIC_ARCH_TOPOLOGY if SMP
> --
> 2.25.1
>
>

Reviewed-by: Atish Patra <atish.patra@wdc.com>

-- 
Regards,
Atish

Re: [PATCH v5 6/6] RISC-V: Force select RISCV_INTC for CONFIG_RISCV

[Atish Patra]

On Thu, May 21, 2020 at 6:34 AM Anup Patel <anup.patel@wdc.com> wrote:
>
> The RISC-V per-HART local interrupt controller driver is mandatory
> for all RISC-V system (with/without MMU) hence we force select it
> for CONFIG_RISCV (just like RISCV_TIMER).
>
> Signed-off-by: Anup Patel <anup.patel@wdc.com>
> ---
>  arch/riscv/Kconfig | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 822cb0e1a380..2cf0c83c1a47 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -51,6 +51,7 @@ config RISCV
>         select THREAD_INFO_IN_TASK
>         select PCI_DOMAINS_GENERIC if PCI
>         select PCI_MSI if PCI
> +       select RISCV_INTC
>         select RISCV_TIMER
>         select GENERIC_IRQ_MULTI_HANDLER
>         select GENERIC_ARCH_TOPOLOGY if SMP
> --
> 2.25.1
>
>

Reviewed-by: Atish Patra <atish.patra@wdc.com>

-- 
Regards,
Atish

Re: [PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[Palmer Dabbelt]

On Thu, 21 May 2020 06:32:55 PDT (-0700), Anup Patel wrote:
> This patchset provides a new RISC-V Local Interrupt Controller Driver
> for managing per-CPU local interrupts. The overall approach is inspired
> from the way per-CPU local interrupts are handled by Linux ARM64 and
> ARM GICv3 driver.
>
> Few advantages of this new driver over previous one are:
> 1. All local interrupts are registered as per-CPU interrupts
> 2. The RISC-V timer driver can register timer interrupt handler
>    using kernel irq subsystem without relying on arch/riscv to
>    explicitly call it's interrupt handler
> 3. The KVM RISC-V can use this driver to implement interrupt
>    handler for per-HART guest external interrupt defined by
>    the RISC-V H-Extension
> 4. In future, we can develop drivers for devices with per-HART
>    interrupts without changing arch code or this driver (example,
>    CLINT timer driver for RISC-V M-mode kernel)
>
> With this patchset, output of "cat /proc/interrupts" looks as follows:
>            CPU0       CPU1       CPU2       CPU3
>   2:        379          0          0          0  SiFive PLIC  10  ttyS0
>   3:        591          0          0          0  SiFive PLIC   8  virtio0
>   5:       5079      10821       8435      12984  RISC-V INTC   5  riscv-timer
> IPI0:      2045       2537        891        870  Rescheduling interrupts
> IPI1:         9        269         91        168  Function call interrupts
> IPI2:         0          0          0          0  CPU stop interrupts
>
> The patchset is based up Linux-5.7-rc6 and can be found at riscv_intc_v5
> branch of: https://github.com/avpatel/linux.git
>
> This series is tested on:
>  1. QEMU RV64 virt machine using Linux RISC-V S-mode
>  2. QEMU RV32 virt machine using Linux RISC-V S-mode
>  3. QEMU RV64 virt machine using Linux RISC-V M-mode (i.e. NoMMU)
>
> Changes since v4:
>  - Rebased to Linux-5.7-rc6 and multi-PLIC improvement patches
>  - Added separate patch to force select RISCV_INTC for CONFIG_RISCV
>  - Fixed the driver for Linux RISC-V NoMMU
>
> Changes since v3:
>  - Rebased to Linux-5.6-rc5 and Atish's PLIC patches
>  - Added separate patch to rename and move plic_find_hart_id()
>    to arch directory
>  - Use riscv_of_parent_hartid() in riscv_intc_init() instead of
>    atomic counter
>
> Changes since v2:
>  - Dropped PATCH2 since it was merged long-time back
>  - Rebased series from Linux-4.19-rc2 to Linux-5.6-rc2
>
> Changes since v1:
>  - Removed changes related to puggable IPI triggering
>  - Separate patch for self-contained IPI handling routine
>  - Removed patch for GENERIC_IRQ kconfig options
>  - Added patch to remove do_IRQ() function
>  - Rebased upon Atish's SMP patches
>
> Anup Patel (6):
>   RISC-V: self-contained IPI handling routine
>   RISC-V: Rename and move plic_find_hart_id() to arch directory
>   irqchip: RISC-V per-HART local interrupt controller driver
>   clocksource/drivers/timer-riscv: Use per-CPU timer interrupt
>   RISC-V: Remove do_IRQ() function
>   RISC-V: Force select RISCV_INTC for CONFIG_RISCV
>
>  arch/riscv/Kconfig                     |   2 +
>  arch/riscv/include/asm/irq.h           |   5 -
>  arch/riscv/include/asm/processor.h     |   1 +
>  arch/riscv/include/asm/smp.h           |   3 +
>  arch/riscv/kernel/cpu.c                |  16 +++
>  arch/riscv/kernel/entry.S              |   4 +-
>  arch/riscv/kernel/irq.c                |  33 +-----
>  arch/riscv/kernel/smp.c                |  11 +-
>  arch/riscv/kernel/traps.c              |   2 -
>  drivers/clocksource/timer-riscv.c      |  30 ++++-
>  drivers/irqchip/Kconfig                |  13 +++
>  drivers/irqchip/Makefile               |   1 +
>  drivers/irqchip/irq-riscv-intc.c       | 150 +++++++++++++++++++++++++
>  drivers/irqchip/irq-sifive-plic.c      |  52 +++++----
>  include/linux/cpuhotplug.h             |   1 +
>  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
>  16 files changed, 280 insertions(+), 64 deletions(-)
>  create mode 100644 drivers/irqchip/irq-riscv-intc.c
>  create mode 100644 include/linux/irqchip/irq-riscv-intc.h

So I read through this a bit, and while I haven't gone through every line of
code I'm somewhat inclined toward taking it.

During the original RISC-V port submission we went back and forth between
having this first-level interrupt controller in arch/riscv/ vs
drivers/irqchip/.  The original deciding factor was that the ISA mandated the
interrupt controller, but as that's proving to be less and less the case every
day (with the CLIC and M-mode Linux) it certainly seem sane to move all our
interrupt controller drivers out of arch/riscv/.

This is certainly a step in the right direction, and it handles some of the
more glaring issues (iscv_timer_interrupt and lacking IRQs for the CLINT).  I
think we should just go ahead and merge it, even though there might be some
more refactoring to do when we eventually end up with another interrupt
controller.

I think it's best if this all goes in through a single tree, as it seems more
work than it's worth to split it up.  I'm happy to take it through my tree if
that's OK with the irqchip folks?

Re: [PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[Palmer Dabbelt]

On Thu, 21 May 2020 06:32:55 PDT (-0700), Anup Patel wrote:
> This patchset provides a new RISC-V Local Interrupt Controller Driver
> for managing per-CPU local interrupts. The overall approach is inspired
> from the way per-CPU local interrupts are handled by Linux ARM64 and
> ARM GICv3 driver.
>
> Few advantages of this new driver over previous one are:
> 1. All local interrupts are registered as per-CPU interrupts
> 2. The RISC-V timer driver can register timer interrupt handler
>    using kernel irq subsystem without relying on arch/riscv to
>    explicitly call it's interrupt handler
> 3. The KVM RISC-V can use this driver to implement interrupt
>    handler for per-HART guest external interrupt defined by
>    the RISC-V H-Extension
> 4. In future, we can develop drivers for devices with per-HART
>    interrupts without changing arch code or this driver (example,
>    CLINT timer driver for RISC-V M-mode kernel)
>
> With this patchset, output of "cat /proc/interrupts" looks as follows:
>            CPU0       CPU1       CPU2       CPU3
>   2:        379          0          0          0  SiFive PLIC  10  ttyS0
>   3:        591          0          0          0  SiFive PLIC   8  virtio0
>   5:       5079      10821       8435      12984  RISC-V INTC   5  riscv-timer
> IPI0:      2045       2537        891        870  Rescheduling interrupts
> IPI1:         9        269         91        168  Function call interrupts
> IPI2:         0          0          0          0  CPU stop interrupts
>
> The patchset is based up Linux-5.7-rc6 and can be found at riscv_intc_v5
> branch of: https://github.com/avpatel/linux.git
>
> This series is tested on:
>  1. QEMU RV64 virt machine using Linux RISC-V S-mode
>  2. QEMU RV32 virt machine using Linux RISC-V S-mode
>  3. QEMU RV64 virt machine using Linux RISC-V M-mode (i.e. NoMMU)
>
> Changes since v4:
>  - Rebased to Linux-5.7-rc6 and multi-PLIC improvement patches
>  - Added separate patch to force select RISCV_INTC for CONFIG_RISCV
>  - Fixed the driver for Linux RISC-V NoMMU
>
> Changes since v3:
>  - Rebased to Linux-5.6-rc5 and Atish's PLIC patches
>  - Added separate patch to rename and move plic_find_hart_id()
>    to arch directory
>  - Use riscv_of_parent_hartid() in riscv_intc_init() instead of
>    atomic counter
>
> Changes since v2:
>  - Dropped PATCH2 since it was merged long-time back
>  - Rebased series from Linux-4.19-rc2 to Linux-5.6-rc2
>
> Changes since v1:
>  - Removed changes related to puggable IPI triggering
>  - Separate patch for self-contained IPI handling routine
>  - Removed patch for GENERIC_IRQ kconfig options
>  - Added patch to remove do_IRQ() function
>  - Rebased upon Atish's SMP patches
>
> Anup Patel (6):
>   RISC-V: self-contained IPI handling routine
>   RISC-V: Rename and move plic_find_hart_id() to arch directory
>   irqchip: RISC-V per-HART local interrupt controller driver
>   clocksource/drivers/timer-riscv: Use per-CPU timer interrupt
>   RISC-V: Remove do_IRQ() function
>   RISC-V: Force select RISCV_INTC for CONFIG_RISCV
>
>  arch/riscv/Kconfig                     |   2 +
>  arch/riscv/include/asm/irq.h           |   5 -
>  arch/riscv/include/asm/processor.h     |   1 +
>  arch/riscv/include/asm/smp.h           |   3 +
>  arch/riscv/kernel/cpu.c                |  16 +++
>  arch/riscv/kernel/entry.S              |   4 +-
>  arch/riscv/kernel/irq.c                |  33 +-----
>  arch/riscv/kernel/smp.c                |  11 +-
>  arch/riscv/kernel/traps.c              |   2 -
>  drivers/clocksource/timer-riscv.c      |  30 ++++-
>  drivers/irqchip/Kconfig                |  13 +++
>  drivers/irqchip/Makefile               |   1 +
>  drivers/irqchip/irq-riscv-intc.c       | 150 +++++++++++++++++++++++++
>  drivers/irqchip/irq-sifive-plic.c      |  52 +++++----
>  include/linux/cpuhotplug.h             |   1 +
>  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
>  16 files changed, 280 insertions(+), 64 deletions(-)
>  create mode 100644 drivers/irqchip/irq-riscv-intc.c
>  create mode 100644 include/linux/irqchip/irq-riscv-intc.h

So I read through this a bit, and while I haven't gone through every line of
code I'm somewhat inclined toward taking it.

During the original RISC-V port submission we went back and forth between
having this first-level interrupt controller in arch/riscv/ vs
drivers/irqchip/.  The original deciding factor was that the ISA mandated the
interrupt controller, but as that's proving to be less and less the case every
day (with the CLIC and M-mode Linux) it certainly seem sane to move all our
interrupt controller drivers out of arch/riscv/.

This is certainly a step in the right direction, and it handles some of the
more glaring issues (iscv_timer_interrupt and lacking IRQs for the CLINT).  I
think we should just go ahead and merge it, even though there might be some
more refactoring to do when we eventually end up with another interrupt
controller.

I think it's best if this all goes in through a single tree, as it seems more
work than it's worth to split it up.  I'm happy to take it through my tree if
that's OK with the irqchip folks?

Re: [PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[Anup Patel]

On Thu, May 28, 2020 at 12:17 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>
> On Thu, 21 May 2020 06:32:55 PDT (-0700), Anup Patel wrote:
> > This patchset provides a new RISC-V Local Interrupt Controller Driver
> > for managing per-CPU local interrupts. The overall approach is inspired
> > from the way per-CPU local interrupts are handled by Linux ARM64 and
> > ARM GICv3 driver.
> >
> > Few advantages of this new driver over previous one are:
> > 1. All local interrupts are registered as per-CPU interrupts
> > 2. The RISC-V timer driver can register timer interrupt handler
> >    using kernel irq subsystem without relying on arch/riscv to
> >    explicitly call it's interrupt handler
> > 3. The KVM RISC-V can use this driver to implement interrupt
> >    handler for per-HART guest external interrupt defined by
> >    the RISC-V H-Extension
> > 4. In future, we can develop drivers for devices with per-HART
> >    interrupts without changing arch code or this driver (example,
> >    CLINT timer driver for RISC-V M-mode kernel)
> >
> > With this patchset, output of "cat /proc/interrupts" looks as follows:
> >            CPU0       CPU1       CPU2       CPU3
> >   2:        379          0          0          0  SiFive PLIC  10  ttyS0
> >   3:        591          0          0          0  SiFive PLIC   8  virtio0
> >   5:       5079      10821       8435      12984  RISC-V INTC   5  riscv-timer
> > IPI0:      2045       2537        891        870  Rescheduling interrupts
> > IPI1:         9        269         91        168  Function call interrupts
> > IPI2:         0          0          0          0  CPU stop interrupts
> >
> > The patchset is based up Linux-5.7-rc6 and can be found at riscv_intc_v5
> > branch of: https://github.com/avpatel/linux.git
> >
> > This series is tested on:
> >  1. QEMU RV64 virt machine using Linux RISC-V S-mode
> >  2. QEMU RV32 virt machine using Linux RISC-V S-mode
> >  3. QEMU RV64 virt machine using Linux RISC-V M-mode (i.e. NoMMU)
> >
> > Changes since v4:
> >  - Rebased to Linux-5.7-rc6 and multi-PLIC improvement patches
> >  - Added separate patch to force select RISCV_INTC for CONFIG_RISCV
> >  - Fixed the driver for Linux RISC-V NoMMU
> >
> > Changes since v3:
> >  - Rebased to Linux-5.6-rc5 and Atish's PLIC patches
> >  - Added separate patch to rename and move plic_find_hart_id()
> >    to arch directory
> >  - Use riscv_of_parent_hartid() in riscv_intc_init() instead of
> >    atomic counter
> >
> > Changes since v2:
> >  - Dropped PATCH2 since it was merged long-time back
> >  - Rebased series from Linux-4.19-rc2 to Linux-5.6-rc2
> >
> > Changes since v1:
> >  - Removed changes related to puggable IPI triggering
> >  - Separate patch for self-contained IPI handling routine
> >  - Removed patch for GENERIC_IRQ kconfig options
> >  - Added patch to remove do_IRQ() function
> >  - Rebased upon Atish's SMP patches
> >
> > Anup Patel (6):
> >   RISC-V: self-contained IPI handling routine
> >   RISC-V: Rename and move plic_find_hart_id() to arch directory
> >   irqchip: RISC-V per-HART local interrupt controller driver
> >   clocksource/drivers/timer-riscv: Use per-CPU timer interrupt
> >   RISC-V: Remove do_IRQ() function
> >   RISC-V: Force select RISCV_INTC for CONFIG_RISCV
> >
> >  arch/riscv/Kconfig                     |   2 +
> >  arch/riscv/include/asm/irq.h           |   5 -
> >  arch/riscv/include/asm/processor.h     |   1 +
> >  arch/riscv/include/asm/smp.h           |   3 +
> >  arch/riscv/kernel/cpu.c                |  16 +++
> >  arch/riscv/kernel/entry.S              |   4 +-
> >  arch/riscv/kernel/irq.c                |  33 +-----
> >  arch/riscv/kernel/smp.c                |  11 +-
> >  arch/riscv/kernel/traps.c              |   2 -
> >  drivers/clocksource/timer-riscv.c      |  30 ++++-
> >  drivers/irqchip/Kconfig                |  13 +++
> >  drivers/irqchip/Makefile               |   1 +
> >  drivers/irqchip/irq-riscv-intc.c       | 150 +++++++++++++++++++++++++
> >  drivers/irqchip/irq-sifive-plic.c      |  52 +++++----
> >  include/linux/cpuhotplug.h             |   1 +
> >  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
> >  16 files changed, 280 insertions(+), 64 deletions(-)
> >  create mode 100644 drivers/irqchip/irq-riscv-intc.c
> >  create mode 100644 include/linux/irqchip/irq-riscv-intc.h
>
> So I read through this a bit, and while I haven't gone through every line of
> code I'm somewhat inclined toward taking it.
>
> During the original RISC-V port submission we went back and forth between
> having this first-level interrupt controller in arch/riscv/ vs
> drivers/irqchip/.  The original deciding factor was that the ISA mandated the
> interrupt controller, but as that's proving to be less and less the case every
> day (with the CLIC and M-mode Linux) it certainly seem sane to move all our
> interrupt controller drivers out of arch/riscv/.
>
> This is certainly a step in the right direction, and it handles some of the
> more glaring issues (iscv_timer_interrupt and lacking IRQs for the CLINT).  I
> think we should just go ahead and merge it, even though there might be some
> more refactoring to do when we eventually end up with another interrupt
> controller.
>
> I think it's best if this all goes in through a single tree, as it seems more
> work than it's worth to split it up.  I'm happy to take it through my tree if
> that's OK with the irqchip folks?

A small heads up...

Marc has queued a few PLIC improvement patches for Linux-next.
(https://patchwork.kernel.org/cover/11555051/)

This series (particularly PATCH3) is based upon above mentioned
PLIC patches.

Apart from above, I don't see any potential merge conflicts.

Regards,
Anup

Re: [PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[Anup Patel]

On Thu, May 28, 2020 at 12:17 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>
> On Thu, 21 May 2020 06:32:55 PDT (-0700), Anup Patel wrote:
> > This patchset provides a new RISC-V Local Interrupt Controller Driver
> > for managing per-CPU local interrupts. The overall approach is inspired
> > from the way per-CPU local interrupts are handled by Linux ARM64 and
> > ARM GICv3 driver.
> >
> > Few advantages of this new driver over previous one are:
> > 1. All local interrupts are registered as per-CPU interrupts
> > 2. The RISC-V timer driver can register timer interrupt handler
> >    using kernel irq subsystem without relying on arch/riscv to
> >    explicitly call it's interrupt handler
> > 3. The KVM RISC-V can use this driver to implement interrupt
> >    handler for per-HART guest external interrupt defined by
> >    the RISC-V H-Extension
> > 4. In future, we can develop drivers for devices with per-HART
> >    interrupts without changing arch code or this driver (example,
> >    CLINT timer driver for RISC-V M-mode kernel)
> >
> > With this patchset, output of "cat /proc/interrupts" looks as follows:
> >            CPU0       CPU1       CPU2       CPU3
> >   2:        379          0          0          0  SiFive PLIC  10  ttyS0
> >   3:        591          0          0          0  SiFive PLIC   8  virtio0
> >   5:       5079      10821       8435      12984  RISC-V INTC   5  riscv-timer
> > IPI0:      2045       2537        891        870  Rescheduling interrupts
> > IPI1:         9        269         91        168  Function call interrupts
> > IPI2:         0          0          0          0  CPU stop interrupts
> >
> > The patchset is based up Linux-5.7-rc6 and can be found at riscv_intc_v5
> > branch of: https://github.com/avpatel/linux.git
> >
> > This series is tested on:
> >  1. QEMU RV64 virt machine using Linux RISC-V S-mode
> >  2. QEMU RV32 virt machine using Linux RISC-V S-mode
> >  3. QEMU RV64 virt machine using Linux RISC-V M-mode (i.e. NoMMU)
> >
> > Changes since v4:
> >  - Rebased to Linux-5.7-rc6 and multi-PLIC improvement patches
> >  - Added separate patch to force select RISCV_INTC for CONFIG_RISCV
> >  - Fixed the driver for Linux RISC-V NoMMU
> >
> > Changes since v3:
> >  - Rebased to Linux-5.6-rc5 and Atish's PLIC patches
> >  - Added separate patch to rename and move plic_find_hart_id()
> >    to arch directory
> >  - Use riscv_of_parent_hartid() in riscv_intc_init() instead of
> >    atomic counter
> >
> > Changes since v2:
> >  - Dropped PATCH2 since it was merged long-time back
> >  - Rebased series from Linux-4.19-rc2 to Linux-5.6-rc2
> >
> > Changes since v1:
> >  - Removed changes related to puggable IPI triggering
> >  - Separate patch for self-contained IPI handling routine
> >  - Removed patch for GENERIC_IRQ kconfig options
> >  - Added patch to remove do_IRQ() function
> >  - Rebased upon Atish's SMP patches
> >
> > Anup Patel (6):
> >   RISC-V: self-contained IPI handling routine
> >   RISC-V: Rename and move plic_find_hart_id() to arch directory
> >   irqchip: RISC-V per-HART local interrupt controller driver
> >   clocksource/drivers/timer-riscv: Use per-CPU timer interrupt
> >   RISC-V: Remove do_IRQ() function
> >   RISC-V: Force select RISCV_INTC for CONFIG_RISCV
> >
> >  arch/riscv/Kconfig                     |   2 +
> >  arch/riscv/include/asm/irq.h           |   5 -
> >  arch/riscv/include/asm/processor.h     |   1 +
> >  arch/riscv/include/asm/smp.h           |   3 +
> >  arch/riscv/kernel/cpu.c                |  16 +++
> >  arch/riscv/kernel/entry.S              |   4 +-
> >  arch/riscv/kernel/irq.c                |  33 +-----
> >  arch/riscv/kernel/smp.c                |  11 +-
> >  arch/riscv/kernel/traps.c              |   2 -
> >  drivers/clocksource/timer-riscv.c      |  30 ++++-
> >  drivers/irqchip/Kconfig                |  13 +++
> >  drivers/irqchip/Makefile               |   1 +
> >  drivers/irqchip/irq-riscv-intc.c       | 150 +++++++++++++++++++++++++
> >  drivers/irqchip/irq-sifive-plic.c      |  52 +++++----
> >  include/linux/cpuhotplug.h             |   1 +
> >  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
> >  16 files changed, 280 insertions(+), 64 deletions(-)
> >  create mode 100644 drivers/irqchip/irq-riscv-intc.c
> >  create mode 100644 include/linux/irqchip/irq-riscv-intc.h
>
> So I read through this a bit, and while I haven't gone through every line of
> code I'm somewhat inclined toward taking it.
>
> During the original RISC-V port submission we went back and forth between
> having this first-level interrupt controller in arch/riscv/ vs
> drivers/irqchip/.  The original deciding factor was that the ISA mandated the
> interrupt controller, but as that's proving to be less and less the case every
> day (with the CLIC and M-mode Linux) it certainly seem sane to move all our
> interrupt controller drivers out of arch/riscv/.
>
> This is certainly a step in the right direction, and it handles some of the
> more glaring issues (iscv_timer_interrupt and lacking IRQs for the CLINT).  I
> think we should just go ahead and merge it, even though there might be some
> more refactoring to do when we eventually end up with another interrupt
> controller.
>
> I think it's best if this all goes in through a single tree, as it seems more
> work than it's worth to split it up.  I'm happy to take it through my tree if
> that's OK with the irqchip folks?

A small heads up...

Marc has queued a few PLIC improvement patches for Linux-next.
(https://patchwork.kernel.org/cover/11555051/)

This series (particularly PATCH3) is based upon above mentioned
PLIC patches.

Apart from above, I don't see any potential merge conflicts.

Regards,
Anup

Re: [PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[Palmer Dabbelt]

On Thu, 28 May 2020 20:57:26 PDT (-0700), anup@brainfault.org wrote:
> On Thu, May 28, 2020 at 12:17 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>>
>> On Thu, 21 May 2020 06:32:55 PDT (-0700), Anup Patel wrote:
>> > This patchset provides a new RISC-V Local Interrupt Controller Driver
>> > for managing per-CPU local interrupts. The overall approach is inspired
>> > from the way per-CPU local interrupts are handled by Linux ARM64 and
>> > ARM GICv3 driver.
>> >
>> > Few advantages of this new driver over previous one are:
>> > 1. All local interrupts are registered as per-CPU interrupts
>> > 2. The RISC-V timer driver can register timer interrupt handler
>> >    using kernel irq subsystem without relying on arch/riscv to
>> >    explicitly call it's interrupt handler
>> > 3. The KVM RISC-V can use this driver to implement interrupt
>> >    handler for per-HART guest external interrupt defined by
>> >    the RISC-V H-Extension
>> > 4. In future, we can develop drivers for devices with per-HART
>> >    interrupts without changing arch code or this driver (example,
>> >    CLINT timer driver for RISC-V M-mode kernel)
>> >
>> > With this patchset, output of "cat /proc/interrupts" looks as follows:
>> >            CPU0       CPU1       CPU2       CPU3
>> >   2:        379          0          0          0  SiFive PLIC  10  ttyS0
>> >   3:        591          0          0          0  SiFive PLIC   8  virtio0
>> >   5:       5079      10821       8435      12984  RISC-V INTC   5  riscv-timer
>> > IPI0:      2045       2537        891        870  Rescheduling interrupts
>> > IPI1:         9        269         91        168  Function call interrupts
>> > IPI2:         0          0          0          0  CPU stop interrupts
>> >
>> > The patchset is based up Linux-5.7-rc6 and can be found at riscv_intc_v5
>> > branch of: https://github.com/avpatel/linux.git
>> >
>> > This series is tested on:
>> >  1. QEMU RV64 virt machine using Linux RISC-V S-mode
>> >  2. QEMU RV32 virt machine using Linux RISC-V S-mode
>> >  3. QEMU RV64 virt machine using Linux RISC-V M-mode (i.e. NoMMU)
>> >
>> > Changes since v4:
>> >  - Rebased to Linux-5.7-rc6 and multi-PLIC improvement patches
>> >  - Added separate patch to force select RISCV_INTC for CONFIG_RISCV
>> >  - Fixed the driver for Linux RISC-V NoMMU
>> >
>> > Changes since v3:
>> >  - Rebased to Linux-5.6-rc5 and Atish's PLIC patches
>> >  - Added separate patch to rename and move plic_find_hart_id()
>> >    to arch directory
>> >  - Use riscv_of_parent_hartid() in riscv_intc_init() instead of
>> >    atomic counter
>> >
>> > Changes since v2:
>> >  - Dropped PATCH2 since it was merged long-time back
>> >  - Rebased series from Linux-4.19-rc2 to Linux-5.6-rc2
>> >
>> > Changes since v1:
>> >  - Removed changes related to puggable IPI triggering
>> >  - Separate patch for self-contained IPI handling routine
>> >  - Removed patch for GENERIC_IRQ kconfig options
>> >  - Added patch to remove do_IRQ() function
>> >  - Rebased upon Atish's SMP patches
>> >
>> > Anup Patel (6):
>> >   RISC-V: self-contained IPI handling routine
>> >   RISC-V: Rename and move plic_find_hart_id() to arch directory
>> >   irqchip: RISC-V per-HART local interrupt controller driver
>> >   clocksource/drivers/timer-riscv: Use per-CPU timer interrupt
>> >   RISC-V: Remove do_IRQ() function
>> >   RISC-V: Force select RISCV_INTC for CONFIG_RISCV
>> >
>> >  arch/riscv/Kconfig                     |   2 +
>> >  arch/riscv/include/asm/irq.h           |   5 -
>> >  arch/riscv/include/asm/processor.h     |   1 +
>> >  arch/riscv/include/asm/smp.h           |   3 +
>> >  arch/riscv/kernel/cpu.c                |  16 +++
>> >  arch/riscv/kernel/entry.S              |   4 +-
>> >  arch/riscv/kernel/irq.c                |  33 +-----
>> >  arch/riscv/kernel/smp.c                |  11 +-
>> >  arch/riscv/kernel/traps.c              |   2 -
>> >  drivers/clocksource/timer-riscv.c      |  30 ++++-
>> >  drivers/irqchip/Kconfig                |  13 +++
>> >  drivers/irqchip/Makefile               |   1 +
>> >  drivers/irqchip/irq-riscv-intc.c       | 150 +++++++++++++++++++++++++
>> >  drivers/irqchip/irq-sifive-plic.c      |  52 +++++----
>> >  include/linux/cpuhotplug.h             |   1 +
>> >  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
>> >  16 files changed, 280 insertions(+), 64 deletions(-)
>> >  create mode 100644 drivers/irqchip/irq-riscv-intc.c
>> >  create mode 100644 include/linux/irqchip/irq-riscv-intc.h
>>
>> So I read through this a bit, and while I haven't gone through every line of
>> code I'm somewhat inclined toward taking it.
>>
>> During the original RISC-V port submission we went back and forth between
>> having this first-level interrupt controller in arch/riscv/ vs
>> drivers/irqchip/.  The original deciding factor was that the ISA mandated the
>> interrupt controller, but as that's proving to be less and less the case every
>> day (with the CLIC and M-mode Linux) it certainly seem sane to move all our
>> interrupt controller drivers out of arch/riscv/.
>>
>> This is certainly a step in the right direction, and it handles some of the
>> more glaring issues (iscv_timer_interrupt and lacking IRQs for the CLINT).  I
>> think we should just go ahead and merge it, even though there might be some
>> more refactoring to do when we eventually end up with another interrupt
>> controller.
>>
>> I think it's best if this all goes in through a single tree, as it seems more
>> work than it's worth to split it up.  I'm happy to take it through my tree if
>> that's OK with the irqchip folks?
>
> A small heads up...
>
> Marc has queued a few PLIC improvement patches for Linux-next.
> (https://patchwork.kernel.org/cover/11555051/)
>
> This series (particularly PATCH3) is based upon above mentioned
> PLIC patches.
>
> Apart from above, I don't see any potential merge conflicts.

Thanks.  I hit some merge issues when pulling it into a staging branch, but
nothing seemed interesting.  I think the best bet here is to just pull it in
through the RISC-V tree.

Aside from this, I'm ready to send out my first 5.8 PR.  I'm going to put this
on a staging branch and send it up as a second 5.8 merge window PR once
everything else settles.  IIRC our other irqchip changes have been around for a
while, so they should be going up early in the merge window.

LMK if anyone has issues with that.

>
> Regards,
> Anup

Re: [PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[Palmer Dabbelt]

On Thu, 28 May 2020 20:57:26 PDT (-0700), anup@brainfault.org wrote:
> On Thu, May 28, 2020 at 12:17 AM Palmer Dabbelt <palmer@dabbelt.com> wrote:
>>
>> On Thu, 21 May 2020 06:32:55 PDT (-0700), Anup Patel wrote:
>> > This patchset provides a new RISC-V Local Interrupt Controller Driver
>> > for managing per-CPU local interrupts. The overall approach is inspired
>> > from the way per-CPU local interrupts are handled by Linux ARM64 and
>> > ARM GICv3 driver.
>> >
>> > Few advantages of this new driver over previous one are:
>> > 1. All local interrupts are registered as per-CPU interrupts
>> > 2. The RISC-V timer driver can register timer interrupt handler
>> >    using kernel irq subsystem without relying on arch/riscv to
>> >    explicitly call it's interrupt handler
>> > 3. The KVM RISC-V can use this driver to implement interrupt
>> >    handler for per-HART guest external interrupt defined by
>> >    the RISC-V H-Extension
>> > 4. In future, we can develop drivers for devices with per-HART
>> >    interrupts without changing arch code or this driver (example,
>> >    CLINT timer driver for RISC-V M-mode kernel)
>> >
>> > With this patchset, output of "cat /proc/interrupts" looks as follows:
>> >            CPU0       CPU1       CPU2       CPU3
>> >   2:        379          0          0          0  SiFive PLIC  10  ttyS0
>> >   3:        591          0          0          0  SiFive PLIC   8  virtio0
>> >   5:       5079      10821       8435      12984  RISC-V INTC   5  riscv-timer
>> > IPI0:      2045       2537        891        870  Rescheduling interrupts
>> > IPI1:         9        269         91        168  Function call interrupts
>> > IPI2:         0          0          0          0  CPU stop interrupts
>> >
>> > The patchset is based up Linux-5.7-rc6 and can be found at riscv_intc_v5
>> > branch of: https://github.com/avpatel/linux.git
>> >
>> > This series is tested on:
>> >  1. QEMU RV64 virt machine using Linux RISC-V S-mode
>> >  2. QEMU RV32 virt machine using Linux RISC-V S-mode
>> >  3. QEMU RV64 virt machine using Linux RISC-V M-mode (i.e. NoMMU)
>> >
>> > Changes since v4:
>> >  - Rebased to Linux-5.7-rc6 and multi-PLIC improvement patches
>> >  - Added separate patch to force select RISCV_INTC for CONFIG_RISCV
>> >  - Fixed the driver for Linux RISC-V NoMMU
>> >
>> > Changes since v3:
>> >  - Rebased to Linux-5.6-rc5 and Atish's PLIC patches
>> >  - Added separate patch to rename and move plic_find_hart_id()
>> >    to arch directory
>> >  - Use riscv_of_parent_hartid() in riscv_intc_init() instead of
>> >    atomic counter
>> >
>> > Changes since v2:
>> >  - Dropped PATCH2 since it was merged long-time back
>> >  - Rebased series from Linux-4.19-rc2 to Linux-5.6-rc2
>> >
>> > Changes since v1:
>> >  - Removed changes related to puggable IPI triggering
>> >  - Separate patch for self-contained IPI handling routine
>> >  - Removed patch for GENERIC_IRQ kconfig options
>> >  - Added patch to remove do_IRQ() function
>> >  - Rebased upon Atish's SMP patches
>> >
>> > Anup Patel (6):
>> >   RISC-V: self-contained IPI handling routine
>> >   RISC-V: Rename and move plic_find_hart_id() to arch directory
>> >   irqchip: RISC-V per-HART local interrupt controller driver
>> >   clocksource/drivers/timer-riscv: Use per-CPU timer interrupt
>> >   RISC-V: Remove do_IRQ() function
>> >   RISC-V: Force select RISCV_INTC for CONFIG_RISCV
>> >
>> >  arch/riscv/Kconfig                     |   2 +
>> >  arch/riscv/include/asm/irq.h           |   5 -
>> >  arch/riscv/include/asm/processor.h     |   1 +
>> >  arch/riscv/include/asm/smp.h           |   3 +
>> >  arch/riscv/kernel/cpu.c                |  16 +++
>> >  arch/riscv/kernel/entry.S              |   4 +-
>> >  arch/riscv/kernel/irq.c                |  33 +-----
>> >  arch/riscv/kernel/smp.c                |  11 +-
>> >  arch/riscv/kernel/traps.c              |   2 -
>> >  drivers/clocksource/timer-riscv.c      |  30 ++++-
>> >  drivers/irqchip/Kconfig                |  13 +++
>> >  drivers/irqchip/Makefile               |   1 +
>> >  drivers/irqchip/irq-riscv-intc.c       | 150 +++++++++++++++++++++++++
>> >  drivers/irqchip/irq-sifive-plic.c      |  52 +++++----
>> >  include/linux/cpuhotplug.h             |   1 +
>> >  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
>> >  16 files changed, 280 insertions(+), 64 deletions(-)
>> >  create mode 100644 drivers/irqchip/irq-riscv-intc.c
>> >  create mode 100644 include/linux/irqchip/irq-riscv-intc.h
>>
>> So I read through this a bit, and while I haven't gone through every line of
>> code I'm somewhat inclined toward taking it.
>>
>> During the original RISC-V port submission we went back and forth between
>> having this first-level interrupt controller in arch/riscv/ vs
>> drivers/irqchip/.  The original deciding factor was that the ISA mandated the
>> interrupt controller, but as that's proving to be less and less the case every
>> day (with the CLIC and M-mode Linux) it certainly seem sane to move all our
>> interrupt controller drivers out of arch/riscv/.
>>
>> This is certainly a step in the right direction, and it handles some of the
>> more glaring issues (iscv_timer_interrupt and lacking IRQs for the CLINT).  I
>> think we should just go ahead and merge it, even though there might be some
>> more refactoring to do when we eventually end up with another interrupt
>> controller.
>>
>> I think it's best if this all goes in through a single tree, as it seems more
>> work than it's worth to split it up.  I'm happy to take it through my tree if
>> that's OK with the irqchip folks?
>
> A small heads up...
>
> Marc has queued a few PLIC improvement patches for Linux-next.
> (https://patchwork.kernel.org/cover/11555051/)
>
> This series (particularly PATCH3) is based upon above mentioned
> PLIC patches.
>
> Apart from above, I don't see any potential merge conflicts.

Thanks.  I hit some merge issues when pulling it into a staging branch, but
nothing seemed interesting.  I think the best bet here is to just pull it in
through the RISC-V tree.

Aside from this, I'm ready to send out my first 5.8 PR.  I'm going to put this
on a staging branch and send it up as a second 5.8 merge window PR once
everything else settles.  IIRC our other irqchip changes have been around for a
while, so they should be going up early in the merge window.

LMK if anyone has issues with that.

>
> Regards,
> Anup

RE: [PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[Anup Patel]

> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org <linux-kernel-
> owner@vger.kernel.org> On Behalf Of Palmer Dabbelt
> Sent: 29 May 2020 09:43
> To: anup@brainfault.org
> Cc: Marc Zyngier <maz@kernel.org>; Anup Patel <Anup.Patel@wdc.com>; Paul
> Walmsley <paul.walmsley@sifive.com>; aou@eecs.berkeley.edu;
> daniel.lezcano@linaro.org; tglx@linutronix.de; jason@lakedaemon.net; Atish
> Patra <Atish.Patra@wdc.com>; Alistair Francis <Alistair.Francis@wdc.com>;
> linux-riscv@lists.infradead.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver
> 
> On Thu, 28 May 2020 20:57:26 PDT (-0700), anup@brainfault.org wrote:
> > On Thu, May 28, 2020 at 12:17 AM Palmer Dabbelt <palmer@dabbelt.com>
> wrote:
> >>
> >> On Thu, 21 May 2020 06:32:55 PDT (-0700), Anup Patel wrote:
> >> > This patchset provides a new RISC-V Local Interrupt Controller
> >> > Driver for managing per-CPU local interrupts. The overall approach
> >> > is inspired from the way per-CPU local interrupts are handled by
> >> > Linux ARM64 and ARM GICv3 driver.
> >> >
> >> > Few advantages of this new driver over previous one are:
> >> > 1. All local interrupts are registered as per-CPU interrupts 2. The
> >> > RISC-V timer driver can register timer interrupt handler
> >> >    using kernel irq subsystem without relying on arch/riscv to
> >> >    explicitly call it's interrupt handler 3. The KVM RISC-V can use
> >> > this driver to implement interrupt
> >> >    handler for per-HART guest external interrupt defined by
> >> >    the RISC-V H-Extension
> >> > 4. In future, we can develop drivers for devices with per-HART
> >> >    interrupts without changing arch code or this driver (example,
> >> >    CLINT timer driver for RISC-V M-mode kernel)
> >> >
> >> > With this patchset, output of "cat /proc/interrupts" looks as follows:
> >> >            CPU0       CPU1       CPU2       CPU3
> >> >   2:        379          0          0          0  SiFive PLIC  10  ttyS0
> >> >   3:        591          0          0          0  SiFive PLIC   8  virtio0
> >> >   5:       5079      10821       8435      12984  RISC-V INTC   5  riscv-timer
> >> > IPI0:      2045       2537        891        870  Rescheduling interrupts
> >> > IPI1:         9        269         91        168  Function call interrupts
> >> > IPI2:         0          0          0          0  CPU stop interrupts
> >> >
> >> > The patchset is based up Linux-5.7-rc6 and can be found at
> >> > riscv_intc_v5 branch of: https://github.com/avpatel/linux.git
> >> >
> >> > This series is tested on:
> >> >  1. QEMU RV64 virt machine using Linux RISC-V S-mode  2. QEMU RV32
> >> > virt machine using Linux RISC-V S-mode  3. QEMU RV64 virt machine
> >> > using Linux RISC-V M-mode (i.e. NoMMU)
> >> >
> >> > Changes since v4:
> >> >  - Rebased to Linux-5.7-rc6 and multi-PLIC improvement patches
> >> >  - Added separate patch to force select RISCV_INTC for CONFIG_RISCV
> >> >  - Fixed the driver for Linux RISC-V NoMMU
> >> >
> >> > Changes since v3:
> >> >  - Rebased to Linux-5.6-rc5 and Atish's PLIC patches
> >> >  - Added separate patch to rename and move plic_find_hart_id()
> >> >    to arch directory
> >> >  - Use riscv_of_parent_hartid() in riscv_intc_init() instead of
> >> >    atomic counter
> >> >
> >> > Changes since v2:
> >> >  - Dropped PATCH2 since it was merged long-time back
> >> >  - Rebased series from Linux-4.19-rc2 to Linux-5.6-rc2
> >> >
> >> > Changes since v1:
> >> >  - Removed changes related to puggable IPI triggering
> >> >  - Separate patch for self-contained IPI handling routine
> >> >  - Removed patch for GENERIC_IRQ kconfig options
> >> >  - Added patch to remove do_IRQ() function
> >> >  - Rebased upon Atish's SMP patches
> >> >
> >> > Anup Patel (6):
> >> >   RISC-V: self-contained IPI handling routine
> >> >   RISC-V: Rename and move plic_find_hart_id() to arch directory
> >> >   irqchip: RISC-V per-HART local interrupt controller driver
> >> >   clocksource/drivers/timer-riscv: Use per-CPU timer interrupt
> >> >   RISC-V: Remove do_IRQ() function
> >> >   RISC-V: Force select RISCV_INTC for CONFIG_RISCV
> >> >
> >> >  arch/riscv/Kconfig                     |   2 +
> >> >  arch/riscv/include/asm/irq.h           |   5 -
> >> >  arch/riscv/include/asm/processor.h     |   1 +
> >> >  arch/riscv/include/asm/smp.h           |   3 +
> >> >  arch/riscv/kernel/cpu.c                |  16 +++
> >> >  arch/riscv/kernel/entry.S              |   4 +-
> >> >  arch/riscv/kernel/irq.c                |  33 +-----
> >> >  arch/riscv/kernel/smp.c                |  11 +-
> >> >  arch/riscv/kernel/traps.c              |   2 -
> >> >  drivers/clocksource/timer-riscv.c      |  30 ++++-
> >> >  drivers/irqchip/Kconfig                |  13 +++
> >> >  drivers/irqchip/Makefile               |   1 +
> >> >  drivers/irqchip/irq-riscv-intc.c       | 150 +++++++++++++++++++++++++
> >> >  drivers/irqchip/irq-sifive-plic.c      |  52 +++++----
> >> >  include/linux/cpuhotplug.h             |   1 +
> >> >  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
> >> >  16 files changed, 280 insertions(+), 64 deletions(-)  create mode
> >> > 100644 drivers/irqchip/irq-riscv-intc.c  create mode 100644
> >> > include/linux/irqchip/irq-riscv-intc.h
> >>
> >> So I read through this a bit, and while I haven't gone through every
> >> line of code I'm somewhat inclined toward taking it.
> >>
> >> During the original RISC-V port submission we went back and forth
> >> between having this first-level interrupt controller in arch/riscv/
> >> vs drivers/irqchip/.  The original deciding factor was that the ISA
> >> mandated the interrupt controller, but as that's proving to be less
> >> and less the case every day (with the CLIC and M-mode Linux) it
> >> certainly seem sane to move all our interrupt controller drivers out of
> arch/riscv/.
> >>
> >> This is certainly a step in the right direction, and it handles some
> >> of the more glaring issues (iscv_timer_interrupt and lacking IRQs for
> >> the CLINT).  I think we should just go ahead and merge it, even
> >> though there might be some more refactoring to do when we eventually
> >> end up with another interrupt controller.
> >>
> >> I think it's best if this all goes in through a single tree, as it
> >> seems more work than it's worth to split it up.  I'm happy to take it
> >> through my tree if that's OK with the irqchip folks?
> >
> > A small heads up...
> >
> > Marc has queued a few PLIC improvement patches for Linux-next.
> > (https://patchwork.kernel.org/cover/11555051/)
> >
> > This series (particularly PATCH3) is based upon above mentioned PLIC
> > patches.
> >
> > Apart from above, I don't see any potential merge conflicts.
> 
> Thanks.  I hit some merge issues when pulling it into a staging branch, but
> nothing seemed interesting.  I think the best bet here is to just pull it in through
> the RISC-V tree.
> 
> Aside from this, I'm ready to send out my first 5.8 PR.  I'm going to put this on a
> staging branch and send it up as a second 5.8 merge window PR once
> everything else settles.  IIRC our other irqchip changes have been around for a
> while, so they should be going up early in the merge window.

Sending this as part of second 5.8 merge window PR will be perfect. It will
also take care of dependency on PLIC patches.

Let me know if you want me to rebase and send v6 before your second 5.8 PR.

Thanks,
Anup

RE: [PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[Anup Patel]

> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org <linux-kernel-
> owner@vger.kernel.org> On Behalf Of Palmer Dabbelt
> Sent: 29 May 2020 09:43
> To: anup@brainfault.org
> Cc: Marc Zyngier <maz@kernel.org>; Anup Patel <Anup.Patel@wdc.com>; Paul
> Walmsley <paul.walmsley@sifive.com>; aou@eecs.berkeley.edu;
> daniel.lezcano@linaro.org; tglx@linutronix.de; jason@lakedaemon.net; Atish
> Patra <Atish.Patra@wdc.com>; Alistair Francis <Alistair.Francis@wdc.com>;
> linux-riscv@lists.infradead.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver
> 
> On Thu, 28 May 2020 20:57:26 PDT (-0700), anup@brainfault.org wrote:
> > On Thu, May 28, 2020 at 12:17 AM Palmer Dabbelt <palmer@dabbelt.com>
> wrote:
> >>
> >> On Thu, 21 May 2020 06:32:55 PDT (-0700), Anup Patel wrote:
> >> > This patchset provides a new RISC-V Local Interrupt Controller
> >> > Driver for managing per-CPU local interrupts. The overall approach
> >> > is inspired from the way per-CPU local interrupts are handled by
> >> > Linux ARM64 and ARM GICv3 driver.
> >> >
> >> > Few advantages of this new driver over previous one are:
> >> > 1. All local interrupts are registered as per-CPU interrupts 2. The
> >> > RISC-V timer driver can register timer interrupt handler
> >> >    using kernel irq subsystem without relying on arch/riscv to
> >> >    explicitly call it's interrupt handler 3. The KVM RISC-V can use
> >> > this driver to implement interrupt
> >> >    handler for per-HART guest external interrupt defined by
> >> >    the RISC-V H-Extension
> >> > 4. In future, we can develop drivers for devices with per-HART
> >> >    interrupts without changing arch code or this driver (example,
> >> >    CLINT timer driver for RISC-V M-mode kernel)
> >> >
> >> > With this patchset, output of "cat /proc/interrupts" looks as follows:
> >> >            CPU0       CPU1       CPU2       CPU3
> >> >   2:        379          0          0          0  SiFive PLIC  10  ttyS0
> >> >   3:        591          0          0          0  SiFive PLIC   8  virtio0
> >> >   5:       5079      10821       8435      12984  RISC-V INTC   5  riscv-timer
> >> > IPI0:      2045       2537        891        870  Rescheduling interrupts
> >> > IPI1:         9        269         91        168  Function call interrupts
> >> > IPI2:         0          0          0          0  CPU stop interrupts
> >> >
> >> > The patchset is based up Linux-5.7-rc6 and can be found at
> >> > riscv_intc_v5 branch of: https://github.com/avpatel/linux.git
> >> >
> >> > This series is tested on:
> >> >  1. QEMU RV64 virt machine using Linux RISC-V S-mode  2. QEMU RV32
> >> > virt machine using Linux RISC-V S-mode  3. QEMU RV64 virt machine
> >> > using Linux RISC-V M-mode (i.e. NoMMU)
> >> >
> >> > Changes since v4:
> >> >  - Rebased to Linux-5.7-rc6 and multi-PLIC improvement patches
> >> >  - Added separate patch to force select RISCV_INTC for CONFIG_RISCV
> >> >  - Fixed the driver for Linux RISC-V NoMMU
> >> >
> >> > Changes since v3:
> >> >  - Rebased to Linux-5.6-rc5 and Atish's PLIC patches
> >> >  - Added separate patch to rename and move plic_find_hart_id()
> >> >    to arch directory
> >> >  - Use riscv_of_parent_hartid() in riscv_intc_init() instead of
> >> >    atomic counter
> >> >
> >> > Changes since v2:
> >> >  - Dropped PATCH2 since it was merged long-time back
> >> >  - Rebased series from Linux-4.19-rc2 to Linux-5.6-rc2
> >> >
> >> > Changes since v1:
> >> >  - Removed changes related to puggable IPI triggering
> >> >  - Separate patch for self-contained IPI handling routine
> >> >  - Removed patch for GENERIC_IRQ kconfig options
> >> >  - Added patch to remove do_IRQ() function
> >> >  - Rebased upon Atish's SMP patches
> >> >
> >> > Anup Patel (6):
> >> >   RISC-V: self-contained IPI handling routine
> >> >   RISC-V: Rename and move plic_find_hart_id() to arch directory
> >> >   irqchip: RISC-V per-HART local interrupt controller driver
> >> >   clocksource/drivers/timer-riscv: Use per-CPU timer interrupt
> >> >   RISC-V: Remove do_IRQ() function
> >> >   RISC-V: Force select RISCV_INTC for CONFIG_RISCV
> >> >
> >> >  arch/riscv/Kconfig                     |   2 +
> >> >  arch/riscv/include/asm/irq.h           |   5 -
> >> >  arch/riscv/include/asm/processor.h     |   1 +
> >> >  arch/riscv/include/asm/smp.h           |   3 +
> >> >  arch/riscv/kernel/cpu.c                |  16 +++
> >> >  arch/riscv/kernel/entry.S              |   4 +-
> >> >  arch/riscv/kernel/irq.c                |  33 +-----
> >> >  arch/riscv/kernel/smp.c                |  11 +-
> >> >  arch/riscv/kernel/traps.c              |   2 -
> >> >  drivers/clocksource/timer-riscv.c      |  30 ++++-
> >> >  drivers/irqchip/Kconfig                |  13 +++
> >> >  drivers/irqchip/Makefile               |   1 +
> >> >  drivers/irqchip/irq-riscv-intc.c       | 150 +++++++++++++++++++++++++
> >> >  drivers/irqchip/irq-sifive-plic.c      |  52 +++++----
> >> >  include/linux/cpuhotplug.h             |   1 +
> >> >  include/linux/irqchip/irq-riscv-intc.h |  20 ++++
> >> >  16 files changed, 280 insertions(+), 64 deletions(-)  create mode
> >> > 100644 drivers/irqchip/irq-riscv-intc.c  create mode 100644
> >> > include/linux/irqchip/irq-riscv-intc.h
> >>
> >> So I read through this a bit, and while I haven't gone through every
> >> line of code I'm somewhat inclined toward taking it.
> >>
> >> During the original RISC-V port submission we went back and forth
> >> between having this first-level interrupt controller in arch/riscv/
> >> vs drivers/irqchip/.  The original deciding factor was that the ISA
> >> mandated the interrupt controller, but as that's proving to be less
> >> and less the case every day (with the CLIC and M-mode Linux) it
> >> certainly seem sane to move all our interrupt controller drivers out of
> arch/riscv/.
> >>
> >> This is certainly a step in the right direction, and it handles some
> >> of the more glaring issues (iscv_timer_interrupt and lacking IRQs for
> >> the CLINT).  I think we should just go ahead and merge it, even
> >> though there might be some more refactoring to do when we eventually
> >> end up with another interrupt controller.
> >>
> >> I think it's best if this all goes in through a single tree, as it
> >> seems more work than it's worth to split it up.  I'm happy to take it
> >> through my tree if that's OK with the irqchip folks?
> >
> > A small heads up...
> >
> > Marc has queued a few PLIC improvement patches for Linux-next.
> > (https://patchwork.kernel.org/cover/11555051/)
> >
> > This series (particularly PATCH3) is based upon above mentioned PLIC
> > patches.
> >
> > Apart from above, I don't see any potential merge conflicts.
> 
> Thanks.  I hit some merge issues when pulling it into a staging branch, but
> nothing seemed interesting.  I think the best bet here is to just pull it in through
> the RISC-V tree.
> 
> Aside from this, I'm ready to send out my first 5.8 PR.  I'm going to put this on a
> staging branch and send it up as a second 5.8 merge window PR once
> everything else settles.  IIRC our other irqchip changes have been around for a
> while, so they should be going up early in the merge window.

Sending this as part of second 5.8 merge window PR will be perfect. It will
also take care of dependency on PLIC patches.

Let me know if you want me to rebase and send v6 before your second 5.8 PR.

Thanks,
Anup

Re: [PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[Marc Zyngier]

On 2020-05-27 19:47, Palmer Dabbelt wrote:

[...]

> I think it's best if this all goes in through a single tree, as it 
> seems more
> work than it's worth to split it up.  I'm happy to take it through my 
> tree if
> that's OK with the irqchip folks?

This still needs some fixing on patch 3 (the exported symbol is pretty 
awful),
but this should definitely go via the RISC-V tree.

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH v5 0/6] New RISC-V Local Interrupt Controller Driver

[Marc Zyngier]

On 2020-05-27 19:47, Palmer Dabbelt wrote:

[...]

> I think it's best if this all goes in through a single tree, as it 
> seems more
> work than it's worth to split it up.  I'm happy to take it through my 
> tree if
> that's OK with the irqchip folks?

This still needs some fixing on patch 3 (the exported symbol is pretty 
awful),
but this should definitely go via the RISC-V tree.

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...