From: Marc Zyngier <maz@kernel.org> To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Will Deacon <will@kernel.org>, Catalin Marinas <catalin.marinas@arm.com>, Russell King <linux@arm.linux.org.uk>, Thomas Gleixner <tglx@linutronix.de>, Jason Cooper <jason@lakedaemon.net>, Sumit Garg <sumit.garg@linaro.org>, Valentin Schneider <Valentin.Schneider@arm.com>, Florian Fainelli <f.fainelli@gmail.com>, Gregory Clement <gregory.clement@bootlin.com>, Andrew Lunn <andrew@lunn.ch>, kernel-team@android.com Subject: [PATCH v2 03/17] arm64: Allow IPIs to be handled as normal interrupts Date: Wed, 24 Jun 2020 20:57:57 +0100 [thread overview] Message-ID: <20200624195811.435857-4-maz@kernel.org> (raw) In-Reply-To: <20200624195811.435857-1-maz@kernel.org> In order to deal with IPIs as normal interrupts, let's add a new way to register them with the architecture code. set_smp_ipi_range() takes a range of interrupts, and allows the arch code to request them as if the were normal interrupts. A standard handler is then called by the core IRQ code to deal with the IPI. This means that we don't need to call irq_enter/irq_exit, and that we don't need to deal with set_irq_regs either. So let's move the dispatcher into its own function, and leave handle_IPI() as a compatibility function. On the sending side, let's make use of ipi_send_mask, which already exists for this purpose. One of the major difference is that we end up, in some cases (such as when performing IRQ time accounting on the scheduler IPI), end up with nested irq_enter()/irq_exit() pairs. Other than the (relatively small) overhead, there should be no consequences to it (these pairs are designed to nest correctly, and the accounting shouldn't be off). Signed-off-by: Marc Zyngier <maz@kernel.org> --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/smp.h | 5 ++ arch/arm64/kernel/smp.c | 91 +++++++++++++++++++++++++++++++----- 3 files changed, 85 insertions(+), 12 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a4a094bedcb2..fe88d28bb985 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -106,6 +106,7 @@ config ARM64 select GENERIC_CPU_VULNERABILITIES select GENERIC_EARLY_IOREMAP select GENERIC_IDLE_POLL_SETUP + select GENERIC_IRQ_IPI select GENERIC_IRQ_MULTI_HANDLER select GENERIC_IRQ_PROBE select GENERIC_IRQ_SHOW diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h index ea268d88b6f7..4537a4620e3d 100644 --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -74,6 +74,11 @@ extern void set_smp_cross_call(void (*)(const struct cpumask *, unsigned int)); extern void (*__smp_cross_call)(const struct cpumask *, unsigned int); +/* + * Register IPI interrupts with the arch SMP code + */ +extern void set_smp_ipi_range(int ipi_base, int nr_ipi); + /* * Called from the secondary holding pen, this is the secondary CPU entry point. */ diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index e43a8ff19f0f..c08950cc09cc 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -76,6 +76,13 @@ enum ipi_msg_type { IPI_WAKEUP }; +static int ipi_irq_base; +static int nr_ipi = NR_IPI; +static struct irq_desc *ipi_desc[NR_IPI]; + +static void ipi_setup(int cpu); +static void ipi_teardown(int cpu); + #ifdef CONFIG_HOTPLUG_CPU static int op_cpu_kill(unsigned int cpu); #else @@ -238,6 +245,8 @@ asmlinkage notrace void secondary_start_kernel(void) */ notify_cpu_starting(cpu); + ipi_setup(cpu); + store_cpu_topology(cpu); numa_add_cpu(cpu); @@ -303,6 +312,7 @@ int __cpu_disable(void) * and we must not schedule until we're ready to give up the cpu. */ set_cpu_online(cpu, false); + ipi_teardown(cpu); /* * OK - migrate IRQs away from this CPU @@ -891,10 +901,9 @@ static void ipi_cpu_crash_stop(unsigned int cpu, struct pt_regs *regs) /* * Main handler for inter-processor interrupts */ -void handle_IPI(int ipinr, struct pt_regs *regs) +static void do_handle_IPI(int ipinr) { unsigned int cpu = smp_processor_id(); - struct pt_regs *old_regs = set_irq_regs(regs); if ((unsigned)ipinr < NR_IPI) { trace_ipi_entry_rcuidle(ipi_types[ipinr]); @@ -907,21 +916,16 @@ void handle_IPI(int ipinr, struct pt_regs *regs) break; case IPI_CALL_FUNC: - irq_enter(); generic_smp_call_function_interrupt(); - irq_exit(); break; case IPI_CPU_STOP: - irq_enter(); local_cpu_stop(); - irq_exit(); break; case IPI_CPU_CRASH_STOP: if (IS_ENABLED(CONFIG_KEXEC_CORE)) { - irq_enter(); - ipi_cpu_crash_stop(cpu, regs); + ipi_cpu_crash_stop(cpu, get_irq_regs()); unreachable(); } @@ -929,17 +933,13 @@ void handle_IPI(int ipinr, struct pt_regs *regs) #ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST case IPI_TIMER: - irq_enter(); tick_receive_broadcast(); - irq_exit(); break; #endif #ifdef CONFIG_IRQ_WORK case IPI_IRQ_WORK: - irq_enter(); irq_work_run(); - irq_exit(); break; #endif @@ -958,9 +958,76 @@ void handle_IPI(int ipinr, struct pt_regs *regs) if ((unsigned)ipinr < NR_IPI) trace_ipi_exit_rcuidle(ipi_types[ipinr]); +} + +/* Legacy version, should go away once all irqchips have been converted */ +void handle_IPI(int ipinr, struct pt_regs *regs) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + + irq_enter(); + do_handle_IPI(ipinr); + irq_exit(); + set_irq_regs(old_regs); } +static irqreturn_t ipi_handler(int irq, void *data) +{ + do_handle_IPI(irq - ipi_irq_base); + return IRQ_HANDLED; +} + +static void ipi_send(const struct cpumask *target, unsigned int ipi) +{ + __ipi_send_mask(ipi_desc[ipi], target); +} + +static void ipi_setup(int cpu) +{ + if (ipi_irq_base) { + int i; + + for (i = 0; i < nr_ipi; i++) + enable_percpu_irq(ipi_irq_base + i, 0); + } +} + +static void ipi_teardown(int cpu) +{ + if (ipi_irq_base) { + int i; + + for (i = 0; i < nr_ipi; i++) + disable_percpu_irq(ipi_irq_base + i); + } +} + +void __init set_smp_ipi_range(int ipi_base, int n) +{ + int i; + + WARN_ON(n < NR_IPI); + nr_ipi = min(n, NR_IPI); + + for (i = 0; i < nr_ipi; i++) { + int err; + + err = request_percpu_irq(ipi_base + i, ipi_handler, + "IPI", &irq_stat); + WARN_ON(err); + + ipi_desc[i] = irq_to_desc(ipi_base + i); + irq_set_status_flags(ipi_base + i, IRQ_HIDDEN); + } + + ipi_irq_base = ipi_base; + __smp_cross_call = ipi_send; + + /* Setup the boot CPU immediately */ + ipi_setup(smp_processor_id()); +} + void smp_send_reschedule(int cpu) { smp_cross_call(cpumask_of(cpu), IPI_RESCHEDULE); -- 2.27.0
WARNING: multiple messages have this Message-ID (diff)
From: Marc Zyngier <maz@kernel.org> To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Sumit Garg <sumit.garg@linaro.org>, Florian Fainelli <f.fainelli@gmail.com>, Russell King <linux@arm.linux.org.uk>, Jason Cooper <jason@lakedaemon.net>, kernel-team@android.com, Andrew Lunn <andrew@lunn.ch>, Catalin Marinas <catalin.marinas@arm.com>, Gregory Clement <gregory.clement@bootlin.com>, Thomas Gleixner <tglx@linutronix.de>, Will Deacon <will@kernel.org>, Valentin Schneider <Valentin.Schneider@arm.com> Subject: [PATCH v2 03/17] arm64: Allow IPIs to be handled as normal interrupts Date: Wed, 24 Jun 2020 20:57:57 +0100 [thread overview] Message-ID: <20200624195811.435857-4-maz@kernel.org> (raw) In-Reply-To: <20200624195811.435857-1-maz@kernel.org> In order to deal with IPIs as normal interrupts, let's add a new way to register them with the architecture code. set_smp_ipi_range() takes a range of interrupts, and allows the arch code to request them as if the were normal interrupts. A standard handler is then called by the core IRQ code to deal with the IPI. This means that we don't need to call irq_enter/irq_exit, and that we don't need to deal with set_irq_regs either. So let's move the dispatcher into its own function, and leave handle_IPI() as a compatibility function. On the sending side, let's make use of ipi_send_mask, which already exists for this purpose. One of the major difference is that we end up, in some cases (such as when performing IRQ time accounting on the scheduler IPI), end up with nested irq_enter()/irq_exit() pairs. Other than the (relatively small) overhead, there should be no consequences to it (these pairs are designed to nest correctly, and the accounting shouldn't be off). Signed-off-by: Marc Zyngier <maz@kernel.org> --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/smp.h | 5 ++ arch/arm64/kernel/smp.c | 91 +++++++++++++++++++++++++++++++----- 3 files changed, 85 insertions(+), 12 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a4a094bedcb2..fe88d28bb985 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -106,6 +106,7 @@ config ARM64 select GENERIC_CPU_VULNERABILITIES select GENERIC_EARLY_IOREMAP select GENERIC_IDLE_POLL_SETUP + select GENERIC_IRQ_IPI select GENERIC_IRQ_MULTI_HANDLER select GENERIC_IRQ_PROBE select GENERIC_IRQ_SHOW diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h index ea268d88b6f7..4537a4620e3d 100644 --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -74,6 +74,11 @@ extern void set_smp_cross_call(void (*)(const struct cpumask *, unsigned int)); extern void (*__smp_cross_call)(const struct cpumask *, unsigned int); +/* + * Register IPI interrupts with the arch SMP code + */ +extern void set_smp_ipi_range(int ipi_base, int nr_ipi); + /* * Called from the secondary holding pen, this is the secondary CPU entry point. */ diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index e43a8ff19f0f..c08950cc09cc 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -76,6 +76,13 @@ enum ipi_msg_type { IPI_WAKEUP }; +static int ipi_irq_base; +static int nr_ipi = NR_IPI; +static struct irq_desc *ipi_desc[NR_IPI]; + +static void ipi_setup(int cpu); +static void ipi_teardown(int cpu); + #ifdef CONFIG_HOTPLUG_CPU static int op_cpu_kill(unsigned int cpu); #else @@ -238,6 +245,8 @@ asmlinkage notrace void secondary_start_kernel(void) */ notify_cpu_starting(cpu); + ipi_setup(cpu); + store_cpu_topology(cpu); numa_add_cpu(cpu); @@ -303,6 +312,7 @@ int __cpu_disable(void) * and we must not schedule until we're ready to give up the cpu. */ set_cpu_online(cpu, false); + ipi_teardown(cpu); /* * OK - migrate IRQs away from this CPU @@ -891,10 +901,9 @@ static void ipi_cpu_crash_stop(unsigned int cpu, struct pt_regs *regs) /* * Main handler for inter-processor interrupts */ -void handle_IPI(int ipinr, struct pt_regs *regs) +static void do_handle_IPI(int ipinr) { unsigned int cpu = smp_processor_id(); - struct pt_regs *old_regs = set_irq_regs(regs); if ((unsigned)ipinr < NR_IPI) { trace_ipi_entry_rcuidle(ipi_types[ipinr]); @@ -907,21 +916,16 @@ void handle_IPI(int ipinr, struct pt_regs *regs) break; case IPI_CALL_FUNC: - irq_enter(); generic_smp_call_function_interrupt(); - irq_exit(); break; case IPI_CPU_STOP: - irq_enter(); local_cpu_stop(); - irq_exit(); break; case IPI_CPU_CRASH_STOP: if (IS_ENABLED(CONFIG_KEXEC_CORE)) { - irq_enter(); - ipi_cpu_crash_stop(cpu, regs); + ipi_cpu_crash_stop(cpu, get_irq_regs()); unreachable(); } @@ -929,17 +933,13 @@ void handle_IPI(int ipinr, struct pt_regs *regs) #ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST case IPI_TIMER: - irq_enter(); tick_receive_broadcast(); - irq_exit(); break; #endif #ifdef CONFIG_IRQ_WORK case IPI_IRQ_WORK: - irq_enter(); irq_work_run(); - irq_exit(); break; #endif @@ -958,9 +958,76 @@ void handle_IPI(int ipinr, struct pt_regs *regs) if ((unsigned)ipinr < NR_IPI) trace_ipi_exit_rcuidle(ipi_types[ipinr]); +} + +/* Legacy version, should go away once all irqchips have been converted */ +void handle_IPI(int ipinr, struct pt_regs *regs) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + + irq_enter(); + do_handle_IPI(ipinr); + irq_exit(); + set_irq_regs(old_regs); } +static irqreturn_t ipi_handler(int irq, void *data) +{ + do_handle_IPI(irq - ipi_irq_base); + return IRQ_HANDLED; +} + +static void ipi_send(const struct cpumask *target, unsigned int ipi) +{ + __ipi_send_mask(ipi_desc[ipi], target); +} + +static void ipi_setup(int cpu) +{ + if (ipi_irq_base) { + int i; + + for (i = 0; i < nr_ipi; i++) + enable_percpu_irq(ipi_irq_base + i, 0); + } +} + +static void ipi_teardown(int cpu) +{ + if (ipi_irq_base) { + int i; + + for (i = 0; i < nr_ipi; i++) + disable_percpu_irq(ipi_irq_base + i); + } +} + +void __init set_smp_ipi_range(int ipi_base, int n) +{ + int i; + + WARN_ON(n < NR_IPI); + nr_ipi = min(n, NR_IPI); + + for (i = 0; i < nr_ipi; i++) { + int err; + + err = request_percpu_irq(ipi_base + i, ipi_handler, + "IPI", &irq_stat); + WARN_ON(err); + + ipi_desc[i] = irq_to_desc(ipi_base + i); + irq_set_status_flags(ipi_base + i, IRQ_HIDDEN); + } + + ipi_irq_base = ipi_base; + __smp_cross_call = ipi_send; + + /* Setup the boot CPU immediately */ + ipi_setup(smp_processor_id()); +} + void smp_send_reschedule(int cpu) { smp_cross_call(cpumask_of(cpu), IPI_RESCHEDULE); -- 2.27.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2020-06-24 19:58 UTC|newest] Thread overview: 99+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-06-24 19:57 [PATCH v2 00/17] arm/arm64: Turning IPIs into normal interrupts Marc Zyngier 2020-06-24 19:57 ` Marc Zyngier 2020-06-24 19:57 ` [PATCH v2 01/17] genirq: Add fasteoi IPI flow Marc Zyngier 2020-06-24 19:57 ` Marc Zyngier 2020-06-24 19:57 ` [PATCH v2 02/17] genirq: Allow interrupts to be excluded from /proc/interrupts Marc Zyngier 2020-06-24 19:57 ` Marc Zyngier 2020-06-24 19:57 ` Marc Zyngier [this message] 2020-06-24 19:57 ` [PATCH v2 03/17] arm64: Allow IPIs to be handled as normal interrupts Marc Zyngier 2020-06-25 18:25 ` Valentin Schneider 2020-06-25 18:25 ` Valentin Schneider 2020-07-10 19:58 ` Valentin Schneider 2020-07-10 19:58 ` Valentin Schneider 2020-06-24 19:57 ` [PATCH v2 04/17] ARM: " Marc Zyngier 2020-06-24 19:57 ` Marc Zyngier 2020-06-25 18:25 ` Valentin Schneider 2020-06-25 18:25 ` Valentin Schneider 2020-06-29 9:37 ` Marc Zyngier 2020-06-29 9:37 ` Marc Zyngier 2020-06-24 19:57 ` [PATCH v2 05/17] irqchip/gic-v3: Describe the SGI range Marc Zyngier 2020-06-24 19:57 ` Marc Zyngier 2020-06-24 19:58 ` [PATCH v2 06/17] irqchip/gic-v3: Configure SGIs as standard interrupts Marc Zyngier 2020-06-24 19:58 ` Marc Zyngier 2020-06-25 18:25 ` Valentin Schneider 2020-06-25 18:25 ` Valentin Schneider 2020-06-30 10:15 ` Marc Zyngier 2020-06-30 10:15 ` Marc Zyngier 2020-07-02 13:23 ` Valentin Schneider 2020-07-02 13:23 ` Valentin Schneider 2020-07-02 13:48 ` Marc Zyngier 2020-07-02 13:48 ` Marc Zyngier 2020-07-02 14:24 ` Valentin Schneider 2020-07-02 14:24 ` Valentin Schneider 2020-06-24 19:58 ` [PATCH v2 07/17] irqchip/gic: Atomically update affinity Marc Zyngier 2020-06-24 19:58 ` Marc Zyngier 2020-07-01 19:33 ` Sasha Levin 2020-07-01 19:33 ` Sasha Levin 2020-07-10 14:02 ` Sasha Levin 2020-07-10 14:02 ` Sasha Levin 2021-09-09 15:22 ` Geert Uytterhoeven 2021-09-09 15:22 ` Geert Uytterhoeven 2021-09-09 15:37 ` Russell King (Oracle) 2021-09-09 15:37 ` Russell King (Oracle) 2021-09-10 10:22 ` Marc Zyngier 2021-09-10 10:22 ` Marc Zyngier 2021-09-10 13:19 ` Geert Uytterhoeven 2021-09-10 13:19 ` Geert Uytterhoeven 2021-09-11 2:49 ` Magnus Damm 2021-09-11 2:49 ` Magnus Damm 2021-09-11 19:32 ` Marc Zyngier 2021-09-11 19:32 ` Marc Zyngier 2021-09-12 5:40 ` Magnus Damm 2021-09-12 5:40 ` Magnus Damm 2021-09-13 8:05 ` Geert Uytterhoeven 2021-09-13 8:05 ` Geert Uytterhoeven 2021-09-15 3:28 ` Magnus Damm 2021-09-15 3:28 ` Magnus Damm 2021-09-22 13:53 ` [irqchip: irq/irqchip-fixes] irqchip/gic: Work around broken Renesas integration irqchip-bot for Marc Zyngier 2020-06-24 19:58 ` [PATCH v2 08/17] irqchip/gic: Refactor SMP configuration Marc Zyngier 2020-06-24 19:58 ` Marc Zyngier 2020-06-24 19:58 ` [PATCH v2 09/17] irqchip/gic: Configure SGIs as standard interrupts Marc Zyngier 2020-06-24 19:58 ` Marc Zyngier 2020-06-24 19:58 ` [PATCH v2 10/17] irqchip/gic-common: Don't enable SGIs by default Marc Zyngier 2020-06-24 19:58 ` Marc Zyngier 2020-06-24 19:58 ` [PATCH v2 11/17] irqchip/bcm2836: Configure mailbox interrupts as standard interrupts Marc Zyngier 2020-06-24 19:58 ` Marc Zyngier 2020-06-24 19:58 ` [PATCH v2 12/17] irqchip/hip04: Configure IPIs " Marc Zyngier 2020-06-24 19:58 ` Marc Zyngier 2020-06-24 19:58 ` [PATCH v2 13/17] irqchip/armada-370-xp: " Marc Zyngier 2020-06-24 19:58 ` Marc Zyngier 2020-06-24 19:58 ` [PATCH v2 14/17] arm64: Kill __smp_cross_call and co Marc Zyngier 2020-06-24 19:58 ` Marc Zyngier 2020-06-25 18:25 ` Valentin Schneider 2020-06-25 18:25 ` Valentin Schneider 2020-07-02 13:37 ` Marc Zyngier 2020-07-02 13:37 ` Marc Zyngier 2020-06-24 19:58 ` [PATCH v2 15/17] arm64: Remove custom IRQ stat accounting Marc Zyngier 2020-06-24 19:58 ` Marc Zyngier 2020-06-25 18:26 ` Valentin Schneider 2020-06-25 18:26 ` Valentin Schneider 2020-06-26 11:58 ` Marc Zyngier 2020-06-26 11:58 ` Marc Zyngier 2020-06-26 23:15 ` Valentin Schneider 2020-06-26 23:15 ` Valentin Schneider 2020-06-27 11:42 ` Marc Zyngier 2020-06-27 11:42 ` Marc Zyngier 2020-07-10 19:58 ` Valentin Schneider 2020-07-10 19:58 ` Valentin Schneider 2020-06-24 19:58 ` [PATCH v2 16/17] ARM: Kill __smp_cross_call and co Marc Zyngier 2020-06-24 19:58 ` Marc Zyngier 2020-06-24 19:58 ` [PATCH v2 17/17] ARM: Remove custom IRQ stat accounting Marc Zyngier 2020-06-24 19:58 ` Marc Zyngier 2020-06-25 18:24 ` [PATCH v2 00/17] arm/arm64: Turning IPIs into normal interrupts Valentin Schneider 2020-06-25 18:24 ` Valentin Schneider 2020-07-10 19:58 ` Valentin Schneider 2020-07-10 19:58 ` Valentin Schneider 2020-08-11 13:15 ` Sumit Garg 2020-08-11 13:15 ` Sumit Garg 2020-08-11 13:58 ` Marc Zyngier 2020-08-11 13:58 ` Marc Zyngier
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200624195811.435857-4-maz@kernel.org \ --to=maz@kernel.org \ --cc=Valentin.Schneider@arm.com \ --cc=andrew@lunn.ch \ --cc=catalin.marinas@arm.com \ --cc=f.fainelli@gmail.com \ --cc=gregory.clement@bootlin.com \ --cc=jason@lakedaemon.net \ --cc=kernel-team@android.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux@arm.linux.org.uk \ --cc=sumit.garg@linaro.org \ --cc=tglx@linutronix.de \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.