From: anup@brainfault.org (Anup Patel) To: linux-riscv@lists.infradead.org Subject: [PATCH 4/4] irqchip: sifive-plic: Implement irq_set_affinity() for SMP host Date: Mon, 22 Oct 2018 17:15:17 +0530 [thread overview] Message-ID: <20181022114517.22748-5-anup@brainfault.org> (raw) In-Reply-To: <20181022114517.22748-1-anup@brainfault.org> Currently on SMP host, all CPUs take external interrupts routed via PLIC. All CPUs will try to claim a given external interrupt but only one of them will succeed while other CPUs would simply resume whatever they were doing before. This means if we have N CPUs then for every external interrupt N-1 CPUs will always fail to claim it and waste their CPU time. Instead of above, external interrupts should be taken by only one CPU and we should have provision to explicity specify IRQ affinity from kernel-space or user-space. This patch provides irq_set_affinity() implementation for PLIC driver. It also updates irq_enable() such that PLIC interrupts are only enabled for one of CPUs specified in IRQ affinity mask. With this patch in-place, we can change IRQ affinity at any-time from user-space using procfs. Example: / # cat /proc/interrupts CPU0 CPU1 CPU2 CPU3 8: 44 0 0 0 SiFive PLIC 8 virtio0 10: 48 0 0 0 SiFive PLIC 10 ttyS0 IPI0: 55 663 58 363 Rescheduling interrupts IPI1: 0 1 3 16 Function call interrupts / # / # / # echo 4 > /proc/irq/10/smp_affinity / # / # cat /proc/interrupts CPU0 CPU1 CPU2 CPU3 8: 45 0 0 0 SiFive PLIC 8 virtio0 10: 160 0 17 0 SiFive PLIC 10 ttyS0 IPI0: 68 693 77 410 Rescheduling interrupts IPI1: 0 2 3 16 Function call interrupts Signed-off-by: Anup Patel <anup@brainfault.org> --- drivers/irqchip/irq-sifive-plic.c | 35 +++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c index eb9e8aee1a1a..5a53ffb3c413 100644 --- a/drivers/irqchip/irq-sifive-plic.c +++ b/drivers/irqchip/irq-sifive-plic.c @@ -97,14 +97,42 @@ static void plic_irq_toggle(const struct cpumask *mask, int hwirq, int enable) static void plic_irq_enable(struct irq_data *d) { - plic_irq_toggle(irq_data_get_affinity_mask(d), d->hwirq, 1); + unsigned int cpu = cpumask_any_and(irq_data_get_affinity_mask(d), + cpu_online_mask); + WARN_ON(cpu >= nr_cpu_ids); + plic_irq_toggle(cpumask_of(cpu), d->hwirq, 1); } static void plic_irq_disable(struct irq_data *d) { - plic_irq_toggle(irq_data_get_affinity_mask(d), d->hwirq, 0); + plic_irq_toggle(cpu_possible_mask, d->hwirq, 0); } +#ifdef CONFIG_SMP +static int plic_set_affinity(struct irq_data *d, const struct cpumask *mask_val, + bool force) +{ + unsigned int cpu; + + if (!force) + cpu = cpumask_any_and(mask_val, cpu_online_mask); + else + cpu = cpumask_first(mask_val); + + if (cpu >= nr_cpu_ids) + return -EINVAL; + + if (!irqd_irq_disabled(d)) { + plic_irq_toggle(cpu_possible_mask, d->hwirq, 0); + plic_irq_toggle(cpumask_of(cpu), d->hwirq, 1); + } + + irq_data_update_effective_affinity(d, cpumask_of(cpu)); + + return IRQ_SET_MASK_OK_DONE; +} +#endif + static struct irq_chip plic_chip = { .name = "SiFive PLIC", /* @@ -113,6 +141,9 @@ static struct irq_chip plic_chip = { */ .irq_enable = plic_irq_enable, .irq_disable = plic_irq_disable, +#ifdef CONFIG_SMP + .irq_set_affinity = plic_set_affinity, +#endif }; static int plic_irqdomain_map(struct irq_domain *d, unsigned int irq, -- 2.17.1
WARNING: multiple messages have this Message-ID (diff)
From: Anup Patel <anup@brainfault.org> To: Palmer Dabbelt <palmer@sifive.com>, Albert Ou <aou@eecs.berkeley.edu>, Daniel Lezcano <daniel.lezcano@linaro.org>, Thomas Gleixner <tglx@linutronix.de>, Jason Cooper <jason@lakedaemon.net>, Marc Zyngier <marc.zyngier@arm.com> Cc: Christoph Hellwig <hch@infradead.org>, Atish Patra <atish.patra@wdc.com>, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Anup Patel <anup@brainfault.org> Subject: [PATCH 4/4] irqchip: sifive-plic: Implement irq_set_affinity() for SMP host Date: Mon, 22 Oct 2018 17:15:17 +0530 [thread overview] Message-ID: <20181022114517.22748-5-anup@brainfault.org> (raw) Message-ID: <20181022114517.IKX7x59GARHya1sCESZyu_pN7uvjL4pqBn6WrQtv8qc@z> (raw) In-Reply-To: <20181022114517.22748-1-anup@brainfault.org> Currently on SMP host, all CPUs take external interrupts routed via PLIC. All CPUs will try to claim a given external interrupt but only one of them will succeed while other CPUs would simply resume whatever they were doing before. This means if we have N CPUs then for every external interrupt N-1 CPUs will always fail to claim it and waste their CPU time. Instead of above, external interrupts should be taken by only one CPU and we should have provision to explicity specify IRQ affinity from kernel-space or user-space. This patch provides irq_set_affinity() implementation for PLIC driver. It also updates irq_enable() such that PLIC interrupts are only enabled for one of CPUs specified in IRQ affinity mask. With this patch in-place, we can change IRQ affinity at any-time from user-space using procfs. Example: / # cat /proc/interrupts CPU0 CPU1 CPU2 CPU3 8: 44 0 0 0 SiFive PLIC 8 virtio0 10: 48 0 0 0 SiFive PLIC 10 ttyS0 IPI0: 55 663 58 363 Rescheduling interrupts IPI1: 0 1 3 16 Function call interrupts / # / # / # echo 4 > /proc/irq/10/smp_affinity / # / # cat /proc/interrupts CPU0 CPU1 CPU2 CPU3 8: 45 0 0 0 SiFive PLIC 8 virtio0 10: 160 0 17 0 SiFive PLIC 10 ttyS0 IPI0: 68 693 77 410 Rescheduling interrupts IPI1: 0 2 3 16 Function call interrupts Signed-off-by: Anup Patel <anup@brainfault.org> --- drivers/irqchip/irq-sifive-plic.c | 35 +++++++++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 2 deletions(-) diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c index eb9e8aee1a1a..5a53ffb3c413 100644 --- a/drivers/irqchip/irq-sifive-plic.c +++ b/drivers/irqchip/irq-sifive-plic.c @@ -97,14 +97,42 @@ static void plic_irq_toggle(const struct cpumask *mask, int hwirq, int enable) static void plic_irq_enable(struct irq_data *d) { - plic_irq_toggle(irq_data_get_affinity_mask(d), d->hwirq, 1); + unsigned int cpu = cpumask_any_and(irq_data_get_affinity_mask(d), + cpu_online_mask); + WARN_ON(cpu >= nr_cpu_ids); + plic_irq_toggle(cpumask_of(cpu), d->hwirq, 1); } static void plic_irq_disable(struct irq_data *d) { - plic_irq_toggle(irq_data_get_affinity_mask(d), d->hwirq, 0); + plic_irq_toggle(cpu_possible_mask, d->hwirq, 0); } +#ifdef CONFIG_SMP +static int plic_set_affinity(struct irq_data *d, const struct cpumask *mask_val, + bool force) +{ + unsigned int cpu; + + if (!force) + cpu = cpumask_any_and(mask_val, cpu_online_mask); + else + cpu = cpumask_first(mask_val); + + if (cpu >= nr_cpu_ids) + return -EINVAL; + + if (!irqd_irq_disabled(d)) { + plic_irq_toggle(cpu_possible_mask, d->hwirq, 0); + plic_irq_toggle(cpumask_of(cpu), d->hwirq, 1); + } + + irq_data_update_effective_affinity(d, cpumask_of(cpu)); + + return IRQ_SET_MASK_OK_DONE; +} +#endif + static struct irq_chip plic_chip = { .name = "SiFive PLIC", /* @@ -113,6 +141,9 @@ static struct irq_chip plic_chip = { */ .irq_enable = plic_irq_enable, .irq_disable = plic_irq_disable, +#ifdef CONFIG_SMP + .irq_set_affinity = plic_set_affinity, +#endif }; static int plic_irqdomain_map(struct irq_domain *d, unsigned int irq, -- 2.17.1 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2018-10-22 11:45 UTC|newest] Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-10-22 11:45 [PATCH 0/4] IRQ affinity support in PLIC driver Anup Patel 2018-10-22 11:45 ` Anup Patel 2018-10-22 11:45 ` [PATCH 1/4] irqchip: sifive-plic: Pre-compute context hart base and enable base Anup Patel 2018-10-22 11:45 ` Anup Patel 2018-11-09 8:42 ` Christoph Hellwig 2018-11-09 8:42 ` Christoph Hellwig 2018-11-12 4:27 ` Anup Patel 2018-11-12 4:27 ` Anup Patel 2018-10-22 11:45 ` [PATCH 2/4] irqchip: sifive-plic: More flexible plic_irq_toggle() Anup Patel 2018-10-22 11:45 ` Anup Patel 2018-11-09 8:43 ` Christoph Hellwig 2018-11-09 8:43 ` Christoph Hellwig 2018-11-12 12:33 ` Anup Patel 2018-11-12 12:33 ` Anup Patel 2018-10-22 11:45 ` [PATCH 3/4] irqchip: sifive-plic: Differentiate between PLIC handler and context Anup Patel 2018-10-22 11:45 ` Anup Patel 2018-10-22 11:45 ` Anup Patel [this message] 2018-10-22 11:45 ` [PATCH 4/4] irqchip: sifive-plic: Implement irq_set_affinity() for SMP host Anup Patel
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20181022114517.22748-5-anup@brainfault.org \ --to=anup@brainfault.org \ --cc=linux-riscv@lists.infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).