* [PATCH 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster @ 2017-10-25 23:37 Paul Burton 2017-10-25 23:37 ` [PATCH 1/8] irqchip: mips-gic: Inline gic_local_irq_domain_map() Paul Burton ` (7 more replies) 0 siblings, 8 replies; 25+ messages in thread From: Paul Burton @ 2017-10-25 23:37 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton This series continues cleaning & fixing up the MIPS GIC irqchip driver whilst laying groundwork to support multi-cluster systems. Patch 1 refactors in order to reduce some duplication and prepare us for the following patches. Patches 2-4 move per-CPU GIC configuration away from being performed all at once when the driver is probed or when interrupts are masked & unmasked, instead performing configuration as CPUs are brought online. This allows us to support reconfiguring after clusters are powered down & back up, generally cleans up and fixes bugs in the process. Patch 5 makes use of num_possible_cpus() to reserve IPIs, rather than the gic_vpes variable. This prepares us for multi-cluster in which gic_vpes is mostly meaningless since it only reflects the local cluster, and it generally makes more sense to use the more standard num_possible_cpus(). Patch 6 removes the now unused gic_vpes variable. Patch 7 is a general clean up but also prepares us for later patches as described in its commit message. Patch 8 is a general clean up marking some variables static. This series by itself continues along the path towards supporting multi-cluster systems such as the MIPS I6500, but does not yet get us the whole way there. If you wish to see my current work in progress which builds out multi-cluster support atop these patches then that can be found in the multicluster branch of: git://git.linux-mips.org/pub/scm/paul/linux.git Or browsed at: https://git.linux-mips.org/cgit/paul/linux.git/log/?h=multicluster This series applies cleanly atop v4.14-rc6. Paul Burton (8): irqchip: mips-gic: Inline gic_local_irq_domain_map() irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs irqchip: mips-gic: Mask local interrupts when CPUs come online irqchip: mips-gic: Configure EIC when CPUs come online irqchip: mips-gic: Use num_possible_cpus() to reserve IPIs irqchip: mips-gic: Remove gic_vpes variable irqchip: mips-gic: Share register writes in gic_set_type() irqchip: mips-gic: Make IPI bitmaps static drivers/irqchip/irq-mips-gic.c | 213 ++++++++++++++++++++++------------------- 1 file changed, 114 insertions(+), 99 deletions(-) -- 2.14.3 ^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH 1/8] irqchip: mips-gic: Inline gic_local_irq_domain_map() 2017-10-25 23:37 [PATCH 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton @ 2017-10-25 23:37 ` Paul Burton 2017-10-25 23:37 ` [PATCH 2/8] irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs Paul Burton ` (6 subsequent siblings) 7 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-25 23:37 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton The gic_local_irq_domain_map() function has only one callsite in gic_irq_domain_map(), and the split between the two functions makes it unclear that they duplicate calculations & checks. Inline gic_local_irq_domain_map() into gic_irq_domain_map() in order to clean this up. Doing this makes the following small issues obvious, and the patch tidies them up: - Both functions used GIC_HWIRQ_TO_LOCAL() to convert a hwirq number to a local IRQ number. We now only do this once. Although the compiler ought to have optimised this away before anyway, the change leaves us with less duplicate code. - gic_local_irq_domain_map() had a check for invalid local interrupt numbers (intr > GIC_LOCAL_INT_FDC). This condition can never occur because any hwirq higher than those used for local interrupts is a shared interrupt, which gic_irq_domain_map() already handles separately. We therefore remove this check. - The decision of whether to map the interrupt to gic_cpu_pin or timer_cpu_pin can be handled within the existing switch statement in gic_irq_domain_map(), shortening the code a little. The change additionally prepares us nicely for the following patch of the series which would otherwise need to duplicate the check for whether a local interrupt should be percpu_devid or just percpu (ie. the switch statement from gic_irq_domain_map()) in gic_local_irq_domain_map(). Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- drivers/irqchip/irq-mips-gic.c | 58 ++++++++++++++++-------------------------- 1 file changed, 22 insertions(+), 36 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index c90976d7e53c..6fdcc1552fab 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -382,39 +382,6 @@ static void gic_irq_dispatch(struct irq_desc *desc) gic_handle_shared_int(true); } -static int gic_local_irq_domain_map(struct irq_domain *d, unsigned int virq, - irq_hw_number_t hw) -{ - int intr = GIC_HWIRQ_TO_LOCAL(hw); - int i; - unsigned long flags; - u32 val; - - if (!gic_local_irq_is_routable(intr)) - return -EPERM; - - if (intr > GIC_LOCAL_INT_FDC) { - pr_err("Invalid local IRQ %d\n", intr); - return -EINVAL; - } - - if (intr == GIC_LOCAL_INT_TIMER) { - /* CONFIG_MIPS_CMP workaround (see __gic_init) */ - val = GIC_MAP_PIN_MAP_TO_PIN | timer_cpu_pin; - } else { - val = GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin; - } - - spin_lock_irqsave(&gic_lock, flags); - for (i = 0; i < gic_vpes; i++) { - write_gic_vl_other(mips_cm_vp_id(i)); - write_gic_vo_map(intr, val); - } - spin_unlock_irqrestore(&gic_lock, flags); - - return 0; -} - static int gic_shared_irq_domain_map(struct irq_domain *d, unsigned int virq, irq_hw_number_t hw, unsigned int cpu) { @@ -457,7 +424,10 @@ static int gic_irq_domain_xlate(struct irq_domain *d, struct device_node *ctrlr, static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, irq_hw_number_t hwirq) { - int err; + unsigned long flags; + unsigned int intr; + int err, i; + u32 map; if (hwirq >= GIC_SHARED_HWIRQ_BASE) { /* verify that shared irqs don't conflict with an IPI irq */ @@ -474,8 +444,14 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, return gic_shared_irq_domain_map(d, virq, hwirq, 0); } - switch (GIC_HWIRQ_TO_LOCAL(hwirq)) { + intr = GIC_HWIRQ_TO_LOCAL(hwirq); + map = GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin; + + switch (intr) { case GIC_LOCAL_INT_TIMER: + /* CONFIG_MIPS_CMP workaround (see __gic_init) */ + map = GIC_MAP_PIN_MAP_TO_PIN | timer_cpu_pin; + /* fall-through */ case GIC_LOCAL_INT_PERFCTR: case GIC_LOCAL_INT_FDC: /* @@ -504,7 +480,17 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, break; } - return gic_local_irq_domain_map(d, virq, hwirq); + if (!gic_local_irq_is_routable(intr)) + return -EPERM; + + spin_lock_irqsave(&gic_lock, flags); + for (i = 0; i < gic_vpes; i++) { + write_gic_vl_other(mips_cm_vp_id(i)); + write_gic_vo_map(intr, map); + } + spin_unlock_irqrestore(&gic_lock, flags); + + return 0; } static int gic_irq_domain_alloc(struct irq_domain *d, unsigned int virq, -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 2/8] irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs 2017-10-25 23:37 [PATCH 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton 2017-10-25 23:37 ` [PATCH 1/8] irqchip: mips-gic: Inline gic_local_irq_domain_map() Paul Burton @ 2017-10-25 23:37 ` Paul Burton 2017-10-30 8:00 ` Marc Zyngier 2017-10-25 23:37 ` [PATCH 3/8] irqchip: mips-gic: Mask local interrupts when CPUs come online Paul Burton ` (5 subsequent siblings) 7 siblings, 1 reply; 25+ messages in thread From: Paul Burton @ 2017-10-25 23:37 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton The gic_all_vpes_local_irq_controller chip currently attempts to operate on all CPUs/VPs in the system when masking or unmasking an interrupt. This has a few drawbacks: - In multi-cluster systems we may not always have access to all CPUs in the system. When all CPUs in a cluster are powered down that cluster's GIC may also power down, in which case we cannot configure its state. - Relatedly, if we power down a cluster after having configured interrupts for CPUs within it then the cluster's GIC may lose state & we need to reconfigure it. The current approach doesn't take this into account. - It's wasteful if we run Linux on fewer VPs than are present in the system. For example if we run a uniprocessor kernel on CPU0 of a system with 16 CPUs then there's no point in us configuring CPUs 1-15. - The implementation is also lacking in that it expects the range 0..gic_vpes-1 to represent valid Linux CPU numbers which may not always be the case - for example if we run on a system with more VPs than the kernel is configured to support. Fix all of these issues by only configuring the affected interrupts for CPUs which are online at the time, and recording the configuration in a new struct gic_all_vpes_chip_data for later use by CPUs being brought online. We register a CPU hotplug state (reusing CPUHP_AP_IRQ_GIC_STARTING which the ARM GIC driver uses, and which seems suitably generic for reuse with the MIPS GIC) and execute irq_cpu_online() in order to configure the interrupts on the newly onlined CPU. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- drivers/irqchip/irq-mips-gic.c | 72 ++++++++++++++++++++++++++++++++---------- 1 file changed, 56 insertions(+), 16 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 6fdcc1552fab..dd9da773db90 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -8,6 +8,7 @@ */ #include <linux/bitmap.h> #include <linux/clocksource.h> +#include <linux/cpuhotplug.h> #include <linux/init.h> #include <linux/interrupt.h> #include <linux/irq.h> @@ -55,6 +56,11 @@ static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller; DECLARE_BITMAP(ipi_resrv, GIC_MAX_INTRS); DECLARE_BITMAP(ipi_available, GIC_MAX_INTRS); +static struct gic_all_vpes_chip_data { + u32 map; + bool mask; +} gic_all_vpes_chip_data[GIC_NUM_LOCAL_INTRS]; + static void gic_clear_pcpu_masks(unsigned int intr) { unsigned int i; @@ -338,13 +344,17 @@ static struct irq_chip gic_local_irq_controller = { static void gic_mask_local_irq_all_vpes(struct irq_data *d) { - int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); - int i; + struct gic_all_vpes_chip_data *cd; unsigned long flags; + int intr, cpu; + + intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); + cd = irq_data_get_irq_chip_data(d); + cd->mask = false; spin_lock_irqsave(&gic_lock, flags); - for (i = 0; i < gic_vpes; i++) { - write_gic_vl_other(mips_cm_vp_id(i)); + for_each_online_cpu(cpu) { + write_gic_vl_other(mips_cm_vp_id(cpu)); write_gic_vo_rmask(BIT(intr)); } spin_unlock_irqrestore(&gic_lock, flags); @@ -352,22 +362,40 @@ static void gic_mask_local_irq_all_vpes(struct irq_data *d) static void gic_unmask_local_irq_all_vpes(struct irq_data *d) { - int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); - int i; + struct gic_all_vpes_chip_data *cd; unsigned long flags; + int intr, cpu; + + intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); + cd = irq_data_get_irq_chip_data(d); + cd->mask = true; spin_lock_irqsave(&gic_lock, flags); - for (i = 0; i < gic_vpes; i++) { - write_gic_vl_other(mips_cm_vp_id(i)); + for_each_online_cpu(cpu) { + write_gic_vl_other(mips_cm_vp_id(cpu)); write_gic_vo_smask(BIT(intr)); } spin_unlock_irqrestore(&gic_lock, flags); } +static void gic_all_vpes_irq_cpu_online(struct irq_data *d) +{ + struct gic_all_vpes_chip_data *cd; + unsigned int intr; + + intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); + cd = irq_data_get_irq_chip_data(d); + + write_gic_vl_map(intr, cd->map); + if (cd->mask) + write_gic_vl_smask(BIT(intr)); +} + static struct irq_chip gic_all_vpes_local_irq_controller = { - .name = "MIPS GIC Local", - .irq_mask = gic_mask_local_irq_all_vpes, - .irq_unmask = gic_unmask_local_irq_all_vpes, + .name = "MIPS GIC Local", + .irq_mask = gic_mask_local_irq_all_vpes, + .irq_unmask = gic_unmask_local_irq_all_vpes, + .irq_cpu_online = gic_all_vpes_irq_cpu_online, }; static void __gic_irq_dispatch(void) @@ -424,9 +452,10 @@ static int gic_irq_domain_xlate(struct irq_domain *d, struct device_node *ctrlr, static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, irq_hw_number_t hwirq) { + struct gic_all_vpes_chip_data *cd; unsigned long flags; unsigned int intr; - int err, i; + int err, cpu; u32 map; if (hwirq >= GIC_SHARED_HWIRQ_BASE) { @@ -459,9 +488,11 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, * the rest of the MIPS kernel code does not use the * percpu IRQ API for them. */ + cd = &gic_all_vpes_chip_data[intr]; + cd->map = map; err = irq_domain_set_hwirq_and_chip(d, virq, hwirq, &gic_all_vpes_local_irq_controller, - NULL); + cd); if (err) return err; @@ -484,8 +515,8 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, return -EPERM; spin_lock_irqsave(&gic_lock, flags); - for (i = 0; i < gic_vpes; i++) { - write_gic_vl_other(mips_cm_vp_id(i)); + for_each_online_cpu(cpu) { + write_gic_vl_other(mips_cm_vp_id(cpu)); write_gic_vo_map(intr, map); } spin_unlock_irqrestore(&gic_lock, flags); @@ -622,6 +653,13 @@ static const struct irq_domain_ops gic_ipi_domain_ops = { .match = gic_ipi_domain_match, }; +static int gic_cpu_startup(unsigned int cpu) +{ + /* Invoke irq_cpu_online callbacks to enable desired interrupts */ + irq_cpu_online(); + + return 0; +} static int __init gic_of_init(struct device_node *node, struct device_node *parent) @@ -768,6 +806,8 @@ static int __init gic_of_init(struct device_node *node, } } - return 0; + return cpuhp_setup_state(CPUHP_AP_IRQ_GIC_STARTING, + "irqchip/mips/gic:starting", + gic_cpu_startup, NULL); } IRQCHIP_DECLARE(mips_gic, "mti,gic", gic_of_init); -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PATCH 2/8] irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs 2017-10-25 23:37 ` [PATCH 2/8] irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs Paul Burton @ 2017-10-30 8:00 ` Marc Zyngier 2017-10-30 16:36 ` Paul Burton 0 siblings, 1 reply; 25+ messages in thread From: Marc Zyngier @ 2017-10-30 8:00 UTC (permalink / raw) To: Paul Burton; +Cc: Jason Cooper, Thomas Gleixner, linux-mips, linux-kernel On Wed, Oct 25 2017 at 5:37:24 pm BST, Paul Burton <paul.burton@mips.com> wrote: > The gic_all_vpes_local_irq_controller chip currently attempts to operate > on all CPUs/VPs in the system when masking or unmasking an interrupt. > This has a few drawbacks: > > - In multi-cluster systems we may not always have access to all CPUs in > the system. When all CPUs in a cluster are powered down that > cluster's GIC may also power down, in which case we cannot configure > its state. > > - Relatedly, if we power down a cluster after having configured > interrupts for CPUs within it then the cluster's GIC may lose state & > we need to reconfigure it. The current approach doesn't take this > into account. > > - It's wasteful if we run Linux on fewer VPs than are present in the > system. For example if we run a uniprocessor kernel on CPU0 of a > system with 16 CPUs then there's no point in us configuring CPUs > 1-15. > > - The implementation is also lacking in that it expects the range > 0..gic_vpes-1 to represent valid Linux CPU numbers which may not > always be the case - for example if we run on a system with more VPs > than the kernel is configured to support. > > Fix all of these issues by only configuring the affected interrupts for > CPUs which are online at the time, and recording the configuration in a > new struct gic_all_vpes_chip_data for later use by CPUs being brought > online. We register a CPU hotplug state (reusing > CPUHP_AP_IRQ_GIC_STARTING which the ARM GIC driver uses, and which seems > suitably generic for reuse with the MIPS GIC) and execute > irq_cpu_online() in order to configure the interrupts on the newly > onlined CPU. > > Signed-off-by: Paul Burton <paul.burton@mips.com> > Cc: Jason Cooper <jason@lakedaemon.net> > Cc: Marc Zyngier <marc.zyngier@arm.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: linux-mips@linux-mips.org > --- > > drivers/irqchip/irq-mips-gic.c | 72 ++++++++++++++++++++++++++++++++---------- > 1 file changed, 56 insertions(+), 16 deletions(-) > > diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c > index 6fdcc1552fab..dd9da773db90 100644 > --- a/drivers/irqchip/irq-mips-gic.c > +++ b/drivers/irqchip/irq-mips-gic.c [...] > @@ -622,6 +653,13 @@ static const struct irq_domain_ops gic_ipi_domain_ops = { > .match = gic_ipi_domain_match, > }; > > +static int gic_cpu_startup(unsigned int cpu) > +{ > + /* Invoke irq_cpu_online callbacks to enable desired interrupts */ > + irq_cpu_online(); > + > + return 0; > +} > > static int __init gic_of_init(struct device_node *node, > struct device_node *parent) > @@ -768,6 +806,8 @@ static int __init gic_of_init(struct device_node *node, > } > } > > - return 0; > + return cpuhp_setup_state(CPUHP_AP_IRQ_GIC_STARTING, > + "irqchip/mips/gic:starting", > + gic_cpu_startup, NULL); I'm wondering about this. CPUHP_AP_IRQ_GIC_STARTING is a symbol that is used on ARM platforms. You're very welcome to use it (as long as nobody builds a system with both an ARM GIC and a MIPS GIC...), but I'm a bit worried that we could end-up breaking things if one of us decides to reorder it in enum cpuhp_state. The safest option would be for you to add your own state value, which would allow the two architecture to evolve independently. Thoughts? M. -- Jazz is not dead. It just smells funny. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 2/8] irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs 2017-10-30 8:00 ` Marc Zyngier @ 2017-10-30 16:36 ` Paul Burton 2017-10-31 1:35 ` Marc Zyngier 0 siblings, 1 reply; 25+ messages in thread From: Paul Burton @ 2017-10-30 16:36 UTC (permalink / raw) To: Marc Zyngier Cc: Paul Burton, Jason Cooper, Thomas Gleixner, linux-mips, linux-kernel Hi Marc, On Mon, Oct 30, 2017 at 08:00:08AM +0000, Marc Zyngier wrote: > > static int __init gic_of_init(struct device_node *node, > > struct device_node *parent) > > @@ -768,6 +806,8 @@ static int __init gic_of_init(struct device_node *node, > > } > > } > > > > - return 0; > > + return cpuhp_setup_state(CPUHP_AP_IRQ_GIC_STARTING, > > + "irqchip/mips/gic:starting", > > + gic_cpu_startup, NULL); > > I'm wondering about this. CPUHP_AP_IRQ_GIC_STARTING is a symbol that is > used on ARM platforms. You're very welcome to use it (as long as nobody > builds a system with both an ARM GIC and a MIPS GIC...), but I'm a bit > worried that we could end-up breaking things if one of us decides to > reorder it in enum cpuhp_state. > > The safest option would be for you to add your own state value, which > would allow the two architecture to evolve independently. I had figured that if something like that ever happens it'd be easy to split into 2 states at that point, but sure - I'm happy to add a MIPS-specific state now to avoid anyone needing to worry about it. Thanks, Paul ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH 2/8] irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs 2017-10-30 16:36 ` Paul Burton @ 2017-10-31 1:35 ` Marc Zyngier 2017-10-31 16:41 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton 0 siblings, 1 reply; 25+ messages in thread From: Marc Zyngier @ 2017-10-31 1:35 UTC (permalink / raw) To: Paul Burton; +Cc: Jason Cooper, Thomas Gleixner, linux-mips, linux-kernel On Mon, Oct 30 2017 at 9:36:16 am GMT, Paul Burton <paul.burton@mips.com> wrote: > Hi Marc, > > On Mon, Oct 30, 2017 at 08:00:08AM +0000, Marc Zyngier wrote: >> > static int __init gic_of_init(struct device_node *node, >> > struct device_node *parent) >> > @@ -768,6 +806,8 @@ static int __init gic_of_init(struct device_node *node, >> > } >> > } >> > >> > - return 0; >> > + return cpuhp_setup_state(CPUHP_AP_IRQ_GIC_STARTING, >> > + "irqchip/mips/gic:starting", >> > + gic_cpu_startup, NULL); >> >> I'm wondering about this. CPUHP_AP_IRQ_GIC_STARTING is a symbol that is >> used on ARM platforms. You're very welcome to use it (as long as nobody >> builds a system with both an ARM GIC and a MIPS GIC...), but I'm a bit >> worried that we could end-up breaking things if one of us decides to >> reorder it in enum cpuhp_state. >> >> The safest option would be for you to add your own state value, which >> would allow the two architecture to evolve independently. > > I had figured that if something like that ever happens it'd be easy to split > into 2 states at that point, but sure - I'm happy to add a MIPS-specific state > now to avoid anyone needing to worry about it. That would be my preferred option. Thanks, M. -- Jazz is not dead. It just smells funny. ^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster 2017-10-31 1:35 ` Marc Zyngier @ 2017-10-31 16:41 ` Paul Burton 2017-10-31 16:41 ` [PATCH v2 1/8] irqchip: mips-gic: Inline gic_local_irq_domain_map() Paul Burton ` (8 more replies) 0 siblings, 9 replies; 25+ messages in thread From: Paul Burton @ 2017-10-31 16:41 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton This series continues cleaning & fixing up the MIPS GIC irqchip driver whilst laying groundwork to support multi-cluster systems. Patch 1 refactors in order to reduce some duplication and prepare us for the following patches. Patches 2-4 move per-CPU GIC configuration away from being performed all at once when the driver is probed or when interrupts are masked & unmasked, instead performing configuration as CPUs are brought online. This allows us to support reconfiguring after clusters are powered down & back up, generally cleans up and fixes bugs in the process. Patch 5 makes use of num_possible_cpus() to reserve IPIs, rather than the gic_vpes variable. This prepares us for multi-cluster in which gic_vpes is mostly meaningless since it only reflects the local cluster, and it generally makes more sense to use the more standard num_possible_cpus(). Patch 6 removes the now unused gic_vpes variable. Patch 7 is a general clean up but also prepares us for later patches as described in its commit message. Patch 8 is a general clean up marking some variables static. This series by itself continues along the path towards supporting multi-cluster systems such as the MIPS I6500, but does not yet get us the whole way there. If you wish to see my current work in progress which builds out multi-cluster support atop these patches then that can be found in the multicluster branch of: git://git.linux-mips.org/pub/scm/paul/linux.git Or browsed at: https://git.linux-mips.org/cgit/paul/linux.git/log/?h=multicluster This series applies cleanly atop v4.14-rc7. Changes in v2: - Add & use CPUHP_AP_IRQ_MIPS_GIC_STARTING state rather than reusing the generically-named CPUHP_AP_IRQ_GIC_STARTING used for the ARM GIC. Paul Burton (8): irqchip: mips-gic: Inline gic_local_irq_domain_map() irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs irqchip: mips-gic: Mask local interrupts when CPUs come online irqchip: mips-gic: Configure EIC when CPUs come online irqchip: mips-gic: Use num_possible_cpus() to reserve IPIs irqchip: mips-gic: Remove gic_vpes variable irqchip: mips-gic: Share register writes in gic_set_type() irqchip: mips-gic: Make IPI bitmaps static drivers/irqchip/irq-mips-gic.c | 213 ++++++++++++++++++++++------------------- include/linux/cpuhotplug.h | 1 + 2 files changed, 115 insertions(+), 99 deletions(-) -- 2.14.3 ^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v2 1/8] irqchip: mips-gic: Inline gic_local_irq_domain_map() 2017-10-31 16:41 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton @ 2017-10-31 16:41 ` Paul Burton 2017-10-31 16:41 ` [PATCH v2 2/8] irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs Paul Burton ` (7 subsequent siblings) 8 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-31 16:41 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton The gic_local_irq_domain_map() function has only one callsite in gic_irq_domain_map(), and the split between the two functions makes it unclear that they duplicate calculations & checks. Inline gic_local_irq_domain_map() into gic_irq_domain_map() in order to clean this up. Doing this makes the following small issues obvious, and the patch tidies them up: - Both functions used GIC_HWIRQ_TO_LOCAL() to convert a hwirq number to a local IRQ number. We now only do this once. Although the compiler ought to have optimised this away before anyway, the change leaves us with less duplicate code. - gic_local_irq_domain_map() had a check for invalid local interrupt numbers (intr > GIC_LOCAL_INT_FDC). This condition can never occur because any hwirq higher than those used for local interrupts is a shared interrupt, which gic_irq_domain_map() already handles separately. We therefore remove this check. - The decision of whether to map the interrupt to gic_cpu_pin or timer_cpu_pin can be handled within the existing switch statement in gic_irq_domain_map(), shortening the code a little. The change additionally prepares us nicely for the following patch of the series which would otherwise need to duplicate the check for whether a local interrupt should be percpu_devid or just percpu (ie. the switch statement from gic_irq_domain_map()) in gic_local_irq_domain_map(). Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- Changes in v2: None drivers/irqchip/irq-mips-gic.c | 58 ++++++++++++++++-------------------------- 1 file changed, 22 insertions(+), 36 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index c90976d7e53c..6fdcc1552fab 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -382,39 +382,6 @@ static void gic_irq_dispatch(struct irq_desc *desc) gic_handle_shared_int(true); } -static int gic_local_irq_domain_map(struct irq_domain *d, unsigned int virq, - irq_hw_number_t hw) -{ - int intr = GIC_HWIRQ_TO_LOCAL(hw); - int i; - unsigned long flags; - u32 val; - - if (!gic_local_irq_is_routable(intr)) - return -EPERM; - - if (intr > GIC_LOCAL_INT_FDC) { - pr_err("Invalid local IRQ %d\n", intr); - return -EINVAL; - } - - if (intr == GIC_LOCAL_INT_TIMER) { - /* CONFIG_MIPS_CMP workaround (see __gic_init) */ - val = GIC_MAP_PIN_MAP_TO_PIN | timer_cpu_pin; - } else { - val = GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin; - } - - spin_lock_irqsave(&gic_lock, flags); - for (i = 0; i < gic_vpes; i++) { - write_gic_vl_other(mips_cm_vp_id(i)); - write_gic_vo_map(intr, val); - } - spin_unlock_irqrestore(&gic_lock, flags); - - return 0; -} - static int gic_shared_irq_domain_map(struct irq_domain *d, unsigned int virq, irq_hw_number_t hw, unsigned int cpu) { @@ -457,7 +424,10 @@ static int gic_irq_domain_xlate(struct irq_domain *d, struct device_node *ctrlr, static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, irq_hw_number_t hwirq) { - int err; + unsigned long flags; + unsigned int intr; + int err, i; + u32 map; if (hwirq >= GIC_SHARED_HWIRQ_BASE) { /* verify that shared irqs don't conflict with an IPI irq */ @@ -474,8 +444,14 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, return gic_shared_irq_domain_map(d, virq, hwirq, 0); } - switch (GIC_HWIRQ_TO_LOCAL(hwirq)) { + intr = GIC_HWIRQ_TO_LOCAL(hwirq); + map = GIC_MAP_PIN_MAP_TO_PIN | gic_cpu_pin; + + switch (intr) { case GIC_LOCAL_INT_TIMER: + /* CONFIG_MIPS_CMP workaround (see __gic_init) */ + map = GIC_MAP_PIN_MAP_TO_PIN | timer_cpu_pin; + /* fall-through */ case GIC_LOCAL_INT_PERFCTR: case GIC_LOCAL_INT_FDC: /* @@ -504,7 +480,17 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, break; } - return gic_local_irq_domain_map(d, virq, hwirq); + if (!gic_local_irq_is_routable(intr)) + return -EPERM; + + spin_lock_irqsave(&gic_lock, flags); + for (i = 0; i < gic_vpes; i++) { + write_gic_vl_other(mips_cm_vp_id(i)); + write_gic_vo_map(intr, map); + } + spin_unlock_irqrestore(&gic_lock, flags); + + return 0; } static int gic_irq_domain_alloc(struct irq_domain *d, unsigned int virq, -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 2/8] irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs 2017-10-31 16:41 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton 2017-10-31 16:41 ` [PATCH v2 1/8] irqchip: mips-gic: Inline gic_local_irq_domain_map() Paul Burton @ 2017-10-31 16:41 ` Paul Burton 2017-10-31 16:41 ` [PATCH v2 3/8] irqchip: mips-gic: Mask local interrupts when CPUs come online Paul Burton ` (6 subsequent siblings) 8 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-31 16:41 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton The gic_all_vpes_local_irq_controller chip currently attempts to operate on all CPUs/VPs in the system when masking or unmasking an interrupt. This has a few drawbacks: - In multi-cluster systems we may not always have access to all CPUs in the system. When all CPUs in a cluster are powered down that cluster's GIC may also power down, in which case we cannot configure its state. - Relatedly, if we power down a cluster after having configured interrupts for CPUs within it then the cluster's GIC may lose state & we need to reconfigure it. The current approach doesn't take this into account. - It's wasteful if we run Linux on fewer VPs than are present in the system. For example if we run a uniprocessor kernel on CPU0 of a system with 16 CPUs then there's no point in us configuring CPUs 1-15. - The implementation is also lacking in that it expects the range 0..gic_vpes-1 to represent valid Linux CPU numbers which may not always be the case - for example if we run on a system with more VPs than the kernel is configured to support. Fix all of these issues by only configuring the affected interrupts for CPUs which are online at the time, and recording the configuration in a new struct gic_all_vpes_chip_data for later use by CPUs being brought online. We register a CPU hotplug state (reusing CPUHP_AP_IRQ_GIC_STARTING which the ARM GIC driver uses, and which seems suitably generic for reuse with the MIPS GIC) and execute irq_cpu_online() in order to configure the interrupts on the newly onlined CPU. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- Changes in v2: - Add & use CPUHP_AP_IRQ_MIPS_GIC_STARTING state rather than reusing the generically-named CPUHP_AP_IRQ_GIC_STARTING used for the ARM GIC. drivers/irqchip/irq-mips-gic.c | 72 ++++++++++++++++++++++++++++++++---------- include/linux/cpuhotplug.h | 1 + 2 files changed, 57 insertions(+), 16 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 6fdcc1552fab..60f644279803 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -8,6 +8,7 @@ */ #include <linux/bitmap.h> #include <linux/clocksource.h> +#include <linux/cpuhotplug.h> #include <linux/init.h> #include <linux/interrupt.h> #include <linux/irq.h> @@ -55,6 +56,11 @@ static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller; DECLARE_BITMAP(ipi_resrv, GIC_MAX_INTRS); DECLARE_BITMAP(ipi_available, GIC_MAX_INTRS); +static struct gic_all_vpes_chip_data { + u32 map; + bool mask; +} gic_all_vpes_chip_data[GIC_NUM_LOCAL_INTRS]; + static void gic_clear_pcpu_masks(unsigned int intr) { unsigned int i; @@ -338,13 +344,17 @@ static struct irq_chip gic_local_irq_controller = { static void gic_mask_local_irq_all_vpes(struct irq_data *d) { - int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); - int i; + struct gic_all_vpes_chip_data *cd; unsigned long flags; + int intr, cpu; + + intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); + cd = irq_data_get_irq_chip_data(d); + cd->mask = false; spin_lock_irqsave(&gic_lock, flags); - for (i = 0; i < gic_vpes; i++) { - write_gic_vl_other(mips_cm_vp_id(i)); + for_each_online_cpu(cpu) { + write_gic_vl_other(mips_cm_vp_id(cpu)); write_gic_vo_rmask(BIT(intr)); } spin_unlock_irqrestore(&gic_lock, flags); @@ -352,22 +362,40 @@ static void gic_mask_local_irq_all_vpes(struct irq_data *d) static void gic_unmask_local_irq_all_vpes(struct irq_data *d) { - int intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); - int i; + struct gic_all_vpes_chip_data *cd; unsigned long flags; + int intr, cpu; + + intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); + cd = irq_data_get_irq_chip_data(d); + cd->mask = true; spin_lock_irqsave(&gic_lock, flags); - for (i = 0; i < gic_vpes; i++) { - write_gic_vl_other(mips_cm_vp_id(i)); + for_each_online_cpu(cpu) { + write_gic_vl_other(mips_cm_vp_id(cpu)); write_gic_vo_smask(BIT(intr)); } spin_unlock_irqrestore(&gic_lock, flags); } +static void gic_all_vpes_irq_cpu_online(struct irq_data *d) +{ + struct gic_all_vpes_chip_data *cd; + unsigned int intr; + + intr = GIC_HWIRQ_TO_LOCAL(d->hwirq); + cd = irq_data_get_irq_chip_data(d); + + write_gic_vl_map(intr, cd->map); + if (cd->mask) + write_gic_vl_smask(BIT(intr)); +} + static struct irq_chip gic_all_vpes_local_irq_controller = { - .name = "MIPS GIC Local", - .irq_mask = gic_mask_local_irq_all_vpes, - .irq_unmask = gic_unmask_local_irq_all_vpes, + .name = "MIPS GIC Local", + .irq_mask = gic_mask_local_irq_all_vpes, + .irq_unmask = gic_unmask_local_irq_all_vpes, + .irq_cpu_online = gic_all_vpes_irq_cpu_online, }; static void __gic_irq_dispatch(void) @@ -424,9 +452,10 @@ static int gic_irq_domain_xlate(struct irq_domain *d, struct device_node *ctrlr, static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, irq_hw_number_t hwirq) { + struct gic_all_vpes_chip_data *cd; unsigned long flags; unsigned int intr; - int err, i; + int err, cpu; u32 map; if (hwirq >= GIC_SHARED_HWIRQ_BASE) { @@ -459,9 +488,11 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, * the rest of the MIPS kernel code does not use the * percpu IRQ API for them. */ + cd = &gic_all_vpes_chip_data[intr]; + cd->map = map; err = irq_domain_set_hwirq_and_chip(d, virq, hwirq, &gic_all_vpes_local_irq_controller, - NULL); + cd); if (err) return err; @@ -484,8 +515,8 @@ static int gic_irq_domain_map(struct irq_domain *d, unsigned int virq, return -EPERM; spin_lock_irqsave(&gic_lock, flags); - for (i = 0; i < gic_vpes; i++) { - write_gic_vl_other(mips_cm_vp_id(i)); + for_each_online_cpu(cpu) { + write_gic_vl_other(mips_cm_vp_id(cpu)); write_gic_vo_map(intr, map); } spin_unlock_irqrestore(&gic_lock, flags); @@ -622,6 +653,13 @@ static const struct irq_domain_ops gic_ipi_domain_ops = { .match = gic_ipi_domain_match, }; +static int gic_cpu_startup(unsigned int cpu) +{ + /* Invoke irq_cpu_online callbacks to enable desired interrupts */ + irq_cpu_online(); + + return 0; +} static int __init gic_of_init(struct device_node *node, struct device_node *parent) @@ -768,6 +806,8 @@ static int __init gic_of_init(struct device_node *node, } } - return 0; + return cpuhp_setup_state(CPUHP_AP_IRQ_MIPS_GIC_STARTING, + "irqchip/mips/gic:starting", + gic_cpu_startup, NULL); } IRQCHIP_DECLARE(mips_gic, "mti,gic", gic_of_init); diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 6d508767e144..1966a45bc453 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -98,6 +98,7 @@ enum cpuhp_state { CPUHP_AP_IRQ_HIP04_STARTING, CPUHP_AP_IRQ_ARMADA_XP_STARTING, CPUHP_AP_IRQ_BCM2836_STARTING, + CPUHP_AP_IRQ_MIPS_GIC_STARTING, CPUHP_AP_ARM_MVEBU_COHERENCY, CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING, CPUHP_AP_PERF_X86_STARTING, -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 3/8] irqchip: mips-gic: Mask local interrupts when CPUs come online 2017-10-31 16:41 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton 2017-10-31 16:41 ` [PATCH v2 1/8] irqchip: mips-gic: Inline gic_local_irq_domain_map() Paul Burton 2017-10-31 16:41 ` [PATCH v2 2/8] irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs Paul Burton @ 2017-10-31 16:41 ` Paul Burton 2017-10-31 16:41 ` [PATCH v2 4/8] irqchip: mips-gic: Configure EIC " Paul Burton ` (5 subsequent siblings) 8 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-31 16:41 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton We currently walk through the range 0..gic_vpes-1, expecting these values all to be valid Linux CPU numbers to provide to mips_cm_vp_id(), and masking all routable local interrupts during boot. This approach has a few drawbacks: - In multi-cluster systems we won't have access to all CPU's GIC local registers when the driver is probed, since clusters (and their GICs) may be powered down at this point & only brought online later. - In multi-cluster systems we may power down clusters at runtime, for example if we offline all CPUs within it via hotplug, and the cluster's GIC may lose state. We therefore need to reinitialise it when powering back up, which this approach does not take into account. - The range 0..gic_vpes-1 may not all be valid Linux CPU numbers, for example if we run a kernel configured to support fewer CPUs than the system it is running on actually has. In this case we'll get garbage values from mips_cm_vp_id() as we read past the end of the cpu_data array. Fix this and simplify the code somewhat by writing an all-bits-set value to the VP-local reset mask register when a CPU is brought online, before any local interrupts are configured for it. This removes the need for us to access all CPUs during driver probe, removing all of the problems described above. In the name of simplicity we drop the checks for routability of interrupts and simply clear the mask bits for all interrupts. Bits for non-routable local interrupts will have no effect so there's no point performing extra work to avoid modifying them. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- Changes in v2: None drivers/irqchip/irq-mips-gic.c | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 60f644279803..bd732b256f67 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -655,6 +655,9 @@ static const struct irq_domain_ops gic_ipi_domain_ops = { static int gic_cpu_startup(unsigned int cpu) { + /* Clear all local IRQ masks (ie. disable all local interrupts) */ + write_gic_vl_rmask(~0); + /* Invoke irq_cpu_online callbacks to enable desired interrupts */ irq_cpu_online(); @@ -664,7 +667,7 @@ static int gic_cpu_startup(unsigned int cpu) static int __init gic_of_init(struct device_node *node, struct device_node *parent) { - unsigned int cpu_vec, i, j, gicconfig, cpu, v[2]; + unsigned int cpu_vec, i, gicconfig, cpu, v[2]; unsigned long reserved; phys_addr_t gic_base; struct resource res; @@ -797,15 +800,6 @@ static int __init gic_of_init(struct device_node *node, write_gic_rmask(i); } - for (i = 0; i < gic_vpes; i++) { - write_gic_vl_other(mips_cm_vp_id(i)); - for (j = 0; j < GIC_NUM_LOCAL_INTRS; j++) { - if (!gic_local_irq_is_routable(j)) - continue; - write_gic_vo_rmask(BIT(j)); - } - } - return cpuhp_setup_state(CPUHP_AP_IRQ_MIPS_GIC_STARTING, "irqchip/mips/gic:starting", gic_cpu_startup, NULL); -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 4/8] irqchip: mips-gic: Configure EIC when CPUs come online 2017-10-31 16:41 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton ` (2 preceding siblings ...) 2017-10-31 16:41 ` [PATCH v2 3/8] irqchip: mips-gic: Mask local interrupts when CPUs come online Paul Burton @ 2017-10-31 16:41 ` Paul Burton 2017-10-31 16:41 ` [PATCH v2 5/8] irqchip: mips-gic: Use num_possible_cpus() to reserve IPIs Paul Burton ` (4 subsequent siblings) 8 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-31 16:41 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton Rather than configuring EIC mode for all CPUs during boot, configure it locally on each when they come online. This will become important with multi-cluster support, since clusters may be powered on & off (for example via hotplug) and would lose the EIC configuration when powered off. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- Changes in v2: None drivers/irqchip/irq-mips-gic.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index bd732b256f67..b1320ccb9f94 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -655,6 +655,10 @@ static const struct irq_domain_ops gic_ipi_domain_ops = { static int gic_cpu_startup(unsigned int cpu) { + /* Enable or disable EIC */ + change_gic_vl_ctl(GIC_VX_CTL_EIC, + cpu_has_veic ? GIC_VX_CTL_EIC : 0); + /* Clear all local IRQ masks (ie. disable all local interrupts) */ write_gic_vl_rmask(~0); @@ -667,7 +671,7 @@ static int gic_cpu_startup(unsigned int cpu) static int __init gic_of_init(struct device_node *node, struct device_node *parent) { - unsigned int cpu_vec, i, gicconfig, cpu, v[2]; + unsigned int cpu_vec, i, gicconfig, v[2]; unsigned long reserved; phys_addr_t gic_base; struct resource res; @@ -722,12 +726,6 @@ static int __init gic_of_init(struct device_node *node, gic_vpes = gic_vpes + 1; if (cpu_has_veic) { - /* Set EIC mode for all VPEs */ - for_each_present_cpu(cpu) { - write_gic_vl_other(mips_cm_vp_id(cpu)); - write_gic_vo_ctl(GIC_VX_CTL_EIC); - } - /* Always use vector 1 in EIC mode */ gic_cpu_pin = 0; timer_cpu_pin = gic_cpu_pin; -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 5/8] irqchip: mips-gic: Use num_possible_cpus() to reserve IPIs 2017-10-31 16:41 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton ` (3 preceding siblings ...) 2017-10-31 16:41 ` [PATCH v2 4/8] irqchip: mips-gic: Configure EIC " Paul Burton @ 2017-10-31 16:41 ` Paul Burton 2017-10-31 16:41 ` [PATCH v2 6/8] irqchip: mips-gic: Remove gic_vpes variable Paul Burton ` (3 subsequent siblings) 8 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-31 16:41 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton Reserving a number of IPIs based upon the number of VPs reported by the GIC makes little sense for a few reasons: - The kernel may have been configured with NR_CPUS less than the number of VPs in the cluster, in which case using gic_vpes causes us to reserve more interrupts for IPIs than we will possibly use. - If a kernel is configured without support for multi-threading & runs on a system with multi-threading & multiple VPs per core then we'll similarly reserve more interrupts for IPIs than we will possibly use. - In systems with multiple clusters the GIC can only provide us with the number of VPs in its cluster, not across all clusters. In this case we'll reserve fewer interrupts for IPIs than we need. Fix these issues by using num_possible_cpus() instead, which in all cases is actually indicative of how many IPIs we may need. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- Changes in v2: None drivers/irqchip/irq-mips-gic.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index b1320ccb9f94..4304283bfb1a 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -671,7 +671,7 @@ static int gic_cpu_startup(unsigned int cpu) static int __init gic_of_init(struct device_node *node, struct device_node *parent) { - unsigned int cpu_vec, i, gicconfig, v[2]; + unsigned int cpu_vec, i, gicconfig, v[2], num_ipis; unsigned long reserved; phys_addr_t gic_base; struct resource res; @@ -781,10 +781,12 @@ static int __init gic_of_init(struct device_node *node, !of_property_read_u32_array(node, "mti,reserved-ipi-vectors", v, 2)) { bitmap_set(ipi_resrv, v[0], v[1]); } else { - /* Make the last 2 * gic_vpes available for IPIs */ - bitmap_set(ipi_resrv, - gic_shared_intrs - 2 * gic_vpes, - 2 * gic_vpes); + /* + * Reserve 2 interrupts per possible CPU/VP for use as IPIs, + * meeting the requirements of arch/mips SMP. + */ + num_ipis = 2 * num_possible_cpus(); + bitmap_set(ipi_resrv, gic_shared_intrs - num_ipis, num_ipis); } bitmap_copy(ipi_available, ipi_resrv, GIC_MAX_INTRS); -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 6/8] irqchip: mips-gic: Remove gic_vpes variable 2017-10-31 16:41 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton ` (4 preceding siblings ...) 2017-10-31 16:41 ` [PATCH v2 5/8] irqchip: mips-gic: Use num_possible_cpus() to reserve IPIs Paul Burton @ 2017-10-31 16:41 ` Paul Burton 2017-10-31 16:41 ` [PATCH v2 7/8] irqchip: mips-gic: Share register writes in gic_set_type() Paul Burton ` (2 subsequent siblings) 8 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-31 16:41 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton Following the past few patches nothing uses the gic_vpes variable any longer. Remove the dead code. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- Changes in v2: None drivers/irqchip/irq-mips-gic.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 4304283bfb1a..48f0f43cd05d 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -49,7 +49,6 @@ static DEFINE_SPINLOCK(gic_lock); static struct irq_domain *gic_irq_domain; static struct irq_domain *gic_ipi_domain; static int gic_shared_intrs; -static int gic_vpes; static unsigned int gic_cpu_pin; static unsigned int timer_cpu_pin; static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller; @@ -721,10 +720,6 @@ static int __init gic_of_init(struct device_node *node, gic_shared_intrs >>= __ffs(GIC_CONFIG_NUMINTERRUPTS); gic_shared_intrs = (gic_shared_intrs + 1) * 8; - gic_vpes = gicconfig & GIC_CONFIG_PVPS; - gic_vpes >>= __ffs(GIC_CONFIG_PVPS); - gic_vpes = gic_vpes + 1; - if (cpu_has_veic) { /* Always use vector 1 in EIC mode */ gic_cpu_pin = 0; -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 7/8] irqchip: mips-gic: Share register writes in gic_set_type() 2017-10-31 16:41 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton ` (5 preceding siblings ...) 2017-10-31 16:41 ` [PATCH v2 6/8] irqchip: mips-gic: Remove gic_vpes variable Paul Burton @ 2017-10-31 16:41 ` Paul Burton 2017-10-31 16:41 ` [PATCH v2 8/8] irqchip: mips-gic: Make IPI bitmaps static Paul Burton 2017-11-01 0:13 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Marc Zyngier 8 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-31 16:41 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton The gic_set_type() function included writes to the MIPS GIC polarity, trigger & dual-trigger registers in each case of a switch statement determining the IRQs type. This is all well & good when we only have a single cluster & thus a single GIC whose register we want to update. It will lead to significant duplication once we have multi-cluster support & multiple GICs to update. Refactor this such that we determine values for the polarity, trigger & dual-trigger registers and then have a single set of register writes following the switch statement. This will allow us to write the same values to each GIC in a multi-cluster system in a later patch, rather than needing to duplicate more register writes in each case. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- Changes in v2: None drivers/irqchip/irq-mips-gic.c | 46 +++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 48f0f43cd05d..b2e83461e2a8 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -199,46 +199,46 @@ static void gic_ack_irq(struct irq_data *d) static int gic_set_type(struct irq_data *d, unsigned int type) { - unsigned int irq = GIC_HWIRQ_TO_SHARED(d->hwirq); + unsigned int irq, pol, trig, dual; unsigned long flags; - bool is_edge; + + irq = GIC_HWIRQ_TO_SHARED(d->hwirq); spin_lock_irqsave(&gic_lock, flags); switch (type & IRQ_TYPE_SENSE_MASK) { case IRQ_TYPE_EDGE_FALLING: - change_gic_pol(irq, GIC_POL_FALLING_EDGE); - change_gic_trig(irq, GIC_TRIG_EDGE); - change_gic_dual(irq, GIC_DUAL_SINGLE); - is_edge = true; + pol = GIC_POL_FALLING_EDGE; + trig = GIC_TRIG_EDGE; + dual = GIC_DUAL_SINGLE; break; case IRQ_TYPE_EDGE_RISING: - change_gic_pol(irq, GIC_POL_RISING_EDGE); - change_gic_trig(irq, GIC_TRIG_EDGE); - change_gic_dual(irq, GIC_DUAL_SINGLE); - is_edge = true; + pol = GIC_POL_RISING_EDGE; + trig = GIC_TRIG_EDGE; + dual = GIC_DUAL_SINGLE; break; case IRQ_TYPE_EDGE_BOTH: - /* polarity is irrelevant in this case */ - change_gic_trig(irq, GIC_TRIG_EDGE); - change_gic_dual(irq, GIC_DUAL_DUAL); - is_edge = true; + pol = 0; /* Doesn't matter */ + trig = GIC_TRIG_EDGE; + dual = GIC_DUAL_DUAL; break; case IRQ_TYPE_LEVEL_LOW: - change_gic_pol(irq, GIC_POL_ACTIVE_LOW); - change_gic_trig(irq, GIC_TRIG_LEVEL); - change_gic_dual(irq, GIC_DUAL_SINGLE); - is_edge = false; + pol = GIC_POL_ACTIVE_LOW; + trig = GIC_TRIG_LEVEL; + dual = GIC_DUAL_SINGLE; break; case IRQ_TYPE_LEVEL_HIGH: default: - change_gic_pol(irq, GIC_POL_ACTIVE_HIGH); - change_gic_trig(irq, GIC_TRIG_LEVEL); - change_gic_dual(irq, GIC_DUAL_SINGLE); - is_edge = false; + pol = GIC_POL_ACTIVE_HIGH; + trig = GIC_TRIG_LEVEL; + dual = GIC_DUAL_SINGLE; break; } - if (is_edge) + change_gic_pol(irq, pol); + change_gic_trig(irq, trig); + change_gic_dual(irq, dual); + + if (trig == GIC_TRIG_EDGE) irq_set_chip_handler_name_locked(d, &gic_edge_irq_controller, handle_edge_irq, NULL); else -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH v2 8/8] irqchip: mips-gic: Make IPI bitmaps static 2017-10-31 16:41 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton ` (6 preceding siblings ...) 2017-10-31 16:41 ` [PATCH v2 7/8] irqchip: mips-gic: Share register writes in gic_set_type() Paul Burton @ 2017-10-31 16:41 ` Paul Burton 2017-11-01 0:13 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Marc Zyngier 8 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-31 16:41 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton We have 2 bitmaps used to keep track of interrupts dedicated to IPIs in the MIPS GIC irqchip driver. These bitmaps are only used from the one compilation unit of that driver, and so can be made static. Do so in order to avoid polluting the symbol table & global namespace. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- Changes in v2: None drivers/irqchip/irq-mips-gic.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index b2e83461e2a8..3ccebb020f40 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -52,8 +52,8 @@ static int gic_shared_intrs; static unsigned int gic_cpu_pin; static unsigned int timer_cpu_pin; static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller; -DECLARE_BITMAP(ipi_resrv, GIC_MAX_INTRS); -DECLARE_BITMAP(ipi_available, GIC_MAX_INTRS); +static DECLARE_BITMAP(ipi_resrv, GIC_MAX_INTRS); +static DECLARE_BITMAP(ipi_available, GIC_MAX_INTRS); static struct gic_all_vpes_chip_data { u32 map; -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster 2017-10-31 16:41 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton ` (7 preceding siblings ...) 2017-10-31 16:41 ` [PATCH v2 8/8] irqchip: mips-gic: Make IPI bitmaps static Paul Burton @ 2017-11-01 0:13 ` Marc Zyngier 2017-11-01 16:40 ` Paul Burton 8 siblings, 1 reply; 25+ messages in thread From: Marc Zyngier @ 2017-11-01 0:13 UTC (permalink / raw) To: Paul Burton; +Cc: Jason Cooper, Thomas Gleixner, linux-mips, linux-kernel On Tue, Oct 31 2017 at 9:41:43 am GMT, Paul Burton <paul.burton@mips.com> wrote: > This series continues cleaning & fixing up the MIPS GIC irqchip driver > whilst laying groundwork to support multi-cluster systems. > > Patch 1 refactors in order to reduce some duplication and prepare us for > the following patches. > > Patches 2-4 move per-CPU GIC configuration away from being performed all > at once when the driver is probed or when interrupts are masked & > unmasked, instead performing configuration as CPUs are brought online. > This allows us to support reconfiguring after clusters are powered down > & back up, generally cleans up and fixes bugs in the process. > > Patch 5 makes use of num_possible_cpus() to reserve IPIs, rather than > the gic_vpes variable. This prepares us for multi-cluster in which > gic_vpes is mostly meaningless since it only reflects the local cluster, > and it generally makes more sense to use the more standard > num_possible_cpus(). > > Patch 6 removes the now unused gic_vpes variable. > > Patch 7 is a general clean up but also prepares us for later patches as > described in its commit message. > > Patch 8 is a general clean up marking some variables static. > > This series by itself continues along the path towards supporting > multi-cluster systems such as the MIPS I6500, but does not yet get us > the whole way there. If you wish to see my current work in progress > which builds out multi-cluster support atop these patches then that can > be found in the multicluster branch of: > > git://git.linux-mips.org/pub/scm/paul/linux.git > > Or browsed at: > > https://git.linux-mips.org/cgit/paul/linux.git/log/?h=multicluster > > This series applies cleanly atop v4.14-rc7. Are those targeting 4.14 or 4.15? It is getting quite late for the former, and it doesn't seem to cleanly apply on tip/irq/core (or my irqchip-4.15 branch) if that's for the latter (patch 6 shouts at me). Thanks, M. -- Jazz is not dead. It just smells funny. ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster 2017-11-01 0:13 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Marc Zyngier @ 2017-11-01 16:40 ` Paul Burton 2017-11-01 16:59 ` Thomas Gleixner 0 siblings, 1 reply; 25+ messages in thread From: Paul Burton @ 2017-11-01 16:40 UTC (permalink / raw) To: Marc Zyngier; +Cc: Jason Cooper, Thomas Gleixner, linux-mips, linux-kernel Hi Marc, On Wed, Nov 01, 2017 at 12:13:16AM +0000, Marc Zyngier wrote: > On Tue, Oct 31 2017 at 9:41:43 am GMT, Paul Burton <paul.burton@mips.com> wrote: > > This series continues cleaning & fixing up the MIPS GIC irqchip driver > > whilst laying groundwork to support multi-cluster systems. <SNIP> > Are those targeting 4.14 or 4.15? It is getting quite late for the > former, and it doesn't seem to cleanly apply on tip/irq/core (or my > irqchip-4.15 branch) if that's for the latter (patch 6 shouts at me). Whichever you're happiest with. If you'd like me to rebase them & resubmit that's fine. I see the conflict with patch 6 atop tip/irq/core - it's because tip/irq/core is based upon v4.14-rc2 which doesn't have commit a08588ea486a ("irqchip/mips-gic: Fix shifts to extract register fields") that went into v4.14-rc3. The correct resolution is to keep the patches version of things (ie. delete the block of code). Thanks, Paul ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster 2017-11-01 16:40 ` Paul Burton @ 2017-11-01 16:59 ` Thomas Gleixner 2017-11-02 10:44 ` Marc Zyngier 0 siblings, 1 reply; 25+ messages in thread From: Thomas Gleixner @ 2017-11-01 16:59 UTC (permalink / raw) To: Paul Burton; +Cc: Marc Zyngier, Jason Cooper, linux-mips, linux-kernel On Wed, 1 Nov 2017, Paul Burton wrote: > Hi Marc, > > On Wed, Nov 01, 2017 at 12:13:16AM +0000, Marc Zyngier wrote: > > On Tue, Oct 31 2017 at 9:41:43 am GMT, Paul Burton <paul.burton@mips.com> wrote: > > > This series continues cleaning & fixing up the MIPS GIC irqchip driver > > > whilst laying groundwork to support multi-cluster systems. > > <SNIP> > > > Are those targeting 4.14 or 4.15? It is getting quite late for the > > former, and it doesn't seem to cleanly apply on tip/irq/core (or my > > irqchip-4.15 branch) if that's for the latter (patch 6 shouts at me). > > Whichever you're happiest with. If you'd like me to rebase them & resubmit > that's fine. > > I see the conflict with patch 6 atop tip/irq/core - it's because tip/irq/core > is based upon v4.14-rc2 which doesn't have commit a08588ea486a > ("irqchip/mips-gic: Fix shifts to extract register fields") that went into > v4.14-rc3. The correct resolution is to keep the patches version of things (ie. > delete the block of code). Marc, simply merge current linus into your branch with the reason: Pick up upstream fixes so dependent patches apply Thanks, tglx ^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster 2017-11-01 16:59 ` Thomas Gleixner @ 2017-11-02 10:44 ` Marc Zyngier 0 siblings, 0 replies; 25+ messages in thread From: Marc Zyngier @ 2017-11-02 10:44 UTC (permalink / raw) To: Thomas Gleixner; +Cc: Paul Burton, Jason Cooper, linux-mips, linux-kernel On Wed, Nov 01 2017 at 5:59:23 pm GMT, Thomas Gleixner <tglx@linutronix.de> wrote: > On Wed, 1 Nov 2017, Paul Burton wrote: > >> Hi Marc, >> >> On Wed, Nov 01, 2017 at 12:13:16AM +0000, Marc Zyngier wrote: >> > On Tue, Oct 31 2017 at 9:41:43 am GMT, Paul Burton >> > <paul.burton@mips.com> wrote: >> > > This series continues cleaning & fixing up the MIPS GIC irqchip driver >> > > whilst laying groundwork to support multi-cluster systems. >> >> <SNIP> >> >> > Are those targeting 4.14 or 4.15? It is getting quite late for the >> > former, and it doesn't seem to cleanly apply on tip/irq/core (or my >> > irqchip-4.15 branch) if that's for the latter (patch 6 shouts at me). >> >> Whichever you're happiest with. If you'd like me to rebase them & resubmit >> that's fine. >> >> I see the conflict with patch 6 atop tip/irq/core - it's because tip/irq/core >> is based upon v4.14-rc2 which doesn't have commit a08588ea486a >> ("irqchip/mips-gic: Fix shifts to extract register fields") that went into >> v4.14-rc3. The correct resolution is to keep the patches version of >> things (ie. >> delete the block of code). > > Marc, simply merge current linus into your branch with the reason: > > Pick up upstream fixes so dependent patches apply Fair enough. I've now merged -rc3 in my branch, and applied those without a hitch. Thanks, M. -- Jazz is not dead, it just smell funny. ^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH 3/8] irqchip: mips-gic: Mask local interrupts when CPUs come online 2017-10-25 23:37 [PATCH 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton 2017-10-25 23:37 ` [PATCH 1/8] irqchip: mips-gic: Inline gic_local_irq_domain_map() Paul Burton 2017-10-25 23:37 ` [PATCH 2/8] irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs Paul Burton @ 2017-10-25 23:37 ` Paul Burton 2017-10-25 23:37 ` [PATCH 4/8] irqchip: mips-gic: Configure EIC " Paul Burton ` (4 subsequent siblings) 7 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-25 23:37 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton We currently walk through the range 0..gic_vpes-1, expecting these values all to be valid Linux CPU numbers to provide to mips_cm_vp_id(), and masking all routable local interrupts during boot. This approach has a few drawbacks: - In multi-cluster systems we won't have access to all CPU's GIC local registers when the driver is probed, since clusters (and their GICs) may be powered down at this point & only brought online later. - In multi-cluster systems we may power down clusters at runtime, for example if we offline all CPUs within it via hotplug, and the cluster's GIC may lose state. We therefore need to reinitialise it when powering back up, which this approach does not take into account. - The range 0..gic_vpes-1 may not all be valid Linux CPU numbers, for example if we run a kernel configured to support fewer CPUs than the system it is running on actually has. In this case we'll get garbage values from mips_cm_vp_id() as we read past the end of the cpu_data array. Fix this and simplify the code somewhat by writing an all-bits-set value to the VP-local reset mask register when a CPU is brought online, before any local interrupts are configured for it. This removes the need for us to access all CPUs during driver probe, removing all of the problems described above. In the name of simplicity we drop the checks for routability of interrupts and simply clear the mask bits for all interrupts. Bits for non-routable local interrupts will have no effect so there's no point performing extra work to avoid modifying them. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- drivers/irqchip/irq-mips-gic.c | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index dd9da773db90..2a31949d09ce 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -655,6 +655,9 @@ static const struct irq_domain_ops gic_ipi_domain_ops = { static int gic_cpu_startup(unsigned int cpu) { + /* Clear all local IRQ masks (ie. disable all local interrupts) */ + write_gic_vl_rmask(~0); + /* Invoke irq_cpu_online callbacks to enable desired interrupts */ irq_cpu_online(); @@ -664,7 +667,7 @@ static int gic_cpu_startup(unsigned int cpu) static int __init gic_of_init(struct device_node *node, struct device_node *parent) { - unsigned int cpu_vec, i, j, gicconfig, cpu, v[2]; + unsigned int cpu_vec, i, gicconfig, cpu, v[2]; unsigned long reserved; phys_addr_t gic_base; struct resource res; @@ -797,15 +800,6 @@ static int __init gic_of_init(struct device_node *node, write_gic_rmask(i); } - for (i = 0; i < gic_vpes; i++) { - write_gic_vl_other(mips_cm_vp_id(i)); - for (j = 0; j < GIC_NUM_LOCAL_INTRS; j++) { - if (!gic_local_irq_is_routable(j)) - continue; - write_gic_vo_rmask(BIT(j)); - } - } - return cpuhp_setup_state(CPUHP_AP_IRQ_GIC_STARTING, "irqchip/mips/gic:starting", gic_cpu_startup, NULL); -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 4/8] irqchip: mips-gic: Configure EIC when CPUs come online 2017-10-25 23:37 [PATCH 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton ` (2 preceding siblings ...) 2017-10-25 23:37 ` [PATCH 3/8] irqchip: mips-gic: Mask local interrupts when CPUs come online Paul Burton @ 2017-10-25 23:37 ` Paul Burton 2017-10-25 23:37 ` [PATCH 5/8] irqchip: mips-gic: Use num_possible_cpus() to reserve IPIs Paul Burton ` (3 subsequent siblings) 7 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-25 23:37 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton Rather than configuring EIC mode for all CPUs during boot, configure it locally on each when they come online. This will become important with multi-cluster support, since clusters may be powered on & off (for example via hotplug) and would lose the EIC configuration when powered off. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- drivers/irqchip/irq-mips-gic.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 2a31949d09ce..95daa96d8b52 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -655,6 +655,10 @@ static const struct irq_domain_ops gic_ipi_domain_ops = { static int gic_cpu_startup(unsigned int cpu) { + /* Enable or disable EIC */ + change_gic_vl_ctl(GIC_VX_CTL_EIC, + cpu_has_veic ? GIC_VX_CTL_EIC : 0); + /* Clear all local IRQ masks (ie. disable all local interrupts) */ write_gic_vl_rmask(~0); @@ -667,7 +671,7 @@ static int gic_cpu_startup(unsigned int cpu) static int __init gic_of_init(struct device_node *node, struct device_node *parent) { - unsigned int cpu_vec, i, gicconfig, cpu, v[2]; + unsigned int cpu_vec, i, gicconfig, v[2]; unsigned long reserved; phys_addr_t gic_base; struct resource res; @@ -722,12 +726,6 @@ static int __init gic_of_init(struct device_node *node, gic_vpes = gic_vpes + 1; if (cpu_has_veic) { - /* Set EIC mode for all VPEs */ - for_each_present_cpu(cpu) { - write_gic_vl_other(mips_cm_vp_id(cpu)); - write_gic_vo_ctl(GIC_VX_CTL_EIC); - } - /* Always use vector 1 in EIC mode */ gic_cpu_pin = 0; timer_cpu_pin = gic_cpu_pin; -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 5/8] irqchip: mips-gic: Use num_possible_cpus() to reserve IPIs 2017-10-25 23:37 [PATCH 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton ` (3 preceding siblings ...) 2017-10-25 23:37 ` [PATCH 4/8] irqchip: mips-gic: Configure EIC " Paul Burton @ 2017-10-25 23:37 ` Paul Burton 2017-10-25 23:37 ` [PATCH 6/8] irqchip: mips-gic: Remove gic_vpes variable Paul Burton ` (2 subsequent siblings) 7 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-25 23:37 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton Reserving a number of IPIs based upon the number of VPs reported by the GIC makes little sense for a few reasons: - The kernel may have been configured with NR_CPUS less than the number of VPs in the cluster, in which case using gic_vpes causes us to reserve more interrupts for IPIs than we will possibly use. - If a kernel is configured without support for multi-threading & runs on a system with multi-threading & multiple VPs per core then we'll similarly reserve more interrupts for IPIs than we will possibly use. - In systems with multiple clusters the GIC can only provide us with the number of VPs in its cluster, not across all clusters. In this case we'll reserve fewer interrupts for IPIs than we need. Fix these issues by using num_possible_cpus() instead, which in all cases is actually indicative of how many IPIs we may need. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- drivers/irqchip/irq-mips-gic.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 95daa96d8b52..70e18026f896 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -671,7 +671,7 @@ static int gic_cpu_startup(unsigned int cpu) static int __init gic_of_init(struct device_node *node, struct device_node *parent) { - unsigned int cpu_vec, i, gicconfig, v[2]; + unsigned int cpu_vec, i, gicconfig, v[2], num_ipis; unsigned long reserved; phys_addr_t gic_base; struct resource res; @@ -781,10 +781,12 @@ static int __init gic_of_init(struct device_node *node, !of_property_read_u32_array(node, "mti,reserved-ipi-vectors", v, 2)) { bitmap_set(ipi_resrv, v[0], v[1]); } else { - /* Make the last 2 * gic_vpes available for IPIs */ - bitmap_set(ipi_resrv, - gic_shared_intrs - 2 * gic_vpes, - 2 * gic_vpes); + /* + * Reserve 2 interrupts per possible CPU/VP for use as IPIs, + * meeting the requirements of arch/mips SMP. + */ + num_ipis = 2 * num_possible_cpus(); + bitmap_set(ipi_resrv, gic_shared_intrs - num_ipis, num_ipis); } bitmap_copy(ipi_available, ipi_resrv, GIC_MAX_INTRS); -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 6/8] irqchip: mips-gic: Remove gic_vpes variable 2017-10-25 23:37 [PATCH 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton ` (4 preceding siblings ...) 2017-10-25 23:37 ` [PATCH 5/8] irqchip: mips-gic: Use num_possible_cpus() to reserve IPIs Paul Burton @ 2017-10-25 23:37 ` Paul Burton 2017-10-25 23:37 ` [PATCH 7/8] irqchip: mips-gic: Share register writes in gic_set_type() Paul Burton 2017-10-25 23:37 ` [PATCH 8/8] irqchip: mips-gic: Make IPI bitmaps static Paul Burton 7 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-25 23:37 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton Following the past few patches nothing uses the gic_vpes variable any longer. Remove the dead code. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- drivers/irqchip/irq-mips-gic.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 70e18026f896..88797e1c58e2 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -49,7 +49,6 @@ static DEFINE_SPINLOCK(gic_lock); static struct irq_domain *gic_irq_domain; static struct irq_domain *gic_ipi_domain; static int gic_shared_intrs; -static int gic_vpes; static unsigned int gic_cpu_pin; static unsigned int timer_cpu_pin; static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller; @@ -721,10 +720,6 @@ static int __init gic_of_init(struct device_node *node, gic_shared_intrs >>= __ffs(GIC_CONFIG_NUMINTERRUPTS); gic_shared_intrs = (gic_shared_intrs + 1) * 8; - gic_vpes = gicconfig & GIC_CONFIG_PVPS; - gic_vpes >>= __ffs(GIC_CONFIG_PVPS); - gic_vpes = gic_vpes + 1; - if (cpu_has_veic) { /* Always use vector 1 in EIC mode */ gic_cpu_pin = 0; -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 7/8] irqchip: mips-gic: Share register writes in gic_set_type() 2017-10-25 23:37 [PATCH 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton ` (5 preceding siblings ...) 2017-10-25 23:37 ` [PATCH 6/8] irqchip: mips-gic: Remove gic_vpes variable Paul Burton @ 2017-10-25 23:37 ` Paul Burton 2017-10-25 23:37 ` [PATCH 8/8] irqchip: mips-gic: Make IPI bitmaps static Paul Burton 7 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-25 23:37 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton The gic_set_type() function included writes to the MIPS GIC polarity, trigger & dual-trigger registers in each case of a switch statement determining the IRQs type. This is all well & good when we only have a single cluster & thus a single GIC whose register we want to update. It will lead to significant duplication once we have multi-cluster support & multiple GICs to update. Refactor this such that we determine values for the polarity, trigger & dual-trigger registers and then have a single set of register writes following the switch statement. This will allow us to write the same values to each GIC in a multi-cluster system in a later patch, rather than needing to duplicate more register writes in each case. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- drivers/irqchip/irq-mips-gic.c | 46 +++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 88797e1c58e2..8c719a9912fc 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -199,46 +199,46 @@ static void gic_ack_irq(struct irq_data *d) static int gic_set_type(struct irq_data *d, unsigned int type) { - unsigned int irq = GIC_HWIRQ_TO_SHARED(d->hwirq); + unsigned int irq, pol, trig, dual; unsigned long flags; - bool is_edge; + + irq = GIC_HWIRQ_TO_SHARED(d->hwirq); spin_lock_irqsave(&gic_lock, flags); switch (type & IRQ_TYPE_SENSE_MASK) { case IRQ_TYPE_EDGE_FALLING: - change_gic_pol(irq, GIC_POL_FALLING_EDGE); - change_gic_trig(irq, GIC_TRIG_EDGE); - change_gic_dual(irq, GIC_DUAL_SINGLE); - is_edge = true; + pol = GIC_POL_FALLING_EDGE; + trig = GIC_TRIG_EDGE; + dual = GIC_DUAL_SINGLE; break; case IRQ_TYPE_EDGE_RISING: - change_gic_pol(irq, GIC_POL_RISING_EDGE); - change_gic_trig(irq, GIC_TRIG_EDGE); - change_gic_dual(irq, GIC_DUAL_SINGLE); - is_edge = true; + pol = GIC_POL_RISING_EDGE; + trig = GIC_TRIG_EDGE; + dual = GIC_DUAL_SINGLE; break; case IRQ_TYPE_EDGE_BOTH: - /* polarity is irrelevant in this case */ - change_gic_trig(irq, GIC_TRIG_EDGE); - change_gic_dual(irq, GIC_DUAL_DUAL); - is_edge = true; + pol = 0; /* Doesn't matter */ + trig = GIC_TRIG_EDGE; + dual = GIC_DUAL_DUAL; break; case IRQ_TYPE_LEVEL_LOW: - change_gic_pol(irq, GIC_POL_ACTIVE_LOW); - change_gic_trig(irq, GIC_TRIG_LEVEL); - change_gic_dual(irq, GIC_DUAL_SINGLE); - is_edge = false; + pol = GIC_POL_ACTIVE_LOW; + trig = GIC_TRIG_LEVEL; + dual = GIC_DUAL_SINGLE; break; case IRQ_TYPE_LEVEL_HIGH: default: - change_gic_pol(irq, GIC_POL_ACTIVE_HIGH); - change_gic_trig(irq, GIC_TRIG_LEVEL); - change_gic_dual(irq, GIC_DUAL_SINGLE); - is_edge = false; + pol = GIC_POL_ACTIVE_HIGH; + trig = GIC_TRIG_LEVEL; + dual = GIC_DUAL_SINGLE; break; } - if (is_edge) + change_gic_pol(irq, pol); + change_gic_trig(irq, trig); + change_gic_dual(irq, dual); + + if (trig == GIC_TRIG_EDGE) irq_set_chip_handler_name_locked(d, &gic_edge_irq_controller, handle_edge_irq, NULL); else -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [PATCH 8/8] irqchip: mips-gic: Make IPI bitmaps static 2017-10-25 23:37 [PATCH 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton ` (6 preceding siblings ...) 2017-10-25 23:37 ` [PATCH 7/8] irqchip: mips-gic: Share register writes in gic_set_type() Paul Burton @ 2017-10-25 23:37 ` Paul Burton 7 siblings, 0 replies; 25+ messages in thread From: Paul Burton @ 2017-10-25 23:37 UTC (permalink / raw) To: Jason Cooper, Marc Zyngier, Thomas Gleixner Cc: linux-mips, linux-kernel, Paul Burton We have 2 bitmaps used to keep track of interrupts dedicated to IPIs in the MIPS GIC irqchip driver. These bitmaps are only used from the one compilation unit of that driver, and so can be made static. Do so in order to avoid polluting the symbol table & global namespace. Signed-off-by: Paul Burton <paul.burton@mips.com> Cc: Jason Cooper <jason@lakedaemon.net> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mips@linux-mips.org --- drivers/irqchip/irq-mips-gic.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/irqchip/irq-mips-gic.c b/drivers/irqchip/irq-mips-gic.c index 8c719a9912fc..108ffdc4f804 100644 --- a/drivers/irqchip/irq-mips-gic.c +++ b/drivers/irqchip/irq-mips-gic.c @@ -52,8 +52,8 @@ static int gic_shared_intrs; static unsigned int gic_cpu_pin; static unsigned int timer_cpu_pin; static struct irq_chip gic_level_irq_controller, gic_edge_irq_controller; -DECLARE_BITMAP(ipi_resrv, GIC_MAX_INTRS); -DECLARE_BITMAP(ipi_available, GIC_MAX_INTRS); +static DECLARE_BITMAP(ipi_resrv, GIC_MAX_INTRS); +static DECLARE_BITMAP(ipi_available, GIC_MAX_INTRS); static struct gic_all_vpes_chip_data { u32 map; -- 2.14.3 ^ permalink raw reply related [flat|nested] 25+ messages in thread
end of thread, other threads:[~2017-11-02 10:44 UTC | newest] Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-10-25 23:37 [PATCH 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton 2017-10-25 23:37 ` [PATCH 1/8] irqchip: mips-gic: Inline gic_local_irq_domain_map() Paul Burton 2017-10-25 23:37 ` [PATCH 2/8] irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs Paul Burton 2017-10-30 8:00 ` Marc Zyngier 2017-10-30 16:36 ` Paul Burton 2017-10-31 1:35 ` Marc Zyngier 2017-10-31 16:41 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Paul Burton 2017-10-31 16:41 ` [PATCH v2 1/8] irqchip: mips-gic: Inline gic_local_irq_domain_map() Paul Burton 2017-10-31 16:41 ` [PATCH v2 2/8] irqchip: mips-gic: Use irq_cpu_online to (un)mask all-VP(E) IRQs Paul Burton 2017-10-31 16:41 ` [PATCH v2 3/8] irqchip: mips-gic: Mask local interrupts when CPUs come online Paul Burton 2017-10-31 16:41 ` [PATCH v2 4/8] irqchip: mips-gic: Configure EIC " Paul Burton 2017-10-31 16:41 ` [PATCH v2 5/8] irqchip: mips-gic: Use num_possible_cpus() to reserve IPIs Paul Burton 2017-10-31 16:41 ` [PATCH v2 6/8] irqchip: mips-gic: Remove gic_vpes variable Paul Burton 2017-10-31 16:41 ` [PATCH v2 7/8] irqchip: mips-gic: Share register writes in gic_set_type() Paul Burton 2017-10-31 16:41 ` [PATCH v2 8/8] irqchip: mips-gic: Make IPI bitmaps static Paul Burton 2017-11-01 0:13 ` [PATCH v2 0/8] irqchip: mips-gic: Cleanups, fixes, prep for multi-cluster Marc Zyngier 2017-11-01 16:40 ` Paul Burton 2017-11-01 16:59 ` Thomas Gleixner 2017-11-02 10:44 ` Marc Zyngier 2017-10-25 23:37 ` [PATCH 3/8] irqchip: mips-gic: Mask local interrupts when CPUs come online Paul Burton 2017-10-25 23:37 ` [PATCH 4/8] irqchip: mips-gic: Configure EIC " Paul Burton 2017-10-25 23:37 ` [PATCH 5/8] irqchip: mips-gic: Use num_possible_cpus() to reserve IPIs Paul Burton 2017-10-25 23:37 ` [PATCH 6/8] irqchip: mips-gic: Remove gic_vpes variable Paul Burton 2017-10-25 23:37 ` [PATCH 7/8] irqchip: mips-gic: Share register writes in gic_set_type() Paul Burton 2017-10-25 23:37 ` [PATCH 8/8] irqchip: mips-gic: Make IPI bitmaps static Paul Burton
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).