[PATCH] parisc: Improve initial IRQ to CPU assignment

[PATCH] parisc: Improve initial IRQ to CPU assignment

[Helge Deller]

On parisc, each IRQ can only be handled by one CPU, and currently CPU0
is choosen as default for handling all IRQs by default.
With this patch we now assign each requested IRQ to one of the online
CPUs (and thus distribute the IRQs across all CPUs), even without an
instance of irqbalance running.

Signed-off-by: Helge Deller <deller@gmx.de>

diff --git a/arch/parisc/kernel/irq.c b/arch/parisc/kernel/irq.c
index fd6d873..04e8755 100644
--- a/arch/parisc/kernel/irq.c
+++ b/arch/parisc/kernel/irq.c
@@ -117,7 +117,10 @@ int cpu_check_affinity(struct irq_data *d, const struct cpumask *dest)
 		return -EINVAL;
 
 	/* whatever mask they set, we just allow one CPU */
-	cpu_dest = cpumask_first_and(dest, cpu_online_mask);
+	cpu_dest = cpumask_next_and(d->irq & (num_online_cpus()-1),
+					dest, cpu_online_mask);
+	if (cpu_dest >= nr_cpu_ids)
+		cpu_dest = cpumask_first_and(dest, cpu_online_mask);
 
 	return cpu_dest;
 }

Re: [PATCH] parisc: Improve initial IRQ to CPU assignment

[Dennis Clarke]

On 1/4/19 6:05 PM, Helge Deller wrote:
> On parisc, each IRQ can only be handled by one CPU, and currently CPU0
> is choosen as default for handling all IRQs by default.
> With this patch we now assign each requested IRQ to one of the online
> CPUs (and thus distribute the IRQs across all CPUs), even without an
> instance of irqbalance running.

Will this take into consideration the features in default_smp_affinity
and allow processor cores to be considered 'non interrupt' service
state?

Dennis

Re: [PATCH] parisc: Improve initial IRQ to CPU assignment

[Helge Deller]

On 05.01.19 00:21, Dennis Clarke wrote:
> On 1/4/19 6:05 PM, Helge Deller wrote:
>> On parisc, each IRQ can only be handled by one CPU, and currently CPU0
>> is choosen as default for handling all IRQs by default.
>> With this patch we now assign each requested IRQ to one of the online
>> CPUs (and thus distribute the IRQs across all CPUs), even without an
>> instance of irqbalance running.
> 
> Will this take into consideration the features in default_smp_affinity
> and allow processor cores to be considered 'non interrupt' service
> state?

It seems the feature of default_smp_affinity to disable specific cores from
servicing interrupts doesn't work on the parisc platform.
On a 2-CPU machine I booted with the "irqaffinity=0" kernel commandline option
(to enable CPU#0 and disable CPU#1), which then led to a hanging kernel when
CPU#1 ist started.
On parisc, each CPU needs to handle at least it's timer and IPMI interrupt.
Any idea what I should try?

Helge

Re: [PATCH] parisc: Improve initial IRQ to CPU assignment

[Dennis Clarke]

On 1/5/19 4:21 PM, Helge Deller wrote:
> On 05.01.19 00:21, Dennis Clarke wrote:
>> On 1/4/19 6:05 PM, Helge Deller wrote:
>>> On parisc, each IRQ can only be handled by one CPU, and currently CPU0
>>> is choosen as default for handling all IRQs by default.
>>> With this patch we now assign each requested IRQ to one of the online
>>> CPUs (and thus distribute the IRQs across all CPUs), even without an
>>> instance of irqbalance running.
>>
>> Will this take into consideration the features in default_smp_affinity
>> and allow processor cores to be considered 'non interrupt' service
>> state?
> 
> It seems the feature of default_smp_affinity to disable specific cores from
> servicing interrupts doesn't work on the parisc platform.
> On a 2-CPU machine I booted with the "irqaffinity=0" kernel commandline option
> (to enable CPU#0 and disable CPU#1), which then led to a hanging kernel when
> CPU#1 ist started.
> On parisc, each CPU needs to handle at least it's timer and IPMI interrupt.
> Any idea what I should try?

Sorry, the first question was "does this work there" and we now know it
is most likely "no". I tried similar on ppc64 and also arrived at
nothing but problems.

I am somewhat focused on digging into RISC-V at the moment and your post
raised the question in my mind.

One of the features I once enjoyed about Solaris was that it could
easily bring cpu cores online and offline as well as disable irq service
disruptions on any core with a trivial one line command. I always wanted
to port something similar over to linux but would start with something
interesting like RISC-V.

I'll get back to you on this one.

Dennis Clarke

ps: Also I still have those black superdomes standing around doing
     nothing and still want to get serial attached, powered up, etc.
     Yet another news years resolution post-it note idea.

Re: [PATCH] parisc: Improve initial IRQ to CPU assignment

[Helge Deller]

On 05.01.19 22:45, Dennis Clarke wrote:
> On 1/5/19 4:21 PM, Helge Deller wrote:
>> On 05.01.19 00:21, Dennis Clarke wrote:
>>> On 1/4/19 6:05 PM, Helge Deller wrote:
>>>> On parisc, each IRQ can only be handled by one CPU, and currently CPU0
>>>> is choosen as default for handling all IRQs by default.
>>>> With this patch we now assign each requested IRQ to one of the online
>>>> CPUs (and thus distribute the IRQs across all CPUs), even without an
>>>> instance of irqbalance running.
>>>
>>> Will this take into consideration the features in default_smp_affinity
>>> and allow processor cores to be considered 'non interrupt' service
>>> state?
>>
>> It seems the feature of default_smp_affinity to disable specific cores from
>> servicing interrupts doesn't work on the parisc platform.
>> On a 2-CPU machine I booted with the "irqaffinity=0" kernel commandline option
>> (to enable CPU#0 and disable CPU#1), which then led to a hanging kernel when
>> CPU#1 ist started.
>> On parisc, each CPU needs to handle at least it's timer and IPMI interrupt.
>> Any idea what I should try?
> 
> Sorry, the first question was "does this work there" and we now know it
> is most likely "no". I tried similar on ppc64 and also arrived at
> nothing but problems.
> 
> I am somewhat focused on digging into RISC-V at the moment and your post
> raised the question in my mind.
> 
> One of the features I once enjoyed about Solaris was that it could
> easily bring cpu cores online and offline as well as disable irq service
> disruptions on any core with a trivial one line command.

That's already possible on Linux (e.g. on x86,power,...).
 > I always wanted to port something similar over to linux but would
> start with something interesting like RISC-V.

There shouldn't be much to port.
Even on parisc, enabling and disabling irqs on specific CPUs is possible
today with simple echo-commands into the files in /proc/irq/#/... (with the
exception of timer and IPI irqs).
Onlining and offlining cpu cores is on my todo list for parisc.
So, I think it shouldn't take much to do that on RISC-V too.

> ps: Also I still have those black superdomes standing around doing
>     nothing and still want to get serial attached, powered up, etc.
>     Yet another news years resolution post-it note idea.

Yes, would be interesting.

Helge