[PATCH] powerpc: re-enable dynticks

[PATCH] powerpc: re-enable dynticks

[Paul Clarke]

implement arch_irq_work_has_interrupt() for powerpc

Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for 
full dynamic ticks to be enabled, by expecting architectures to 
implement a suitable arch_irq_work_has_interrupt() routine.

Several arches have implemented this routine, including x86 (3010279f) 
and arm (09f6edd4), but powerpc was omitted.

This patch implements this routine for powerpc.

The symptom, at boot (on powerpc arch systems) with "nohz_full=<CPU 
list>" is displayed:
     NO_HZ: Can't run full dynticks because arch doesn't support irq 
work self-IPIs

after this patch:
     NO_HZ: Full dynticks CPUs: <CPU list>.

Tested against 3.19.

CC: Frederic Weisbecker <fweisbec@gmail.com>
CC: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Paul A. Clarke <pc@us.ibm.com>

diff --git a/arch/powerpc/include/asm/irq_work.h 
b/arch/powerpc/include/asm/irq_work.h
new file mode 100644
index 0000000..18365ec
--- /dev/null
+++ b/arch/powerpc/include/asm/irq_work.h
@@ -0,0 +1,11 @@
+#ifndef _ASM_IRQ_WORK_H
+#define _ASM_IRQ_WORK_H
+
+#include <asm/processor.h>
+
+static inline bool arch_irq_work_has_interrupt(void)
+{
+	return 1;
+}
+
+#endif /* _ASM_IRQ_WORK_H */

--
Regards,
Paul Clarke, IBM

Re: [PATCH] powerpc: re-enable dynticks

[Paul E. McKenney]

On Fri, Feb 13, 2015 at 01:38:36PM -0600, Paul Clarke wrote:
> implement arch_irq_work_has_interrupt() for powerpc
> 
> Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for
> full dynamic ticks to be enabled, by expecting architectures to
> implement a suitable arch_irq_work_has_interrupt() routine.
> 
> Several arches have implemented this routine, including x86
> (3010279f) and arm (09f6edd4), but powerpc was omitted.
> 
> This patch implements this routine for powerpc.
> 
> The symptom, at boot (on powerpc arch systems) with "nohz_full=<CPU
> list>" is displayed:
>     NO_HZ: Can't run full dynticks because arch doesn't support irq
> work self-IPIs
> 
> after this patch:
>     NO_HZ: Full dynticks CPUs: <CPU list>.
> 
> Tested against 3.19.
> 
> CC: Frederic Weisbecker <fweisbec@gmail.com>
> CC: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Signed-off-by: Paul A. Clarke <pc@us.ibm.com>

Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

> diff --git a/arch/powerpc/include/asm/irq_work.h
> b/arch/powerpc/include/asm/irq_work.h
> new file mode 100644
> index 0000000..18365ec
> --- /dev/null
> +++ b/arch/powerpc/include/asm/irq_work.h
> @@ -0,0 +1,11 @@
> +#ifndef _ASM_IRQ_WORK_H
> +#define _ASM_IRQ_WORK_H
> +
> +#include <asm/processor.h>
> +
> +static inline bool arch_irq_work_has_interrupt(void)
> +{
> +	return 1;
> +}
> +
> +#endif /* _ASM_IRQ_WORK_H */
> 
> --
> Regards,
> Paul Clarke, IBM
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH] powerpc: re-enable dynticks

[Michael Ellerman]

On Fri, 2015-02-13 at 13:38 -0600, Paul Clarke wrote:
> implement arch_irq_work_has_interrupt() for powerpc
> 
> Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for 
> full dynamic ticks to be enabled, by expecting architectures to 
> implement a suitable arch_irq_work_has_interrupt() routine.
> 
> Several arches have implemented this routine, including x86 (3010279f) 
> and arm (09f6edd4), but powerpc was omitted.
> 
> This patch implements this routine for powerpc.
> 
> The symptom, at boot (on powerpc arch systems) with "nohz_full=<CPU 
> list>" is displayed:
>      NO_HZ: Can't run full dynticks because arch doesn't support irq 
> work self-IPIs
> 
> after this patch:
>      NO_HZ: Full dynticks CPUs: <CPU list>.
> 
> Tested against 3.19.

It makes the message change, but is that correct? ie. do we actually implement
"IRQ work self-IPIs"?

> diff --git a/arch/powerpc/include/asm/irq_work.h 
> b/arch/powerpc/include/asm/irq_work.h
> new file mode 100644
> index 0000000..18365ec
> --- /dev/null
> +++ b/arch/powerpc/include/asm/irq_work.h
> @@ -0,0 +1,11 @@
> +#ifndef _ASM_IRQ_WORK_H
> +#define _ASM_IRQ_WORK_H
> +
> +#include <asm/processor.h>
> +
> +static inline bool arch_irq_work_has_interrupt(void)
> +{
> +	return 1;

Should be "true";

> +}

cheers

Re: [PATCH] powerpc: re-enable dynticks

[Benjamin Herrenschmidt]

On Mon, 2015-02-16 at 11:08 +1100, Michael Ellerman wrote:
> On Fri, 2015-02-13 at 13:38 -0600, Paul Clarke wrote:
> > implement arch_irq_work_has_interrupt() for powerpc
> > 
> > Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for 
> > full dynamic ticks to be enabled, by expecting architectures to 
> > implement a suitable arch_irq_work_has_interrupt() routine.
> > 
> > Several arches have implemented this routine, including x86 (3010279f) 
> > and arm (09f6edd4), but powerpc was omitted.
> > 
> > This patch implements this routine for powerpc.
> > 
 .../...
> 
> It makes the message change, but is that correct? ie. do we actually implement
> "IRQ work self-IPIs"?

I think so... Fred, do you think what we do will work ? We hijack our
decrementer (local timer) by making it shoot almost immediately (1 tick
away) and run the irq work at the beginning of __timer_interrupt().

At that point we are on our irq stack and have done irq_enter but that's
about it.

Cheers,
Ben.

> > diff --git a/arch/powerpc/include/asm/irq_work.h 
> > b/arch/powerpc/include/asm/irq_work.h
> > new file mode 100644
> > index 0000000..18365ec
> > --- /dev/null
> > +++ b/arch/powerpc/include/asm/irq_work.h
> > @@ -0,0 +1,11 @@
> > +#ifndef _ASM_IRQ_WORK_H
> > +#define _ASM_IRQ_WORK_H
> > +
> > +#include <asm/processor.h>
> > +
> > +static inline bool arch_irq_work_has_interrupt(void)
> > +{
> > +	return 1;
> 
> Should be "true";
> 
> > +}
> 
> cheers
> 
> 
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/linuxppc-dev

[PATCH v2] powerpc: re-enable dynticks

[Paul Clarke]

implement arch_irq_work_has_interrupt() for powerpc

Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for 
full dynamic ticks to be enabled, by expecting architectures to 
implement a suitable arch_irq_work_has_interrupt() routine.

Several arches have implemented this routine, including x86 (3010279f) 
and arm (09f6edd4), but powerpc was omitted.

This patch implements this routine for powerpc.

The symptom, at boot (on powerpc arch systems) with "nohz_full=<CPU 
list>" is displayed:
     NO_HZ: Can't run full dynticks because arch doesn't support irq 
work self-IPIs

after this patch:
     NO_HZ: Full dynticks CPUs: <CPU list>.

Tested against 3.19.

v2: changed "return 1" to "return true", per Michael Ellerman

CC: Frederic Weisbecker <fweisbec@gmail.com>
CC: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Paul A. Clarke <pc@us.ibm.com>

diff --git a/arch/powerpc/include/asm/irq_work.h 
b/arch/powerpc/include/asm/irq_work.h
new file mode 100644
index 0000000..99cc0aa
--- /dev/null
+++ b/arch/powerpc/include/asm/irq_work.h
@@ -0,0 +1,11 @@
+#ifndef _ASM_IRQ_WORK_H
+#define _ASM_IRQ_WORK_H
+
+#include <asm/processor.h>
+
+static inline bool arch_irq_work_has_interrupt(void)
+{
+	return true;
+}
+
+#endif /* _ASM_IRQ_WORK_H */

[PATCH v2] powerpc: re-enable dynticks

[Paul Clarke]

implement arch_irq_work_has_interrupt() for powerpc

(resending because I messed up the e-mail addresses)

Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for 
full dynamic ticks to be enabled, by expecting architectures to 
implement a suitable arch_irq_work_has_interrupt() routine.

Several arches have implemented this routine, including x86 (3010279f) 
and arm (09f6edd4), but powerpc was omitted.

This patch implements this routine for powerpc.

The symptom, at boot (on powerpc arch systems) with "nohz_full=<CPU 
list>" is displayed:
     NO_HZ: Can't run full dynticks because arch doesn't support irq 
work self-IPIs

after this patch:
     NO_HZ: Full dynticks CPUs: <CPU list>.

Tested against 3.19.

v2: changed "return 1" to "return true", per Michael Ellerman

CC: Frederic Weisbecker <fweisbec@gmail.com>
CC: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Paul A. Clarke <pc@us.ibm.com>

diff --git a/arch/powerpc/include/asm/irq_work.h 
b/arch/powerpc/include/asm/irq_work.h
new file mode 100644
index 0000000..99cc0aa
--- /dev/null
+++ b/arch/powerpc/include/asm/irq_work.h
@@ -0,0 +1,11 @@
+#ifndef _ASM_IRQ_WORK_H
+#define _ASM_IRQ_WORK_H
+
+#include <asm/processor.h>
+
+static inline bool arch_irq_work_has_interrupt(void)
+{
+    return true;
+}
+
+#endif /* _ASM_IRQ_WORK_H */

Re: [PATCH v2] powerpc: re-enable dynticks

[Paul E. McKenney]

On Fri, Feb 20, 2015 at 11:13:33AM -0600, Paul Clarke wrote:
> 
> implement arch_irq_work_has_interrupt() for powerpc
> 
> (resending because I messed up the e-mail addresses)
> 
> Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for
> full dynamic ticks to be enabled, by expecting architectures to
> implement a suitable arch_irq_work_has_interrupt() routine.
> 
> Several arches have implemented this routine, including x86
> (3010279f) and arm (09f6edd4), but powerpc was omitted.
> 
> This patch implements this routine for powerpc.
> 
> The symptom, at boot (on powerpc arch systems) with "nohz_full=<CPU
> list>" is displayed:
>     NO_HZ: Can't run full dynticks because arch doesn't support irq
> work self-IPIs
> 
> after this patch:
>     NO_HZ: Full dynticks CPUs: <CPU list>.
> 
> Tested against 3.19.
> 
> v2: changed "return 1" to "return true", per Michael Ellerman
> 
> CC: Frederic Weisbecker <fweisbec@gmail.com>
> CC: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Signed-off-by: Paul A. Clarke <pc@us.ibm.com>

Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

> diff --git a/arch/powerpc/include/asm/irq_work.h
> b/arch/powerpc/include/asm/irq_work.h
> new file mode 100644
> index 0000000..99cc0aa
> --- /dev/null
> +++ b/arch/powerpc/include/asm/irq_work.h
> @@ -0,0 +1,11 @@
> +#ifndef _ASM_IRQ_WORK_H
> +#define _ASM_IRQ_WORK_H
> +
> +#include <asm/processor.h>
> +
> +static inline bool arch_irq_work_has_interrupt(void)
> +{
> +    return true;
> +}
> +
> +#endif /* _ASM_IRQ_WORK_H */

Re: [PATCH] powerpc: re-enable dynticks

[Frederic Weisbecker]

Hi Ben,

2015-02-16 5:06 GMT+01:00 Benjamin Herrenschmidt <benh@kernel.crashing.org>:
> On Mon, 2015-02-16 at 11:08 +1100, Michael Ellerman wrote:
>> On Fri, 2015-02-13 at 13:38 -0600, Paul Clarke wrote:
>> > implement arch_irq_work_has_interrupt() for powerpc
>> >
>> > Commit 9b01f5bf3 introduced a dependency on "IRQ work self-IPIs" for
>> > full dynamic ticks to be enabled, by expecting architectures to
>> > implement a suitable arch_irq_work_has_interrupt() routine.
>> >
>> > Several arches have implemented this routine, including x86 (3010279f)
>> > and arm (09f6edd4), but powerpc was omitted.
>> >
>> > This patch implements this routine for powerpc.
>> >
>  .../...
>>
>> It makes the message change, but is that correct? ie. do we actually implement
>> "IRQ work self-IPIs"?
>
> I think so... Fred, do you think what we do will work ? We hijack our
> decrementer (local timer) by making it shoot almost immediately (1 tick
> away) and run the irq work at the beginning of __timer_interrupt().
>
> At that point we are on our irq stack and have done irq_enter but that's
> about it.

Yes that should work. After all "self-IPI" is an oxymoron. One would
expect an IPI to be triggered by an irq controller but if such
operation isn't supported with the current CPU being both source and
destination, anything triggering the desired callback in an interrupt
context in a reasonable amount of time ahead does the job here.

I thought well that's what powerpc was doing for irq work but I wasn't
sure I understood the code correctly. I should have pinged people
about that, sorry.

Thanks.

>
> Cheers,
> Ben.
>
>> > diff --git a/arch/powerpc/include/asm/irq_work.h
>> > b/arch/powerpc/include/asm/irq_work.h
>> > new file mode 100644
>> > index 0000000..18365ec
>> > --- /dev/null
>> > +++ b/arch/powerpc/include/asm/irq_work.h
>> > @@ -0,0 +1,11 @@
>> > +#ifndef _ASM_IRQ_WORK_H
>> > +#define _ASM_IRQ_WORK_H
>> > +
>> > +#include <asm/processor.h>
>> > +
>> > +static inline bool arch_irq_work_has_interrupt(void)
>> > +{
>> > +   return 1;
>>
>> Should be "true";
>>
>> > +}
>>
>> cheers
>>
>>
>> _______________________________________________
>> Linuxppc-dev mailing list
>> Linuxppc-dev@lists.ozlabs.org
>> https://lists.ozlabs.org/listinfo/linuxppc-dev
>
>

Re: [PATCH] powerpc: re-enable dynticks

[Benjamin Herrenschmidt]

On Sun, 2015-02-22 at 23:13 +0100, Frederic Weisbecker wrote:
> Yes that should work. After all "self-IPI" is an oxymoron. One would
> expect an IPI to be triggered by an irq controller but if such
> operation isn't supported with the current CPU being both source and
> destination, anything triggering the desired callback in an interrupt
> context in a reasonable amount of time ahead does the job here.

We could do self-IPI on platforms that have an SMP-capable interrupt
controller too but it would probably have higher overhead and would
require verifying that the code for each of our different interrupt
controllers is safe to be called from NMIs (hint: ioremap space isn't
safe to access from NMIs for us on some CPU families...).

We might be able to do better than using the decrementer on some CPUs by
using local doorbells, but for now this will do.

> I thought well that's what powerpc was doing for irq work but I wasn't
> sure I understood the code correctly. I should have pinged people
> about that, sorry.

No worries,

Cheers,
Ben.