[PATCH RFC] ARM: omap2: fix lack of timer interrupts on CPU1 after hotplug

[PATCH RFC] ARM: omap2: fix lack of timer interrupts on CPU1 after hotplug

[Russell King]

If we have a kernel configured for periodic timer interrupts, and we
have cpuidle enabled, then we end up with CPU1 losing timer interupts
after a hotplug.

This can manifest itself in RCU stall warnings, or userspace becoming
unresponsive.

The problem is that the kernel initially wants to use the TWD timer
for interrupts, but the TWD loses context when we enter the C3 cpuidle
state.  Nothing reprograms the TWD after idle.

We have solved this in the past by switching to broadcast timer ticks,
and cpuidle44xx switches to that mode at boot time.  However, there is
nothing to switch from periodic mode local timers after a hotplug
operation.

We call tick_broadcast_enter() in omap_enter_idle_coupled(), which one
would expect would take care of the issue, but internally this only
deals with one-shot local timers - tick_broadcast_enable() on the other
hand only deals with periodic local timers.  So, we need to call both.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
---
 arch/arm/mach-omap2/cpuidle44xx.c | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)

diff --git a/arch/arm/mach-omap2/cpuidle44xx.c b/arch/arm/mach-omap2/cpuidle44xx.c
index a8b291f00109..dae514c8276a 100644
--- a/arch/arm/mach-omap2/cpuidle44xx.c
+++ b/arch/arm/mach-omap2/cpuidle44xx.c
@@ -152,6 +152,10 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,
 	mpuss_can_lose_context = (cx->mpu_state == PWRDM_POWER_RET) &&
 				 (cx->mpu_logic_state == PWRDM_POWER_OFF);
 
+	/* Enter broadcast mode for periodic timers */
+	tick_broadcast_enable();
+
+	/* Enter broadcast mode for one-shot timers */
 	tick_broadcast_enter();
 
 	/*
@@ -218,15 +222,6 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,
 	return index;
 }
 
-/*
- * For each cpu, setup the broadcast timer because local timers
- * stops for the states above C1.
- */
-static void omap_setup_broadcast_timer(void *arg)
-{
-	tick_broadcast_enable();
-}
-
 static struct cpuidle_driver omap4_idle_driver = {
 	.name				= "omap4_idle",
 	.owner				= THIS_MODULE,
@@ -319,8 +314,5 @@ int __init omap4_idle_init(void)
 	if (!cpu_clkdm[0] || !cpu_clkdm[1])
 		return -ENODEV;
 
-	/* Configure the broadcast timer on each cpu */
-	on_each_cpu(omap_setup_broadcast_timer, NULL, 1);
-
 	return cpuidle_register(idle_driver, cpu_online_mask);
 }
-- 
2.7.4

[PATCH RFC] ARM: omap2: fix lack of timer interrupts on CPU1 after hotplug

[Russell King]

If we have a kernel configured for periodic timer interrupts, and we
have cpuidle enabled, then we end up with CPU1 losing timer interupts
after a hotplug.

This can manifest itself in RCU stall warnings, or userspace becoming
unresponsive.

The problem is that the kernel initially wants to use the TWD timer
for interrupts, but the TWD loses context when we enter the C3 cpuidle
state.  Nothing reprograms the TWD after idle.

We have solved this in the past by switching to broadcast timer ticks,
and cpuidle44xx switches to that mode at boot time.  However, there is
nothing to switch from periodic mode local timers after a hotplug
operation.

We call tick_broadcast_enter() in omap_enter_idle_coupled(), which one
would expect would take care of the issue, but internally this only
deals with one-shot local timers - tick_broadcast_enable() on the other
hand only deals with periodic local timers.  So, we need to call both.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
---
 arch/arm/mach-omap2/cpuidle44xx.c | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)

diff --git a/arch/arm/mach-omap2/cpuidle44xx.c b/arch/arm/mach-omap2/cpuidle44xx.c
index a8b291f00109..dae514c8276a 100644
--- a/arch/arm/mach-omap2/cpuidle44xx.c
+++ b/arch/arm/mach-omap2/cpuidle44xx.c
@@ -152,6 +152,10 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,
 	mpuss_can_lose_context = (cx->mpu_state == PWRDM_POWER_RET) &&
 				 (cx->mpu_logic_state == PWRDM_POWER_OFF);
 
+	/* Enter broadcast mode for periodic timers */
+	tick_broadcast_enable();
+
+	/* Enter broadcast mode for one-shot timers */
 	tick_broadcast_enter();
 
 	/*
@@ -218,15 +222,6 @@ static int omap_enter_idle_coupled(struct cpuidle_device *dev,
 	return index;
 }
 
-/*
- * For each cpu, setup the broadcast timer because local timers
- * stops for the states above C1.
- */
-static void omap_setup_broadcast_timer(void *arg)
-{
-	tick_broadcast_enable();
-}
-
 static struct cpuidle_driver omap4_idle_driver = {
 	.name				= "omap4_idle",
 	.owner				= THIS_MODULE,
@@ -319,8 +314,5 @@ int __init omap4_idle_init(void)
 	if (!cpu_clkdm[0] || !cpu_clkdm[1])
 		return -ENODEV;
 
-	/* Configure the broadcast timer on each cpu */
-	on_each_cpu(omap_setup_broadcast_timer, NULL, 1);
-
 	return cpuidle_register(idle_driver, cpu_online_mask);
 }
-- 
2.7.4


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC] ARM: omap2: fix lack of timer interrupts on CPU1 after hotplug

[Tony Lindgren]

* Russell King <rmk+kernel@armlinux.org.uk> [181212 11:49]:
> We call tick_broadcast_enter() in omap_enter_idle_coupled(), which one
> would expect would take care of the issue, but internally this only
> deals with one-shot local timers - tick_broadcast_enable() on the other
> hand only deals with periodic local timers.  So, we need to call both.

This works just fine for me for NO_HZ and cpuidle:

Tested-by: Tony Lindgren <tony@atomide.com>

Re: [PATCH RFC] ARM: omap2: fix lack of timer interrupts on CPU1 after hotplug

[Tony Lindgren]

* Russell King <rmk+kernel@armlinux.org.uk> [181212 11:49]:
> We call tick_broadcast_enter() in omap_enter_idle_coupled(), which one
> would expect would take care of the issue, but internally this only
> deals with one-shot local timers - tick_broadcast_enable() on the other
> hand only deals with periodic local timers.  So, we need to call both.

This works just fine for me for NO_HZ and cpuidle:

Tested-by: Tony Lindgren <tony@atomide.com>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC] ARM: omap2: fix lack of timer interrupts on CPU1 after hotplug

[Tony Lindgren]

* Tony Lindgren <tony@atomide.com> [181212 17:53]:
> * Russell King <rmk+kernel@armlinux.org.uk> [181212 11:49]:
> > We call tick_broadcast_enter() in omap_enter_idle_coupled(), which one
> > would expect would take care of the issue, but internally this only
> > deals with one-shot local timers - tick_broadcast_enable() on the other
> > hand only deals with periodic local timers.  So, we need to call both.
> 
> This works just fine for me for NO_HZ and cpuidle:
>
> Tested-by: Tony Lindgren <tony@atomide.com>

And also works for me without NO_HZ too FYI.

Tony

Re: [PATCH RFC] ARM: omap2: fix lack of timer interrupts on CPU1 after hotplug

[Tony Lindgren]

* Tony Lindgren <tony@atomide.com> [181212 17:53]:
> * Russell King <rmk+kernel@armlinux.org.uk> [181212 11:49]:
> > We call tick_broadcast_enter() in omap_enter_idle_coupled(), which one
> > would expect would take care of the issue, but internally this only
> > deals with one-shot local timers - tick_broadcast_enable() on the other
> > hand only deals with periodic local timers.  So, we need to call both.
> 
> This works just fine for me for NO_HZ and cpuidle:
>
> Tested-by: Tony Lindgren <tony@atomide.com>

And also works for me without NO_HZ too FYI.

Tony

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC] ARM: omap2: fix lack of timer interrupts on CPU1 after hotplug

[Russell King - ARM Linux admin]

On Wed, Dec 12, 2018 at 10:57:41AM -0800, Tony Lindgren wrote:
> * Tony Lindgren <tony@atomide.com> [181212 17:53]:
> > * Russell King <rmk+kernel@armlinux.org.uk> [181212 11:49]:
> > > We call tick_broadcast_enter() in omap_enter_idle_coupled(), which one
> > > would expect would take care of the issue, but internally this only
> > > deals with one-shot local timers - tick_broadcast_enable() on the other
> > > hand only deals with periodic local timers.  So, we need to call both.
> > 
> > This works just fine for me for NO_HZ and cpuidle:
> >
> > Tested-by: Tony Lindgren <tony@atomide.com>
> 
> And also works for me without NO_HZ too FYI.

Hi Tony,

Could you pick this patch up and submit for 5.0-rc please?

Thanks.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

Re: [PATCH RFC] ARM: omap2: fix lack of timer interrupts on CPU1 after hotplug

[Russell King - ARM Linux admin]

On Wed, Dec 12, 2018 at 10:57:41AM -0800, Tony Lindgren wrote:
> * Tony Lindgren <tony@atomide.com> [181212 17:53]:
> > * Russell King <rmk+kernel@armlinux.org.uk> [181212 11:49]:
> > > We call tick_broadcast_enter() in omap_enter_idle_coupled(), which one
> > > would expect would take care of the issue, but internally this only
> > > deals with one-shot local timers - tick_broadcast_enable() on the other
> > > hand only deals with periodic local timers.  So, we need to call both.
> > 
> > This works just fine for me for NO_HZ and cpuidle:
> >
> > Tested-by: Tony Lindgren <tony@atomide.com>
> 
> And also works for me without NO_HZ too FYI.

Hi Tony,

Could you pick this patch up and submit for 5.0-rc please?

Thanks.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC] ARM: omap2: fix lack of timer interrupts on CPU1 after hotplug

[Tony Lindgren]

* Russell King - ARM Linux admin <linux@armlinux.org.uk> [190207 13:46]:
> On Wed, Dec 12, 2018 at 10:57:41AM -0800, Tony Lindgren wrote:
> > * Tony Lindgren <tony@atomide.com> [181212 17:53]:
> > > * Russell King <rmk+kernel@armlinux.org.uk> [181212 11:49]:
> > > > We call tick_broadcast_enter() in omap_enter_idle_coupled(), which one
> > > > would expect would take care of the issue, but internally this only
> > > > deals with one-shot local timers - tick_broadcast_enable() on the other
> > > > hand only deals with periodic local timers.  So, we need to call both.
> > > 
> > > This works just fine for me for NO_HZ and cpuidle:
> > >
> > > Tested-by: Tony Lindgren <tony@atomide.com>
> > 
> > And also works for me without NO_HZ too FYI.
> 
> Hi Tony,
> 
> Could you pick this patch up and submit for 5.0-rc please?

Oh sorry. I thought you had already merged it since I added
my Tested-by. Hmm I guess I need to start adding also an
Acked-by to signal it's OK to pick some misc patches.

Anyways, applying into omap-for-v5.0/fixes-v2.

Thanks,

Tony

Re: [PATCH RFC] ARM: omap2: fix lack of timer interrupts on CPU1 after hotplug

[Tony Lindgren]

* Russell King - ARM Linux admin <linux@armlinux.org.uk> [190207 13:46]:
> On Wed, Dec 12, 2018 at 10:57:41AM -0800, Tony Lindgren wrote:
> > * Tony Lindgren <tony@atomide.com> [181212 17:53]:
> > > * Russell King <rmk+kernel@armlinux.org.uk> [181212 11:49]:
> > > > We call tick_broadcast_enter() in omap_enter_idle_coupled(), which one
> > > > would expect would take care of the issue, but internally this only
> > > > deals with one-shot local timers - tick_broadcast_enable() on the other
> > > > hand only deals with periodic local timers.  So, we need to call both.
> > > 
> > > This works just fine for me for NO_HZ and cpuidle:
> > >
> > > Tested-by: Tony Lindgren <tony@atomide.com>
> > 
> > And also works for me without NO_HZ too FYI.
> 
> Hi Tony,
> 
> Could you pick this patch up and submit for 5.0-rc please?

Oh sorry. I thought you had already merged it since I added
my Tested-by. Hmm I guess I need to start adding also an
Acked-by to signal it's OK to pick some misc patches.

Anyways, applying into omap-for-v5.0/fixes-v2.

Thanks,

Tony

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel