Re: [patch 1/2] tick/broadcast: Prevent deep idle states if no broadcast device available

From: Thomas Gleixner <tglx@linutronix.de>
To: Sudeep Holla <sudeep.holla@arm.com>
Cc: LKML <linux-kernel@vger.kernel.org>,
	Ingo Molnar <mingo@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Preeti U Murthy <preeti@linux.vnet.ibm.com>,
	Suzuki Poulose <Suzuki.Poulose@arm.com>,
	Lorenzo Pieralisi <Lorenzo.Pieralisi@arm.com>,
	Catalin Marinas <Catalin.Marinas@arm.com>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>
Subject: Re: [patch 1/2] tick/broadcast: Prevent deep idle states if no broadcast device available
Date: Mon, 6 Jul 2015 17:35:26 +0200 (CEST)	[thread overview]
Message-ID: <alpine.DEB.2.11.1507061712590.3916@nanos> (raw)
In-Reply-To: <559A9A2B.2030705@arm.com>

On Mon, 6 Jul 2015, Sudeep Holla wrote:
> On 05/07/15 21:53, Thomas Gleixner wrote:
> > If no broadcast device is installed and the cpu local timers stop in
> > deeper idle states, then there is currently nothing telling the idle
> > code that it should not go into deep idle states, so the timers stop
> > and nothing wakes up the cpus.
> > 
> > Make the broadcast_enter/exit() functions independent of the
> > configuration options and always check on enter:
> > 
> > - whether the cpu local device is affected by idle states
> > - whether a broadcast device is available
> > 
> > This covers all possible config combinations including
> > CONFIG_BROADCAST=n.
> > 
> > Reported-by: Sudeep Holla <sudeep.holla@arm.com>
> 
> Sorry for the delay, took a while testing few configuration:
> 
> +--------------+--------+--------------+--------------------+
> |   Configs    | PERIOD | HRTimers+NOHz|Cmdline(HR+NoHZ=off)|
> +--------------+--------+--------------+--------------------+
> | UP w/o H/W BC|   OK   |      OK      |        OK          |
> +--------------+--------+--------------+--------------------+
> | UP w/ H/W BC |   OK   |      OK      |        OK          |
> +--------------+--------+--------------+--------------------+
> |SMP w/o H/W BC|   OK*  |      OK      |        Not OK(**)  |
> +--------------+--------+--------------+--------------------+
> |SMP w/ H/W BC |   OK   |      OK      |        OK          |
> +--------------+--------+--------------+--------------------+
> 
> H/W BC - Hardware Broadcast Timer source
> 
> (*) None of the CPUs enters deeper idle states losing local timers
> 
> (**)SMP build without Hardware Broadcast Timer source(i.e. one cpu is
> the broadcast source) with HRTimers+NOHz configs but disabled in cmdline
> fails to boot. 

That's using the hrtimer broadcast mechanism, right?

> On connecting debugger, I found all the cpus are in
> shallow idle state(i.e. WFI in ARM) but with interrupts disabled.

And that means?

> I am not really keen on the failing configuration. We have never tested
> that before, though I found it working with CPUIdle disabled.

Well, we should figure out what happens while we are at it before
everything gets paged out again.

In the case of CONFIG_NOHZ=n and CONFIG_HIGHRES=n the broadcast
hrtimer is not compiled as it depends on CONFIG_TICK_ONESHOT, so it
works via the bc.evtdev == NULL check.

With either option enabled CONFIG_TICK_ONESHOT gets set, so the
broadcast timer gets installed but somehow does not work proper if
nohz and highres are disabled on the kernel command line.

Thanks,

	tglx