[PATCH 0/3] cpuidle: Rework the handling of the poll state

[PATCH 0/3] cpuidle: Rework the handling of the poll state

[Rafael J. Wysocki]

Hi,

On x86 the fist idle state is a polling one, but the way it is set up is far
from straightforward and then it is avoided by governors in rather somewhat
convoluted fashion.

Make this more clear by explicitly flagging that state as "polling" and
checking its flag where it needs to be avoided instead of using
arch-dependent numbering of idle states (patch [1/3]), move the
polling state code from driver.c to a separate C file (patch [2/3]) and
move the initialization of it from the core to the relevant cpuidle drivers -
ACPI and intel_idle (patch [3/3]).

Thanks,
Rafael

[PATCH 1/3] cpuidle: Eliminate the CPUIDLE_DRIVER_STATE_START symbol

[Rafael J. Wysocki]

From: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

On some architectures the first (index 0) idle state is a polling
one and it doesn't really save energy, so there is the
CPUIDLE_DRIVER_STATE_START symbol allowing some pieces of
cpuidle code to avoid using that state.

However, this makes the code rather hard to follow.  It is better
to explicitly avoid the polling state, so add a new cpuidle state
flag CPUIDLE_FLAG_POLLING to mark it and make the relevant code
check that flag for the first state instead of using the
CPUIDLE_DRIVER_STATE_START symbol.

In the ACPI processor driver that cannot always rely on the state
flags (like before the states table has been set up) define
a new internal symbol ACPI_IDLE_STATE_START equivalent to the
CPUIDLE_DRIVER_STATE_START one and drop the latter.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
---
 drivers/acpi/processor_idle.c      |   10 ++++++----
 drivers/cpuidle/driver.c           |    1 +
 drivers/cpuidle/governors/ladder.c |   14 ++++++++------
 drivers/cpuidle/governors/menu.c   |   13 +++++--------
 include/linux/cpuidle.h            |    7 +------
 5 files changed, 21 insertions(+), 24 deletions(-)

Index: linux-pm/include/linux/cpuidle.h
===================================================================
--- linux-pm.orig/include/linux/cpuidle.h
+++ linux-pm/include/linux/cpuidle.h
@@ -63,6 +63,7 @@ struct cpuidle_state {
 
 /* Idle State Flags */
 #define CPUIDLE_FLAG_NONE       (0x00)
+#define CPUIDLE_FLAG_POLLING	(0x01) /* polling state */
 #define CPUIDLE_FLAG_COUPLED	(0x02) /* state applies to multiple cpus */
 #define CPUIDLE_FLAG_TIMER_STOP (0x04)  /* timer is stopped on this state */
 
@@ -250,12 +251,6 @@ static inline int cpuidle_register_gover
 {return 0;}
 #endif
 
-#ifdef CONFIG_ARCH_HAS_CPU_RELAX
-#define CPUIDLE_DRIVER_STATE_START	1
-#else
-#define CPUIDLE_DRIVER_STATE_START	0
-#endif
-
 #define CPU_PM_CPU_IDLE_ENTER(low_level_idle_enter, idx)	\
 ({								\
 	int __ret;						\
Index: linux-pm/drivers/cpuidle/driver.c
===================================================================
--- linux-pm.orig/drivers/cpuidle/driver.c
+++ linux-pm/drivers/cpuidle/driver.c
@@ -204,6 +204,7 @@ static void poll_idle_init(struct cpuidl
 	state->power_usage = -1;
 	state->enter = poll_idle;
 	state->disabled = false;
+	state->flags = CPUIDLE_FLAG_POLLING;
 }
 #else
 static void poll_idle_init(struct cpuidle_driver *drv) {}
Index: linux-pm/drivers/cpuidle/governors/menu.c
===================================================================
--- linux-pm.orig/drivers/cpuidle/governors/menu.c
+++ linux-pm/drivers/cpuidle/governors/menu.c
@@ -324,8 +324,9 @@ static int menu_select(struct cpuidle_dr
 	expected_interval = get_typical_interval(data);
 	expected_interval = min(expected_interval, data->next_timer_us);
 
-	if (CPUIDLE_DRIVER_STATE_START > 0) {
-		struct cpuidle_state *s = &drv->states[CPUIDLE_DRIVER_STATE_START];
+	first_idx = 0;
+	if (drv->states[0].flags & CPUIDLE_FLAG_POLLING) {
+		struct cpuidle_state *s = &drv->states[1];
 		unsigned int polling_threshold;
 
 		/*
@@ -336,12 +337,8 @@ static int menu_select(struct cpuidle_dr
 		polling_threshold = max_t(unsigned int, 20, s->target_residency);
 		if (data->next_timer_us > polling_threshold &&
 		    latency_req > s->exit_latency && !s->disabled &&
-		    !dev->states_usage[CPUIDLE_DRIVER_STATE_START].disable)
-			first_idx = CPUIDLE_DRIVER_STATE_START;
-		else
-			first_idx = CPUIDLE_DRIVER_STATE_START - 1;
-	} else {
-		first_idx = 0;
+		    !dev->states_usage[1].disable)
+			first_idx = 1;
 	}
 
 	/*
Index: linux-pm/drivers/cpuidle/governors/ladder.c
===================================================================
--- linux-pm.orig/drivers/cpuidle/governors/ladder.c
+++ linux-pm/drivers/cpuidle/governors/ladder.c
@@ -69,6 +69,7 @@ static int ladder_select_state(struct cp
 	struct ladder_device *ldev = this_cpu_ptr(&ladder_devices);
 	struct ladder_device_state *last_state;
 	int last_residency, last_idx = ldev->last_state_idx;
+	int first_idx = drv->states[0].flags & CPUIDLE_FLAG_POLLING ? 1 : 0;
 	int latency_req = pm_qos_request(PM_QOS_CPU_DMA_LATENCY);
 
 	/* Special case when user has set very strict latency requirement */
@@ -96,13 +97,13 @@ static int ladder_select_state(struct cp
 	}
 
 	/* consider demotion */
-	if (last_idx > CPUIDLE_DRIVER_STATE_START &&
+	if (last_idx > first_idx &&
 	    (drv->states[last_idx].disabled ||
 	    dev->states_usage[last_idx].disable ||
 	    drv->states[last_idx].exit_latency > latency_req)) {
 		int i;
 
-		for (i = last_idx - 1; i > CPUIDLE_DRIVER_STATE_START; i--) {
+		for (i = last_idx - 1; i > first_idx; i--) {
 			if (drv->states[i].exit_latency <= latency_req)
 				break;
 		}
@@ -110,7 +111,7 @@ static int ladder_select_state(struct cp
 		return i;
 	}
 
-	if (last_idx > CPUIDLE_DRIVER_STATE_START &&
+	if (last_idx > first_idx &&
 	    last_residency < last_state->threshold.demotion_time) {
 		last_state->stats.demotion_count++;
 		last_state->stats.promotion_count = 0;
@@ -133,13 +134,14 @@ static int ladder_enable_device(struct c
 				struct cpuidle_device *dev)
 {
 	int i;
+	int first_idx = drv->states[0].flags & CPUIDLE_FLAG_POLLING ? 1 : 0;
 	struct ladder_device *ldev = &per_cpu(ladder_devices, dev->cpu);
 	struct ladder_device_state *lstate;
 	struct cpuidle_state *state;
 
-	ldev->last_state_idx = CPUIDLE_DRIVER_STATE_START;
+	ldev->last_state_idx = first_idx;
 
-	for (i = CPUIDLE_DRIVER_STATE_START; i < drv->state_count; i++) {
+	for (i = first_idx; i < drv->state_count; i++) {
 		state = &drv->states[i];
 		lstate = &ldev->states[i];
 
@@ -151,7 +153,7 @@ static int ladder_enable_device(struct c
 
 		if (i < drv->state_count - 1)
 			lstate->threshold.promotion_time = state->exit_latency;
-		if (i > CPUIDLE_DRIVER_STATE_START)
+		if (i > first_idx)
 			lstate->threshold.demotion_time = state->exit_latency;
 	}
 
Index: linux-pm/drivers/acpi/processor_idle.c
===================================================================
--- linux-pm.orig/drivers/acpi/processor_idle.c
+++ linux-pm/drivers/acpi/processor_idle.c
@@ -48,6 +48,8 @@
 #define _COMPONENT              ACPI_PROCESSOR_COMPONENT
 ACPI_MODULE_NAME("processor_idle");
 
+#define ACPI_IDLE_STATE_START	(IS_ENABLED(CONFIG_ARCH_HAS_CPU_RELAX) ? 1 : 0)
+
 static unsigned int max_cstate __read_mostly = ACPI_PROCESSOR_MAX_POWER;
 module_param(max_cstate, uint, 0000);
 static unsigned int nocst __read_mostly;
@@ -761,7 +763,7 @@ static int acpi_idle_enter(struct cpuidl
 
 	if (cx->type != ACPI_STATE_C1) {
 		if (acpi_idle_fallback_to_c1(pr) && num_online_cpus() > 1) {
-			index = CPUIDLE_DRIVER_STATE_START;
+			index = ACPI_IDLE_STATE_START;
 			cx = per_cpu(acpi_cstate[index], dev->cpu);
 		} else if (cx->type == ACPI_STATE_C3 && pr->flags.bm_check) {
 			if (cx->bm_sts_skip || !acpi_idle_bm_check()) {
@@ -813,7 +815,7 @@ static void acpi_idle_enter_s2idle(struc
 static int acpi_processor_setup_cpuidle_cx(struct acpi_processor *pr,
 					   struct cpuidle_device *dev)
 {
-	int i, count = CPUIDLE_DRIVER_STATE_START;
+	int i, count = ACPI_IDLE_STATE_START;
 	struct acpi_processor_cx *cx;
 
 	if (max_cstate == 0)
@@ -840,7 +842,7 @@ static int acpi_processor_setup_cpuidle_
 
 static int acpi_processor_setup_cstates(struct acpi_processor *pr)
 {
-	int i, count = CPUIDLE_DRIVER_STATE_START;
+	int i, count = ACPI_IDLE_STATE_START;
 	struct acpi_processor_cx *cx;
 	struct cpuidle_state *state;
 	struct cpuidle_driver *drv = &acpi_idle_driver;
@@ -1291,7 +1293,7 @@ static int acpi_processor_setup_cpuidle_
 		return -EINVAL;
 
 	drv->safe_state_index = -1;
-	for (i = CPUIDLE_DRIVER_STATE_START; i < CPUIDLE_STATE_MAX; i++) {
+	for (i = ACPI_IDLE_STATE_START; i < CPUIDLE_STATE_MAX; i++) {
 		drv->states[i].name[0] = '\0';
 		drv->states[i].desc[0] = '\0';
 	}

[PATCH 2/3] cpuidle: Move polling state initialization code to separate file

[Rafael J. Wysocki]

From: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Move the polling state initialization code to a separate file built
conditionally on CONFIG_ARCH_HAS_CPU_RELAX to get rid of the #ifdef
in driver.c.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
---
 drivers/cpuidle/Makefile     |    1 +
 drivers/cpuidle/driver.c     |   31 -------------------------------
 drivers/cpuidle/poll_state.c |   36 ++++++++++++++++++++++++++++++++++++
 include/linux/cpuidle.h      |    6 ++++++
 4 files changed, 43 insertions(+), 31 deletions(-)

Index: linux-pm/drivers/cpuidle/Makefile
===================================================================
--- linux-pm.orig/drivers/cpuidle/Makefile
+++ linux-pm/drivers/cpuidle/Makefile
@@ -5,6 +5,7 @@
 obj-y += cpuidle.o driver.o governor.o sysfs.o governors/
 obj-$(CONFIG_ARCH_NEEDS_CPU_IDLE_COUPLED) += coupled.o
 obj-$(CONFIG_DT_IDLE_STATES)		  += dt_idle_states.o
+obj-$(CONFIG_ARCH_HAS_CPU_RELAX)	  += poll_state.o
 
 ##################################################################################
 # ARM SoC drivers
Index: linux-pm/drivers/cpuidle/poll_state.c
===================================================================
--- /dev/null
+++ linux-pm/drivers/cpuidle/poll_state.c
@@ -0,0 +1,36 @@
+/*
+ * poll_state.c - Polling idle state
+ *
+ * This file is released under the GPLv2.
+ */
+
+#include <linux/cpuidle.h>
+#include <linux/sched.h>
+#include <linux/sched/idle.h>
+
+static int __cpuidle poll_idle(struct cpuidle_device *dev,
+			       struct cpuidle_driver *drv, int index)
+{
+	local_irq_enable();
+	if (!current_set_polling_and_test()) {
+		while (!need_resched())
+			cpu_relax();
+	}
+	current_clr_polling();
+
+	return index;
+}
+
+void poll_idle_init(struct cpuidle_driver *drv)
+{
+	struct cpuidle_state *state = &drv->states[0];
+
+	snprintf(state->name, CPUIDLE_NAME_LEN, "POLL");
+	snprintf(state->desc, CPUIDLE_DESC_LEN, "CPUIDLE CORE POLL IDLE");
+	state->exit_latency = 0;
+	state->target_residency = 0;
+	state->power_usage = -1;
+	state->enter = poll_idle;
+	state->disabled = false;
+	state->flags = CPUIDLE_FLAG_POLLING;
+}
Index: linux-pm/include/linux/cpuidle.h
===================================================================
--- linux-pm.orig/include/linux/cpuidle.h
+++ linux-pm/include/linux/cpuidle.h
@@ -225,6 +225,12 @@ static inline void cpuidle_coupled_paral
 }
 #endif
 
+#ifdef CONFIG_ARCH_HAS_CPU_RELAX
+void poll_idle_init(struct cpuidle_driver *drv);
+#else
+static void poll_idle_init(struct cpuidle_driver *drv) {}
+#endif
+
 /******************************
  * CPUIDLE GOVERNOR INTERFACE *
  ******************************/
Index: linux-pm/drivers/cpuidle/driver.c
===================================================================
--- linux-pm.orig/drivers/cpuidle/driver.c
+++ linux-pm/drivers/cpuidle/driver.c
@@ -179,37 +179,6 @@ static void __cpuidle_driver_init(struct
 	}
 }
 
-#ifdef CONFIG_ARCH_HAS_CPU_RELAX
-static int __cpuidle poll_idle(struct cpuidle_device *dev,
-			       struct cpuidle_driver *drv, int index)
-{
-	local_irq_enable();
-	if (!current_set_polling_and_test()) {
-		while (!need_resched())
-			cpu_relax();
-	}
-	current_clr_polling();
-
-	return index;
-}
-
-static void poll_idle_init(struct cpuidle_driver *drv)
-{
-	struct cpuidle_state *state = &drv->states[0];
-
-	snprintf(state->name, CPUIDLE_NAME_LEN, "POLL");
-	snprintf(state->desc, CPUIDLE_DESC_LEN, "CPUIDLE CORE POLL IDLE");
-	state->exit_latency = 0;
-	state->target_residency = 0;
-	state->power_usage = -1;
-	state->enter = poll_idle;
-	state->disabled = false;
-	state->flags = CPUIDLE_FLAG_POLLING;
-}
-#else
-static void poll_idle_init(struct cpuidle_driver *drv) {}
-#endif /* !CONFIG_ARCH_HAS_CPU_RELAX */
-
 /**
  * __cpuidle_register_driver: register the driver
  * @drv: a valid pointer to a struct cpuidle_driver

[PATCH 3/3] cpuidle: Make drivers initialize polling state

[Rafael J. Wysocki]

From: Rafael J. Wysocki <rafael.j.wysocki@intel.com>

Make the drivers that want to include the polling state into their
states table initialize it explicitly and drop the initialization of
it (which in fact is conditional, but that is not obvious from the
code) from the core.

Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
---
 drivers/acpi/processor_idle.c |    9 ++++++++-
 drivers/cpuidle/driver.c      |    2 --
 drivers/cpuidle/poll_state.c  |    3 ++-
 drivers/idle/intel_idle.c     |    1 +
 include/linux/cpuidle.h       |    4 +---
 5 files changed, 12 insertions(+), 7 deletions(-)

Index: linux-pm/drivers/acpi/processor_idle.c
===================================================================
--- linux-pm.orig/drivers/acpi/processor_idle.c
+++ linux-pm/drivers/acpi/processor_idle.c
@@ -842,7 +842,7 @@ static int acpi_processor_setup_cpuidle_
 
 static int acpi_processor_setup_cstates(struct acpi_processor *pr)
 {
-	int i, count = ACPI_IDLE_STATE_START;
+	int i, count;
 	struct acpi_processor_cx *cx;
 	struct cpuidle_state *state;
 	struct cpuidle_driver *drv = &acpi_idle_driver;
@@ -850,6 +850,13 @@ static int acpi_processor_setup_cstates(
 	if (max_cstate == 0)
 		max_cstate = 1;
 
+	if (IS_ENABLED(CONFIG_ARCH_HAS_CPU_RELAX)) {
+		cpuidle_poll_state_init(drv);
+		count = 1;
+	} else {
+		count = 0;
+	}
+
 	for (i = 1; i < ACPI_PROCESSOR_MAX_POWER && i <= max_cstate; i++) {
 		cx = &pr->power.states[i];
 
Index: linux-pm/drivers/cpuidle/driver.c
===================================================================
--- linux-pm.orig/drivers/cpuidle/driver.c
+++ linux-pm/drivers/cpuidle/driver.c
@@ -216,8 +216,6 @@ static int __cpuidle_register_driver(str
 		on_each_cpu_mask(drv->cpumask, cpuidle_setup_broadcast_timer,
 				 (void *)1, 1);
 
-	poll_idle_init(drv);
-
 	return 0;
 }
 
Index: linux-pm/drivers/cpuidle/poll_state.c
===================================================================
--- linux-pm.orig/drivers/cpuidle/poll_state.c
+++ linux-pm/drivers/cpuidle/poll_state.c
@@ -21,7 +21,7 @@ static int __cpuidle poll_idle(struct cp
 	return index;
 }
 
-void poll_idle_init(struct cpuidle_driver *drv)
+void cpuidle_poll_state_init(struct cpuidle_driver *drv)
 {
 	struct cpuidle_state *state = &drv->states[0];
 
@@ -34,3 +34,4 @@ void poll_idle_init(struct cpuidle_drive
 	state->disabled = false;
 	state->flags = CPUIDLE_FLAG_POLLING;
 }
+EXPORT_SYMBOL_GPL(cpuidle_poll_state_init);
Index: linux-pm/include/linux/cpuidle.h
===================================================================
--- linux-pm.orig/include/linux/cpuidle.h
+++ linux-pm/include/linux/cpuidle.h
@@ -226,9 +226,7 @@ static inline void cpuidle_coupled_paral
 #endif
 
 #ifdef CONFIG_ARCH_HAS_CPU_RELAX
-void poll_idle_init(struct cpuidle_driver *drv);
-#else
-static void poll_idle_init(struct cpuidle_driver *drv) {}
+void cpuidle_poll_state_init(struct cpuidle_driver *drv);
 #endif
 
 /******************************
Index: linux-pm/drivers/idle/intel_idle.c
===================================================================
--- linux-pm.orig/drivers/idle/intel_idle.c
+++ linux-pm/drivers/idle/intel_idle.c
@@ -1331,6 +1331,7 @@ static void __init intel_idle_cpuidle_dr
 
 	intel_idle_state_table_update();
 
+	cpuidle_poll_state_init(drv);
 	drv->state_count = 1;
 
 	for (cstate = 0; cstate < CPUIDLE_STATE_MAX; ++cstate) {

Re: [PATCH 0/3] cpuidle: Rework the handling of the poll state

[Sudeep Holla]

On 23/08/17 22:18, Rafael J. Wysocki wrote:
> Hi,
> 
> On x86 the fist idle state is a polling one, but the way it is set up is far
> from straightforward and then it is avoided by governors in rather somewhat
> convoluted fashion.
> 
> Make this more clear by explicitly flagging that state as "polling" and
> checking its flag where it needs to be avoided instead of using
> arch-dependent numbering of idle states (patch [1/3]), move the
> polling state code from driver.c to a separate C file (patch [2/3]) and
> move the initialization of it from the core to the relevant cpuidle drivers -
> ACPI and intel_idle (patch [3/3]).
> 

Tested this on ARM64 platform(both DT and ACPI/LPI) and everything
continues to work fine.
Tested-by: Sudeep Holla <sudeep.holla@arm.com>

-- 
Regards,
Sudeep

Re: [PATCH 0/3] cpuidle: Rework the handling of the poll state

[Rafael J. Wysocki]

On Thursday, August 24, 2017 11:48:19 AM CEST Sudeep Holla wrote:
> 
> On 23/08/17 22:18, Rafael J. Wysocki wrote:
> > Hi,
> > 
> > On x86 the fist idle state is a polling one, but the way it is set up is far
> > from straightforward and then it is avoided by governors in rather somewhat
> > convoluted fashion.
> > 
> > Make this more clear by explicitly flagging that state as "polling" and
> > checking its flag where it needs to be avoided instead of using
> > arch-dependent numbering of idle states (patch [1/3]), move the
> > polling state code from driver.c to a separate C file (patch [2/3]) and
> > move the initialization of it from the core to the relevant cpuidle drivers -
> > ACPI and intel_idle (patch [3/3]).
> > 
> 
> Tested this on ARM64 platform(both DT and ACPI/LPI) and everything
> continues to work fine.
> Tested-by: Sudeep Holla <sudeep.holla@arm.com>

Thanks Sudeep!

I haven't seen any more comments on this which I'm taking as a green light for
it, so I'm going to queue it up for 4.14.

Thanks,
Rafael

Re: [PATCH 0/3] cpuidle: Rework the handling of the poll state

[Daniel Lezcano]

On 28/08/2017 23:17, Rafael J. Wysocki wrote:
> On Thursday, August 24, 2017 11:48:19 AM CEST Sudeep Holla wrote:
>>
>> On 23/08/17 22:18, Rafael J. Wysocki wrote:
>>> Hi,
>>>
>>> On x86 the fist idle state is a polling one, but the way it is set up is far
>>> from straightforward and then it is avoided by governors in rather somewhat
>>> convoluted fashion.
>>>
>>> Make this more clear by explicitly flagging that state as "polling" and
>>> checking its flag where it needs to be avoided instead of using
>>> arch-dependent numbering of idle states (patch [1/3]), move the
>>> polling state code from driver.c to a separate C file (patch [2/3]) and
>>> move the initialization of it from the core to the relevant cpuidle drivers -
>>> ACPI and intel_idle (patch [3/3]).
>>>
>>
>> Tested this on ARM64 platform(both DT and ACPI/LPI) and everything
>> continues to work fine.
>> Tested-by: Sudeep Holla <sudeep.holla@arm.com>
> 
> Thanks Sudeep!
> 
> I haven't seen any more comments on this which I'm taking as a green light for
> it, so I'm going to queue it up for 4.14.

Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org>


-- 
 <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog