* [PATCH V2 1/2] cpuidle: Replace use_deepest_state flag by forced_idle_latency_limit_ns
@ 2019-11-16 13:16 Daniel Lezcano
2019-11-16 13:16 ` [PATCH V2 2/2] cpuidle: Use the latency to call find_deepest_idle_state Daniel Lezcano
2019-11-20 11:05 ` [PATCH V2 1/2] cpuidle: Replace use_deepest_state flag by forced_idle_latency_limit_ns Rafael J. Wysocki
0 siblings, 2 replies; 4+ messages in thread
From: Daniel Lezcano @ 2019-11-16 13:16 UTC (permalink / raw)
To: rafael; +Cc: linux-pm, linux-kernel, ulf.hansson
We want to specify a latency constraint when choosing an idle state at
play_idle time. Instead of duplicating the information in the
structure or propagate the latency in the call stack, change the
use_deepest_state by forced_latency_limit_ns to introduce this
constraint. The idea being that when it is set, idle is forced
(i.e. no governors), but there is a latency limit for the state to
use.
A zero latency constraint means "do not use the deepest idle state
path" as the 'use_deepest_state' boolean was used in the
cpuidle_idle_call.
In addition add the play_idle_precise() function getting a nanosec
base latency constraint as parameter. The play_idle() function becomes
a wrapper passing an U64_MAX to play_idle_precise().
Suggested-by: Rafael J. Wysocki <rafael@kernel.org>
Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
---
drivers/cpuidle/cpuidle.c | 10 +++++-----
include/linux/cpu.h | 6 +++++-
include/linux/cpuidle.h | 6 +++---
kernel/sched/idle.c | 14 +++++++-------
4 files changed, 20 insertions(+), 16 deletions(-)
diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
index 44ae39f2b47a..62226fadc02d 100644
--- a/drivers/cpuidle/cpuidle.c
+++ b/drivers/cpuidle/cpuidle.c
@@ -100,19 +100,19 @@ static int find_deepest_state(struct cpuidle_driver *drv,
/**
* cpuidle_use_deepest_state - Set/clear governor override flag.
- * @enable: New value of the flag.
+ * @latency_limit_ns: A latency limit constraint
*
- * Set/unset the current CPU to use the deepest idle state (override governors
- * going forward if set).
+ * Set/unset the current CPU to use the deepest idle state with the exit
+ * latency within @latency_limit_ns
*/
-void cpuidle_use_deepest_state(bool enable)
+void cpuidle_use_deepest_state(u64 latency_limit_ns)
{
struct cpuidle_device *dev;
preempt_disable();
dev = cpuidle_get_device();
if (dev)
- dev->use_deepest_state = enable;
+ dev->forced_idle_latency_limit_ns = latency_limit_ns;
preempt_enable();
}
diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index d0633ebdaa9c..4c9694e209a5 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -179,7 +179,11 @@ void arch_cpu_idle_dead(void);
int cpu_report_state(int cpu);
int cpu_check_up_prepare(int cpu);
void cpu_set_state_online(int cpu);
-void play_idle(unsigned long duration_us);
+void play_idle_precise(u64 duration_ns, u64 latency_ns);
+static inline void play_idle(unsigned long duration_us)
+{
+ play_idle_precise(duration_us * NSEC_PER_USEC, U64_MAX);
+}
#ifdef CONFIG_HOTPLUG_CPU
bool cpu_wait_death(unsigned int cpu, int seconds);
diff --git a/include/linux/cpuidle.h b/include/linux/cpuidle.h
index d23a3b1ddcf6..1f3f4dd01e48 100644
--- a/include/linux/cpuidle.h
+++ b/include/linux/cpuidle.h
@@ -83,7 +83,6 @@ struct cpuidle_driver_kobj;
struct cpuidle_device {
unsigned int registered:1;
unsigned int enabled:1;
- unsigned int use_deepest_state:1;
unsigned int poll_time_limit:1;
unsigned int cpu;
ktime_t next_hrtimer;
@@ -91,6 +90,7 @@ struct cpuidle_device {
int last_state_idx;
int last_residency;
u64 poll_limit_ns;
+ u64 forced_idle_latency_limit_ns;
struct cpuidle_state_usage states_usage[CPUIDLE_STATE_MAX];
struct cpuidle_state_kobj *kobjs[CPUIDLE_STATE_MAX];
struct cpuidle_driver_kobj *kobj_driver;
@@ -210,7 +210,7 @@ extern int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
struct cpuidle_device *dev);
extern int cpuidle_enter_s2idle(struct cpuidle_driver *drv,
struct cpuidle_device *dev);
-extern void cpuidle_use_deepest_state(bool enable);
+extern void cpuidle_use_deepest_state(u64 latency_limit_ns);
#else
static inline int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
struct cpuidle_device *dev)
@@ -218,7 +218,7 @@ static inline int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
static inline int cpuidle_enter_s2idle(struct cpuidle_driver *drv,
struct cpuidle_device *dev)
{return -ENODEV; }
-static inline void cpuidle_use_deepest_state(bool enable)
+static inline void cpuidle_use_deepest_state(u64 latency_limit_ns)
{
}
#endif
diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
index 8dad5aa600ea..0a817e907192 100644
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -165,7 +165,7 @@ static void cpuidle_idle_call(void)
* until a proper wakeup interrupt happens.
*/
- if (idle_should_enter_s2idle() || dev->use_deepest_state) {
+ if (idle_should_enter_s2idle() || dev->forced_idle_latency_limit_ns) {
if (idle_should_enter_s2idle()) {
rcu_idle_enter();
@@ -311,7 +311,7 @@ static enum hrtimer_restart idle_inject_timer_fn(struct hrtimer *timer)
return HRTIMER_NORESTART;
}
-void play_idle(unsigned long duration_us)
+void play_idle_precise(u64 duration_ns, u64 latency_ns)
{
struct idle_timer it;
@@ -323,29 +323,29 @@ void play_idle(unsigned long duration_us)
WARN_ON_ONCE(current->nr_cpus_allowed != 1);
WARN_ON_ONCE(!(current->flags & PF_KTHREAD));
WARN_ON_ONCE(!(current->flags & PF_NO_SETAFFINITY));
- WARN_ON_ONCE(!duration_us);
+ WARN_ON_ONCE(!duration_ns);
rcu_sleep_check();
preempt_disable();
current->flags |= PF_IDLE;
- cpuidle_use_deepest_state(true);
+ cpuidle_use_deepest_state(latency_ns);
it.done = 0;
hrtimer_init_on_stack(&it.timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
it.timer.function = idle_inject_timer_fn;
- hrtimer_start(&it.timer, ns_to_ktime(duration_us * NSEC_PER_USEC),
+ hrtimer_start(&it.timer, ns_to_ktime(duration_ns),
HRTIMER_MODE_REL_PINNED);
while (!READ_ONCE(it.done))
do_idle();
- cpuidle_use_deepest_state(false);
+ cpuidle_use_deepest_state(0);
current->flags &= ~PF_IDLE;
preempt_fold_need_resched();
preempt_enable();
}
-EXPORT_SYMBOL_GPL(play_idle);
+EXPORT_SYMBOL_GPL(play_idle_precise);
void cpu_startup_entry(enum cpuhp_state state)
{
--
2.17.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH V2 2/2] cpuidle: Use the latency to call find_deepest_idle_state
2019-11-16 13:16 [PATCH V2 1/2] cpuidle: Replace use_deepest_state flag by forced_idle_latency_limit_ns Daniel Lezcano
@ 2019-11-16 13:16 ` Daniel Lezcano
2019-11-20 11:05 ` [PATCH V2 1/2] cpuidle: Replace use_deepest_state flag by forced_idle_latency_limit_ns Rafael J. Wysocki
1 sibling, 0 replies; 4+ messages in thread
From: Daniel Lezcano @ 2019-11-16 13:16 UTC (permalink / raw)
To: rafael; +Cc: linux-pm, linux-kernel, ulf.hansson
As the dev->forced_idle_latency_limit_ns is filled with the latency
value when this function is called, use it as a parameter to the
find_deepest_idle_state() function.
Suggested-by: Rafael J. Wysocki <rafael@kernel.org>
Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
---
drivers/cpuidle/cpuidle.c | 7 +++++--
include/linux/cpuidle.h | 6 ++++--
kernel/sched/idle.c | 8 +++++++-
3 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
index 62226fadc02d..a02b701fc289 100644
--- a/drivers/cpuidle/cpuidle.c
+++ b/drivers/cpuidle/cpuidle.c
@@ -122,9 +122,12 @@ void cpuidle_use_deepest_state(u64 latency_limit_ns)
* @dev: cpuidle device for the given CPU.
*/
int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
- struct cpuidle_device *dev)
+ struct cpuidle_device *dev,
+ u64 latency_limit_ns)
{
- return find_deepest_state(drv, dev, UINT_MAX, 0, false);
+ return find_deepest_state(drv, dev,
+ div_u64(latency_limit_ns, NSEC_PER_USEC),
+ 0, false);
}
#ifdef CONFIG_SUSPEND
diff --git a/include/linux/cpuidle.h b/include/linux/cpuidle.h
index 1f3f4dd01e48..b60f35b7d53e 100644
--- a/include/linux/cpuidle.h
+++ b/include/linux/cpuidle.h
@@ -207,13 +207,15 @@ static inline struct cpuidle_device *cpuidle_get_device(void) {return NULL; }
#ifdef CONFIG_CPU_IDLE
extern int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
- struct cpuidle_device *dev);
+ struct cpuidle_device *dev,
+ u64 latency_limit_ns);
extern int cpuidle_enter_s2idle(struct cpuidle_driver *drv,
struct cpuidle_device *dev);
extern void cpuidle_use_deepest_state(u64 latency_limit_ns);
#else
static inline int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
- struct cpuidle_device *dev)
+ struct cpuidle_device *dev,
+ u64 latency_limit_ns)
{return -ENODEV; }
static inline int cpuidle_enter_s2idle(struct cpuidle_driver *drv,
struct cpuidle_device *dev)
diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
index 0a817e907192..fd4a8747e602 100644
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -166,6 +166,9 @@ static void cpuidle_idle_call(void)
*/
if (idle_should_enter_s2idle() || dev->forced_idle_latency_limit_ns) {
+
+ u64 latency_limit_ns = dev->forced_idle_latency_limit_ns;
+
if (idle_should_enter_s2idle()) {
rcu_idle_enter();
@@ -176,12 +179,15 @@ static void cpuidle_idle_call(void)
}
rcu_idle_exit();
+
+ latency_limit_ns = U64_MAX;
}
tick_nohz_idle_stop_tick();
rcu_idle_enter();
- next_state = cpuidle_find_deepest_state(drv, dev);
+ next_state = cpuidle_find_deepest_state(drv, dev,
+ latency_limit_ns);
call_cpuidle(drv, dev, next_state);
} else {
bool stop_tick = true;
--
2.17.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH V2 1/2] cpuidle: Replace use_deepest_state flag by forced_idle_latency_limit_ns
2019-11-16 13:16 [PATCH V2 1/2] cpuidle: Replace use_deepest_state flag by forced_idle_latency_limit_ns Daniel Lezcano
2019-11-16 13:16 ` [PATCH V2 2/2] cpuidle: Use the latency to call find_deepest_idle_state Daniel Lezcano
@ 2019-11-20 11:05 ` Rafael J. Wysocki
2019-11-20 11:32 ` Daniel Lezcano
1 sibling, 1 reply; 4+ messages in thread
From: Rafael J. Wysocki @ 2019-11-20 11:05 UTC (permalink / raw)
To: Daniel Lezcano; +Cc: rafael, linux-pm, linux-kernel, ulf.hansson
On Saturday, November 16, 2019 2:16:12 PM CET Daniel Lezcano wrote:
> We want to specify a latency constraint when choosing an idle state at
> play_idle time. Instead of duplicating the information in the
> structure or propagate the latency in the call stack, change the
> use_deepest_state by forced_latency_limit_ns to introduce this
> constraint. The idea being that when it is set, idle is forced
> (i.e. no governors), but there is a latency limit for the state to
> use.
>
> A zero latency constraint means "do not use the deepest idle state
> path" as the 'use_deepest_state' boolean was used in the
> cpuidle_idle_call.
>
> In addition add the play_idle_precise() function getting a nanosec
> base latency constraint as parameter. The play_idle() function becomes
> a wrapper passing an U64_MAX to play_idle_precise().
>
> Suggested-by: Rafael J. Wysocki <rafael@kernel.org>
> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
> ---
> drivers/cpuidle/cpuidle.c | 10 +++++-----
> include/linux/cpu.h | 6 +++++-
> include/linux/cpuidle.h | 6 +++---
> kernel/sched/idle.c | 14 +++++++-------
> 4 files changed, 20 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c
> index 44ae39f2b47a..62226fadc02d 100644
> --- a/drivers/cpuidle/cpuidle.c
> +++ b/drivers/cpuidle/cpuidle.c
> @@ -100,19 +100,19 @@ static int find_deepest_state(struct cpuidle_driver *drv,
>
> /**
> * cpuidle_use_deepest_state - Set/clear governor override flag.
> - * @enable: New value of the flag.
> + * @latency_limit_ns: A latency limit constraint
> *
> - * Set/unset the current CPU to use the deepest idle state (override governors
> - * going forward if set).
> + * Set/unset the current CPU to use the deepest idle state with the exit
> + * latency within @latency_limit_ns
> */
> -void cpuidle_use_deepest_state(bool enable)
> +void cpuidle_use_deepest_state(u64 latency_limit_ns)
> {
> struct cpuidle_device *dev;
>
> preempt_disable();
> dev = cpuidle_get_device();
> if (dev)
> - dev->use_deepest_state = enable;
> + dev->forced_idle_latency_limit_ns = latency_limit_ns;
> preempt_enable();
> }
>
> diff --git a/include/linux/cpu.h b/include/linux/cpu.h
> index d0633ebdaa9c..4c9694e209a5 100644
> --- a/include/linux/cpu.h
> +++ b/include/linux/cpu.h
> @@ -179,7 +179,11 @@ void arch_cpu_idle_dead(void);
> int cpu_report_state(int cpu);
> int cpu_check_up_prepare(int cpu);
> void cpu_set_state_online(int cpu);
> -void play_idle(unsigned long duration_us);
> +void play_idle_precise(u64 duration_ns, u64 latency_ns);
> +static inline void play_idle(unsigned long duration_us)
> +{
> + play_idle_precise(duration_us * NSEC_PER_USEC, U64_MAX);
> +}
>
> #ifdef CONFIG_HOTPLUG_CPU
> bool cpu_wait_death(unsigned int cpu, int seconds);
> diff --git a/include/linux/cpuidle.h b/include/linux/cpuidle.h
> index d23a3b1ddcf6..1f3f4dd01e48 100644
> --- a/include/linux/cpuidle.h
> +++ b/include/linux/cpuidle.h
> @@ -83,7 +83,6 @@ struct cpuidle_driver_kobj;
> struct cpuidle_device {
> unsigned int registered:1;
> unsigned int enabled:1;
> - unsigned int use_deepest_state:1;
> unsigned int poll_time_limit:1;
> unsigned int cpu;
> ktime_t next_hrtimer;
> @@ -91,6 +90,7 @@ struct cpuidle_device {
> int last_state_idx;
> int last_residency;
> u64 poll_limit_ns;
> + u64 forced_idle_latency_limit_ns;
> struct cpuidle_state_usage states_usage[CPUIDLE_STATE_MAX];
> struct cpuidle_state_kobj *kobjs[CPUIDLE_STATE_MAX];
> struct cpuidle_driver_kobj *kobj_driver;
> @@ -210,7 +210,7 @@ extern int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
> struct cpuidle_device *dev);
> extern int cpuidle_enter_s2idle(struct cpuidle_driver *drv,
> struct cpuidle_device *dev);
> -extern void cpuidle_use_deepest_state(bool enable);
> +extern void cpuidle_use_deepest_state(u64 latency_limit_ns);
> #else
> static inline int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
> struct cpuidle_device *dev)
> @@ -218,7 +218,7 @@ static inline int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
> static inline int cpuidle_enter_s2idle(struct cpuidle_driver *drv,
> struct cpuidle_device *dev)
> {return -ENODEV; }
> -static inline void cpuidle_use_deepest_state(bool enable)
> +static inline void cpuidle_use_deepest_state(u64 latency_limit_ns)
> {
> }
> #endif
> diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
> index 8dad5aa600ea..0a817e907192 100644
> --- a/kernel/sched/idle.c
> +++ b/kernel/sched/idle.c
> @@ -165,7 +165,7 @@ static void cpuidle_idle_call(void)
> * until a proper wakeup interrupt happens.
> */
>
> - if (idle_should_enter_s2idle() || dev->use_deepest_state) {
> + if (idle_should_enter_s2idle() || dev->forced_idle_latency_limit_ns) {
> if (idle_should_enter_s2idle()) {
> rcu_idle_enter();
>
> @@ -311,7 +311,7 @@ static enum hrtimer_restart idle_inject_timer_fn(struct hrtimer *timer)
> return HRTIMER_NORESTART;
> }
>
> -void play_idle(unsigned long duration_us)
> +void play_idle_precise(u64 duration_ns, u64 latency_ns)
> {
> struct idle_timer it;
>
> @@ -323,29 +323,29 @@ void play_idle(unsigned long duration_us)
> WARN_ON_ONCE(current->nr_cpus_allowed != 1);
> WARN_ON_ONCE(!(current->flags & PF_KTHREAD));
> WARN_ON_ONCE(!(current->flags & PF_NO_SETAFFINITY));
> - WARN_ON_ONCE(!duration_us);
> + WARN_ON_ONCE(!duration_ns);
>
> rcu_sleep_check();
> preempt_disable();
> current->flags |= PF_IDLE;
> - cpuidle_use_deepest_state(true);
> + cpuidle_use_deepest_state(latency_ns);
>
> it.done = 0;
> hrtimer_init_on_stack(&it.timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
> it.timer.function = idle_inject_timer_fn;
> - hrtimer_start(&it.timer, ns_to_ktime(duration_us * NSEC_PER_USEC),
> + hrtimer_start(&it.timer, ns_to_ktime(duration_ns),
> HRTIMER_MODE_REL_PINNED);
>
> while (!READ_ONCE(it.done))
> do_idle();
>
> - cpuidle_use_deepest_state(false);
> + cpuidle_use_deepest_state(0);
> current->flags &= ~PF_IDLE;
>
> preempt_fold_need_resched();
> preempt_enable();
> }
> -EXPORT_SYMBOL_GPL(play_idle);
> +EXPORT_SYMBOL_GPL(play_idle_precise);
>
> void cpu_startup_entry(enum cpuhp_state state)
> {
>
Queued up for 5.5 along with the [2/2], but I have made quite a few changes
to the patches, so please double check the result in my bleeding-edge branch.
Thanks!
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH V2 1/2] cpuidle: Replace use_deepest_state flag by forced_idle_latency_limit_ns
2019-11-20 11:05 ` [PATCH V2 1/2] cpuidle: Replace use_deepest_state flag by forced_idle_latency_limit_ns Rafael J. Wysocki
@ 2019-11-20 11:32 ` Daniel Lezcano
0 siblings, 0 replies; 4+ messages in thread
From: Daniel Lezcano @ 2019-11-20 11:32 UTC (permalink / raw)
To: Rafael J. Wysocki; +Cc: rafael, linux-pm, linux-kernel, ulf.hansson
On 20/11/2019 12:05, Rafael J. Wysocki wrote:
> On Saturday, November 16, 2019 2:16:12 PM CET Daniel Lezcano wrote:
>> We want to specify a latency constraint when choosing an idle state at
>> play_idle time. Instead of duplicating the information in the
>> structure or propagate the latency in the call stack, change the
>> use_deepest_state by forced_latency_limit_ns to introduce this
>> constraint. The idea being that when it is set, idle is forced
>> (i.e. no governors), but there is a latency limit for the state to
>> use.
>>
>> A zero latency constraint means "do not use the deepest idle state
>> path" as the 'use_deepest_state' boolean was used in the
>> cpuidle_idle_call.
>>
>> In addition add the play_idle_precise() function getting a nanosec
>> base latency constraint as parameter. The play_idle() function becomes
>> a wrapper passing an U64_MAX to play_idle_precise().
>>
>> Suggested-by: Rafael J. Wysocki <rafael@kernel.org>
>> Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
>> ---
[ ... ]
> Queued up for 5.5 along with the [2/2], but I have made quite a few changes
> to the patches, so please double check the result in my bleeding-edge branch.
>
> Thanks!
They look good to me.
Thanks
-- Daniel
--
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs
Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-11-20 11:32 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-16 13:16 [PATCH V2 1/2] cpuidle: Replace use_deepest_state flag by forced_idle_latency_limit_ns Daniel Lezcano
2019-11-16 13:16 ` [PATCH V2 2/2] cpuidle: Use the latency to call find_deepest_idle_state Daniel Lezcano
2019-11-20 11:05 ` [PATCH V2 1/2] cpuidle: Replace use_deepest_state flag by forced_idle_latency_limit_ns Rafael J. Wysocki
2019-11-20 11:32 ` Daniel Lezcano
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).