* [PATCH RESEND v2] sched/nohz: Optimize get_nohz_timer_target()
@ 2020-01-13 0:50 Wanpeng Li
2020-01-20 6:14 ` Wanpeng Li
2020-01-29 11:32 ` [tip: sched/core] " tip-bot2 for Wanpeng Li
0 siblings, 2 replies; 3+ messages in thread
From: Wanpeng Li @ 2020-01-13 0:50 UTC (permalink / raw)
To: linux-kernel
Cc: Ingo Molnar, Peter Zijlstra, Thomas Gleixner, Ingo Molnar,
Frederic Weisbecker
From: Wanpeng Li <wanpengli@tencent.com>
On a machine, cpu 0 is used for housekeeping, the other 39 cpus in the
same socket are in nohz_full mode. We can observe huge time burn in the
loop for seaching nearest busy housekeeper cpu by ftrace.
2) | get_nohz_timer_target() {
2) 0.240 us | housekeeping_test_cpu();
2) 0.458 us | housekeeping_test_cpu();
...
2) 0.292 us | housekeeping_test_cpu();
2) 0.240 us | housekeeping_test_cpu();
2) 0.227 us | housekeeping_any_cpu();
2) + 43.460 us | }
This patch optimizes the searching logic by finding a nearest housekeeper
cpu in the housekeeping cpumask, it can minimize the worst searching time
from ~44us to < 10us in my testing. In addition, the last iterated busy
housekeeper can become a random candidate while current CPU is a better
fallback if it is a housekeeper.
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
---
v1 -> v2:
* current CPU is a better fallback if it is a housekeeper
kernel/sched/core.c | 19 ++++++++++++-------
1 file changed, 12 insertions(+), 7 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 102dfcf..04a0f6a 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -539,27 +539,32 @@ void resched_cpu(int cpu)
*/
int get_nohz_timer_target(void)
{
- int i, cpu = smp_processor_id();
+ int i, cpu = smp_processor_id(), default_cpu = -1;
struct sched_domain *sd;
- if (!idle_cpu(cpu) && housekeeping_cpu(cpu, HK_FLAG_TIMER))
- return cpu;
+ if (housekeeping_cpu(cpu, HK_FLAG_TIMER)) {
+ if (!idle_cpu(cpu))
+ return cpu;
+ default_cpu = cpu;
+ }
rcu_read_lock();
for_each_domain(cpu, sd) {
- for_each_cpu(i, sched_domain_span(sd)) {
+ for_each_cpu_and(i, sched_domain_span(sd),
+ housekeeping_cpumask(HK_FLAG_TIMER)) {
if (cpu == i)
continue;
- if (!idle_cpu(i) && housekeeping_cpu(i, HK_FLAG_TIMER)) {
+ if (!idle_cpu(i)) {
cpu = i;
goto unlock;
}
}
}
- if (!housekeeping_cpu(cpu, HK_FLAG_TIMER))
- cpu = housekeeping_any_cpu(HK_FLAG_TIMER);
+ if (default_cpu == -1)
+ default_cpu = housekeeping_any_cpu(HK_FLAG_TIMER);
+ cpu = default_cpu;
unlock:
rcu_read_unlock();
return cpu;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH RESEND v2] sched/nohz: Optimize get_nohz_timer_target()
2020-01-13 0:50 [PATCH RESEND v2] sched/nohz: Optimize get_nohz_timer_target() Wanpeng Li
@ 2020-01-20 6:14 ` Wanpeng Li
2020-01-29 11:32 ` [tip: sched/core] " tip-bot2 for Wanpeng Li
1 sibling, 0 replies; 3+ messages in thread
From: Wanpeng Li @ 2020-01-20 6:14 UTC (permalink / raw)
To: LKML
Cc: Ingo Molnar, Peter Zijlstra, Thomas Gleixner, Ingo Molnar,
Frederic Weisbecker
ping Thomas,
On Mon, 13 Jan 2020 at 08:50, Wanpeng Li <kernellwp@gmail.com> wrote:
>
> From: Wanpeng Li <wanpengli@tencent.com>
>
> On a machine, cpu 0 is used for housekeeping, the other 39 cpus in the
> same socket are in nohz_full mode. We can observe huge time burn in the
> loop for seaching nearest busy housekeeper cpu by ftrace.
>
> 2) | get_nohz_timer_target() {
> 2) 0.240 us | housekeeping_test_cpu();
> 2) 0.458 us | housekeeping_test_cpu();
>
> ...
>
> 2) 0.292 us | housekeeping_test_cpu();
> 2) 0.240 us | housekeeping_test_cpu();
> 2) 0.227 us | housekeeping_any_cpu();
> 2) + 43.460 us | }
>
> This patch optimizes the searching logic by finding a nearest housekeeper
> cpu in the housekeeping cpumask, it can minimize the worst searching time
> from ~44us to < 10us in my testing. In addition, the last iterated busy
> housekeeper can become a random candidate while current CPU is a better
> fallback if it is a housekeeper.
>
> Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Frederic Weisbecker <frederic@kernel.org>
> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> ---
> v1 -> v2:
> * current CPU is a better fallback if it is a housekeeper
>
> kernel/sched/core.c | 19 ++++++++++++-------
> 1 file changed, 12 insertions(+), 7 deletions(-)
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 102dfcf..04a0f6a 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -539,27 +539,32 @@ void resched_cpu(int cpu)
> */
> int get_nohz_timer_target(void)
> {
> - int i, cpu = smp_processor_id();
> + int i, cpu = smp_processor_id(), default_cpu = -1;
> struct sched_domain *sd;
>
> - if (!idle_cpu(cpu) && housekeeping_cpu(cpu, HK_FLAG_TIMER))
> - return cpu;
> + if (housekeeping_cpu(cpu, HK_FLAG_TIMER)) {
> + if (!idle_cpu(cpu))
> + return cpu;
> + default_cpu = cpu;
> + }
>
> rcu_read_lock();
> for_each_domain(cpu, sd) {
> - for_each_cpu(i, sched_domain_span(sd)) {
> + for_each_cpu_and(i, sched_domain_span(sd),
> + housekeeping_cpumask(HK_FLAG_TIMER)) {
> if (cpu == i)
> continue;
>
> - if (!idle_cpu(i) && housekeeping_cpu(i, HK_FLAG_TIMER)) {
> + if (!idle_cpu(i)) {
> cpu = i;
> goto unlock;
> }
> }
> }
>
> - if (!housekeeping_cpu(cpu, HK_FLAG_TIMER))
> - cpu = housekeeping_any_cpu(HK_FLAG_TIMER);
> + if (default_cpu == -1)
> + default_cpu = housekeeping_any_cpu(HK_FLAG_TIMER);
> + cpu = default_cpu;
> unlock:
> rcu_read_unlock();
> return cpu;
> --
> 1.8.3.1
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* [tip: sched/core] sched/nohz: Optimize get_nohz_timer_target()
2020-01-13 0:50 [PATCH RESEND v2] sched/nohz: Optimize get_nohz_timer_target() Wanpeng Li
2020-01-20 6:14 ` Wanpeng Li
@ 2020-01-29 11:32 ` tip-bot2 for Wanpeng Li
1 sibling, 0 replies; 3+ messages in thread
From: tip-bot2 for Wanpeng Li @ 2020-01-29 11:32 UTC (permalink / raw)
To: linux-tip-commits
Cc: Wanpeng Li, Peter Zijlstra (Intel),
Ingo Molnar, Frederic Weisbecker, x86, LKML
The following commit has been merged into the sched/core branch of tip:
Commit-ID: e938b9c94164e4d981039f1cf6007d7453883e5a
Gitweb: https://git.kernel.org/tip/e938b9c94164e4d981039f1cf6007d7453883e5a
Author: Wanpeng Li <wanpengli@tencent.com>
AuthorDate: Mon, 13 Jan 2020 08:50:27 +08:00
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Tue, 28 Jan 2020 21:36:57 +01:00
sched/nohz: Optimize get_nohz_timer_target()
On a machine, CPU 0 is used for housekeeping, the other 39 CPUs in the
same socket are in nohz_full mode. We can observe huge time burn in the
loop for seaching nearest busy housekeeper cpu by ftrace.
2) | get_nohz_timer_target() {
2) 0.240 us | housekeeping_test_cpu();
2) 0.458 us | housekeeping_test_cpu();
...
2) 0.292 us | housekeeping_test_cpu();
2) 0.240 us | housekeeping_test_cpu();
2) 0.227 us | housekeeping_any_cpu();
2) + 43.460 us | }
This patch optimizes the searching logic by finding a nearest housekeeper
CPU in the housekeeping cpumask, it can minimize the worst searching time
from ~44us to < 10us in my testing. In addition, the last iterated busy
housekeeper can become a random candidate while current CPU is a better
fallback if it is a housekeeper.
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://lkml.kernel.org/r/1578876627-11938-1-git-send-email-wanpengli@tencent.com
---
kernel/sched/core.c | 19 ++++++++++++-------
1 file changed, 12 insertions(+), 7 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 55b9a9c..a8a5d5b 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -552,27 +552,32 @@ void resched_cpu(int cpu)
*/
int get_nohz_timer_target(void)
{
- int i, cpu = smp_processor_id();
+ int i, cpu = smp_processor_id(), default_cpu = -1;
struct sched_domain *sd;
- if (!idle_cpu(cpu) && housekeeping_cpu(cpu, HK_FLAG_TIMER))
- return cpu;
+ if (housekeeping_cpu(cpu, HK_FLAG_TIMER)) {
+ if (!idle_cpu(cpu))
+ return cpu;
+ default_cpu = cpu;
+ }
rcu_read_lock();
for_each_domain(cpu, sd) {
- for_each_cpu(i, sched_domain_span(sd)) {
+ for_each_cpu_and(i, sched_domain_span(sd),
+ housekeeping_cpumask(HK_FLAG_TIMER)) {
if (cpu == i)
continue;
- if (!idle_cpu(i) && housekeeping_cpu(i, HK_FLAG_TIMER)) {
+ if (!idle_cpu(i)) {
cpu = i;
goto unlock;
}
}
}
- if (!housekeeping_cpu(cpu, HK_FLAG_TIMER))
- cpu = housekeeping_any_cpu(HK_FLAG_TIMER);
+ if (default_cpu == -1)
+ default_cpu = housekeeping_any_cpu(HK_FLAG_TIMER);
+ cpu = default_cpu;
unlock:
rcu_read_unlock();
return cpu;
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-01-29 11:33 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-13 0:50 [PATCH RESEND v2] sched/nohz: Optimize get_nohz_timer_target() Wanpeng Li
2020-01-20 6:14 ` Wanpeng Li
2020-01-29 11:32 ` [tip: sched/core] " tip-bot2 for Wanpeng Li
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.