* [PATCH 0/2] Reduce SIS scanning
@ 2021-08-04 11:58 Mel Gorman
2021-08-04 11:58 ` [PATCH 1/2] sched/fair: Use prev instead of new target as recent_used_cpu Mel Gorman
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Mel Gorman @ 2021-08-04 11:58 UTC (permalink / raw)
To: Ingo Molnar
Cc: Peter Zijlstra, Vincent Guittot, Valentin Schneider,
Song Bao Hua, LKML, Mel Gorman
This is the first two patches from "Modify and/or delete SIS_PROP" and
focues on improving the hit rate for recent_used_cpu and avoids rescanning
the target CPU when it's known to be busy. These are relatively simply
patches in comparison to the full series which could be a regression
magnet and that series had at least one mistake in it.
In general, the usual suspects showed mostly small gains with a few
exceptions. Usual suspects were NAS, hackbench, schbench (Facebook
latency-sensitive workload), perf pipe, kernel building, git test suite,
dbench and redis.
From here, the next obvious candidates are either improving has_idle_cores
or continuing to try remove/improve SIS_PROP.
kernel/sched/fair.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
--
2.31.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 1/2] sched/fair: Use prev instead of new target as recent_used_cpu
2021-08-04 11:58 [PATCH 0/2] Reduce SIS scanning Mel Gorman
@ 2021-08-04 11:58 ` Mel Gorman
2021-08-05 9:34 ` [tip: sched/core] " tip-bot2 for Mel Gorman
2021-08-04 11:58 ` [PATCH 2/2] sched/fair: Avoid a second scan of target in select_idle_cpu Mel Gorman
2021-08-04 13:11 ` [PATCH 0/2] Reduce SIS scanning Peter Zijlstra
2 siblings, 1 reply; 6+ messages in thread
From: Mel Gorman @ 2021-08-04 11:58 UTC (permalink / raw)
To: Ingo Molnar
Cc: Peter Zijlstra, Vincent Guittot, Valentin Schneider,
Song Bao Hua, LKML, Mel Gorman
After select_idle_sibling, p->recent_used_cpu is set to the
new target. However on the next wakeup, prev will be the same as
recent_used_cpu unless the load balancer has moved the task since the
last wakeup. It still works, but is less efficient than it could be.
This patch preserves recent_used_cpu for longer.
The impact on SIS efficiency is tiny so the SIS statistic patches were
used to track the hit rate for using recent_used_cpu. With perf bench
pipe on a 2-socket Cascadelake machine, the hit rate went from 57.14%
to 85.32%. For more intensive wakeup loads like hackbench, the hit rate
is almost negligible but rose from 0.21% to 6.64%. For scaling loads
like tbench, the hit rate goes from almost 0% to 25.42% overall. Broadly
speaking, on tbench, the success rate is much higher for lower thread
counts and drops to almost 0 as the workload scales to towards saturation.
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
kernel/sched/fair.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 44c452072a1b..8ad7666f387c 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6376,6 +6376,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
/* Check a recently used CPU as a potential idle candidate: */
recent_used_cpu = p->recent_used_cpu;
+ p->recent_used_cpu = prev;
if (recent_used_cpu != prev &&
recent_used_cpu != target &&
cpus_share_cache(recent_used_cpu, target) &&
@@ -6902,9 +6903,6 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags)
} else if (wake_flags & WF_TTWU) { /* XXX always ? */
/* Fast path */
new_cpu = select_idle_sibling(p, prev_cpu, new_cpu);
-
- if (want_affine)
- current->recent_used_cpu = cpu;
}
rcu_read_unlock();
--
2.31.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/2] sched/fair: Avoid a second scan of target in select_idle_cpu
2021-08-04 11:58 [PATCH 0/2] Reduce SIS scanning Mel Gorman
2021-08-04 11:58 ` [PATCH 1/2] sched/fair: Use prev instead of new target as recent_used_cpu Mel Gorman
@ 2021-08-04 11:58 ` Mel Gorman
2021-08-05 9:34 ` [tip: sched/core] " tip-bot2 for Mel Gorman
2021-08-04 13:11 ` [PATCH 0/2] Reduce SIS scanning Peter Zijlstra
2 siblings, 1 reply; 6+ messages in thread
From: Mel Gorman @ 2021-08-04 11:58 UTC (permalink / raw)
To: Ingo Molnar
Cc: Peter Zijlstra, Vincent Guittot, Valentin Schneider,
Song Bao Hua, LKML, Mel Gorman
When select_idle_cpu starts scanning for an idle CPU, it starts with
a target CPU that has already been checked by select_idle_sibling.
This patch starts with the next CPU instead.
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
---
kernel/sched/fair.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 8ad7666f387c..f412b4504378 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6249,7 +6249,7 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
time = cpu_clock(this);
}
- for_each_cpu_wrap(cpu, cpus, target) {
+ for_each_cpu_wrap(cpu, cpus, target + 1) {
if (has_idle_core) {
i = select_idle_core(p, cpu, cpus, &idle_cpu);
if ((unsigned int)i < nr_cpumask_bits)
--
2.31.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 0/2] Reduce SIS scanning
2021-08-04 11:58 [PATCH 0/2] Reduce SIS scanning Mel Gorman
2021-08-04 11:58 ` [PATCH 1/2] sched/fair: Use prev instead of new target as recent_used_cpu Mel Gorman
2021-08-04 11:58 ` [PATCH 2/2] sched/fair: Avoid a second scan of target in select_idle_cpu Mel Gorman
@ 2021-08-04 13:11 ` Peter Zijlstra
2 siblings, 0 replies; 6+ messages in thread
From: Peter Zijlstra @ 2021-08-04 13:11 UTC (permalink / raw)
To: Mel Gorman
Cc: Ingo Molnar, Vincent Guittot, Valentin Schneider, Song Bao Hua, LKML
On Wed, Aug 04, 2021 at 12:58:55PM +0100, Mel Gorman wrote:
> This is the first two patches from "Modify and/or delete SIS_PROP" and
> focues on improving the hit rate for recent_used_cpu and avoids rescanning
> the target CPU when it's known to be busy. These are relatively simply
> patches in comparison to the full series which could be a regression
> magnet and that series had at least one mistake in it.
>
> In general, the usual suspects showed mostly small gains with a few
> exceptions. Usual suspects were NAS, hackbench, schbench (Facebook
> latency-sensitive workload), perf pipe, kernel building, git test suite,
> dbench and redis.
>
> From here, the next obvious candidates are either improving has_idle_cores
> or continuing to try remove/improve SIS_PROP.
Thanks!
^ permalink raw reply [flat|nested] 6+ messages in thread
* [tip: sched/core] sched/fair: Avoid a second scan of target in select_idle_cpu
2021-08-04 11:58 ` [PATCH 2/2] sched/fair: Avoid a second scan of target in select_idle_cpu Mel Gorman
@ 2021-08-05 9:34 ` tip-bot2 for Mel Gorman
0 siblings, 0 replies; 6+ messages in thread
From: tip-bot2 for Mel Gorman @ 2021-08-05 9:34 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Mel Gorman, Peter Zijlstra (Intel), x86, linux-kernel
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 56498cfb045d7147cdcba33795d19429afcd1d00
Gitweb: https://git.kernel.org/tip/56498cfb045d7147cdcba33795d19429afcd1d00
Author: Mel Gorman <mgorman@techsingularity.net>
AuthorDate: Wed, 04 Aug 2021 12:58:57 +01:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Wed, 04 Aug 2021 15:16:44 +02:00
sched/fair: Avoid a second scan of target in select_idle_cpu
When select_idle_cpu starts scanning for an idle CPU, it starts with
a target CPU that has already been checked by select_idle_sibling.
This patch starts with the next CPU instead.
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20210804115857.6253-3-mgorman@techsingularity.net
---
kernel/sched/fair.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 7ee394b..47a0fbf 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6220,7 +6220,7 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
time = cpu_clock(this);
}
- for_each_cpu_wrap(cpu, cpus, target) {
+ for_each_cpu_wrap(cpu, cpus, target + 1) {
if (has_idle_core) {
i = select_idle_core(p, cpu, cpus, &idle_cpu);
if ((unsigned int)i < nr_cpumask_bits)
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [tip: sched/core] sched/fair: Use prev instead of new target as recent_used_cpu
2021-08-04 11:58 ` [PATCH 1/2] sched/fair: Use prev instead of new target as recent_used_cpu Mel Gorman
@ 2021-08-05 9:34 ` tip-bot2 for Mel Gorman
0 siblings, 0 replies; 6+ messages in thread
From: tip-bot2 for Mel Gorman @ 2021-08-05 9:34 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Mel Gorman, Peter Zijlstra (Intel), x86, linux-kernel
The following commit has been merged into the sched/core branch of tip:
Commit-ID: 89aafd67f28c9e3b725aa30b44b7f61ad3e348ce
Gitweb: https://git.kernel.org/tip/89aafd67f28c9e3b725aa30b44b7f61ad3e348ce
Author: Mel Gorman <mgorman@techsingularity.net>
AuthorDate: Wed, 04 Aug 2021 12:58:56 +01:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Wed, 04 Aug 2021 15:16:44 +02:00
sched/fair: Use prev instead of new target as recent_used_cpu
After select_idle_sibling, p->recent_used_cpu is set to the
new target. However on the next wakeup, prev will be the same as
recent_used_cpu unless the load balancer has moved the task since the
last wakeup. It still works, but is less efficient than it could be.
This patch preserves recent_used_cpu for longer.
The impact on SIS efficiency is tiny so the SIS statistic patches were
used to track the hit rate for using recent_used_cpu. With perf bench
pipe on a 2-socket Cascadelake machine, the hit rate went from 57.14%
to 85.32%. For more intensive wakeup loads like hackbench, the hit rate
is almost negligible but rose from 0.21% to 6.64%. For scaling loads
like tbench, the hit rate goes from almost 0% to 25.42% overall. Broadly
speaking, on tbench, the success rate is much higher for lower thread
counts and drops to almost 0 as the workload scales to towards saturation.
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20210804115857.6253-2-mgorman@techsingularity.net
---
kernel/sched/fair.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 13c3fd4..7ee394b 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6347,6 +6347,7 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
/* Check a recently used CPU as a potential idle candidate: */
recent_used_cpu = p->recent_used_cpu;
+ p->recent_used_cpu = prev;
if (recent_used_cpu != prev &&
recent_used_cpu != target &&
cpus_share_cache(recent_used_cpu, target) &&
@@ -6873,9 +6874,6 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags)
} else if (wake_flags & WF_TTWU) { /* XXX always ? */
/* Fast path */
new_cpu = select_idle_sibling(p, prev_cpu, new_cpu);
-
- if (want_affine)
- current->recent_used_cpu = cpu;
}
rcu_read_unlock();
^ permalink raw reply related [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-08-05 9:34 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-04 11:58 [PATCH 0/2] Reduce SIS scanning Mel Gorman
2021-08-04 11:58 ` [PATCH 1/2] sched/fair: Use prev instead of new target as recent_used_cpu Mel Gorman
2021-08-05 9:34 ` [tip: sched/core] " tip-bot2 for Mel Gorman
2021-08-04 11:58 ` [PATCH 2/2] sched/fair: Avoid a second scan of target in select_idle_cpu Mel Gorman
2021-08-05 9:34 ` [tip: sched/core] " tip-bot2 for Mel Gorman
2021-08-04 13:11 ` [PATCH 0/2] Reduce SIS scanning Peter Zijlstra
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.