From: Dietmar Eggemann <dietmar.eggemann@arm.com>
To: Ingo Molnar <mingo@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Vincent Guittot <vincent.guittot@linaro.org>
Cc: Qais Yousef <qyousef@layalina.io>,
Kajetan Puchalski <kajetan.puchalski@arm.com>,
Morten Rasmussen <morten.rasmussen@arm.com>,
Vincent Donnefort <vdonnefort@google.com>,
Quentin Perret <qperret@google.com>,
Abhijeet Dharmapurikar <adharmap@quicinc.com>,
linux-kernel@vger.kernel.org
Subject: [PATCH 1/1] sched: Consider CPU contention in frequency & load-balance busiest CPU selection
Date: Thu, 6 Apr 2023 17:50:30 +0200 [thread overview]
Message-ID: <20230406155030.1989554-2-dietmar.eggemann@arm.com> (raw)
In-Reply-To: <20230406155030.1989554-1-dietmar.eggemann@arm.com>
Use new cpu_boosted_util_cfs() instead of cpu_util_cfs().
The former returns max(util_avg, runnable_avg) capped by max CPU
capacity. CPU contention is thereby considered through runnable_avg.
The change in load-balance only affects migration type `migrate_util`.
Suggested-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
---
kernel/sched/cpufreq_schedutil.c | 3 ++-
kernel/sched/fair.c | 2 +-
kernel/sched/sched.h | 19 +++++++++++++++++++
3 files changed, 22 insertions(+), 2 deletions(-)
diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c
index e3211455b203..728b186cd367 100644
--- a/kernel/sched/cpufreq_schedutil.c
+++ b/kernel/sched/cpufreq_schedutil.c
@@ -158,7 +158,8 @@ static void sugov_get_util(struct sugov_cpu *sg_cpu)
struct rq *rq = cpu_rq(sg_cpu->cpu);
sg_cpu->bw_dl = cpu_bw_dl(rq);
- sg_cpu->util = effective_cpu_util(sg_cpu->cpu, cpu_util_cfs(sg_cpu->cpu),
+ sg_cpu->util = effective_cpu_util(sg_cpu->cpu,
+ cpu_boosted_util_cfs(sg_cpu->cpu),
FREQUENCY_UTIL, NULL);
}
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index bc358dc4faeb..5ae36224a1c2 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -10481,7 +10481,7 @@ static struct rq *find_busiest_queue(struct lb_env *env,
break;
case migrate_util:
- util = cpu_util_cfs(i);
+ util = cpu_boosted_util_cfs(i);
/*
* Don't try to pull utilization from a CPU with one
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 060616944d7a..f42c859579d9 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -2994,6 +2994,25 @@ static inline unsigned long cpu_util_cfs(int cpu)
return min(util, capacity_orig_of(cpu));
}
+/*
+ * cpu_boosted_util_cfs() - Estimates the amount of CPU capacity used by
+ * CFS tasks.
+ *
+ * Similar to cpu_util_cfs() but also take possible CPU contention into
+ * consideration.
+ */
+static inline unsigned long cpu_boosted_util_cfs(int cpu)
+{
+ unsigned long runnable;
+ struct cfs_rq *cfs_rq;
+
+ cfs_rq = &cpu_rq(cpu)->cfs;
+ runnable = READ_ONCE(cfs_rq->avg.runnable_avg);
+ runnable = min(runnable, capacity_orig_of(cpu));
+
+ return max(cpu_util_cfs(cpu), runnable);
+}
+
static inline unsigned long cpu_util_rt(struct rq *rq)
{
return READ_ONCE(rq->avg_rt.util_avg);
--
2.25.1
next prev parent reply other threads:[~2023-04-06 15:50 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-06 15:50 [PATCH 0/1] sched: Consider CPU contention in frequency & load-balance busiest CPU selection Dietmar Eggemann
2023-04-06 15:50 ` Dietmar Eggemann [this message]
2023-04-29 14:58 ` [PATCH 1/1] " Peter Zijlstra
2023-05-03 17:13 ` Dietmar Eggemann
2023-05-04 15:23 ` Qais Yousef
2023-05-11 15:25 ` Dietmar Eggemann
2023-05-15 19:18 ` Qais Yousef
2023-05-03 16:08 ` Vincent Guittot
2023-05-04 17:11 ` Dietmar Eggemann
2023-05-05 8:22 ` Vincent Guittot
2023-05-05 18:16 ` Dietmar Eggemann
2023-05-05 7:10 ` Chen Yu
2023-05-05 18:02 ` Dietmar Eggemann
2023-05-07 2:19 ` Chen Yu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230406155030.1989554-2-dietmar.eggemann@arm.com \
--to=dietmar.eggemann@arm.com \
--cc=adharmap@quicinc.com \
--cc=kajetan.puchalski@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=morten.rasmussen@arm.com \
--cc=peterz@infradead.org \
--cc=qperret@google.com \
--cc=qyousef@layalina.io \
--cc=vdonnefort@google.com \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).