From: tip-bot for Morten Rasmussen <tipbot@zytor.com>
To: linux-tip-commits@vger.kernel.org
Cc: tglx@linutronix.de, hpa@zytor.com, linux-kernel@vger.kernel.org,
torvalds@linux-foundation.org, mingo@kernel.org,
peterz@infradead.org, morten.rasmussen@arm.com
Subject: [tip:sched/core] sched/topology: Add static_key for asymmetric CPU capacity optimizations
Date: Mon, 10 Sep 2018 03:13:19 -0700 [thread overview]
Message-ID: <tip-df054e8445a4011e3d693c2268129c0456108663@git.kernel.org> (raw)
In-Reply-To: <1530699470-29808-2-git-send-email-morten.rasmussen@arm.com>
Commit-ID: df054e8445a4011e3d693c2268129c0456108663
Gitweb: https://git.kernel.org/tip/df054e8445a4011e3d693c2268129c0456108663
Author: Morten Rasmussen <morten.rasmussen@arm.com>
AuthorDate: Wed, 4 Jul 2018 11:17:39 +0100
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Mon, 10 Sep 2018 11:05:48 +0200
sched/topology: Add static_key for asymmetric CPU capacity optimizations
The existing asymmetric CPU capacity code should cause minimal overhead
for others. Putting it behind a static_key, it has been done for SMT
optimizations, would make it easier to extend and improve without
causing harm to others moving forward.
Signed-off-by: Morten Rasmussen <morten.rasmussen@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: dietmar.eggemann@arm.com
Cc: gaku.inami.xh@renesas.com
Cc: valentin.schneider@arm.com
Cc: vincent.guittot@linaro.org
Link: http://lkml.kernel.org/r/1530699470-29808-2-git-send-email-morten.rasmussen@arm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
kernel/sched/fair.c | 3 +++
kernel/sched/sched.h | 1 +
kernel/sched/topology.c | 9 ++++++++-
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index f808ddf2a868..3e5071aeb117 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -6188,6 +6188,9 @@ static int wake_cap(struct task_struct *p, int cpu, int prev_cpu)
{
long min_cap, max_cap;
+ if (!static_branch_unlikely(&sched_asym_cpucapacity))
+ return 0;
+
min_cap = min(capacity_orig_of(prev_cpu), capacity_orig_of(cpu));
max_cap = cpu_rq(cpu)->rd->max_cpu_capacity;
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 4a2e8cae63c4..0f36adc31ba5 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1185,6 +1185,7 @@ DECLARE_PER_CPU(int, sd_llc_id);
DECLARE_PER_CPU(struct sched_domain_shared *, sd_llc_shared);
DECLARE_PER_CPU(struct sched_domain *, sd_numa);
DECLARE_PER_CPU(struct sched_domain *, sd_asym);
+extern struct static_key_false sched_asym_cpucapacity;
struct sched_group_capacity {
atomic_t ref;
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 5c4d583d53ee..b0cdf5e95bda 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -398,6 +398,7 @@ DEFINE_PER_CPU(int, sd_llc_id);
DEFINE_PER_CPU(struct sched_domain_shared *, sd_llc_shared);
DEFINE_PER_CPU(struct sched_domain *, sd_numa);
DEFINE_PER_CPU(struct sched_domain *, sd_asym);
+DEFINE_STATIC_KEY_FALSE(sched_asym_cpucapacity);
static void update_top_cache_domain(int cpu)
{
@@ -1705,6 +1706,7 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
struct rq *rq = NULL;
int i, ret = -ENOMEM;
struct sched_domain_topology_level *tl_asym;
+ bool has_asym = false;
alloc_state = __visit_domain_allocation_hell(&d, cpu_map);
if (alloc_state != sa_rootdomain)
@@ -1720,8 +1722,10 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
for_each_sd_topology(tl) {
int dflags = 0;
- if (tl == tl_asym)
+ if (tl == tl_asym) {
dflags |= SD_ASYM_CPUCAPACITY;
+ has_asym = true;
+ }
sd = build_sched_domain(tl, cpu_map, attr, sd, dflags, i);
@@ -1773,6 +1777,9 @@ build_sched_domains(const struct cpumask *cpu_map, struct sched_domain_attr *att
}
rcu_read_unlock();
+ if (has_asym)
+ static_branch_enable_cpuslocked(&sched_asym_cpucapacity);
+
if (rq && sched_debug_enabled) {
pr_info("root domain span: %*pbl (max cpu_capacity = %lu)\n",
cpumask_pr_args(cpu_map), rq->rd->max_cpu_capacity);
next prev parent reply other threads:[~2018-09-10 10:13 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-04 10:17 [PATCHv4 00/12] sched/fair: Migrate 'misfit' tasks on asymmetric capacity systems Morten Rasmussen
2018-07-04 10:17 ` [PATCHv4 01/12] sched: Add static_key for asymmetric cpu capacity optimizations Morten Rasmussen
2018-07-31 10:59 ` Peter Zijlstra
2018-08-02 15:15 ` Morten Rasmussen
2018-09-10 10:13 ` tip-bot for Morten Rasmussen [this message]
2018-07-04 10:17 ` [PATCHv4 02/12] sched/fair: Add group_misfit_task load-balance type Morten Rasmussen
2018-09-10 10:13 ` [tip:sched/core] sched/fair: Add 'group_misfit_task' " tip-bot for Morten Rasmussen
2018-07-04 10:17 ` [PATCHv4 03/12] sched: Add sched_group per-cpu max capacity Morten Rasmussen
2018-09-10 10:14 ` [tip:sched/core] sched/fair: Add sched_group per-CPU " tip-bot for Morten Rasmussen
2018-07-04 10:17 ` [PATCHv4 04/12] sched/fair: Consider misfit tasks when load-balancing Morten Rasmussen
2018-09-10 10:14 ` [tip:sched/core] " tip-bot for Morten Rasmussen
2018-07-04 10:17 ` [PATCHv4 05/12] sched/fair: Kick nohz balance if rq->misfit_task_load Morten Rasmussen
2018-09-10 10:15 ` [tip:sched/core] " tip-bot for Valentin Schneider
2018-07-04 10:17 ` [PATCHv4 06/12] sched/fair: Change prefer_sibling type to bool Morten Rasmussen
2018-09-10 10:15 ` [tip:sched/core] sched/fair: Change 'prefer_sibling' " tip-bot for Valentin Schneider
2018-07-04 10:17 ` [PATCHv4 07/12] sched: Change root_domain->overload type to int Morten Rasmussen
2018-09-10 10:16 ` [tip:sched/core] sched/core: " tip-bot for Valentin Schneider
2018-07-04 10:17 ` [PATCHv4 08/12] sched: Wrap rq->rd->overload accesses with READ/WRITE_ONCE Morten Rasmussen
2018-09-10 10:17 ` [tip:sched/core] sched/fair: Wrap rq->rd->overload accesses with READ/WRITE_ONCE() tip-bot for Valentin Schneider
2018-07-04 10:17 ` [PATCHv4 09/12] sched/fair: Set rq->rd->overload when misfit Morten Rasmussen
2018-09-10 10:17 ` [tip:sched/core] " tip-bot for Valentin Schneider
2018-07-04 10:17 ` [PATCHv4 10/12] sched/fair: Don't move tasks to lower capacity cpus unless necessary Morten Rasmussen
2018-09-10 10:18 ` [tip:sched/core] sched/fair: Don't move tasks to lower capacity CPUs " tip-bot for Chris Redpath
2018-07-04 10:17 ` [PATCHv4 11/12] sched/core: Disable SD_ASYM_CPUCAPACITY for root_domains without asymmetry Morten Rasmussen
2018-07-05 13:31 ` Quentin Perret
2018-07-05 14:13 ` Morten Rasmussen
2018-07-05 15:03 ` Quentin Perret
2018-07-20 13:54 ` Morten Rasmussen
2018-07-04 10:17 ` [PATCHv4 12/12] sched/core: Disable SD_PREFER_SIBLING on asymmetric cpu capacity domains Morten Rasmussen
2018-07-06 10:18 ` Vincent Guittot
2018-07-06 14:31 ` Morten Rasmussen
2018-07-31 12:17 ` Vincent Guittot
2018-07-31 12:33 ` Valentin Schneider
2018-08-06 10:20 ` Vincent Guittot
2018-08-06 10:53 ` Valentin Schneider
2018-08-10 9:30 ` Vincent Guittot
2018-09-10 10:18 ` [tip:sched/core] sched/core: Disable SD_PREFER_SIBLING on asymmetric CPU " tip-bot for Morten Rasmussen
2018-07-06 10:18 ` [PATCHv4 00/12] sched/fair: Migrate 'misfit' tasks on asymmetric capacity systems Vincent Guittot
2018-07-09 15:08 ` Morten Rasmussen
2018-07-26 17:14 ` Valentin Schneider
2018-07-30 14:30 ` Dietmar Eggemann
2018-07-31 12:13 ` Vincent Guittot
2018-07-31 12:14 ` Dietmar Eggemann
2018-07-31 12:11 ` Vincent Guittot
2018-07-31 12:00 ` Peter Zijlstra
2018-07-31 12:10 ` Valentin Schneider
2018-07-31 12:50 ` Morten Rasmussen
2018-08-17 1:57 ` Joel Fernandes
2018-08-20 2:50 ` Gaku Inami
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=tip-df054e8445a4011e3d693c2268129c0456108663@git.kernel.org \
--to=tipbot@zytor.com \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=morten.rasmussen@arm.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).