From: Jiri Olsa <jolsa@kernel.org>
To: Ingo Molnar <mingo@kernel.org>, Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: lkml <linux-kernel@vger.kernel.org>,
James Hartsock <hartsjc@redhat.com>,
Rik van Riel <riel@redhat.com>,
Srivatsa Vaddagiri <vatsa@linux.vnet.ibm.com>,
Kirill Tkhai <ktkhai@parallels.com>
Subject: [PATCH 2/4] sched/fair: Introduce idle enter/exit balance callbacks
Date: Mon, 20 Jun 2016 14:15:12 +0200 [thread overview]
Message-ID: <1466424914-8981-3-git-send-email-jolsa@kernel.org> (raw)
In-Reply-To: <1466424914-8981-1-git-send-email-jolsa@kernel.org>
Introducing idle enter/exit balance callbacks to keep
balance.idle_cpus_mask cpumask of current idle cpus
in system.
It's used only when REBALANCE_AFFINITY feature is
switched on. The code functionality of this feature
is introduced in following patch.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
kernel/sched/fair.c | 32 ++++++++++++++++++++++++++++++++
kernel/sched/idle.c | 2 ++
kernel/sched/sched.h | 3 +++
3 files changed, 37 insertions(+)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index f19c9435c64d..78c4127f2f3a 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7802,6 +7802,37 @@ static inline int on_null_domain(struct rq *rq)
return unlikely(!rcu_dereference_sched(rq->sd));
}
+static struct {
+ cpumask_var_t idle_cpus_mask;
+ atomic_t nr_cpus;
+} balance ____cacheline_aligned;
+
+void sched_idle_enter(int cpu)
+{
+ if (!sched_feat(REBALANCE_AFFINITY))
+ return;
+
+ if (!cpu_active(cpu))
+ return;
+
+ if (on_null_domain(cpu_rq(cpu)))
+ return;
+
+ cpumask_set_cpu(cpu, balance.idle_cpus_mask);
+ atomic_inc(&balance.nr_cpus);
+}
+
+void sched_idle_exit(int cpu)
+{
+ if (!sched_feat(REBALANCE_AFFINITY))
+ return;
+
+ if (likely(cpumask_test_cpu(cpu, balance.idle_cpus_mask))) {
+ cpumask_clear_cpu(cpu, balance.idle_cpus_mask);
+ atomic_dec(&balance.nr_cpus);
+ }
+}
+
#ifdef CONFIG_NO_HZ_COMMON
/*
* idle load balancing details
@@ -8731,6 +8762,7 @@ __init void init_sched_fair_class(void)
nohz.next_balance = jiffies;
zalloc_cpumask_var(&nohz.idle_cpus_mask, GFP_NOWAIT);
#endif
+ zalloc_cpumask_var(&balance.idle_cpus_mask, GFP_NOWAIT);
#endif /* SMP */
}
diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
index db4ff7c100b9..0cc0109eacd3 100644
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -216,6 +216,7 @@ static void cpu_idle_loop(void)
__current_set_polling();
quiet_vmstat();
tick_nohz_idle_enter();
+ sched_idle_enter(cpu);
while (!need_resched()) {
check_pgt_cache();
@@ -256,6 +257,7 @@ static void cpu_idle_loop(void)
*/
preempt_set_need_resched();
tick_nohz_idle_exit();
+ sched_idle_exit(cpu);
__current_clr_polling();
/*
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 72f1f3087b04..d1a6224cd140 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1692,6 +1692,9 @@ extern void init_dl_rq(struct dl_rq *dl_rq);
extern void cfs_bandwidth_usage_inc(void);
extern void cfs_bandwidth_usage_dec(void);
+void sched_idle_enter(int cpu);
+void sched_idle_exit(int cpu);
+
#ifdef CONFIG_NO_HZ_COMMON
enum rq_nohz_flag_bits {
NOHZ_TICK_STOPPED,
--
2.4.11
next prev parent reply other threads:[~2016-06-20 12:23 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-20 12:15 [RFC 0/4] sched/fair: Rebalance tasks based on affinity Jiri Olsa
2016-06-20 12:15 ` [PATCH 1/4] sched/fair: Introduce sched_entity::dont_balance Jiri Olsa
2016-06-20 14:28 ` Peter Zijlstra
2016-06-20 15:59 ` Jiri Olsa
2016-06-21 8:09 ` Peter Zijlstra
2016-06-20 12:15 ` Jiri Olsa [this message]
2016-06-20 14:30 ` [PATCH 2/4] sched/fair: Introduce idle enter/exit balance callbacks Peter Zijlstra
2016-06-20 16:27 ` Jiri Olsa
2016-06-21 8:12 ` Peter Zijlstra
2016-06-20 12:15 ` [PATCH 3/4] sched/fair: Add REBALANCE_AFFINITY rebalancing code Jiri Olsa
2016-06-30 10:58 ` Peter Zijlstra
2016-07-01 7:35 ` Jiri Olsa
2016-07-01 8:24 ` Peter Zijlstra
[not found] ` <CAM1qU8Ms2ZO5fnYRVX51uiBC5pZX6Hpa2W3Wqj5NjnYp3t8kOA@mail.gmail.com>
2016-07-01 14:58 ` Peter Zijlstra
2016-06-20 12:15 ` [PATCH 4/4] sched/fair: Add schedstat debug values for REBALANCE_AFFINITY Jiri Olsa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1466424914-8981-3-git-send-email-jolsa@kernel.org \
--to=jolsa@kernel.org \
--cc=a.p.zijlstra@chello.nl \
--cc=hartsjc@redhat.com \
--cc=ktkhai@parallels.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=riel@redhat.com \
--cc=vatsa@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).