From: Peter Zijlstra <peterz@infradead.org>
To: Mike Galbraith <mgalbraith@suse.de>
Cc: Suresh Siddha <suresh.b.siddha@intel.com>,
linux-kernel <linux-kernel@vger.kernel.org>,
Ingo Molnar <mingo@elte.hu>, Paul Turner <pjt@google.com>,
Chris Mason <chris.mason@oracle.com>,
Dave Kleikamp <dave.kleikamp@oracle.com>
Subject: Re: [patch 7/7] sched: only use TTWU_QUEUE when waker/wakee CPUs do not share top level cache
Date: Wed, 23 Nov 2011 13:08:16 +0100 [thread overview]
Message-ID: <1322050096.14799.49.camel@twins> (raw)
In-Reply-To: <1321971985.6855.26.camel@marge.simson.net>
I changed this a little, no point in iterating the domains when
SD_SHARE_PKG_RESOURCES fails.
---
Subject: sched: Only use TTWU_QUEUE when waker/wakee CPUs do not share top level cache
From: Mike Galbraith <mgalbraith@suse.de>
Date: Tue, 22 Nov 2011 15:26:25 +0100
TTWU_QUEUE IPI overhead was measured to be as much as 13% of netperf TCP_RR
overhead when waking to a shared cache. Don't IPI unless we're waking cross
cache, where it can be a winner.
Cc: Chris Mason <chris.mason@oracle.com>
Cc: Dave Kleikamp <dave.kleikamp@oracle.com>
Suggested-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: Mike Galbraith <efault@gmx.de>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/r/1321971985.6855.26.camel@marge.simson.net
---
kernel/sched/core.c | 26 +++++++++++++++++++++++++-
1 file changed, 25 insertions(+), 1 deletion(-)
Index: linux-2.6/kernel/sched/core.c
===================================================================
--- linux-2.6.orig/kernel/sched/core.c
+++ linux-2.6/kernel/sched/core.c
@@ -1480,12 +1480,36 @@ static int ttwu_activate_remote(struct t
#endif /* __ARCH_WANT_INTERRUPTS_ON_CTXSW */
#endif /* CONFIG_SMP */
+static int ttwu_share_cache(int this_cpu, int cpu)
+{
+#ifndef CONFIG_X86
+ struct sched_domain *sd;
+ int ret = 0;
+
+ rcu_read_lock();
+ for_each_domain(this_cpu, sd) {
+ if (!(sd->flags & SD_SHARE_PKG_RESOURCES))
+ break;
+
+ if (cpumask_test_cpu(cpu, sched_domain_span(sd))) {
+ ret = 1;
+ break;
+ }
+ }
+ rcu_read_unlock();
+
+ return ret;
+#else
+ return per_cpu(cpu_llc_id, this_cpu) == per_cpu(cpu_llc_id, cpu);
+#endif
+}
+
static void ttwu_queue(struct task_struct *p, int cpu)
{
struct rq *rq = cpu_rq(cpu);
#if defined(CONFIG_SMP)
- if (sched_feat(TTWU_QUEUE) && cpu != smp_processor_id()) {
+ if (sched_feat(TTWU_QUEUE) && !ttwu_share_cache(smp_processor_id(), cpu)) {
sched_clock_cpu(cpu); /* sync clocks x-cpu */
ttwu_queue_remote(p, cpu);
return;
next prev parent reply other threads:[~2011-11-23 12:08 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-11-15 9:46 sched: Avoid SMT siblings in select_idle_sibling() if possible Peter Zijlstra
2011-11-16 1:14 ` Suresh Siddha
2011-11-16 9:24 ` Mike Galbraith
2011-11-16 18:37 ` Suresh Siddha
2011-11-17 1:59 ` Mike Galbraith
2011-11-17 15:38 ` Mike Galbraith
2011-11-17 15:56 ` Peter Zijlstra
2011-11-17 16:38 ` Mike Galbraith
2011-11-17 17:36 ` Suresh Siddha
2011-11-18 15:14 ` Mike Galbraith
2011-11-18 15:17 ` [patch 1/6] sched: use rt.nr_cpus_allowed to recover select_task_rq() cycles Mike Galbraith
2011-11-18 15:35 ` Peter Zijlstra
2011-11-18 17:34 ` Mike Galbraith
2011-11-22 14:17 ` Mike Galbraith
2011-11-22 14:18 ` [patch 1/7] " Mike Galbraith
2011-12-06 9:50 ` [tip:sched/core] sched: Use " tip-bot for Mike Galbraith
2011-11-22 14:20 ` [patch 2/7] sched: save some hrtick_start_fair cycles Mike Galbraith
2011-12-06 20:20 ` [tip:sched/core] sched: Save " tip-bot for Mike Galbraith
2011-11-22 14:21 ` [patch 3/7] sched: set skip_clock_update in yield_task_fair() Mike Galbraith
2011-11-23 11:53 ` Peter Zijlstra
2011-11-23 12:06 ` Mike Galbraith
2011-11-23 14:48 ` Peter Zijlstra
2011-11-24 3:50 ` Mike Galbraith
2011-11-24 10:12 ` Peter Zijlstra
2011-11-25 6:39 ` Mike Galbraith
2011-12-06 9:51 ` [tip:sched/core] sched: Set " tip-bot for Mike Galbraith
2011-11-22 14:22 ` [patch 4/7] sched: convert rq->avg_idle to rq->avg_event Mike Galbraith
2011-11-23 11:55 ` Peter Zijlstra
2011-11-23 12:09 ` Mike Galbraith
2011-11-23 12:27 ` Peter Zijlstra
2011-11-23 12:57 ` Mike Galbraith
2011-11-23 14:21 ` Mike Galbraith
2011-11-22 14:23 ` [patch 5/7] sched: ratelimit select_idle_sibling()for sync wakeups Mike Galbraith
2011-11-22 14:24 ` [patch 6/7] sched: use rq->avg_event to resurrect nohz ratelimiting Mike Galbraith
2011-11-23 11:57 ` Peter Zijlstra
2011-11-23 12:35 ` Mike Galbraith
2011-11-22 14:26 ` [patch 7/7] sched: only use TTWU_QUEUE when waker/wakee CPUs do not share top level cache Mike Galbraith
2011-11-23 12:08 ` Peter Zijlstra [this message]
2011-11-18 15:39 ` [patch 1/6] sched: use rt.nr_cpus_allowed to recover select_task_rq() cycles Hillf Danton
2011-11-18 15:18 ` [patch 2/6] sched: convert rq->avg_idle to rq->avg_event Mike Galbraith
2011-11-18 15:19 ` [patch 3/6] sched: use rq->avg_event to resurrect nohz ratelimiting Mike Galbraith
2011-11-18 15:36 ` Peter Zijlstra
2011-11-18 17:42 ` Mike Galbraith
2011-11-19 0:51 ` Van De Ven, Arjan
2011-11-19 4:15 ` Mike Galbraith
2011-11-18 15:20 ` [patch 4/6] sched: ratelimit select_idle_sibling()for sync wakeups Mike Galbraith
2011-11-18 15:22 ` [patch 5/6] sched: save some hrtick_start_fair cycles Mike Galbraith
2011-11-18 15:23 ` [patch 6/6] sched: set skip_clock_update in yield_task_fair() Mike Galbraith
2012-02-20 14:41 ` sched: Avoid SMT siblings in select_idle_sibling() if possible Peter Zijlstra
2012-02-20 15:03 ` Srivatsa Vaddagiri
2012-02-20 18:25 ` Mike Galbraith
2012-02-21 0:06 ` Srivatsa Vaddagiri
2012-02-21 6:37 ` Mike Galbraith
2012-02-21 8:09 ` Srivatsa Vaddagiri
2012-02-20 18:14 ` Mike Galbraith
2012-02-20 18:15 ` Peter Zijlstra
2012-02-20 19:07 ` Peter Zijlstra
2012-02-21 5:43 ` Mike Galbraith
2012-02-21 8:32 ` Srivatsa Vaddagiri
2012-02-21 9:21 ` Mike Galbraith
2012-02-21 10:37 ` Peter Zijlstra
2012-02-21 14:58 ` Srivatsa Vaddagiri
2012-02-23 10:49 ` Srivatsa Vaddagiri
2012-02-23 11:19 ` Ingo Molnar
2012-02-23 12:18 ` Srivatsa Vaddagiri
2012-02-23 11:20 ` Srivatsa Vaddagiri
2012-02-23 11:26 ` Ingo Molnar
2012-02-23 11:32 ` Srivatsa Vaddagiri
2012-02-23 16:17 ` Ingo Molnar
2012-02-23 11:21 ` Mike Galbraith
2012-02-25 6:54 ` Srivatsa Vaddagiri
2012-02-25 8:30 ` Mike Galbraith
2012-02-27 22:11 ` Suresh Siddha
2012-02-28 5:05 ` Mike Galbraith
2011-11-17 19:08 ` Suresh Siddha
2011-11-18 15:12 ` Peter Zijlstra
2011-11-18 15:26 ` Mike Galbraith
2011-12-06 9:49 ` [tip:sched/core] sched: Clean up domain traversal in select_idle_sibling() tip-bot for Suresh Siddha
2011-11-18 23:40 ` [tip:sched/core] sched: Avoid SMT siblings in select_idle_sibling() if possible tip-bot for Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1322050096.14799.49.camel@twins \
--to=peterz@infradead.org \
--cc=chris.mason@oracle.com \
--cc=dave.kleikamp@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgalbraith@suse.de \
--cc=mingo@elte.hu \
--cc=pjt@google.com \
--cc=suresh.b.siddha@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).