From: Mike Galbraith <umgwanakikbuti@gmail.com>
To: Chris Metcalf <cmetcalf@ezchip.com>
Cc: "Peter Zijlstra (Intel)" <peterz@infradead.org>,
Frederic Weisbecker <fweisbec@gmail.com>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
Martin Schwidefsky <schwidefsky@de.ibm.com>,
Ingo Molnar <mingo@redhat.com>,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH 2/2] nohz: make nohz_full imply isolcpus
Date: Sat, 04 Apr 2015 05:43:09 +0200 [thread overview]
Message-ID: <1428118989.3916.34.camel@gmail.com> (raw)
In-Reply-To: <1428113039.3189.6.camel@gmail.com>
On Sat, 2015-04-04 at 04:03 +0200, Mike Galbraith wrote:
> On Fri, 2015-04-03 at 15:21 -0400, Chris Metcalf wrote:
> > On 04/03/2015 02:08 PM, Mike Galbraith wrote:
> > > On Fri, 2015-04-03 at 12:24 -0400, cmetcalf@ezchip.comwrote:
> > > > From: Chris Metcalf <cmetcalf@ezchip.com>
> > > >
> > > > It's not clear that nohz_full is useful without isolcpus also
> > > > set, since otherwise the scheduler has to run periodically to
> > > > try to determine whether to steal work from other cores.
> > > >
> > > > Signed-off-by: Chris Metcalf <cmetcalf@ezchip.com>
> > > Ack! nohz_full= as currently defined makes zero sense when the
> > > cpu
> > > set (which should be spelled cpuset) remains connected to the
> > > scheduler. Perturbation of tasks to PREVENT cpu domination is
> > > what
> > > the scheduler does for a living. Sprinkling microsecond savers
> > > all
> > > over the kernel is pretty silly if you don't shut down the
> > > mother
> > > lode
> > > of perturbation.
> >
> > Sounds like a thumbs up for this patch, then? :-)
>
> Yup. The other thumb turns in the up direction when folks start
> spelling cpuset properly ;-) Static isolcpus was supposed to go
> away.
Speaking of microsecond savers, the (ick) deferment experiment below
cut 60 core jitter in half. Shooting the clocksource watchdog fixes
alternating ~15us/~5us tick on my desktop box.
With workqueue twiddles and whatnot floating around, the thing is
starting to look viable.
---
kernel/sched/core.c | 5 +++--
kernel/time/clocksource.c | 5 +++++
2 files changed, 8 insertions(+), 2 deletions(-)
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2604,12 +2604,13 @@ u64 scheduler_tick_max_deferment(void)
struct rq *rq = this_rq();
unsigned long next, now = ACCESS_ONCE(jiffies);
- next = rq->last_sched_tick + HZ;
+ next = (rq->last_sched_tick + HZ) | (rq->clock & 0x3f);
if (time_before_eq(next, now))
return 0;
- return jiffies_to_nsecs(next - now);
+ /* Add noise to avoid CPUs colliding at tick boundaries */
+ return jiffies_to_nsecs(next - now) | (rq->clock & 0xfffff);
}
#endif
--- a/kernel/time/clocksource.c
+++ b/kernel/time/clocksource.c
@@ -267,8 +267,13 @@ static void clocksource_watchdog(unsigne
* to each other.
*/
next_cpu = cpumask_next(raw_smp_processor_id(), cpu_online_mask);
+skip_nohz_full:
if (next_cpu >= nr_cpu_ids)
next_cpu = cpumask_first(cpu_online_mask);
+ if (next_cpu && tick_nohz_full_cpu(next_cpu)) {
+ next_cpu = cpumask_next(next_cpu, cpu_online_mask);
+ goto skip_nohz_full;
+ }
watchdog_timer.expires += WATCHDOG_INTERVAL;
add_timer_on(&watchdog_timer, next_cpu);
out:
next prev parent reply other threads:[~2015-04-04 3:43 UTC|newest]
Thread overview: 89+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-04-03 16:24 [PATCH 1/2] nohz: add tick_nohz_full_set_cpus() API cmetcalf
2015-04-03 16:24 ` [PATCH 2/2] nohz: make nohz_full imply isolcpus cmetcalf
2015-04-03 17:42 ` Frederic Weisbecker
2015-04-03 19:20 ` Chris Metcalf
2015-04-04 14:10 ` Rik van Riel
2015-04-04 17:09 ` Chris Metcalf
2015-04-05 5:05 ` Ingo Molnar
2015-04-06 17:15 ` Chris Metcalf
2015-04-06 18:16 ` [PATCH v2 1/2] nohz: add tick_nohz_full_clear_cpus() and _set_cpus() APIs cmetcalf
2015-04-06 18:16 ` [PATCH v2 2/2] nohz: make nohz_full imply isolcpus cmetcalf
2015-04-07 22:29 ` Frederic Weisbecker
2015-04-08 9:41 ` Peter Zijlstra
2015-04-08 14:04 ` Chris Metcalf
2015-04-08 14:26 ` Peter Zijlstra
2015-04-08 15:21 ` Chris Metcalf
2015-04-08 17:24 ` Frederic Weisbecker
2015-04-08 19:20 ` [PATCH v3 1/2] nohz: add tick_nohz_full_clear_cpus() and _set_cpus() APIs cmetcalf
2015-04-08 19:20 ` [PATCH v3 2/2] nohz: set isolcpus when nohz_full is set cmetcalf
2015-04-08 17:27 ` [PATCH v2 2/2] nohz: make nohz_full imply isolcpus Peter Zijlstra
2015-04-08 18:12 ` Chris Metcalf
2015-04-09 8:29 ` Peter Zijlstra
2015-04-09 12:02 ` Chris Metcalf
2015-04-09 12:45 ` Frederic Weisbecker
2015-04-09 16:59 ` [PATCH v5] nohz: set isolcpus when nohz_full is set Chris Metcalf
2015-04-09 17:12 ` Peter Zijlstra
2015-04-10 1:05 ` Mike Galbraith
2015-04-10 15:33 ` Chris Metcalf
2015-04-10 15:57 ` Mike Galbraith
2015-04-09 17:00 ` [PATCH v4 1/2] nohz: add tick_nohz_full_cpumask_or() and _andnot() APIs Chris Metcalf
2015-04-09 17:00 ` [PATCH v4 2/2] nohz: set isolcpus when nohz_full is set Chris Metcalf
2015-04-09 17:07 ` [PATCH v4 1/2] nohz: add tick_nohz_full_cpumask_or() and _andnot() APIs Peter Zijlstra
2015-04-09 17:24 ` Chris Metcalf
2015-04-09 17:42 ` Peter Zijlstra
2015-04-09 18:01 ` [PATCH v6 1/2] nohz: add tick_nohz_full_add_cpus_to() and _remove_cpus_from() APIs Chris Metcalf
2015-04-09 18:01 ` [PATCH v6 2/2] nohz: set isolcpus when nohz_full is set Chris Metcalf
2015-04-10 1:37 ` Frederic Weisbecker
2015-04-10 20:53 ` [PATCH v7 1/2] nohz: add tick_nohz_full_add_cpus_to() and _remove_cpus_from() APIs Chris Metcalf
2015-04-10 20:53 ` [PATCH v7 2/2] nohz: set isolcpus when nohz_full is set Chris Metcalf
2015-04-14 0:37 ` Frederic Weisbecker
2015-04-14 15:17 ` [PATCH v8 1/2] nohz: add tick_nohz_full_add_cpus_to() API Chris Metcalf
2015-04-14 15:17 ` [PATCH v8 2/2] nohz: set isolcpus when nohz_full is set Chris Metcalf
2015-04-14 15:26 ` Frederic Weisbecker
2015-04-14 16:45 ` Peter Zijlstra
2015-04-14 0:33 ` [PATCH v7 1/2] nohz: add tick_nohz_full_add_cpus_to() and _remove_cpus_from() APIs Frederic Weisbecker
2015-04-14 0:49 ` Chris Metcalf
2015-04-14 15:34 ` Frederic Weisbecker
2015-04-10 1:34 ` [PATCH v6 " Frederic Weisbecker
2015-04-10 15:31 ` Chris Metcalf
2015-04-06 18:29 ` [PATCH v2 1/2] nohz: add tick_nohz_full_clear_cpus() and _set_cpus() APIs Frederic Weisbecker
2015-04-06 19:09 ` Chris Metcalf
2015-04-07 9:33 ` Ingo Molnar
2015-04-03 18:08 ` [PATCH 2/2] nohz: make nohz_full imply isolcpus Mike Galbraith
2015-04-03 19:21 ` Chris Metcalf
2015-04-04 2:03 ` Mike Galbraith
2015-04-04 3:43 ` Mike Galbraith [this message]
2015-04-06 19:28 ` Rik van Riel
2015-04-07 3:10 ` Mike Galbraith
2015-05-06 16:04 [GIT PULL] nohz: A few improvements v4 Frederic Weisbecker
2015-05-06 16:04 ` [PATCH 1/4] context_tracking: Protect against recursion Frederic Weisbecker
2015-05-07 9:58 ` Ingo Molnar
2015-05-07 11:53 ` Frederic Weisbecker
2015-05-07 11:31 ` [tip:timers/nohz] " tip-bot for Frederic Weisbecker
2015-05-06 16:04 ` [PATCH 2/4] context_tracking: Inherit TIF_NOHZ through forks instead of context switches Frederic Weisbecker
2015-05-07 11:31 ` [tip:timers/nohz] " tip-bot for Frederic Weisbecker
2015-05-06 16:04 ` [PATCH 3/4] nohz: Add tick_nohz_full_add_cpus_to() API Frederic Weisbecker
2015-05-07 11:31 ` [tip:timers/nohz] " tip-bot for Chris Metcalf
2015-05-06 16:04 ` [PATCH 4/4] nohz: Set isolcpus when nohz_full is set Frederic Weisbecker
2015-05-07 11:32 ` [tip:timers/nohz] " tip-bot for Chris Metcalf
2015-05-16 19:39 ` [PATCH 4/4] " Sasha Levin
2015-05-17 5:30 ` Mike Galbraith
2015-05-18 2:17 ` Rik van Riel
2015-05-18 3:29 ` Mike Galbraith
2015-05-18 14:07 ` Rik van Riel
2015-05-18 14:22 ` Mike Galbraith
2015-05-18 14:52 ` Rik van Riel
2015-05-19 2:30 ` Mike Galbraith
2015-06-12 19:12 ` Rik van Riel
2015-05-20 20:38 ` Afzal Mohammed
2015-05-20 21:00 ` Paul E. McKenney
2015-05-21 12:12 ` Afzal Mohammed
2015-05-21 12:57 ` Paul E. McKenney
2015-05-21 13:06 ` Frederic Weisbecker
2015-05-21 13:27 ` Paul E. McKenney
2015-05-21 13:29 ` Afzal Mohammed
2015-05-21 14:14 ` Paul E. McKenney
2015-05-21 14:46 ` Frederic Weisbecker
2015-05-21 18:59 ` Mike Galbraith
2015-05-22 14:39 ` Frederic Weisbecker
2015-05-22 15:20 ` Mike Galbraith
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1428118989.3916.34.camel@gmail.com \
--to=umgwanakikbuti@gmail.com \
--cc=cmetcalf@ezchip.com \
--cc=fweisbec@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=rafael.j.wysocki@intel.com \
--cc=schwidefsky@de.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).