All of lore.kernel.org
 help / color / mirror / Atom feed
From: Christoph Lameter <cl@linux.com>
To: Mike Galbraith <efault@gmx.de>
Cc: Frederic Weisbecker <fweisbec@gmail.com>,
	LKML <linux-kernel@vger.kernel.org>,
	linaro-sched-sig@lists.linaro.org,
	Alessio Igor Bogani <abogani@kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Avi Kivity <avi@redhat.com>, Chris Metcalf <cmetcalf@tilera.com>,
	Daniel Lezcano <daniel.lezcano@linaro.org>,
	Geoff Levand <geoff@infradead.org>,
	Gilad Ben Yossef <gilad@benyossef.com>,
	Ingo Molnar <mingo@kernel.org>,
	Max Krasnyansky <maxk@qualcomm.com>,
	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Stephen Hemminger <shemminger@vyatta.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Sven-Thorsten Dietrich <thebigcorporation@gmail.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Zen Lin <zen@openhuawei.org>
Subject: Re: [PATCH 07/32] cpuset: Set up interface for nohz flag
Date: Thu, 22 Mar 2012 11:26:14 -0500 (CDT)	[thread overview]
Message-ID: <alpine.DEB.2.00.1203221118550.25011@router.home> (raw)
In-Reply-To: <1332389033.5759.52.camel@marge.simpson.net>

On Thu, 22 Mar 2012, Mike Galbraith wrote:

> > > We use here a per cpu refcounter. As long as a CPU
> > > is contained into at least one cpuset that has the
> > > nohz flag set, it is part of the set of CPUs that
> > > run into adaptive nohz mode.
> >
> > What are the drawbacks for nohz?
>
> For nohz in general, latency.  To make it at all usable for rt loads, I

Well nohz while a process is running on a dedicated cpu means the cpu is
running full power and no disruptions occur. This is a tremendous benefit.

> had to make isolated cores immune from playing load balancer.  Even so,
> to achieve target latency, I had to hack up cpusets to let the user
> dynamically switch nohz off for specified sets (and the tick has to be
> skewed in both cases or you can just forget it).  With nohz, I can't
> quite achieve 30us jitter target, turn it off, I get single digit.  Out

Less than 10us jitter can alrady be accomplished by building a kernel with
certain options off (like for example preemption...) and ensuring that
stuff stays off certain processors. Lets not confuse realtime with low
latency. Real time in the sense of deterministic execution is bad for
latency because overhead is added to ensure the determinism which
increases latency.

I have a patch here that adds a system call to simply switch off the timer
interrupt cold turkey and with that I can get down to 1-2 usecs jitter.

> of the current box, triple digit for simple synchronized frame timers +
> compute worker-bees load on 64 cores.  Patch 4 probably helps that, but
> don't _think_ it'll fix it.  If you (currently) ever become balancer,
> you're latency target is smoking wreckage.

Yes so we need something to tell the system which cpu is the sacrificial
lamb that will not run low latency applications.

  reply	other threads:[~2012-03-22 16:26 UTC|newest]

Thread overview: 96+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-03-21 13:58 [RFC][PATCH 00/32] Nohz cpusets v2 (adaptive tickless kernel) Frederic Weisbecker
2012-03-21 13:58 ` Frederic Weisbecker
2012-04-04 15:33   ` warning in tick_nohz_irq_exit Stephen Hemminger
2012-04-04 20:45     ` Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 01/32] nohz: Separate idle sleeping time accounting from nohz logic Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 02/32] nohz: Make nohz API agnostic against idle ticks cputime accounting Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 03/32] nohz: Rename ts->idle_tick to ts->last_tick Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 04/32] nohz: Move nohz load balancer selection into idle logic Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 05/32] nohz: Move ts->idle_calls incrementation into strict " Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 06/32] nohz: Move next idle expiry time record into idle logic area Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 07/32] cpuset: Set up interface for nohz flag Frederic Weisbecker
2012-03-21 14:50   ` Christoph Lameter
2012-03-22  4:03     ` Mike Galbraith
2012-03-22 16:26       ` Christoph Lameter [this message]
2012-03-22 19:20         ` Mike Galbraith
2012-03-27 11:22       ` Frederic Weisbecker
2012-03-27 11:53         ` Mike Galbraith
2012-03-27 11:56           ` Frederic Weisbecker
2012-03-27 12:31             ` Mike Galbraith
2012-03-27 11:19     ` Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 08/32] nohz: Try not to give the timekeeping duty to an adaptive tickless cpu Frederic Weisbecker
2012-03-21 14:52   ` Christoph Lameter
2012-03-27 10:50     ` Frederic Weisbecker
2012-03-27 16:08       ` Christoph Lameter
2012-03-27 16:47         ` Peter Zijlstra
2012-03-28  1:12           ` Christoph Lameter
2012-03-28  8:39             ` Peter Zijlstra
2012-03-28 13:11               ` Dimitri Sivanich
2012-03-28 15:51               ` Chris Metcalf
2012-03-30  1:34         ` Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 09/32] x86: New cpuset nohz irq vector Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 10/32] nohz: Adaptive tick stop and restart on nohz cpuset Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 11/32] nohz/cpuset: Don't turn off the tick if rcu needs it Frederic Weisbecker
2012-03-21 14:54   ` Christoph Lameter
2012-03-22  7:38     ` Gilad Ben-Yossef
2012-03-22 16:18       ` Christoph Lameter
2012-03-27 15:21         ` Gilad Ben-Yossef
2012-03-28 12:39           ` Frederic Weisbecker
2012-03-28 12:57             ` Gilad Ben-Yossef
2012-03-28 13:38               ` Frederic Weisbecker
2012-03-22 17:18       ` Chris Metcalf
2012-03-27 15:31         ` Gilad Ben-Yossef
2012-03-27 15:43           ` Chris Metcalf
2012-03-28  8:36             ` Gilad Ben-Yossef
2012-03-27 12:13     ` Frederic Weisbecker
2012-03-27 16:13       ` Christoph Lameter
2012-03-27 16:24         ` Steven Rostedt
2012-03-28  0:42           ` Christoph Lameter
2012-03-28  1:06             ` Steven Rostedt
2012-03-28  1:19               ` Christoph Lameter
2012-03-28  1:35                 ` Steven Rostedt
2012-03-28  3:17                   ` Steven Rostedt
2012-03-28  7:55                     ` Gilad Ben-Yossef
2012-03-28 12:21                       ` Frederic Weisbecker
2012-03-28 12:41                         ` Gilad Ben-Yossef
2012-03-28 14:02                       ` Steven Rostedt
2012-03-28 11:53         ` Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 12/32] nohz/cpuset: Wake up adaptive nohz CPU when a timer gets enqueued Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 13/32] nohz/cpuset: Don't stop the tick if posix cpu timers are running Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 14/32] nohz/cpuset: Restart tick when nohz flag is cleared on cpuset Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 15/32] nohz/cpuset: Restart the tick if printk needs it Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 16/32] rcu: Restart the tick on non-responding adaptive nohz CPUs Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 17/32] rcu: Restart tick if we enqueue a callback in a nohz/cpuset CPU Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 18/32] nohz: Generalize tickless cpu time accounting Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 19/32] nohz/cpuset: Account user and system times in adaptive nohz mode Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 20/32] nohz/cpuset: New API to flush cputimes on nohz cpusets Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 21/32] nohz/cpuset: Flush cputime on threads in nohz cpusets when waiting leader Frederic Weisbecker
2012-03-27 14:10   ` Gilad Ben-Yossef
2012-03-27 14:23     ` Gilad Ben-Yossef
2012-03-28 11:20       ` Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 22/32] nohz/cpuset: Flush cputimes on procfs stat file read Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 23/32] nohz/cpuset: Flush cputimes for getrusage() and times() syscalls Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 24/32] x86: Syscall hooks for nohz cpusets Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 25/32] x86: Exception " Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 26/32] x86: Add adaptive tickless hooks on do_notify_resume() Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 27/32] nohz: Don't restart the tick before scheduling to idle Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 28/32] rcu: New rcu_user_enter() and rcu_user_exit() APIs Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 29/32] rcu: New rcu_user_enter_irq() and rcu_user_exit_irq() APIs Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 30/32] rcu: Switch to extended quiescent state in userspace from nohz cpuset Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 31/32] nohz: Exit RCU idle mode when we schedule before resuming userspace Frederic Weisbecker
2012-03-21 13:58 ` [PATCH 32/32] nohz/cpuset: Disable under some configs Frederic Weisbecker
2012-03-27 15:02 ` [RFC][PATCH 00/32] Nohz cpusets v2 (adaptive tickless kernel) Gilad Ben-Yossef
2012-03-27 15:04   ` Gilad Ben-Yossef
2012-03-27 15:05     ` Gilad Ben-Yossef
2012-03-27 16:22       ` Christoph Lameter
2012-03-28  6:47         ` Gilad Ben-Yossef
2012-03-27 15:10   ` Peter Zijlstra
2012-03-27 15:18     ` Gilad Ben-Yossef
2012-05-22 21:31     ` Thomas Gleixner
2012-05-22 21:50       ` Steven Rostedt
2012-05-22 22:22         ` Thomas Gleixner
2012-03-28 11:43   ` Frederic Weisbecker
2012-03-30  0:33 ` Kevin Hilman
2012-03-30  0:45   ` Frederic Weisbecker
2012-03-30  2:07     ` Geoff Levand
2012-03-30 14:10       ` Kevin Hilman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.DEB.2.00.1203221118550.25011@router.home \
    --to=cl@linux.com \
    --cc=abogani@kernel.org \
    --cc=akpm@linux-foundation.org \
    --cc=avi@redhat.com \
    --cc=cmetcalf@tilera.com \
    --cc=daniel.lezcano@linaro.org \
    --cc=efault@gmx.de \
    --cc=fweisbec@gmail.com \
    --cc=geoff@infradead.org \
    --cc=gilad@benyossef.com \
    --cc=linaro-sched-sig@lists.linaro.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maxk@qualcomm.com \
    --cc=mingo@kernel.org \
    --cc=paulmck@linux.vnet.ibm.com \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=shemminger@vyatta.com \
    --cc=tglx@linutronix.de \
    --cc=thebigcorporation@gmail.com \
    --cc=zen@openhuawei.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.