From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
Lai Jiangshan <laijs@cn.fujitsu.com>,
Joe Korty <joe.korty@ccur.com>,
mathieu.desnoyers@efficios.com, dhowells@redhat.com,
loic.minier@linaro.org, dhaval.giani@gmail.com,
tglx@linutronix.de, linux-kernel@vger.kernel.org,
josh@joshtriplett.org, houston.jim@comcast.net
Subject: Re: [PATCH] a local-timer-free version of RCU
Date: Tue, 16 Nov 2010 07:51:04 -0800 [thread overview]
Message-ID: <20101116155104.GB2497@linux.vnet.ibm.com> (raw)
In-Reply-To: <20101116135230.GA5362@nowhere>
On Tue, Nov 16, 2010 at 02:52:34PM +0100, Frederic Weisbecker wrote:
> On Mon, Nov 15, 2010 at 05:28:46PM -0800, Paul E. McKenney wrote:
> > My concern is not the tick -- it is really easy to work around lack of a
> > tick from an RCU viewpoint. In fact, this happens automatically given the
> > current implementations! If there is a callback anywhere in the system,
> > then RCU will prevent the corresponding CPU from entering dyntick-idle
> > mode, and that CPU's clock will drive the rest of RCU as needed via
> > force_quiescent_state().
>
> Now, I'm confused, I thought a CPU entering idle nohz had nothing to do
> if it has no local callbacks, and rcu_enter_nohz already deals with
> everything.
>
> There is certainly tons of subtle things in RCU anyway :)
Well, I wasn't being all that clear above, apologies!!!
If a given CPU hasn't responded to the current RCU grace period,
perhaps due to being in a longer-than-average irq handler, then it
doesn't necessarily need its own scheduler tick enabled. If there is a
callback anywhere else in the system, then there is some other CPU with
its scheduler tick enabled. That other CPU can drive the slow-to-respond
CPU through the grace-period process.
The current RCU code should work in the common case. There are probably
a few bugs, but I will make you a deal. You find them, I will fix them.
Particularly if you are willing to test the fixes.
> > The force_quiescent_state() workings would
> > want to be slightly different for dyntick-hpc, but not significantly so
> > (especially once I get TREE_RCU moved to kthreads).
> >
> > My concern is rather all the implicit RCU-sched read-side critical
> > sections, particularly those that arch-specific code is creating.
> > And it recently occurred to me that there are necessarily more implicit
> > irq/preempt disables than there are exception entries.
>
> Doh! You're right, I don't know why I thought that adaptive tick would
> solve the implicit rcu sched/bh cases, my vision took a shortcut.
Yeah, and I was clearly suffering from a bit of sleep deprivation when
we discussed this in Boston. :-/
> > So would you be OK with telling RCU about kernel entries/exits, but
> > simply not enabling the tick?
>
> Let's try that.
Cool!!!
> > The irq and NMI kernel entries/exits are
> > already covered, of course.
>
> Yep.
>
> > This seems to me to work out as follows:
> >
> > 1. If there are no RCU callbacks anywhere in the system, RCU
> > is quiescent and does not cause any IPIs or interrupts of
> > any kind. For HPC workloads, this should be the common case.
>
> Right.
>
> > 2. If there is an RCU callback, then one CPU keeps a tick going
> > and drives RCU core processing on all CPUs. (This probably
> > works with RCU as is, but somewhat painfully.) This results
> > in some IPIs, but only to those CPUs that remain running in
> > the kernel for extended time periods. Appropriate adjustment
> > of RCU_JIFFIES_TILL_FORCE_QS, possibly promoted to be a
> > kernel configuration parameter, should make such IPIs
> > -extremely- rare. After all, how many kernel code paths
> > are going to consume (say) 10 jiffies of CPU time? (Keep
> > in mind that if the system call blocks, the CPU will enter
> > dyntick-idle mode, and RCU will still recognize it as an
> > innocent bystander without needing to IPI it.)
>
> Makes all sense. Also there may be periods when these "isolated" CPUs
> will restart the tick, like when there is more than one task running
> on that CPU, in which case we can of course fall back to usual
> grace periods processing.
Yep!
> > 3. The implicit RCU-sched read-side critical sections just work
> > as they do today.
> >
> > Or am I missing some other problems with this approach?
>
> No, looks good, now I'm going to implement/test a draft of these ideas.
>
> Thanks a lot!
Very cool, and thank you!!! I am sure that you will not be shy about
letting me know of any RCU problems that you might encounter. ;-)
Thanx, Paul
next prev parent reply other threads:[~2010-11-16 15:51 UTC|newest]
Thread overview: 63+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-11-04 23:21 dyntick-hpc and RCU Paul E. McKenney
2010-11-05 5:27 ` Frederic Weisbecker
2010-11-05 5:38 ` Frederic Weisbecker
2010-11-05 15:06 ` Paul E. McKenney
2010-11-05 20:06 ` Dhaval Giani
2010-11-05 15:04 ` Paul E. McKenney
2010-11-08 14:10 ` Frederic Weisbecker
2010-11-05 21:00 ` [PATCH] a local-timer-free version of RCU Joe Korty
2010-11-06 19:28 ` Paul E. McKenney
2010-11-06 19:34 ` Mathieu Desnoyers
2010-11-06 19:42 ` Mathieu Desnoyers
2010-11-06 19:44 ` Paul E. McKenney
2010-11-08 2:11 ` Udo A. Steinberg
2010-11-08 2:19 ` Udo A. Steinberg
2010-11-08 2:54 ` Paul E. McKenney
2010-11-08 15:32 ` Frederic Weisbecker
2010-11-08 19:38 ` Paul E. McKenney
2010-11-08 20:40 ` Frederic Weisbecker
2010-11-10 18:08 ` Paul E. McKenney
2010-11-08 15:06 ` Frederic Weisbecker
2010-11-08 15:18 ` Joe Korty
2010-11-08 19:50 ` Paul E. McKenney
2010-11-08 19:49 ` Paul E. McKenney
2010-11-08 20:51 ` Frederic Weisbecker
2010-11-06 20:03 ` Mathieu Desnoyers
2010-11-09 9:22 ` Lai Jiangshan
2010-11-10 15:54 ` Frederic Weisbecker
2010-11-10 17:31 ` Peter Zijlstra
2010-11-10 17:45 ` Frederic Weisbecker
2010-11-11 4:19 ` Paul E. McKenney
2010-11-13 22:30 ` Frederic Weisbecker
2010-11-16 1:28 ` Paul E. McKenney
2010-11-16 13:52 ` Frederic Weisbecker
2010-11-16 15:51 ` Paul E. McKenney [this message]
2010-11-17 0:52 ` Frederic Weisbecker
2010-11-17 1:25 ` Paul E. McKenney
2011-03-07 20:31 ` [PATCH] An RCU for SMP with a single CPU garbage collector Joe Korty
[not found] ` <20110307210157.GG3104@linux.vnet.ibm.com>
2011-03-07 21:16 ` Joe Korty
2011-03-07 21:33 ` Joe Korty
2011-03-07 22:51 ` Joe Korty
2011-03-08 9:07 ` Paul E. McKenney
2011-03-08 15:57 ` Joe Korty
2011-03-08 22:53 ` Joe Korty
2011-03-10 0:30 ` Paul E. McKenney
2011-03-10 0:28 ` Paul E. McKenney
2011-03-09 22:29 ` Frederic Weisbecker
2011-03-09 22:15 ` [PATCH 2/4] jrcu: tap rcu_read_unlock Joe Korty
2011-03-10 0:34 ` Paul E. McKenney
2011-03-10 19:50 ` JRCU Theory of Operation Joe Korty
2011-03-12 14:36 ` Paul E. McKenney
2011-03-13 0:43 ` Joe Korty
2011-03-13 5:56 ` Paul E. McKenney
2011-03-13 23:53 ` Joe Korty
2011-03-14 0:50 ` Paul E. McKenney
2011-03-14 0:55 ` Josh Triplett
2011-03-09 22:16 ` [PATCH 3/4] jrcu: tap might_resched() Joe Korty
2011-03-09 22:17 ` [PATCH 4/4] jrcu: add new stat to /sys/kernel/debug/rcu/rcudata Joe Korty
2011-03-09 22:19 ` [PATCH 1/4] jrcu: remove preempt_enable() tap [resend] Joe Korty
2011-03-12 14:36 ` [PATCH] An RCU for SMP with a single CPU garbage collector Paul E. McKenney
2011-03-13 1:25 ` Joe Korty
2011-03-13 6:09 ` Paul E. McKenney
[not found] <757455806.950179.1289232791283.JavaMail.root@sz0076a.westchester.pa.mail.comcast.net>
2010-11-08 16:15 ` [PATCH] a local-timer-free version of RCU houston.jim
2010-11-08 19:52 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20101116155104.GB2497@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=dhaval.giani@gmail.com \
--cc=dhowells@redhat.com \
--cc=fweisbec@gmail.com \
--cc=houston.jim@comcast.net \
--cc=joe.korty@ccur.com \
--cc=josh@joshtriplett.org \
--cc=laijs@cn.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=loic.minier@linaro.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).