From: "Paul E. McKenney" <paulmck@kernel.org>
To: Frederic Weisbecker <frederic@kernel.org>
Cc: Joel Fernandes <joel@joelfernandes.org>,
LKML <linux-kernel@vger.kernel.org>,
Steven Rostedt <rostedt@goodmis.org>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
Lai Jiangshan <jiangshanlai@gmail.com>,
Josh Triplett <josh@joshtriplett.org>
Subject: Re: [PATCH 01/10] rcu: Directly lock rdp->nocb_lock on nocb code entrypoints
Date: Wed, 10 Jun 2020 16:21:42 -0700 [thread overview]
Message-ID: <20200610232142.GA4455@paulmck-ThinkPad-P72> (raw)
In-Reply-To: <20200610221245.GA3833@lenoir>
On Thu, Jun 11, 2020 at 12:12:46AM +0200, Frederic Weisbecker wrote:
> On Wed, Jun 10, 2020 at 07:02:10AM -0700, Paul E. McKenney wrote:
> > And just to argue against myself...
> >
> > Another approach is to maintain explicit multiple states for each
> > ->cblist, perhaps something like this:
> >
> > 1. In softirq. Transition code advances to next.
> > 2. To no-CB 1. Either GP or CB kthread for the transitioning
> > CPU advances to next. Note that the fact that the
> > transition code runs on the transitioning CPU means that
> > the RCU softirq handler doesn't need to be involved.
> > 3. To no-CB 2. Either GP or CB kthread for the transitioning
> > CPU advances to next.
>
> Just to clarify, if GP has set NO_CB2 in (2), we want CB to set NO_CB3
> in 3), right? OTOH if CB has set NO_CB2 in (2), we want GP to set NO_CB3
> in (3), right?
>
> The point being to make sure that both threads acknowledge the transition?
Exactly!
> > 4. To no-CB 3. Transitioning code advances to next.
> > At this point, the no-CBs setup is fully functional.
>
> And softirq can stop processing callbacks from that point on.
You got it!
> > 5. No-CB. Transitioning code advances to next.
> > Again, the fact that the transitioning code is running
> > on the transitioning CPU with interrupts disabled means
> > that the RCU softirq handler need not be explicitly
> > involved.
> > 6. To softirq 1. The RCU softirq handler for the transitioning
> > CPU advances to next.
> > At this point, the RCU softirq handler is processing callbacks.
> > 7. To softirq 2. Either GP or CB kthread for the transitioning
> > CPU advances to next.
> > At this point, the softirq handler is processing callbacks.
>
> SOFTIRQ2 should be part of what happens in SOFTIRQ1. The transitioning
> CPU sets SOFTIRQ1, which is immediately visible by local softirqs,
> and wakes up CB/GP. CB/GP sets SOFTIRQ2, CB/GP sets SOFTIRQ3 and
> we go back to transitioning code that sets IN_SOFTIRQ.
>
> Or did I miss something?
I was perhaps being overly paranoid. You might well be right.
> > 8. To softirq 3. Either GP or CB kthread for the transitioning
> > CPU advances to next.
> > At this point, the no-CBs setup is fully shut down.
> > 9. To softirq 4. Transitioning code advances to next,
> > which is the first, "In softirq".
> > (This one -might- be unnecessary, but...)
> >
> > All transitions are of course with the ->nocb_lock held.
> >
> > When there is only one CPU during early boot near rcu_init() time,
> > the transition from "In softirq" to "No-CB" can remain be instantaneous.
> >
> > This has the advantage of not slowing things down just because there
> > is an RCU callback flood in progress. It also uses an explicit
> > protocol that should (give or take bugs) maintain full safety both
> > in protection of ->cblist and in dealing with RCU callback floods.
> >
> > Thoughts?
>
> Agreed. And I really like that it details the whole process in a very
> explicit way.
>
> Thanks!
Glad you like it! And of course please adjust it as needed, up to and
including doing something completely different that works better. ;-)
Thanx, Paul
next prev parent reply other threads:[~2020-06-10 23:21 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-13 16:47 [PATCH 00/10] rcu: Allow a CPU to leave and reenter NOCB state Frederic Weisbecker
2020-05-13 16:47 ` [PATCH 01/10] rcu: Directly lock rdp->nocb_lock on nocb code entrypoints Frederic Weisbecker
2020-05-20 12:29 ` Joel Fernandes
2020-05-22 17:57 ` Paul E. McKenney
2020-05-26 15:21 ` Joel Fernandes
2020-05-26 16:29 ` Paul E. McKenney
2020-05-26 20:18 ` Joel Fernandes
2020-05-26 21:09 ` Paul E. McKenney
2020-05-26 21:27 ` Joel Fernandes
2020-05-26 22:29 ` Paul E. McKenney
2020-05-27 0:45 ` Joel Fernandes
2020-05-27 0:58 ` Paul E. McKenney
2020-06-04 11:41 ` Frederic Weisbecker
2020-06-04 16:36 ` Paul E. McKenney
2020-06-08 12:57 ` Frederic Weisbecker
2020-06-09 18:02 ` Paul E. McKenney
2020-06-10 13:12 ` Frederic Weisbecker
2020-06-10 14:02 ` Paul E. McKenney
2020-06-10 22:12 ` Frederic Weisbecker
2020-06-10 23:21 ` Paul E. McKenney [this message]
2020-06-11 1:32 ` Joel Fernandes
2020-05-13 16:47 ` [PATCH 02/10] rcu: Use direct rdp->nocb_lock operations on local calls Frederic Weisbecker
2020-05-13 16:47 ` [PATCH 03/10] rcu: Make locking explicit in do_nocb_deferred_wakeup_common() Frederic Weisbecker
2020-05-26 19:54 ` Joel Fernandes
2020-05-26 19:59 ` Joel Fernandes
2020-05-13 16:47 ` [PATCH 04/10] rcu: Implement rcu_segcblist_is_offloaded() config dependent Frederic Weisbecker
2020-05-13 18:20 ` Paul E. McKenney
2020-05-13 23:03 ` Frederic Weisbecker
2020-05-14 15:47 ` Paul E. McKenney
2020-05-13 16:47 ` [PATCH 05/10] rcu: Remove useless conditional nocb unlock Frederic Weisbecker
2020-05-13 16:47 ` [PATCH 06/10] rcu: Make nocb_cb kthread parkable Frederic Weisbecker
2020-06-11 1:34 ` Joel Fernandes
2020-05-13 16:47 ` [PATCH 07/10] rcu: Temporarily assume that nohz full CPUs might not be NOCB Frederic Weisbecker
2020-05-13 18:25 ` Paul E. McKenney
2020-05-13 23:08 ` Frederic Weisbecker
2020-05-14 15:50 ` Paul E. McKenney
2020-05-14 22:49 ` Frederic Weisbecker
2020-05-13 16:47 ` [PATCH 08/10] rcu: Allow to deactivate nocb on a CPU Frederic Weisbecker
2020-05-13 18:38 ` Paul E. McKenney
2020-05-13 22:45 ` Frederic Weisbecker
2020-05-14 15:47 ` Paul E. McKenney
2020-05-14 22:30 ` Frederic Weisbecker
2020-05-14 22:47 ` Paul E. McKenney
2020-05-14 22:55 ` Frederic Weisbecker
2020-05-26 21:20 ` Joel Fernandes
2020-05-26 22:49 ` Joel Fernandes
2020-06-04 13:10 ` Frederic Weisbecker
2020-06-11 1:32 ` Joel Fernandes
2020-06-11 17:03 ` Paul E. McKenney
2020-06-04 13:14 ` Frederic Weisbecker
2020-05-13 16:47 ` [PATCH 09/10] rcu: Allow to re-offload a CPU that used to be nocb Frederic Weisbecker
2020-05-13 18:41 ` Paul E. McKenney
2020-05-13 16:47 ` [PATCH 10/10] rcu: Nocb (de)activate through sysfs Frederic Weisbecker
2020-05-13 18:42 ` Paul E. McKenney
2020-05-13 23:23 ` Frederic Weisbecker
2020-05-14 15:51 ` Paul E. McKenney
2020-05-13 18:15 ` [PATCH 00/10] rcu: Allow a CPU to leave and reenter NOCB state Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200610232142.GA4455@paulmck-ThinkPad-P72 \
--to=paulmck@kernel.org \
--cc=frederic@kernel.org \
--cc=jiangshanlai@gmail.com \
--cc=joel@joelfernandes.org \
--cc=josh@joshtriplett.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=rostedt@goodmis.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).