From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Josh Triplett <josh@joshtriplett.org>
Cc: linux-kernel@vger.kernel.org, mingo@elte.hu,
laijs@cn.fujitsu.com, dipankar@in.ibm.com,
akpm@linux-foundation.org, mathieu.desnoyers@polymtl.ca,
niv@us.ibm.com, tglx@linutronix.de, peterz@infradead.org,
rostedt@goodmis.org, Valdis.Kletnieks@vt.edu,
dhowells@redhat.com, eric.dumazet@gmail.com, darren@dvhart.com,
fweisbec@gmail.com, sbw@mit.edu, patches@linaro.org,
"Paul E. McKenney" <paul.mckenney@linaro.org>
Subject: Re: [PATCH tip/core/rcu 2/3] rcu: Disallow callback registry on offline CPUs
Date: Tue, 4 Sep 2012 16:53:21 -0700 [thread overview]
Message-ID: <20120904235321.GZ2593@linux.vnet.ibm.com> (raw)
In-Reply-To: <20120831162130.GB21673@leaf>
On Fri, Aug 31, 2012 at 09:21:30AM -0700, Josh Triplett wrote:
> On Thu, Aug 30, 2012 at 12:03:02PM -0700, Paul E. McKenney wrote:
> > From: "Paul E. McKenney" <paul.mckenney@linaro.org>
> >
> > Posting a callback after the CPU_DEAD notifier effectively leaks
> > that callback unless/until that CPU comes back online. Silence is
> > unhelpful when attempting to track down such leaks, so this commit emits
> > a WARN_ON_ONCE() and unconditionally leaks the callback when an offline
> > CPU attempts to register a callback. The rdp->nxttail[RCU_NEXT_TAIL] is
> > set to NULL in the CPU_DEAD notifier and restored in the CPU_UP_PREPARE
> > notifier, allowing _call_rcu() to determine exactly when posting callbacks
> > is illegal.
> >
> > Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
> > Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
>
> One suggestion below; with or without that change:
>
> Reviewed-by: Josh Triplett <josh@joshtriplett.org>
>
> > ---
> > kernel/rcutree.c | 10 ++++++++++
> > 1 files changed, 10 insertions(+), 0 deletions(-)
> >
> > diff --git a/kernel/rcutree.c b/kernel/rcutree.c
> > index 9854a00..5f8c4dd 100644
> > --- a/kernel/rcutree.c
> > +++ b/kernel/rcutree.c
> > @@ -1503,6 +1503,9 @@ static void rcu_cleanup_dead_cpu(int cpu, struct rcu_state *rsp)
> > WARN_ONCE(rdp->qlen != 0 || rdp->nxtlist != NULL,
> > "rcu_cleanup_dead_cpu: Callbacks on offline CPU %d: qlen=%lu, nxtlist=%p\n",
> > cpu, rdp->qlen, rdp->nxtlist);
> > + init_callback_list(rdp);
> > + /* Disallow further callbacks on this CPU. */
> > + rdp->nxttail[RCU_NEXT_TAIL] = NULL;
> > }
> >
> > #else /* #ifdef CONFIG_HOTPLUG_CPU */
> > @@ -1925,6 +1928,12 @@ __call_rcu(struct rcu_head *head, void (*func)(struct rcu_head *rcu),
> > rdp = this_cpu_ptr(rsp->rda);
> >
> > /* Add the callback to our list. */
> > + if (unlikely(rdp->nxttail[RCU_NEXT_TAIL] == NULL)) {
> > + /* _call_rcu() is illegal on offline CPU; leak the callback. */
> > + WARN_ON_ONCE(1);
>
> You can write this as:
>
> if (WARN_ON_ONCE(rdp->nxttail[RCU_NEXT_TAIL] == NULL))
>
> WARN_ON_ONCE also has a built-in unlikely() already.
Indeed, I missed the fact that WARN_ON_ONCE() returns its argument.
I believe that there are a number of other places in RCU where I could
bury a WARN_ON_ONCE(1) into the preceding "if" statement, so added a
todo-list item to check all of them and convert as appropriate.
Thanx, Paul
> > + local_irq_restore(flags);
> > + return;
> > + }
> > ACCESS_ONCE(rdp->qlen)++;
> > if (lazy)
> > rdp->qlen_lazy++;
> > @@ -2462,6 +2471,7 @@ rcu_init_percpu_data(int cpu, struct rcu_state *rsp, int preemptible)
> > rdp->qlen_last_fqs_check = 0;
> > rdp->n_force_qs_snap = rsp->n_force_qs;
> > rdp->blimit = blimit;
> > + init_callback_list(rdp); /* Re-enable callbacks on this CPU. */
> > rdp->dynticks->dynticks_nesting = DYNTICK_TASK_EXIT_IDLE;
> > atomic_set(&rdp->dynticks->dynticks,
> > (atomic_read(&rdp->dynticks->dynticks) & ~0x1) + 1);
> > --
> > 1.7.8
> >
>
next prev parent reply other threads:[~2012-09-04 23:53 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-08-30 19:02 [PATCH tip/core/rcu 0/3] CPU-hotplug changes Paul E. McKenney
2012-08-30 19:03 ` [PATCH tip/core/rcu 1/3] rcu: Remove _rcu_barrier() dependency on __stop_machine() Paul E. McKenney
2012-08-30 19:03 ` [PATCH tip/core/rcu 2/3] rcu: Disallow callback registry on offline CPUs Paul E. McKenney
2012-08-31 16:21 ` Josh Triplett
2012-09-04 23:53 ` Paul E. McKenney [this message]
2012-08-30 19:03 ` [PATCH tip/core/rcu 3/3] sched: Fix load avg vs cpu-hotplug Paul E. McKenney
2012-08-31 16:09 ` [PATCH tip/core/rcu 1/3] rcu: Remove _rcu_barrier() dependency on __stop_machine() Josh Triplett
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120904235321.GZ2593@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=Valdis.Kletnieks@vt.edu \
--cc=akpm@linux-foundation.org \
--cc=darren@dvhart.com \
--cc=dhowells@redhat.com \
--cc=dipankar@in.ibm.com \
--cc=eric.dumazet@gmail.com \
--cc=fweisbec@gmail.com \
--cc=josh@joshtriplett.org \
--cc=laijs@cn.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@polymtl.ca \
--cc=mingo@elte.hu \
--cc=niv@us.ibm.com \
--cc=patches@linaro.org \
--cc=paul.mckenney@linaro.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=sbw@mit.edu \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).