From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: linux-kernel@vger.kernel.org, mingo@kernel.org,
laijs@cn.fujitsu.com, dipankar@in.ibm.com,
akpm@linux-foundation.org, mathieu.desnoyers@efficios.com,
josh@joshtriplett.org, tglx@linutronix.de, rostedt@goodmis.org,
dhowells@redhat.com, edumazet@google.com, dvhart@linux.intel.com,
fweisbec@gmail.com, oleg@redhat.com, bobby.prani@gmail.com,
<""@rjwysocki.net>,
tianyu.lan@intel.com
Subject: Re: [PATCH RFC tip/core/rcu] Eliminate deadlock between CPU hotplug and expedited grace periods
Date: Wed, 3 Sep 2014 08:03:06 -0700 [thread overview]
Message-ID: <20140903150306.GU5001@linux.vnet.ibm.com> (raw)
In-Reply-To: <20140903113112.GM4783@worktop.ger.corp.intel.com>
On Wed, Sep 03, 2014 at 01:31:12PM +0200, Peter Zijlstra wrote:
> On Tue, Sep 02, 2014 at 09:36:56AM -0700, Paul E. McKenney wrote:
> > On Mon, Sep 01, 2014 at 06:17:35PM +0200, Peter Zijlstra wrote:
> > > On Mon, Sep 01, 2014 at 09:05:50AM -0700, Paul E. McKenney wrote:
> > > > > URGH.. I really hate that. The hotplug interface is already too
> > > > > horrible, we should not add such hacks to it.
> > > >
> > > > We do have try_ interfaces to a number of other subsystems, so I don't
> > > > believe that it qualifies as such a hack.
> > >
> > > We do indeed, but I'm not sure about adding this to the hotplug stuff.
> >
> > Looks pretty straightforward to me.
> >
> > > Also; not really understanding the problem doesn't help.
> >
> > The current implementation of synchronize_sched_expedited()
> > calls get_online_cpus(). Some of the ACPI code needs to hold the
> > acpi_ioremap_lock mutex across synchronize_sched_expedited(), and
> > also needs to acquire this same mutex from a CPU hotplug notifier.
> > This results in deadlock between the cpu_hotplug.lock mutex and the
> > acpi_ioremap_lock mutex.
>
>
> acpi_ioremap_lock cpu_hotplug_begin()
> synchronize_sched() acpi_ioremap_lock
> get_online_cpus()
>
> So yes, AB-BA.
>
> > Normal RCU grace periods avoid this by synchronizing on a lock acquired by
> > the RCU CPU-hotplug notifiers, but this does not work for the expedited
> > grace periods because the outgoing CPU can be running random tasks for
> > quite some time after RCU's notifier executes. So the fix is just to
> > drop back to a normal grace period when there is a CPU-hotplug operation
> > in progress.
>
> So why are we 'normally' doing an expedited call here anyhow?
Presumably because they set either the boot parameter or
the sysfs variable that causes synchronize_sched() to so
synchronize_sched_expedited().
> > > > > How about ripping that rcu_expedited stuff out instead? That's all
> > > > > conditional anyhow, so might as well not do it.
> > > >
> > > > In what way is the expedited stuff conditional?
> > >
> > > synchronize_sched() conditionally calls synchronize_sched_expedited()
> > > and its condition: rcu_expedited, gets set/cleared on pm notifiers and
> > > nr_cpu_ids.
> >
> > There are also direct calls to both synchronize_sched_expedited() and
> > synchronize_rcu_expedited().
>
> But those are not within hotplug bits. Also weren't we removing them? I
> thought we didn't appreciate spraying IPIs like they do?
I hadn't heard anything about removing them, but making the
expedited primitives a bit less IPI-happy is on my list.
Thanx, Paul
next prev parent reply other threads:[~2014-09-03 15:12 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-08-28 19:47 [PATCH RFC tip/core/rcu] Eliminate deadlock between CPU hotplug and expedited grace periods Paul E. McKenney
2014-08-29 6:54 ` Lan Tianyu
2014-08-29 13:11 ` Paul E. McKenney
2014-09-01 11:20 ` Peter Zijlstra
2014-09-01 16:05 ` Paul E. McKenney
2014-09-01 16:17 ` Peter Zijlstra
2014-09-02 16:36 ` Paul E. McKenney
2014-09-03 11:31 ` Peter Zijlstra
2014-09-03 15:03 ` Paul E. McKenney [this message]
2014-09-03 15:28 ` Peter Zijlstra
2014-09-03 16:38 ` Paul E. McKenney
2014-09-17 7:11 ` Lan Tianyu
2014-09-17 13:10 ` Paul E. McKenney
2014-09-18 7:15 ` Lan Tianyu
2014-09-18 12:38 ` Paul E. McKenney
2014-09-18 22:55 ` Rafael J. Wysocki
2014-09-18 22:57 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140903150306.GU5001@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=bobby.prani@gmail.com \
--cc=dhowells@redhat.com \
--cc=dipankar@in.ibm.com \
--cc=dvhart@linux.intel.com \
--cc=edumazet@google.com \
--cc=fweisbec@gmail.com \
--cc=josh@joshtriplett.org \
--cc=laijs@cn.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mingo@kernel.org \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=tianyu.lan@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).