From: Scott Wood <swood@redhat.com>
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Joel Fernandes <joel@joelfernandes.org>,
linux-rt-users@vger.kernel.org, linux-kernel@vger.kernel.org,
"Paul E . McKenney" <paulmck@linux.ibm.com>,
Thomas Gleixner <tglx@linutronix.de>,
Steven Rostedt <rostedt@goodmis.org>,
Peter Zijlstra <peterz@infradead.org>,
Juri Lelli <juri.lelli@redhat.com>,
Clark Williams <williams@redhat.com>
Subject: Re: [PATCH RT v3 5/5] rcutorture: Avoid problematic critical section nesting on RT
Date: Tue, 17 Sep 2019 09:36:22 -0500 [thread overview]
Message-ID: <26dbecfee2c02456ddfda3647df1bcd56d9cc520.camel@redhat.com> (raw)
In-Reply-To: <20190917100728.wnhdvmbbzzxolef4@linutronix.de>
On Tue, 2019-09-17 at 12:07 +0200, Sebastian Andrzej Siewior wrote:
> On 2019-09-16 11:55:57 [-0500], Scott Wood wrote:
> > On Thu, 2019-09-12 at 18:17 -0400, Joel Fernandes wrote:
> > > On Wed, Sep 11, 2019 at 05:57:29PM +0100, Scott Wood wrote:
> > > > rcutorture was generating some nesting scenarios that are not
> > > > reasonable. Constrain the state selection to avoid them.
> > > >
> > > > Example #1:
> > > >
> > > > 1. preempt_disable()
> > > > 2. local_bh_disable()
> > > > 3. preempt_enable()
> > > > 4. local_bh_enable()
> > > >
> > > > On PREEMPT_RT, BH disabling takes a local lock only when called in
> > > > non-atomic context. Thus, atomic context must be retained until
> > > > after
> > > > BH
> > > > is re-enabled. Likewise, if BH is initially disabled in non-atomic
> > > > context, it cannot be re-enabled in atomic context.
> > > >
> > > > Example #2:
> > > >
> > > > 1. rcu_read_lock()
> > > > 2. local_irq_disable()
> > > > 3. rcu_read_unlock()
> > > > 4. local_irq_enable()
> > >
> > > If I understand correctly, these examples are not unrealistic in the
> > > real
> > > world unless RCU is used in the scheduler.
> >
> > I hope you mean "not realistic", at least when it comes to explicit
> > preempt/irq disabling rather than spinlock variants that don't disable
> > preempt/irqs on PREEMPT_RT.
>
> We have:
> - local_irq_disable() (+save)
> - spin_lock()
> - local_bh_disable()
> - preempt_disable()
>
> On non-RT you can (but should not) use the counter part of the function
> in random order like:
> local_bh_disable();
> local_irq_disable();
> local_bh_enable();
> local_irq_enable();
Actually even non-RT will assert if you do local_bh_enable() with IRQs
disabled -- but the other combinations do work, and are used some places via
spinlocks. If they are used via direct calls to preempt_disable() or
local_irq_disable() (or via raw spinlocks), then that will not go away on RT
and we'll have a problem.
> The non-RT will survive this. On RT the counterpart functions have to be
> used in reverse order:
> local_bh_disable();
> local_irq_disable();
> local_irq_enable();
> local_bh_enable();
>
> or the kernel will fall apart.
>
> Since you _can_ use it in random order Paul wants to test that the
> random use of those function does not break RCU in any way. Since they
> can not be used on RT in random order it has been agreed that we keep
> the test for !RT but disable it on RT.
For now, yes. Long term it would be good to keep track of when
preemption/irqs would be disabled on RT, even when running a non-RT debug
kernel, and assert when bad things are done with it (assuming an RT-capable
arch). Besides detecting these fairly unusual patterns, it could also
detect earlier the much more common problem of nesting a non-raw spinlock
inside a raw spinlock or other RT-atomic context.
-Scott
next prev parent reply other threads:[~2019-09-17 14:36 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-11 16:57 [PATCH RT v3 0/5] RCU fixes Scott Wood
2019-09-11 16:57 ` [PATCH RT v3 1/5] rcu: Acquire RCU lock when disabling BHs Scott Wood
2019-09-12 22:09 ` Joel Fernandes
2019-09-17 7:44 ` Sebastian Andrzej Siewior
2019-09-17 14:06 ` Scott Wood
2019-09-17 14:42 ` Sebastian Andrzej Siewior
2019-09-17 16:12 ` Scott Wood
2019-09-23 16:41 ` Sebastian Andrzej Siewior
2019-09-11 16:57 ` [PATCH RT v3 2/5] sched: Rename sleeping_lock to rt_invol_sleep Scott Wood
2019-09-17 7:52 ` Sebastian Andrzej Siewior
2019-09-11 16:57 ` [PATCH RT v3 3/5] sched: migrate_dis/enable: Use rt_invol_sleep Scott Wood
2019-09-17 7:59 ` Sebastian Andrzej Siewior
2019-09-17 14:06 ` Scott Wood
2019-09-23 16:59 ` Scott Wood
2019-09-23 17:52 ` Sebastian Andrzej Siewior
2019-09-24 11:21 ` Sebastian Andrzej Siewior
2019-09-24 13:53 ` Scott Wood
2019-09-24 15:25 ` Sebastian Andrzej Siewior
2019-09-24 15:47 ` Scott Wood
2019-09-24 16:05 ` Sebastian Andrzej Siewior
2019-09-24 16:35 ` Scott Wood
2019-10-04 16:45 ` Sebastian Andrzej Siewior
2019-09-11 16:57 ` [PATCH RT v3 4/5] rcu: Disable use_softirq on PREEMPT_RT Scott Wood
2019-09-12 21:38 ` Joel Fernandes
2019-09-12 22:19 ` Joel Fernandes
2019-09-17 9:31 ` Sebastian Andrzej Siewior
2019-09-17 14:08 ` Scott Wood
2019-09-11 16:57 ` [PATCH RT v3 5/5] rcutorture: Avoid problematic critical section nesting on RT Scott Wood
2019-09-12 22:17 ` Joel Fernandes
2019-09-16 16:55 ` Scott Wood
2019-09-17 10:07 ` Sebastian Andrzej Siewior
2019-09-17 14:36 ` Scott Wood [this message]
2019-09-17 14:50 ` Sebastian Andrzej Siewior
2019-09-17 16:32 ` Scott Wood
2019-09-23 16:25 ` Sebastian Andrzej Siewior
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=26dbecfee2c02456ddfda3647df1bcd56d9cc520.camel@redhat.com \
--to=swood@redhat.com \
--cc=bigeasy@linutronix.de \
--cc=joel@joelfernandes.org \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=paulmck@linux.ibm.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=williams@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).