From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Borislav Petkov <bp@alien8.de>, x86-ml <x86@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
lkml <linux-kernel@vger.kernel.org>,
joel@joelfernandes.org
Subject: Re: 4.19-rc1: ./include/linux/rcupdate.h:631 rcu_read_lock() used illegally while idle!
Date: Tue, 4 Sep 2018 13:53:21 -0700 [thread overview]
Message-ID: <20180904205321.GQ4225@linux.vnet.ibm.com> (raw)
In-Reply-To: <20180904162611.6a120068@gandalf.local.home>
On Tue, Sep 04, 2018 at 04:26:11PM -0400, Steven Rostedt wrote:
> On Sat, 1 Sep 2018 21:16:39 -0700
> "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
>
> > On Sat, Sep 01, 2018 at 06:45:31PM -0400, Steven Rostedt wrote:
> > > On Sat, 1 Sep 2018 10:54:42 -0700
> > > "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
> > >
> > > > On Sat, Sep 01, 2018 at 07:35:59PM +0200, Borislav Petkov wrote:
> > > > > This is a huge splat! It haz some perf* and sched* in it, I guess for
> > > > > peterz to stare at. And lemme add Paul for good measure too :)
> > > > >
> > > > > Kernel is -rc1 + 3 microcode loader patches ontop which should not be
> > > > > related.
> > > >
> > > > It really is tracing from the idle loop. But I thought that the event
> > > > tracing took care of that. Adding Steve and Joel for their thoughts.
> > > >
> > > > Thanx, Paul
> > > >
> > > > > Thx.
> > > > >
> > > > > ---
> > > > > [ 62.409125] =============================
> > > > > [ 62.409129] WARNING: suspicious RCU usage
> > > > > [ 62.409133] 4.19.0-rc1+ #1 Not tainted
> > > > > [ 62.409136] -----------------------------
> > > > > [ 62.409140] ./include/linux/rcupdate.h:631 rcu_read_lock() used illegally while idle!
> > > > > [ 62.409143]
> > > > > other info that might help us debug this:
> > > > >
> > > > > [ 62.409147]
> > > > > RCU used illegally from idle CPU!
> > > > > rcu_scheduler_active = 2, debug_locks = 1
> > > > > [ 62.409151] RCU used illegally from extended quiescent state!
> > > > > [ 62.409155] 1 lock held by swapper/0/0:
> > > > > [ 62.409158] #0: 000000004557ee0e (rcu_read_lock){....}, at: perf_event_output_forward+0x0/0x130
> > > > > [ 62.409175]
> > > > > stack backtrace:
> > > > > [ 62.409180] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.19.0-rc1+ #1
> > > > > [ 62.409183] Hardware name: LENOVO 2320CTO/2320CTO, BIOS G2ET86WW (2.06 ) 11/13/2012
> > > > > [ 62.409187] Call Trace:
> > > > > [ 62.409196] dump_stack+0x85/0xcb
> > > > > [ 62.409203] perf_event_output_forward+0xf6/0x130
> > >
> > > I think this is because we switched the trace point code to be
> > > protected by srcu instead of rcu_lock_sched() and a song and dance to
> > > "make RCU watch again" if it is not, but perf is using normal
> > > rcu_read_lock() internally even though it is hooked into the
> > > tracepoint code. Should perf switch to SRCU, or perhaps it can do the
> > > song and dance to make RCU watch again?
> >
> > Well, this is a regression, so in theory we could push my three SRCU
> > patches into the current merge window, which would allow perf going
> > to SRCU, thus fixing the above splat. I am OK either way. What would
> > you prefer?
>
> I wonder if this partial revert will fix things?
I must defer to Borislav on this one. Assuming it has the desired
effect, I am good with it. Nicer and more contained fix than three
SRCU patches! ;-)
Thanx, Paul
> -- Steve
>
>
> diff --git a/include/linux/tracepoint.h b/include/linux/tracepoint.h
> index 7f2e16e76ac4..041f7e56a289 100644
> --- a/include/linux/tracepoint.h
> +++ b/include/linux/tracepoint.h
> @@ -158,8 +158,10 @@ extern void syscall_unregfunc(void);
> * For rcuidle callers, use srcu since sched-rcu \
> * doesn't work from the idle path. \
> */ \
> - if (rcuidle) \
> + if (rcuidle) { \
> idx = srcu_read_lock_notrace(&tracepoint_srcu); \
> + rcu_irq_enter_irqson(); \
> + } \
> \
> it_func_ptr = rcu_dereference_raw((tp)->funcs); \
> \
> @@ -171,8 +173,10 @@ extern void syscall_unregfunc(void);
> } while ((++it_func_ptr)->func); \
> } \
> \
> - if (rcuidle) \
> + if (rcuidle) { \
> + rcu_irq_exit_irqson(); \
> srcu_read_unlock_notrace(&tracepoint_srcu, idx);\
> + } \
> \
> preempt_enable_notrace(); \
> } while (0)
>
next prev parent reply other threads:[~2018-09-04 20:53 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-01 17:35 4.19-rc1: ./include/linux/rcupdate.h:631 rcu_read_lock() used illegally while idle! Borislav Petkov
2018-09-01 17:54 ` Paul E. McKenney
2018-09-01 22:45 ` Steven Rostedt
2018-09-02 4:16 ` Paul E. McKenney
2018-09-04 20:26 ` Steven Rostedt
2018-09-04 20:53 ` Paul E. McKenney [this message]
2018-09-05 8:22 ` Borislav Petkov
2018-09-05 12:59 ` Paul E. McKenney
2018-09-05 13:07 ` Steven Rostedt
2018-09-05 13:26 ` [PATCH] tracing: Add back in rcu_irq_enter/exit_irqson() for rcuidle tracepoints Steven Rostedt
2018-09-05 15:05 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180904205321.GQ4225@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=bp@alien8.de \
--cc=joel@joelfernandes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).