From: Abdul Haleem <abdhalee@linux.vnet.ibm.com>
To: Nicholas Piggin <npiggin@gmail.com>
Cc: sachinp <sachinp@linux.vnet.ibm.com>,
Stephen Rothwell <sfr@canb.auug.org.au>,
Paul McKenney <Paul.McKenney@us.ibm.com>,
linux-kernel <linux-kernel@vger.kernel.org>,
linux-next <linux-next@vger.kernel.org>,
paulus@samba.org, paulmck@linux.vnet.ibm.com,
linuxppc-dev <linuxppc-dev@lists.ozlabs.org>
Subject: Re: [linux-next] cpus stalls detected few hours after booting next kernel
Date: Fri, 07 Jul 2017 16:36:10 +0530 [thread overview]
Message-ID: <1499425570.19784.38.camel@abdul.in.ibm.com> (raw)
In-Reply-To: <20170630172802.6d4ddca2@roar.ozlabs.ibm.com>
On Fri, 2017-06-30 at 17:28 +1000, Nicholas Piggin wrote:
> On Fri, 30 Jun 2017 10:52:18 +0530
> Abdul Haleem <abdhalee@linux.vnet.ibm.com> wrote:
>
> > On Fri, 2017-06-30 at 00:45 +1000, Nicholas Piggin wrote:
> > > On Thu, 29 Jun 2017 20:23:05 +1000
> > > Nicholas Piggin <npiggin@gmail.com> wrote:
> > >
> > > > On Thu, 29 Jun 2017 19:36:14 +1000
> > > > Nicholas Piggin <npiggin@gmail.com> wrote:
> > >
> > > > > I don't *think* the replay-wakeup-interrupt patch is directly involved, but
> > > > > it's likely to be one of the idle patches.
> > >
> > > Okay this turned out to be misconfigured sleep states I added for the
> > > simulator, sorry for the false alarm.
> > >
> > > > Although you have this in the backtrace. I wonder if that's a stuck
> > > > lock in rcu_process_callbacks?
> > >
> > > So this spinlock becomes top of the list of suspects. Can you try
> > > enabling lockdep and try to reproduce it?
> >
> > Yes, recreated again with CONFIG_LOCKDEP=y & CONFIG_DEBUG_LOCKDEP=y set.
> > I do not see any difference in trace messages with and without LOCKDEP
> > enabled.
> >
> > Please find the attached log file.
>
> Can you get an rcu_invoke_callback event trace that Paul suggested?
Yes, I was able to collect the perf data for rcu_invoke_callback event
on recent next kernel (4.12.0-next-20170705). the issue is rare to hit.
After booting the next kernel, I started this command 'perf record -e
rcu:rcu_invoke_callback -a -g -- cat' and waited for 30 minutes.
five minutes after seeing the stalls messages, I did CTRL-C to end the
perf command.
@Nicholas : the perf.data report is too huge to attach here, shall I
ping you the internal location of file on slack/mail ? Also the machine
is in the same state if you want to use it ?
>
> Does this bug show up with just the powerpc next branch?
>
> Thanks,
> Nick
>
--
Regard's
Abdul Haleem
IBM Linux Technology Centre
next prev parent reply other threads:[~2017-07-07 11:06 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-06-29 9:21 [linux-next] cpus stalls detected few hours after booting next kernel Abdul Haleem
2017-06-29 9:36 ` Nicholas Piggin
2017-06-29 10:23 ` Nicholas Piggin
2017-06-29 14:45 ` Nicholas Piggin
2017-06-29 15:47 ` [linux-next] " Paul McKenney
2017-06-30 5:22 ` Abdul Haleem
2017-06-30 7:28 ` Nicholas Piggin
2017-06-30 18:15 ` Paul E. McKenney
2017-07-04 14:14 ` [linux-next] " Abdul Haleem
2017-07-04 23:31 ` Paul E. McKenney
2017-07-07 11:06 ` Abdul Haleem [this message]
2017-07-25 5:19 ` Abdul Haleem
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1499425570.19784.38.camel@abdul.in.ibm.com \
--to=abdhalee@linux.vnet.ibm.com \
--cc=Paul.McKenney@us.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-next@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=npiggin@gmail.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=paulus@samba.org \
--cc=sachinp@linux.vnet.ibm.com \
--cc=sfr@canb.auug.org.au \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).