RCU Archive on lore.kernel.org
 help / color / Atom feed
From: "Paul E. McKenney" <paulmck@linux.ibm.com>
To: Joel Fernandes <joel@joelfernandes.org>
Cc: Frederic Weisbecker <frederic@kernel.org>,
	linux-kernel@vger.kernel.org, rcu@vger.kernel.org
Subject: Re: [PATCH -rcu dev 3/3] RFC: rcu/tree: Read dynticks_nmi_nesting in advance
Date: Mon, 19 Aug 2019 09:17:36 -0700
Message-ID: <20190819161736.GZ28441@linux.ibm.com> (raw)
In-Reply-To: <20190819154636.GC117548@google.com>

On Mon, Aug 19, 2019 at 11:46:36AM -0400, Joel Fernandes wrote:
> On Mon, Aug 19, 2019 at 07:41:08AM -0700, Paul E. McKenney wrote:
> > On Mon, Aug 19, 2019 at 10:22:08AM -0400, Joel Fernandes wrote:
> > > On Mon, Aug 19, 2019 at 02:59:08PM +0200, Frederic Weisbecker wrote:
> > > > On Fri, Aug 16, 2019 at 09:52:42AM -0700, Paul E. McKenney wrote:
> > > > > On Fri, Aug 16, 2019 at 12:24:04PM -0400, Joel Fernandes wrote:
> > > > > > On Thu, Aug 15, 2019 at 10:53:11PM -0400, Joel Fernandes (Google) wrote:
> > > > > > > I really cannot explain this patch, but without it, the "else if" block
> > > > > > > just doesn't execute thus causing the tick's dep mask to not be set and
> > > > > > > causes the tick to be turned off.
> > > > > > > 
> > > > > > > I tried various _ONCE() macros but the only thing that works is this
> > > > > > > patch.
> > > > > > > 
> > > > > > > Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> > > > > > > ---
> > > > > > >  kernel/rcu/tree.c | 3 ++-
> > > > > > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > > > > > 
> > > > > > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> > > > > > > index 856d3c9f1955..ac6bcf7614d7 100644
> > > > > > > --- a/kernel/rcu/tree.c
> > > > > > > +++ b/kernel/rcu/tree.c
> > > > > > > @@ -802,6 +802,7 @@ static __always_inline void rcu_nmi_enter_common(bool irq)
> > > > > > >  {
> > > > > > >  	struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
> > > > > > >  	long incby = 2;
> > > > > > > +	int dnn = rdp->dynticks_nmi_nesting;
> > > > > > 
> > > > > > I believe the accidental sign extension / conversion from long to int was
> > > > > > giving me an illusion since things started working well. Changing the 'int
> > > > > > dnn' to 'long dnn' gives similar behavior as without this patch! At least I
> > > > > > know now. Please feel free to ignore this particular RFC patch while I debug
> > > > > > this more (over the weekend or early next week). The first 2 patches are
> > > > > > good, just ignore this one.
> > > > > 
> > > > > Ah, good point on the type!  So you were ending up with zero due to the
> > > > > low-order 32 bits of DYNTICK_IRQ_NONIDLE being zero, correct?  If so,
> > > > > the "!rdp->dynticks_nmi_nesting" instead needs to be something like
> > > > > "rdp->dynticks_nmi_nesting == DYNTICK_IRQ_NONIDLE", which sounds like
> > > > > it is actually worse then the earlier comparison against the constant 2.
> > > > > 
> > > > > Sounds like I should revert the -rcu commit 805a16eaefc3 ("rcu: Force
> > > > > nohz_full tick on upon irq enter instead of exit").
> > > > 
> > > > I can't find that patch so all I can say so far is that its title doesn't
> > > > inspire me much. Do you still need that change for some reason?
> > > 
> > > No we don't need it. Paul's dev branch fixed it by checking DYNTICK_IRQ_NONIDLE:
> > > https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/commit/?h=dev&id=227482fd4f3ede0502b586da28a59971dfbac0b0
> > 
> > Ah, so you have tested reverting this?  If so, thank you very much!
> 
> Just tried reverting, and found a bug if done in the reverted way. Sent you
> email with a proposed change which is essentially the top of tree:
> https://github.com/joelagnel/linux-kernel/commits/rcu/nohz-test-3
> 
> Also for Frederick, I wanted to mention why my pure hack above (dnn variable)
> seemed to work. The reason was because of long to int conversion of
> rdp->dynticks_nmi_nesting which I surprisingly did not get a compiler warning
> for. dynticks_nmi_nesting getting converted to int was truncating the
> DYNTICK_IRQ_NONIDLE bit (in fact I believe this was due to the cltq
> instruction in x86). This caused the "else if" condition to always evaluate
> to true and turn off the tick.
> 
> Paul, I wanted to see if I can create a repeatable test case for this issue.
> Not a full blown RCU torture test, but something that one could run and get a
> PASS or FAIL. Do you think this could be useful? And what is the best place
> for such a test?
> Essentially the test would be:
> 1. Run a test and dump some traces.
> 2. Parse the traces and see if things are sane (such as the tick not turning
>    off for this issue).
> 3. Report pass or fail.
> 
> The other way instead of parsing traces could be, a kernel module that does
> trace_probe_register on various tracepoints and tries to see if the tick
> indeed could stay turned on. Then report pass/fail at the end of the module's
> execution.

Or you could increment a per-CPU counter in rcu_sched_clock_irq() and use
that to verify the tick.  Maybe you could use the existing ->ticks_this_gp,
though that does get zeroed at the beginning of each grace period, which
would make sampling it a bit trickier.

							Thanx, Paul


  reply index

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-16  2:53 [PATCH -rcu dev 1/3] rcu/tree: tick_dep_set/clear_cpu should accept bits instead of masks Joel Fernandes (Google)
2019-08-16  2:53 ` [PATCH -rcu dev 2/3] rcu/tree: Fix issue where sometimes rcu_urgent_qs is not set on IPI Joel Fernandes (Google)
2019-08-16  2:57   ` Joel Fernandes
2019-08-16  2:53 ` [PATCH -rcu dev 3/3] RFC: rcu/tree: Read dynticks_nmi_nesting in advance Joel Fernandes (Google)
2019-08-16 16:24   ` Joel Fernandes
2019-08-16 16:52     ` Paul E. McKenney
2019-08-16 17:07       ` Joel Fernandes
2019-08-16 17:30         ` Paul E. McKenney
2019-08-19 12:59       ` Frederic Weisbecker
2019-08-19 14:22         ` Joel Fernandes
2019-08-19 14:41           ` Paul E. McKenney
2019-08-19 15:46             ` Joel Fernandes
2019-08-19 16:17               ` Paul E. McKenney [this message]
2019-08-19 14:40         ` Paul E. McKenney
2019-08-16  2:59 ` [PATCH v2 -rcu dev 2/3] rcu/tree: Fix issue where sometimes rcu_urgent_qs is not set on IPI Joel Fernandes (Google)
2019-08-16 17:04   ` Paul E. McKenney
2019-08-16 17:25 ` [PATCH -rcu dev 1/3] rcu/tree: tick_dep_set/clear_cpu should accept bits instead of masks Paul E. McKenney
2019-08-16 17:35   ` Joel Fernandes
2019-08-19 12:38 ` Frederic Weisbecker
2019-08-19 14:46   ` Paul E. McKenney
2019-08-19 16:32     ` Frederic Weisbecker
2019-08-19 16:44       ` Paul E. McKenney
2019-08-20 12:08         ` Frederic Weisbecker
2019-08-20 14:44           ` Paul E. McKenney

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190819161736.GZ28441@linux.ibm.com \
    --to=paulmck@linux.ibm.com \
    --cc=frederic@kernel.org \
    --cc=joel@joelfernandes.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=rcu@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

RCU Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/rcu/0 rcu/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 rcu rcu/ https://lore.kernel.org/rcu \
		rcu@vger.kernel.org
	public-inbox-index rcu

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.rcu


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git