RCU Archive on lore.kernel.org
 help / color / Atom feed
From: Joel Fernandes <joel@joelfernandes.org>
To: "Paul E. McKenney" <paulmck@linux.ibm.com>
Cc: linux-kernel@vger.kernel.org, rcu@vger.kernel.org, frederic@kernel.org
Subject: Re: [PATCH -rcu dev 3/3] RFC: rcu/tree: Read dynticks_nmi_nesting in advance
Date: Fri, 16 Aug 2019 13:07:00 -0400
Message-ID: <20190816170700.GC10481@google.com> (raw)
In-Reply-To: <20190816165242.GS28441@linux.ibm.com>

On Fri, Aug 16, 2019 at 09:52:42AM -0700, Paul E. McKenney wrote:
> On Fri, Aug 16, 2019 at 12:24:04PM -0400, Joel Fernandes wrote:
> > On Thu, Aug 15, 2019 at 10:53:11PM -0400, Joel Fernandes (Google) wrote:
> > > I really cannot explain this patch, but without it, the "else if" block
> > > just doesn't execute thus causing the tick's dep mask to not be set and
> > > causes the tick to be turned off.
> > > 
> > > I tried various _ONCE() macros but the only thing that works is this
> > > patch.
> > > 
> > > Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> > > ---
> > >  kernel/rcu/tree.c | 3 ++-
> > >  1 file changed, 2 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> > > index 856d3c9f1955..ac6bcf7614d7 100644
> > > --- a/kernel/rcu/tree.c
> > > +++ b/kernel/rcu/tree.c
> > > @@ -802,6 +802,7 @@ static __always_inline void rcu_nmi_enter_common(bool irq)
> > >  {
> > >  	struct rcu_data *rdp = this_cpu_ptr(&rcu_data);
> > >  	long incby = 2;
> > > +	int dnn = rdp->dynticks_nmi_nesting;
> > 
> > I believe the accidental sign extension / conversion from long to int was
> > giving me an illusion since things started working well. Changing the 'int
> > dnn' to 'long dnn' gives similar behavior as without this patch! At least I
> > know now. Please feel free to ignore this particular RFC patch while I debug
> > this more (over the weekend or early next week). The first 2 patches are
> > good, just ignore this one.
> 
> Ah, good point on the type!  So you were ending up with zero due to the
> low-order 32 bits of DYNTICK_IRQ_NONIDLE being zero, correct?  If so,
> the "!rdp->dynticks_nmi_nesting" instead needs to be something like
> "rdp->dynticks_nmi_nesting == DYNTICK_IRQ_NONIDLE", which sounds like
> it is actually worse then the earlier comparison against the constant 2.
> 
> Sounds like I should revert the -rcu commit 805a16eaefc3 ("rcu: Force
> nohz_full tick on upon irq enter instead of exit").

I think just using doing " == DYNTICK_IRQ_NONIDLE" as you mentioned should
make it work. I'll test that soon, thanks!

I would prefer not to revert that commit, and just make the above change.
Just because I feel this is safer. Since the tick is turned off in the IRQ
exit path, I am a bit worried about timing (does the tick turn off before RCU
sees the IRQ exit, or after it?). Either way, doing it on IRQ entry makes the
question irrelevant and immune to future changes in the timing.

Would you think the check for the nesting variable is more expensive to do on
IRQ entry than exit? If so, we could discuss doing it in the exit path,
otherwise we could doing on entry with just the above change in the equality
condition.

thanks,

 - Joel

> 
> 								Thanx, Paul
> 
> > thanks,
> > 
> >  - Joel
> > 
> > 
> > >  
> > >  	/* Complain about underflow. */
> > >  	WARN_ON_ONCE(rdp->dynticks_nmi_nesting < 0);
> > > @@ -826,7 +827,7 @@ static __always_inline void rcu_nmi_enter_common(bool irq)
> > >  
> > >  		incby = 1;
> > >  	} else if (tick_nohz_full_cpu(rdp->cpu) &&
> > > -		   !rdp->dynticks_nmi_nesting &&
> > > +		   !dnn &&
> > >  		   rdp->rcu_urgent_qs && !rdp->rcu_forced_tick) {
> > >  		rdp->rcu_forced_tick = true;
> > >  		tick_dep_set_cpu(rdp->cpu, TICK_DEP_BIT_RCU);
> > > -- 
> > > 2.23.0.rc1.153.gdeed80330f-goog
> > > 
> > 

  reply index

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-16  2:53 [PATCH -rcu dev 1/3] rcu/tree: tick_dep_set/clear_cpu should accept bits instead of masks Joel Fernandes (Google)
2019-08-16  2:53 ` [PATCH -rcu dev 2/3] rcu/tree: Fix issue where sometimes rcu_urgent_qs is not set on IPI Joel Fernandes (Google)
2019-08-16  2:57   ` Joel Fernandes
2019-08-16  2:53 ` [PATCH -rcu dev 3/3] RFC: rcu/tree: Read dynticks_nmi_nesting in advance Joel Fernandes (Google)
2019-08-16 16:24   ` Joel Fernandes
2019-08-16 16:52     ` Paul E. McKenney
2019-08-16 17:07       ` Joel Fernandes [this message]
2019-08-16 17:30         ` Paul E. McKenney
2019-08-19 12:59       ` Frederic Weisbecker
2019-08-19 14:22         ` Joel Fernandes
2019-08-19 14:41           ` Paul E. McKenney
2019-08-19 15:46             ` Joel Fernandes
2019-08-19 16:17               ` Paul E. McKenney
2019-08-19 14:40         ` Paul E. McKenney
2019-08-16  2:59 ` [PATCH v2 -rcu dev 2/3] rcu/tree: Fix issue where sometimes rcu_urgent_qs is not set on IPI Joel Fernandes (Google)
2019-08-16 17:04   ` Paul E. McKenney
2019-08-16 17:25 ` [PATCH -rcu dev 1/3] rcu/tree: tick_dep_set/clear_cpu should accept bits instead of masks Paul E. McKenney
2019-08-16 17:35   ` Joel Fernandes
2019-08-19 12:38 ` Frederic Weisbecker
2019-08-19 14:46   ` Paul E. McKenney
2019-08-19 16:32     ` Frederic Weisbecker
2019-08-19 16:44       ` Paul E. McKenney
2019-08-20 12:08         ` Frederic Weisbecker
2019-08-20 14:44           ` Paul E. McKenney

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190816170700.GC10481@google.com \
    --to=joel@joelfernandes.org \
    --cc=frederic@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=paulmck@linux.ibm.com \
    --cc=rcu@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

RCU Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/rcu/0 rcu/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 rcu rcu/ https://lore.kernel.org/rcu \
		rcu@vger.kernel.org
	public-inbox-index rcu

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.rcu


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git