Re: [PATCH tip/core/rcu 02/18] rcu: Move rcu_report_exp_rnp() to allow consolidation

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Tue, Oct 06, 2015 at 10:29:37PM +0200, Peter Zijlstra wrote:
> On Tue, Oct 06, 2015 at 09:29:21AM -0700, Paul E. McKenney wrote:
> > +static void __maybe_unused rcu_report_exp_rnp(struct rcu_state *rsp,
> > +					      struct rcu_node *rnp, bool wake)
> > +{
> > +	unsigned long flags;
> > +	unsigned long mask;
> > +
> > +	raw_spin_lock_irqsave(&rnp->lock, flags);
> 
> Normally we require a comment with barriers, explaining the order and
> the pairing etc.. :-)
> 
> > +	smp_mb__after_unlock_lock();

Hmmmm...  That is not good.

Worse yet, I am missing comments on most of the pre-existing barriers
of this form.

The purpose is to enforce the heavy-weight grace-period memory-ordering
guarantees documented in the synchronize_sched() header comment and
elsewhere.  They pair with anything you might use to check for violation
of these guarantees, or, simiarly, any ordering that you might use when
relying on these guarantees.

I could add something like  "/* Enforce GP memory ordering. */"

Or perhaps "/* See synchronize_sched() header. */"

I do not propose reproducing the synchronize_sched() header on each
of these.  That would be verbose, even for me!  ;-)

Other thoughts?

							Thanx, Paul

> > +	for (;;) {
> > +		if (!sync_rcu_preempt_exp_done(rnp)) {
> > +			raw_spin_unlock_irqrestore(&rnp->lock, flags);
> > +			break;
> > +		}
> > +		if (rnp->parent == NULL) {
> > +			raw_spin_unlock_irqrestore(&rnp->lock, flags);
> > +			if (wake) {
> > +				smp_mb(); /* EGP done before wake_up(). */
> > +				wake_up(&rsp->expedited_wq);
> > +			}
> > +			break;
> > +		}
> > +		mask = rnp->grpmask;
> > +		raw_spin_unlock(&rnp->lock); /* irqs remain disabled */
> > +		rnp = rnp->parent;
> > +		raw_spin_lock(&rnp->lock); /* irqs already disabled */
> > +		smp_mb__after_unlock_lock();
> > +		rnp->expmask &= ~mask;
> > +	}
> > +}
>