Re: [PATCH] refcount: provide same memory ordering guarantees as in atomic_t

From: Alan Stern <stern@rowland.harvard.edu>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Will Deacon <will.deacon@arm.com>,
	"Reshetova, Elena" <elena.reshetova@intel.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
	"keescook@chromium.org" <keescook@chromium.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"ishkamiel@gmail.com" <ishkamiel@gmail.com>,
	Paul McKenney <paulmck@linux.vnet.ibm.com>,
	<parri.andrea@gmail.com>, <boqun.feng@gmail.com>,
	<dhowells@redhat.com>, <david@fromorbit.com>
Subject: Re: [PATCH] refcount: provide same memory ordering guarantees as in atomic_t
Date: Wed, 15 Nov 2017 15:22:33 -0500 (EST)	[thread overview]
Message-ID: <Pine.LNX.4.44L0.1711151516380.1363-100000@iolanthe.rowland.org> (raw)
In-Reply-To: <20171115200307.ns4ja7xjwhunen65@hirez.programming.kicks-ass.net>

On Wed, 15 Nov 2017, Peter Zijlstra wrote:

> On Wed, Nov 15, 2017 at 02:15:19PM -0500, Alan Stern wrote:
> > On Wed, 15 Nov 2017, Will Deacon wrote:
> > 
> > > On Thu, Nov 02, 2017 at 04:21:56PM -0400, Alan Stern wrote:
> > > > I was trying to think of something completely different.  If you have a
> > > > release/acquire to the same address, it creates a happens-before
> > > > ordering:
> > > > 
> > > > 	Access x
> > > > 	Release a
> > > > 	Acquire a
> > > > 	Access y
> > > > 
> > > > Here is the access to x happens-before the access to y.  This is true
> > > > even on x86, even in the presence of forwarding -- the CPU still has to
> > > > execute the instructions in order.  But if the release and acquire are
> > > > to different addresses:
> > > > 
> > > > 	Access x
> > > > 	Release a
> > > > 	Acquire b
> > > > 	Access y
> > > > 
> > > > then there is no happens-before ordering for x and y -- the CPU can
> > > > execute the last two instructions before the first two.  x86 and
> > > > PowerPC won't do this, but I believe ARMv8 can.  (Please correct me if
> > > > it can't.)
> > > 
> > > Release/Acquire are RCsc on ARMv8, so they are ordered irrespective of
> > > address.
> > 
> > Ah, okay, thanks.
> > 
> > In any case, we have considered removing this ordering constraint
> > (store-release followed by load-acquire for the same location) from the
> > Linux-kernel memory model.
> 
> Why? Its a perfectly sensible construct.
> 
> > I'm not aware of any code in the kernel that depends on it.  Do any of
> > you happen to know of any examples?
> 
> All locks? Something like:
> 
> 	spin_lock(&x)
> 	/* foo */
> 	spin_unlock(&x)
> 	spin_lock(&x)
> 	/* bar */
> 	spin_unlock(&x);
> 
> Has a fairly high foo happens-before bar expectation level.
> 
> And in specific things like:
> 
>   135e8c9250dd5
>   ecf7d01c229d1
> 
> which use the release of rq->lock paired with the next acquire of the
> same rq->lock to match with an smp_rmb().

You know, sometimes I feel like I'm losing my mind.

Yes, of course -- this was in fact the original reason for adding that
constraint to the memory model in the first place!  An unlock-to-lock
link between two CPUs would naturally create an ordering relation, and
we wanted the same to be true when everything occurred on a single CPU.

I'll shut up now...

Alan