From: Alan Stern <stern@rowland.harvard.edu>
To: Will Deacon <will.deacon@arm.com>
Cc: LKMM Maintainers -- Akira Yokosawa <akiyks@gmail.com>,
Andrea Parri <andrea.parri@amarulasolutions.com>,
Boqun Feng <boqun.feng@gmail.com>,
David Howells <dhowells@redhat.com>,
Jade Alglave <j.alglave@ucl.ac.uk>,
Luc Maranget <luc.maranget@inria.fr>,
Nicholas Piggin <npiggin@gmail.com>,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Peter Zijlstra <peterz@infradead.org>,
Kernel development list <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 2/2] tools/memory-model: Add write ordering by release-acquire and by locks
Date: Fri, 22 Jun 2018 15:11:37 -0400 (EDT) [thread overview]
Message-ID: <Pine.LNX.4.44L0.1806221458480.1454-100000@iolanthe.rowland.org> (raw)
In-Reply-To: <20180622183007.GD1802@arm.com>
On Fri, 22 Jun 2018, Will Deacon wrote:
> Hi Alan,
>
> On Fri, Jun 22, 2018 at 02:09:04PM -0400, Alan Stern wrote:
> > On Fri, 22 Jun 2018, Will Deacon wrote:
> > > On Thu, Jun 21, 2018 at 01:27:12PM -0400, Alan Stern wrote:
> > > > More than one kernel developer has expressed the opinion that the LKMM
> > > > should enforce ordering of writes by release-acquire chains and by
> > > > locking. In other words, given the following code:
> > > >
> > > > WRITE_ONCE(x, 1);
> > > > spin_unlock(&s):
> > > > spin_lock(&s);
> > > > WRITE_ONCE(y, 1);
> > > >
> > > > or the following:
> > > >
> > > > smp_store_release(&x, 1);
> > > > r1 = smp_load_acquire(&x); // r1 = 1
> > > > WRITE_ONCE(y, 1);
> > > >
> > > > the stores to x and y should be propagated in order to all other CPUs,
> > > > even though those other CPUs might not access the lock s or be part of
> > > > the release-acquire chain. In terms of the memory model, this means
> > > > that rel-rf-acq-po should be part of the cumul-fence relation.
> > > >
> > > > All the architectures supported by the Linux kernel (including RISC-V)
> > > > do behave this way, albeit for varying reasons. Therefore this patch
> > > > changes the model in accordance with the developers' wishes.
> > >
> > > Interesting...
> > >
> > > I think the second example would preclude us using LDAPR for load-acquire,
> >
> > What are the semantics of LDAPR? That instruction isn't included in my
> > year-old copy of the ARMv8.1 manual; the closest it comes is LDAR and
> > LDAXP.
>
> It's part of 8.3 and is documented in the latest Arm Arm:
>
> https://static.docs.arm.com/ddi0487/ca/DDI0487C_a_armv8_arm.pdf
>
> It's also included in the upstream armv8.cat file using the 'Q' set.
I'll have to look at that.
> > > so I'm surprised that RISC-V is ok with this. For example, the first test
> > > below is allowed on arm64.
> >
> > Does ARMv8 use LDAPR for smp_load_aquire()? If it doesn't, this is a
> > moot point.
>
> I don't think it's a moot point. We want new architectures to implement
> acquire/release efficiently, and it's not unlikely that they will have
> acquire loads that are similar in semantics to LDAPR. This patch prevents
> them from doing so, and it also breaks Power and RISC-V without any clear
> justification for the stronger semantics.
>
> > > I also think this would break if we used DMB LD to implement load-acquire
> > > (second test below).
> >
> > Same question.
>
> Same answer (and RISC-V is a concrete example of an architecture building
> acquire using a load->load+store fence).
>
> > > So I'm not a big fan of this change, and I'm surprised this works on all
> > > architectures. What's the justification?
> >
> > For ARMv8, I've been going by something you wrote in an earlier email
> > to the effect that store-release and load-acquire are fully ordered,
> > and therefore a release can never be forwarded to an acquire. Is that
> > still true? But evidently it only justifies patch 1 in this series,
> > not patch 2.
>
> LDAR and STLR are RCsc, so that remains true. arm64 is not broken by this
> patch, but I'm still objecting to the change in semantics.
>
> > For RISC-V, I've been going by Andrea's and Luc's comments.
>
> https://is.gd/WhV1xz
>
> From that state of rmem, you can propagate the writes out of order on
> RISC-V.
>
> > > > Reading back some of the old threads [1], it seems the direct
> > > > translation of the first into acquire-release would be:
> > > >
> > > > WRITE_ONCE(x, 1);
> > > > smp_store_release(&s, 1);
> > > > r1 = smp_load_acquire(&s);
> > > > WRITE_ONCE(y, 1);
> > > >
> > > > Which is I think easier to make happen than the second example you give.
> > >
> > > It's easier, but it will still break on architectures with native support
> > > for RCpc acquire/release.
> >
> > Again, do we want the kernel to support that?
>
> Yes, I think we do. That's the most common interpretation of
> acquire/release, it matches what C11 has done and it facilitates
> construction of acquire using a load->load+store fence.
>
> > For that matter, what would happen if someone were to try using RCpc
> > semantics for lock/unlock? Or to put it another way, why do you
> > contemplate the possibility of RCpc acquire/release but not RCpc
> > lock/unlock?
>
> I think lock/unlock is a higher-level abstraction than acquire/release
> and therefore should be simpler to use and easier to reason about.
> acquire/release are building blocks for more complicated synchronisation
> mechanisms and we shouldn't be penalising their implementation without good
> reason.
>
> > > Could we drop the acquire/release stuff from the patch and limit this change
> > > to locking instead?
> >
> > The LKMM uses the same CAT code for acquire/release and lock/unlock.
> > (In essence, it considers a lock to be an acquire and an unlock to be a
> > release; everything else follows from that.) Treating one differently
> > from the other in these tests would require some significant changes.
> > It wouldn't be easy.
>
> It would be boring if it was easy ;) I think this is a case of the tail
> wagging the dog.
>
> Paul -- please can you drop this patch until we've resolved this discussion?
Agreed. It sounds like we'll need two versions of the Rel and Acq sets
in the memory model; one for RCpc and one for RCsc. smp_load_acquire
and smp_store_release will use the former, and locking will use the
latter.
Would it suffice to have this duplication just for release, using a
single version of acquire? What would happen on ARMv8 or RISC-V if an
RCsc release was read by an RCpc acquire? Or vice versa?
Alan
next prev parent reply other threads:[~2018-06-22 19:11 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-21 17:27 [PATCH 2/2] tools/memory-model: Add write ordering by release-acquire and by locks Alan Stern
2018-06-21 18:04 ` Peter Zijlstra
2018-06-22 3:34 ` Paul E. McKenney
2018-06-22 8:08 ` Peter Zijlstra
2018-06-22 8:09 ` Will Deacon
2018-06-22 9:55 ` Will Deacon
2018-06-22 10:31 ` Peter Zijlstra
2018-06-22 10:38 ` Will Deacon
2018-06-22 11:25 ` Andrea Parri
2018-06-22 16:40 ` Paul E. McKenney
2018-06-22 18:09 ` Alan Stern
2018-06-22 18:30 ` Will Deacon
2018-06-22 19:11 ` Alan Stern [this message]
2018-06-22 20:53 ` Paul E. McKenney
2018-07-04 11:53 ` Will Deacon
2018-06-25 8:19 ` Andrea Parri
2018-07-03 17:28 ` Alan Stern
2018-07-04 11:28 ` Paul E. McKenney
2018-07-04 12:13 ` Will Deacon
2018-07-05 14:23 ` Alan Stern
2018-07-05 15:31 ` Paul E. McKenney
2018-07-04 12:11 ` Will Deacon
2018-07-05 14:00 ` Andrea Parri
2018-07-05 14:44 ` Will Deacon
2018-07-05 15:16 ` Daniel Lustig
2018-07-05 15:35 ` Daniel Lustig
2018-07-05 14:21 ` Alan Stern
2018-07-05 14:46 ` Will Deacon
2018-07-05 14:57 ` Alan Stern
2018-07-05 15:15 ` Will Deacon
2018-07-05 15:09 ` Andrea Parri
2018-07-06 20:37 ` Alan Stern
2018-07-06 21:10 ` Paul E. McKenney
2018-07-09 16:52 ` Will Deacon
2018-07-09 17:29 ` Daniel Lustig
2018-07-09 19:18 ` Alan Stern
2018-07-05 15:31 ` Paul E. McKenney
2018-07-05 15:39 ` Andrea Parri
2018-07-05 16:58 ` Paul E. McKenney
2018-07-05 17:06 ` Andrea Parri
2018-07-05 15:44 ` Daniel Lustig
2018-07-05 16:22 ` Will Deacon
2018-07-05 16:56 ` Paul E. McKenney
2018-07-05 18:12 ` Daniel Lustig
2018-07-05 18:38 ` Andrea Parri
2018-07-05 18:44 ` Andrea Parri
2018-07-05 23:32 ` Paul E. McKenney
2018-07-05 23:31 ` Paul E. McKenney
2018-07-06 9:25 ` Will Deacon
2018-07-06 14:14 ` Paul E. McKenney
2018-06-25 7:32 ` Peter Zijlstra
2018-06-25 8:29 ` Andrea Parri
2018-06-25 9:06 ` Peter Zijlstra
2018-06-22 9:06 ` Andrea Parri
2018-06-22 19:23 ` Alan Stern
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Pine.LNX.4.44L0.1806221458480.1454-100000@iolanthe.rowland.org \
--to=stern@rowland.harvard.edu \
--cc=akiyks@gmail.com \
--cc=andrea.parri@amarulasolutions.com \
--cc=boqun.feng@gmail.com \
--cc=dhowells@redhat.com \
--cc=j.alglave@ucl.ac.uk \
--cc=linux-kernel@vger.kernel.org \
--cc=luc.maranget@inria.fr \
--cc=npiggin@gmail.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).