From: "Paul E. McKenney" <paulmck@kernel.org>
To: maranget <luc.maranget@inria.fr>
Cc: Randy Dunlap <rdunlap@infradead.org>,
szyhb810501.student@sina.com, stern <stern@rowland.harvard.edu>,
"parri.andrea" <parri.andrea@gmail.com>, will <will@kernel.org>,
peterz <peterz@infradead.org>,
"boqun.feng" <boqun.feng@gmail.com>, npiggin <npiggin@gmail.com>,
dhowells <dhowells@redhat.com>, "j.alglave" <j.alglave@ucl.ac.uk>,
linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: Documentation/memory-barriers.txt: Is "stores are not speculated" correct?
Date: Mon, 26 Apr 2021 08:34:37 -0700 [thread overview]
Message-ID: <20210426153437.GZ975577@paulmck-ThinkPad-P17-Gen-1> (raw)
In-Reply-To: <8846FB06-D0CD-4880-93A5-E6D959AF23C1@inria.fr>
On Mon, Apr 26, 2021 at 05:16:15PM +0200, maranget wrote:
>
>
> > On 26 Apr 2021, at 17:13, Randy Dunlap <rdunlap@infradead.org> wrote:
> >
> > On 4/26/21 2:30 AM, Luc Maranget wrote:
> >>> On Mon, Apr 26, 2021 at 10:23:09AM +0800, szyhb810501.student@sina.com wrote:
> >>>>
> >>>> Hello everyone, I have a question."Documentation/memory-barriers.txt"
> >>>> says:However, stores are not speculated. This means that ordering -is-
> >>>> providedfor load-store control dependencies, as in the following example:
> >>> q = READ_ONCE(a);
> >>> if (q) {
> >>> WRITE_ONCE(b, 1);
> >>> }
> >>>> Is "stores are not speculated" correct? I
> >>>> think store instructions can be executed speculatively.
> >>>> "https://stackoverflow.com/questions/64141366/can-a-speculatively-executed-cpu-branch-contain-opcodes-that-access-ram"
> >>>> says:Store instructions can also be executed speculatively thanks to the
> >>>> store buffer. The actual execution of a store just writes the address and
> >>>> data into the store buffer.Commit to L1d cache happens some time after
> >>>> the store instruction retires from the ROB, i.e. when the store is known
> >>>> to be non-speculative, the associated store-buffer entry "graduates"
> >>>> and becomes eligible to commit to cache and become globally visible.
> >>>
> >>>> From the viewpoint of other CPUs, the store hasn't really happened
> >>> until it finds its way into a cacheline. As you yourself note above,
> >>> if the store is still in the store buffer, it might be squashed when
> >>> speculation fails.
> >>>
> >>> So Documentation/memory-barriers.txt and that stackoverflow entry are
> >>> not really in conflict, but are instead using words a bit differently
> >>> from each other. The stackoverflow entry is considering a store to have
> >>> in some sense happened during a time when it might later be squashed.
> >>> In contrast, the Documentation/memory-barriers.txt document only considers
> >>> a store to have completed once it is visible outside of the CPU executing
> >>> that store.
> >>>
> >>> So from a stackoverflow viewpoint, stores can be speculated, but until
> >>> they are finalized, they must be hidden from other CPUs.
> >>>
> >>>> From a Documentation/memory-barriers.txt viewpoint, stores don't complete
> >>> until they update their cachelines, and stores may not be speculated.
> >>> Some of the actions that lead up to the completion of a store may be
> >>> speculated, but not the completion of the store itself.
> >>>
> >>> Different words, but same effect. Welcome to our world! ;-)
> >>>
> >>> Thanx, Paul
> >>
> >> Hi all,
> >>
> >> Here is a complement to Paul's excellent answer.
> >>
> >> The "CPU-local" speculation of stores can be observed
> >> by the following test (in C11)
> >>
> >> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
> >>
> >> C PPOCA
> >>
> >> {}
> >>
> >> P0(volatile int* y, volatile int* x) {
> >>
> >> atomic_store(x,1);
> >> atomic_store(y,1);
> >>
> >> }
> >>
> >> P1(volatile int* z, volatile int* y, volatile int* x) {
> >>
> >> int r1=-1; int r2=-1;
> >> int r0 = atomic_load_explicit(y,memory_order_relaxed);
> >> if (r0) {
> >> atomic_store_explicit(z,1,memory_order_relaxed);
> >> r1 = atomic_load_explicit(z,memory_order_relaxed);
> >> r2 = atomic_load_explicit(x+(r1 & 128),memory_order_relaxed);
> >> }
> >>
> >> }
> >>
> >>
> >> This is a variation on the MP test.
> >>
> >> Because of tht conditionnal "if (..) { S }" Statements "S" can be executed
> >> speculatively.
> >>
> >> More precisely, the store statement writes value 1 into the CPU local
> >> structure for variable z. The next load statement reads the value,
> >> and the last load statement can be peformed (speculatively)
> >> as its address is known.
> >>
> >> The resulting outcomme is observed for instance on a RaspBerry Pi3,
> >> see attached file.
> >
> > ?attached file?
> >
> > --
> > ~Randy
> >
>
> Oups, sorry I forgot the attachement:
>
> —Luc
> Mon Apr 26 09:07:19 UTC 2021
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%
> % Results for PPOCA.litmus %
> %%%%%%%%%%%%%%%%%%%%%%%%%%%%
> C PPOCA
>
> {}
>
> P0(volatile int* y, volatile int* x) {
>
> atomic_store(x,1);
> atomic_store(y,1);
>
> }
>
> P1(volatile int* z, volatile int* y, volatile int* x) {
>
> int r1=-1; int r2=-1;
> int r0 = atomic_load_explicit(y,memory_order_relaxed);
> if (r0) {
> atomic_store_explicit(z,1,memory_order_relaxed);
> r1 = atomic_load_explicit(z,memory_order_relaxed);
> r2 = atomic_load_explicit(x+(r1 & 128),memory_order_relaxed);
> }
>
> }
>
>
> exists (1:r0=1 /\ 1:r1=1 /\ 1:r2=0)
>
> Histogram (3 states)
> 11057696:>1:r0=0; 1:r1=-1; 1:r2=-1;
> 2 *>1:r0=1; 1:r1=1; 1:r2=0;
Fun!!! ;-)
Thanx, Paul
> 8942302:>1:r0=1; 1:r1=1; 1:r2=1;
> Ok
>
> Witnesses
> Positive: 2, Negative: 19999998
> Condition exists (1:r0=1 /\ 1:r1=1 /\ 1:r2=0) is validated
> Hash=bb2426936c19f1555410d1483dd31452
> Observation PPOCA Sometimes 2 19999998
> Time PPOCA 3.30
> Revision 45690d9d0f7a956a6d3dbaf9e912efb22835756e, version 7.56+02~dev
> Command line: litmus7 -mach vougeot -c11 true -o R.tar PPOCA.litmus
> Parameters
> #define SIZE_OF_TEST 10000
> #define NUMBER_OF_RUN 100
> #define AVAIL 4
> #define STRIDE 1
> #define MAX_LOOP 0
> /* gcc options: -Wall -std=gnu11 -O2 -pthread */
> /* barrier: userfence */
> /* launch: changing */
> /* affinity: none */
> /* alloc: dynamic */
> /* memory: direct */
> /* stride: 1 */
> /* safer: write */
> /* preload: random */
> /* speedcheck: no */
> /* proc used: 4 */
> GCC=gcc
> LITMUSOPTS=-s 5k -r 2k -st 1
> Mon Apr 26 09:07:23 UTC 2021
>
>
prev parent reply other threads:[~2021-04-26 15:34 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20210426022309.2333D4640475@webmail.sinamail.sina.com.cn>
2021-04-26 3:50 ` Documentation/memory-barriers.txt: Is "stores are not speculated" correct? Paul E. McKenney
2021-04-26 9:30 ` Luc Maranget
2021-04-26 15:13 ` Randy Dunlap
2021-04-26 15:16 ` maranget
2021-04-26 15:34 ` Paul E. McKenney [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210426153437.GZ975577@paulmck-ThinkPad-P17-Gen-1 \
--to=paulmck@kernel.org \
--cc=boqun.feng@gmail.com \
--cc=dhowells@redhat.com \
--cc=j.alglave@ucl.ac.uk \
--cc=linux-kernel@vger.kernel.org \
--cc=luc.maranget@inria.fr \
--cc=npiggin@gmail.com \
--cc=parri.andrea@gmail.com \
--cc=peterz@infradead.org \
--cc=rdunlap@infradead.org \
--cc=stern@rowland.harvard.edu \
--cc=szyhb810501.student@sina.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).