From: "Paul E. McKenney" <paulmck@linux.ibm.com>
To: Alan Stern <stern@rowland.harvard.edu>
Cc: Andrea Parri <andrea.parri@amarulasolutions.com>,
LKMM Maintainers -- Akira Yokosawa <akiyks@gmail.com>,
Boqun Feng <boqun.feng@gmail.com>,
Daniel Lustig <dlustig@nvidia.com>,
David Howells <dhowells@redhat.com>,
Jade Alglave <j.alglave@ucl.ac.uk>,
Luc Maranget <luc.maranget@inria.fr>,
Nicholas Piggin <npiggin@gmail.com>,
Peter Zijlstra <peterz@infradead.org>,
Will Deacon <will.deacon@arm.com>,
Daniel Kroening <kroening@cs.ox.ac.uk>,
Kernel development list <linux-kernel@vger.kernel.org>
Subject: Re: Adding plain accesses and detecting data races in the LKMM
Date: Thu, 18 Apr 2019 11:39:19 -0700 [thread overview]
Message-ID: <20190418183919.GO14111@linux.ibm.com> (raw)
In-Reply-To: <Pine.LNX.4.44L0.1904181324420.1303-100000@iolanthe.rowland.org>
On Thu, Apr 18, 2019 at 01:44:36PM -0400, Alan Stern wrote:
> On Thu, 18 Apr 2019, Andrea Parri wrote:
>
> > > Another question is "should the kernel permit smp_mb__{before,after}*()
> > > anywhere other than immediately before or after the primitive being
> > > strengthened?"
> >
> > Mmh, I do think that keeping these barriers "immediately before or after
> > the primitive being strengthened" is a good practice (readability, and
> > all that), if this is what you're suggesting.
> >
> > However, a first auditing of the callsites showed that this practice is
> > in fact not always applied, notably... ;-)
> >
> > kernel/rcu/tree_exp.h:sync_exp_work_done
> > kernel/sched/cpupri.c:cpupri_set
> >
> > So there appear, at least, to be some exceptions/reasons for not always
> > following it? Thoughts?
> >
> > BTW, while auditing these callsites, I've stumbled across the following
> > snippet (from kernel/futex.c):
> >
> > *futex = newval;
> > sys_futex(WAKE, futex);
> > futex_wake(futex);
> > smp_mb(); (B)
> > if (waiters)
> > ...
> >
> > where B is actually (c.f., futex_get_mm()):
> >
> > atomic_inc(...->mm_count);
> > smp_mb__after_atomic();
> >
> > It seems worth mentioning the fact that, AFAICT, this sequence does not
> > necessarily provide ordering when plain accesses are involved: consider,
> > e.g., the following variant of the snippet:
> >
> > A:*x = 1;
> > /*
> > * I've "ignored" the syscall, which should provide
> > * (at least) a compiler barrier...
> > */
> > atomic_inc(u);
> > smp_mb__after_atomic();
> > B:r0 = *y;
> >
> > On x86, AFAICT, the compiler can do this:
> >
> > atomic_inc(u);
> > A:*x = 1;
> > smp_mb__after_atomic();
> > B:r0 = *y;
> >
> > (the implementation of atomic_inc() contains no compiler barrier), then
> > the CPU can "reorder" A and B (smp_mb__after_atomic() being #defined to
> > a compiler barrier).
>
> Are you saying that on x86, atomic_inc() acts as a full memory barrier
> but not as a compiler barrier, and vice versa for
> smp_mb__after_atomic()? Or that neither atomic_inc() nor
> smp_mb__after_atomic() implements a full memory barrier?
>
> Either one seems like a very dangerous situation indeed.
If I am following the macro-name breadcrumb trails correctly, x86's
atomic_inc() does have a compiler barrier. But this is an accident
of implementation -- from what I can see, it is not required to do so.
So smb_mb__after_atomic() is only guaranteed to order the atomic_inc()
before B, not A. To order A before B in the above example, an
smp_mb__before_atomic() is also needed.
But now that I look, LKMM looks to be stating a stronger guarantee:
([M] ; fencerel(Before-atomic) ; [RMW] ; po? ; [M]) |
([M] ; po? ; [RMW] ; fencerel(After-atomic) ; [M]) |
([M] ; po? ; [LKW] ; fencerel(After-spinlock) ; [M]) |
([M] ; po ; [UL] ; (co | po) ; [LKW] ;
fencerel(After-unlock-lock) ; [M])
Maybe something like this?
([M] ; fencerel(Before-atomic) ; [RMW] ; fencerel(After-atomic) ; [M]) |
([M] ; fencerel(Before-atomic) ; [RMW] |
( [RMW] ; fencerel(After-atomic) ; [M]) |
([M] ; po? ; [LKW] ; fencerel(After-spinlock) ; [M]) |
([M] ; po ; [UL] ; (co | po) ; [LKW] ;
fencerel(After-unlock-lock) ; [M])
Who is the lead maintainer for this stuff, anyway??? ;-)
Thanx, Paul
> Alan
>
> > The mips implementation seems also affected by such "reorderings": I am
> > not familiar with this implementation but, AFAICT, it does not enforce
> > ordering from A to B in the following snippet:
> >
> > A:*x = 1;
> > atomic_inc(u);
> > smp_mb__after_atomic();
> > B:WRITE_ONCE(*y, 1);
> >
> > when CONFIG_WEAK_ORDERING=y, CONFIG_WEAK_REORDERING_BEYOND_LLSC=n.
> >
> > Do these observations make sense to you? Thoughts?
> >
> > Andrea
>
next prev parent reply other threads:[~2019-04-18 18:39 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-03-19 19:38 Adding plain accesses and detecting data races in the LKMM Alan Stern
2019-04-02 14:42 ` Andrea Parri
2019-04-02 18:06 ` Alan Stern
2019-04-06 0:49 ` Andrea Parri
2019-04-06 16:03 ` Alan Stern
2019-04-08 5:51 ` Andrea Parri
2019-04-08 14:18 ` Alan Stern
2019-04-09 1:36 ` Andrea Parri
2019-04-09 15:01 ` Paul E. McKenney
2019-04-13 21:39 ` Andrea Parri
2019-04-15 13:35 ` Paul E. McKenney
2019-04-15 13:50 ` Alan Stern
2019-04-15 13:53 ` Paul E. McKenney
2019-04-18 12:54 ` Andrea Parri
2019-04-18 17:44 ` Alan Stern
2019-04-18 18:39 ` Paul E. McKenney [this message]
2019-04-18 20:19 ` Alan Stern
2019-04-19 0:53 ` Andrea Parri
2019-04-19 12:47 ` Paul E. McKenney
2019-04-19 14:34 ` Alan Stern
2019-04-19 17:17 ` Paul E. McKenney
2019-04-19 15:06 ` Akira Yokosawa
2019-04-19 16:37 ` Andrea Parri
2019-04-19 18:06 ` Paul E. McKenney
2019-04-20 14:50 ` Akira Yokosawa
2019-04-21 19:38 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190418183919.GO14111@linux.ibm.com \
--to=paulmck@linux.ibm.com \
--cc=akiyks@gmail.com \
--cc=andrea.parri@amarulasolutions.com \
--cc=boqun.feng@gmail.com \
--cc=dhowells@redhat.com \
--cc=dlustig@nvidia.com \
--cc=j.alglave@ucl.ac.uk \
--cc=kroening@cs.ox.ac.uk \
--cc=linux-kernel@vger.kernel.org \
--cc=luc.maranget@inria.fr \
--cc=npiggin@gmail.com \
--cc=peterz@infradead.org \
--cc=stern@rowland.harvard.edu \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).