linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Nicholas Piggin <npiggin@gmail.com>
To: "Paul E. McKenney" <paulmck@kernel.org>, Will Deacon <will@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	dlustig@nvidia.com, Stephane Eranian <eranian@google.com>,
	Peter Anvin <hpa@zytor.com>, Jiri Olsa <jolsa@redhat.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-tip-commits@vger.kernel.org, Ingo Molnar <mingo@kernel.org>,
	mpe@ellerman.id.au, palmer@dabbelt.com,
	Andrea Parri <parri.andrea@gmail.com>,
	paul.walmsley@sifive.com, Peter Zijlstra <peterz@infradead.org>,
	Alan Stern <stern@rowland.harvard.edu>,
	Thomas Gleixner <tglx@linutronix.de>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Vince Weaver <vincent.weaver@maine.edu>
Subject: Re: [tip:locking/core] tools/memory-model: Add extra ordering for locks and remove it for ordinary release/acquire
Date: Fri, 17 Sep 2021 15:31:29 +1000	[thread overview]
Message-ID: <1631855601.07l1who4w0.astroid@bobo.none> (raw)
In-Reply-To: <1631847849.o57vj41jx3.astroid@bobo.none>

Excerpts from Nicholas Piggin's message of September 17, 2021 1:21 pm:
> Excerpts from Will Deacon's message of September 10, 2021 9:08 pm:
>> Hi Paul,
>> 
>> On Thu, Sep 09, 2021 at 10:46:35AM -0700, Paul E. McKenney wrote:
>>> On Thu, Sep 09, 2021 at 02:35:36PM +0100, Will Deacon wrote:
>>> > On Thu, Sep 09, 2021 at 09:25:30AM +0200, Peter Zijlstra wrote:
>>> > > On Wed, Sep 08, 2021 at 09:08:33AM -0700, Linus Torvalds wrote:
>>> > > > then I think it's entirely reasonable to
>>> > > > 
>>> > > >         spin_unlock(&r);
>>> > > >         spin_lock(&s);
>>> > > > 
>>> > > > cannot be reordered.
>>> > > 
>>> > > I'm obviously completely in favour of that :-)
>>> > 
>>> > I don't think we should require the accesses to the actual lockwords to
>>> > be ordered here, as it becomes pretty onerous for relaxed LL/SC
>>> > architectures where you'd end up with an extra barrier either after the
>>> > unlock() or before the lock() operation. However, I remain absolutely in
>>> > favour of strengthening the ordering of the _critical sections_ guarded by
>>> > the locks to be RCsc.
>>> 
>>> If by this you mean the critical sections when observed only by other
>>> critical sections for a given lock, then everyone is already there.
>> 
>> No, I mean the case where somebody without the lock (but using memory
>> barriers) can observe the critical sections out of order (i.e. W -> R
>> order is not maintained).
> 
> This is a sincere question, why is this important? I mean _any_ 
> restriction on reordering makes things easier by definition I can't 
> argue with that, but why is this one in particular seen as a problem?
> It just seems disproportionate.
> 
> We naturally think of accesses within locks as atomic as a whole 
> (provided the other parties are doing the proper locking too). So like 
> atomic operations, aligned stores, etc can be reordered, I don't see why
> these should have any particular ordering either, or why a unlock() 
> should pair with a later lock() of an unrelated lock to provide some
> ordering.
> 
> It gives the idea that individual lock operations in isolation should be 
> or do something special, but I think that's the wrong way to think about 
> it, the lock and the unlock operate on a specific lock word and protect 
> specific data vs other processors that access the same data under the
> same locks.
> 
> If you don't know what you're doing or don't want to think about 
> ordering, perform accesses under locks. If you don't lock, you get to
> think about ordering. At which point sure two sets of operations from
> different critical sections could go out of order, but so can any two
> stores. Or two stores from inside the one critical section if you are
> not holding the correct lock.

It doesn't actually really relieve the burden of thinking about barriers 
mcuh at all, come to think of it.

spin_lock(&foo);
x = 1;
spin_unlock(&foo);
spin_lock(&bar);
y = 1;
spin_unlock(&bar);

vs

if (READ_ONCE(y) == 1)
    // spin_unlock(&foo)+spin_lock(&bar) provides store ordering
    smp_rmb();
    BUG_ON(READ_ONCE(x) == 0);

Then if you didn't comment the store ordering requirement in the first
code, then you patch things to simplify or add functionality:

spin_lock(&foo);
x = 1;
spin_lock(&bar);
y = 1;
spin_unlock(&bar);
z = 1;
spin_unlock(&foo);

or

spin_lock(&baz);
x = 1;
y = 1;
spin_unlock(&baz);

Then you broke it. Because you thought being clever and avoiding
thinking about or talking about ordering in your code which performs 
memory accesses outside of locks was improving the situation.

It's not good practice. If there is *any* unlocked memory access, you 
should always think about and comment *all* memory orderings. Locking
should not ever be relied on to give you some kind of implicit semantics
that you think should be obvious. All the accesses always need thought
and they always need comments.

spin_lock(&foo);
// The spin_unlock+spin_lock orders this store before the store to y, 
// the corresponding smp_rmb is at function().
x = 1;
spin_unlock(&foo);
spin_lock(&bar);
// See x
y = 1;
spin_unlock(&bar);

Once you do that, does it *really* break the bank to add a line of code?

spin_lock(&foo);
// See smp_unlock_lock_mb() below
x = 1;
spin_unlock(&foo);
// This orders this store of x above before the
// store to y below. 
smp_unlock_lock_mb();
spin_lock(&bar);
// See smp_unlock_lock_mb() above
y = 1;
spin_unlock(&bar);

I can't see how that line of code created a fundamentally a bigger 
problem.

If you don't need the performance and don't want to deal with ordering, 
*always use locks*. If you absolutely can't always use locks, *always 
document important memory accesses ordering that is or can affect 
unlocked memory accesses*. We are all agreed on this rule, right? So
what am I missing?

Thanks,
Nick

  reply	other threads:[~2021-09-17  5:32 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-09-26 18:28 [PATCH memory-model 0/5] Updates to the formal memory model Paul E. McKenney
2018-09-26 18:29 ` [PATCH memory-model 1/5] tools/memory-model: Add litmus-test naming scheme Paul E. McKenney
2018-10-02 10:10   ` [tip:locking/core] " tip-bot for Paul E. McKenney
2018-09-26 18:29 ` [PATCH memory-model 2/5] tools/memory-model: Add extra ordering for locks and remove it for ordinary release/acquire Paul E. McKenney
2018-10-02 10:11   ` [tip:locking/core] " tip-bot for Alan Stern
2021-09-08 11:00     ` Peter Zijlstra
2021-09-08 11:44       ` Peter Zijlstra
2021-09-08 14:42         ` Alan Stern
2021-09-08 15:12           ` Peter Zijlstra
2021-09-08 16:08           ` Linus Torvalds
2021-09-09  7:25             ` Peter Zijlstra
2021-09-09 13:35               ` Will Deacon
2021-09-09 17:02                 ` Linus Torvalds
2021-09-09 18:59                   ` Alan Stern
2021-09-09 17:03                 ` Dan Lustig
2021-09-09 18:00                   ` Paul E. McKenney
2021-09-10 14:20                     ` Boqun Feng
2021-09-10 15:33                       ` Palmer Dabbelt
2021-09-10 16:36                       ` Alan Stern
2021-09-10 17:12                         ` Peter Zijlstra
2021-09-10 17:56                           ` Alan Stern
2021-09-10 17:17                         ` Peter Zijlstra
2021-09-12  0:26                         ` Boqun Feng
2021-09-10  0:01                   ` Boqun Feng
2021-09-10  5:37                     ` Boqun Feng
2021-09-10  9:33                     ` Peter Zijlstra
2021-09-10 10:04                       ` Boqun Feng
2021-09-10 13:48                         ` Dan Lustig
2021-09-10 14:15                           ` Boqun Feng
2021-09-09 17:46                 ` Paul E. McKenney
2021-09-10 11:08                   ` Will Deacon
2021-09-17  3:21                     ` Nicholas Piggin
2021-09-17  5:31                       ` Nicholas Piggin [this message]
2021-09-17 14:36                     ` Michael Ellerman
2018-09-26 18:29 ` [PATCH memory-model 3/5] tools/memory-model: Fix a README typo Paul E. McKenney
2018-10-02 10:11   ` [tip:locking/core] " tip-bot for SeongJae Park
2018-09-26 18:29 ` [PATCH memory-model 4/5] tools/memory-model: Add more LKMM limitations Paul E. McKenney
2018-10-02 10:12   ` [tip:locking/core] " tip-bot for Paul E. McKenney
2018-09-26 18:29 ` [PATCH memory-model 5/5] doc: Replace smp_cond_acquire() with smp_cond_load_acquire() Paul E. McKenney
2018-10-02 10:12   ` [tip:locking/core] locking/memory-barriers: " tip-bot for Andrea Parri
2018-10-02  8:28 ` [PATCH memory-model 0/5] Updates to the formal memory model Ingo Molnar

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1631855601.07l1who4w0.astroid@bobo.none \
    --to=npiggin@gmail.com \
    --cc=acme@redhat.com \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=dlustig@nvidia.com \
    --cc=eranian@google.com \
    --cc=hpa@zytor.com \
    --cc=jolsa@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-tip-commits@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=mpe@ellerman.id.au \
    --cc=palmer@dabbelt.com \
    --cc=parri.andrea@gmail.com \
    --cc=paul.walmsley@sifive.com \
    --cc=paulmck@kernel.org \
    --cc=peterz@infradead.org \
    --cc=stern@rowland.harvard.edu \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    --cc=vincent.weaver@maine.edu \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).