Re: [RFC][PATCH 2/5] mips/atomic: Fix loongson_llsc_mb() wreckage

From: Peter Zijlstra <peterz@infradead.org>
To: Paul Burton <paul.burton@mips.com>
Cc: "stern@rowland.harvard.edu" <stern@rowland.harvard.edu>,
	"akiyks@gmail.com" <akiyks@gmail.com>,
	"andrea.parri@amarulasolutions.com" 
	<andrea.parri@amarulasolutions.com>,
	"boqun.feng@gmail.com" <boqun.feng@gmail.com>,
	"dlustig@nvidia.com" <dlustig@nvidia.com>,
	"dhowells@redhat.com" <dhowells@redhat.com>,
	"j.alglave@ucl.ac.uk" <j.alglave@ucl.ac.uk>,
	"luc.maranget@inria.fr" <luc.maranget@inria.fr>,
	"npiggin@gmail.com" <npiggin@gmail.com>,
	"paulmck@linux.ibm.com" <paulmck@linux.ibm.com>,
	"will.deacon@arm.com" <will.deacon@arm.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"torvalds@linux-foundation.org" <torvalds@linux-foundation.org>,
	Huacai Chen <chenhc@lemote.com>, Huang Pei <huangpei@loongson.cn>
Subject: Re: [RFC][PATCH 2/5] mips/atomic: Fix loongson_llsc_mb() wreckage
Date: Thu, 25 Apr 2019 09:15:28 +0200	[thread overview]
Message-ID: <20190425071528.GU11158@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <20190424211759.52xraajqwudc2fza@pburton-laptop>

On Wed, Apr 24, 2019 at 09:18:04PM +0000, Paul Burton wrote:
> Hi Peter,
> 
> On Wed, Apr 24, 2019 at 02:36:58PM +0200, Peter Zijlstra wrote:
> > The comment describing the loongson_llsc_mb() reorder case doesn't
> > make any sense what so ever. Instruction re-ordering is not an SMP
> > artifact, but rather a CPU local phenomenon. This means that _every_
> > LL/SC loop needs this barrier right in front to avoid the CPU from
> > leaking a memop inside it.
> 
> Does it?

It does, however..

> The Loongson bug being described here causes an sc to succeed
> erroneously if certain loads or stores are executed between the ll &
> associated sc, including speculatively. On a UP system there's no code
> running on other cores to race with us & cause our sc to fail - ie. sc
> should always succeed anyway, so if the bug hits & the sc succeeds
> what's the big deal? It would have succeeded anyway. At least that's my
> understanding based on discussions with Loongson engineers a while ago.

Ah! So that wasn't spelled out as such. This basically says that: Yes,
it also screws with SC on UP, however the failure case is harmless.

(Also the comment with loongson_llsc_mb() seems incomplete in that it
doesn't mention the SC can also erroneously fail; typically that isn't a
problem because we'll just get an extra loop around and succeed
eventually.)

That said; I'm not entirely sure. The reason we use LL/SC even for
CPU-local variables is because of interrupts and the like. Would not a
false positive be a problem if it _should_ fail because of an interrupt?

> Having said that, if you have a strong preference for adding the barrier
> in UP systems anyway then I don't really object. It's not like anyone's
> likely to want to run a UP kernel on the affected systems, nevermind
> care about a miniscule performance impact.

It mostly all didn't make sense to me; and having a consistent recipie
for LL/SC loops is easier on the person occasionally looking at all
this (me, mostly :-).

(also, you should probably have a look at
include/asm-generic/bitops/atomic.h)

> One possibility your change could benefit would be if someone ran Linux
> on a subset of cores & some non-Linux code on other cores, in which case
> there could be something to cause the sc to fail. I've no idea if that's
> something these Loongson systems ever do though.

Or a bunch of UP guests ?

> > For the branch speculation case; if futex_atomic_cmpxchg_inatomic()
> > needs one at the bne branch target, then surely the normal
> > __cmpxch_asmg() implementation does too. We cannot rely on the
> 
> s/cmpxch_asmg/cmpxchg_asm/

Typing hard :-)