From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752030AbdHAShk (ORCPT ); Tue, 1 Aug 2017 14:37:40 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:36142 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751049AbdHAShi (ORCPT ); Tue, 1 Aug 2017 14:37:38 -0400 Date: Tue, 1 Aug 2017 11:37:33 -0700 From: "Paul E. McKenney" To: Will Deacon Cc: Peter Zijlstra , Boqun Feng , linux-kernel@vger.kernel.org, Ingo Molnar , Thomas Gleixner , Randy Dunlap , stern@rowland.harvard.edu Subject: Re: [RFC][PATCH v3]: documentation,atomic: Add new documents Reply-To: paulmck@linux.vnet.ibm.com References: <20170726115328.2sxiitivlnlq64dk@hirez.programming.kicks-ass.net> <20170726124750.vktrn5zi2gmpzfru@tardis> <20170731090535.rjgnoewqg7mhzr55@hirez.programming.kicks-ass.net> <20170731110403.ou3zqsp3uviqorkz@tardis> <20170731174345.GL3730@linux.vnet.ibm.com> <20170801090121.edo7mekhw3sann4h@hirez.programming.kicks-ass.net> <20170801101900.GB8702@arm.com> <20170801114744.evjjfviqhu5kgu7v@hirez.programming.kicks-ass.net> <20170801121713.GH8702@arm.com> <20170801161412.GW3730@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170801161412.GW3730@linux.vnet.ibm.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 17080118-0044-0000-0000-0000037686E4 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007466; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000215; SDB=6.00896116; UDB=6.00448253; IPR=6.00676303; BA=6.00005506; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00016483; XFM=3.00000015; UTC=2017-08-01 18:37:35 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17080118-0045-0000-0000-000007A4984A Message-Id: <20170801183733.GA22638@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-08-01_09:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1706020000 definitions=main-1708010301 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Aug 01, 2017 at 09:14:12AM -0700, Paul E. McKenney wrote: > On Tue, Aug 01, 2017 at 01:17:13PM +0100, Will Deacon wrote: > > On Tue, Aug 01, 2017 at 01:47:44PM +0200, Peter Zijlstra wrote: > > > On Tue, Aug 01, 2017 at 11:19:00AM +0100, Will Deacon wrote: > > > > On Tue, Aug 01, 2017 at 11:01:21AM +0200, Peter Zijlstra wrote: > > > > > On Mon, Jul 31, 2017 at 10:43:45AM -0700, Paul E. McKenney wrote: > > > > > > > > > > > Why wouldn't the following have ACQUIRE semantics? > > > > > > > > > > > > atomic_inc(&var); > > > > > > smp_mb__after_atomic(); > > > > > > > > > > > > Is the issue that there is no actual value returned or some such? > > > > > > > > > > Yes, so that the inc is a load-store, and thus there is a load, we loose > > > > > the value. > > > > > > > > > > But I see your point I think. Irrespective of still having the value, > > > > > the ordering is preserved and nothing should pass across that. > > > > > > > > > > > So if I have something like this, the assertion really can trigger? > > > > > > > > > > > > WRITE_ONCE(x, 1); atomic_inc(&y); > > > > > > r0 = xchg_release(&y, 5); smp_mb__after_atomic(); > > > > > > r1 = READ_ONCE(x); > > > > > > > > > > > > > > > > > > WARN_ON(r0 == 0 && r1 == 0); > > > > > > > > > > > > I must confess that I am not seeing why we would want to allow this > > > > > > outcome. > > > > > > > > > > No you are indeed quite right. I just wasn't creative enough. Thanks for > > > > > the inspiration. > > > > > > > > Just to close this out, we agree that an smp_rmb() instead of > > > > smp_mb__after_atomic() would *not* forbid this outcome, right? > > > > > > So that really hurts my brain. Per the normal rules that smp_rmb() would > > > order the read of @x against the last ll of @y and per ll/sc ordering > > > you then still don't get to make the WARN happen. > > > > > > On IRC you explained that your 8.1 LSE instructions are not in fact > > > ordered by a smp_rmb, only by smp_wmb, which is 'surprising' since you > > > really need to load the old value to compute the new value. > > > > To be clear, it's only the ST* variants of the LSE instructions that are > > treated as a write for the purposes of memory ordering, so these are the > > non-*_return variants. It's not unlikely that other architectures will > > exhibit the same behaviour (e.g. Power, RISC-V), because the CPU can > > treat non-return atomics as "fire-and-forget" and have them handled > > elsewhere in the memory subsystem, causing them to be treated similarly > > to posted writes. > > > > For the code snippet above, the second thread has no idea about the value > > of y and so smp_rmb() is the wrong thing to be using imo. It really cares > > about ordering the store to y before the read of x, so needs a full mb (i.e. > > the test is more like 'R' than 'MP'). > > > > Also, wouldn't this problem also arise if your atomics were built using a > > spinlock where unlock had release semantics? And responding more directly to the bit about spinlocks after a side discussion with Alan Stern, both the xchg_release() and the atomic_inc() are operating on the same variable, namely "y". Even by the very weak "roach motel" locking semantics, this outcome would be forbidden. But if you have hardware that allows this, let's all discuss and get it hashed out... Thanx, Paul Thanx, Paul