From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932195AbcEWM0K (ORCPT ); Mon, 23 May 2016 08:26:10 -0400 Received: from bombadil.infradead.org ([198.137.202.9]:34485 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752964AbcEWM0I (ORCPT ); Mon, 23 May 2016 08:26:08 -0400 Date: Mon, 23 May 2016 14:25:54 +0200 From: Peter Zijlstra To: Linus Torvalds Cc: Boqun Feng , Davidlohr Bueso , Manfred Spraul , Waiman Long , Ingo Molnar , ggherdovich@suse.com, Mel Gorman , Linux Kernel Mailing List , Paul McKenney , Will Deacon Subject: Re: sem_lock() vs qspinlocks Message-ID: <20160523122554.GH15728@worktop.ger.corp.intel.com> References: <20160520053926.GC31084@linux-uzut.site> <20160520115819.GF3193@twins.programming.kicks-ass.net> <20160520140533.GA20726@insomnia> <20160520152149.GH3193@twins.programming.kicks-ass.net> <20160520160436.GQ3205@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.22.1 (2013-10-16) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, May 20, 2016 at 10:00:45AM -0700, Linus Torvalds wrote: > So I do wonder if we should make that smp_mb() be something the > *caller* has to do, and document rules for it. IOW, introduce a new > spinlock primitive called "spin_lock_synchronize()", and then spinlock > implementations that have this non-atomic behavior with an unordered > store would do something like > > static inline void queued_spin_lock_synchronize(struct qspinlock > *a, struct qspinlock *b) > { > smp_mb(); > } > > and then we'd document that *if* yuou need ordering guarantees between > > spin_lock(a); > .. spin_is_locked/spin_wait_lock(b) .. > > you have to have a > > spin_lock_synchronize(a, b); > > in between. So I think I favour the explicit barrier. But my 'problem' is that we now have _two_ different scenarios in which we need to order two different spinlocks. The first is the RCpc vs RCsc spinlock situation (currently only on PowerPC). Where the spin_unlock() spin_lock() 'barier' is not transitive. And the second is this 'new' situation, where the store is unordered and is not observable until a release, which is fundamentally so on PPC and ARM64 but also possible due to lock implementation choices like with our qspinlock, which makes it manifest even on x86. Now, ideally we'd be able to use one barrier construct for both; but given that, while there is overlap, they're not the same. And I'd be somewhat reluctant to issue superfluous smp_mb()s just because; it is an expensive instruction. Paul has smp_mb__after_unlock_lock() for the RCpc 'upgrade'. How about something like: smp_mb__after_lock() ? OTOH; even if we document this, it is something that is easy to forget or miss. It is not like Documentation/memory-barriers.txt is in want of more complexity.