From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751558AbbIRIvZ (ORCPT ); Fri, 18 Sep 2015 04:51:25 -0400 Received: from terminus.zytor.com ([198.137.202.10]:44811 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752160AbbIRIvT (ORCPT ); Fri, 18 Sep 2015 04:51:19 -0400 Date: Fri, 18 Sep 2015 01:50:35 -0700 From: tip-bot for Davidlohr Bueso Message-ID: Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, Waiman.Long@hpe.com, hpa@zytor.com, dave@stgolabs.net, mingo@kernel.org, akpm@linux-foundation.org, dbueso@suse.de, paulmck@linux.vnet.ibm.com, tglx@linutronix.de, peterz@infradead.org Reply-To: dbueso@suse.de, akpm@linux-foundation.org, paulmck@linux.vnet.ibm.com, peterz@infradead.org, tglx@linutronix.de, mingo@kernel.org, dave@stgolabs.net, hpa@zytor.com, Waiman.Long@hpe.com, torvalds@linux-foundation.org, linux-kernel@vger.kernel.org In-Reply-To: <1442216244-4409-3-git-send-email-dave@stgolabs.net> References: <1442216244-4409-3-git-send-email-dave@stgolabs.net> To: linux-tip-commits@vger.kernel.org Subject: [tip:locking/core] locking/osq: Relax atomic semantics Git-Commit-ID: c55a6ffa6285e29f874ed403979472631ec70bff X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit-ID: c55a6ffa6285e29f874ed403979472631ec70bff Gitweb: http://git.kernel.org/tip/c55a6ffa6285e29f874ed403979472631ec70bff Author: Davidlohr Bueso AuthorDate: Mon, 14 Sep 2015 00:37:24 -0700 Committer: Ingo Molnar CommitDate: Fri, 18 Sep 2015 09:27:29 +0200 locking/osq: Relax atomic semantics ... by using acquire/release for ops around the lock->tail. As such, weakly ordered archs can benefit from more relaxed use of barriers when issuing atomics. Signed-off-by: Davidlohr Bueso Signed-off-by: Peter Zijlstra (Intel) Cc: Andrew Morton Cc: Linus Torvalds Cc: Paul E. McKenney Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Waiman Long Link: http://lkml.kernel.org/r/1442216244-4409-3-git-send-email-dave@stgolabs.net Signed-off-by: Ingo Molnar --- kernel/locking/osq_lock.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/kernel/locking/osq_lock.c b/kernel/locking/osq_lock.c index dc85ee2..d092a0c 100644 --- a/kernel/locking/osq_lock.c +++ b/kernel/locking/osq_lock.c @@ -50,7 +50,7 @@ osq_wait_next(struct optimistic_spin_queue *lock, for (;;) { if (atomic_read(&lock->tail) == curr && - atomic_cmpxchg(&lock->tail, curr, old) == curr) { + atomic_cmpxchg_acquire(&lock->tail, curr, old) == curr) { /* * We were the last queued, we moved @lock back. @prev * will now observe @lock and will complete its @@ -92,7 +92,11 @@ bool osq_lock(struct optimistic_spin_queue *lock) node->next = NULL; node->cpu = curr; - old = atomic_xchg(&lock->tail, curr); + /* + * ACQUIRE semantics, pairs with corresponding RELEASE + * in unlock() uncontended, or fastpath. + */ + old = atomic_xchg_acquire(&lock->tail, curr); if (old == OSQ_UNLOCKED_VAL) return true; @@ -184,7 +188,8 @@ void osq_unlock(struct optimistic_spin_queue *lock) /* * Fast path for the uncontended case. */ - if (likely(atomic_cmpxchg(&lock->tail, curr, OSQ_UNLOCKED_VAL) == curr)) + if (likely(atomic_cmpxchg_release(&lock->tail, curr, + OSQ_UNLOCKED_VAL) == curr)) return; /*