From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753827AbbBRRNN (ORCPT ); Wed, 18 Feb 2015 12:13:13 -0500 Received: from terminus.zytor.com ([198.137.202.10]:39291 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752985AbbBRRNK (ORCPT ); Wed, 18 Feb 2015 12:13:10 -0500 Date: Wed, 18 Feb 2015 09:12:31 -0800 From: tip-bot for Davidlohr Bueso Message-ID: Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, torvalds@linux-foundation.org, tim.c.chen@linux.intel.com, peterz@infradead.org, dave@stgolabs.net, tglx@linutronix.de, hpa@zytor.com, walken@google.com, dbueso@suse.de, paulmck@linux.vnet.ibm.com, jason.low2@hp.com Reply-To: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, mingo@kernel.org, walken@google.com, dave@stgolabs.net, tim.c.chen@linux.intel.com, peterz@infradead.org, tglx@linutronix.de, hpa@zytor.com, paulmck@linux.vnet.ibm.com, jason.low2@hp.com, dbueso@suse.de In-Reply-To: <1422609267-15102-6-git-send-email-dave@stgolabs.net> References: <1422609267-15102-6-git-send-email-dave@stgolabs.net> To: linux-tip-commits@vger.kernel.org Subject: [tip:locking/core] locking/rwsem: Check for active lock before bailing on spinning Git-Commit-ID: 1a99367023f6ac664365a37fa508b059e31d0e88 X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit-ID: 1a99367023f6ac664365a37fa508b059e31d0e88 Gitweb: http://git.kernel.org/tip/1a99367023f6ac664365a37fa508b059e31d0e88 Author: Davidlohr Bueso AuthorDate: Fri, 30 Jan 2015 01:14:27 -0800 Committer: Ingo Molnar CommitDate: Wed, 18 Feb 2015 16:57:18 +0100 locking/rwsem: Check for active lock before bailing on spinning 37e9562453b ("locking/rwsem: Allow conservative optimistic spinning when readers have lock") forced the default for optimistic spinning to be disabled if the lock owner was nil, which makes much sense for readers. However, while it is not our priority, we can make some optimizations for write-mostly workloads. We can bail the spinning step and still be conservative if there are any active tasks, otherwise there's really no reason not to spin, as the semaphore is most likely unlocked. This patch recovers most of a Unixbench 'execl' benchmark throughput by sleeping less and making better average system usage: before: CPU %user %nice %system %iowait %steal %idle all 0.60 0.00 8.02 0.00 0.00 91.38 after: CPU %user %nice %system %iowait %steal %idle all 1.22 0.00 70.18 0.00 0.00 28.60 Signed-off-by: Davidlohr Bueso Signed-off-by: Peter Zijlstra (Intel) Acked-by: Jason Low Cc: Linus Torvalds Cc: Michel Lespinasse Cc: Paul E. McKenney Cc: Tim Chen Link: http://lkml.kernel.org/r/1422609267-15102-6-git-send-email-dave@stgolabs.net Signed-off-by: Ingo Molnar --- kernel/locking/rwsem-xadd.c | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/kernel/locking/rwsem-xadd.c b/kernel/locking/rwsem-xadd.c index 1c0d11e..e4ad019 100644 --- a/kernel/locking/rwsem-xadd.c +++ b/kernel/locking/rwsem-xadd.c @@ -298,23 +298,30 @@ static inline bool rwsem_try_write_lock_unqueued(struct rw_semaphore *sem) static inline bool rwsem_can_spin_on_owner(struct rw_semaphore *sem) { struct task_struct *owner; - bool on_cpu = false; + bool ret = true; if (need_resched()) return false; rcu_read_lock(); owner = ACCESS_ONCE(sem->owner); - if (owner) - on_cpu = owner->on_cpu; - rcu_read_unlock(); + if (!owner) { + long count = ACCESS_ONCE(sem->count); + /* + * If sem->owner is not set, yet we have just recently entered the + * slowpath with the lock being active, then there is a possibility + * reader(s) may have the lock. To be safe, bail spinning in these + * situations. + */ + if (count & RWSEM_ACTIVE_MASK) + ret = false; + goto done; + } - /* - * If sem->owner is not set, yet we have just recently entered the - * slowpath, then there is a possibility reader(s) may have the lock. - * To be safe, avoid spinning in these situations. - */ - return on_cpu; + ret = owner->on_cpu; +done: + rcu_read_unlock(); + return ret; } static inline bool owner_running(struct rw_semaphore *sem,