From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751144AbaGPT1E (ORCPT ); Wed, 16 Jul 2014 15:27:04 -0400 Received: from terminus.zytor.com ([198.137.202.10]:44672 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030519AbaGPT0T (ORCPT ); Wed, 16 Jul 2014 15:26:19 -0400 Date: Wed, 16 Jul 2014 12:24:18 -0700 From: tip-bot for Jason Low Message-ID: Cc: mingo@kernel.org, konrad.wilk@oracle.com, torvalds@linux-foundation.org, peterz@infradead.org, jason.low2@hp.com, clm@fb.com, riel@redhat.com, rostedt@goodmis.org, akpm@linux-foundation.org, tglx@linutronix.de, david@fromorbit.com, davidlohr@hp.com, scott.norton@hp.com, linux-kernel@vger.kernel.org, hpa@zytor.com, waiman.long@hp.com, tim.c.chen@linux.intel.com, paulmck@linux.vnet.ibm.com, jbacik@fusionio.com, aswin@hp.com Reply-To: mingo@kernel.org, konrad.wilk@oracle.com, torvalds@linux-foundation.org, peterz@infradead.org, jason.low2@hp.com, clm@fb.com, riel@redhat.com, akpm@linux-foundation.org, rostedt@goodmis.org, david@fromorbit.com, tglx@linutronix.de, scott.norton@hp.com, davidlohr@hp.com, linux-kernel@vger.kernel.org, hpa@zytor.com, waiman.long@hp.com, tim.c.chen@linux.intel.com, paulmck@linux.vnet.ibm.com, jbacik@fusionio.com, aswin@hp.com In-Reply-To: <1405358872-3732-6-git-send-email-jason.low2@hp.com> References: <1405358872-3732-6-git-send-email-jason.low2@hp.com> To: linux-tip-commits@vger.kernel.org Subject: [tip:locking/urgent] locking/rwsem: Reduce the size of struct rw_semaphore Git-Commit-ID: ce069fc920e5734558b3d9cbef1ab06cf01ee793 X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit-ID: ce069fc920e5734558b3d9cbef1ab06cf01ee793 Gitweb: http://git.kernel.org/tip/ce069fc920e5734558b3d9cbef1ab06cf01ee793 Author: Jason Low AuthorDate: Mon, 14 Jul 2014 10:27:52 -0700 Committer: Ingo Molnar CommitDate: Wed, 16 Jul 2014 14:57:03 +0200 locking/rwsem: Reduce the size of struct rw_semaphore Recent optimistic spinning additions to rwsem provide significant performance benefits on many workloads on large machines. The cost of it was increasing the size of the rwsem structure by up to 128 bits. However, now that the previous patches in this series bring the overhead of struct optimistic_spin_queue to 32 bits, this patch reorders some fields in struct rw_semaphore such that we can reduce the overhead of the rwsem structure by 64 bits (on 64 bit systems). The extra overhead required for rwsem optimistic spinning would now be up to 8 additional bytes instead of up to 16 bytes. Additionally, the size of rwsem would now be more in line with mutexes. Signed-off-by: Jason Low Signed-off-by: Peter Zijlstra Cc: Scott Norton Cc: "Paul E. McKenney" Cc: Dave Chinner Cc: Waiman Long Cc: Davidlohr Bueso Cc: Rik van Riel Cc: Andrew Morton Cc: "H. Peter Anvin" Cc: Steven Rostedt Cc: Tim Chen Cc: Konrad Rzeszutek Wilk Cc: Aswin Chandramouleeswaran Cc: Linus Torvalds Cc: Chris Mason Cc: Josef Bacik Link: http://lkml.kernel.org/r/1405358872-3732-6-git-send-email-jason.low2@hp.com Signed-off-by: Ingo Molnar --- include/linux/rwsem.h | 25 +++++++++++-------------- 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/include/linux/rwsem.h b/include/linux/rwsem.h index 25cd9aa..716807f 100644 --- a/include/linux/rwsem.h +++ b/include/linux/rwsem.h @@ -24,15 +24,15 @@ struct rw_semaphore; /* All arch specific implementations share the same struct */ struct rw_semaphore { long count; - raw_spinlock_t wait_lock; struct list_head wait_list; + raw_spinlock_t wait_lock; #ifdef CONFIG_SMP + struct optimistic_spin_queue osq; /* spinner MCS lock */ /* * Write owner. Used as a speculative check to see * if the owner is running on the cpu. */ struct task_struct *owner; - struct optimistic_spin_queue osq; /* spinner MCS lock */ #endif #ifdef CONFIG_DEBUG_LOCK_ALLOC struct lockdep_map dep_map; @@ -64,21 +64,18 @@ static inline int rwsem_is_locked(struct rw_semaphore *sem) #endif #if defined(CONFIG_SMP) && !defined(CONFIG_RWSEM_GENERIC_SPINLOCK) -#define __RWSEM_INITIALIZER(name) \ - { RWSEM_UNLOCKED_VALUE, \ - __RAW_SPIN_LOCK_UNLOCKED(name.wait_lock), \ - LIST_HEAD_INIT((name).wait_list), \ - NULL, /* owner */ \ - OSQ_LOCK_UNLOCKED /* osq */ \ - __RWSEM_DEP_MAP_INIT(name) } +#define __RWSEM_OPT_INIT(lockname) , .osq = OSQ_LOCK_UNLOCKED, .owner = NULL #else -#define __RWSEM_INITIALIZER(name) \ - { RWSEM_UNLOCKED_VALUE, \ - __RAW_SPIN_LOCK_UNLOCKED(name.wait_lock), \ - LIST_HEAD_INIT((name).wait_list) \ - __RWSEM_DEP_MAP_INIT(name) } +#define __RWSEM_OPT_INIT(lockname) #endif +#define __RWSEM_INITIALIZER(name) \ + { .count = RWSEM_UNLOCKED_VALUE, \ + .wait_list = LIST_HEAD_INIT((name).wait_list), \ + .wait_lock = __RAW_SPIN_LOCK_UNLOCKED(name.wait_lock) \ + __RWSEM_OPT_INIT(name) \ + __RWSEM_DEP_MAP_INIT(name) } + #define DECLARE_RWSEM(name) \ struct rw_semaphore name = __RWSEM_INITIALIZER(name)