From: Peter Zijlstra <a.p.zijlstra@chello.nl> To: Waiman.Long@hp.com, tglx@linutronix.de, mingo@kernel.org Cc: linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, xen-devel@lists.xenproject.org, kvm@vger.kernel.org, paolo.bonzini@gmail.com, konrad.wilk@oracle.com, boris.ostrovsky@oracle.com, paulmck@linux.vnet.ibm.com, riel@redhat.com, torvalds@linux-foundation.org, raghavendra.kt@linux.vnet.ibm.com, david.vrabel@citrix.com, oleg@redhat.com, gleb@redhat.com, scott.norton@hp.com, chegu_vinod@hp.com, Peter Zijlstra <peterz@infradead.org> Subject: [PATCH 08/11] qspinlock: Revert to test-and-set on hypervisors Date: Sun, 15 Jun 2014 14:47:05 +0200 [thread overview] Message-ID: <20140615130153.940699466@chello.nl> (raw) In-Reply-To: 20140615124657.264658593@chello.nl [-- Attachment #1: peterz-qspinlock-test-and-set.patch --] [-- Type: text/plain, Size: 1777 bytes --] When we detect a hypervisor (!paravirt, see later patches), revert to a simple test-and-set lock to avoid the horrors of queue preemption. Signed-off-by: Peter Zijlstra <peterz@infradead.org> --- arch/x86/include/asm/qspinlock.h | 14 ++++++++++++++ include/asm-generic/qspinlock.h | 7 +++++++ kernel/locking/qspinlock.c | 3 +++ 3 files changed, 24 insertions(+) --- a/arch/x86/include/asm/qspinlock.h +++ b/arch/x86/include/asm/qspinlock.h @@ -1,6 +1,7 @@ #ifndef _ASM_X86_QSPINLOCK_H #define _ASM_X86_QSPINLOCK_H +#include <asm/cpufeature.h> #include <asm-generic/qspinlock_types.h> #if !defined(CONFIG_X86_OOSTORE) && !defined(CONFIG_X86_PPRO_FENCE) @@ -20,6 +21,19 @@ static inline void queue_spin_unlock(str #endif /* !CONFIG_X86_OOSTORE && !CONFIG_X86_PPRO_FENCE */ +#define virt_queue_spin_lock virt_queue_spin_lock + +static inline bool virt_queue_spin_lock(struct qspinlock *lock) +{ + if (!static_cpu_has(X86_FEATURE_HYPERVISOR)) + return false; + + while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0) + cpu_relax(); + + return true; +} + #include <asm-generic/qspinlock.h> #endif /* _ASM_X86_QSPINLOCK_H */ --- a/include/asm-generic/qspinlock.h +++ b/include/asm-generic/qspinlock.h @@ -98,6 +98,13 @@ static __always_inline void queue_spin_u } #endif +#ifndef virt_queue_spin_lock +static __always_inline bool virt_queue_spin_lock(struct qspinlock *lock) +{ + return false; +} +#endif + /* * Initializier */ --- a/kernel/locking/qspinlock.c +++ b/kernel/locking/qspinlock.c @@ -247,6 +247,9 @@ void queue_spin_lock_slowpath(struct qsp BUILD_BUG_ON(CONFIG_NR_CPUS >= (1U << _Q_TAIL_CPU_BITS)); + if (virt_queue_spin_lock(lock)) + return; + /* * wait for in-progress pending->locked hand-overs *
WARNING: multiple messages have this Message-ID (diff)
From: Peter Zijlstra <a.p.zijlstra@chello.nl> To: Waiman.Long@hp.com, tglx@linutronix.de, mingo@kernel.org Cc: linux-arch@vger.kernel.org, riel@redhat.com, gleb@redhat.com, kvm@vger.kernel.org, konrad.wilk@oracle.com, Peter Zijlstra <peterz@infradead.org>, scott.norton@hp.com, raghavendra.kt@linux.vnet.ibm.com, paolo.bonzini@gmail.com, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, chegu_vinod@hp.com, david.vrabel@citrix.com, oleg@redhat.com, xen-devel@lists.xenproject.org, boris.ostrovsky@oracle.com, paulmck@linux.vnet.ibm.com, torvalds@linux-foundation.org Subject: [PATCH 08/11] qspinlock: Revert to test-and-set on hypervisors Date: Sun, 15 Jun 2014 14:47:05 +0200 [thread overview] Message-ID: <20140615130153.940699466@chello.nl> (raw) In-Reply-To: 20140615124657.264658593@chello.nl [-- Attachment #1: peterz-qspinlock-test-and-set.patch --] [-- Type: text/plain, Size: 1775 bytes --] When we detect a hypervisor (!paravirt, see later patches), revert to a simple test-and-set lock to avoid the horrors of queue preemption. Signed-off-by: Peter Zijlstra <peterz@infradead.org> --- arch/x86/include/asm/qspinlock.h | 14 ++++++++++++++ include/asm-generic/qspinlock.h | 7 +++++++ kernel/locking/qspinlock.c | 3 +++ 3 files changed, 24 insertions(+) --- a/arch/x86/include/asm/qspinlock.h +++ b/arch/x86/include/asm/qspinlock.h @@ -1,6 +1,7 @@ #ifndef _ASM_X86_QSPINLOCK_H #define _ASM_X86_QSPINLOCK_H +#include <asm/cpufeature.h> #include <asm-generic/qspinlock_types.h> #if !defined(CONFIG_X86_OOSTORE) && !defined(CONFIG_X86_PPRO_FENCE) @@ -20,6 +21,19 @@ static inline void queue_spin_unlock(str #endif /* !CONFIG_X86_OOSTORE && !CONFIG_X86_PPRO_FENCE */ +#define virt_queue_spin_lock virt_queue_spin_lock + +static inline bool virt_queue_spin_lock(struct qspinlock *lock) +{ + if (!static_cpu_has(X86_FEATURE_HYPERVISOR)) + return false; + + while (atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL) != 0) + cpu_relax(); + + return true; +} + #include <asm-generic/qspinlock.h> #endif /* _ASM_X86_QSPINLOCK_H */ --- a/include/asm-generic/qspinlock.h +++ b/include/asm-generic/qspinlock.h @@ -98,6 +98,13 @@ static __always_inline void queue_spin_u } #endif +#ifndef virt_queue_spin_lock +static __always_inline bool virt_queue_spin_lock(struct qspinlock *lock) +{ + return false; +} +#endif + /* * Initializier */ --- a/kernel/locking/qspinlock.c +++ b/kernel/locking/qspinlock.c @@ -247,6 +247,9 @@ void queue_spin_lock_slowpath(struct qsp BUILD_BUG_ON(CONFIG_NR_CPUS >= (1U << _Q_TAIL_CPU_BITS)); + if (virt_queue_spin_lock(lock)) + return; + /* * wait for in-progress pending->locked hand-overs *
next prev parent reply other threads:[~2014-06-15 13:16 UTC|newest] Thread overview: 191+ messages / expand[flat|nested] mbox.gz Atom feed top 2014-06-15 12:46 [PATCH 00/11] qspinlock with paravirt support Peter Zijlstra 2014-06-15 12:46 ` Peter Zijlstra 2014-06-15 12:46 ` [PATCH 01/11] qspinlock: A simple generic 4-byte queue spinlock Peter Zijlstra 2014-06-15 12:46 ` Peter Zijlstra 2014-06-15 12:46 ` Peter Zijlstra 2014-06-16 20:49 ` Konrad Rzeszutek Wilk 2014-06-16 20:49 ` Konrad Rzeszutek Wilk 2014-06-16 20:49 ` Konrad Rzeszutek Wilk 2014-06-17 20:03 ` Konrad Rzeszutek Wilk 2014-06-17 20:03 ` Konrad Rzeszutek Wilk 2014-06-17 20:03 ` Konrad Rzeszutek Wilk 2014-06-17 20:03 ` Konrad Rzeszutek Wilk 2014-06-23 16:12 ` Peter Zijlstra 2014-06-23 16:12 ` Peter Zijlstra 2014-06-23 16:12 ` Peter Zijlstra 2014-06-23 16:20 ` Konrad Rzeszutek Wilk 2014-06-23 16:20 ` Konrad Rzeszutek Wilk 2014-06-23 16:20 ` Konrad Rzeszutek Wilk 2014-06-17 20:03 ` Konrad Rzeszutek Wilk 2014-06-23 15:56 ` Peter Zijlstra 2014-06-23 15:56 ` Peter Zijlstra 2014-06-23 15:56 ` Peter Zijlstra 2014-06-23 16:16 ` Konrad Rzeszutek Wilk 2014-06-23 16:16 ` Konrad Rzeszutek Wilk 2014-06-23 16:16 ` Konrad Rzeszutek Wilk 2014-06-17 20:05 ` Konrad Rzeszutek Wilk 2014-06-17 20:05 ` Konrad Rzeszutek Wilk 2014-06-17 20:05 ` Konrad Rzeszutek Wilk 2014-06-17 20:05 ` Konrad Rzeszutek Wilk 2014-06-23 16:26 ` Peter Zijlstra 2014-06-23 16:26 ` Peter Zijlstra 2014-06-23 16:45 ` Konrad Rzeszutek Wilk 2014-06-23 16:45 ` Konrad Rzeszutek Wilk 2014-06-23 16:45 ` Konrad Rzeszutek Wilk 2014-06-23 16:26 ` Peter Zijlstra 2014-06-17 20:05 ` Konrad Rzeszutek Wilk 2014-06-15 12:46 ` [PATCH 02/11] qspinlock, x86: Enable x86-64 to use " Peter Zijlstra 2014-06-15 12:46 ` Peter Zijlstra 2014-06-15 12:46 ` Peter Zijlstra 2014-06-15 12:47 ` [PATCH 03/11] qspinlock: Add pending bit Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-17 20:36 ` Konrad Rzeszutek Wilk 2014-06-17 20:36 ` Konrad Rzeszutek Wilk 2014-06-17 20:36 ` Konrad Rzeszutek Wilk 2014-06-17 20:51 ` Waiman Long 2014-06-17 20:51 ` Waiman Long 2014-06-17 20:51 ` Waiman Long 2014-06-17 21:07 ` Konrad Rzeszutek Wilk 2014-06-17 21:07 ` Konrad Rzeszutek Wilk 2014-06-17 21:07 ` Konrad Rzeszutek Wilk 2014-06-17 21:10 ` Konrad Rzeszutek Wilk 2014-06-17 21:10 ` Konrad Rzeszutek Wilk 2014-06-17 22:25 ` Waiman Long 2014-06-17 22:25 ` Waiman Long 2014-06-17 22:25 ` Waiman Long 2014-06-17 21:10 ` Konrad Rzeszutek Wilk 2014-06-24 8:24 ` Peter Zijlstra 2014-06-24 8:24 ` Peter Zijlstra 2014-06-24 8:24 ` Peter Zijlstra 2014-06-18 11:29 ` Paolo Bonzini 2014-06-18 11:29 ` Paolo Bonzini 2014-06-18 11:29 ` Paolo Bonzini 2014-06-18 13:36 ` Konrad Rzeszutek Wilk 2014-06-18 13:36 ` Konrad Rzeszutek Wilk 2014-06-18 13:36 ` Konrad Rzeszutek Wilk 2014-06-23 16:35 ` Peter Zijlstra 2014-06-23 16:35 ` Peter Zijlstra 2014-06-23 16:35 ` Peter Zijlstra 2014-06-15 12:47 ` [PATCH 04/11] qspinlock: Extract out the exchange of tail code word Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-17 20:55 ` Konrad Rzeszutek Wilk 2014-06-17 20:55 ` Konrad Rzeszutek Wilk 2014-06-18 11:37 ` Paolo Bonzini 2014-06-18 11:37 ` Paolo Bonzini 2014-06-18 11:37 ` Paolo Bonzini 2014-06-18 13:50 ` Konrad Rzeszutek Wilk 2014-06-18 13:50 ` Konrad Rzeszutek Wilk 2014-06-18 13:50 ` Konrad Rzeszutek Wilk 2014-06-18 15:46 ` Waiman Long 2014-06-18 15:46 ` Waiman Long 2014-06-18 15:49 ` Paolo Bonzini 2014-06-18 15:49 ` Paolo Bonzini 2014-06-18 15:49 ` Paolo Bonzini 2014-06-18 16:02 ` Konrad Rzeszutek Wilk 2014-06-18 16:02 ` Konrad Rzeszutek Wilk 2014-06-18 16:02 ` Konrad Rzeszutek Wilk 2014-06-18 15:46 ` Waiman Long 2014-06-24 10:47 ` Peter Zijlstra 2014-06-24 10:47 ` Peter Zijlstra 2014-06-24 10:47 ` Peter Zijlstra 2014-06-17 20:55 ` Konrad Rzeszutek Wilk 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` [PATCH 05/11] qspinlock: Optimize for smaller NR_CPUS Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-18 11:39 ` Paolo Bonzini 2014-06-18 11:39 ` Paolo Bonzini 2014-06-18 11:39 ` Paolo Bonzini 2014-07-07 14:35 ` Peter Zijlstra 2014-07-07 14:35 ` Peter Zijlstra 2014-07-07 14:35 ` Peter Zijlstra 2014-07-07 15:08 ` Paolo Bonzini 2014-07-07 15:08 ` Paolo Bonzini 2014-07-07 15:08 ` Paolo Bonzini 2014-07-07 15:35 ` Peter Zijlstra 2014-07-07 15:35 ` Peter Zijlstra 2014-07-07 16:10 ` Paolo Bonzini 2014-07-07 16:10 ` Paolo Bonzini 2014-07-07 16:10 ` Paolo Bonzini 2014-07-07 15:35 ` Peter Zijlstra 2014-06-18 15:57 ` Konrad Rzeszutek Wilk 2014-06-18 15:57 ` Konrad Rzeszutek Wilk 2014-07-07 14:33 ` Peter Zijlstra 2014-07-07 14:33 ` Peter Zijlstra 2014-07-07 14:33 ` Peter Zijlstra 2014-06-18 15:57 ` Konrad Rzeszutek Wilk 2014-06-15 12:47 ` [PATCH 06/11] qspinlock: Optimize pending bit Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-18 11:42 ` Paolo Bonzini 2014-06-18 11:42 ` Paolo Bonzini 2014-06-18 11:42 ` Paolo Bonzini 2014-06-15 12:47 ` [PATCH 07/11] qspinlock: Use a simple write to grab the lock, if applicable Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-18 16:36 ` Konrad Rzeszutek Wilk 2014-06-18 16:36 ` Konrad Rzeszutek Wilk 2014-06-18 16:36 ` Konrad Rzeszutek Wilk 2014-07-07 14:51 ` Peter Zijlstra 2014-07-07 14:51 ` Peter Zijlstra 2014-07-07 14:51 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` [PATCH 08/11] qspinlock: Revert to test-and-set on hypervisors Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra [this message] 2014-06-15 12:47 ` Peter Zijlstra 2014-06-16 21:57 ` Waiman Long 2014-06-16 21:57 ` Waiman Long 2014-06-16 21:57 ` Waiman Long 2014-06-18 16:40 ` Konrad Rzeszutek Wilk 2014-06-18 16:40 ` Konrad Rzeszutek Wilk 2014-06-18 16:40 ` Konrad Rzeszutek Wilk 2014-06-15 12:47 ` [PATCH 09/11] pvqspinlock, x86: Rename paravirt_ticketlocks_enabled Peter Zijlstra 2014-06-18 16:43 ` Konrad Rzeszutek Wilk 2014-06-18 16:43 ` Konrad Rzeszutek Wilk 2014-06-18 16:43 ` Konrad Rzeszutek Wilk 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` [PATCH 10/11] qspinlock: Paravirt support Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-16 22:08 ` Waiman Long 2014-06-18 12:03 ` Paolo Bonzini 2014-06-18 12:03 ` Paolo Bonzini 2014-06-18 15:26 ` Waiman Long 2014-06-18 15:26 ` Waiman Long 2014-06-18 15:26 ` Waiman Long 2014-07-07 15:20 ` Peter Zijlstra 2014-07-07 15:20 ` Peter Zijlstra 2014-07-07 15:20 ` Peter Zijlstra 2014-06-18 12:03 ` Paolo Bonzini 2014-07-07 15:20 ` Peter Zijlstra 2014-07-07 15:20 ` Peter Zijlstra 2014-07-07 15:20 ` Peter Zijlstra 2014-06-16 22:08 ` Waiman Long 2014-06-17 0:53 ` Waiman Long 2014-06-17 0:53 ` Waiman Long 2014-06-17 0:53 ` Waiman Long 2014-06-18 12:04 ` Paolo Bonzini 2014-06-18 12:04 ` Paolo Bonzini 2014-06-18 12:04 ` Paolo Bonzini 2014-06-20 13:46 ` Konrad Rzeszutek Wilk 2014-06-20 13:46 ` Konrad Rzeszutek Wilk 2014-06-20 13:46 ` Konrad Rzeszutek Wilk 2014-07-07 15:27 ` Peter Zijlstra 2014-07-15 14:23 ` Konrad Rzeszutek Wilk 2014-07-15 14:23 ` Konrad Rzeszutek Wilk 2014-07-15 14:23 ` Konrad Rzeszutek Wilk 2014-07-07 15:27 ` Peter Zijlstra 2014-07-07 15:27 ` Peter Zijlstra 2014-06-15 12:47 ` [PATCH 11/11] qspinlock, kvm: Add paravirt support Peter Zijlstra 2014-06-22 16:36 ` Raghavendra K T 2014-06-22 16:36 ` Raghavendra K T 2014-07-07 15:23 ` Peter Zijlstra 2014-07-07 15:23 ` Peter Zijlstra 2014-07-07 15:23 ` Peter Zijlstra 2014-06-22 16:36 ` Raghavendra K T 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-16 20:52 ` [PATCH 00/11] qspinlock with " Konrad Rzeszutek Wilk 2014-06-16 20:52 ` Konrad Rzeszutek Wilk 2014-06-16 20:52 ` Konrad Rzeszutek Wilk
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20140615130153.940699466@chello.nl \ --to=a.p.zijlstra@chello.nl \ --cc=Waiman.Long@hp.com \ --cc=boris.ostrovsky@oracle.com \ --cc=chegu_vinod@hp.com \ --cc=david.vrabel@citrix.com \ --cc=gleb@redhat.com \ --cc=konrad.wilk@oracle.com \ --cc=kvm@vger.kernel.org \ --cc=linux-arch@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mingo@kernel.org \ --cc=oleg@redhat.com \ --cc=paolo.bonzini@gmail.com \ --cc=paulmck@linux.vnet.ibm.com \ --cc=peterz@infradead.org \ --cc=raghavendra.kt@linux.vnet.ibm.com \ --cc=riel@redhat.com \ --cc=scott.norton@hp.com \ --cc=tglx@linutronix.de \ --cc=torvalds@linux-foundation.org \ --cc=virtualization@lists.linux-foundation.org \ --cc=xen-devel@lists.xenproject.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.