From: Waiman Long <Waiman.Long@hp.com> To: Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>, Peter Zijlstra <peterz@infradead.org> Cc: linux-arch@vger.kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, xen-devel@lists.xenproject.org, kvm@vger.kernel.org, Paolo Bonzini <paolo.bonzini@gmail.com>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, Boris Ostrovsky <boris.ostrovsky@oracle.com>, "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>, Rik van Riel <riel@redhat.com>, Linus Torvalds <torvalds@linux-foundation.org>, Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>, David Vrabel <david.vrabel@citrix.com>, Oleg Nesterov <oleg@redhat.com>, Daniel J Blueman <daniel@numascale.com>, Scott J Norton <scott.norton@hp.com>, Douglas Hatch <doug.hatch@hp.com>, Waiman Long <Waiman.Long@hp.com> Subject: [PATCH v16 10/14] pvqspinlock, x86: Enable PV qspinlock for KVM Date: Fri, 24 Apr 2015 14:56:39 -0400 [thread overview] Message-ID: <1429901803-29771-11-git-send-email-Waiman.Long@hp.com> (raw) In-Reply-To: <1429901803-29771-1-git-send-email-Waiman.Long@hp.com> This patch adds the necessary KVM specific code to allow KVM to support the CPU halting and kicking operations needed by the queue spinlock PV code. Signed-off-by: Waiman Long <Waiman.Long@hp.com> --- arch/x86/kernel/kvm.c | 43 +++++++++++++++++++++++++++++++++++++++++++ kernel/Kconfig.locks | 2 +- 2 files changed, 44 insertions(+), 1 deletions(-) diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index e354cc6..4bb42c0 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -584,6 +584,39 @@ static void kvm_kick_cpu(int cpu) kvm_hypercall2(KVM_HC_KICK_CPU, flags, apicid); } + +#ifdef CONFIG_QUEUE_SPINLOCK + +#include <asm/qspinlock.h> + +static void kvm_wait(u8 *ptr, u8 val) +{ + unsigned long flags; + + if (in_nmi()) + return; + + local_irq_save(flags); + + if (READ_ONCE(*ptr) != val) + goto out; + + /* + * halt until it's our turn and kicked. Note that we do safe halt + * for irq enabled case to avoid hang when lock info is overwritten + * in irq spinlock slowpath and no spurious interrupt occur to save us. + */ + if (arch_irqs_disabled_flags(flags)) + halt(); + else + safe_halt(); + +out: + local_irq_restore(flags); +} + +#else /* !CONFIG_QUEUE_SPINLOCK */ + enum kvm_contention_stat { TAKEN_SLOW, TAKEN_SLOW_PICKUP, @@ -817,6 +850,8 @@ static void kvm_unlock_kick(struct arch_spinlock *lock, __ticket_t ticket) } } +#endif /* !CONFIG_QUEUE_SPINLOCK */ + /* * Setup pv_lock_ops to exploit KVM_FEATURE_PV_UNHALT if present. */ @@ -828,8 +863,16 @@ void __init kvm_spinlock_init(void) if (!kvm_para_has_feature(KVM_FEATURE_PV_UNHALT)) return; +#ifdef CONFIG_QUEUE_SPINLOCK + __pv_init_lock_hash(); + pv_lock_ops.queue_spin_lock_slowpath = __pv_queue_spin_lock_slowpath; + pv_lock_ops.queue_spin_unlock = PV_CALLEE_SAVE(__pv_queue_spin_unlock); + pv_lock_ops.wait = kvm_wait; + pv_lock_ops.kick = kvm_kick_cpu; +#else /* !CONFIG_QUEUE_SPINLOCK */ pv_lock_ops.lock_spinning = PV_CALLEE_SAVE(kvm_lock_spinning); pv_lock_ops.unlock_kick = kvm_unlock_kick; +#endif } static __init int kvm_spinlock_init_jump(void) diff --git a/kernel/Kconfig.locks b/kernel/Kconfig.locks index c6a8f7c..537b13e 100644 --- a/kernel/Kconfig.locks +++ b/kernel/Kconfig.locks @@ -240,7 +240,7 @@ config ARCH_USE_QUEUE_SPINLOCK config QUEUE_SPINLOCK def_bool y if ARCH_USE_QUEUE_SPINLOCK - depends on SMP && !PARAVIRT_SPINLOCKS + depends on SMP && (!PARAVIRT_SPINLOCKS || !XEN) config ARCH_USE_QUEUE_RWLOCK bool -- 1.7.1
WARNING: multiple messages have this Message-ID (diff)
From: Waiman Long <Waiman.Long@hp.com> To: Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>, Peter Zijlstra <peterz@infradead.org> Cc: linux-arch@vger.kernel.org, Waiman Long <Waiman.Long@hp.com>, Rik van Riel <riel@redhat.com>, Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>, kvm@vger.kernel.org, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, Daniel J Blueman <daniel@numascale.com>, x86@kernel.org, Paolo Bonzini <paolo.bonzini@gmail.com>, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, Scott J Norton <scott.norton@hp.com>, David Vrabel <david.vrabel@citrix.com>, Oleg Nesterov <oleg@redhat.com>, xen-devel@lists.xenproject.org, Boris Ostrovsky <boris.ostrovsky@oracle.com>, "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>, Linus Torvalds <torvalds@linux-foundation.org>, Douglas Hatch <doug.hatch@hp.com> Subject: [PATCH v16 10/14] pvqspinlock, x86: Enable PV qspinlock for KVM Date: Fri, 24 Apr 2015 14:56:39 -0400 [thread overview] Message-ID: <1429901803-29771-11-git-send-email-Waiman.Long@hp.com> (raw) In-Reply-To: <1429901803-29771-1-git-send-email-Waiman.Long@hp.com> This patch adds the necessary KVM specific code to allow KVM to support the CPU halting and kicking operations needed by the queue spinlock PV code. Signed-off-by: Waiman Long <Waiman.Long@hp.com> --- arch/x86/kernel/kvm.c | 43 +++++++++++++++++++++++++++++++++++++++++++ kernel/Kconfig.locks | 2 +- 2 files changed, 44 insertions(+), 1 deletions(-) diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index e354cc6..4bb42c0 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -584,6 +584,39 @@ static void kvm_kick_cpu(int cpu) kvm_hypercall2(KVM_HC_KICK_CPU, flags, apicid); } + +#ifdef CONFIG_QUEUE_SPINLOCK + +#include <asm/qspinlock.h> + +static void kvm_wait(u8 *ptr, u8 val) +{ + unsigned long flags; + + if (in_nmi()) + return; + + local_irq_save(flags); + + if (READ_ONCE(*ptr) != val) + goto out; + + /* + * halt until it's our turn and kicked. Note that we do safe halt + * for irq enabled case to avoid hang when lock info is overwritten + * in irq spinlock slowpath and no spurious interrupt occur to save us. + */ + if (arch_irqs_disabled_flags(flags)) + halt(); + else + safe_halt(); + +out: + local_irq_restore(flags); +} + +#else /* !CONFIG_QUEUE_SPINLOCK */ + enum kvm_contention_stat { TAKEN_SLOW, TAKEN_SLOW_PICKUP, @@ -817,6 +850,8 @@ static void kvm_unlock_kick(struct arch_spinlock *lock, __ticket_t ticket) } } +#endif /* !CONFIG_QUEUE_SPINLOCK */ + /* * Setup pv_lock_ops to exploit KVM_FEATURE_PV_UNHALT if present. */ @@ -828,8 +863,16 @@ void __init kvm_spinlock_init(void) if (!kvm_para_has_feature(KVM_FEATURE_PV_UNHALT)) return; +#ifdef CONFIG_QUEUE_SPINLOCK + __pv_init_lock_hash(); + pv_lock_ops.queue_spin_lock_slowpath = __pv_queue_spin_lock_slowpath; + pv_lock_ops.queue_spin_unlock = PV_CALLEE_SAVE(__pv_queue_spin_unlock); + pv_lock_ops.wait = kvm_wait; + pv_lock_ops.kick = kvm_kick_cpu; +#else /* !CONFIG_QUEUE_SPINLOCK */ pv_lock_ops.lock_spinning = PV_CALLEE_SAVE(kvm_lock_spinning); pv_lock_ops.unlock_kick = kvm_unlock_kick; +#endif } static __init int kvm_spinlock_init_jump(void) diff --git a/kernel/Kconfig.locks b/kernel/Kconfig.locks index c6a8f7c..537b13e 100644 --- a/kernel/Kconfig.locks +++ b/kernel/Kconfig.locks @@ -240,7 +240,7 @@ config ARCH_USE_QUEUE_SPINLOCK config QUEUE_SPINLOCK def_bool y if ARCH_USE_QUEUE_SPINLOCK - depends on SMP && !PARAVIRT_SPINLOCKS + depends on SMP && (!PARAVIRT_SPINLOCKS || !XEN) config ARCH_USE_QUEUE_RWLOCK bool -- 1.7.1
next prev parent reply other threads:[~2015-04-24 19:00 UTC|newest] Thread overview: 90+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-04-24 18:56 [PATCH v16 00/14] qspinlock: a 4-byte queue spinlock with PV support Waiman Long 2015-04-24 18:56 ` [PATCH v16 01/14] qspinlock: A simple generic 4-byte queue spinlock Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-05-08 13:25 ` [tip:locking/core] locking/qspinlock: Introduce a simple generic 4-byte queued spinlock tip-bot for Waiman Long 2015-04-24 18:56 ` [PATCH v16 02/14] qspinlock, x86: Enable x86-64 to use queue spinlock Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-05-08 13:25 ` [tip:locking/core] locking/qspinlock, x86: Enable x86-64 to use queued spinlocks tip-bot for Waiman Long 2015-04-24 18:56 ` [PATCH v16 03/14] qspinlock: Add pending bit Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-05-08 13:26 ` [tip:locking/core] locking/qspinlock: " tip-bot for Peter Zijlstra (Intel) 2015-04-24 18:56 ` [PATCH v16 04/14] qspinlock: Extract out code snippets for the next patch Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-05-08 13:26 ` [tip:locking/core] locking/qspinlock: " tip-bot for Waiman Long 2015-04-24 18:56 ` [PATCH v16 05/14] qspinlock: Optimize for smaller NR_CPUS Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-05-08 13:26 ` [tip:locking/core] locking/qspinlock: " tip-bot for Peter Zijlstra (Intel) 2015-04-24 18:56 ` [PATCH v16 06/14] qspinlock: Use a simple write to grab the lock Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-05-08 13:27 ` [tip:locking/core] locking/qspinlock: " tip-bot for Waiman Long 2015-04-24 18:56 ` [PATCH v16 07/14] qspinlock: Revert to test-and-set on hypervisors Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-05-08 13:27 ` [tip:locking/core] locking/qspinlock: " tip-bot for Peter Zijlstra (Intel) 2015-04-24 18:56 ` [PATCH v16 07/14] qspinlock: " Waiman Long 2015-04-24 18:56 ` [PATCH v16 08/14] pvqspinlock: Implement simple paravirt support for the qspinlock Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-05-04 14:20 ` Peter Zijlstra 2015-05-04 14:20 ` Peter Zijlstra 2015-05-04 14:20 ` Peter Zijlstra 2015-05-04 17:15 ` Waiman Long 2015-05-04 17:15 ` Waiman Long 2015-05-04 17:15 ` Waiman Long 2015-05-08 13:27 ` [tip:locking/core] locking/pvqspinlock: " tip-bot for Waiman Long 2015-04-24 18:56 ` [PATCH v16 08/14] pvqspinlock: " Waiman Long 2015-04-24 18:56 ` [PATCH v16 09/14] pvqspinlock, x86: Implement the paravirt qspinlock call patching Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-05-08 13:27 ` [tip:locking/core] locking/pvqspinlock, " tip-bot for Peter Zijlstra (Intel) 2015-05-30 4:09 ` Sasha Levin 2015-05-31 18:29 ` Waiman Long 2015-04-24 18:56 ` [PATCH v16 09/14] pvqspinlock, " Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` [PATCH v16 10/14] pvqspinlock, x86: Enable PV qspinlock for KVM Waiman Long 2015-04-24 18:56 ` Waiman Long [this message] 2015-04-24 18:56 ` Waiman Long 2015-05-08 13:28 ` [tip:locking/core] locking/pvqspinlock, " tip-bot for Waiman Long 2015-04-24 18:56 ` [PATCH v16 11/14] pvqspinlock, x86: Enable PV qspinlock for Xen Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-05-08 13:28 ` [tip:locking/core] locking/pvqspinlock, " tip-bot for David Vrabel 2015-04-24 18:56 ` [PATCH v16 12/14] pvqspinlock: Only kick CPU at unlock time Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` [PATCH v16 13/14] pvqspinlock: Improve slowpath performance by avoiding cmpxchg Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-29 18:11 ` Peter Zijlstra 2015-04-29 18:27 ` Linus Torvalds 2015-04-29 18:27 ` Linus Torvalds 2015-04-29 18:27 ` Linus Torvalds 2015-04-30 18:56 ` Waiman Long 2015-04-30 18:56 ` Waiman Long 2015-04-30 18:56 ` Waiman Long 2015-04-30 18:56 ` Waiman Long 2015-04-30 18:56 ` Waiman Long 2015-04-30 18:49 ` Waiman Long 2015-04-30 18:49 ` Waiman Long 2015-04-30 18:49 ` Waiman Long 2015-05-04 14:05 ` Peter Zijlstra 2015-05-04 14:05 ` Peter Zijlstra 2015-05-04 14:05 ` Peter Zijlstra 2015-05-04 17:18 ` Waiman Long 2015-05-04 17:18 ` Waiman Long 2015-05-04 17:18 ` Waiman Long 2015-04-29 18:11 ` Peter Zijlstra 2015-04-29 18:11 ` Peter Zijlstra 2015-04-24 18:56 ` [PATCH v16 14/14] pvqspinlock: Collect slowpath lock statistics Waiman Long 2015-04-24 18:56 ` Waiman Long 2015-04-24 18:56 ` Waiman Long
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1429901803-29771-11-git-send-email-Waiman.Long@hp.com \ --to=waiman.long@hp.com \ --cc=boris.ostrovsky@oracle.com \ --cc=daniel@numascale.com \ --cc=david.vrabel@citrix.com \ --cc=doug.hatch@hp.com \ --cc=hpa@zytor.com \ --cc=konrad.wilk@oracle.com \ --cc=kvm@vger.kernel.org \ --cc=linux-arch@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mingo@redhat.com \ --cc=oleg@redhat.com \ --cc=paolo.bonzini@gmail.com \ --cc=paulmck@linux.vnet.ibm.com \ --cc=peterz@infradead.org \ --cc=raghavendra.kt@linux.vnet.ibm.com \ --cc=riel@redhat.com \ --cc=scott.norton@hp.com \ --cc=tglx@linutronix.de \ --cc=torvalds@linux-foundation.org \ --cc=virtualization@lists.linux-foundation.org \ --cc=x86@kernel.org \ --cc=xen-devel@lists.xenproject.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.