All of lore.kernel.org
 help / color / mirror / Atom feed
From: Waiman Long <waiman.long@hp.com>
To: David Vrabel <david.vrabel@citrix.com>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>,
	Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>,
	kvm@vger.kernel.org, Peter Zijlstra <peterz@infradead.org>,
	virtualization@lists.linux-foundation.org,
	Andi Kleen <andi@firstfloor.org>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Michel Lespinasse <walken@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	linux-arch@vger.kernel.org, Gleb Natapov <gleb@redhat.com>,
	x86@kernel.org, Ingo Molnar <mingo@redhat.com>,
	xen-devel@lists.xenproject.org,
	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
	Rik van Riel <riel@redhat.com>, Arnd Bergmann <arnd@arndb.de>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Scott J Norton <scott.norton@hp.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Chris Wright <chrisw@sous-sol.org>,
	Oleg Nesterov <oleg@redhat.com>,
	Alok Kataria <akataria@vmware.com>,
	Aswin Chandramouleeswaran <aswin@hp.com>, Chegu Vinod <chegu>
Subject: Re: [PATCH v6 05/11] pvqspinlock, x86: Allow unfair spinlock in a PV guest
Date: Thu, 13 Mar 2014 15:03:01 -0400	[thread overview]
Message-ID: <532200E5.2050107@hp.com> (raw)
In-Reply-To: <53218E7A.8090707@citrix.com>

On 03/13/2014 06:54 AM, David Vrabel wrote:
> On 12/03/14 18:54, Waiman Long wrote:
>> Locking is always an issue in a virtualized environment as the virtual
>> CPU that is waiting on a lock may get scheduled out and hence block
>> any progress in lock acquisition even when the lock has been freed.
>>
>> One solution to this problem is to allow unfair lock in a
>> para-virtualized environment. In this case, a new lock acquirer can
>> come and steal the lock if the next-in-line CPU to get the lock is
>> scheduled out. Unfair lock in a native environment is generally not a
>> good idea as there is a possibility of lock starvation for a heavily
>> contended lock.
> I do not think this is a good idea -- the problems with unfair locks are
> worse in a virtualized guest.  If a waiting VCPU deschedules and has to
> be kicked to grab a lock then it is very likely to lose a race with
> another running VCPU trying to take a lock (since it takes time for the
> VCPU to be rescheduled).

I have seen figure that it will take about 1000 cycles to kick in a CPU. 
As long as the critical section isn't that long, there is enough time 
for a lock stealer to come in, grab the lock, do whatever it needs to do 
and leave without introducing too much latency to the kicked-in CPU.

Anyway there are people who ask for unfair lock. In fact, RHEL6 ship a 
virtual guest with unfair lock. So I provide an option for those people 
who want unfair lock to enable it in their virtual guest. For those who 
don't want it, they can always turn them off when building the kernel.

>> With the unfair locking activated on bare metal 4-socket Westmere-EX
>> box, the execution times (in ms) of a spinlock micro-benchmark were
>> as follows:
>>
>>    # of    Ticket       Fair	    Unfair
>>    tasks    lock     queue lock    queue lock
>>    ------  -------   ----------    ----------
>>      1       135        135	     137
>>      2      1045       1120	     747
>>      3      1827       2345     	    1084
>>      4      2689       2934	    1438
>>      5      3736       3658	    1722
>>      6      4942       4434	    2092
>>      7      6304       5176          2245
>>      8      7736       5955          2388
> Are these figures with or without the later PV support patches?

This is without the PV patch.

Regards,
Longman

WARNING: multiple messages have this Message-ID (diff)
From: Waiman Long <waiman.long@hp.com>
To: David Vrabel <david.vrabel@citrix.com>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>,
	Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com>,
	kvm@vger.kernel.org, Peter Zijlstra <peterz@infradead.org>,
	virtualization@lists.linux-foundation.org,
	Andi Kleen <andi@firstfloor.org>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Michel Lespinasse <walken@google.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	linux-arch@vger.kernel.org, Gleb Natapov <gleb@redhat.com>,
	x86@kernel.org, Ingo Molnar <mingo@redhat.com>,
	xen-devel@lists.xenproject.org,
	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
	Rik van Riel <riel@redhat.com>, Arnd Bergmann <arnd@arndb.de>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Scott J Norton <scott.norton@hp.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Chris Wright <chrisw@sous-sol.org>,
	Oleg Nesterov <oleg@redhat.com>,
	Alok Kataria <akataria@vmware.com>,
	Aswin Chandramouleeswaran <aswin@hp.com>,
	Chegu Vinod <chegu
Subject: Re: [PATCH v6 05/11] pvqspinlock, x86: Allow unfair spinlock in a PV guest
Date: Thu, 13 Mar 2014 15:03:01 -0400	[thread overview]
Message-ID: <532200E5.2050107@hp.com> (raw)
In-Reply-To: <53218E7A.8090707@citrix.com>

On 03/13/2014 06:54 AM, David Vrabel wrote:
> On 12/03/14 18:54, Waiman Long wrote:
>> Locking is always an issue in a virtualized environment as the virtual
>> CPU that is waiting on a lock may get scheduled out and hence block
>> any progress in lock acquisition even when the lock has been freed.
>>
>> One solution to this problem is to allow unfair lock in a
>> para-virtualized environment. In this case, a new lock acquirer can
>> come and steal the lock if the next-in-line CPU to get the lock is
>> scheduled out. Unfair lock in a native environment is generally not a
>> good idea as there is a possibility of lock starvation for a heavily
>> contended lock.
> I do not think this is a good idea -- the problems with unfair locks are
> worse in a virtualized guest.  If a waiting VCPU deschedules and has to
> be kicked to grab a lock then it is very likely to lose a race with
> another running VCPU trying to take a lock (since it takes time for the
> VCPU to be rescheduled).

I have seen figure that it will take about 1000 cycles to kick in a CPU. 
As long as the critical section isn't that long, there is enough time 
for a lock stealer to come in, grab the lock, do whatever it needs to do 
and leave without introducing too much latency to the kicked-in CPU.

Anyway there are people who ask for unfair lock. In fact, RHEL6 ship a 
virtual guest with unfair lock. So I provide an option for those people 
who want unfair lock to enable it in their virtual guest. For those who 
don't want it, they can always turn them off when building the kernel.

>> With the unfair locking activated on bare metal 4-socket Westmere-EX
>> box, the execution times (in ms) of a spinlock micro-benchmark were
>> as follows:
>>
>>    # of    Ticket       Fair	    Unfair
>>    tasks    lock     queue lock    queue lock
>>    ------  -------   ----------    ----------
>>      1       135        135	     137
>>      2      1045       1120	     747
>>      3      1827       2345     	    1084
>>      4      2689       2934	    1438
>>      5      3736       3658	    1722
>>      6      4942       4434	    2092
>>      7      6304       5176          2245
>>      8      7736       5955          2388
> Are these figures with or without the later PV support patches?

This is without the PV patch.

Regards,
Longman

  parent reply	other threads:[~2014-03-13 19:03 UTC|newest]

Thread overview: 135+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-03-12 18:54 [PATCH v6 00/11] qspinlock: a 4-byte queue spinlock with PV support Waiman Long
2014-03-12 18:54 ` [PATCH v6 01/11] qspinlock: A generic 4-byte queue spinlock implementation Waiman Long
2014-03-12 18:54 ` Waiman Long
2014-03-12 18:54 ` [PATCH v6 02/11] qspinlock, x86: Enable x86-64 to use queue spinlock Waiman Long
2014-03-12 18:54 ` Waiman Long
2014-03-12 18:54 ` [PATCH v6 03/11] qspinlock: More optimized code for smaller NR_CPUS Waiman Long
2014-03-12 18:54 ` Waiman Long
2014-03-12 18:54 ` [PATCH v6 04/11] qspinlock: Optimized code path for 2 contending tasks Waiman Long
2014-03-12 19:08   ` Waiman Long
2014-03-12 19:08     ` Waiman Long
2014-03-13 13:57     ` Peter Zijlstra
2014-03-13 13:57     ` Peter Zijlstra
2014-03-13 13:57       ` Peter Zijlstra
2014-03-17 17:23       ` Waiman Long
2014-03-17 17:23       ` Waiman Long
2014-03-17 17:23         ` Waiman Long
2014-03-12 19:08   ` Waiman Long
2014-03-12 18:54 ` Waiman Long
2014-03-12 18:54 ` [PATCH v6 05/11] pvqspinlock, x86: Allow unfair spinlock in a PV guest Waiman Long
2014-03-13 10:54   ` David Vrabel
2014-03-13 10:54   ` David Vrabel
2014-03-13 10:54     ` David Vrabel
2014-03-13 13:16     ` Paolo Bonzini
2014-03-13 13:16     ` Paolo Bonzini
2014-03-13 13:16       ` Paolo Bonzini
2014-03-13 13:16       ` Paolo Bonzini
2014-03-17 19:05       ` Konrad Rzeszutek Wilk
2014-03-17 19:05       ` Konrad Rzeszutek Wilk
2014-03-17 19:05         ` Konrad Rzeszutek Wilk
2014-03-17 19:05         ` Konrad Rzeszutek Wilk
2014-03-18  8:14         ` Paolo Bonzini
2014-03-18  8:14         ` Paolo Bonzini
2014-03-18  8:14           ` Paolo Bonzini
2014-03-19  3:15           ` Waiman Long
2014-03-19  3:15           ` Waiman Long
2014-03-19  3:15           ` Waiman Long
2014-03-19  3:15             ` Waiman Long
2014-03-19  3:15             ` Waiman Long
2014-03-19 10:07             ` Paolo Bonzini
2014-03-19 16:58               ` Waiman Long
2014-03-19 16:58               ` Waiman Long
2014-03-19 16:58               ` Waiman Long
2014-03-19 17:08                 ` Paolo Bonzini
2014-03-19 17:08                 ` Paolo Bonzini
2014-03-19 17:08                   ` Paolo Bonzini
2014-03-19 10:07             ` Paolo Bonzini
2014-03-19 10:07             ` Paolo Bonzini
2014-03-13 19:03     ` Waiman Long
2014-03-13 19:03     ` Waiman Long [this message]
2014-03-13 19:03       ` Waiman Long
2014-03-13 15:15   ` Peter Zijlstra
2014-03-13 15:15     ` Peter Zijlstra
2014-03-13 20:05     ` Waiman Long
2014-03-13 20:05     ` Waiman Long
2014-03-13 20:05       ` Waiman Long
2014-03-14  8:30       ` Peter Zijlstra
2014-03-14  8:30         ` Peter Zijlstra
2014-03-14  8:48         ` Paolo Bonzini
2014-03-14  8:48         ` Paolo Bonzini
2014-03-14  8:48           ` Paolo Bonzini
2014-03-17 17:44         ` Waiman Long
2014-03-17 17:44         ` Waiman Long
2014-03-17 17:44           ` Waiman Long
2014-03-17 18:54           ` Peter Zijlstra
2014-03-17 18:54           ` Peter Zijlstra
2014-03-17 18:54             ` Peter Zijlstra
2014-03-18  8:16             ` Paolo Bonzini
2014-03-18  8:16               ` Paolo Bonzini
2014-03-18  8:16             ` Paolo Bonzini
2014-03-19  3:08             ` Waiman Long
2014-03-19  3:08             ` Waiman Long
2014-03-19  3:08               ` Waiman Long
2014-03-17 19:10           ` Konrad Rzeszutek Wilk
2014-03-17 19:10             ` Konrad Rzeszutek Wilk
2014-03-19  3:11             ` Waiman Long
2014-03-19  3:11             ` Waiman Long
2014-03-19  3:11               ` Waiman Long
2014-03-19 15:25               ` Konrad Rzeszutek Wilk
2014-03-19 15:25               ` Konrad Rzeszutek Wilk
2014-03-19 15:25                 ` Konrad Rzeszutek Wilk
2014-03-17 19:10           ` Konrad Rzeszutek Wilk
2014-03-14  8:30       ` Peter Zijlstra
2014-03-13 15:15   ` Peter Zijlstra
2014-03-12 18:54 ` Waiman Long
2014-03-12 18:54 ` [PATCH v6 06/11] pvqspinlock, x86: Allow unfair queue spinlock in a KVM guest Waiman Long
2014-03-12 18:54 ` Waiman Long
2014-03-12 18:54 ` [PATCH v6 07/11] pvqspinlock, x86: Allow unfair queue spinlock in a XEN guest Waiman Long
2014-03-12 18:54 ` Waiman Long
2014-03-12 18:54 ` [PATCH v6 08/11] pvqspinlock, x86: Rename paravirt_ticketlocks_enabled Waiman Long
2014-03-12 18:54 ` Waiman Long
2014-03-12 18:54 ` [PATCH RFC v6 09/11] pvqspinlock, x86: Add qspinlock para-virtualization support Waiman Long
2014-03-12 18:54 ` Waiman Long
2014-03-13 11:21   ` David Vrabel
2014-03-13 11:21     ` David Vrabel
2014-03-13 13:57     ` Paolo Bonzini
2014-03-13 13:57     ` Paolo Bonzini
2014-03-13 13:57       ` Paolo Bonzini
2014-03-13 13:57       ` Paolo Bonzini
2014-03-13 19:49       ` Waiman Long
2014-03-13 19:49       ` Waiman Long
2014-03-13 19:49       ` Waiman Long
2014-03-13 19:49         ` Waiman Long
2014-03-13 19:49         ` Waiman Long
2014-03-14  9:44         ` Paolo Bonzini
2014-03-14  9:44         ` Paolo Bonzini
2014-03-14  9:44         ` Paolo Bonzini
2014-03-13 13:57     ` Paolo Bonzini
2014-03-13 19:05     ` Waiman Long
2014-03-13 19:05       ` Waiman Long
2014-03-13 19:05     ` Waiman Long
2014-03-13 11:21   ` David Vrabel
2014-03-12 18:54 ` [PATCH RFC v6 10/11] pvqspinlock, x86: Enable qspinlock PV support for KVM Waiman Long
2014-03-12 18:54 ` Waiman Long
2014-03-13 13:59   ` Paolo Bonzini
2014-03-13 13:59     ` Paolo Bonzini
2014-03-13 19:13     ` Waiman Long
2014-03-13 19:13     ` Waiman Long
2014-03-14  8:42       ` Paolo Bonzini
2014-03-14  8:42       ` Paolo Bonzini
2014-03-14  8:42         ` Paolo Bonzini
2014-03-17 17:47         ` Waiman Long
2014-03-17 17:47         ` Waiman Long
2014-03-17 17:47           ` Waiman Long
2014-03-18  8:18           ` Paolo Bonzini
2014-03-18  8:18             ` Paolo Bonzini
2014-03-18  8:18           ` Paolo Bonzini
2014-03-13 13:59   ` Paolo Bonzini
2014-03-13 15:25   ` Peter Zijlstra
2014-03-13 15:25   ` Peter Zijlstra
2014-03-13 15:25     ` Peter Zijlstra
2014-03-13 20:09     ` Waiman Long
2014-03-13 20:09     ` Waiman Long
2014-03-13 20:09       ` Waiman Long
2014-03-12 18:54 ` [PATCH RFC v6 11/11] pvqspinlock, x86: Enable qspinlock PV support for XEN Waiman Long
2014-03-12 18:54 ` Waiman Long

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=532200E5.2050107@hp.com \
    --to=waiman.long@hp.com \
    --cc=akataria@vmware.com \
    --cc=andi@firstfloor.org \
    --cc=arnd@arndb.de \
    --cc=aswin@hp.com \
    --cc=chrisw@sous-sol.org \
    --cc=david.vrabel@citrix.com \
    --cc=gleb@redhat.com \
    --cc=hpa@zytor.com \
    --cc=jeremy@goop.org \
    --cc=konrad.wilk@oracle.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=oleg@redhat.com \
    --cc=paulmck@linux.vnet.ibm.com \
    --cc=peterz@infradead.org \
    --cc=raghavendra.kt@linux.vnet.ibm.com \
    --cc=riel@redhat.com \
    --cc=rostedt@goodmis.org \
    --cc=scott.norton@hp.com \
    --cc=tglx@linutronix.de \
    --cc=virtualization@lists.linux-foundation.org \
    --cc=walken@google.com \
    --cc=x86@kernel.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.