From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> To: Peter Zijlstra <peterz@infradead.org> Cc: Waiman.Long@hp.com, tglx@linutronix.de, mingo@kernel.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, xen-devel@lists.xenproject.org, kvm@vger.kernel.org, paolo.bonzini@gmail.com, boris.ostrovsky@oracle.com, paulmck@linux.vnet.ibm.com, riel@redhat.com, torvalds@linux-foundation.org, raghavendra.kt@linux.vnet.ibm.com, david.vrabel@citrix.com, oleg@redhat.com, gleb@redhat.com, scott.norton@hp.com, chegu_vinod@hp.com Subject: Re: [PATCH 10/11] qspinlock: Paravirt support Date: Tue, 15 Jul 2014 10:23:05 -0400 [thread overview] Message-ID: <20140715142305.GA3403@laptop.dumpdata.com> (raw) In-Reply-To: <20140707152734.GX6758@twins.programming.kicks-ass.net> On Mon, Jul 07, 2014 at 05:27:34PM +0200, Peter Zijlstra wrote: > On Fri, Jun 20, 2014 at 09:46:08AM -0400, Konrad Rzeszutek Wilk wrote: > > I dug in the code and I have some comments about it, but before > > I post them I was wondering if you have any plans to run any performance > > tests against the PV ticketlock with normal and over-committed scenarios? > > I can barely boot a guest.. I'm not sure I can make them do anything > much at all yet. All this virt crap is totally painful. HA! The reason I asked about that is from a pen-and-paper view it looks suboptimal in the worst case scenario compared to PV ticketlock. The 'worst case scenario' is when we over-commit (more CPUs than there are physical CPUs) or have to delay guests (the sum of all virtual CPUs > physical CPUs and all of the guests are compiling kernels). In those cases the PV ticketlock goes to sleep and gets woken up once the ticket holder has finished. In the PV qspinlock we do wake up the first in queue, but we also wake the next one in queue so it can progress further. And so on. Perhaps a better mechanism is just ditch the queue part and utilize the byte part and under KVM and Xen just do bytelocking (since we have 8 bits). For the PV halt/waking we can stash in the 'struct mcs' the current lock that each CPU is waiting for. And the unlocker can iterate over all of those and wake them all up. Perhaps make the iteration random. Anyhow, that is how the old PV bytelock under Xen worked (before 3.11) and it had worked pretty well (it didn't do it random thought - always started with 'for_each_online_cpu'). Squashing in the ticketlock concept in qspinlock for PV looks scary. And as I said - this is all pen-and-paper - so it might be that this 'wake-up-go-sleep-on-the-queue' kick is actually not that bad? Lastly - thank you for taking a stab at this. >
WARNING: multiple messages have this Message-ID (diff)
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> To: Peter Zijlstra <peterz@infradead.org> Cc: Waiman.Long@hp.com, linux-arch@vger.kernel.org, riel@redhat.com, gleb@redhat.com, kvm@vger.kernel.org, boris.ostrovsky@oracle.com, scott.norton@hp.com, raghavendra.kt@linux.vnet.ibm.com, paolo.bonzini@gmail.com, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, chegu_vinod@hp.com, david.vrabel@citrix.com, oleg@redhat.com, xen-devel@lists.xenproject.org, tglx@linutronix.de, paulmck@linux.vnet.ibm.com, torvalds@linux-foundation.org, mingo@kernel.org Subject: Re: [PATCH 10/11] qspinlock: Paravirt support Date: Tue, 15 Jul 2014 10:23:05 -0400 [thread overview] Message-ID: <20140715142305.GA3403@laptop.dumpdata.com> (raw) In-Reply-To: <20140707152734.GX6758@twins.programming.kicks-ass.net> On Mon, Jul 07, 2014 at 05:27:34PM +0200, Peter Zijlstra wrote: > On Fri, Jun 20, 2014 at 09:46:08AM -0400, Konrad Rzeszutek Wilk wrote: > > I dug in the code and I have some comments about it, but before > > I post them I was wondering if you have any plans to run any performance > > tests against the PV ticketlock with normal and over-committed scenarios? > > I can barely boot a guest.. I'm not sure I can make them do anything > much at all yet. All this virt crap is totally painful. HA! The reason I asked about that is from a pen-and-paper view it looks suboptimal in the worst case scenario compared to PV ticketlock. The 'worst case scenario' is when we over-commit (more CPUs than there are physical CPUs) or have to delay guests (the sum of all virtual CPUs > physical CPUs and all of the guests are compiling kernels). In those cases the PV ticketlock goes to sleep and gets woken up once the ticket holder has finished. In the PV qspinlock we do wake up the first in queue, but we also wake the next one in queue so it can progress further. And so on. Perhaps a better mechanism is just ditch the queue part and utilize the byte part and under KVM and Xen just do bytelocking (since we have 8 bits). For the PV halt/waking we can stash in the 'struct mcs' the current lock that each CPU is waiting for. And the unlocker can iterate over all of those and wake them all up. Perhaps make the iteration random. Anyhow, that is how the old PV bytelock under Xen worked (before 3.11) and it had worked pretty well (it didn't do it random thought - always started with 'for_each_online_cpu'). Squashing in the ticketlock concept in qspinlock for PV looks scary. And as I said - this is all pen-and-paper - so it might be that this 'wake-up-go-sleep-on-the-queue' kick is actually not that bad? Lastly - thank you for taking a stab at this. >
next prev parent reply other threads:[~2014-07-15 14:24 UTC|newest] Thread overview: 191+ messages / expand[flat|nested] mbox.gz Atom feed top 2014-06-15 12:46 [PATCH 00/11] qspinlock with paravirt support Peter Zijlstra 2014-06-15 12:46 ` Peter Zijlstra 2014-06-15 12:46 ` [PATCH 01/11] qspinlock: A simple generic 4-byte queue spinlock Peter Zijlstra 2014-06-15 12:46 ` Peter Zijlstra 2014-06-15 12:46 ` Peter Zijlstra 2014-06-16 20:49 ` Konrad Rzeszutek Wilk 2014-06-16 20:49 ` Konrad Rzeszutek Wilk 2014-06-16 20:49 ` Konrad Rzeszutek Wilk 2014-06-17 20:03 ` Konrad Rzeszutek Wilk 2014-06-17 20:03 ` Konrad Rzeszutek Wilk 2014-06-17 20:03 ` Konrad Rzeszutek Wilk 2014-06-17 20:03 ` Konrad Rzeszutek Wilk 2014-06-23 16:12 ` Peter Zijlstra 2014-06-23 16:12 ` Peter Zijlstra 2014-06-23 16:12 ` Peter Zijlstra 2014-06-23 16:20 ` Konrad Rzeszutek Wilk 2014-06-23 16:20 ` Konrad Rzeszutek Wilk 2014-06-23 16:20 ` Konrad Rzeszutek Wilk 2014-06-17 20:03 ` Konrad Rzeszutek Wilk 2014-06-23 15:56 ` Peter Zijlstra 2014-06-23 15:56 ` Peter Zijlstra 2014-06-23 15:56 ` Peter Zijlstra 2014-06-23 16:16 ` Konrad Rzeszutek Wilk 2014-06-23 16:16 ` Konrad Rzeszutek Wilk 2014-06-23 16:16 ` Konrad Rzeszutek Wilk 2014-06-17 20:05 ` Konrad Rzeszutek Wilk 2014-06-17 20:05 ` Konrad Rzeszutek Wilk 2014-06-17 20:05 ` Konrad Rzeszutek Wilk 2014-06-17 20:05 ` Konrad Rzeszutek Wilk 2014-06-23 16:26 ` Peter Zijlstra 2014-06-23 16:26 ` Peter Zijlstra 2014-06-23 16:45 ` Konrad Rzeszutek Wilk 2014-06-23 16:45 ` Konrad Rzeszutek Wilk 2014-06-23 16:45 ` Konrad Rzeszutek Wilk 2014-06-23 16:26 ` Peter Zijlstra 2014-06-17 20:05 ` Konrad Rzeszutek Wilk 2014-06-15 12:46 ` [PATCH 02/11] qspinlock, x86: Enable x86-64 to use " Peter Zijlstra 2014-06-15 12:46 ` Peter Zijlstra 2014-06-15 12:46 ` Peter Zijlstra 2014-06-15 12:47 ` [PATCH 03/11] qspinlock: Add pending bit Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-17 20:36 ` Konrad Rzeszutek Wilk 2014-06-17 20:36 ` Konrad Rzeszutek Wilk 2014-06-17 20:36 ` Konrad Rzeszutek Wilk 2014-06-17 20:51 ` Waiman Long 2014-06-17 20:51 ` Waiman Long 2014-06-17 20:51 ` Waiman Long 2014-06-17 21:07 ` Konrad Rzeszutek Wilk 2014-06-17 21:07 ` Konrad Rzeszutek Wilk 2014-06-17 21:07 ` Konrad Rzeszutek Wilk 2014-06-17 21:10 ` Konrad Rzeszutek Wilk 2014-06-17 21:10 ` Konrad Rzeszutek Wilk 2014-06-17 22:25 ` Waiman Long 2014-06-17 22:25 ` Waiman Long 2014-06-17 22:25 ` Waiman Long 2014-06-17 21:10 ` Konrad Rzeszutek Wilk 2014-06-24 8:24 ` Peter Zijlstra 2014-06-24 8:24 ` Peter Zijlstra 2014-06-24 8:24 ` Peter Zijlstra 2014-06-18 11:29 ` Paolo Bonzini 2014-06-18 11:29 ` Paolo Bonzini 2014-06-18 11:29 ` Paolo Bonzini 2014-06-18 13:36 ` Konrad Rzeszutek Wilk 2014-06-18 13:36 ` Konrad Rzeszutek Wilk 2014-06-18 13:36 ` Konrad Rzeszutek Wilk 2014-06-23 16:35 ` Peter Zijlstra 2014-06-23 16:35 ` Peter Zijlstra 2014-06-23 16:35 ` Peter Zijlstra 2014-06-15 12:47 ` [PATCH 04/11] qspinlock: Extract out the exchange of tail code word Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-17 20:55 ` Konrad Rzeszutek Wilk 2014-06-17 20:55 ` Konrad Rzeszutek Wilk 2014-06-18 11:37 ` Paolo Bonzini 2014-06-18 11:37 ` Paolo Bonzini 2014-06-18 11:37 ` Paolo Bonzini 2014-06-18 13:50 ` Konrad Rzeszutek Wilk 2014-06-18 13:50 ` Konrad Rzeszutek Wilk 2014-06-18 13:50 ` Konrad Rzeszutek Wilk 2014-06-18 15:46 ` Waiman Long 2014-06-18 15:46 ` Waiman Long 2014-06-18 15:49 ` Paolo Bonzini 2014-06-18 15:49 ` Paolo Bonzini 2014-06-18 15:49 ` Paolo Bonzini 2014-06-18 16:02 ` Konrad Rzeszutek Wilk 2014-06-18 16:02 ` Konrad Rzeszutek Wilk 2014-06-18 16:02 ` Konrad Rzeszutek Wilk 2014-06-18 15:46 ` Waiman Long 2014-06-24 10:47 ` Peter Zijlstra 2014-06-24 10:47 ` Peter Zijlstra 2014-06-24 10:47 ` Peter Zijlstra 2014-06-17 20:55 ` Konrad Rzeszutek Wilk 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` [PATCH 05/11] qspinlock: Optimize for smaller NR_CPUS Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-18 11:39 ` Paolo Bonzini 2014-06-18 11:39 ` Paolo Bonzini 2014-06-18 11:39 ` Paolo Bonzini 2014-07-07 14:35 ` Peter Zijlstra 2014-07-07 14:35 ` Peter Zijlstra 2014-07-07 14:35 ` Peter Zijlstra 2014-07-07 15:08 ` Paolo Bonzini 2014-07-07 15:08 ` Paolo Bonzini 2014-07-07 15:08 ` Paolo Bonzini 2014-07-07 15:35 ` Peter Zijlstra 2014-07-07 15:35 ` Peter Zijlstra 2014-07-07 16:10 ` Paolo Bonzini 2014-07-07 16:10 ` Paolo Bonzini 2014-07-07 16:10 ` Paolo Bonzini 2014-07-07 15:35 ` Peter Zijlstra 2014-06-18 15:57 ` Konrad Rzeszutek Wilk 2014-06-18 15:57 ` Konrad Rzeszutek Wilk 2014-07-07 14:33 ` Peter Zijlstra 2014-07-07 14:33 ` Peter Zijlstra 2014-07-07 14:33 ` Peter Zijlstra 2014-06-18 15:57 ` Konrad Rzeszutek Wilk 2014-06-15 12:47 ` [PATCH 06/11] qspinlock: Optimize pending bit Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-18 11:42 ` Paolo Bonzini 2014-06-18 11:42 ` Paolo Bonzini 2014-06-18 11:42 ` Paolo Bonzini 2014-06-15 12:47 ` [PATCH 07/11] qspinlock: Use a simple write to grab the lock, if applicable Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-18 16:36 ` Konrad Rzeszutek Wilk 2014-06-18 16:36 ` Konrad Rzeszutek Wilk 2014-06-18 16:36 ` Konrad Rzeszutek Wilk 2014-07-07 14:51 ` Peter Zijlstra 2014-07-07 14:51 ` Peter Zijlstra 2014-07-07 14:51 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` [PATCH 08/11] qspinlock: Revert to test-and-set on hypervisors Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-16 21:57 ` Waiman Long 2014-06-16 21:57 ` Waiman Long 2014-06-16 21:57 ` Waiman Long 2014-06-18 16:40 ` Konrad Rzeszutek Wilk 2014-06-18 16:40 ` Konrad Rzeszutek Wilk 2014-06-18 16:40 ` Konrad Rzeszutek Wilk 2014-06-15 12:47 ` [PATCH 09/11] pvqspinlock, x86: Rename paravirt_ticketlocks_enabled Peter Zijlstra 2014-06-18 16:43 ` Konrad Rzeszutek Wilk 2014-06-18 16:43 ` Konrad Rzeszutek Wilk 2014-06-18 16:43 ` Konrad Rzeszutek Wilk 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` [PATCH 10/11] qspinlock: Paravirt support Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-16 22:08 ` Waiman Long 2014-06-18 12:03 ` Paolo Bonzini 2014-06-18 12:03 ` Paolo Bonzini 2014-06-18 15:26 ` Waiman Long 2014-06-18 15:26 ` Waiman Long 2014-06-18 15:26 ` Waiman Long 2014-07-07 15:20 ` Peter Zijlstra 2014-07-07 15:20 ` Peter Zijlstra 2014-07-07 15:20 ` Peter Zijlstra 2014-06-18 12:03 ` Paolo Bonzini 2014-07-07 15:20 ` Peter Zijlstra 2014-07-07 15:20 ` Peter Zijlstra 2014-07-07 15:20 ` Peter Zijlstra 2014-06-16 22:08 ` Waiman Long 2014-06-17 0:53 ` Waiman Long 2014-06-17 0:53 ` Waiman Long 2014-06-17 0:53 ` Waiman Long 2014-06-18 12:04 ` Paolo Bonzini 2014-06-18 12:04 ` Paolo Bonzini 2014-06-18 12:04 ` Paolo Bonzini 2014-06-20 13:46 ` Konrad Rzeszutek Wilk 2014-06-20 13:46 ` Konrad Rzeszutek Wilk 2014-06-20 13:46 ` Konrad Rzeszutek Wilk 2014-07-07 15:27 ` Peter Zijlstra 2014-07-15 14:23 ` Konrad Rzeszutek Wilk 2014-07-15 14:23 ` Konrad Rzeszutek Wilk [this message] 2014-07-15 14:23 ` Konrad Rzeszutek Wilk 2014-07-07 15:27 ` Peter Zijlstra 2014-07-07 15:27 ` Peter Zijlstra 2014-06-15 12:47 ` [PATCH 11/11] qspinlock, kvm: Add paravirt support Peter Zijlstra 2014-06-22 16:36 ` Raghavendra K T 2014-06-22 16:36 ` Raghavendra K T 2014-07-07 15:23 ` Peter Zijlstra 2014-07-07 15:23 ` Peter Zijlstra 2014-07-07 15:23 ` Peter Zijlstra 2014-06-22 16:36 ` Raghavendra K T 2014-06-15 12:47 ` Peter Zijlstra 2014-06-15 12:47 ` Peter Zijlstra 2014-06-16 20:52 ` [PATCH 00/11] qspinlock with " Konrad Rzeszutek Wilk 2014-06-16 20:52 ` Konrad Rzeszutek Wilk 2014-06-16 20:52 ` Konrad Rzeszutek Wilk
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20140715142305.GA3403@laptop.dumpdata.com \ --to=konrad.wilk@oracle.com \ --cc=Waiman.Long@hp.com \ --cc=boris.ostrovsky@oracle.com \ --cc=chegu_vinod@hp.com \ --cc=david.vrabel@citrix.com \ --cc=gleb@redhat.com \ --cc=kvm@vger.kernel.org \ --cc=linux-arch@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mingo@kernel.org \ --cc=oleg@redhat.com \ --cc=paolo.bonzini@gmail.com \ --cc=paulmck@linux.vnet.ibm.com \ --cc=peterz@infradead.org \ --cc=raghavendra.kt@linux.vnet.ibm.com \ --cc=riel@redhat.com \ --cc=scott.norton@hp.com \ --cc=tglx@linutronix.de \ --cc=torvalds@linux-foundation.org \ --cc=virtualization@lists.linux-foundation.org \ --cc=xen-devel@lists.xenproject.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.