From: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com> To: Alexander Graf <agraf@suse.de> Cc: Jeremy Fitzhardinge <jeremy@goop.org>, Greg Kroah-Hartman <gregkh@suse.de>, linux-doc@vger.kernel.org, Peter Zijlstra <peterz@infradead.org>, Jan Kiszka <jan.kiszka@siemens.com>, Srivatsa Vaddagiri <vatsa@linux.vnet.ibm.com>, Paul Mackerras <paulus@samba.org>, "H. Peter Anvin" <hpa@zytor.com>, Stefano Stabellini <stefano.stabellini@eu.citrix.com>, Xen <xen-devel@lists.xensource.com>, Dave Jiang <dave.jiang@intel.com>, KVM <kvm@vger.kernel.org>, Glauber Costa <glommer@redhat.com>, X86 <x86@kernel.org>, Ingo Molnar <mingo@redhat.com>, Avi Kivity <avi@redhat.com>, Rik van Riel <riel@redhat.com>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, Sasha Levin <levinsasha928@gmail.com>, Sedat Dilek <sedat.dilek@gmail.com>, Thomas Gleixner <tglx@linutronix.de>, Virtualization <virtualization@lists.linux-foundation.org>, LKML <linux-kernel@vger.kernel.org>, Dave Hansen <dave@linu Subject: Re: [PATCH RFC V4 0/5] kvm : Paravirt-spinlock support for KVM guests Date: Wed, 18 Jan 2012 00:06:30 +0530 [thread overview] Message-ID: <4F15BFAE.7060500@linux.vnet.ibm.com> (raw) In-Reply-To: <1485A122-9D48-46E3-A01E-E37B5C9EC54A@suse.de> On 01/17/2012 11:09 PM, Alexander Graf wrote: > > On 17.01.2012, at 18:27, Raghavendra K T wrote: > >> On 01/17/2012 12:12 AM, Alexander Graf wrote: >>> >>> On 16.01.2012, at 19:38, Raghavendra K T wrote: >>> >>>> On 01/16/2012 07:53 PM, Alexander Graf wrote: >>>>> >>>>> On 16.01.2012, at 15:20, Srivatsa Vaddagiri wrote: >>>>> >>>>>> * Alexander Graf<agraf@suse.de> [2012-01-16 04:57:45]: >>>>>> >>>>>>> Speaking of which - have you benchmarked performance degradation of pv ticket locks on bare metal? >>>>>> >>>>>> You mean, run kernel on bare metal with CONFIG_PARAVIRT_SPINLOCKS >>>>>> enabled and compare how it performs with CONFIG_PARAVIRT_SPINLOCKS disabled for >>>>>> some workload(s)? >>>>> >>>>> Yup >>>>> >>>>>> >>>>>> In some sense, the 1x overcommitcase results posted does measure the overhead >>>>>> of (pv-)spinlocks no? We don't see any overhead in that case for atleast >>>>>> kernbench .. >>>>>> >>>>>>> Result for Non PLE machine : >>>>>>> ============================ >>>>>> >>>>>> [snip] >>>>>> >>>>>>> Kernbench: >>>>>>> BASE BASE+patch >>>>> >>>>> What is BASE really? Is BASE already with the PV spinlocks enabled? I'm having a hard time understanding which tree you're working against, since the prerequisites aren't upstream yet. >>>>> >>>>> >>>>> Alex >>>> >>>> Sorry for confusion, I think I was little imprecise on the BASE. >>>> >>>> The BASE is pre 3.2.0 + Jeremy's following patches: >>>> xadd (https://lkml.org/lkml/2011/10/4/328) >>>> x86/ticketlocklock (https://lkml.org/lkml/2011/10/12/496). >>>> So this would have ticketlock cleanups from Jeremy and >>>> CONFIG_PARAVIRT_SPINLOCKS=y >>>> >>>> BASE+patch = pre 3.2.0 + Jeremy's above patches + above V5 PV spinlock >>>> series and CONFIG_PARAVIRT_SPINLOCKS=y >>>> >>>> In both the cases CONFIG_PARAVIRT_SPINLOCKS=y. >>>> >>>> So let, >>>> A. pre-3.2.0 with CONFIG_PARAVIRT_SPINLOCKS = n >>>> B. pre-3.2.0 + Jeremy's above patches with CONFIG_PARAVIRT_SPINLOCKS = n >>>> C. pre-3.2.0 + Jeremy's above patches with CONFIG_PARAVIRT_SPINLOCKS = y >>>> D. pre-3.2.0 + Jeremy's above patches + V5 patches with CONFIG_PARAVIRT_SPINLOCKS = n >>>> E. pre-3.2.0 + Jeremy's above patches + V5 patches with CONFIG_PARAVIRT_SPINLOCKS = y >>>> >>>> is it performance of A vs E ? (currently C vs E) >>> >>> Since D and E only matter with KVM in use, yes, I'm mostly interested in A, B and C :). >>> >>> >>> Alex >>> >>> >> setup : >> Native: IBM xSeries with Intel(R) Xeon(R) x5570 2.93GHz CPU with 8 core , 64GB RAM, (16 cpu online) >> >> Guest : Single guest with 8 VCPU 4GB Ram. >> benchmark : kernbench -f -H -M -o 20 >> >> Here is the result : >> Native Run >> ============ >> case A case B %improvement case C %improvement >> 56.1917 (2.57125) 56.035 (2.02439) 0.278867 56.27 (2.40401) -0.139344 > > This looks a lot like statistical derivation. How often did you execute the test case? Did you make sure to have a clean base state every time? > > Maybe it'd be a good idea to create a small in-kernel microbenchmark with a couple threads that take spinlocks, then do work for a specified number of cycles, then release them again and start anew. At the end of it, we can check how long the whole thing took for n runs. That would enable us to measure the worst case scenario. > It was a quick test. two iteration of kernbench (=6runs) and had ensured cache is cleared. echo "1" > /proc/sys/vm/drop_caches ccache -C. Yes may be I can run test as you mentioned.. >> >> Guest Run >> ============ >> case A case B %improvement case C %improvement >> 166.999 (15.7613) 161.876 (14.4874) 3.06768 161.24 (12.6497) 3.44852 > > Is this the same machine? Why is the guest 3x slower? Yes non - ple machine but with all 16 cpus online. 3x slower you meant case A is slower (pre-3.2.0 with CONFIG_PARAVIRT_SPINLOCKS = n) ? > > > Alex > >> >> We do not see much overhead in native run with CONFIG_PARAVIRT_SPINLOCKS = y >> > >
WARNING: multiple messages have this Message-ID (diff)
From: Raghavendra K T <raghavendra.kt@linux.vnet.ibm.com> To: Alexander Graf <agraf@suse.de> Cc: Jeremy Fitzhardinge <jeremy@goop.org>, Greg Kroah-Hartman <gregkh@suse.de>, linux-doc@vger.kernel.org, Peter Zijlstra <peterz@infradead.org>, Jan Kiszka <jan.kiszka@siemens.com>, Srivatsa Vaddagiri <vatsa@linux.vnet.ibm.com>, Paul Mackerras <paulus@samba.org>, "H. Peter Anvin" <hpa@zytor.com>, Stefano Stabellini <stefano.stabellini@eu.citrix.com>, Xen <xen-devel@lists.xensource.com>, Dave Jiang <dave.jiang@intel.com>, KVM <kvm@vger.kernel.org>, Glauber Costa <glommer@redhat.com>, X86 <x86@kernel.org>, Ingo Molnar <mingo@redhat.com>, Avi Kivity <avi@redhat.com>, Rik van Riel <riel@redhat.com>, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>, Sasha Levin <levinsasha928@gmail.com>, Sedat Dilek <sedat.dilek@gmail.com>, Thomas Gleixner <tglx@linutronix.de>, Virtualization <virtualization@lists.linux-foundation.org>, LKML <linux-kernel@vger.kernel.org>, Dave Hansen <dave@linu> Subject: Re: [PATCH RFC V4 0/5] kvm : Paravirt-spinlock support for KVM guests Date: Wed, 18 Jan 2012 00:06:30 +0530 [thread overview] Message-ID: <4F15BFAE.7060500@linux.vnet.ibm.com> (raw) In-Reply-To: <1485A122-9D48-46E3-A01E-E37B5C9EC54A@suse.de> On 01/17/2012 11:09 PM, Alexander Graf wrote: > > On 17.01.2012, at 18:27, Raghavendra K T wrote: > >> On 01/17/2012 12:12 AM, Alexander Graf wrote: >>> >>> On 16.01.2012, at 19:38, Raghavendra K T wrote: >>> >>>> On 01/16/2012 07:53 PM, Alexander Graf wrote: >>>>> >>>>> On 16.01.2012, at 15:20, Srivatsa Vaddagiri wrote: >>>>> >>>>>> * Alexander Graf<agraf@suse.de> [2012-01-16 04:57:45]: >>>>>> >>>>>>> Speaking of which - have you benchmarked performance degradation of pv ticket locks on bare metal? >>>>>> >>>>>> You mean, run kernel on bare metal with CONFIG_PARAVIRT_SPINLOCKS >>>>>> enabled and compare how it performs with CONFIG_PARAVIRT_SPINLOCKS disabled for >>>>>> some workload(s)? >>>>> >>>>> Yup >>>>> >>>>>> >>>>>> In some sense, the 1x overcommitcase results posted does measure the overhead >>>>>> of (pv-)spinlocks no? We don't see any overhead in that case for atleast >>>>>> kernbench .. >>>>>> >>>>>>> Result for Non PLE machine : >>>>>>> ============================ >>>>>> >>>>>> [snip] >>>>>> >>>>>>> Kernbench: >>>>>>> BASE BASE+patch >>>>> >>>>> What is BASE really? Is BASE already with the PV spinlocks enabled? I'm having a hard time understanding which tree you're working against, since the prerequisites aren't upstream yet. >>>>> >>>>> >>>>> Alex >>>> >>>> Sorry for confusion, I think I was little imprecise on the BASE. >>>> >>>> The BASE is pre 3.2.0 + Jeremy's following patches: >>>> xadd (https://lkml.org/lkml/2011/10/4/328) >>>> x86/ticketlocklock (https://lkml.org/lkml/2011/10/12/496). >>>> So this would have ticketlock cleanups from Jeremy and >>>> CONFIG_PARAVIRT_SPINLOCKS=y >>>> >>>> BASE+patch = pre 3.2.0 + Jeremy's above patches + above V5 PV spinlock >>>> series and CONFIG_PARAVIRT_SPINLOCKS=y >>>> >>>> In both the cases CONFIG_PARAVIRT_SPINLOCKS=y. >>>> >>>> So let, >>>> A. pre-3.2.0 with CONFIG_PARAVIRT_SPINLOCKS = n >>>> B. pre-3.2.0 + Jeremy's above patches with CONFIG_PARAVIRT_SPINLOCKS = n >>>> C. pre-3.2.0 + Jeremy's above patches with CONFIG_PARAVIRT_SPINLOCKS = y >>>> D. pre-3.2.0 + Jeremy's above patches + V5 patches with CONFIG_PARAVIRT_SPINLOCKS = n >>>> E. pre-3.2.0 + Jeremy's above patches + V5 patches with CONFIG_PARAVIRT_SPINLOCKS = y >>>> >>>> is it performance of A vs E ? (currently C vs E) >>> >>> Since D and E only matter with KVM in use, yes, I'm mostly interested in A, B and C :). >>> >>> >>> Alex >>> >>> >> setup : >> Native: IBM xSeries with Intel(R) Xeon(R) x5570 2.93GHz CPU with 8 core , 64GB RAM, (16 cpu online) >> >> Guest : Single guest with 8 VCPU 4GB Ram. >> benchmark : kernbench -f -H -M -o 20 >> >> Here is the result : >> Native Run >> ============ >> case A case B %improvement case C %improvement >> 56.1917 (2.57125) 56.035 (2.02439) 0.278867 56.27 (2.40401) -0.139344 > > This looks a lot like statistical derivation. How often did you execute the test case? Did you make sure to have a clean base state every time? > > Maybe it'd be a good idea to create a small in-kernel microbenchmark with a couple threads that take spinlocks, then do work for a specified number of cycles, then release them again and start anew. At the end of it, we can check how long the whole thing took for n runs. That would enable us to measure the worst case scenario. > It was a quick test. two iteration of kernbench (=6runs) and had ensured cache is cleared. echo "1" > /proc/sys/vm/drop_caches ccache -C. Yes may be I can run test as you mentioned.. >> >> Guest Run >> ============ >> case A case B %improvement case C %improvement >> 166.999 (15.7613) 161.876 (14.4874) 3.06768 161.24 (12.6497) 3.44852 > > Is this the same machine? Why is the guest 3x slower? Yes non - ple machine but with all 16 cpus online. 3x slower you meant case A is slower (pre-3.2.0 with CONFIG_PARAVIRT_SPINLOCKS = n) ? > > > Alex > >> >> We do not see much overhead in native run with CONFIG_PARAVIRT_SPINLOCKS = y >> > >
next prev parent reply other threads:[~2012-01-17 18:36 UTC|newest] Thread overview: 139+ messages / expand[flat|nested] mbox.gz Atom feed top 2012-01-14 18:25 [PATCH RFC V4 0/5] kvm : Paravirt-spinlock support for KVM guests Raghavendra K T 2012-01-14 18:25 ` Raghavendra K T 2012-01-14 18:25 ` Raghavendra K T 2012-01-14 18:25 ` [PATCH RFC V4 1/5] debugfs: Add support to print u32 array in debugfs Raghavendra K T 2012-01-14 18:25 ` Raghavendra K T 2012-01-14 18:25 ` Raghavendra K T 2012-01-14 18:25 ` [PATCH RFC V4 2/5] kvm hypervisor : Add a hypercall to KVM hypervisor to support pv-ticketlocks Raghavendra K T 2012-01-14 18:25 ` Raghavendra K T 2012-01-14 18:25 ` Raghavendra K T 2012-01-16 3:24 ` Alexander Graf 2012-01-16 3:24 ` Alexander Graf 2012-01-16 8:43 ` Raghavendra K T 2012-01-16 8:43 ` Raghavendra K T 2012-01-16 9:03 ` Avi Kivity 2012-01-16 9:03 ` Avi Kivity 2012-01-16 9:55 ` Raghavendra K T 2012-01-16 9:55 ` Raghavendra K T 2012-01-14 18:26 ` [PATCH RFC V4 3/5] kvm guest : Added configuration support to enable debug information for KVM Guests Raghavendra K T 2012-01-14 18:26 ` Raghavendra K T 2012-01-14 18:26 ` Raghavendra K T 2012-01-14 18:26 ` [PATCH RFC V4 4/5] kvm : pv-ticketlocks support for linux guests running on KVM hypervisor Raghavendra K T 2012-01-14 18:26 ` Raghavendra K T 2012-01-14 18:26 ` Raghavendra K T 2012-01-16 3:12 ` Alexander Graf 2012-01-16 3:12 ` Alexander Graf 2012-01-16 7:25 ` Raghavendra K T 2012-01-16 7:25 ` Raghavendra K T 2012-01-16 9:05 ` Avi Kivity 2012-01-16 9:05 ` Avi Kivity 2012-01-16 14:13 ` Raghavendra K T 2012-01-16 14:13 ` Raghavendra K T 2012-01-16 14:47 ` Avi Kivity 2012-01-16 14:47 ` Avi Kivity 2012-01-16 23:49 ` Jeremy Fitzhardinge 2012-01-16 23:49 ` Jeremy Fitzhardinge 2012-01-17 11:02 ` Marcelo Tosatti 2012-01-17 11:02 ` Marcelo Tosatti 2012-01-17 11:33 ` Srivatsa Vaddagiri 2012-01-17 11:33 ` Srivatsa Vaddagiri 2012-01-18 1:34 ` Jeremy Fitzhardinge 2012-01-18 1:34 ` Jeremy Fitzhardinge 2012-01-18 13:54 ` Srivatsa Vaddagiri 2012-01-18 13:54 ` Srivatsa Vaddagiri 2012-01-18 21:52 ` Jeremy Fitzhardinge 2012-01-18 21:52 ` Jeremy Fitzhardinge 2012-01-24 14:08 ` Avi Kivity 2012-01-24 14:08 ` Avi Kivity 2012-01-24 18:51 ` Raghavendra K T 2012-01-24 18:51 ` Raghavendra K T 2012-01-17 18:57 ` Raghavendra K T 2012-01-17 18:57 ` Raghavendra K T 2012-01-24 19:01 ` Raghavendra K T 2012-01-14 18:27 ` [PATCH RFC V4 5/5] Documentation/kvm : Add documentation on Hypercalls and features used for PV spinlock Raghavendra K T 2012-01-14 18:27 ` Raghavendra K T 2012-01-14 18:27 ` Raghavendra K T 2012-01-14 18:27 ` Raghavendra K T 2012-01-16 3:23 ` Alexander Graf 2012-01-16 3:23 ` Alexander Graf 2012-01-16 3:51 ` Srivatsa Vaddagiri 2012-01-16 3:51 ` Srivatsa Vaddagiri 2012-01-16 4:00 ` Alexander Graf 2012-01-16 4:00 ` Alexander Graf 2012-01-16 8:47 ` Avi Kivity 2012-01-16 8:44 ` Raghavendra K T 2012-01-16 8:44 ` Raghavendra K T 2012-01-16 10:26 ` Alexander Graf 2012-01-16 10:26 ` Alexander Graf 2012-01-16 9:00 ` Avi Kivity 2012-01-16 9:00 ` Avi Kivity 2012-01-16 9:40 ` Srivatsa Vaddagiri 2012-01-16 10:14 ` Avi Kivity 2012-01-16 14:11 ` Srivatsa Vaddagiri 2012-01-17 9:14 ` Gleb Natapov 2012-01-17 9:14 ` Gleb Natapov 2012-01-17 12:26 ` Srivatsa Vaddagiri 2012-01-17 12:26 ` Srivatsa Vaddagiri 2012-01-17 12:51 ` Gleb Natapov 2012-01-17 12:51 ` Gleb Natapov 2012-01-17 13:11 ` Srivatsa Vaddagiri 2012-01-17 13:11 ` Srivatsa Vaddagiri 2012-01-17 13:20 ` Gleb Natapov 2012-01-17 13:20 ` Gleb Natapov 2012-01-17 14:28 ` Srivatsa Vaddagiri 2012-01-17 14:28 ` Srivatsa Vaddagiri 2012-01-17 15:32 ` Gleb Natapov 2012-01-17 15:32 ` Gleb Natapov 2012-01-17 15:53 ` Marcelo Tosatti 2012-01-17 15:53 ` Marcelo Tosatti 2012-01-20 15:09 ` Srivatsa Vaddagiri 2012-01-17 13:13 ` Raghavendra K T 2012-01-17 13:13 ` Raghavendra K T 2012-01-16 3:57 ` [PATCH RFC V4 0/5] kvm : Paravirt-spinlock support for KVM guests Alexander Graf 2012-01-16 3:57 ` Alexander Graf 2012-01-16 6:40 ` Jeremy Fitzhardinge 2012-01-16 6:40 ` Jeremy Fitzhardinge 2012-01-16 8:55 ` Avi Kivity 2012-01-16 8:55 ` Avi Kivity 2012-01-16 23:59 ` Jeremy Fitzhardinge 2012-01-16 23:59 ` Jeremy Fitzhardinge 2012-01-18 10:48 ` Raghavendra K T 2012-01-18 10:48 ` Raghavendra K T 2012-01-16 10:24 ` Alexander Graf 2012-01-16 10:24 ` Alexander Graf 2012-01-17 0:30 ` Jeremy Fitzhardinge 2012-01-17 0:30 ` Jeremy Fitzhardinge 2012-01-18 10:23 ` Raghavendra K T 2012-01-18 10:23 ` Raghavendra K T 2012-01-16 13:43 ` Raghavendra K T 2012-01-16 13:43 ` Raghavendra K T 2012-01-16 13:49 ` Avi Kivity 2012-01-16 13:49 ` Avi Kivity 2012-01-16 18:48 ` Raghavendra K T 2012-01-16 18:48 ` Raghavendra K T 2012-01-16 14:20 ` Srivatsa Vaddagiri 2012-01-16 14:20 ` Srivatsa Vaddagiri 2012-01-16 14:23 ` Alexander Graf 2012-01-16 14:23 ` Alexander Graf 2012-01-16 18:38 ` Raghavendra K T 2012-01-16 18:38 ` Raghavendra K T 2012-01-16 18:42 ` Alexander Graf 2012-01-16 18:42 ` Alexander Graf 2012-01-17 17:27 ` Raghavendra K T 2012-01-17 17:27 ` Raghavendra K T 2012-01-17 17:39 ` Alexander Graf 2012-01-17 17:39 ` Alexander Graf 2012-01-17 18:36 ` Raghavendra K T [this message] 2012-01-17 18:36 ` Raghavendra K T 2012-01-17 21:57 ` Dave Hansen 2012-01-17 21:57 ` Dave Hansen 2012-01-18 2:27 ` Raghavendra K T 2012-01-18 2:27 ` Raghavendra K T 2012-01-25 8:55 ` Raghavendra K T 2012-01-25 8:55 ` Raghavendra K T 2012-01-25 16:35 ` Konrad Rzeszutek Wilk 2012-01-25 16:35 ` Konrad Rzeszutek Wilk 2012-01-25 17:45 ` Raghavendra K T 2012-01-25 17:45 ` Raghavendra K T 2012-01-25 19:05 ` Konrad Rzeszutek Wilk 2012-01-25 19:05 ` Konrad Rzeszutek Wilk
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=4F15BFAE.7060500@linux.vnet.ibm.com \ --to=raghavendra.kt@linux.vnet.ibm.com \ --cc=agraf@suse.de \ --cc=avi@redhat.com \ --cc=dave.jiang@intel.com \ --cc=dave@linu \ --cc=glommer@redhat.com \ --cc=gregkh@suse.de \ --cc=hpa@zytor.com \ --cc=jan.kiszka@siemens.com \ --cc=jeremy@goop.org \ --cc=konrad.wilk@oracle.com \ --cc=kvm@vger.kernel.org \ --cc=levinsasha928@gmail.com \ --cc=linux-doc@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mingo@redhat.com \ --cc=paulus@samba.org \ --cc=peterz@infradead.org \ --cc=riel@redhat.com \ --cc=sedat.dilek@gmail.com \ --cc=stefano.stabellini@eu.citrix.com \ --cc=tglx@linutronix.de \ --cc=vatsa@linux.vnet.ibm.com \ --cc=virtualization@lists.linux-foundation.org \ --cc=x86@kernel.org \ --cc=xen-devel@lists.xensource.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.