* [PATCH] KVM: Boosting vCPUs that are delivering interrupts
@ 2019-07-12 7:10 Wanpeng Li
2019-07-15 10:53 ` Christian Borntraeger
0 siblings, 1 reply; 3+ messages in thread
From: Wanpeng Li @ 2019-07-12 7:10 UTC (permalink / raw)
To: linux-kernel, kvm
Cc: Paolo Bonzini, Radim Krčmář, Christian Borntraeger
From: Wanpeng Li <wanpengli@tencent.com>
Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during vcpu wakeup
and interrupt delivery), except the lock holder, we want to also boost vCPUs
that are delivering interrupts. Actually most smp_call_function_many calls are
synchronous ipi calls, the ipi target vCPUs are also good yield candidates.
This patch sets preempted flag during wakeup and interrupt delivery time.
Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
ebizzy -M
vanilla boosting improved
1VM 23000 21232 -9%
2VM 2800 8000 180%
3VM 1800 3100 72%
Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs,
one running ebizzy -M, the other running 'stress --cpu 2':
w/ boosting + w/o pv sched yield(vanilla)
vanilla boosting improved
1570 4000 55%
w/ boosting + w/ pv sched yield(vanilla)
vanilla boosting improved
1844 5157 79%
w/o boosting, perf top in VM:
72.33% [kernel] [k] smp_call_function_many
4.22% [kernel] [k] call_function_i
3.71% [kernel] [k] async_page_fault
w/ boosting, perf top in VM:
38.43% [kernel] [k] smp_call_function_many
6.31% [kernel] [k] async_page_fault
6.13% libc-2.23.so [.] __memcpy_avx_unaligned
4.88% [kernel] [k] call_function_interrupt
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
---
virt/kvm/kvm_main.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index b4ab59d..2c46705 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2404,8 +2404,10 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
int me;
int cpu = vcpu->cpu;
- if (kvm_vcpu_wake_up(vcpu))
+ if (kvm_vcpu_wake_up(vcpu)) {
+ vcpu->preempted = true;
return;
+ }
me = get_cpu();
if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))
--
2.7.4
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] KVM: Boosting vCPUs that are delivering interrupts
2019-07-12 7:10 [PATCH] KVM: Boosting vCPUs that are delivering interrupts Wanpeng Li
@ 2019-07-15 10:53 ` Christian Borntraeger
2019-07-15 11:15 ` Wanpeng Li
0 siblings, 1 reply; 3+ messages in thread
From: Christian Borntraeger @ 2019-07-15 10:53 UTC (permalink / raw)
To: Wanpeng Li, linux-kernel, kvm; +Cc: Paolo Bonzini, Radim Krčmář
On 12.07.19 09:10, Wanpeng Li wrote:
> From: Wanpeng Li <wanpengli@tencent.com>
>
> Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during vcpu wakeup
> and interrupt delivery), except the lock holder, we want to also boost vCPUs
> that are delivering interrupts. Actually most smp_call_function_many calls are
> synchronous ipi calls, the ipi target vCPUs are also good yield candidates.
> This patch sets preempted flag during wakeup and interrupt delivery time.
>
> Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
> ebizzy -M
>
> vanilla boosting improved
> 1VM 23000 21232 -9%
> 2VM 2800 8000 180%
> 3VM 1800 3100 72%
>
> Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs,
> one running ebizzy -M, the other running 'stress --cpu 2':
>
> w/ boosting + w/o pv sched yield(vanilla)
>
> vanilla boosting improved
> 1570 4000 55%
>
> w/ boosting + w/ pv sched yield(vanilla)
>
> vanilla boosting improved
> 1844 5157 79%
>
> w/o boosting, perf top in VM:
>
> 72.33% [kernel] [k] smp_call_function_many
> 4.22% [kernel] [k] call_function_i
> 3.71% [kernel] [k] async_page_fault
>
> w/ boosting, perf top in VM:
>
> 38.43% [kernel] [k] smp_call_function_many
> 6.31% [kernel] [k] async_page_fault
> 6.13% libc-2.23.so [.] __memcpy_avx_unaligned
> 4.88% [kernel] [k] call_function_interrupt
This certainly made sense for s390 so I guess that this also makes sense
for others.
Nnote we (s390) do not use kvm_vcpu_kick, so this should not cause
any issue for s390.
>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Krčmář <rkrcmar@redhat.com>
> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> ---
> virt/kvm/kvm_main.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index b4ab59d..2c46705 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -2404,8 +2404,10 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
> int me;
> int cpu = vcpu->cpu;
>
> - if (kvm_vcpu_wake_up(vcpu))
> + if (kvm_vcpu_wake_up(vcpu)) {
> + vcpu->preempted = true;
> return;
> + }
>
> me = get_cpu();
> if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] KVM: Boosting vCPUs that are delivering interrupts
2019-07-15 10:53 ` Christian Borntraeger
@ 2019-07-15 11:15 ` Wanpeng Li
0 siblings, 0 replies; 3+ messages in thread
From: Wanpeng Li @ 2019-07-15 11:15 UTC (permalink / raw)
To: Christian Borntraeger
Cc: LKML, kvm, Paolo Bonzini, Radim Krčmář,
Paul Mackerras, Marc Zyngier
Cc arm and powerpc people,
On Mon, 15 Jul 2019 at 18:53, Christian Borntraeger
<borntraeger@de.ibm.com> wrote:
>
>
>
> On 12.07.19 09:10, Wanpeng Li wrote:
> > From: Wanpeng Li <wanpengli@tencent.com>
> >
> > Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during vcpu wakeup
> > and interrupt delivery), except the lock holder, we want to also boost vCPUs
> > that are delivering interrupts. Actually most smp_call_function_many calls are
> > synchronous ipi calls, the ipi target vCPUs are also good yield candidates.
> > This patch sets preempted flag during wakeup and interrupt delivery time.
> >
> > Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
> > ebizzy -M
> >
> > vanilla boosting improved
> > 1VM 23000 21232 -9%
> > 2VM 2800 8000 180%
> > 3VM 1800 3100 72%
> >
> > Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs,
> > one running ebizzy -M, the other running 'stress --cpu 2':
> >
> > w/ boosting + w/o pv sched yield(vanilla)
> >
> > vanilla boosting improved
> > 1570 4000 55%
> >
> > w/ boosting + w/ pv sched yield(vanilla)
> >
> > vanilla boosting improved
> > 1844 5157 79%
> >
> > w/o boosting, perf top in VM:
> >
> > 72.33% [kernel] [k] smp_call_function_many
> > 4.22% [kernel] [k] call_function_i
> > 3.71% [kernel] [k] async_page_fault
> >
> > w/ boosting, perf top in VM:
> >
> > 38.43% [kernel] [k] smp_call_function_many
> > 6.31% [kernel] [k] async_page_fault
> > 6.13% libc-2.23.so [.] __memcpy_avx_unaligned
> > 4.88% [kernel] [k] call_function_interrupt
> This certainly made sense for s390 so I guess that this also makes sense
> for others.
> Nnote we (s390) do not use kvm_vcpu_kick, so this should not cause
> any issue for s390.
>
>
> >
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Cc: Radim Krčmář <rkrcmar@redhat.com>
> > Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> > Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > ---
> > virt/kvm/kvm_main.c | 4 +++-
> > 1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> > index b4ab59d..2c46705 100644
> > --- a/virt/kvm/kvm_main.c
> > +++ b/virt/kvm/kvm_main.c
> > @@ -2404,8 +2404,10 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
> > int me;
> > int cpu = vcpu->cpu;
> >
> > - if (kvm_vcpu_wake_up(vcpu))
> > + if (kvm_vcpu_wake_up(vcpu)) {
> > + vcpu->preempted = true;
> > return;
> > + }
> >
> > me = get_cpu();
> > if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))
> >
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2019-07-15 11:15 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-12 7:10 [PATCH] KVM: Boosting vCPUs that are delivering interrupts Wanpeng Li
2019-07-15 10:53 ` Christian Borntraeger
2019-07-15 11:15 ` Wanpeng Li
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).