* [PATCH] KVM: Avoid atomic operations when kicking the running vCPU
@ 2021-10-20 14:52 Paolo Bonzini
2021-10-20 19:34 ` Sean Christopherson
0 siblings, 1 reply; 3+ messages in thread
From: Paolo Bonzini @ 2021-10-20 14:52 UTC (permalink / raw)
To: linux-kernel, kvm; +Cc: wanpengli, seanjc
If we do have the vcpu mutex, as is the case if kvm_running_vcpu is set
to the target vcpu of the kick, changes to vcpu->mode do not need atomic
operations; cmpxchg is only needed _outside_ the mutex to ensure that
the IN_GUEST_MODE->EXITING_GUEST_MODE change does not race with the vcpu
thread going OUTSIDE_GUEST_MODE.
Use this to optimize the case of a vCPU sending an interrupt to itself.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
virt/kvm/kvm_main.c | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 3f6d450355f0..9f45f26fce4f 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -3325,6 +3325,19 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
if (kvm_vcpu_wake_up(vcpu))
return;
+ me = get_cpu();
+ /*
+ * The only state change done outside the vcpu mutex is IN_GUEST_MODE
+ * to EXITING_GUEST_MODE. Therefore the moderately expensive "should
+ * kick" check does not need atomic operations if kvm_vcpu_kick is used
+ * within the vCPU thread itself.
+ */
+ if (vcpu == __this_cpu_read(kvm_running_vcpu)) {
+ if (vcpu->mode == IN_GUEST_MODE)
+ WRITE_ONCE(vcpu->mode, EXITING_GUEST_MODE);
+ goto out;
+ }
+
/*
* Note, the vCPU could get migrated to a different pCPU at any point
* after kvm_arch_vcpu_should_kick(), which could result in sending an
@@ -3332,12 +3345,12 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
* IPI is to force the vCPU to leave IN_GUEST_MODE, and migrating the
* vCPU also requires it to leave IN_GUEST_MODE.
*/
- me = get_cpu();
if (kvm_arch_vcpu_should_kick(vcpu)) {
cpu = READ_ONCE(vcpu->cpu);
if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))
smp_send_reschedule(cpu);
}
+out:
put_cpu();
}
EXPORT_SYMBOL_GPL(kvm_vcpu_kick);
--
2.27.0
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [PATCH] KVM: Avoid atomic operations when kicking the running vCPU
2021-10-20 14:52 [PATCH] KVM: Avoid atomic operations when kicking the running vCPU Paolo Bonzini
@ 2021-10-20 19:34 ` Sean Christopherson
2021-10-20 21:32 ` Paolo Bonzini
0 siblings, 1 reply; 3+ messages in thread
From: Sean Christopherson @ 2021-10-20 19:34 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: linux-kernel, kvm, wanpengli
On Wed, Oct 20, 2021, Paolo Bonzini wrote:
> If we do have the vcpu mutex, as is the case if kvm_running_vcpu is set
> to the target vcpu of the kick, changes to vcpu->mode do not need atomic
> operations; cmpxchg is only needed _outside_ the mutex to ensure that
> the IN_GUEST_MODE->EXITING_GUEST_MODE change does not race with the vcpu
> thread going OUTSIDE_GUEST_MODE.
>
> Use this to optimize the case of a vCPU sending an interrupt to itself.
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
> virt/kvm/kvm_main.c | 15 ++++++++++++++-
> 1 file changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index 3f6d450355f0..9f45f26fce4f 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -3325,6 +3325,19 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
> if (kvm_vcpu_wake_up(vcpu))
> return;
>
> + me = get_cpu();
> + /*
> + * The only state change done outside the vcpu mutex is IN_GUEST_MODE
> + * to EXITING_GUEST_MODE. Therefore the moderately expensive "should
> + * kick" check does not need atomic operations if kvm_vcpu_kick is used
> + * within the vCPU thread itself.
> + */
> + if (vcpu == __this_cpu_read(kvm_running_vcpu)) {
> + if (vcpu->mode == IN_GUEST_MODE)
> + WRITE_ONCE(vcpu->mode, EXITING_GUEST_MODE);
Fun. I had a whole thing typed out about this being unsafe because it implicitly
relies on a pending request and that there's a kvm_vcpu_exit_request() check _after_
this kick. Then I saw your other patches, and then I realized we already have this
bug in the kvm_arch_vcpu_should_kick() below.
Anyways, I also think we should add do:
if (vcpu == __this_cpu_read(kvm_running_vcpu)) {
if (vcpu->mode == IN_GUEST_MODE &&
!WARN_ON_ONCE(!kvm_request_pending(vcpu)))
WRITE_ONCE(vcpu->mode, EXITING_GUEST_MODE);
goto out;
}
The idea being that delaying or even missing an event in case of a KVM bug is
preferable to letting the vCPU state become invalid due to running in the guest
with EXITING_GUEST_MODE.
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] KVM: Avoid atomic operations when kicking the running vCPU
2021-10-20 19:34 ` Sean Christopherson
@ 2021-10-20 21:32 ` Paolo Bonzini
0 siblings, 0 replies; 3+ messages in thread
From: Paolo Bonzini @ 2021-10-20 21:32 UTC (permalink / raw)
To: Sean Christopherson; +Cc: linux-kernel, kvm, wanpengli
On 20/10/21 21:34, Sean Christopherson wrote:
>>
>> + /*
>> + * The only state change done outside the vcpu mutex is IN_GUEST_MODE
>> + * to EXITING_GUEST_MODE. Therefore the moderately expensive "should
>> + * kick" check does not need atomic operations if kvm_vcpu_kick is used
>> + * within the vCPU thread itself.
>> + */
>> + if (vcpu == __this_cpu_read(kvm_running_vcpu)) {
>> + if (vcpu->mode == IN_GUEST_MODE)
>> + WRITE_ONCE(vcpu->mode, EXITING_GUEST_MODE);
>
> Fun. I had a whole thing typed out about this being unsafe because it implicitly
> relies on a pending request and that there's a kvm_vcpu_exit_request() check _after_
> this kick. Then I saw your other patches, and then I realized we already have this
> bug in the kvm_arch_vcpu_should_kick() below.
Yeah, the three patches are independent but part of the same rabbit hole.
> Anyways, I also think we should add do:
>
> if (vcpu == __this_cpu_read(kvm_running_vcpu)) {
> if (vcpu->mode == IN_GUEST_MODE &&
> !WARN_ON_ONCE(!kvm_request_pending(vcpu)))
> WRITE_ONCE(vcpu->mode, EXITING_GUEST_MODE);
> goto out;
> }
>
> The idea being that delaying or even missing an event in case of a KVM bug is
> preferable to letting the vCPU state become invalid due to running in the guest
> with EXITING_GUEST_MODE.
On one hand I like the idea of having a safety net; for example a test
similar to this one would have triggered for the naked
kvm_vcpu_exiting_guest_mode(vcpu) call in vmx_sync_pir_to_irr.
On the other hand, "request-less VCPU kicks", as
Documentation/virt/kvm/vcpu-requests.rst calls them, are a thing; PPC
book3s_hv does not use vcpu->requests at all. For an artificial but more
relatable example, the ON bit takes the role of vcpu->requests when
processing PIR. Hence the code below would be suboptimal but still correct:
for (;;) {
exit_fastpath = static_call(kvm_x86_run)(vcpu);
if (likely(exit_fastpath !=
EXIT_FASTPATH_REENTER_GUEST))
break;
if (vcpu->arch.apicv_active && pi_test_on(vcpu))
kvm_vcpu_kick(vcpu);
if (unlikely(kvm_vcpu_exit_request(vcpu))) {
exit_fastpath = EXIT_FASTPATH_EXIT_HANDLED;
break;
}
}
All that really matters is that every call to kvm_x86_run is guarded by
kvm_vcpu_exit_request(vcpu), and indeed that's what is restored by "KVM:
x86: check for interrupts before deciding whether to exit the fast
path". The other architectures also have similar checks, though again
it's a bit hard to find it for book3s_hv (due to not using
vcpu->requests) and MIPS (which only uses KVM_REQ_TLB_FLUSH).
Paolo
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-10-20 21:32 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-20 14:52 [PATCH] KVM: Avoid atomic operations when kicking the running vCPU Paolo Bonzini
2021-10-20 19:34 ` Sean Christopherson
2021-10-20 21:32 ` Paolo Bonzini
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.