[PATCH RESEND v2] KVM: LAPIC: Enable timer posted-interrupt when mwait/hlt is advertised

[PATCH RESEND v2] KVM: LAPIC: Enable timer posted-interrupt when mwait/hlt is advertised

[Wanpeng Li]

From: Wanpeng Li <wanpengli@tencent.com>

As commit 0c5f81dad46 (KVM: LAPIC: Inject timer interrupt via posted interrupt) 
mentioned that the host admin should well tune the guest setup, so that vCPUs 
are placed on isolated pCPUs, and with several pCPUs surplus for *busy* housekeeping.
It is better to disable mwait/hlt/pause vmexits to keep the vCPUs in non-root 
mode. However, we may isolate pCPUs for other purpose like DPDK or we can make 
some guests isolated and others not, we may lose vmx preemption timer/timer fastpath 
due to not well tuned setup, and the checking in kvm_can_post_timer_interrupt() 
is not enough. Let's guarantee mwait/hlt is advertised before enabling posted-interrupt 
interrupt. vmx preemption timer/timer fastpath can continue to work if both of them 
are not advertised.

Reported-by: Aili Yao <yaoaili@kingsoft.com>
Reviewed-by: Sean Christopherson <seanjc@google.com>
Cc: Aili Yao <yaoaili@kingsoft.com>
Cc: Sean Christopherson <seanjc@google.com>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
---
v1 -> v2:
 * also check kvm_hlt_in_guest since sometime mwait is disabled on host

 arch/x86/kvm/lapic.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index f206fc3..fdb7c81 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -113,7 +113,8 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)
 
 static bool kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu)
 {
-	return pi_inject_timer && kvm_vcpu_apicv_active(vcpu);
+	return pi_inject_timer && kvm_vcpu_apicv_active(vcpu) &&
+		(kvm_mwait_in_guest(vcpu->kvm) || kvm_hlt_in_guest(vcpu->kvm));
 }
 
 bool kvm_can_use_hv_timer(struct kvm_vcpu *vcpu)
-- 
2.7.4

Re: [PATCH RESEND v2] KVM: LAPIC: Enable timer posted-interrupt when mwait/hlt is advertised

[Paolo Bonzini]

On 1/25/22 13:08, Wanpeng Li wrote:
> As commit 0c5f81dad46 (KVM: LAPIC: Inject timer interrupt via posted interrupt)
> mentioned that the host admin should well tune the guest setup, so that vCPUs
> are placed on isolated pCPUs, and with several pCPUs surplus for*busy*  housekeeping.
> It is better to disable mwait/hlt/pause vmexits to keep the vCPUs in non-root
> mode. However, we may isolate pCPUs for other purpose like DPDK or we can make
> some guests isolated and others not, we may lose vmx preemption timer/timer fastpath
> due to not well tuned setup, and the checking in kvm_can_post_timer_interrupt()
> is not enough. Let's guarantee mwait/hlt is advertised before enabling posted-interrupt
> interrupt. vmx preemption timer/timer fastpath can continue to work if both of them
> are not advertised.

Is this the same thing that you meant?

--------
As commit 0c5f81dad46 ("KVM: LAPIC: Inject timer interrupt via posted interrupt")
mentioned that the host admin should well tune the guest setup, so that vCPUs
are placed on isolated pCPUs, and with several pCPUs surplus for *busy* housekeeping.
In this setup, it is preferrable to disable mwait/hlt/pause vmexits to
keep the vCPUs in non-root mode.

However, if only some guests isolated and others not, they would not have
any benefit from posted timer interrupts, and at the same time lose
VMX preemption timer fast paths because kvm_can_post_timer_interrupt()
returns true and therefore forces kvm_can_use_hv_timer() to false.

By guaranteeing that posted-interrupt timer is only used if MWAIT or HLT
are done without vmexit, KVM can make a better choice and use the
VMX preemption timer and the corresponding fast paths.
--------

Thanks,

Paolo

Re: [PATCH RESEND v2] KVM: LAPIC: Enable timer posted-interrupt when mwait/hlt is advertised

[Wanpeng Li]

On Tue, 1 Feb 2022 at 20:11, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 1/25/22 13:08, Wanpeng Li wrote:
> > As commit 0c5f81dad46 (KVM: LAPIC: Inject timer interrupt via posted interrupt)
> > mentioned that the host admin should well tune the guest setup, so that vCPUs
> > are placed on isolated pCPUs, and with several pCPUs surplus for*busy*  housekeeping.
> > It is better to disable mwait/hlt/pause vmexits to keep the vCPUs in non-root
> > mode. However, we may isolate pCPUs for other purpose like DPDK or we can make
> > some guests isolated and others not, we may lose vmx preemption timer/timer fastpath
> > due to not well tuned setup, and the checking in kvm_can_post_timer_interrupt()
> > is not enough. Let's guarantee mwait/hlt is advertised before enabling posted-interrupt
> > interrupt. vmx preemption timer/timer fastpath can continue to work if both of them
> > are not advertised.
>
> Is this the same thing that you meant?
>
> --------
> As commit 0c5f81dad46 ("KVM: LAPIC: Inject timer interrupt via posted interrupt")
> mentioned that the host admin should well tune the guest setup, so that vCPUs
> are placed on isolated pCPUs, and with several pCPUs surplus for *busy* housekeeping.
> In this setup, it is preferrable to disable mwait/hlt/pause vmexits to
> keep the vCPUs in non-root mode.
>
> However, if only some guests isolated and others not, they would not have
> any benefit from posted timer interrupts, and at the same time lose
> VMX preemption timer fast paths because kvm_can_post_timer_interrupt()
> returns true and therefore forces kvm_can_use_hv_timer() to false.
>
> By guaranteeing that posted-interrupt timer is only used if MWAIT or HLT
> are done without vmexit, KVM can make a better choice and use the
> VMX preemption timer and the corresponding fast paths.
> --------

Looks better, thanks Paolo! :)

    Wanpeng

Re: [PATCH RESEND v2] KVM: LAPIC: Enable timer posted-interrupt when mwait/hlt is advertised

[Paolo Bonzini]

On 2/1/22 14:57, Wanpeng Li wrote:
> On Tue, 1 Feb 2022 at 20:11, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>> On 1/25/22 13:08, Wanpeng Li wrote:
>>> As commit 0c5f81dad46 (KVM: LAPIC: Inject timer interrupt via posted interrupt)
>>> mentioned that the host admin should well tune the guest setup, so that vCPUs
>>> are placed on isolated pCPUs, and with several pCPUs surplus for*busy*  housekeeping.
>>> It is better to disable mwait/hlt/pause vmexits to keep the vCPUs in non-root
>>> mode. However, we may isolate pCPUs for other purpose like DPDK or we can make
>>> some guests isolated and others not, we may lose vmx preemption timer/timer fastpath
>>> due to not well tuned setup, and the checking in kvm_can_post_timer_interrupt()
>>> is not enough. Let's guarantee mwait/hlt is advertised before enabling posted-interrupt
>>> interrupt. vmx preemption timer/timer fastpath can continue to work if both of them
>>> are not advertised.
>>
>> Is this the same thing that you meant?
>>
>> --------
>> As commit 0c5f81dad46 ("KVM: LAPIC: Inject timer interrupt via posted interrupt")
>> mentioned that the host admin should well tune the guest setup, so that vCPUs
>> are placed on isolated pCPUs, and with several pCPUs surplus for *busy* housekeeping.
>> In this setup, it is preferrable to disable mwait/hlt/pause vmexits to
>> keep the vCPUs in non-root mode.
>>
>> However, if only some guests isolated and others not, they would not have
>> any benefit from posted timer interrupts, and at the same time lose
>> VMX preemption timer fast paths because kvm_can_post_timer_interrupt()
>> returns true and therefore forces kvm_can_use_hv_timer() to false.
>>
>> By guaranteeing that posted-interrupt timer is only used if MWAIT or HLT
>> are done without vmexit, KVM can make a better choice and use the
>> VMX preemption timer and the corresponding fast paths.
>> --------
> 
> Looks better, thanks Paolo! :)

Queued then, thanks!

Paolo