linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2] KVM: LAPIC: Fix lapic timer injection delay
@ 2017-06-23 11:59 Wanpeng Li
  2017-06-27 16:10 ` Paolo Bonzini
  0 siblings, 1 reply; 4+ messages in thread
From: Wanpeng Li @ 2017-06-23 11:59 UTC (permalink / raw)
  To: linux-kernel, kvm; +Cc: Paolo Bonzini, Radim Krčmář, Wanpeng Li

From: Wanpeng Li <wanpeng.li@hotmail.com>

Linux as a guest will prevent to program the next event to the clock event 
device when the event is in the past. However, it is not guaranteed by all 
the guests, the guest like kvm-unit-tests/apic.flat will write current tsc 
value visible in guest to MSR_IA32_TSCDEADLINE, lapic timer which is emulated 
by vmx preemption timer will program the absolute target tsc value to vmcs 
preemption timer field w/ delta == 0, then plays a vmentry and an upcoming 
vmx preemption timer fire vmexit dance, the lapic timer injection is delayed 
for this duration. Actually the lapic timer which is emulated by hrtimer can 
handle this correctly.

This patch fixes it by firing the lapic timer and injecting a timer interrupt 
immediately during the next vmentry if guest programs an expired timer to 
the emulated timer device. This saves ~300 cycles on the tsc_deadline_timer
test of apic.flat.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
---
 arch/x86/kvm/lapic.c | 7 ++++++-
 arch/x86/kvm/vmx.c   | 3 +++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index d24c874..08cf73d 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -1504,12 +1504,17 @@ static void cancel_hv_timer(struct kvm_lapic *apic)
 static bool start_hv_timer(struct kvm_lapic *apic)
 {
 	u64 tscdeadline = apic->lapic_timer.tscdeadline;
+	int ret = 0;
 
 	if ((atomic_read(&apic->lapic_timer.pending) &&
 		!apic_lvtt_period(apic)) ||
-		kvm_x86_ops->set_hv_timer(apic->vcpu, tscdeadline)) {
+		(ret = kvm_x86_ops->set_hv_timer(apic->vcpu, tscdeadline))) {
 		if (apic->lapic_timer.hv_timer_in_use)
 			cancel_hv_timer(apic);
+		if (ret == 1) {
+			apic_timer_expired(apic);
+			return true;
+		}
 	} else {
 		apic->lapic_timer.hv_timer_in_use = true;
 		hrtimer_cancel(&apic->lapic_timer.timer);
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 2e906cf..2008e9b 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -11149,6 +11149,9 @@ static int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc)
 	u64 guest_tscl = kvm_read_l1_tsc(vcpu, tscl);
 	u64 delta_tsc = max(guest_deadline_tsc, guest_tscl) - guest_tscl;
 
+	if (delta_tsc == 0)
+		return 1;
+
 	/* Convert to host delta tsc if tsc scaling is enabled */
 	if (vcpu->arch.tsc_scaling_ratio != kvm_default_tsc_scaling_ratio &&
 			u64_shl_div_u64(delta_tsc,
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [PATCH v2] KVM: LAPIC: Fix lapic timer injection delay
  2017-06-23 11:59 [PATCH v2] KVM: LAPIC: Fix lapic timer injection delay Wanpeng Li
@ 2017-06-27 16:10 ` Paolo Bonzini
  2017-06-27 22:09   ` Wanpeng Li
  0 siblings, 1 reply; 4+ messages in thread
From: Paolo Bonzini @ 2017-06-27 16:10 UTC (permalink / raw)
  To: Wanpeng Li, linux-kernel, kvm; +Cc: Radim Krčmář, Wanpeng Li



On 23/06/2017 13:59, Wanpeng Li wrote:
> From: Wanpeng Li <wanpeng.li@hotmail.com>
> 
> Linux as a guest will prevent to program the next event to the clock event 
> device when the event is in the past. However, it is not guaranteed by all 
> the guests, the guest like kvm-unit-tests/apic.flat will write current tsc 
> value visible in guest to MSR_IA32_TSCDEADLINE, lapic timer which is emulated 
> by vmx preemption timer will program the absolute target tsc value to vmcs 
> preemption timer field w/ delta == 0, then plays a vmentry and an upcoming 
> vmx preemption timer fire vmexit dance, the lapic timer injection is delayed 
> for this duration. Actually the lapic timer which is emulated by hrtimer can 
> handle this correctly.
> 
> This patch fixes it by firing the lapic timer and injecting a timer interrupt 
> immediately during the next vmentry if guest programs an expired timer to 
> the emulated timer device. This saves ~300 cycles on the tsc_deadline_timer
> test of apic.flat.

If it doesn't matter for actual guests, I'd rather not complicate the logic.

Paolo

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH v2] KVM: LAPIC: Fix lapic timer injection delay
  2017-06-27 16:10 ` Paolo Bonzini
@ 2017-06-27 22:09   ` Wanpeng Li
  2017-06-27 22:10     ` Wanpeng Li
  0 siblings, 1 reply; 4+ messages in thread
From: Wanpeng Li @ 2017-06-27 22:09 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: linux-kernel, kvm, Radim Krčmář, Wanpeng Li

2017-06-28 0:10 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>:
>
>
> On 23/06/2017 13:59, Wanpeng Li wrote:
>> From: Wanpeng Li <wanpeng.li@hotmail.com>
>>
>> Linux as a guest will prevent to program the next event to the clock event
>> device when the event is in the past. However, it is not guaranteed by all
>> the guests, the guest like kvm-unit-tests/apic.flat will write current tsc
>> value visible in guest to MSR_IA32_TSCDEADLINE, lapic timer which is emulated
>> by vmx preemption timer will program the absolute target tsc value to vmcs
>> preemption timer field w/ delta == 0, then plays a vmentry and an upcoming
>> vmx preemption timer fire vmexit dance, the lapic timer injection is delayed
>> for this duration. Actually the lapic timer which is emulated by hrtimer can
>> handle this correctly.
>>
>> This patch fixes it by firing the lapic timer and injecting a timer interrupt
>> immediately during the next vmentry if guest programs an expired timer to
>> the emulated timer device. This saves ~300 cycles on the tsc_deadline_timer
>> test of apic.flat.
>
> If it doesn't matter for actual guests, I'd rather not complicate the logic.

Sorry, maybe my description is not that accurate. It can also benefit
if the tsc deadline timer is programmed really close to the deadline.
That's why we introduce commit 9175d2e97b ("KVM: vmx: fix underflow in
TSC deadline calculation"). In addition, I capture there is frequently
programmed close to the deadline in vmx_set_hv_timer(delta_tsc == 0):

 qemu-system-x86-2352  [003] ....   457.709435: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [004] ....   457.941090: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [005] ....   458.381340: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [005] ....   458.381565: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [005] ....   458.382079: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [005] ....   458.382277: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [005] ....   458.382576: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [005] ....   458.383065: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [005] ....   458.565506: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [005] ....   458.565718: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [005] ....   458.566199: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [005] ....   458.566431: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [005] ....   458.566680: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [005] ....   458.567154: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [006] ....   458.989358: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [006] ....   458.989554: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [006] ....   458.990075: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [003] ....   459.397456: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [003] ....   459.397618: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [003] ....   459.397877: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [003] ....   459.398091: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [003] ....   459.398307: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [003] ....   459.398672: vmx_set_hv_timer: expired timer
 qemu-system-x86-2352  [003] ....   459.502826: vmx_set_hv_timer: expired timer

Regards,
Wanpeng Li

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH v2] KVM: LAPIC: Fix lapic timer injection delay
  2017-06-27 22:09   ` Wanpeng Li
@ 2017-06-27 22:10     ` Wanpeng Li
  0 siblings, 0 replies; 4+ messages in thread
From: Wanpeng Li @ 2017-06-27 22:10 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: linux-kernel, kvm, Radim Krčmář, Wanpeng Li

2017-06-28 6:09 GMT+08:00 Wanpeng Li <kernellwp@gmail.com>:
> 2017-06-28 0:10 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>:
>>
>>
>> On 23/06/2017 13:59, Wanpeng Li wrote:
>>> From: Wanpeng Li <wanpeng.li@hotmail.com>
>>>
>>> Linux as a guest will prevent to program the next event to the clock event
>>> device when the event is in the past. However, it is not guaranteed by all
>>> the guests, the guest like kvm-unit-tests/apic.flat will write current tsc
>>> value visible in guest to MSR_IA32_TSCDEADLINE, lapic timer which is emulated
>>> by vmx preemption timer will program the absolute target tsc value to vmcs
>>> preemption timer field w/ delta == 0, then plays a vmentry and an upcoming
>>> vmx preemption timer fire vmexit dance, the lapic timer injection is delayed
>>> for this duration. Actually the lapic timer which is emulated by hrtimer can
>>> handle this correctly.
>>>
>>> This patch fixes it by firing the lapic timer and injecting a timer interrupt
>>> immediately during the next vmentry if guest programs an expired timer to
>>> the emulated timer device. This saves ~300 cycles on the tsc_deadline_timer
>>> test of apic.flat.
>>
>> If it doesn't matter for actual guests, I'd rather not complicate the logic.
>
> Sorry, maybe my description is not that accurate. It can also benefit
> if the tsc deadline timer is programmed really close to the deadline.
> That's why we introduce commit 9175d2e97b ("KVM: vmx: fix underflow in
> TSC deadline calculation"). In addition, I capture there is frequently
> programmed close to the deadline in vmx_set_hv_timer(delta_tsc == 0):

I test a linux guest.

>
>  qemu-system-x86-2352  [003] ....   457.709435: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [004] ....   457.941090: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [005] ....   458.381340: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [005] ....   458.381565: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [005] ....   458.382079: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [005] ....   458.382277: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [005] ....   458.382576: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [005] ....   458.383065: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [005] ....   458.565506: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [005] ....   458.565718: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [005] ....   458.566199: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [005] ....   458.566431: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [005] ....   458.566680: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [005] ....   458.567154: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [006] ....   458.989358: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [006] ....   458.989554: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [006] ....   458.990075: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [003] ....   459.397456: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [003] ....   459.397618: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [003] ....   459.397877: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [003] ....   459.398091: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [003] ....   459.398307: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [003] ....   459.398672: vmx_set_hv_timer: expired timer
>  qemu-system-x86-2352  [003] ....   459.502826: vmx_set_hv_timer: expired timer
>
> Regards,
> Wanpeng Li

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2017-06-27 22:10 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-06-23 11:59 [PATCH v2] KVM: LAPIC: Fix lapic timer injection delay Wanpeng Li
2017-06-27 16:10 ` Paolo Bonzini
2017-06-27 22:09   ` Wanpeng Li
2017-06-27 22:10     ` Wanpeng Li

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).