From: Wanpeng Li <kernellwp@gmail.com>
To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
"Radim Krčmář" <rkrcmar@redhat.com>,
"Marcelo Tosatti" <mtosatti@redhat.com>
Subject: [PATCH v4 0/5] KVM: LAPIC: Implement Exitless Timer
Date: Mon, 17 Jun 2019 19:24:42 +0800 [thread overview]
Message-ID: <1560770687-23227-1-git-send-email-wanpengli@tencent.com> (raw)
Dedicated instances are currently disturbed by unnecessary jitter due
to the emulated lapic timers fire on the same pCPUs which vCPUs resident.
There is no hardware virtual timer on Intel for guest like ARM. Both
programming timer in guest and the emulated timer fires incur vmexits.
This patchset tries to avoid vmexit which is incurred by the emulated
timer fires in dedicated instance scenario.
When nohz_full is enabled in dedicated instances scenario, the unpinned
timer will be moved to the nearest busy housekeepers after commit
9642d18eee2cd (nohz: Affine unpinned timers to housekeepers) and commit
444969223c8 ("sched/nohz: Fix affine unpinned timers mess"). However,
KVM always makes lapic timer pinned to the pCPU which vCPU residents, the
reason is explained by commit 61abdbe0 (kvm: x86: make lapic hrtimer
pinned). Actually, these emulated timers can be offload to the housekeeping
cpus since APICv is really common in recent years. The guest timer interrupt
is injected by posted-interrupt which is delivered by housekeeping cpu
once the emulated timer fires.
The host admin should fine tuned, e.g. dedicated instances scenario w/
nohz_full cover the pCPUs which vCPUs resident, several pCPUs surplus
for busy housekeeping, disable mwait/hlt/pause vmexits to keep in non-root
mode, ~3% redis performance benefit can be observed on Skylake server.
w/o patchset:
VM-EXIT Samples Samples% Time% Min Time Max Time Avg time
EXTERNAL_INTERRUPT 42916 49.43% 39.30% 0.47us 106.09us 0.71us ( +- 1.09% )
w/ patchset:
VM-EXIT Samples Samples% Time% Min Time Max Time Avg time
EXTERNAL_INTERRUPT 6871 9.29% 2.96% 0.44us 57.88us 0.72us ( +- 4.02% )
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Marcelo Tosatti <mtosatti@redhat.com>
v3 -> v4:
* drop the HRTIMER_MODE_ABS_PINNED, add kick after set pending timer
* don't posted inject already-expired timer
v2 -> v3:
* disarming the vmx preemption timer when posted_interrupt_inject_timer_enabled()
* check kvm_hlt_in_guest instead
v1 -> v2:
* check vcpu_halt_in_guest
* move module parameter from kvm-intel to kvm
* add housekeeping_enabled
* rename apic_timer_expired_pi to kvm_apic_inject_pending_timer_irqs
Wanpeng Li (5):
KVM: LAPIC: Make lapic timer unpinned
KVM: LAPIC: inject lapic timer interrupt by posted interrupt
KVM: LAPIC: Ignore timer migration when lapic timer is injected by pi
KVM: LAPIC: Don't posted inject already-expired timer
KVM: LAPIC: add advance timer support to pi_inject_timer
arch/x86/kvm/lapic.c | 62 ++++++++++++++++++++++++++++-------------
arch/x86/kvm/lapic.h | 3 +-
arch/x86/kvm/svm.c | 2 +-
arch/x86/kvm/vmx/vmx.c | 5 ++--
arch/x86/kvm/x86.c | 11 ++++----
arch/x86/kvm/x86.h | 2 ++
include/linux/sched/isolation.h | 2 ++
kernel/sched/isolation.c | 6 ++++
8 files changed, 64 insertions(+), 29 deletions(-)
--
2.7.4
next reply other threads:[~2019-06-17 11:25 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-17 11:24 Wanpeng Li [this message]
2019-06-17 11:24 ` [PATCH v4 1/5] KVM: LAPIC: Make lapic timer unpinned Wanpeng Li
2019-06-17 11:48 ` Peter Xu
2019-06-18 0:38 ` Wanpeng Li
2019-06-17 11:24 ` [PATCH v4 2/5] KVM: LAPIC: inject lapic timer interrupt by posted interrupt Wanpeng Li
2019-06-18 13:35 ` Marcelo Tosatti
2019-06-19 0:36 ` Wanpeng Li
2019-06-19 21:03 ` Marcelo Tosatti
2019-06-20 0:52 ` Wanpeng Li
2019-06-21 1:42 ` Wanpeng Li
2019-06-21 21:42 ` Marcelo Tosatti
2019-06-24 8:53 ` Wanpeng Li
2019-06-25 19:00 ` Marcelo Tosatti
2019-06-26 11:02 ` Wanpeng Li
2019-06-26 16:44 ` Marcelo Tosatti
2019-06-28 8:26 ` Wanpeng Li
2019-06-25 17:02 ` Paolo Bonzini
2019-06-17 11:24 ` [PATCH v4 3/5] KVM: LAPIC: Ignore timer migration when lapic timer is injected by pi Wanpeng Li
2019-06-17 11:24 ` [PATCH v4 4/5] KVM: LAPIC: Don't posted inject already-expired timer Wanpeng Li
2019-06-17 11:24 ` [PATCH v4 5/5] KVM: LAPIC: add advance timer support to pi_inject_timer Wanpeng Li
2019-06-17 21:32 ` Radim Krčmář
2019-06-18 0:44 ` Wanpeng Li
2019-06-18 0:57 ` Wanpeng Li
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1560770687-23227-1-git-send-email-wanpengli@tencent.com \
--to=kernellwp@gmail.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mtosatti@redhat.com \
--cc=pbonzini@redhat.com \
--cc=rkrcmar@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).