From: Wanpeng Li <kernellwp@gmail.com>
To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Cc: "Paolo Bonzini" <pbonzini@redhat.com>,
"Radim Krčmář" <rkrcmar@redhat.com>,
"Vitaly Kuznetsov" <vkuznets@redhat.com>
Subject: [PATCH v3 4/6] KVM: X86: Implement PV IPIs send hypercall
Date: Tue, 3 Jul 2018 14:21:29 +0800 [thread overview]
Message-ID: <1530598891-21370-5-git-send-email-wanpengli@tencent.com> (raw)
In-Reply-To: <1530598891-21370-1-git-send-email-wanpengli@tencent.com>
From: Wanpeng Li <wanpengli@tencent.com>
Using hypercall to send IPIs by one vmexit instead of one by one for
xAPIC/x2APIC physical mode and one vmexit per-cluster for x2APIC cluster
mode. Intel guest can enter x2apic cluster mode when interrupt remmaping
is enabled in qemu, however, latest AMD EPYC still just supports xapic
mode which can get great improvement by PV IPIs.
Hardware: Xeon Skylake 2.5GHz, 2 sockets, 40 cores, 80 threads, the VM
is 80 vCPUs, IPI microbenchmark(https://lkml.org/lkml/2017/12/19/141):
x2apic cluster mode, vanilla
Dry-run: 0, 2392199 ns
Self-IPI: 6907514, 15027589 ns
Normal IPI: 223910476, 251301666 ns
Broadcast IPI: 0, 9282161150 ns
Broadcast lock: 0, 8812934104 ns
x2apic cluster mode, pv-ipi
Dry-run: 0, 2449341 ns
Self-IPI: 6720360, 15028732 ns
Normal IPI: 228643307, 255708477 ns
Broadcast IPI: 0, 7572293590 ns => 22% performance boost
Broadcast lock: 0, 8316124651 ns
x2apic physical mode, vanilla
Dry-run: 0, 3135933 ns
Self-IPI: 8572670, 17901757 ns
Normal IPI: 226444334, 255421709 ns
Broadcast IPI: 0, 19845070887 ns
Broadcast lock: 0, 19827383656 ns
x2apic physical mode, pv-ipi
Dry-run: 0, 2446381 ns
Self-IPI: 6788217, 15021056 ns
Normal IPI: 219454441, 249583458 ns
Broadcast IPI: 0, 7806540019 ns => 154% performance boost
Broadcast lock: 0, 9143618799 ns
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
---
Documentation/virtual/kvm/hypercalls.txt | 6 ++++++
arch/x86/kvm/x86.c | 36 ++++++++++++++++++++++++++++++++
2 files changed, 42 insertions(+)
diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virtual/kvm/hypercalls.txt
index a890529..a771ee8 100644
--- a/Documentation/virtual/kvm/hypercalls.txt
+++ b/Documentation/virtual/kvm/hypercalls.txt
@@ -121,3 +121,9 @@ compute the CLOCK_REALTIME for its clock, at the same instant.
Returns KVM_EOPNOTSUPP if the host does not use TSC clocksource,
or if clock type is different than KVM_CLOCK_PAIRING_WALLCLOCK.
+
+6. KVM_HC_SEND_IPI
+------------------------
+Architecture: x86
+Status: active
+Purpose: Hypercall used to send IPIs.
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 0046aa7..46657ef 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -6689,6 +6689,39 @@ static void kvm_pv_kick_cpu_op(struct kvm *kvm, unsigned long flags, int apicid)
kvm_irq_delivery_to_apic(kvm, NULL, &lapic_irq, NULL);
}
+/*
+ * Return 0 if successfully added and 1 if discarded.
+ */
+static int kvm_pv_send_ipi(struct kvm *kvm, unsigned long ipi_bitmap_low,
+ unsigned long ipi_bitmap_high, int vector)
+{
+ int i;
+ struct kvm_apic_map *map;
+ struct kvm_vcpu *vcpu;
+ struct kvm_lapic_irq irq = {
+ .delivery_mode = APIC_DM_FIXED,
+ .vector = vector,
+ };
+
+ rcu_read_lock();
+ map = rcu_dereference(kvm->arch.apic_map);
+
+ for_each_set_bit(i, &ipi_bitmap_low, BITS_PER_LONG) {
+ vcpu = map->phys_map[i]->vcpu;
+ if (!kvm_apic_set_irq(vcpu, &irq, NULL))
+ return 1;
+ }
+
+ for_each_set_bit(i, &ipi_bitmap_high, BITS_PER_LONG) {
+ vcpu = map->phys_map[i + BITS_PER_LONG]->vcpu;
+ if (!kvm_apic_set_irq(vcpu, &irq, NULL))
+ return 1;
+ }
+
+ rcu_read_unlock();
+ return 0;
+}
+
void kvm_vcpu_deactivate_apicv(struct kvm_vcpu *vcpu)
{
vcpu->arch.apicv_active = false;
@@ -6737,6 +6770,9 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
case KVM_HC_CLOCK_PAIRING:
ret = kvm_pv_clock_pairing(vcpu, a0, a1);
break;
+ case KVM_HC_SEND_IPI:
+ ret = kvm_pv_send_ipi(vcpu->kvm, a0, a1, a2);
+ break;
#endif
default:
ret = -KVM_ENOSYS;
--
2.7.4
next prev parent reply other threads:[~2018-07-03 6:21 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-03 6:21 [PATCH v3 0/6] KVM: X86: Implement PV IPIs support Wanpeng Li
2018-07-03 6:21 ` [PATCH v3 1/6] KVM: X86: Add kvm hypervisor init time platform setup callback Wanpeng Li
2018-07-03 6:21 ` [PATCH v3 2/6] KVM: X86: Implement PV IPIs in linux guest Wanpeng Li
2018-07-19 16:28 ` Radim Krčmář
2018-07-19 16:47 ` Paolo Bonzini
2018-07-19 17:22 ` Radim Krčmář
2018-07-20 3:35 ` Wanpeng Li
2018-07-20 5:58 ` Wanpeng Li
2018-07-20 8:06 ` Paolo Bonzini
2018-07-20 3:33 ` Wanpeng Li
2018-07-20 9:51 ` Radim Krcmar
2018-07-20 10:17 ` Wanpeng Li
2018-07-19 23:05 ` David Matlack
2018-07-20 3:45 ` Wanpeng Li
2018-07-20 13:12 ` Radim Krcmar
2018-07-03 6:21 ` [PATCH v3 3/6] KVM: X86: Fallback to original apic hooks when bad happens Wanpeng Li
2018-07-03 6:21 ` Wanpeng Li [this message]
2018-07-19 16:47 ` [PATCH v3 4/6] KVM: X86: Implement PV IPIs send hypercall Paolo Bonzini
2018-07-20 3:49 ` Wanpeng Li
2018-07-03 6:21 ` [PATCH v3 5/6] KVM: X86: Add NMI support to PV IPIs Wanpeng Li
2018-07-19 16:31 ` Radim Krčmář
2018-07-20 3:53 ` Wanpeng Li
2018-07-20 8:04 ` Paolo Bonzini
2018-07-20 13:26 ` Radim Krcmar
2018-07-23 0:52 ` Wanpeng Li
2018-07-03 6:21 ` [PATCH v3 6/6] KVM: X86: Expose PV_SEND_IPI CPUID feature bit to guest Wanpeng Li
2018-07-18 3:00 ` [PATCH v3 0/6] KVM: X86: Implement PV IPIs support Wanpeng Li
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1530598891-21370-5-git-send-email-wanpengli@tencent.com \
--to=kernellwp@gmail.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=rkrcmar@redhat.com \
--cc=vkuznets@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).