From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: kvm@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>,
Wanpeng Li <wanpengli@tencent.com>,
Jim Mattson <jmattson@google.com>,
"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
Nitesh Narayan Lal <nitesh@redhat.com>,
linux-kernel@vger.kernel.org
Subject: [PATCH v2 3/4] KVM: Optimize kvm_make_vcpus_request_mask() a bit
Date: Mon, 23 Aug 2021 16:30:27 +0200 [thread overview]
Message-ID: <20210823143028.649818-4-vkuznets@redhat.com> (raw)
In-Reply-To: <20210823143028.649818-1-vkuznets@redhat.com>
Iterating over set bits in 'vcpu_bitmap' should be faster than going
through all vCPUs, especially when just a few bits are set.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
virt/kvm/kvm_main.c | 83 ++++++++++++++++++++++++++-------------------
1 file changed, 49 insertions(+), 34 deletions(-)
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 82c5280dd5ce..e9c2ce2d273f 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -257,49 +257,64 @@ static inline bool kvm_kick_many_cpus(const struct cpumask *cpus, bool wait)
return true;
}
+static void kvm_make_vcpu_request(struct kvm *kvm, struct kvm_vcpu *vcpu,
+ unsigned int req, cpumask_var_t tmp,
+ int current_cpu)
+{
+ int cpu = vcpu->cpu;
+
+ kvm_make_request(req, vcpu);
+
+ if (!(req & KVM_REQUEST_NO_WAKEUP) && kvm_vcpu_wake_up(vcpu))
+ return;
+
+ /*
+ * tmp can be NULL if cpumasks are allocated off stack, as allocation of
+ * the mask is deliberately not fatal and is handled by falling back to
+ * kicking all online CPUs.
+ */
+ if (IS_ENABLED(CONFIG_CPUMASK_OFFSTACK) && !tmp)
+ return;
+
+ /*
+ * Note, the vCPU could get migrated to a different pCPU at any point
+ * after kvm_request_needs_ipi(), which could result in sending an IPI
+ * to the previous pCPU. But, that's ok because the purpose of the IPI
+ * is to ensure the vCPU returns to OUTSIDE_GUEST_MODE, which is
+ * satisfied if the vCPU migrates. Entering READING_SHADOW_PAGE_TABLES
+ * after this point is also ok, as the requirement is only that KVM wait
+ * for vCPUs that were reading SPTEs _before_ any changes were
+ * finalized. See kvm_vcpu_kick() for more details on handling requests.
+ */
+ if (kvm_request_needs_ipi(vcpu, req)) {
+ cpu = READ_ONCE(vcpu->cpu);
+ if (cpu != -1 && cpu != current_cpu)
+ __cpumask_set_cpu(cpu, tmp);
+ }
+}
+
bool kvm_make_vcpus_request_mask(struct kvm *kvm, unsigned int req,
struct kvm_vcpu *except,
unsigned long *vcpu_bitmap, cpumask_var_t tmp)
{
- int i, cpu, me;
+ int i, me;
struct kvm_vcpu *vcpu;
bool called;
me = get_cpu();
- kvm_for_each_vcpu(i, vcpu, kvm) {
- if ((vcpu_bitmap && !test_bit(i, vcpu_bitmap)) ||
- vcpu == except)
- continue;
-
- kvm_make_request(req, vcpu);
-
- if (!(req & KVM_REQUEST_NO_WAKEUP) && kvm_vcpu_wake_up(vcpu))
- continue;
-
- /*
- * tmp can be NULL if cpumasks are allocated off stack, as
- * allocation of the mask is deliberately not fatal and is
- * handled by falling back to kicking all online CPUs.
- */
- if (IS_ENABLED(CONFIG_CPUMASK_OFFSTACK) && !tmp)
- continue;
-
- /*
- * Note, the vCPU could get migrated to a different pCPU at any
- * point after kvm_request_needs_ipi(), which could result in
- * sending an IPI to the previous pCPU. But, that's ok because
- * the purpose of the IPI is to ensure the vCPU returns to
- * OUTSIDE_GUEST_MODE, which is satisfied if the vCPU migrates.
- * Entering READING_SHADOW_PAGE_TABLES after this point is also
- * ok, as the requirement is only that KVM wait for vCPUs that
- * were reading SPTEs _before_ any changes were finalized. See
- * kvm_vcpu_kick() for more details on handling requests.
- */
- if (kvm_request_needs_ipi(vcpu, req)) {
- cpu = READ_ONCE(vcpu->cpu);
- if (cpu != -1 && cpu != me)
- __cpumask_set_cpu(cpu, tmp);
+ if (vcpu_bitmap) {
+ for_each_set_bit(i, vcpu_bitmap, KVM_MAX_VCPUS) {
+ vcpu = kvm_get_vcpu(kvm, i);
+ if (!vcpu || vcpu == except)
+ continue;
+ kvm_make_vcpu_request(kvm, vcpu, req, tmp, me);
+ }
+ } else {
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (vcpu == except)
+ continue;
+ kvm_make_vcpu_request(kvm, vcpu, req, tmp, me);
}
}
--
2.31.1
next prev parent reply other threads:[~2021-08-23 14:30 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-23 14:30 [PATCH v2 0/4] KVM: Various fixes and improvements around kicking vCPUs Vitaly Kuznetsov
2021-08-23 14:30 ` [PATCH v2 1/4] KVM: Clean up benign vcpu->cpu data races when " Vitaly Kuznetsov
2021-08-23 14:30 ` [PATCH v2 2/4] KVM: Guard cpusmask NULL check with CONFIG_CPUMASK_OFFSTACK Vitaly Kuznetsov
2021-08-23 14:30 ` Vitaly Kuznetsov [this message]
2021-08-23 14:30 ` [PATCH v2 4/4] KVM: x86: Fix stack-out-of-bounds memory access from ioapic_write_indirect() Vitaly Kuznetsov
2021-08-23 18:58 ` Eduardo Habkost
2021-08-24 7:13 ` Vitaly Kuznetsov
2021-08-24 14:23 ` Eduardo Habkost
2021-08-24 14:42 ` Vitaly Kuznetsov
2021-08-24 16:07 ` Maxim Levitsky
2021-08-24 17:40 ` Sean Christopherson
2021-08-25 8:26 ` Vitaly Kuznetsov
2021-08-25 8:21 ` Vitaly Kuznetsov
2021-08-25 9:11 ` Maxim Levitsky
2021-08-25 9:43 ` Vitaly Kuznetsov
2021-08-25 10:41 ` Maxim Levitsky
2021-08-25 13:19 ` Eduardo Habkost
2021-08-26 12:40 ` Vitaly Kuznetsov
2021-08-26 14:52 ` Eduardo Habkost
2021-08-26 18:01 ` Sean Christopherson
2021-08-26 18:13 ` Eduardo Habkost
2021-08-26 19:27 ` Eduardo Habkost
2021-08-30 19:47 ` Nitesh Lal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210823143028.649818-4-vkuznets@redhat.com \
--to=vkuznets@redhat.com \
--cc=dgilbert@redhat.com \
--cc=jmattson@google.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nitesh@redhat.com \
--cc=pbonzini@redhat.com \
--cc=seanjc@google.com \
--cc=wanpengli@tencent.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).