kvm.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Sean Christopherson <sean.j.christopherson@intel.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: Sean Christopherson <sean.j.christopherson@intel.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
	Ben Gardon <bgardon@google.com>,
	Junaid Shahid <junaids@google.com>,
	Liran Alon <liran.alon@oracle.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	John Haxby <john.haxby@oracle.com>,
	Miaohe Lin <linmiaohe@huawei.com>,
	Tom Lendacky <thomas.lendacky@amd.com>
Subject: [PATCH v2 23/32] KVM: nVMX: Add helper to handle TLB flushes on nested VM-Enter/VM-Exit
Date: Mon, 16 Mar 2020 21:52:29 -0700	[thread overview]
Message-ID: <20200317045238.30434-24-sean.j.christopherson@intel.com> (raw)
In-Reply-To: <20200317045238.30434-1-sean.j.christopherson@intel.com>

Add a helper to determine whether or not a full TLB flush needs to be
performed on nested VM-Enter/VM-Exit, as the logic is identical for both
flows and needs a fairly beefy comment to boot.  This also provides a
common point to make future adjustments to the logic.

Skip the INVVPID if a TLB flush is pending, which mostly preserves the
existing logic, but also skips INVVPID in the unlikely event that a TLB
flush was requested for some other reason.

Remove the explicit enable_vpid from prepare_vmcs02() as its implied by
nested_cpu_has_vpid(), which can return true if and only if VPID is
enabled in KVM (L0).

Cc: Liran Alon <liran.alon@oracle.com>
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
---
 arch/x86/kvm/vmx/nested.c | 83 +++++++++++++++++++--------------------
 1 file changed, 40 insertions(+), 43 deletions(-)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 960ecbab5ebe..a7cc41e69948 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -1154,6 +1154,31 @@ static bool nested_has_guest_tlb_tag(struct kvm_vcpu *vcpu)
 	       (nested_cpu_has_vpid(vmcs12) && to_vmx(vcpu)->nested.vpid02);
 }
 
+static void nested_vmx_transition_tlb_flush(struct kvm_vcpu *vcpu,
+					    struct vmcs12 *vmcs12)
+{
+	/*
+	 * If VPID is disabled, linear and combined mappings are flushed on
+	 * VM-Enter/VM-Exit, and guest-physical mappings are valid only for
+	 * their associated EPTP.
+	 *
+	 * If vmcs12 doesn't use VPID, L1 expects linear and combined mappings
+	 * for *all* contexts to be flushed on VM-Enter/VM-Exit.
+	 *
+	 * If VPID is enabled and used by vmc12, but L2 does not have a unique
+	 * TLB tag (ASID), i.e. EPT is disabled and KVM was unable to allocate
+	 * a VPID for L2, flush the TLB as the effective ASID is common to both
+	 * L1 and L2.
+	 *
+	 * Defer the flush so that it runs after vmcs02.EPTP has been set by
+	 * KVM_REQ_LOAD_MMU_PGD (if nested EPT is enabled) and to avoid
+	 * redundant flushes further down the nested pipeline.
+	 */
+	if (enable_vpid &&
+	    (!nested_cpu_has_vpid(vmcs12) || !nested_has_guest_tlb_tag(vcpu)))
+		kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu);
+}
+
 static bool is_bitwise_subset(u64 superset, u64 subset, u64 mask)
 {
 	superset &= mask;
@@ -2462,31 +2487,20 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,
 	if (kvm_has_tsc_control)
 		decache_tsc_multiplier(vmx);
 
-	if (enable_vpid) {
-		/*
-		 * There is no direct mapping between vpid02 and vpid12, the
-		 * vpid02 is per-vCPU for L0 and reused while the value of
-		 * vpid12 is changed w/ one invvpid during nested vmentry.
-		 * The vpid12 is allocated by L1 for L2, so it will not
-		 * influence global bitmap(for vpid01 and vpid02 allocation)
-		 * even if spawn a lot of nested vCPUs.
-		 */
-		if (nested_cpu_has_vpid(vmcs12) && nested_has_guest_tlb_tag(vcpu)) {
-			if (vmcs12->virtual_processor_id != vmx->nested.last_vpid) {
-				vmx->nested.last_vpid = vmcs12->virtual_processor_id;
-				vpid_sync_context(nested_get_vpid02(vcpu));
-			}
-		} else {
-			/*
-			 * If L1 use EPT, then L0 needs to execute INVEPT on
-			 * EPTP02 instead of EPTP01. Therefore, delay TLB
-			 * flush until vmcs02->eptp is fully updated by
-			 * KVM_REQ_LOAD_MMU_PGD. Note that this assumes
-			 * KVM_REQ_TLB_FLUSH is evaluated after
-			 * KVM_REQ_LOAD_MMU_PGD in vcpu_enter_guest().
-			 */
-			kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu);
-		}
+	nested_vmx_transition_tlb_flush(vcpu, vmcs12);
+
+	/*
+	 * There is no direct mapping between vpid02 and vpid12, vpid02 is
+	 * per-vCPU and reused for all nested vCPUs.  If vpid12 is changing
+	 * then the new "virtual" VPID will reuse the same "real" VPID,
+	 * vpid02, and so needs to be sync'd.  Skip the sync if a TLB flush
+	 * has already been requested, but always update the last used VPID.
+	 */
+	if (nested_cpu_has_vpid(vmcs12) && nested_has_guest_tlb_tag(vcpu) &&
+	    vmcs12->virtual_processor_id != vmx->nested.last_vpid) {
+		vmx->nested.last_vpid = vmcs12->virtual_processor_id;
+		if (!kvm_test_request(KVM_REQ_TLB_FLUSH, vcpu))
+			vpid_sync_context(nested_get_vpid02(vcpu));
 	}
 
 	if (nested_cpu_has_ept(vmcs12))
@@ -4054,24 +4068,7 @@ static void load_vmcs12_host_state(struct kvm_vcpu *vcpu,
 	if (!enable_ept)
 		vcpu->arch.walk_mmu->inject_page_fault = kvm_inject_page_fault;
 
-	/*
-	 * If vmcs01 doesn't use VPID, CPU flushes TLB on every
-	 * VMEntry/VMExit. Thus, no need to flush TLB.
-	 *
-	 * If vmcs12 doesn't use VPID, L1 expects TLB to be
-	 * flushed on every VMEntry/VMExit.
-	 *
-	 * Otherwise, we can preserve TLB entries as long as we are
-	 * able to tag L1 TLB entries differently than L2 TLB entries.
-	 *
-	 * If vmcs12 uses EPT, we need to execute this flush on EPTP01
-	 * and therefore we request the TLB flush to happen only after VMCS EPTP
-	 * has been set by KVM_REQ_LOAD_MMU_PGD.
-	 */
-	if (enable_vpid &&
-	    (!nested_cpu_has_vpid(vmcs12) || !nested_has_guest_tlb_tag(vcpu))) {
-		kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu);
-	}
+	nested_vmx_transition_tlb_flush(vcpu, vmcs12);
 
 	vmcs_write32(GUEST_SYSENTER_CS, vmcs12->host_ia32_sysenter_cs);
 	vmcs_writel(GUEST_SYSENTER_ESP, vmcs12->host_ia32_sysenter_esp);
-- 
2.24.1


  parent reply	other threads:[~2020-03-17  4:54 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-17  4:52 [PATCH v2 00/32] KVM: x86: TLB flushing fixes and enhancements Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 01/32] KVM: VMX: Flush all EPTP/VPID contexts on remote TLB flush Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 02/32] KVM: nVMX: Validate the EPTP when emulating INVEPT(EXTENT_CONTEXT) Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 03/32] KVM: nVMX: Invalidate all EPTP contexts when emulating INVEPT for L1 Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 04/32] KVM: nVMX: Invalidate all L2 roots when emulating INVVPID without EPT Sean Christopherson
2020-03-20  4:11   ` Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 05/32] KVM: x86: Export kvm_propagate_fault() (as kvm_inject_emulated_page_fault) Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 06/32] KVM: x86: Consolidate logic for injecting page faults to L1 Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 07/32] KVM: x86: Sync SPTEs when injecting page/EPT fault into L1 Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 08/32] KVM: VMX: Skip global INVVPID fallback if vpid==0 in vpid_sync_context() Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 09/32] KVM: VMX: Use vpid_sync_context() directly when possible Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 10/32] KVM: VMX: Move vpid_sync_vcpu_addr() down a few lines Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 11/32] KVM: VMX: Handle INVVPID fallback logic in vpid_sync_vcpu_addr() Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 12/32] KVM: VMX: Drop redundant capability checks in low level INVVPID helpers Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 13/32] KVM: nVMX: Use vpid_sync_vcpu_addr() to emulate INVVPID with address Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 14/32] KVM: x86: Move "flush guest's TLB" logic to separate kvm_x86_ops hook Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 15/32] KVM: VMX: Clean up vmx_flush_tlb_gva() Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 16/32] KVM: x86: Drop @invalidate_gpa param from kvm_x86_ops' tlb_flush() Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 17/32] KVM: SVM: Wire up ->tlb_flush_guest() directly to svm_flush_tlb() Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 18/32] KVM: VMX: Move vmx_flush_tlb() to vmx.c Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 19/32] KVM: nVMX: Move nested_get_vpid02() to vmx/nested.h Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 20/32] KVM: VMX: Introduce vmx_flush_tlb_current() Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 21/32] KVM: SVM: Document the ASID logic in svm_flush_tlb() Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 22/32] KVM: x86: Rename ->tlb_flush() to ->tlb_flush_all() Sean Christopherson
2020-03-17  4:52 ` Sean Christopherson [this message]
2020-03-17 17:17   ` [PATCH v2 23/32] KVM: nVMX: Add helper to handle TLB flushes on nested VM-Enter/VM-Exit Paolo Bonzini
2020-03-17 18:18     ` Sean Christopherson
2020-03-18 10:45       ` Paolo Bonzini
2020-03-18 16:09         ` Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 24/32] KVM: x86: Introduce KVM_REQ_TLB_FLUSH_CURRENT to flush current ASID Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 25/32] KVM: x86/mmu: Use KVM_REQ_TLB_FLUSH_CURRENT for MMU specific flushes Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 26/32] KVM: nVMX: Selectively use TLB_FLUSH_CURRENT for nested VM-Enter/VM-Exit Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 27/32] KVM: nVMX: Reload APIC access page on nested VM-Exit only if necessary Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 28/32] KVM: VMX: Retrieve APIC access page HPA only when necessary Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 29/32] KVM: VMX: Don't reload APIC access page if its control is disabled Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 30/32] KVM: x86/mmu: Add module param to force TLB flush on root reuse Sean Christopherson
2020-03-17  4:52 ` [PATCH v2 31/32] KVM: nVMX: Don't flush TLB on nested VM transition with EPT enabled Sean Christopherson
2020-03-17 17:18   ` Paolo Bonzini
2020-03-17 18:22     ` Sean Christopherson
2020-03-18 10:36       ` Paolo Bonzini
2020-03-18 17:02         ` Sean Christopherson
2020-03-18 17:11           ` Paolo Bonzini
2020-03-18 17:26             ` Sean Christopherson
2020-03-18 17:38               ` Paolo Bonzini
2020-03-17  4:52 ` [PATCH v2 32/32] KVM: nVMX: Free only the affected contexts when emulating INVEPT Sean Christopherson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200317045238.30434-24-sean.j.christopherson@intel.com \
    --to=sean.j.christopherson@intel.com \
    --cc=bgardon@google.com \
    --cc=boris.ostrovsky@oracle.com \
    --cc=jmattson@google.com \
    --cc=john.haxby@oracle.com \
    --cc=joro@8bytes.org \
    --cc=junaids@google.com \
    --cc=kvm@vger.kernel.org \
    --cc=linmiaohe@huawei.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=liran.alon@oracle.com \
    --cc=pbonzini@redhat.com \
    --cc=thomas.lendacky@amd.com \
    --cc=vkuznets@redhat.com \
    --cc=wanpengli@tencent.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).