From: Ben Gardon <bgardon@google.com>
To: Sean Christopherson <seanjc@google.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
Vitaly Kuznetsov <vkuznets@redhat.com>,
Wanpeng Li <wanpengli@tencent.com>,
Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
kvm <kvm@vger.kernel.org>, LKML <linux-kernel@vger.kernel.org>,
Brijesh Singh <brijesh.singh@amd.com>,
Tom Lendacky <thomas.lendacky@amd.com>
Subject: Re: [PATCH 03/15] KVM: x86/mmu: Ensure MMU pages are available when allocating roots
Date: Tue, 2 Mar 2021 16:21:46 -0800 [thread overview]
Message-ID: <CANgfPd95G-01ObzeKeMTUu0yXBJ=0+ZGQwr_5WCNH-NmR03f9w@mail.gmail.com> (raw)
In-Reply-To: <20210302184540.2829328-4-seanjc@google.com>
On Tue, Mar 2, 2021 at 10:46 AM Sean Christopherson <seanjc@google.com> wrote:
>
> Hold the mmu_lock for write for the entire duration of allocating and
> initializing an MMU's roots. This ensures there are MMU pages available
> and thus prevents root allocations from failing. That in turn fixes a
> bug where KVM would fail to free valid PAE roots if a one of the later
> roots failed to allocate.
>
> Note, KVM still leaks the PAE roots if the lm_root allocation fails.
> This will be addressed in a future commit.
>
> Cc: Ben Gardon <bgardon@google.com>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Ben Gardon <bgardon@google.com>
Very tidy cleanup!
> ---
> arch/x86/kvm/mmu/mmu.c | 41 ++++++++++++--------------------------
> arch/x86/kvm/mmu/tdp_mmu.c | 23 +++++----------------
> 2 files changed, 18 insertions(+), 46 deletions(-)
>
> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> index 2ed3fac1244e..1f129001a30c 100644
> --- a/arch/x86/kvm/mmu/mmu.c
> +++ b/arch/x86/kvm/mmu/mmu.c
> @@ -2398,6 +2398,9 @@ static int make_mmu_pages_available(struct kvm_vcpu *vcpu)
> {
> unsigned long avail = kvm_mmu_available_pages(vcpu->kvm);
>
> + /* Ensure all four PAE roots can be allocated in a single pass. */
> + BUILD_BUG_ON(KVM_MIN_FREE_MMU_PAGES < 4);
> +
For a second I thought that this should be 5 since a page is needed to
hold the 4 PAE roots, but that page is allocated at vCPU creation and
reused, so no need to check for it here.
> if (likely(avail >= KVM_MIN_FREE_MMU_PAGES))
> return 0;
>
> @@ -3220,16 +3223,9 @@ static hpa_t mmu_alloc_root(struct kvm_vcpu *vcpu, gfn_t gfn, gva_t gva,
> {
> struct kvm_mmu_page *sp;
>
> - write_lock(&vcpu->kvm->mmu_lock);
> -
> - if (make_mmu_pages_available(vcpu)) {
> - write_unlock(&vcpu->kvm->mmu_lock);
> - return INVALID_PAGE;
> - }
> sp = kvm_mmu_get_page(vcpu, gfn, gva, level, direct, ACC_ALL);
> ++sp->root_count;
>
> - write_unlock(&vcpu->kvm->mmu_lock);
> return __pa(sp->spt);
> }
>
> @@ -3241,16 +3237,10 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *vcpu)
>
> if (is_tdp_mmu_enabled(vcpu->kvm)) {
> root = kvm_tdp_mmu_get_vcpu_root_hpa(vcpu);
> -
> - if (!VALID_PAGE(root))
> - return -ENOSPC;
> vcpu->arch.mmu->root_hpa = root;
> } else if (shadow_root_level >= PT64_ROOT_4LEVEL) {
> root = mmu_alloc_root(vcpu, 0, 0, shadow_root_level,
> true);
> -
> - if (!VALID_PAGE(root))
> - return -ENOSPC;
There's so much going on in mmu_alloc_root that removing this check
makes me nervous, but I think it should be safe.
I checked though the function because I was worried it might yield
somewhere in there, which could result in the page cache being emptied
and the allocation failing, but I don't think mmu_alloc_root this
function will yield.
> vcpu->arch.mmu->root_hpa = root;
> } else if (shadow_root_level == PT32E_ROOT_LEVEL) {
> for (i = 0; i < 4; ++i) {
> @@ -3258,8 +3248,6 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *vcpu)
>
> root = mmu_alloc_root(vcpu, i << (30 - PAGE_SHIFT),
> i << 30, PT32_ROOT_LEVEL, true);
> - if (!VALID_PAGE(root))
> - return -ENOSPC;
> vcpu->arch.mmu->pae_root[i] = root | PT_PRESENT_MASK;
> }
> vcpu->arch.mmu->root_hpa = __pa(vcpu->arch.mmu->pae_root);
> @@ -3294,8 +3282,6 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu)
>
> root = mmu_alloc_root(vcpu, root_gfn, 0,
> vcpu->arch.mmu->shadow_root_level, false);
> - if (!VALID_PAGE(root))
> - return -ENOSPC;
> vcpu->arch.mmu->root_hpa = root;
> goto set_root_pgd;
> }
> @@ -3325,6 +3311,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu)
>
> for (i = 0; i < 4; ++i) {
> MMU_WARN_ON(VALID_PAGE(vcpu->arch.mmu->pae_root[i]));
> +
> if (vcpu->arch.mmu->root_level == PT32E_ROOT_LEVEL) {
> pdptr = vcpu->arch.mmu->get_pdptr(vcpu, i);
> if (!(pdptr & PT_PRESENT_MASK)) {
> @@ -3338,8 +3325,6 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu)
>
> root = mmu_alloc_root(vcpu, root_gfn, i << 30,
> PT32_ROOT_LEVEL, false);
> - if (!VALID_PAGE(root))
> - return -ENOSPC;
> vcpu->arch.mmu->pae_root[i] = root | pm_mask;
> }
> vcpu->arch.mmu->root_hpa = __pa(vcpu->arch.mmu->pae_root);
> @@ -3373,14 +3358,6 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu)
> return 0;
> }
>
> -static int mmu_alloc_roots(struct kvm_vcpu *vcpu)
> -{
> - if (vcpu->arch.mmu->direct_map)
> - return mmu_alloc_direct_roots(vcpu);
> - else
> - return mmu_alloc_shadow_roots(vcpu);
> -}
> -
> void kvm_mmu_sync_roots(struct kvm_vcpu *vcpu)
> {
> int i;
> @@ -4822,7 +4799,15 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu)
> r = mmu_topup_memory_caches(vcpu, !vcpu->arch.mmu->direct_map);
> if (r)
> goto out;
> - r = mmu_alloc_roots(vcpu);
> + write_lock(&vcpu->kvm->mmu_lock);
> + if (make_mmu_pages_available(vcpu))
> + r = -ENOSPC;
> + else if (vcpu->arch.mmu->direct_map)
> + r = mmu_alloc_direct_roots(vcpu);
> + else
> + r = mmu_alloc_shadow_roots(vcpu);
> + write_unlock(&vcpu->kvm->mmu_lock);
> +
> kvm_mmu_sync_roots(vcpu);
> if (r)
> goto out;
> diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
> index 70226e0875fe..50ef757c5586 100644
> --- a/arch/x86/kvm/mmu/tdp_mmu.c
> +++ b/arch/x86/kvm/mmu/tdp_mmu.c
> @@ -137,22 +137,21 @@ static struct kvm_mmu_page *alloc_tdp_mmu_page(struct kvm_vcpu *vcpu, gfn_t gfn,
> return sp;
> }
>
> -static struct kvm_mmu_page *get_tdp_mmu_vcpu_root(struct kvm_vcpu *vcpu)
> +hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu)
> {
> union kvm_mmu_page_role role;
> struct kvm *kvm = vcpu->kvm;
> struct kvm_mmu_page *root;
>
> + lockdep_assert_held_write(&kvm->mmu_lock);
> +
> role = page_role_for_level(vcpu, vcpu->arch.mmu->shadow_root_level);
>
> - write_lock(&kvm->mmu_lock);
> -
> /* Check for an existing root before allocating a new one. */
> for_each_tdp_mmu_root(kvm, root) {
> if (root->role.word == role.word) {
> kvm_mmu_get_root(kvm, root);
> - write_unlock(&kvm->mmu_lock);
> - return root;
> + goto out;
> }
> }
>
> @@ -161,19 +160,7 @@ static struct kvm_mmu_page *get_tdp_mmu_vcpu_root(struct kvm_vcpu *vcpu)
>
> list_add(&root->link, &kvm->arch.tdp_mmu_roots);
>
> - write_unlock(&kvm->mmu_lock);
> -
> - return root;
> -}
> -
> -hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu)
> -{
> - struct kvm_mmu_page *root;
> -
> - root = get_tdp_mmu_vcpu_root(vcpu);
> - if (!root)
> - return INVALID_PAGE;
> -
> +out:
> return __pa(root->spt);
> }
>
> --
> 2.30.1.766.gb4fecdf3b7-goog
>
next prev parent reply other threads:[~2021-03-03 11:03 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-02 18:45 [PATCH 00/15] KVM: x86/mmu: Lots of bug fixes Sean Christopherson
2021-03-02 18:45 ` [PATCH 01/15] KVM: nSVM: Set the shadow root level to the TDP level for nested NPT Sean Christopherson
2021-03-02 18:45 ` [PATCH 02/15] KVM: x86/mmu: Alloc page for PDPTEs when shadowing 32-bit NPT with 64-bit Sean Christopherson
2021-03-03 17:28 ` Ben Gardon
2021-03-02 18:45 ` [PATCH 03/15] KVM: x86/mmu: Ensure MMU pages are available when allocating roots Sean Christopherson
2021-03-03 0:21 ` Ben Gardon [this message]
2021-03-03 16:46 ` Sean Christopherson
2021-03-02 18:45 ` [PATCH 04/15] KVM: x86/mmu: Allocate the lm_root before allocating PAE roots Sean Christopherson
2021-03-02 18:45 ` [PATCH 05/15] KVM: x86/mmu: Check PDPTRs " Sean Christopherson
2021-03-02 18:45 ` [PATCH 06/15] KVM: x86/mmu: Fix and unconditionally enable WARNs to detect PAE leaks Sean Christopherson
2021-03-02 18:45 ` [PATCH 07/15] KVM: x86/mmu: Use '0' as the one and only value for an invalid PAE root Sean Christopherson
2021-03-02 18:45 ` [PATCH 08/15] KVM: x86/mmu: Set the C-bit in the PDPTRs and LM pseudo-PDPTRs Sean Christopherson
2021-03-02 18:45 ` [PATCH 09/15] KVM: x86/mmu: Mark the PAE roots as decrypted for shadow paging Sean Christopherson
2021-03-02 18:45 ` [PATCH 10/15] KVM: SVM: Don't strip the C-bit from CR2 on #PF interception Sean Christopherson
2021-03-02 18:45 ` [PATCH 11/15] KVM: nVMX: Defer the MMU reload to the normal path on an EPTP switch Sean Christopherson
2021-03-02 18:45 ` [PATCH 12/15] KVM: x86: Defer the MMU unload to the normal path on an global INVPCID Sean Christopherson
2021-03-02 18:45 ` [PATCH 13/15] KVM: x86/mmu: Unexport MMU load/unload functions Sean Christopherson
2021-03-02 18:45 ` [PATCH 14/15] KVM: x86/mmu: Sync roots after MMU load iff load as successful Sean Christopherson
2021-03-02 18:45 ` [PATCH 15/15] KVM: x86/mmu: WARN on NULL pae_root and bad shadow root level Sean Christopherson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CANgfPd95G-01ObzeKeMTUu0yXBJ=0+ZGQwr_5WCNH-NmR03f9w@mail.gmail.com' \
--to=bgardon@google.com \
--cc=brijesh.singh@amd.com \
--cc=jmattson@google.com \
--cc=joro@8bytes.org \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=seanjc@google.com \
--cc=thomas.lendacky@amd.com \
--cc=vkuznets@redhat.com \
--cc=wanpengli@tencent.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.