From: Michael Roth <michael.roth@amd.com>
To: <x86@kernel.org>
Cc: <kvm@vger.kernel.org>, <linux-coco@lists.linux.dev>,
<linux-mm@kvack.org>, <linux-crypto@vger.kernel.org>,
<linux-kernel@vger.kernel.org>, <tglx@linutronix.de>,
<mingo@redhat.com>, <jroedel@suse.de>, <thomas.lendacky@amd.com>,
<hpa@zytor.com>, <ardb@kernel.org>, <pbonzini@redhat.com>,
<seanjc@google.com>, <vkuznets@redhat.com>, <jmattson@google.com>,
<luto@kernel.org>, <dave.hansen@linux.intel.com>,
<slp@redhat.com>, <pgonda@google.com>, <peterz@infradead.org>,
<srinivas.pandruvada@linux.intel.com>, <rientjes@google.com>,
<tobin@ibm.com>, <bp@alien8.de>, <vbabka@suse.cz>,
<kirill@shutemov.name>, <ak@linux.intel.com>,
<tony.luck@intel.com>,
<sathyanarayanan.kuppuswamy@linux.intel.com>,
<alpergun@google.com>, <jarkko@kernel.org>,
<ashish.kalra@amd.com>, <nikunj.dadhania@amd.com>,
<pankaj.gupta@amd.com>, <liam.merwick@oracle.com>,
<zhi.a.wang@intel.com>, Brijesh Singh <brijesh.singh@amd.com>,
Marc Orr <marcorr@google.com>
Subject: [PATCH v1 22/26] KVM: SEV: Make AVIC backing, VMSA and VMCB memory allocation SNP safe
Date: Sat, 30 Dec 2023 10:19:50 -0600 [thread overview]
Message-ID: <20231230161954.569267-23-michael.roth@amd.com> (raw)
In-Reply-To: <20231230161954.569267-1-michael.roth@amd.com>
From: Brijesh Singh <brijesh.singh@amd.com>
Implement a workaround for an SNP erratum where the CPU will incorrectly
signal an RMP violation #PF if a hugepage (2MB or 1GB) collides with the
RMP entry of a VMCB, VMSA or AVIC backing page.
When SEV-SNP is globally enabled, the CPU marks the VMCB, VMSA, and AVIC
backing pages as "in-use" via a reserved bit in the corresponding RMP
entry after a successful VMRUN. This is done for _all_ VMs, not just
SNP-Active VMs.
If the hypervisor accesses an in-use page through a writable
translation, the CPU will throw an RMP violation #PF. On early SNP
hardware, if an in-use page is 2MB-aligned and software accesses any
part of the associated 2MB region with a hugepage, the CPU will
incorrectly treat the entire 2MB region as in-use and signal a an RMP
violation #PF.
To avoid this, the recommendation is to not use a 2MB-aligned page for
the VMCB, VMSA or AVIC pages. Add a generic allocator that will ensure
that the page returned is not 2MB-aligned and is safe to be used when
SEV-SNP is enabled. Also implement similar handling for the VMCB/VMSA
pages of nested guests.
Co-developed-by: Marc Orr <marcorr@google.com>
Signed-off-by: Marc Orr <marcorr@google.com>
Reported-by: Alper Gun <alpergun@google.com> # for nested VMSA case
Co-developed-by: Ashish Kalra <ashish.kalra@amd.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
Acked-by: Vlastimil Babka <vbabka@suse.cz>
[mdr: squash in nested guest handling from Ashish, commit msg fixups]
Signed-off-by: Michael Roth <michael.roth@amd.com>
---
arch/x86/include/asm/kvm-x86-ops.h | 1 +
arch/x86/include/asm/kvm_host.h | 1 +
arch/x86/kvm/lapic.c | 5 ++++-
arch/x86/kvm/svm/nested.c | 2 +-
arch/x86/kvm/svm/sev.c | 32 ++++++++++++++++++++++++++++++
arch/x86/kvm/svm/svm.c | 17 +++++++++++++---
arch/x86/kvm/svm/svm.h | 1 +
7 files changed, 54 insertions(+), 5 deletions(-)
diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h
index 378ed944b849..ab24ce207988 100644
--- a/arch/x86/include/asm/kvm-x86-ops.h
+++ b/arch/x86/include/asm/kvm-x86-ops.h
@@ -138,6 +138,7 @@ KVM_X86_OP(complete_emulated_msr)
KVM_X86_OP(vcpu_deliver_sipi_vector)
KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
KVM_X86_OP_OPTIONAL(get_untagged_addr)
+KVM_X86_OP_OPTIONAL(alloc_apic_backing_page)
#undef KVM_X86_OP
#undef KVM_X86_OP_OPTIONAL
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 7bc1daf68741..9b0f18d096ed 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1794,6 +1794,7 @@ struct kvm_x86_ops {
unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu, gva_t gva, unsigned int flags);
+ void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
};
struct kvm_x86_nested_ops {
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 3242f3da2457..1edf93ee3395 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -2815,7 +2815,10 @@ int kvm_create_lapic(struct kvm_vcpu *vcpu, int timer_advance_ns)
vcpu->arch.apic = apic;
- apic->regs = (void *)get_zeroed_page(GFP_KERNEL_ACCOUNT);
+ if (kvm_x86_ops.alloc_apic_backing_page)
+ apic->regs = static_call(kvm_x86_alloc_apic_backing_page)(vcpu);
+ else
+ apic->regs = (void *)get_zeroed_page(GFP_KERNEL_ACCOUNT);
if (!apic->regs) {
printk(KERN_ERR "malloc apic regs error for vcpu %x\n",
vcpu->vcpu_id);
diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index dee62362a360..55b9a6d96bcf 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -1181,7 +1181,7 @@ int svm_allocate_nested(struct vcpu_svm *svm)
if (svm->nested.initialized)
return 0;
- vmcb02_page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+ vmcb02_page = snp_safe_alloc_page(&svm->vcpu);
if (!vmcb02_page)
return -ENOMEM;
svm->nested.vmcb02.ptr = page_address(vmcb02_page);
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 58e19d023d70..2efe3ed89808 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -3144,3 +3144,35 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)
ghcb_set_sw_exit_info_2(svm->sev_es.ghcb, 1);
}
+
+struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu)
+{
+ unsigned long pfn;
+ struct page *p;
+
+ if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+ return alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+
+ /*
+ * Allocate an SNP-safe page to workaround the SNP erratum where
+ * the CPU will incorrectly signal an RMP violation #PF if a
+ * hugepage (2MB or 1GB) collides with the RMP entry of a
+ * 2MB-aligned VMCB, VMSA, or AVIC backing page.
+ *
+ * Allocate one extra page, choose a page which is not
+ * 2MB-aligned, and free the other.
+ */
+ p = alloc_pages(GFP_KERNEL_ACCOUNT | __GFP_ZERO, 1);
+ if (!p)
+ return NULL;
+
+ split_page(p, 1);
+
+ pfn = page_to_pfn(p);
+ if (IS_ALIGNED(pfn, PTRS_PER_PMD))
+ __free_page(p++);
+ else
+ __free_page(p + 1);
+
+ return p;
+}
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 13cacaba229c..b6179696861a 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -702,7 +702,7 @@ static int svm_cpu_init(int cpu)
int ret = -ENOMEM;
memset(sd, 0, sizeof(struct svm_cpu_data));
- sd->save_area = alloc_page(GFP_KERNEL | __GFP_ZERO);
+ sd->save_area = snp_safe_alloc_page(NULL);
if (!sd->save_area)
return ret;
@@ -1420,7 +1420,7 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu)
svm = to_svm(vcpu);
err = -ENOMEM;
- vmcb01_page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+ vmcb01_page = snp_safe_alloc_page(vcpu);
if (!vmcb01_page)
goto out;
@@ -1429,7 +1429,7 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu)
* SEV-ES guests require a separate VMSA page used to contain
* the encrypted register state of the guest.
*/
- vmsa_page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+ vmsa_page = snp_safe_alloc_page(vcpu);
if (!vmsa_page)
goto error_free_vmcb_page;
@@ -4899,6 +4899,16 @@ static int svm_vm_init(struct kvm *kvm)
return 0;
}
+static void *svm_alloc_apic_backing_page(struct kvm_vcpu *vcpu)
+{
+ struct page *page = snp_safe_alloc_page(vcpu);
+
+ if (!page)
+ return NULL;
+
+ return page_address(page);
+}
+
static struct kvm_x86_ops svm_x86_ops __initdata = {
.name = KBUILD_MODNAME,
@@ -5030,6 +5040,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
.vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector,
.vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons,
+ .alloc_apic_backing_page = svm_alloc_apic_backing_page,
};
/*
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index 59adff7bbf55..9ed9d72546b3 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -694,6 +694,7 @@ void sev_es_vcpu_reset(struct vcpu_svm *svm);
void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector);
void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa);
void sev_es_unmap_ghcb(struct vcpu_svm *svm);
+struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu);
/* vmenter.S */
--
2.25.1
next prev parent reply other threads:[~2023-12-30 16:26 UTC|newest]
Thread overview: 102+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-30 16:19 [PATCH v1 00/26] Add AMD Secure Nested Paging (SEV-SNP) Initialization Support Michael Roth
2023-12-30 16:19 ` [PATCH v1 01/26] x86/cpufeatures: Add SEV-SNP CPU feature Michael Roth
2023-12-31 11:50 ` Borislav Petkov
2023-12-31 16:44 ` Michael Roth
2023-12-30 16:19 ` [PATCH v1 02/26] x86/speculation: Do not enable Automatic IBRS if SEV SNP is enabled Michael Roth
2023-12-30 16:19 ` [PATCH v1 03/26] iommu/amd: Don't rely on external callers to enable IOMMU SNP support Michael Roth
2024-01-04 10:30 ` Borislav Petkov
2024-01-04 10:58 ` Joerg Roedel
2023-12-30 16:19 ` [PATCH v1 04/26] x86/sev: Add the host SEV-SNP initialization support Michael Roth
2024-01-04 11:05 ` Jeremi Piotrowski
2024-01-05 16:09 ` Borislav Petkov
2024-01-05 16:21 ` Borislav Petkov
2024-01-08 16:49 ` Jeremi Piotrowski
2024-01-08 17:04 ` Borislav Petkov
2024-01-09 11:56 ` Jeremi Piotrowski
2024-01-09 12:29 ` Borislav Petkov
2024-01-09 12:44 ` Borislav Petkov
2024-02-14 16:56 ` Jeremi Piotrowski
2024-01-04 11:16 ` Borislav Petkov
2024-01-04 14:42 ` Borislav Petkov
2024-01-05 19:19 ` Borislav Petkov
2024-01-05 21:27 ` Borislav Petkov
2023-12-30 16:19 ` [PATCH v1 05/26] x86/mtrr: Don't print errors if MtrrFixDramModEn is set when SNP enabled Michael Roth
2023-12-30 16:19 ` [PATCH v1 06/26] x86/sev: Add RMP entry lookup helpers Michael Roth
2023-12-30 16:19 ` [PATCH v1 07/26] x86/fault: Add helper for dumping RMP entries Michael Roth
2024-01-10 9:59 ` Borislav Petkov
2024-01-10 20:18 ` Jarkko Sakkinen
2024-01-10 22:14 ` Borislav Petkov
2024-01-10 11:13 ` Borislav Petkov
2024-01-10 15:20 ` Tom Lendacky
2024-01-10 15:27 ` Borislav Petkov
2024-01-10 15:51 ` Tom Lendacky
2024-01-10 15:55 ` Borislav Petkov
2024-01-10 15:10 ` Tom Lendacky
2023-12-30 16:19 ` [PATCH v1 08/26] x86/traps: Define RMP violation #PF error code Michael Roth
2023-12-30 16:19 ` [PATCH v1 09/26] x86/fault: Dump RMP table information when RMP page faults occur Michael Roth
2023-12-30 16:19 ` [PATCH v1 10/26] x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction Michael Roth
2024-01-12 14:49 ` Borislav Petkov
2023-12-30 16:19 ` [PATCH v1 11/26] x86/sev: Invalidate pages from the direct map when adding them to the RMP table Michael Roth
2024-01-12 19:48 ` Borislav Petkov
2024-01-12 20:00 ` Dave Hansen
2024-01-12 20:07 ` Borislav Petkov
2024-01-12 20:27 ` Vlastimil Babka
2024-01-15 9:06 ` Borislav Petkov
2024-01-15 9:14 ` Vlastimil Babka
2024-01-15 9:16 ` Mike Rapoport
2024-01-15 9:20 ` Borislav Petkov
2024-01-12 20:28 ` Tom Lendacky
2024-01-12 20:37 ` Dave Hansen
2024-01-15 9:23 ` Vlastimil Babka
2024-01-16 16:19 ` Michael Roth
2024-01-16 16:50 ` Michael Roth
2024-01-16 20:12 ` Mike Rapoport
2024-01-26 1:49 ` Michael Roth
2024-01-16 18:22 ` Borislav Petkov
2024-01-16 20:22 ` Dave Hansen
2024-01-26 1:35 ` Michael Roth
2024-01-15 9:09 ` Borislav Petkov
2024-01-16 16:21 ` Dave Hansen
2024-01-17 9:34 ` Borislav Petkov
2024-01-15 9:01 ` Borislav Petkov
2023-12-30 16:19 ` [PATCH v1 12/26] crypto: ccp: Define the SEV-SNP commands Michael Roth
2024-01-15 9:41 ` Borislav Petkov
2024-01-26 1:56 ` Michael Roth
2023-12-30 16:19 ` [PATCH v1 13/26] crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP Michael Roth
2024-01-15 11:19 ` Borislav Petkov
2024-01-15 19:53 ` Borislav Petkov
2024-01-26 2:48 ` Michael Roth
2023-12-30 16:19 ` [PATCH v1 14/26] crypto: ccp: Provide API to issue SEV and SNP commands Michael Roth
2024-01-17 9:48 ` Borislav Petkov
2023-12-30 16:19 ` [PATCH v1 15/26] x86/sev: Introduce snp leaked pages list Michael Roth
2024-01-08 10:45 ` Vlastimil Babka
2024-01-09 22:19 ` Kalra, Ashish
2024-01-10 8:59 ` Vlastimil Babka
2023-12-30 16:19 ` [PATCH v1 16/26] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled Michael Roth
2023-12-30 16:19 ` [PATCH v1 17/26] crypto: ccp: Handle non-volatile INIT_EX data " Michael Roth
2024-01-18 14:03 ` Borislav Petkov
2023-12-30 16:19 ` [PATCH v1 18/26] crypto: ccp: Handle legacy SEV commands " Michael Roth
2024-01-19 17:18 ` Borislav Petkov
2024-01-19 17:36 ` Tom Lendacky
2024-01-19 17:48 ` Borislav Petkov
2024-01-26 13:29 ` Michael Roth
2023-12-30 16:19 ` [PATCH v1 19/26] iommu/amd: Clean up RMP entries for IOMMU pages during SNP shutdown Michael Roth
2023-12-30 16:19 ` [PATCH v1 20/26] crypto: ccp: Add debug support for decrypting pages Michael Roth
2024-01-10 14:59 ` Sean Christopherson
2024-01-11 0:50 ` Michael Roth
2023-12-30 16:19 ` [PATCH v1 21/26] crypto: ccp: Add panic notifier for SEV/SNP firmware shutdown on kdump Michael Roth
2024-01-21 11:49 ` Borislav Petkov
2024-01-26 3:03 ` Kalra, Ashish
2024-01-26 13:38 ` Michael Roth
2023-12-30 16:19 ` Michael Roth [this message]
2024-01-21 11:51 ` [PATCH v1 22/26] KVM: SEV: Make AVIC backing, VMSA and VMCB memory allocation SNP safe Borislav Petkov
2024-01-26 3:44 ` Michael Roth
2023-12-30 16:19 ` [PATCH v1 23/26] x86/cpufeatures: Enable/unmask SEV-SNP CPU feature Michael Roth
2023-12-30 16:19 ` [PATCH v1 24/26] crypto: ccp: Add the SNP_PLATFORM_STATUS command Michael Roth
2024-01-21 12:29 ` Borislav Petkov
2024-01-26 3:32 ` Michael Roth
2023-12-30 16:19 ` [PATCH v1 25/26] crypto: ccp: Add the SNP_COMMIT command Michael Roth
2024-01-21 12:35 ` Borislav Petkov
2023-12-30 16:19 ` [PATCH v1 26/26] crypto: ccp: Add the SNP_SET_CONFIG command Michael Roth
2024-01-21 12:41 ` Borislav Petkov
2024-01-26 13:30 ` Michael Roth
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231230161954.569267-23-michael.roth@amd.com \
--to=michael.roth@amd.com \
--cc=ak@linux.intel.com \
--cc=alpergun@google.com \
--cc=ardb@kernel.org \
--cc=ashish.kalra@amd.com \
--cc=bp@alien8.de \
--cc=brijesh.singh@amd.com \
--cc=dave.hansen@linux.intel.com \
--cc=hpa@zytor.com \
--cc=jarkko@kernel.org \
--cc=jmattson@google.com \
--cc=jroedel@suse.de \
--cc=kirill@shutemov.name \
--cc=kvm@vger.kernel.org \
--cc=liam.merwick@oracle.com \
--cc=linux-coco@lists.linux.dev \
--cc=linux-crypto@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=luto@kernel.org \
--cc=marcorr@google.com \
--cc=mingo@redhat.com \
--cc=nikunj.dadhania@amd.com \
--cc=pankaj.gupta@amd.com \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=pgonda@google.com \
--cc=rientjes@google.com \
--cc=sathyanarayanan.kuppuswamy@linux.intel.com \
--cc=seanjc@google.com \
--cc=slp@redhat.com \
--cc=srinivas.pandruvada@linux.intel.com \
--cc=tglx@linutronix.de \
--cc=thomas.lendacky@amd.com \
--cc=tobin@ibm.com \
--cc=tony.luck@intel.com \
--cc=vbabka@suse.cz \
--cc=vkuznets@redhat.com \
--cc=x86@kernel.org \
--cc=zhi.a.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).