* [PATCH v3 0/8] x86/seves: Support 32-bit boot path and other updates
@ 2021-03-12 12:38 Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 1/8] x86/boot/compressed/64: Cleanup exception handling before booting kernel Joerg Roedel
` (7 more replies)
0 siblings, 8 replies; 22+ messages in thread
From: Joerg Roedel @ 2021-03-12 12:38 UTC (permalink / raw)
To: x86
Cc: Joerg Roedel, Joerg Roedel, hpa, Andy Lutomirski, Dave Hansen,
Peter Zijlstra, Jiri Slaby, Dan Williams, Tom Lendacky,
Juergen Gross, Kees Cook, David Rientjes, Cfir Cohen,
Erdem Aktas, Masami Hiramatsu, Mike Stunes, Sean Christopherson,
Martin Radev, Arvind Sankar, linux-kernel, kvm, virtualization
From: Joerg Roedel <jroedel@suse.de>
Hi,
these patches add support for the 32-bit boot in the decompressor
code. This is needed to boot an SEV-ES guest on some firmware and grub
versions. The patches also add the necessary CPUID sanity checks and a
32-bit version of the C-bit check.
Other updates included here:
1. Add code to shut down exception handling in the
decompressor code before jumping to the real kernel.
Once in the real kernel it is not safe anymore to jump
back to the decompressor code via exceptions.
2. Replace open-coded hlt loops with proper calls to
sev_es_terminate().
Please review.
Thanks,
Joerg
Changes v2->v3:
- Added a patch to remove the check for the Hypervisor CPUID
bit for detecting SEV
Changes v1->v2:
- Addressed Boris' review comments.
- Fixed a bug which caused the cbit-check to never be
executed even in an SEV guest.
Joerg Roedel (8):
x86/boot/compressed/64: Cleanup exception handling before booting
kernel
x86/sev: Do not require Hypervisor CPUID bit for SEV guests
x86/boot/compressed/64: Reload CS in startup_32
x86/boot/compressed/64: Setup IDT in startup_32 boot path
x86/boot/compressed/64: Add 32-bit boot #VC handler
x86/boot/compressed/64: Add CPUID sanity check to 32-bit boot-path
x86/boot/compressed/64: Check SEV encryption in 32-bit boot-path
x86/sev-es: Replace open-coded hlt-loops with sev_es_terminate()
arch/x86/boot/compressed/head_64.S | 170 ++++++++++++++++++++++++-
arch/x86/boot/compressed/idt_64.c | 14 ++
arch/x86/boot/compressed/mem_encrypt.S | 130 ++++++++++++++++++-
arch/x86/boot/compressed/misc.c | 7 +-
arch/x86/boot/compressed/misc.h | 6 +
arch/x86/boot/compressed/sev-es.c | 12 +-
arch/x86/kernel/sev-es-shared.c | 16 +--
arch/x86/mm/mem_encrypt_identity.c | 35 ++---
8 files changed, 340 insertions(+), 50 deletions(-)
--
2.30.1
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v3 1/8] x86/boot/compressed/64: Cleanup exception handling before booting kernel
2021-03-12 12:38 [PATCH v3 0/8] x86/seves: Support 32-bit boot path and other updates Joerg Roedel
@ 2021-03-12 12:38 ` Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 2/8] x86/sev: Do not require Hypervisor CPUID bit for SEV guests Joerg Roedel
` (6 subsequent siblings)
7 siblings, 1 reply; 22+ messages in thread
From: Joerg Roedel @ 2021-03-12 12:38 UTC (permalink / raw)
To: x86
Cc: Joerg Roedel, Joerg Roedel, hpa, Andy Lutomirski, Dave Hansen,
Peter Zijlstra, Jiri Slaby, Dan Williams, Tom Lendacky,
Juergen Gross, Kees Cook, David Rientjes, Cfir Cohen,
Erdem Aktas, Masami Hiramatsu, Mike Stunes, Sean Christopherson,
Martin Radev, Arvind Sankar, linux-kernel, kvm, virtualization
From: Joerg Roedel <jroedel@suse.de>
Disable the exception handling before booting the kernel to make sure
any exceptions that happen during early kernel boot are not directed to
the pre-decompression code.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
arch/x86/boot/compressed/idt_64.c | 14 ++++++++++++++
arch/x86/boot/compressed/misc.c | 7 ++-----
arch/x86/boot/compressed/misc.h | 6 ++++++
3 files changed, 22 insertions(+), 5 deletions(-)
diff --git a/arch/x86/boot/compressed/idt_64.c b/arch/x86/boot/compressed/idt_64.c
index 804a502ee0d2..9b93567d663a 100644
--- a/arch/x86/boot/compressed/idt_64.c
+++ b/arch/x86/boot/compressed/idt_64.c
@@ -52,3 +52,17 @@ void load_stage2_idt(void)
load_boot_idt(&boot_idt_desc);
}
+
+void cleanup_exception_handling(void)
+{
+ /*
+ * Flush GHCB from cache and map it encrypted again when running as
+ * SEV-ES guest.
+ */
+ sev_es_shutdown_ghcb();
+
+ /* Set a null-idt, disabling #PF and #VC handling */
+ boot_idt_desc.size = 0;
+ boot_idt_desc.address = 0;
+ load_boot_idt(&boot_idt_desc);
+}
diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c
index 267e7f93050e..cc9fd0e8766a 100644
--- a/arch/x86/boot/compressed/misc.c
+++ b/arch/x86/boot/compressed/misc.c
@@ -443,11 +443,8 @@ asmlinkage __visible void *extract_kernel(void *rmode, memptr heap,
handle_relocations(output, output_len, virt_addr);
debug_putstr("done.\nBooting the kernel.\n");
- /*
- * Flush GHCB from cache and map it encrypted again when running as
- * SEV-ES guest.
- */
- sev_es_shutdown_ghcb();
+ /* Disable exception handling before booting the kernel */
+ cleanup_exception_handling();
return output;
}
diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h
index 901ea5ebec22..e5612f035498 100644
--- a/arch/x86/boot/compressed/misc.h
+++ b/arch/x86/boot/compressed/misc.h
@@ -155,6 +155,12 @@ extern pteval_t __default_kernel_pte_mask;
extern gate_desc boot_idt[BOOT_IDT_ENTRIES];
extern struct desc_ptr boot_idt_desc;
+#ifdef CONFIG_X86_64
+void cleanup_exception_handling(void);
+#else
+static inline void cleanup_exception_handling(void) { }
+#endif
+
/* IDT Entry Points */
void boot_page_fault(void);
void boot_stage1_vc(void);
--
2.30.1
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 2/8] x86/sev: Do not require Hypervisor CPUID bit for SEV guests
2021-03-12 12:38 [PATCH v3 0/8] x86/seves: Support 32-bit boot path and other updates Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 1/8] x86/boot/compressed/64: Cleanup exception handling before booting kernel Joerg Roedel
@ 2021-03-12 12:38 ` Joerg Roedel
2021-03-17 15:04 ` Tom Lendacky
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 3/8] x86/boot/compressed/64: Reload CS in startup_32 Joerg Roedel
` (5 subsequent siblings)
7 siblings, 2 replies; 22+ messages in thread
From: Joerg Roedel @ 2021-03-12 12:38 UTC (permalink / raw)
To: x86
Cc: Joerg Roedel, Joerg Roedel, hpa, Andy Lutomirski, Dave Hansen,
Peter Zijlstra, Jiri Slaby, Dan Williams, Tom Lendacky,
Juergen Gross, Kees Cook, David Rientjes, Cfir Cohen,
Erdem Aktas, Masami Hiramatsu, Mike Stunes, Sean Christopherson,
Martin Radev, Arvind Sankar, linux-kernel, kvm, virtualization
From: Joerg Roedel <jroedel@suse.de>
A malicious hypervisor could disable the CPUID intercept for an SEV or
SEV-ES guest and trick it into the no-SEV boot path, where it could
potentially reveal secrets. This is not an issue for SEV-SNP guests,
as the CPUID intercept can't be disabled for those.
Remove the Hypervisor CPUID bit check from the SEV detection code to
protect against this kind of attack and add a Hypervisor bit equals
zero check to the SME detection path to prevent non-SEV guests from
trying to enable SME.
This handles the following cases:
1) SEV(-ES) guest where CPUID intercept is disabled. The guest
will still see leaf 0x8000001f and the SEV bit. It can
retrieve the C-bit and boot normally.
2) Non-SEV guests with intercepted CPUID will check SEV_STATUS
MSR and find it 0 and will try to enable SME. This will
fail when the guest finds MSR_K8_SYSCFG to be zero, as it
is emulated by KVM. But we can't rely on that, as there
might be other hypervisors which return this MSR with bit
23 set. The Hypervisor bit check will prevent that the
guest tries to enable SME in this case.
3) Non-SEV guests on SEV capable hosts with CPUID intercept
disabled (by a malicious hypervisor) will try to boot into
the SME path. This will fail, but it is also not considered
a problem because non-encrypted guests have no protection
against the hypervisor anyway.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
arch/x86/boot/compressed/mem_encrypt.S | 6 -----
arch/x86/kernel/sev-es-shared.c | 6 +----
arch/x86/mm/mem_encrypt_identity.c | 35 ++++++++++++++------------
3 files changed, 20 insertions(+), 27 deletions(-)
diff --git a/arch/x86/boot/compressed/mem_encrypt.S b/arch/x86/boot/compressed/mem_encrypt.S
index aa561795efd1..a6dea4e8a082 100644
--- a/arch/x86/boot/compressed/mem_encrypt.S
+++ b/arch/x86/boot/compressed/mem_encrypt.S
@@ -23,12 +23,6 @@ SYM_FUNC_START(get_sev_encryption_bit)
push %ecx
push %edx
- /* Check if running under a hypervisor */
- movl $1, %eax
- cpuid
- bt $31, %ecx /* Check the hypervisor bit */
- jnc .Lno_sev
-
movl $0x80000000, %eax /* CPUID to check the highest leaf */
cpuid
cmpl $0x8000001f, %eax /* See if 0x8000001f is available */
diff --git a/arch/x86/kernel/sev-es-shared.c b/arch/x86/kernel/sev-es-shared.c
index cdc04d091242..387b71669818 100644
--- a/arch/x86/kernel/sev-es-shared.c
+++ b/arch/x86/kernel/sev-es-shared.c
@@ -186,7 +186,6 @@ void __init do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code)
* make it accessible to the hypervisor.
*
* In particular, check for:
- * - Hypervisor CPUID bit
* - Availability of CPUID leaf 0x8000001f
* - SEV CPUID bit.
*
@@ -194,10 +193,7 @@ void __init do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code)
* can't be checked here.
*/
- if ((fn == 1 && !(regs->cx & BIT(31))))
- /* Hypervisor bit */
- goto fail;
- else if (fn == 0x80000000 && (regs->ax < 0x8000001f))
+ if (fn == 0x80000000 && (regs->ax < 0x8000001f))
/* SEV leaf check */
goto fail;
else if ((fn == 0x8000001f && !(regs->ax & BIT(1))))
diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c
index 6c5eb6f3f14f..a19374d26101 100644
--- a/arch/x86/mm/mem_encrypt_identity.c
+++ b/arch/x86/mm/mem_encrypt_identity.c
@@ -503,14 +503,10 @@ void __init sme_enable(struct boot_params *bp)
#define AMD_SME_BIT BIT(0)
#define AMD_SEV_BIT BIT(1)
- /*
- * Set the feature mask (SME or SEV) based on whether we are
- * running under a hypervisor.
- */
- eax = 1;
- ecx = 0;
- native_cpuid(&eax, &ebx, &ecx, &edx);
- feature_mask = (ecx & BIT(31)) ? AMD_SEV_BIT : AMD_SME_BIT;
+
+ /* Check the SEV MSR whether SEV or SME is enabled */
+ sev_status = __rdmsr(MSR_AMD64_SEV);
+ feature_mask = (sev_status & MSR_AMD64_SEV_ENABLED) ? AMD_SEV_BIT : AMD_SME_BIT;
/*
* Check for the SME/SEV feature:
@@ -530,19 +526,26 @@ void __init sme_enable(struct boot_params *bp)
/* Check if memory encryption is enabled */
if (feature_mask == AMD_SME_BIT) {
+ /*
+ * No SME if Hypervisor bit is set. This check is here to
+ * prevent a guest from trying to enable SME. For running as a
+ * KVM guest the MSR_K8_SYSCFG will be sufficient, but there
+ * might be other hypervisors which emulate that MSR as non-zero
+ * or even pass it through to the guest.
+ * A malicious hypervisor can still trick a guest into this
+ * path, but there is no way to protect against that.
+ */
+ eax = 1;
+ ecx = 0;
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+ if (ecx & BIT(31))
+ return;
+
/* For SME, check the SYSCFG MSR */
msr = __rdmsr(MSR_K8_SYSCFG);
if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
return;
} else {
- /* For SEV, check the SEV MSR */
- msr = __rdmsr(MSR_AMD64_SEV);
- if (!(msr & MSR_AMD64_SEV_ENABLED))
- return;
-
- /* Save SEV_STATUS to avoid reading MSR again */
- sev_status = msr;
-
/* SEV state cannot be controlled by a command line option */
sme_me_mask = me_mask;
sev_enabled = true;
--
2.30.1
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 3/8] x86/boot/compressed/64: Reload CS in startup_32
2021-03-12 12:38 [PATCH v3 0/8] x86/seves: Support 32-bit boot path and other updates Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 1/8] x86/boot/compressed/64: Cleanup exception handling before booting kernel Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 2/8] x86/sev: Do not require Hypervisor CPUID bit for SEV guests Joerg Roedel
@ 2021-03-12 12:38 ` Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 4/8] x86/boot/compressed/64: Setup IDT in startup_32 boot path Joerg Roedel
` (4 subsequent siblings)
7 siblings, 1 reply; 22+ messages in thread
From: Joerg Roedel @ 2021-03-12 12:38 UTC (permalink / raw)
To: x86
Cc: Joerg Roedel, Joerg Roedel, hpa, Andy Lutomirski, Dave Hansen,
Peter Zijlstra, Jiri Slaby, Dan Williams, Tom Lendacky,
Juergen Gross, Kees Cook, David Rientjes, Cfir Cohen,
Erdem Aktas, Masami Hiramatsu, Mike Stunes, Sean Christopherson,
Martin Radev, Arvind Sankar, linux-kernel, kvm, virtualization
From: Joerg Roedel <jroedel@suse.de>
Exception handling in the startup_32 boot path requires the CS
selector to be correctly set up. Reload it from the current GDT.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
arch/x86/boot/compressed/head_64.S | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index e94874f4bbc1..c59c80ca546d 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -107,9 +107,16 @@ SYM_FUNC_START(startup_32)
movl %eax, %gs
movl %eax, %ss
-/* setup a stack and make sure cpu supports long mode. */
+ /* Setup a stack and load CS from current GDT */
leal rva(boot_stack_end)(%ebp), %esp
+ pushl $__KERNEL32_CS
+ leal rva(1f)(%ebp), %eax
+ pushl %eax
+ lretl
+1:
+
+ /* Make sure cpu supports long mode. */
call verify_cpu
testl %eax, %eax
jnz .Lno_longmode
--
2.30.1
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 4/8] x86/boot/compressed/64: Setup IDT in startup_32 boot path
2021-03-12 12:38 [PATCH v3 0/8] x86/seves: Support 32-bit boot path and other updates Joerg Roedel
` (2 preceding siblings ...)
2021-03-12 12:38 ` [PATCH v3 3/8] x86/boot/compressed/64: Reload CS in startup_32 Joerg Roedel
@ 2021-03-12 12:38 ` Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 5/8] x86/boot/compressed/64: Add 32-bit boot #VC handler Joerg Roedel
` (3 subsequent siblings)
7 siblings, 1 reply; 22+ messages in thread
From: Joerg Roedel @ 2021-03-12 12:38 UTC (permalink / raw)
To: x86
Cc: Joerg Roedel, Joerg Roedel, hpa, Andy Lutomirski, Dave Hansen,
Peter Zijlstra, Jiri Slaby, Dan Williams, Tom Lendacky,
Juergen Gross, Kees Cook, David Rientjes, Cfir Cohen,
Erdem Aktas, Masami Hiramatsu, Mike Stunes, Sean Christopherson,
Martin Radev, Arvind Sankar, linux-kernel, kvm, virtualization
From: Joerg Roedel <jroedel@suse.de>
This boot path needs exception handling when it is used with SEV-ES.
Setup an IDT and provide a helper function to write IDT entries for
use in 32-bit protected mode.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
arch/x86/boot/compressed/head_64.S | 72 ++++++++++++++++++++++++++++++
1 file changed, 72 insertions(+)
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index c59c80ca546d..2001c3bf0748 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -116,6 +116,9 @@ SYM_FUNC_START(startup_32)
lretl
1:
+ /* Setup Exception handling for SEV-ES */
+ call startup32_load_idt
+
/* Make sure cpu supports long mode. */
call verify_cpu
testl %eax, %eax
@@ -701,6 +704,19 @@ SYM_DATA_START(boot_idt)
.endr
SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end)
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+SYM_DATA_START(boot32_idt_desc)
+ .word boot32_idt_end - boot32_idt - 1
+ .long 0
+SYM_DATA_END(boot32_idt_desc)
+ .balign 8
+SYM_DATA_START(boot32_idt)
+ .rept 32
+ .quad 0
+ .endr
+SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
+#endif
+
#ifdef CONFIG_EFI_STUB
SYM_DATA(image_offset, .long 0)
#endif
@@ -793,6 +809,62 @@ SYM_DATA_START_LOCAL(loaded_image_proto)
SYM_DATA_END(loaded_image_proto)
#endif
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+ __HEAD
+ .code32
+/*
+ * Write an IDT entry into boot32_idt
+ *
+ * Parameters:
+ *
+ * %eax: Handler address
+ * %edx: Vector number
+ *
+ * Physical offset is expected in %ebp
+ */
+SYM_FUNC_START(startup32_set_idt_entry)
+ push %ebx
+ push %ecx
+
+ /* IDT entry address to %ebx */
+ leal rva(boot32_idt)(%ebp), %ebx
+ shl $3, %edx
+ addl %edx, %ebx
+
+ /* Build IDT entry, lower 4 bytes */
+ movl %eax, %edx
+ andl $0x0000ffff, %edx # Target code segment offset [15:0]
+ movl $__KERNEL32_CS, %ecx # Target code segment selector
+ shl $16, %ecx
+ orl %ecx, %edx
+
+ /* Store lower 4 bytes to IDT */
+ movl %edx, (%ebx)
+
+ /* Build IDT entry, upper 4 bytes */
+ movl %eax, %edx
+ andl $0xffff0000, %edx # Target code segment offset [31:16]
+ orl $0x00008e00, %edx # Present, Type 32-bit Interrupt Gate
+
+ /* Store upper 4 bytes to IDT */
+ movl %edx, 4(%ebx)
+
+ pop %ecx
+ pop %ebx
+ ret
+SYM_FUNC_END(startup32_set_idt_entry)
+#endif
+
+SYM_FUNC_START(startup32_load_idt)
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+ /* Load IDT */
+ leal rva(boot32_idt)(%ebp), %eax
+ movl %eax, rva(boot32_idt_desc+2)(%ebp)
+ lidt rva(boot32_idt_desc)(%ebp)
+#endif
+ ret
+SYM_FUNC_END(startup32_load_idt)
+
/*
* Stack and heap for uncompression
*/
--
2.30.1
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 5/8] x86/boot/compressed/64: Add 32-bit boot #VC handler
2021-03-12 12:38 [PATCH v3 0/8] x86/seves: Support 32-bit boot path and other updates Joerg Roedel
` (3 preceding siblings ...)
2021-03-12 12:38 ` [PATCH v3 4/8] x86/boot/compressed/64: Setup IDT in startup_32 boot path Joerg Roedel
@ 2021-03-12 12:38 ` Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 6/8] x86/boot/compressed/64: Add CPUID sanity check to 32-bit boot-path Joerg Roedel
` (2 subsequent siblings)
7 siblings, 2 replies; 22+ messages in thread
From: Joerg Roedel @ 2021-03-12 12:38 UTC (permalink / raw)
To: x86
Cc: Joerg Roedel, Joerg Roedel, hpa, Andy Lutomirski, Dave Hansen,
Peter Zijlstra, Jiri Slaby, Dan Williams, Tom Lendacky,
Juergen Gross, Kees Cook, David Rientjes, Cfir Cohen,
Erdem Aktas, Masami Hiramatsu, Mike Stunes, Sean Christopherson,
Martin Radev, Arvind Sankar, linux-kernel, kvm, virtualization
From: Joerg Roedel <jroedel@suse.de>
Add a #VC exception handler which is used when the kernel still executes
in protected mode. This boot-path already uses CPUID, which will cause #VC
exceptions in an SEV-ES guest.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
arch/x86/boot/compressed/head_64.S | 6 ++
arch/x86/boot/compressed/mem_encrypt.S | 96 +++++++++++++++++++++++++-
2 files changed, 101 insertions(+), 1 deletion(-)
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index 2001c3bf0748..ee448aedb8b0 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -34,6 +34,7 @@
#include <asm/asm-offsets.h>
#include <asm/bootparam.h>
#include <asm/desc_defs.h>
+#include <asm/trapnr.h>
#include "pgtable.h"
/*
@@ -857,6 +858,11 @@ SYM_FUNC_END(startup32_set_idt_entry)
SYM_FUNC_START(startup32_load_idt)
#ifdef CONFIG_AMD_MEM_ENCRYPT
+ /* #VC handler */
+ leal rva(startup32_vc_handler)(%ebp), %eax
+ movl $X86_TRAP_VC, %edx
+ call startup32_set_idt_entry
+
/* Load IDT */
leal rva(boot32_idt)(%ebp), %eax
movl %eax, rva(boot32_idt_desc+2)(%ebp)
diff --git a/arch/x86/boot/compressed/mem_encrypt.S b/arch/x86/boot/compressed/mem_encrypt.S
index a6dea4e8a082..e915f4906477 100644
--- a/arch/x86/boot/compressed/mem_encrypt.S
+++ b/arch/x86/boot/compressed/mem_encrypt.S
@@ -61,10 +61,104 @@ SYM_FUNC_START(get_sev_encryption_bit)
ret
SYM_FUNC_END(get_sev_encryption_bit)
+/**
+ * sev_es_req_cpuid - Request a CPUID value from the Hypervisor using
+ * the GHCB MSR protocol
+ *
+ * @%eax: Register to request (0=EAX, 1=EBX, 2=ECX, 3=EDX)
+ * @%edx: CPUID Function
+ *
+ * Returns 0 in %eax on sucess, non-zero on failure
+ * %edx returns CPUID value on success
+ */
+SYM_CODE_START_LOCAL(sev_es_req_cpuid)
+ shll $30, %eax
+ orl $0x00000004, %eax
+ movl $MSR_AMD64_SEV_ES_GHCB, %ecx
+ wrmsr
+ rep; vmmcall # VMGEXIT
+ rdmsr
+
+ /* Check response */
+ movl %eax, %ecx
+ andl $0x3ffff000, %ecx # Bits [12-29] MBZ
+ jnz 2f
+
+ /* Check return code */
+ andl $0xfff, %eax
+ cmpl $5, %eax
+ jne 2f
+
+ /* All good - return success */
+ xorl %eax, %eax
+1:
+ ret
+2:
+ movl $-1, %eax
+ jmp 1b
+SYM_CODE_END(sev_es_req_cpuid)
+
+SYM_CODE_START(startup32_vc_handler)
+ pushl %eax
+ pushl %ebx
+ pushl %ecx
+ pushl %edx
+
+ /* Keep CPUID function in %ebx */
+ movl %eax, %ebx
+
+ /* Check if error-code == SVM_EXIT_CPUID */
+ cmpl $0x72, 16(%esp)
+ jne .Lfail
+
+ movl $0, %eax # Request CPUID[fn].EAX
+ movl %ebx, %edx # CPUID fn
+ call sev_es_req_cpuid # Call helper
+ testl %eax, %eax # Check return code
+ jnz .Lfail
+ movl %edx, 12(%esp) # Store result
+
+ movl $1, %eax # Request CPUID[fn].EBX
+ movl %ebx, %edx # CPUID fn
+ call sev_es_req_cpuid # Call helper
+ testl %eax, %eax # Check return code
+ jnz .Lfail
+ movl %edx, 8(%esp) # Store result
+
+ movl $2, %eax # Request CPUID[fn].ECX
+ movl %ebx, %edx # CPUID fn
+ call sev_es_req_cpuid # Call helper
+ testl %eax, %eax # Check return code
+ jnz .Lfail
+ movl %edx, 4(%esp) # Store result
+
+ movl $3, %eax # Request CPUID[fn].EDX
+ movl %ebx, %edx # CPUID fn
+ call sev_es_req_cpuid # Call helper
+ testl %eax, %eax # Check return code
+ jnz .Lfail
+ movl %edx, 0(%esp) # Store result
+
+ popl %edx
+ popl %ecx
+ popl %ebx
+ popl %eax
+
+ /* Remove error code */
+ addl $4, %esp
+
+ /* Jump over CPUID instruction */
+ addl $2, (%esp)
+
+ iret
+.Lfail:
+ hlt
+ jmp .Lfail
+SYM_CODE_END(startup32_vc_handler)
+
.code64
#include "../../kernel/sev_verify_cbit.S"
-
SYM_FUNC_START(set_sev_encryption_mask)
#ifdef CONFIG_AMD_MEM_ENCRYPT
push %rbp
--
2.30.1
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 6/8] x86/boot/compressed/64: Add CPUID sanity check to 32-bit boot-path
2021-03-12 12:38 [PATCH v3 0/8] x86/seves: Support 32-bit boot path and other updates Joerg Roedel
` (4 preceding siblings ...)
2021-03-12 12:38 ` [PATCH v3 5/8] x86/boot/compressed/64: Add 32-bit boot #VC handler Joerg Roedel
@ 2021-03-12 12:38 ` Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 7/8] x86/boot/compressed/64: Check SEV encryption in " Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 8/8] x86/sev-es: Replace open-coded hlt-loops with sev_es_terminate() Joerg Roedel
7 siblings, 2 replies; 22+ messages in thread
From: Joerg Roedel @ 2021-03-12 12:38 UTC (permalink / raw)
To: x86
Cc: Joerg Roedel, Joerg Roedel, hpa, Andy Lutomirski, Dave Hansen,
Peter Zijlstra, Jiri Slaby, Dan Williams, Tom Lendacky,
Juergen Gross, Kees Cook, David Rientjes, Cfir Cohen,
Erdem Aktas, Masami Hiramatsu, Mike Stunes, Sean Christopherson,
Martin Radev, Arvind Sankar, linux-kernel, kvm, virtualization
From: Joerg Roedel <jroedel@suse.de>
The 32-bit #VC handler has no GHCB and can only handle CPUID exit codes.
It is needed by the early boot code to handle #VC exceptions raised in
verify_cpu() and to get the position of the C bit.
But the CPUID information comes from the hypervisor, which is untrusted
and might return results which trick the guest into the no-SEV boot path
with no C bit set in the page-tables. All data written to memory would
then be unencrypted and could leak sensitive data to the hypervisor.
Add sanity checks to the 32-bit boot #VC handler to make sure the
hypervisor does not pretend that SEV is not enabled.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
arch/x86/boot/compressed/mem_encrypt.S | 28 ++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/arch/x86/boot/compressed/mem_encrypt.S b/arch/x86/boot/compressed/mem_encrypt.S
index e915f4906477..0211eddefeb0 100644
--- a/arch/x86/boot/compressed/mem_encrypt.S
+++ b/arch/x86/boot/compressed/mem_encrypt.S
@@ -139,6 +139,26 @@ SYM_CODE_START(startup32_vc_handler)
jnz .Lfail
movl %edx, 0(%esp) # Store result
+ /*
+ * Sanity check CPUID results from the Hypervisor. See comment in
+ * do_vc_no_ghcb() for more details on why this is necessary.
+ */
+
+ /* Fail if SEV leaf not available in CPUID[0x80000000].EAX */
+ cmpl $0x80000000, %ebx
+ jne .Lcheck_sev
+ cmpl $0x8000001f, 12(%esp)
+ jb .Lfail
+ jmp .Ldone
+
+.Lcheck_sev:
+ /* Fail if SEV bit not set in CPUID[0x8000001f].EAX[1] */
+ cmpl $0x8000001f, %ebx
+ jne .Ldone
+ btl $1, 12(%esp)
+ jnc .Lfail
+
+.Ldone:
popl %edx
popl %ecx
popl %ebx
@@ -152,6 +172,14 @@ SYM_CODE_START(startup32_vc_handler)
iret
.Lfail:
+ /* Send terminate request to Hypervisor */
+ movl $0x100, %eax
+ xorl %edx, %edx
+ movl $MSR_AMD64_SEV_ES_GHCB, %ecx
+ wrmsr
+ rep; vmmcall
+
+ /* If request fails, go to hlt loop */
hlt
jmp .Lfail
SYM_CODE_END(startup32_vc_handler)
--
2.30.1
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 7/8] x86/boot/compressed/64: Check SEV encryption in 32-bit boot-path
2021-03-12 12:38 [PATCH v3 0/8] x86/seves: Support 32-bit boot path and other updates Joerg Roedel
` (5 preceding siblings ...)
2021-03-12 12:38 ` [PATCH v3 6/8] x86/boot/compressed/64: Add CPUID sanity check to 32-bit boot-path Joerg Roedel
@ 2021-03-12 12:38 ` Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] x86/boot/compressed/64: Check SEV encryption in the " tip-bot2 for Joerg Roedel
2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 8/8] x86/sev-es: Replace open-coded hlt-loops with sev_es_terminate() Joerg Roedel
7 siblings, 2 replies; 22+ messages in thread
From: Joerg Roedel @ 2021-03-12 12:38 UTC (permalink / raw)
To: x86
Cc: Joerg Roedel, Joerg Roedel, hpa, Andy Lutomirski, Dave Hansen,
Peter Zijlstra, Jiri Slaby, Dan Williams, Tom Lendacky,
Juergen Gross, Kees Cook, David Rientjes, Cfir Cohen,
Erdem Aktas, Masami Hiramatsu, Mike Stunes, Sean Christopherson,
Martin Radev, Arvind Sankar, linux-kernel, kvm, virtualization
From: Joerg Roedel <jroedel@suse.de>
Check whether the hypervisor reported the correct C-bit when running as
an SEV guest. Using a wrong C-bit position could be used to leak
sensitive data from the guest to the hypervisor.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
arch/x86/boot/compressed/head_64.S | 83 ++++++++++++++++++++++++++++++
1 file changed, 83 insertions(+)
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index ee448aedb8b0..7c5c2698a96e 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -183,11 +183,21 @@ SYM_FUNC_START(startup_32)
*/
call get_sev_encryption_bit
xorl %edx, %edx
+#ifdef CONFIG_AMD_MEM_ENCRYPT
testl %eax, %eax
jz 1f
subl $32, %eax /* Encryption bit is always above bit 31 */
bts %eax, %edx /* Set encryption mask for page tables */
+ /*
+ * Mark SEV as active in sev_status so that startup32_check_sev_cbit()
+ * will do a check. The sev_status memory will be fully initialized
+ * with the contents of MSR_AMD_SEV_STATUS later in
+ * set_sev_encryption_mask(). For now it is sufficient to know that SEV
+ * is active.
+ */
+ movl $1, rva(sev_status)(%ebp)
1:
+#endif
/* Initialize Page tables to 0 */
leal rva(pgtable)(%ebx), %edi
@@ -272,6 +282,9 @@ SYM_FUNC_START(startup_32)
movl %esi, %edx
1:
#endif
+ /* Check if the C-bit position is correct when SEV is active */
+ call startup32_check_sev_cbit
+
pushl $__KERNEL_CS
pushl %eax
@@ -871,6 +884,76 @@ SYM_FUNC_START(startup32_load_idt)
ret
SYM_FUNC_END(startup32_load_idt)
+/*
+ * Check for the correct C-bit position when the startup_32 boot-path is used.
+ *
+ * The check makes use of the fact that all memory is encrypted when paging is
+ * disabled. The function creates 64 bits of random data using the RDRAND
+ * instruction. RDRAND is mandatory for SEV guests, so always available. If the
+ * hypervisor violates that the kernel will crash right here.
+ *
+ * The 64 bits of random data are stored to a memory location and at the same
+ * time kept in the %eax and %ebx registers. Since encryption is always active
+ * when paging is off the random data will be stored encrypted in main memory.
+ *
+ * Then paging is enabled. When the C-bit position is correct all memory is
+ * still mapped encrypted and comparing the register values with memory will
+ * succeed. An incorrect C-bit position will map all memory unencrypted, so that
+ * the compare will use the encrypted random data and fail.
+ */
+SYM_FUNC_START(startup32_check_sev_cbit)
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+ pushl %eax
+ pushl %ebx
+ pushl %ecx
+ pushl %edx
+
+ /* Check for non-zero sev_status */
+ movl rva(sev_status)(%ebp), %eax
+ testl %eax, %eax
+ jz 4f
+
+ /*
+ * Get two 32-bit random values - Don't bail out if RDRAND fails
+ * because it is better to prevent forward progress if no random value
+ * can be gathered.
+ */
+1: rdrand %eax
+ jnc 1b
+2: rdrand %ebx
+ jnc 2b
+
+ /* Store to memory and keep it in the registers */
+ movl %eax, rva(sev_check_data)(%ebp)
+ movl %ebx, rva(sev_check_data+4)(%ebp)
+
+ /* Enable paging to see if encryption is active */
+ movl %cr0, %edx /* Backup %cr0 in %edx */
+ movl $(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */
+ movl %ecx, %cr0
+
+ cmpl %eax, rva(sev_check_data)(%ebp)
+ jne 3f
+ cmpl %ebx, rva(sev_check_data+4)(%ebp)
+ jne 3f
+
+ movl %edx, %cr0 /* Restore previous %cr0 */
+
+ jmp 4f
+
+3: /* Check failed - hlt the machine */
+ hlt
+ jmp 3b
+
+4:
+ popl %edx
+ popl %ecx
+ popl %ebx
+ popl %eax
+#endif
+ ret
+SYM_FUNC_END(startup32_check_sev_cbit)
+
/*
* Stack and heap for uncompression
*/
--
2.30.1
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [PATCH v3 8/8] x86/sev-es: Replace open-coded hlt-loops with sev_es_terminate()
2021-03-12 12:38 [PATCH v3 0/8] x86/seves: Support 32-bit boot path and other updates Joerg Roedel
` (6 preceding siblings ...)
2021-03-12 12:38 ` [PATCH v3 7/8] x86/boot/compressed/64: Check SEV encryption in " Joerg Roedel
@ 2021-03-12 12:38 ` Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
7 siblings, 2 replies; 22+ messages in thread
From: Joerg Roedel @ 2021-03-12 12:38 UTC (permalink / raw)
To: x86
Cc: Joerg Roedel, Joerg Roedel, hpa, Andy Lutomirski, Dave Hansen,
Peter Zijlstra, Jiri Slaby, Dan Williams, Tom Lendacky,
Juergen Gross, Kees Cook, David Rientjes, Cfir Cohen,
Erdem Aktas, Masami Hiramatsu, Mike Stunes, Sean Christopherson,
Martin Radev, Arvind Sankar, linux-kernel, kvm, virtualization
From: Joerg Roedel <jroedel@suse.de>
There are a few places left in the SEV-ES C code where hlt loops and/or
terminate requests are implemented. Replace them all with calls to
sev_es_terminate().
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
arch/x86/boot/compressed/sev-es.c | 12 +++---------
arch/x86/kernel/sev-es-shared.c | 10 +++-------
2 files changed, 6 insertions(+), 16 deletions(-)
diff --git a/arch/x86/boot/compressed/sev-es.c b/arch/x86/boot/compressed/sev-es.c
index 27826c265aab..d904bd56b3e3 100644
--- a/arch/x86/boot/compressed/sev-es.c
+++ b/arch/x86/boot/compressed/sev-es.c
@@ -200,14 +200,8 @@ void do_boot_stage2_vc(struct pt_regs *regs, unsigned long exit_code)
}
finish:
- if (result == ES_OK) {
+ if (result == ES_OK)
vc_finish_insn(&ctxt);
- } else if (result != ES_RETRY) {
- /*
- * For now, just halt the machine. That makes debugging easier,
- * later we just call sev_es_terminate() here.
- */
- while (true)
- asm volatile("hlt\n");
- }
+ else if (result != ES_RETRY)
+ sev_es_terminate(GHCB_SEV_ES_REASON_GENERAL_REQUEST);
}
diff --git a/arch/x86/kernel/sev-es-shared.c b/arch/x86/kernel/sev-es-shared.c
index 387b71669818..0aa9f13efd57 100644
--- a/arch/x86/kernel/sev-es-shared.c
+++ b/arch/x86/kernel/sev-es-shared.c
@@ -24,7 +24,7 @@ static bool __init sev_es_check_cpu_features(void)
return true;
}
-static void sev_es_terminate(unsigned int reason)
+static void __noreturn sev_es_terminate(unsigned int reason)
{
u64 val = GHCB_SEV_TERMINATE;
@@ -206,12 +206,8 @@ void __init do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code)
return;
fail:
- sev_es_wr_ghcb_msr(GHCB_SEV_TERMINATE);
- VMGEXIT();
-
- /* Shouldn't get here - if we do halt the machine */
- while (true)
- asm volatile("hlt\n");
+ /* Terminate the guest */
+ sev_es_terminate(GHCB_SEV_ES_REASON_GENERAL_REQUEST);
}
static enum es_result vc_insn_string_read(struct es_em_ctxt *ctxt,
--
2.30.1
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: [PATCH v3 2/8] x86/sev: Do not require Hypervisor CPUID bit for SEV guests
2021-03-12 12:38 ` [PATCH v3 2/8] x86/sev: Do not require Hypervisor CPUID bit for SEV guests Joerg Roedel
@ 2021-03-17 15:04 ` Tom Lendacky
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
1 sibling, 0 replies; 22+ messages in thread
From: Tom Lendacky @ 2021-03-17 15:04 UTC (permalink / raw)
To: Joerg Roedel, x86
Cc: Joerg Roedel, hpa, Andy Lutomirski, Dave Hansen, Peter Zijlstra,
Jiri Slaby, Dan Williams, Juergen Gross, Kees Cook,
David Rientjes, Cfir Cohen, Erdem Aktas, Masami Hiramatsu,
Mike Stunes, Sean Christopherson, Martin Radev, Arvind Sankar,
linux-kernel, kvm, virtualization
On 3/12/21 6:38 AM, Joerg Roedel wrote:
> From: Joerg Roedel <jroedel@suse.de>
>
> A malicious hypervisor could disable the CPUID intercept for an SEV or
> SEV-ES guest and trick it into the no-SEV boot path, where it could
> potentially reveal secrets. This is not an issue for SEV-SNP guests,
> as the CPUID intercept can't be disabled for those.
>
> Remove the Hypervisor CPUID bit check from the SEV detection code to
> protect against this kind of attack and add a Hypervisor bit equals
> zero check to the SME detection path to prevent non-SEV guests from
> trying to enable SME.
>
> This handles the following cases:
>
> 1) SEV(-ES) guest where CPUID intercept is disabled. The guest
> will still see leaf 0x8000001f and the SEV bit. It can
> retrieve the C-bit and boot normally.
>
> 2) Non-SEV guests with intercepted CPUID will check SEV_STATUS
> MSR and find it 0 and will try to enable SME. This will
> fail when the guest finds MSR_K8_SYSCFG to be zero, as it
> is emulated by KVM. But we can't rely on that, as there
> might be other hypervisors which return this MSR with bit
> 23 set. The Hypervisor bit check will prevent that the
> guest tries to enable SME in this case.
>
> 3) Non-SEV guests on SEV capable hosts with CPUID intercept
> disabled (by a malicious hypervisor) will try to boot into
> the SME path. This will fail, but it is also not considered
> a problem because non-encrypted guests have no protection
> against the hypervisor anyway.
>
> Signed-off-by: Joerg Roedel <jroedel@suse.de>
Acked-by: Tom Lendacky <thomas.lendacky@amd.com>
> ---
> arch/x86/boot/compressed/mem_encrypt.S | 6 -----
> arch/x86/kernel/sev-es-shared.c | 6 +----
> arch/x86/mm/mem_encrypt_identity.c | 35 ++++++++++++++------------
> 3 files changed, 20 insertions(+), 27 deletions(-)
>
> diff --git a/arch/x86/boot/compressed/mem_encrypt.S b/arch/x86/boot/compressed/mem_encrypt.S
> index aa561795efd1..a6dea4e8a082 100644
> --- a/arch/x86/boot/compressed/mem_encrypt.S
> +++ b/arch/x86/boot/compressed/mem_encrypt.S
> @@ -23,12 +23,6 @@ SYM_FUNC_START(get_sev_encryption_bit)
> push %ecx
> push %edx
>
> - /* Check if running under a hypervisor */
> - movl $1, %eax
> - cpuid
> - bt $31, %ecx /* Check the hypervisor bit */
> - jnc .Lno_sev
> -
> movl $0x80000000, %eax /* CPUID to check the highest leaf */
> cpuid
> cmpl $0x8000001f, %eax /* See if 0x8000001f is available */
> diff --git a/arch/x86/kernel/sev-es-shared.c b/arch/x86/kernel/sev-es-shared.c
> index cdc04d091242..387b71669818 100644
> --- a/arch/x86/kernel/sev-es-shared.c
> +++ b/arch/x86/kernel/sev-es-shared.c
> @@ -186,7 +186,6 @@ void __init do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code)
> * make it accessible to the hypervisor.
> *
> * In particular, check for:
> - * - Hypervisor CPUID bit
> * - Availability of CPUID leaf 0x8000001f
> * - SEV CPUID bit.
> *
> @@ -194,10 +193,7 @@ void __init do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code)
> * can't be checked here.
> */
>
> - if ((fn == 1 && !(regs->cx & BIT(31))))
> - /* Hypervisor bit */
> - goto fail;
> - else if (fn == 0x80000000 && (regs->ax < 0x8000001f))
> + if (fn == 0x80000000 && (regs->ax < 0x8000001f))
> /* SEV leaf check */
> goto fail;
> else if ((fn == 0x8000001f && !(regs->ax & BIT(1))))
> diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c
> index 6c5eb6f3f14f..a19374d26101 100644
> --- a/arch/x86/mm/mem_encrypt_identity.c
> +++ b/arch/x86/mm/mem_encrypt_identity.c
> @@ -503,14 +503,10 @@ void __init sme_enable(struct boot_params *bp)
>
> #define AMD_SME_BIT BIT(0)
> #define AMD_SEV_BIT BIT(1)
> - /*
> - * Set the feature mask (SME or SEV) based on whether we are
> - * running under a hypervisor.
> - */
> - eax = 1;
> - ecx = 0;
> - native_cpuid(&eax, &ebx, &ecx, &edx);
> - feature_mask = (ecx & BIT(31)) ? AMD_SEV_BIT : AMD_SME_BIT;
> +
> + /* Check the SEV MSR whether SEV or SME is enabled */
> + sev_status = __rdmsr(MSR_AMD64_SEV);
> + feature_mask = (sev_status & MSR_AMD64_SEV_ENABLED) ? AMD_SEV_BIT : AMD_SME_BIT;
>
> /*
> * Check for the SME/SEV feature:
> @@ -530,19 +526,26 @@ void __init sme_enable(struct boot_params *bp)
>
> /* Check if memory encryption is enabled */
> if (feature_mask == AMD_SME_BIT) {
> + /*
> + * No SME if Hypervisor bit is set. This check is here to
> + * prevent a guest from trying to enable SME. For running as a
> + * KVM guest the MSR_K8_SYSCFG will be sufficient, but there
> + * might be other hypervisors which emulate that MSR as non-zero
> + * or even pass it through to the guest.
> + * A malicious hypervisor can still trick a guest into this
> + * path, but there is no way to protect against that.
> + */
> + eax = 1;
> + ecx = 0;
> + native_cpuid(&eax, &ebx, &ecx, &edx);
> + if (ecx & BIT(31))
> + return;
> +
> /* For SME, check the SYSCFG MSR */
> msr = __rdmsr(MSR_K8_SYSCFG);
> if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
> return;
> } else {
> - /* For SEV, check the SEV MSR */
> - msr = __rdmsr(MSR_AMD64_SEV);
> - if (!(msr & MSR_AMD64_SEV_ENABLED))
> - return;
> -
> - /* Save SEV_STATUS to avoid reading MSR again */
> - sev_status = msr;
> -
> /* SEV state cannot be controlled by a command line option */
> sme_me_mask = me_mask;
> sev_enabled = true;
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [tip: x86/seves] x86/sev-es: Replace open-coded hlt-loops with sev_es_terminate()
2021-03-12 12:38 ` [PATCH v3 8/8] x86/sev-es: Replace open-coded hlt-loops with sev_es_terminate() Joerg Roedel
@ 2021-03-18 19:34 ` tip-bot2 for Joerg Roedel
2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
1 sibling, 0 replies; 22+ messages in thread
From: tip-bot2 for Joerg Roedel @ 2021-03-18 19:34 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Joerg Roedel, Borislav Petkov, x86, linux-kernel
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: 4fbe2c3b9dd04f44608d710ad2ae83d7f1c04182
Gitweb: https://git.kernel.org/tip/4fbe2c3b9dd04f44608d710ad2ae83d7f1c04182
Author: Joerg Roedel <jroedel@suse.de>
AuthorDate: Fri, 12 Mar 2021 13:38:24 +01:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 18 Mar 2021 16:44:59 +01:00
x86/sev-es: Replace open-coded hlt-loops with sev_es_terminate()
There are a few places left in the SEV-ES C code where hlt loops and/or
terminate requests are implemented. Replace them all with calls to
sev_es_terminate().
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210312123824.306-9-joro@8bytes.org
---
arch/x86/boot/compressed/sev-es.c | 12 +++---------
arch/x86/kernel/sev-es-shared.c | 10 +++-------
2 files changed, 6 insertions(+), 16 deletions(-)
diff --git a/arch/x86/boot/compressed/sev-es.c b/arch/x86/boot/compressed/sev-es.c
index 27826c2..d904bd5 100644
--- a/arch/x86/boot/compressed/sev-es.c
+++ b/arch/x86/boot/compressed/sev-es.c
@@ -200,14 +200,8 @@ void do_boot_stage2_vc(struct pt_regs *regs, unsigned long exit_code)
}
finish:
- if (result == ES_OK) {
+ if (result == ES_OK)
vc_finish_insn(&ctxt);
- } else if (result != ES_RETRY) {
- /*
- * For now, just halt the machine. That makes debugging easier,
- * later we just call sev_es_terminate() here.
- */
- while (true)
- asm volatile("hlt\n");
- }
+ else if (result != ES_RETRY)
+ sev_es_terminate(GHCB_SEV_ES_REASON_GENERAL_REQUEST);
}
diff --git a/arch/x86/kernel/sev-es-shared.c b/arch/x86/kernel/sev-es-shared.c
index 387b716..0aa9f13 100644
--- a/arch/x86/kernel/sev-es-shared.c
+++ b/arch/x86/kernel/sev-es-shared.c
@@ -24,7 +24,7 @@ static bool __init sev_es_check_cpu_features(void)
return true;
}
-static void sev_es_terminate(unsigned int reason)
+static void __noreturn sev_es_terminate(unsigned int reason)
{
u64 val = GHCB_SEV_TERMINATE;
@@ -206,12 +206,8 @@ void __init do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code)
return;
fail:
- sev_es_wr_ghcb_msr(GHCB_SEV_TERMINATE);
- VMGEXIT();
-
- /* Shouldn't get here - if we do halt the machine */
- while (true)
- asm volatile("hlt\n");
+ /* Terminate the guest */
+ sev_es_terminate(GHCB_SEV_ES_REASON_GENERAL_REQUEST);
}
static enum es_result vc_insn_string_read(struct es_em_ctxt *ctxt,
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [tip: x86/seves] x86/boot/compressed/64: Check SEV encryption in the 32-bit boot-path
2021-03-12 12:38 ` [PATCH v3 7/8] x86/boot/compressed/64: Check SEV encryption in " Joerg Roedel
@ 2021-03-18 19:34 ` tip-bot2 for Joerg Roedel
2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
1 sibling, 0 replies; 22+ messages in thread
From: tip-bot2 for Joerg Roedel @ 2021-03-18 19:34 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Joerg Roedel, Borislav Petkov, x86, linux-kernel
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: 769eb023aa77062cf15c2a179fc8d13b43422c9b
Gitweb: https://git.kernel.org/tip/769eb023aa77062cf15c2a179fc8d13b43422c9b
Author: Joerg Roedel <jroedel@suse.de>
AuthorDate: Fri, 12 Mar 2021 13:38:23 +01:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 18 Mar 2021 16:44:54 +01:00
x86/boot/compressed/64: Check SEV encryption in the 32-bit boot-path
Check whether the hypervisor reported the correct C-bit when running
as an SEV guest. Using a wrong C-bit position could be used to leak
sensitive data from the guest to the hypervisor.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210312123824.306-8-joro@8bytes.org
---
arch/x86/boot/compressed/head_64.S | 83 +++++++++++++++++++++++++++++-
1 file changed, 83 insertions(+)
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index ee448ae..91ea0d5 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -183,11 +183,21 @@ SYM_FUNC_START(startup_32)
*/
call get_sev_encryption_bit
xorl %edx, %edx
+#ifdef CONFIG_AMD_MEM_ENCRYPT
testl %eax, %eax
jz 1f
subl $32, %eax /* Encryption bit is always above bit 31 */
bts %eax, %edx /* Set encryption mask for page tables */
+ /*
+ * Mark SEV as active in sev_status so that startup32_check_sev_cbit()
+ * will do a check. The sev_status memory will be fully initialized
+ * with the contents of MSR_AMD_SEV_STATUS later in
+ * set_sev_encryption_mask(). For now it is sufficient to know that SEV
+ * is active.
+ */
+ movl $1, rva(sev_status)(%ebp)
1:
+#endif
/* Initialize Page tables to 0 */
leal rva(pgtable)(%ebx), %edi
@@ -272,6 +282,9 @@ SYM_FUNC_START(startup_32)
movl %esi, %edx
1:
#endif
+ /* Check if the C-bit position is correct when SEV is active */
+ call startup32_check_sev_cbit
+
pushl $__KERNEL_CS
pushl %eax
@@ -872,6 +885,76 @@ SYM_FUNC_START(startup32_load_idt)
SYM_FUNC_END(startup32_load_idt)
/*
+ * Check for the correct C-bit position when the startup_32 boot-path is used.
+ *
+ * The check makes use of the fact that all memory is encrypted when paging is
+ * disabled. The function creates 64 bits of random data using the RDRAND
+ * instruction. RDRAND is mandatory for SEV guests, so always available. If the
+ * hypervisor violates that the kernel will crash right here.
+ *
+ * The 64 bits of random data are stored to a memory location and at the same
+ * time kept in the %eax and %ebx registers. Since encryption is always active
+ * when paging is off the random data will be stored encrypted in main memory.
+ *
+ * Then paging is enabled. When the C-bit position is correct all memory is
+ * still mapped encrypted and comparing the register values with memory will
+ * succeed. An incorrect C-bit position will map all memory unencrypted, so that
+ * the compare will use the encrypted random data and fail.
+ */
+SYM_FUNC_START(startup32_check_sev_cbit)
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+ pushl %eax
+ pushl %ebx
+ pushl %ecx
+ pushl %edx
+
+ /* Check for non-zero sev_status */
+ movl rva(sev_status)(%ebp), %eax
+ testl %eax, %eax
+ jz 4f
+
+ /*
+ * Get two 32-bit random values - Don't bail out if RDRAND fails
+ * because it is better to prevent forward progress if no random value
+ * can be gathered.
+ */
+1: rdrand %eax
+ jnc 1b
+2: rdrand %ebx
+ jnc 2b
+
+ /* Store to memory and keep it in the registers */
+ movl %eax, rva(sev_check_data)(%ebp)
+ movl %ebx, rva(sev_check_data+4)(%ebp)
+
+ /* Enable paging to see if encryption is active */
+ movl %cr0, %edx /* Backup %cr0 in %edx */
+ movl $(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */
+ movl %ecx, %cr0
+
+ cmpl %eax, rva(sev_check_data)(%ebp)
+ jne 3f
+ cmpl %ebx, rva(sev_check_data+4)(%ebp)
+ jne 3f
+
+ movl %edx, %cr0 /* Restore previous %cr0 */
+
+ jmp 4f
+
+3: /* Check failed - hlt the machine */
+ hlt
+ jmp 3b
+
+4:
+ popl %edx
+ popl %ecx
+ popl %ebx
+ popl %eax
+#endif
+ ret
+SYM_FUNC_END(startup32_check_sev_cbit)
+
+/*
* Stack and heap for uncompression
*/
.bss
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [tip: x86/seves] x86/boot/compressed/64: Add CPUID sanity check to 32-bit boot-path
2021-03-12 12:38 ` [PATCH v3 6/8] x86/boot/compressed/64: Add CPUID sanity check to 32-bit boot-path Joerg Roedel
@ 2021-03-18 19:34 ` tip-bot2 for Joerg Roedel
2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
1 sibling, 0 replies; 22+ messages in thread
From: tip-bot2 for Joerg Roedel @ 2021-03-18 19:34 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Joerg Roedel, Borislav Petkov, x86, linux-kernel
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: 0dd986f3a1e31bd827d2f1f52f07c8a82cd83143
Gitweb: https://git.kernel.org/tip/0dd986f3a1e31bd827d2f1f52f07c8a82cd83143
Author: Joerg Roedel <jroedel@suse.de>
AuthorDate: Fri, 12 Mar 2021 13:38:22 +01:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 18 Mar 2021 16:44:52 +01:00
x86/boot/compressed/64: Add CPUID sanity check to 32-bit boot-path
The 32-bit #VC handler has no GHCB and can only handle CPUID exit codes.
It is needed by the early boot code to handle #VC exceptions raised in
verify_cpu() and to get the position of the C-bit.
But the CPUID information comes from the hypervisor which is untrusted
and might return results which trick the guest into the no-SEV boot path
with no C-bit set in the page-tables. All data written to memory would
then be unencrypted and could leak sensitive data to the hypervisor.
Add sanity checks to the 32-bit boot #VC handler to make sure the
hypervisor does not pretend that SEV is not enabled.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210312123824.306-7-joro@8bytes.org
---
arch/x86/boot/compressed/mem_encrypt.S | 28 +++++++++++++++++++++++++-
1 file changed, 28 insertions(+)
diff --git a/arch/x86/boot/compressed/mem_encrypt.S b/arch/x86/boot/compressed/mem_encrypt.S
index ebc4a29..c1e81a8 100644
--- a/arch/x86/boot/compressed/mem_encrypt.S
+++ b/arch/x86/boot/compressed/mem_encrypt.S
@@ -139,6 +139,26 @@ SYM_CODE_START(startup32_vc_handler)
jnz .Lfail
movl %edx, 0(%esp) # Store result
+ /*
+ * Sanity check CPUID results from the Hypervisor. See comment in
+ * do_vc_no_ghcb() for more details on why this is necessary.
+ */
+
+ /* Fail if SEV leaf not available in CPUID[0x80000000].EAX */
+ cmpl $0x80000000, %ebx
+ jne .Lcheck_sev
+ cmpl $0x8000001f, 12(%esp)
+ jb .Lfail
+ jmp .Ldone
+
+.Lcheck_sev:
+ /* Fail if SEV bit not set in CPUID[0x8000001f].EAX[1] */
+ cmpl $0x8000001f, %ebx
+ jne .Ldone
+ btl $1, 12(%esp)
+ jnc .Lfail
+
+.Ldone:
popl %edx
popl %ecx
popl %ebx
@@ -152,6 +172,14 @@ SYM_CODE_START(startup32_vc_handler)
iret
.Lfail:
+ /* Send terminate request to Hypervisor */
+ movl $0x100, %eax
+ xorl %edx, %edx
+ movl $MSR_AMD64_SEV_ES_GHCB, %ecx
+ wrmsr
+ rep; vmmcall
+
+ /* If request fails, go to hlt loop */
hlt
jmp .Lfail
SYM_CODE_END(startup32_vc_handler)
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [tip: x86/seves] x86/boot/compressed/64: Reload CS in startup_32
2021-03-12 12:38 ` [PATCH v3 3/8] x86/boot/compressed/64: Reload CS in startup_32 Joerg Roedel
@ 2021-03-18 19:34 ` tip-bot2 for Joerg Roedel
0 siblings, 0 replies; 22+ messages in thread
From: tip-bot2 for Joerg Roedel @ 2021-03-18 19:34 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Joerg Roedel, Borislav Petkov, x86, linux-kernel
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: 0c289ff81c24033777fab23019039f11e1449ba4
Gitweb: https://git.kernel.org/tip/0c289ff81c24033777fab23019039f11e1449ba4
Author: Joerg Roedel <jroedel@suse.de>
AuthorDate: Wed, 10 Mar 2021 09:43:20 +01:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 18 Mar 2021 16:44:43 +01:00
x86/boot/compressed/64: Reload CS in startup_32
Exception handling in the startup_32 boot path requires the CS
selector to be correctly set up. Reload it from the current GDT.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210312123824.306-4-joro@8bytes.org
---
arch/x86/boot/compressed/head_64.S | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index e94874f..c59c80c 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -107,9 +107,16 @@ SYM_FUNC_START(startup_32)
movl %eax, %gs
movl %eax, %ss
-/* setup a stack and make sure cpu supports long mode. */
+ /* Setup a stack and load CS from current GDT */
leal rva(boot_stack_end)(%ebp), %esp
+ pushl $__KERNEL32_CS
+ leal rva(1f)(%ebp), %eax
+ pushl %eax
+ lretl
+1:
+
+ /* Make sure cpu supports long mode. */
call verify_cpu
testl %eax, %eax
jnz .Lno_longmode
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [tip: x86/seves] x86/boot/compressed/64: Add 32-bit boot #VC handler
2021-03-12 12:38 ` [PATCH v3 5/8] x86/boot/compressed/64: Add 32-bit boot #VC handler Joerg Roedel
@ 2021-03-18 19:34 ` tip-bot2 for Joerg Roedel
2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
1 sibling, 0 replies; 22+ messages in thread
From: tip-bot2 for Joerg Roedel @ 2021-03-18 19:34 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Joerg Roedel, Borislav Petkov, x86, linux-kernel
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: 9e373ba233b236a831d0d9578be095a4b7435abe
Gitweb: https://git.kernel.org/tip/9e373ba233b236a831d0d9578be095a4b7435abe
Author: Joerg Roedel <jroedel@suse.de>
AuthorDate: Wed, 10 Mar 2021 09:43:22 +01:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 18 Mar 2021 16:44:48 +01:00
x86/boot/compressed/64: Add 32-bit boot #VC handler
Add a #VC exception handler which is used when the kernel still executes
in protected mode. This boot-path already uses CPUID, which will cause
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210312123824.306-6-joro@8bytes.org
---
arch/x86/boot/compressed/head_64.S | 6 ++-
arch/x86/boot/compressed/mem_encrypt.S | 96 ++++++++++++++++++++++++-
2 files changed, 101 insertions(+), 1 deletion(-)
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index 2001c3b..ee448ae 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -34,6 +34,7 @@
#include <asm/asm-offsets.h>
#include <asm/bootparam.h>
#include <asm/desc_defs.h>
+#include <asm/trapnr.h>
#include "pgtable.h"
/*
@@ -857,6 +858,11 @@ SYM_FUNC_END(startup32_set_idt_entry)
SYM_FUNC_START(startup32_load_idt)
#ifdef CONFIG_AMD_MEM_ENCRYPT
+ /* #VC handler */
+ leal rva(startup32_vc_handler)(%ebp), %eax
+ movl $X86_TRAP_VC, %edx
+ call startup32_set_idt_entry
+
/* Load IDT */
leal rva(boot32_idt)(%ebp), %eax
movl %eax, rva(boot32_idt_desc+2)(%ebp)
diff --git a/arch/x86/boot/compressed/mem_encrypt.S b/arch/x86/boot/compressed/mem_encrypt.S
index a6dea4e..ebc4a29 100644
--- a/arch/x86/boot/compressed/mem_encrypt.S
+++ b/arch/x86/boot/compressed/mem_encrypt.S
@@ -61,10 +61,104 @@ SYM_FUNC_START(get_sev_encryption_bit)
ret
SYM_FUNC_END(get_sev_encryption_bit)
+/**
+ * sev_es_req_cpuid - Request a CPUID value from the Hypervisor using
+ * the GHCB MSR protocol
+ *
+ * @%eax: Register to request (0=EAX, 1=EBX, 2=ECX, 3=EDX)
+ * @%edx: CPUID Function
+ *
+ * Returns 0 in %eax on success, non-zero on failure
+ * %edx returns CPUID value on success
+ */
+SYM_CODE_START_LOCAL(sev_es_req_cpuid)
+ shll $30, %eax
+ orl $0x00000004, %eax
+ movl $MSR_AMD64_SEV_ES_GHCB, %ecx
+ wrmsr
+ rep; vmmcall # VMGEXIT
+ rdmsr
+
+ /* Check response */
+ movl %eax, %ecx
+ andl $0x3ffff000, %ecx # Bits [12-29] MBZ
+ jnz 2f
+
+ /* Check return code */
+ andl $0xfff, %eax
+ cmpl $5, %eax
+ jne 2f
+
+ /* All good - return success */
+ xorl %eax, %eax
+1:
+ ret
+2:
+ movl $-1, %eax
+ jmp 1b
+SYM_CODE_END(sev_es_req_cpuid)
+
+SYM_CODE_START(startup32_vc_handler)
+ pushl %eax
+ pushl %ebx
+ pushl %ecx
+ pushl %edx
+
+ /* Keep CPUID function in %ebx */
+ movl %eax, %ebx
+
+ /* Check if error-code == SVM_EXIT_CPUID */
+ cmpl $0x72, 16(%esp)
+ jne .Lfail
+
+ movl $0, %eax # Request CPUID[fn].EAX
+ movl %ebx, %edx # CPUID fn
+ call sev_es_req_cpuid # Call helper
+ testl %eax, %eax # Check return code
+ jnz .Lfail
+ movl %edx, 12(%esp) # Store result
+
+ movl $1, %eax # Request CPUID[fn].EBX
+ movl %ebx, %edx # CPUID fn
+ call sev_es_req_cpuid # Call helper
+ testl %eax, %eax # Check return code
+ jnz .Lfail
+ movl %edx, 8(%esp) # Store result
+
+ movl $2, %eax # Request CPUID[fn].ECX
+ movl %ebx, %edx # CPUID fn
+ call sev_es_req_cpuid # Call helper
+ testl %eax, %eax # Check return code
+ jnz .Lfail
+ movl %edx, 4(%esp) # Store result
+
+ movl $3, %eax # Request CPUID[fn].EDX
+ movl %ebx, %edx # CPUID fn
+ call sev_es_req_cpuid # Call helper
+ testl %eax, %eax # Check return code
+ jnz .Lfail
+ movl %edx, 0(%esp) # Store result
+
+ popl %edx
+ popl %ecx
+ popl %ebx
+ popl %eax
+
+ /* Remove error code */
+ addl $4, %esp
+
+ /* Jump over CPUID instruction */
+ addl $2, (%esp)
+
+ iret
+.Lfail:
+ hlt
+ jmp .Lfail
+SYM_CODE_END(startup32_vc_handler)
+
.code64
#include "../../kernel/sev_verify_cbit.S"
-
SYM_FUNC_START(set_sev_encryption_mask)
#ifdef CONFIG_AMD_MEM_ENCRYPT
push %rbp
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [tip: x86/seves] x86/boot/compressed/64: Setup IDT in startup_32 boot path
2021-03-12 12:38 ` [PATCH v3 4/8] x86/boot/compressed/64: Setup IDT in startup_32 boot path Joerg Roedel
@ 2021-03-18 19:34 ` tip-bot2 for Joerg Roedel
0 siblings, 0 replies; 22+ messages in thread
From: tip-bot2 for Joerg Roedel @ 2021-03-18 19:34 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Joerg Roedel, Borislav Petkov, x86, linux-kernel
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: 79419e13e8082cc15d174df979a363528e31f2e7
Gitweb: https://git.kernel.org/tip/79419e13e8082cc15d174df979a363528e31f2e7
Author: Joerg Roedel <jroedel@suse.de>
AuthorDate: Wed, 10 Mar 2021 09:43:21 +01:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 18 Mar 2021 16:44:46 +01:00
x86/boot/compressed/64: Setup IDT in startup_32 boot path
This boot path needs exception handling when it is used with SEV-ES.
Setup an IDT and provide a helper function to write IDT entries for
use in 32-bit protected mode.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210312123824.306-5-joro@8bytes.org
---
arch/x86/boot/compressed/head_64.S | 72 +++++++++++++++++++++++++++++-
1 file changed, 72 insertions(+)
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index c59c80c..2001c3b 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -116,6 +116,9 @@ SYM_FUNC_START(startup_32)
lretl
1:
+ /* Setup Exception handling for SEV-ES */
+ call startup32_load_idt
+
/* Make sure cpu supports long mode. */
call verify_cpu
testl %eax, %eax
@@ -701,6 +704,19 @@ SYM_DATA_START(boot_idt)
.endr
SYM_DATA_END_LABEL(boot_idt, SYM_L_GLOBAL, boot_idt_end)
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+SYM_DATA_START(boot32_idt_desc)
+ .word boot32_idt_end - boot32_idt - 1
+ .long 0
+SYM_DATA_END(boot32_idt_desc)
+ .balign 8
+SYM_DATA_START(boot32_idt)
+ .rept 32
+ .quad 0
+ .endr
+SYM_DATA_END_LABEL(boot32_idt, SYM_L_GLOBAL, boot32_idt_end)
+#endif
+
#ifdef CONFIG_EFI_STUB
SYM_DATA(image_offset, .long 0)
#endif
@@ -793,6 +809,62 @@ SYM_DATA_START_LOCAL(loaded_image_proto)
SYM_DATA_END(loaded_image_proto)
#endif
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+ __HEAD
+ .code32
+/*
+ * Write an IDT entry into boot32_idt
+ *
+ * Parameters:
+ *
+ * %eax: Handler address
+ * %edx: Vector number
+ *
+ * Physical offset is expected in %ebp
+ */
+SYM_FUNC_START(startup32_set_idt_entry)
+ push %ebx
+ push %ecx
+
+ /* IDT entry address to %ebx */
+ leal rva(boot32_idt)(%ebp), %ebx
+ shl $3, %edx
+ addl %edx, %ebx
+
+ /* Build IDT entry, lower 4 bytes */
+ movl %eax, %edx
+ andl $0x0000ffff, %edx # Target code segment offset [15:0]
+ movl $__KERNEL32_CS, %ecx # Target code segment selector
+ shl $16, %ecx
+ orl %ecx, %edx
+
+ /* Store lower 4 bytes to IDT */
+ movl %edx, (%ebx)
+
+ /* Build IDT entry, upper 4 bytes */
+ movl %eax, %edx
+ andl $0xffff0000, %edx # Target code segment offset [31:16]
+ orl $0x00008e00, %edx # Present, Type 32-bit Interrupt Gate
+
+ /* Store upper 4 bytes to IDT */
+ movl %edx, 4(%ebx)
+
+ pop %ecx
+ pop %ebx
+ ret
+SYM_FUNC_END(startup32_set_idt_entry)
+#endif
+
+SYM_FUNC_START(startup32_load_idt)
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+ /* Load IDT */
+ leal rva(boot32_idt)(%ebp), %eax
+ movl %eax, rva(boot32_idt_desc+2)(%ebp)
+ lidt rva(boot32_idt_desc)(%ebp)
+#endif
+ ret
+SYM_FUNC_END(startup32_load_idt)
+
/*
* Stack and heap for uncompression
*/
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [tip: x86/seves] x86/boot/compressed/64: Cleanup exception handling before booting kernel
2021-03-12 12:38 ` [PATCH v3 1/8] x86/boot/compressed/64: Cleanup exception handling before booting kernel Joerg Roedel
@ 2021-03-18 19:34 ` tip-bot2 for Joerg Roedel
0 siblings, 0 replies; 22+ messages in thread
From: tip-bot2 for Joerg Roedel @ 2021-03-18 19:34 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Joerg Roedel, Borislav Petkov, x86, linux-kernel
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: b099155e2df7dadf8b1ad9828158b89f5639f654
Gitweb: https://git.kernel.org/tip/b099155e2df7dadf8b1ad9828158b89f5639f654
Author: Joerg Roedel <jroedel@suse.de>
AuthorDate: Wed, 10 Mar 2021 09:43:19 +01:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 18 Mar 2021 16:44:36 +01:00
x86/boot/compressed/64: Cleanup exception handling before booting kernel
Disable the exception handling before booting the kernel to make sure
any exceptions that happen during early kernel boot are not directed to
the pre-decompression code.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210312123824.306-2-joro@8bytes.org
---
arch/x86/boot/compressed/idt_64.c | 14 ++++++++++++++
arch/x86/boot/compressed/misc.c | 7 ++-----
arch/x86/boot/compressed/misc.h | 6 ++++++
3 files changed, 22 insertions(+), 5 deletions(-)
diff --git a/arch/x86/boot/compressed/idt_64.c b/arch/x86/boot/compressed/idt_64.c
index 804a502..9b93567 100644
--- a/arch/x86/boot/compressed/idt_64.c
+++ b/arch/x86/boot/compressed/idt_64.c
@@ -52,3 +52,17 @@ void load_stage2_idt(void)
load_boot_idt(&boot_idt_desc);
}
+
+void cleanup_exception_handling(void)
+{
+ /*
+ * Flush GHCB from cache and map it encrypted again when running as
+ * SEV-ES guest.
+ */
+ sev_es_shutdown_ghcb();
+
+ /* Set a null-idt, disabling #PF and #VC handling */
+ boot_idt_desc.size = 0;
+ boot_idt_desc.address = 0;
+ load_boot_idt(&boot_idt_desc);
+}
diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c
index 267e7f9..cc9fd0e 100644
--- a/arch/x86/boot/compressed/misc.c
+++ b/arch/x86/boot/compressed/misc.c
@@ -443,11 +443,8 @@ asmlinkage __visible void *extract_kernel(void *rmode, memptr heap,
handle_relocations(output, output_len, virt_addr);
debug_putstr("done.\nBooting the kernel.\n");
- /*
- * Flush GHCB from cache and map it encrypted again when running as
- * SEV-ES guest.
- */
- sev_es_shutdown_ghcb();
+ /* Disable exception handling before booting the kernel */
+ cleanup_exception_handling();
return output;
}
diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/misc.h
index 901ea5e..e5612f0 100644
--- a/arch/x86/boot/compressed/misc.h
+++ b/arch/x86/boot/compressed/misc.h
@@ -155,6 +155,12 @@ extern pteval_t __default_kernel_pte_mask;
extern gate_desc boot_idt[BOOT_IDT_ENTRIES];
extern struct desc_ptr boot_idt_desc;
+#ifdef CONFIG_X86_64
+void cleanup_exception_handling(void);
+#else
+static inline void cleanup_exception_handling(void) { }
+#endif
+
/* IDT Entry Points */
void boot_page_fault(void);
void boot_stage1_vc(void);
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [tip: x86/seves] x86/sev: Do not require Hypervisor CPUID bit for SEV guests
2021-03-12 12:38 ` [PATCH v3 2/8] x86/sev: Do not require Hypervisor CPUID bit for SEV guests Joerg Roedel
2021-03-17 15:04 ` Tom Lendacky
@ 2021-03-18 19:34 ` tip-bot2 for Joerg Roedel
1 sibling, 0 replies; 22+ messages in thread
From: tip-bot2 for Joerg Roedel @ 2021-03-18 19:34 UTC (permalink / raw)
To: linux-tip-commits
Cc: Joerg Roedel, Borislav Petkov, Tom Lendacky, x86, linux-kernel
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: eab696d8e8b9c9d600be6fad8dd8dfdfaca6ca7c
Gitweb: https://git.kernel.org/tip/eab696d8e8b9c9d600be6fad8dd8dfdfaca6ca7c
Author: Joerg Roedel <jroedel@suse.de>
AuthorDate: Fri, 12 Mar 2021 13:38:18 +01:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 18 Mar 2021 16:44:40 +01:00
x86/sev: Do not require Hypervisor CPUID bit for SEV guests
A malicious hypervisor could disable the CPUID intercept for an SEV or
SEV-ES guest and trick it into the no-SEV boot path, where it could
potentially reveal secrets. This is not an issue for SEV-SNP guests,
as the CPUID intercept can't be disabled for those.
Remove the Hypervisor CPUID bit check from the SEV detection code to
protect against this kind of attack and add a Hypervisor bit equals zero
check to the SME detection path to prevent non-encrypted guests from
trying to enable SME.
This handles the following cases:
1) SEV(-ES) guest where CPUID intercept is disabled. The guest
will still see leaf 0x8000001f and the SEV bit. It can
retrieve the C-bit and boot normally.
2) Non-encrypted guests with intercepted CPUID will check
the SEV_STATUS MSR and find it 0 and will try to enable SME.
This will fail when the guest finds MSR_K8_SYSCFG to be zero,
as it is emulated by KVM. But we can't rely on that, as there
might be other hypervisors which return this MSR with bit
23 set. The Hypervisor bit check will prevent that the guest
tries to enable SME in this case.
3) Non-encrypted guests on SEV capable hosts with CPUID intercept
disabled (by a malicious hypervisor) will try to boot into
the SME path. This will fail, but it is also not considered
a problem because non-encrypted guests have no protection
against the hypervisor anyway.
[ bp: s/non-SEV/non-encrypted/g ]
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Acked-by: Tom Lendacky <thomas.lendacky@amd.com>
Link: https://lkml.kernel.org/r/20210312123824.306-3-joro@8bytes.org
---
arch/x86/boot/compressed/mem_encrypt.S | 6 +----
arch/x86/kernel/sev-es-shared.c | 6 +----
arch/x86/mm/mem_encrypt_identity.c | 35 +++++++++++++------------
3 files changed, 20 insertions(+), 27 deletions(-)
diff --git a/arch/x86/boot/compressed/mem_encrypt.S b/arch/x86/boot/compressed/mem_encrypt.S
index aa56179..a6dea4e 100644
--- a/arch/x86/boot/compressed/mem_encrypt.S
+++ b/arch/x86/boot/compressed/mem_encrypt.S
@@ -23,12 +23,6 @@ SYM_FUNC_START(get_sev_encryption_bit)
push %ecx
push %edx
- /* Check if running under a hypervisor */
- movl $1, %eax
- cpuid
- bt $31, %ecx /* Check the hypervisor bit */
- jnc .Lno_sev
-
movl $0x80000000, %eax /* CPUID to check the highest leaf */
cpuid
cmpl $0x8000001f, %eax /* See if 0x8000001f is available */
diff --git a/arch/x86/kernel/sev-es-shared.c b/arch/x86/kernel/sev-es-shared.c
index cdc04d0..387b716 100644
--- a/arch/x86/kernel/sev-es-shared.c
+++ b/arch/x86/kernel/sev-es-shared.c
@@ -186,7 +186,6 @@ void __init do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code)
* make it accessible to the hypervisor.
*
* In particular, check for:
- * - Hypervisor CPUID bit
* - Availability of CPUID leaf 0x8000001f
* - SEV CPUID bit.
*
@@ -194,10 +193,7 @@ void __init do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code)
* can't be checked here.
*/
- if ((fn == 1 && !(regs->cx & BIT(31))))
- /* Hypervisor bit */
- goto fail;
- else if (fn == 0x80000000 && (regs->ax < 0x8000001f))
+ if (fn == 0x80000000 && (regs->ax < 0x8000001f))
/* SEV leaf check */
goto fail;
else if ((fn == 0x8000001f && !(regs->ax & BIT(1))))
diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_identity.c
index 6c5eb6f..a19374d 100644
--- a/arch/x86/mm/mem_encrypt_identity.c
+++ b/arch/x86/mm/mem_encrypt_identity.c
@@ -503,14 +503,10 @@ void __init sme_enable(struct boot_params *bp)
#define AMD_SME_BIT BIT(0)
#define AMD_SEV_BIT BIT(1)
- /*
- * Set the feature mask (SME or SEV) based on whether we are
- * running under a hypervisor.
- */
- eax = 1;
- ecx = 0;
- native_cpuid(&eax, &ebx, &ecx, &edx);
- feature_mask = (ecx & BIT(31)) ? AMD_SEV_BIT : AMD_SME_BIT;
+
+ /* Check the SEV MSR whether SEV or SME is enabled */
+ sev_status = __rdmsr(MSR_AMD64_SEV);
+ feature_mask = (sev_status & MSR_AMD64_SEV_ENABLED) ? AMD_SEV_BIT : AMD_SME_BIT;
/*
* Check for the SME/SEV feature:
@@ -530,19 +526,26 @@ void __init sme_enable(struct boot_params *bp)
/* Check if memory encryption is enabled */
if (feature_mask == AMD_SME_BIT) {
+ /*
+ * No SME if Hypervisor bit is set. This check is here to
+ * prevent a guest from trying to enable SME. For running as a
+ * KVM guest the MSR_K8_SYSCFG will be sufficient, but there
+ * might be other hypervisors which emulate that MSR as non-zero
+ * or even pass it through to the guest.
+ * A malicious hypervisor can still trick a guest into this
+ * path, but there is no way to protect against that.
+ */
+ eax = 1;
+ ecx = 0;
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+ if (ecx & BIT(31))
+ return;
+
/* For SME, check the SYSCFG MSR */
msr = __rdmsr(MSR_K8_SYSCFG);
if (!(msr & MSR_K8_SYSCFG_MEM_ENCRYPT))
return;
} else {
- /* For SEV, check the SEV MSR */
- msr = __rdmsr(MSR_AMD64_SEV);
- if (!(msr & MSR_AMD64_SEV_ENABLED))
- return;
-
- /* Save SEV_STATUS to avoid reading MSR again */
- sev_status = msr;
-
/* SEV state cannot be controlled by a command line option */
sme_me_mask = me_mask;
sev_enabled = true;
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [tip: x86/seves] x86/sev-es: Replace open-coded hlt-loops with sev_es_terminate()
2021-03-12 12:38 ` [PATCH v3 8/8] x86/sev-es: Replace open-coded hlt-loops with sev_es_terminate() Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
@ 2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
1 sibling, 0 replies; 22+ messages in thread
From: tip-bot2 for Joerg Roedel @ 2021-03-18 22:10 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Joerg Roedel, Borislav Petkov, x86, linux-kernel
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: f15a0a732aefb46f999c2a8aa8f9f16e71cec5b2
Gitweb: https://git.kernel.org/tip/f15a0a732aefb46f999c2a8aa8f9f16e71cec5b2
Author: Joerg Roedel <jroedel@suse.de>
AuthorDate: Fri, 12 Mar 2021 13:38:24 +01:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 18 Mar 2021 23:04:12 +01:00
x86/sev-es: Replace open-coded hlt-loops with sev_es_terminate()
There are a few places left in the SEV-ES C code where hlt loops and/or
terminate requests are implemented. Replace them all with calls to
sev_es_terminate().
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210312123824.306-9-joro@8bytes.org
---
arch/x86/boot/compressed/sev-es.c | 12 +++---------
arch/x86/kernel/sev-es-shared.c | 10 +++-------
2 files changed, 6 insertions(+), 16 deletions(-)
diff --git a/arch/x86/boot/compressed/sev-es.c b/arch/x86/boot/compressed/sev-es.c
index 27826c2..d904bd5 100644
--- a/arch/x86/boot/compressed/sev-es.c
+++ b/arch/x86/boot/compressed/sev-es.c
@@ -200,14 +200,8 @@ void do_boot_stage2_vc(struct pt_regs *regs, unsigned long exit_code)
}
finish:
- if (result == ES_OK) {
+ if (result == ES_OK)
vc_finish_insn(&ctxt);
- } else if (result != ES_RETRY) {
- /*
- * For now, just halt the machine. That makes debugging easier,
- * later we just call sev_es_terminate() here.
- */
- while (true)
- asm volatile("hlt\n");
- }
+ else if (result != ES_RETRY)
+ sev_es_terminate(GHCB_SEV_ES_REASON_GENERAL_REQUEST);
}
diff --git a/arch/x86/kernel/sev-es-shared.c b/arch/x86/kernel/sev-es-shared.c
index 387b716..0aa9f13 100644
--- a/arch/x86/kernel/sev-es-shared.c
+++ b/arch/x86/kernel/sev-es-shared.c
@@ -24,7 +24,7 @@ static bool __init sev_es_check_cpu_features(void)
return true;
}
-static void sev_es_terminate(unsigned int reason)
+static void __noreturn sev_es_terminate(unsigned int reason)
{
u64 val = GHCB_SEV_TERMINATE;
@@ -206,12 +206,8 @@ void __init do_vc_no_ghcb(struct pt_regs *regs, unsigned long exit_code)
return;
fail:
- sev_es_wr_ghcb_msr(GHCB_SEV_TERMINATE);
- VMGEXIT();
-
- /* Shouldn't get here - if we do halt the machine */
- while (true)
- asm volatile("hlt\n");
+ /* Terminate the guest */
+ sev_es_terminate(GHCB_SEV_ES_REASON_GENERAL_REQUEST);
}
static enum es_result vc_insn_string_read(struct es_em_ctxt *ctxt,
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [tip: x86/seves] x86/boot/compressed/64: Check SEV encryption in the 32-bit boot-path
2021-03-12 12:38 ` [PATCH v3 7/8] x86/boot/compressed/64: Check SEV encryption in " Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] x86/boot/compressed/64: Check SEV encryption in the " tip-bot2 for Joerg Roedel
@ 2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
1 sibling, 0 replies; 22+ messages in thread
From: tip-bot2 for Joerg Roedel @ 2021-03-18 22:10 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Joerg Roedel, Borislav Petkov, x86, linux-kernel
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: fef81c86262879d4b1176ef51a834c15b805ebb9
Gitweb: https://git.kernel.org/tip/fef81c86262879d4b1176ef51a834c15b805ebb9
Author: Joerg Roedel <jroedel@suse.de>
AuthorDate: Fri, 12 Mar 2021 13:38:23 +01:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 18 Mar 2021 23:04:12 +01:00
x86/boot/compressed/64: Check SEV encryption in the 32-bit boot-path
Check whether the hypervisor reported the correct C-bit when running
as an SEV guest. Using a wrong C-bit position could be used to leak
sensitive data from the guest to the hypervisor.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210312123824.306-8-joro@8bytes.org
---
arch/x86/boot/compressed/head_64.S | 83 +++++++++++++++++++++++++++++-
1 file changed, 83 insertions(+)
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index ee448ae..91ea0d5 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -183,11 +183,21 @@ SYM_FUNC_START(startup_32)
*/
call get_sev_encryption_bit
xorl %edx, %edx
+#ifdef CONFIG_AMD_MEM_ENCRYPT
testl %eax, %eax
jz 1f
subl $32, %eax /* Encryption bit is always above bit 31 */
bts %eax, %edx /* Set encryption mask for page tables */
+ /*
+ * Mark SEV as active in sev_status so that startup32_check_sev_cbit()
+ * will do a check. The sev_status memory will be fully initialized
+ * with the contents of MSR_AMD_SEV_STATUS later in
+ * set_sev_encryption_mask(). For now it is sufficient to know that SEV
+ * is active.
+ */
+ movl $1, rva(sev_status)(%ebp)
1:
+#endif
/* Initialize Page tables to 0 */
leal rva(pgtable)(%ebx), %edi
@@ -272,6 +282,9 @@ SYM_FUNC_START(startup_32)
movl %esi, %edx
1:
#endif
+ /* Check if the C-bit position is correct when SEV is active */
+ call startup32_check_sev_cbit
+
pushl $__KERNEL_CS
pushl %eax
@@ -872,6 +885,76 @@ SYM_FUNC_START(startup32_load_idt)
SYM_FUNC_END(startup32_load_idt)
/*
+ * Check for the correct C-bit position when the startup_32 boot-path is used.
+ *
+ * The check makes use of the fact that all memory is encrypted when paging is
+ * disabled. The function creates 64 bits of random data using the RDRAND
+ * instruction. RDRAND is mandatory for SEV guests, so always available. If the
+ * hypervisor violates that the kernel will crash right here.
+ *
+ * The 64 bits of random data are stored to a memory location and at the same
+ * time kept in the %eax and %ebx registers. Since encryption is always active
+ * when paging is off the random data will be stored encrypted in main memory.
+ *
+ * Then paging is enabled. When the C-bit position is correct all memory is
+ * still mapped encrypted and comparing the register values with memory will
+ * succeed. An incorrect C-bit position will map all memory unencrypted, so that
+ * the compare will use the encrypted random data and fail.
+ */
+SYM_FUNC_START(startup32_check_sev_cbit)
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+ pushl %eax
+ pushl %ebx
+ pushl %ecx
+ pushl %edx
+
+ /* Check for non-zero sev_status */
+ movl rva(sev_status)(%ebp), %eax
+ testl %eax, %eax
+ jz 4f
+
+ /*
+ * Get two 32-bit random values - Don't bail out if RDRAND fails
+ * because it is better to prevent forward progress if no random value
+ * can be gathered.
+ */
+1: rdrand %eax
+ jnc 1b
+2: rdrand %ebx
+ jnc 2b
+
+ /* Store to memory and keep it in the registers */
+ movl %eax, rva(sev_check_data)(%ebp)
+ movl %ebx, rva(sev_check_data+4)(%ebp)
+
+ /* Enable paging to see if encryption is active */
+ movl %cr0, %edx /* Backup %cr0 in %edx */
+ movl $(X86_CR0_PG | X86_CR0_PE), %ecx /* Enable Paging and Protected mode */
+ movl %ecx, %cr0
+
+ cmpl %eax, rva(sev_check_data)(%ebp)
+ jne 3f
+ cmpl %ebx, rva(sev_check_data+4)(%ebp)
+ jne 3f
+
+ movl %edx, %cr0 /* Restore previous %cr0 */
+
+ jmp 4f
+
+3: /* Check failed - hlt the machine */
+ hlt
+ jmp 3b
+
+4:
+ popl %edx
+ popl %ecx
+ popl %ebx
+ popl %eax
+#endif
+ ret
+SYM_FUNC_END(startup32_check_sev_cbit)
+
+/*
* Stack and heap for uncompression
*/
.bss
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [tip: x86/seves] x86/boot/compressed/64: Add CPUID sanity check to 32-bit boot-path
2021-03-12 12:38 ` [PATCH v3 6/8] x86/boot/compressed/64: Add CPUID sanity check to 32-bit boot-path Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
@ 2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
1 sibling, 0 replies; 22+ messages in thread
From: tip-bot2 for Joerg Roedel @ 2021-03-18 22:10 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Joerg Roedel, Borislav Petkov, x86, linux-kernel
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: e927e62d8e370ebfc0d702fec22bc752249ebcef
Gitweb: https://git.kernel.org/tip/e927e62d8e370ebfc0d702fec22bc752249ebcef
Author: Joerg Roedel <jroedel@suse.de>
AuthorDate: Fri, 12 Mar 2021 13:38:22 +01:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 18 Mar 2021 23:04:12 +01:00
x86/boot/compressed/64: Add CPUID sanity check to 32-bit boot-path
The 32-bit #VC handler has no GHCB and can only handle CPUID exit codes.
It is needed by the early boot code to handle #VC exceptions raised in
verify_cpu() and to get the position of the C-bit.
But the CPUID information comes from the hypervisor which is untrusted
and might return results which trick the guest into the no-SEV boot path
with no C-bit set in the page-tables. All data written to memory would
then be unencrypted and could leak sensitive data to the hypervisor.
Add sanity checks to the 32-bit boot #VC handler to make sure the
hypervisor does not pretend that SEV is not enabled.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210312123824.306-7-joro@8bytes.org
---
arch/x86/boot/compressed/mem_encrypt.S | 28 +++++++++++++++++++++++++-
1 file changed, 28 insertions(+)
diff --git a/arch/x86/boot/compressed/mem_encrypt.S b/arch/x86/boot/compressed/mem_encrypt.S
index ebc4a29..c1e81a8 100644
--- a/arch/x86/boot/compressed/mem_encrypt.S
+++ b/arch/x86/boot/compressed/mem_encrypt.S
@@ -139,6 +139,26 @@ SYM_CODE_START(startup32_vc_handler)
jnz .Lfail
movl %edx, 0(%esp) # Store result
+ /*
+ * Sanity check CPUID results from the Hypervisor. See comment in
+ * do_vc_no_ghcb() for more details on why this is necessary.
+ */
+
+ /* Fail if SEV leaf not available in CPUID[0x80000000].EAX */
+ cmpl $0x80000000, %ebx
+ jne .Lcheck_sev
+ cmpl $0x8000001f, 12(%esp)
+ jb .Lfail
+ jmp .Ldone
+
+.Lcheck_sev:
+ /* Fail if SEV bit not set in CPUID[0x8000001f].EAX[1] */
+ cmpl $0x8000001f, %ebx
+ jne .Ldone
+ btl $1, 12(%esp)
+ jnc .Lfail
+
+.Ldone:
popl %edx
popl %ecx
popl %ebx
@@ -152,6 +172,14 @@ SYM_CODE_START(startup32_vc_handler)
iret
.Lfail:
+ /* Send terminate request to Hypervisor */
+ movl $0x100, %eax
+ xorl %edx, %edx
+ movl $MSR_AMD64_SEV_ES_GHCB, %ecx
+ wrmsr
+ rep; vmmcall
+
+ /* If request fails, go to hlt loop */
hlt
jmp .Lfail
SYM_CODE_END(startup32_vc_handler)
^ permalink raw reply related [flat|nested] 22+ messages in thread
* [tip: x86/seves] x86/boot/compressed/64: Add 32-bit boot #VC handler
2021-03-12 12:38 ` [PATCH v3 5/8] x86/boot/compressed/64: Add 32-bit boot #VC handler Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
@ 2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
1 sibling, 0 replies; 22+ messages in thread
From: tip-bot2 for Joerg Roedel @ 2021-03-18 22:10 UTC (permalink / raw)
To: linux-tip-commits; +Cc: Joerg Roedel, Borislav Petkov, x86, linux-kernel
The following commit has been merged into the x86/seves branch of tip:
Commit-ID: 1ccdbf748d862bc2ea106fa9f2300983c77860fe
Gitweb: https://git.kernel.org/tip/1ccdbf748d862bc2ea106fa9f2300983c77860fe
Author: Joerg Roedel <jroedel@suse.de>
AuthorDate: Wed, 10 Mar 2021 09:43:22 +01:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Thu, 18 Mar 2021 23:03:43 +01:00
x86/boot/compressed/64: Add 32-bit boot #VC handler
Add a #VC exception handler which is used when the kernel still executes
in protected mode. This boot-path already uses CPUID, which will cause #VC
exceptions in an SEV-ES guest.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Borislav Petkov <bp@suse.de>
Link: https://lkml.kernel.org/r/20210312123824.306-6-joro@8bytes.org
---
arch/x86/boot/compressed/head_64.S | 6 ++-
arch/x86/boot/compressed/mem_encrypt.S | 96 ++++++++++++++++++++++++-
2 files changed, 101 insertions(+), 1 deletion(-)
diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index 2001c3b..ee448ae 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -34,6 +34,7 @@
#include <asm/asm-offsets.h>
#include <asm/bootparam.h>
#include <asm/desc_defs.h>
+#include <asm/trapnr.h>
#include "pgtable.h"
/*
@@ -857,6 +858,11 @@ SYM_FUNC_END(startup32_set_idt_entry)
SYM_FUNC_START(startup32_load_idt)
#ifdef CONFIG_AMD_MEM_ENCRYPT
+ /* #VC handler */
+ leal rva(startup32_vc_handler)(%ebp), %eax
+ movl $X86_TRAP_VC, %edx
+ call startup32_set_idt_entry
+
/* Load IDT */
leal rva(boot32_idt)(%ebp), %eax
movl %eax, rva(boot32_idt_desc+2)(%ebp)
diff --git a/arch/x86/boot/compressed/mem_encrypt.S b/arch/x86/boot/compressed/mem_encrypt.S
index a6dea4e..ebc4a29 100644
--- a/arch/x86/boot/compressed/mem_encrypt.S
+++ b/arch/x86/boot/compressed/mem_encrypt.S
@@ -61,10 +61,104 @@ SYM_FUNC_START(get_sev_encryption_bit)
ret
SYM_FUNC_END(get_sev_encryption_bit)
+/**
+ * sev_es_req_cpuid - Request a CPUID value from the Hypervisor using
+ * the GHCB MSR protocol
+ *
+ * @%eax: Register to request (0=EAX, 1=EBX, 2=ECX, 3=EDX)
+ * @%edx: CPUID Function
+ *
+ * Returns 0 in %eax on success, non-zero on failure
+ * %edx returns CPUID value on success
+ */
+SYM_CODE_START_LOCAL(sev_es_req_cpuid)
+ shll $30, %eax
+ orl $0x00000004, %eax
+ movl $MSR_AMD64_SEV_ES_GHCB, %ecx
+ wrmsr
+ rep; vmmcall # VMGEXIT
+ rdmsr
+
+ /* Check response */
+ movl %eax, %ecx
+ andl $0x3ffff000, %ecx # Bits [12-29] MBZ
+ jnz 2f
+
+ /* Check return code */
+ andl $0xfff, %eax
+ cmpl $5, %eax
+ jne 2f
+
+ /* All good - return success */
+ xorl %eax, %eax
+1:
+ ret
+2:
+ movl $-1, %eax
+ jmp 1b
+SYM_CODE_END(sev_es_req_cpuid)
+
+SYM_CODE_START(startup32_vc_handler)
+ pushl %eax
+ pushl %ebx
+ pushl %ecx
+ pushl %edx
+
+ /* Keep CPUID function in %ebx */
+ movl %eax, %ebx
+
+ /* Check if error-code == SVM_EXIT_CPUID */
+ cmpl $0x72, 16(%esp)
+ jne .Lfail
+
+ movl $0, %eax # Request CPUID[fn].EAX
+ movl %ebx, %edx # CPUID fn
+ call sev_es_req_cpuid # Call helper
+ testl %eax, %eax # Check return code
+ jnz .Lfail
+ movl %edx, 12(%esp) # Store result
+
+ movl $1, %eax # Request CPUID[fn].EBX
+ movl %ebx, %edx # CPUID fn
+ call sev_es_req_cpuid # Call helper
+ testl %eax, %eax # Check return code
+ jnz .Lfail
+ movl %edx, 8(%esp) # Store result
+
+ movl $2, %eax # Request CPUID[fn].ECX
+ movl %ebx, %edx # CPUID fn
+ call sev_es_req_cpuid # Call helper
+ testl %eax, %eax # Check return code
+ jnz .Lfail
+ movl %edx, 4(%esp) # Store result
+
+ movl $3, %eax # Request CPUID[fn].EDX
+ movl %ebx, %edx # CPUID fn
+ call sev_es_req_cpuid # Call helper
+ testl %eax, %eax # Check return code
+ jnz .Lfail
+ movl %edx, 0(%esp) # Store result
+
+ popl %edx
+ popl %ecx
+ popl %ebx
+ popl %eax
+
+ /* Remove error code */
+ addl $4, %esp
+
+ /* Jump over CPUID instruction */
+ addl $2, (%esp)
+
+ iret
+.Lfail:
+ hlt
+ jmp .Lfail
+SYM_CODE_END(startup32_vc_handler)
+
.code64
#include "../../kernel/sev_verify_cbit.S"
-
SYM_FUNC_START(set_sev_encryption_mask)
#ifdef CONFIG_AMD_MEM_ENCRYPT
push %rbp
^ permalink raw reply related [flat|nested] 22+ messages in thread
end of thread, other threads:[~2021-03-18 22:11 UTC | newest]
Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-12 12:38 [PATCH v3 0/8] x86/seves: Support 32-bit boot path and other updates Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 1/8] x86/boot/compressed/64: Cleanup exception handling before booting kernel Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 2/8] x86/sev: Do not require Hypervisor CPUID bit for SEV guests Joerg Roedel
2021-03-17 15:04 ` Tom Lendacky
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 3/8] x86/boot/compressed/64: Reload CS in startup_32 Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 4/8] x86/boot/compressed/64: Setup IDT in startup_32 boot path Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 5/8] x86/boot/compressed/64: Add 32-bit boot #VC handler Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 6/8] x86/boot/compressed/64: Add CPUID sanity check to 32-bit boot-path Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 7/8] x86/boot/compressed/64: Check SEV encryption in " Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] x86/boot/compressed/64: Check SEV encryption in the " tip-bot2 for Joerg Roedel
2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
2021-03-12 12:38 ` [PATCH v3 8/8] x86/sev-es: Replace open-coded hlt-loops with sev_es_terminate() Joerg Roedel
2021-03-18 19:34 ` [tip: x86/seves] " tip-bot2 for Joerg Roedel
2021-03-18 22:10 ` tip-bot2 for Joerg Roedel
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).