* [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug
@ 2021-10-15 19:55 Jim Mattson
2021-10-15 19:55 ` [kvm-unit-tests PATCH v2 1/3] x86: Fix operand size for lldt Jim Mattson
` (5 more replies)
0 siblings, 6 replies; 9+ messages in thread
From: Jim Mattson @ 2021-10-15 19:55 UTC (permalink / raw)
To: kvm, pbonzini; +Cc: Jim Mattson
In Linux commit afc8de0118be ("KVM: nVMX: Set LDTR to its
architecturally defined value on nested VM-Exit"), Sean suggested that
this bug was likely benign, but it turns out that--for us, at
least--it can result in live migration failures. On restore, we call
KVM_SET_SREGS before KVM_SET_NESTED_STATE, so when L2 is active at the
time of save/restore, the target vmcs01 is temporarily populated with
L2 values. Hence, the LDTR visible to L1 after the next emulated
VM-exit is L2's, rather than its own.
This issue is significant enough that it warrants a regression
test. Unfortunately, at the moment, the best we can do is check for
the LDTR persistence bug. I'd like to be able to trigger a
save/restore from within the L2 guest, but AFAICT, there's no way to
do that under qemu. Does anyone want to implement a qemu ISA test
device that triggers a save/restore when its configured I/O port is
written to?
Jim Mattson (3):
x86: Fix operand size for lldt
x86: Make set_gdt_entry usable in 64-bit mode
x86: Add a regression test for L1 LDTR persistence bug
v1 -> v2:
Reworded report messages at Sean's suggestion.
lib/x86/desc.c | 41 +++++++++++++++++++++++++++++++----------
lib/x86/desc.h | 3 ++-
lib/x86/processor.h | 2 +-
x86/cstart64.S | 1 +
x86/vmx_tests.c | 39 +++++++++++++++++++++++++++++++++++++++
5 files changed, 74 insertions(+), 12 deletions(-)
--
2.33.0.1079.g6e70778dc9-goog
^ permalink raw reply [flat|nested] 9+ messages in thread
* [kvm-unit-tests PATCH v2 1/3] x86: Fix operand size for lldt
2021-10-15 19:55 [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug Jim Mattson
@ 2021-10-15 19:55 ` Jim Mattson
2021-10-15 19:55 ` [kvm-unit-tests PATCH v2 2/3] x86: Make set_gdt_entry usable in 64-bit mode Jim Mattson
` (4 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Jim Mattson @ 2021-10-15 19:55 UTC (permalink / raw)
To: kvm, pbonzini; +Cc: Jim Mattson
The lldt instruction takes an r/m16 operand.
Fixes: 7d36db351752 ("Initial commit from qemu-kvm.git kvm/test/")
Signed-off-by: Jim Mattson <jmattson@google.com>
---
lib/x86/processor.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/x86/processor.h b/lib/x86/processor.h
index eaf24d491499..fe5add548261 100644
--- a/lib/x86/processor.h
+++ b/lib/x86/processor.h
@@ -443,7 +443,7 @@ static inline void sidt(struct descriptor_table_ptr *ptr)
asm volatile ("sidt %0" : "=m"(*ptr));
}
-static inline void lldt(unsigned val)
+static inline void lldt(u16 val)
{
asm volatile ("lldt %0" : : "rm"(val));
}
--
2.33.0.1079.g6e70778dc9-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [kvm-unit-tests PATCH v2 2/3] x86: Make set_gdt_entry usable in 64-bit mode
2021-10-15 19:55 [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug Jim Mattson
2021-10-15 19:55 ` [kvm-unit-tests PATCH v2 1/3] x86: Fix operand size for lldt Jim Mattson
@ 2021-10-15 19:55 ` Jim Mattson
2021-10-15 19:55 ` [kvm-unit-tests PATCH v2 3/3] x86: Add a regression test for L1 LDTR persistence bug Jim Mattson
` (3 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Jim Mattson @ 2021-10-15 19:55 UTC (permalink / raw)
To: kvm, pbonzini; +Cc: Jim Mattson
The omission of a 64-bit implementation of set_gdt_entry was probably
just an oversight, since the declaration is in scope.
Signed-off-by: Jim Mattson <jmattson@google.com>
---
lib/x86/desc.c | 41 +++++++++++++++++++++++++++++++----------
lib/x86/desc.h | 3 ++-
x86/cstart64.S | 1 +
3 files changed, 34 insertions(+), 11 deletions(-)
diff --git a/lib/x86/desc.c b/lib/x86/desc.c
index e7378c1d372a..87e2850f46c6 100644
--- a/lib/x86/desc.c
+++ b/lib/x86/desc.c
@@ -279,23 +279,39 @@ bool exception_rflags_rf(void)
static char intr_alt_stack[4096];
-#ifndef __x86_64__
-void set_gdt_entry(int sel, u32 base, u32 limit, u8 access, u8 gran)
+static void __set_gdt_entry(int sel, uintptr_t base, u32 limit, u8 access,
+ u8 gran, gdt_entry_t *gdt)
{
int num = sel >> 3;
/* Setup the descriptor base address */
- gdt32[num].base_low = (base & 0xFFFF);
- gdt32[num].base_middle = (base >> 16) & 0xFF;
- gdt32[num].base_high = (base >> 24) & 0xFF;
+ gdt[num].base_low = (base & 0xFFFF);
+ gdt[num].base_middle = (base >> 16) & 0xFF;
+ gdt[num].base_high = (base >> 24) & 0xFF;
/* Setup the descriptor limits */
- gdt32[num].limit_low = (limit & 0xFFFF);
- gdt32[num].granularity = ((limit >> 16) & 0x0F);
+ gdt[num].limit_low = (limit & 0xFFFF);
+ gdt[num].granularity = ((limit >> 16) & 0x0F);
+
+ /* Penultimately, set up the granularity and access flags */
+ gdt[num].granularity |= (gran & 0xF0);
+ gdt[num].access = access;
+
+#ifdef __x86_64__
+ /* 64-bit system descriptors take up two slots */
+ if (!(access & 0x10000)) {
+ u32 *p = (u32 *)&gdt[num + 1];
- /* Finally, set up the granularity and access flags */
- gdt32[num].granularity |= (gran & 0xF0);
- gdt32[num].access = access;
+ p[0] = base >> 32;
+ p[1] = 0;
+ }
+#endif
+}
+
+#ifndef __x86_64__
+void set_gdt_entry(int sel, uintptr_t base, u32 limit, u8 access, u8 gran)
+{
+ __set_gdt_entry(sel, base, limit, access, gran, gdt32);
}
void set_gdt_task_gate(u16 sel, u16 tss_sel)
@@ -366,6 +382,11 @@ void print_current_tss_info(void)
tr, tr ? "interrupt" : "main", tss.prev, tss_intr.prev);
}
#else
+void set_gdt_entry(int sel, uintptr_t base, u32 limit, u8 access, u8 gran)
+{
+ __set_gdt_entry(sel, base, limit, access, gran, gdt64);
+}
+
void set_intr_alt_stack(int e, void *addr)
{
set_idt_entry(e, addr, 0);
diff --git a/lib/x86/desc.h b/lib/x86/desc.h
index a6ffb38c79a1..994f1d791130 100644
--- a/lib/x86/desc.h
+++ b/lib/x86/desc.h
@@ -208,6 +208,7 @@ void set_idt_task_gate(int vec, u16 sel);
void set_intr_task_gate(int vec, void *fn);
void setup_tss32(void);
#else
+extern gdt_entry_t gdt64[];
extern tss64_t tss;
#endif
@@ -218,7 +219,7 @@ bool exception_rflags_rf(void);
void set_idt_entry(int vec, void *addr, int dpl);
void set_idt_sel(int vec, u16 sel);
void set_idt_dpl(int vec, u16 dpl);
-void set_gdt_entry(int sel, u32 base, u32 limit, u8 access, u8 gran);
+void set_gdt_entry(int sel, uintptr_t base, u32 limit, u8 access, u8 gran);
void set_intr_alt_stack(int e, void *fn);
void print_current_tss_info(void);
handler handle_exception(u8 v, handler fn);
diff --git a/x86/cstart64.S b/x86/cstart64.S
index 5c6ad38543cc..62ace3512bb0 100644
--- a/x86/cstart64.S
+++ b/x86/cstart64.S
@@ -62,6 +62,7 @@ gdt64_desc:
.word gdt64_end - gdt64 - 1
.quad gdt64
+.globl gdt64
gdt64:
.quad 0
.quad 0x00af9b000000ffff // 64-bit code segment
--
2.33.0.1079.g6e70778dc9-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [kvm-unit-tests PATCH v2 3/3] x86: Add a regression test for L1 LDTR persistence bug
2021-10-15 19:55 [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug Jim Mattson
2021-10-15 19:55 ` [kvm-unit-tests PATCH v2 1/3] x86: Fix operand size for lldt Jim Mattson
2021-10-15 19:55 ` [kvm-unit-tests PATCH v2 2/3] x86: Make set_gdt_entry usable in 64-bit mode Jim Mattson
@ 2021-10-15 19:55 ` Jim Mattson
2021-11-10 19:50 ` [kvm-unit-tests PATCH v2 0/3] Regression " Jim Mattson
` (2 subsequent siblings)
5 siblings, 0 replies; 9+ messages in thread
From: Jim Mattson @ 2021-10-15 19:55 UTC (permalink / raw)
To: kvm, pbonzini; +Cc: Jim Mattson
Add a regression test for Linux commit afc8de0118be ("KVM: nVMX: Set
LDTR to its architecturally defined value on nested VM-Exit"). L1's
LDTR should be 0 after an emulated VM-exit from L2.
Signed-off-by: Jim Mattson <jmattson@google.com>
---
x86/vmx_tests.c | 39 +++++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
diff --git a/x86/vmx_tests.c b/x86/vmx_tests.c
index 3b97cfa6ed10..6093a90fd4ac 100644
--- a/x86/vmx_tests.c
+++ b/x86/vmx_tests.c
@@ -8363,6 +8363,44 @@ static void vmentry_movss_shadow_test(void)
vmcs_write(GUEST_RFLAGS, X86_EFLAGS_FIXED);
}
+static void vmx_ldtr_test_guest(void)
+{
+ u16 ldtr = sldt();
+
+ report(ldtr == NP_SEL, "Expected %x for L2 LDTR selector (got %x)",
+ NP_SEL, ldtr);
+}
+
+/*
+ * Ensure that the L1 LDTR is set to 0 on VM-exit.
+ */
+static void vmx_ldtr_test(void)
+{
+ const u8 ldt_ar = 0x82; /* Present LDT */
+ u16 sel = FIRST_SPARE_SEL;
+
+ /* Set up a non-zero L1 LDTR prior to VM-entry. */
+ set_gdt_entry(sel, 0, 0, ldt_ar, 0);
+ lldt(sel);
+
+ test_set_guest(vmx_ldtr_test_guest);
+ /*
+ * Set up a different LDTR for L2. The actual GDT contents are
+ * irrelevant, since we stuff the hidden descriptor state
+ * straight into the VMCS rather than reading it from the GDT.
+ */
+ vmcs_write(GUEST_SEL_LDTR, NP_SEL);
+ vmcs_write(GUEST_AR_LDTR, ldt_ar);
+ enter_guest();
+
+ /*
+ * VM-exit should clear LDTR (and make it unusable, but we
+ * won't verify that here).
+ */
+ sel = sldt();
+ report(!sel, "Expected 0 for L1 LDTR selector (got %x)", sel);
+}
+
static void vmx_single_vmcall_guest(void)
{
vmcall();
@@ -10724,6 +10762,7 @@ struct vmx_test vmx_tests[] = {
/* VMCS Shadowing tests */
TEST(vmx_vmcs_shadow_test),
/* Regression tests */
+ TEST(vmx_ldtr_test),
TEST(vmx_cr_load_test),
TEST(vmx_cr4_osxsave_test),
TEST(vmx_nm_test),
--
2.33.0.1079.g6e70778dc9-goog
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug
2021-10-15 19:55 [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug Jim Mattson
` (2 preceding siblings ...)
2021-10-15 19:55 ` [kvm-unit-tests PATCH v2 3/3] x86: Add a regression test for L1 LDTR persistence bug Jim Mattson
@ 2021-11-10 19:50 ` Jim Mattson
2021-11-10 21:44 ` Paolo Bonzini
2021-11-11 17:44 ` Paolo Bonzini
5 siblings, 0 replies; 9+ messages in thread
From: Jim Mattson @ 2021-11-10 19:50 UTC (permalink / raw)
To: kvm, pbonzini
On Fri, Oct 15, 2021 at 12:55 PM Jim Mattson <jmattson@google.com> wrote:
>
> In Linux commit afc8de0118be ("KVM: nVMX: Set LDTR to its
> architecturally defined value on nested VM-Exit"), Sean suggested that
> this bug was likely benign, but it turns out that--for us, at
> least--it can result in live migration failures. On restore, we call
> KVM_SET_SREGS before KVM_SET_NESTED_STATE, so when L2 is active at the
> time of save/restore, the target vmcs01 is temporarily populated with
> L2 values. Hence, the LDTR visible to L1 after the next emulated
> VM-exit is L2's, rather than its own.
>
> This issue is significant enough that it warrants a regression
> test. Unfortunately, at the moment, the best we can do is check for
> the LDTR persistence bug. I'd like to be able to trigger a
> save/restore from within the L2 guest, but AFAICT, there's no way to
> do that under qemu. Does anyone want to implement a qemu ISA test
> device that triggers a save/restore when its configured I/O port is
> written to?
>
> Jim Mattson (3):
> x86: Fix operand size for lldt
> x86: Make set_gdt_entry usable in 64-bit mode
> x86: Add a regression test for L1 LDTR persistence bug
>
> v1 -> v2:
> Reworded report messages at Sean's suggestion.
>
> lib/x86/desc.c | 41 +++++++++++++++++++++++++++++++----------
> lib/x86/desc.h | 3 ++-
> lib/x86/processor.h | 2 +-
> x86/cstart64.S | 1 +
> x86/vmx_tests.c | 39 +++++++++++++++++++++++++++++++++++++++
> 5 files changed, 74 insertions(+), 12 deletions(-)
>
> --
> 2.33.0.1079.g6e70778dc9-goog
>
Ping.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug
2021-10-15 19:55 [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug Jim Mattson
` (3 preceding siblings ...)
2021-11-10 19:50 ` [kvm-unit-tests PATCH v2 0/3] Regression " Jim Mattson
@ 2021-11-10 21:44 ` Paolo Bonzini
2021-11-29 22:19 ` David Matlack
2021-11-11 17:44 ` Paolo Bonzini
5 siblings, 1 reply; 9+ messages in thread
From: Paolo Bonzini @ 2021-11-10 21:44 UTC (permalink / raw)
To: Jim Mattson, kvm
On 10/15/21 21:55, Jim Mattson wrote:
> This issue is significant enough that it warrants a regression
> test. Unfortunately, at the moment, the best we can do is check for
> the LDTR persistence bug. I'd like to be able to trigger a
> save/restore from within the L2 guest, but AFAICT, there's no way to
> do that under qemu. Does anyone want to implement a qemu ISA test
> device that triggers a save/restore when its configured I/O port is
> written to?
The selftests infrastructure already has save/restore tests at
instruction granularity (state_test.c) so you should have more luck that
way; these tests are worthwhile anyway.
Paolo
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug
2021-10-15 19:55 [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug Jim Mattson
` (4 preceding siblings ...)
2021-11-10 21:44 ` Paolo Bonzini
@ 2021-11-11 17:44 ` Paolo Bonzini
5 siblings, 0 replies; 9+ messages in thread
From: Paolo Bonzini @ 2021-11-11 17:44 UTC (permalink / raw)
To: Jim Mattson, kvm
On 10/15/21 21:55, Jim Mattson wrote:
> In Linux commit afc8de0118be ("KVM: nVMX: Set LDTR to its
> architecturally defined value on nested VM-Exit"), Sean suggested that
> this bug was likely benign, but it turns out that--for us, at
> least--it can result in live migration failures. On restore, we call
> KVM_SET_SREGS before KVM_SET_NESTED_STATE, so when L2 is active at the
> time of save/restore, the target vmcs01 is temporarily populated with
> L2 values. Hence, the LDTR visible to L1 after the next emulated
> VM-exit is L2's, rather than its own.
>
> This issue is significant enough that it warrants a regression
> test. Unfortunately, at the moment, the best we can do is check for
> the LDTR persistence bug. I'd like to be able to trigger a
> save/restore from within the L2 guest, but AFAICT, there's no way to
> do that under qemu. Does anyone want to implement a qemu ISA test
> device that triggers a save/restore when its configured I/O port is
> written to?
>
> Jim Mattson (3):
> x86: Fix operand size for lldt
> x86: Make set_gdt_entry usable in 64-bit mode
> x86: Add a regression test for L1 LDTR persistence bug
>
> v1 -> v2:
> Reworded report messages at Sean's suggestion.
>
> lib/x86/desc.c | 41 +++++++++++++++++++++++++++++++----------
> lib/x86/desc.h | 3 ++-
> lib/x86/processor.h | 2 +-
> x86/cstart64.S | 1 +
> x86/vmx_tests.c | 39 +++++++++++++++++++++++++++++++++++++++
> 5 files changed, 74 insertions(+), 12 deletions(-)
>
Queued patches 1 and 3; the function of 2 is also done by my own GDT/IDT
cleanup series, which is also present in Aaron's #PF tests.
Paolo
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug
2021-11-10 21:44 ` Paolo Bonzini
@ 2021-11-29 22:19 ` David Matlack
2021-11-30 8:42 ` Paolo Bonzini
0 siblings, 1 reply; 9+ messages in thread
From: David Matlack @ 2021-11-29 22:19 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: Jim Mattson, kvm
On Wed, Nov 10, 2021 at 1:45 PM Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 10/15/21 21:55, Jim Mattson wrote:
> > This issue is significant enough that it warrants a regression
> > test. Unfortunately, at the moment, the best we can do is check for
> > the LDTR persistence bug. I'd like to be able to trigger a
> > save/restore from within the L2 guest, but AFAICT, there's no way to
> > do that under qemu. Does anyone want to implement a qemu ISA test
> > device that triggers a save/restore when its configured I/O port is
> > written to?
>
> The selftests infrastructure already has save/restore tests at
> instruction granularity (state_test.c) so you should have more luck that
> way; these tests are worthwhile anyway.
There is also (I just discovered) support for guest-triggerable
migration in the kvm-unit-tests [1]. You could use that to trigger a
save/restore.
[1] Example: https://gitlab.com/kvm-unit-tests/kvm-unit-tests/-/blob/master/arm/gic.c#L798
>
> Paolo
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug
2021-11-29 22:19 ` David Matlack
@ 2021-11-30 8:42 ` Paolo Bonzini
0 siblings, 0 replies; 9+ messages in thread
From: Paolo Bonzini @ 2021-11-30 8:42 UTC (permalink / raw)
To: David Matlack; +Cc: Jim Mattson, kvm
On 11/29/21 23:19, David Matlack wrote:
>> The selftests infrastructure already has save/restore tests at
>> instruction granularity (state_test.c) so you should have more luck that
>> way; these tests are worthwhile anyway.
> There is also (I just discovered) support for guest-triggerable
> migration in the kvm-unit-tests [1]. You could use that to trigger a
> save/restore.
There is, but I don't think it's good. Save/restore bugs almost
invariably depend on very specific processor state, and it's hard to
ensure that in kvm-unit-tests (except probabilistically, but we can do
better).
Paolo
> [1] Example:https://gitlab.com/kvm-unit-tests/kvm-unit-tests/-/blob/master/arm/gic.c#L798
>
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-11-30 8:42 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-15 19:55 [kvm-unit-tests PATCH v2 0/3] Regression test for L1 LDTR persistence bug Jim Mattson
2021-10-15 19:55 ` [kvm-unit-tests PATCH v2 1/3] x86: Fix operand size for lldt Jim Mattson
2021-10-15 19:55 ` [kvm-unit-tests PATCH v2 2/3] x86: Make set_gdt_entry usable in 64-bit mode Jim Mattson
2021-10-15 19:55 ` [kvm-unit-tests PATCH v2 3/3] x86: Add a regression test for L1 LDTR persistence bug Jim Mattson
2021-11-10 19:50 ` [kvm-unit-tests PATCH v2 0/3] Regression " Jim Mattson
2021-11-10 21:44 ` Paolo Bonzini
2021-11-29 22:19 ` David Matlack
2021-11-30 8:42 ` Paolo Bonzini
2021-11-11 17:44 ` Paolo Bonzini
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).