* [PATCH v4 0/3] Enable notify VM exit
@ 2022-05-24 14:02 Chenyi Qiang
2022-05-24 14:03 ` [PATCH v4 1/3] linux-header: update linux header Chenyi Qiang
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Chenyi Qiang @ 2022-05-24 14:02 UTC (permalink / raw)
To: Paolo Bonzini, Sean Christopherson, Richard Henderson,
Eduardo Habkost, Marcelo Tosatti, Xiaoyao Li
Cc: Chenyi Qiang, qemu-devel, kvm
Notify VM exit is introduced to mitigate the potential DOS attach from
malicious VM. This series is the userspace part to enable this feature
through a new KVM capability KVM_CAP_X86_NOTIFY_VMEXIT. The detailed
info can be seen in Patch 3.
The corresponding KVM patches are avaiable at:
https://lore.kernel.org/lkml/20220524135624.22988-1-chenyi.qiang@intel.com/
---
Change logs:
v3 -> v4
- Add a new KVM cap KVM_CAP_TRIPLE_FAULT_EVENT to guard the extension of triple fault
event save&restore.
- v3: https://lore.kernel.org/qemu-devel/20220421074028.18196-1-chenyi.qiang@intel.com/
v2 -> v3
- Extend the argument to include both the notify window and some flags
when enabling KVM_CAP_X86_BUS_LOCK_EXIT CAP.
- Change to use KVM_VCPUEVENTS_VALID_TRIPLE_FAULT in flags field and add
pending_triple_fault field in struct kvm_vcpu_events.
- v2: https://lore.kernel.org/qemu-devel/20220318082934.25030-1-chenyi.qiang@intel.com/
v1 -> v2
- Add some commit message to explain why we disable Notify VM exit by default.
- Rename KVM_VCPUEVENT_SHUTDOWN to KVM_VCPUEVENT_TRIPLE_FAULT.
- Do the corresponding change to use the KVM_VCPUEVENTS_TRIPLE_FAULT
to save/restore the triple fault event to avoid lose some synthesized
triple fault from KVM.
- v1: https://lore.kernel.org/qemu-devel/20220310090205.10645-1-chenyi.qiang@intel.com/
---
Chenyi Qiang (3):
linux-header: update linux header
i386: kvm: extend kvm_{get, put}_vcpu_events to support pending triple
fault
i386: Add notify VM exit support
hw/i386/x86.c | 45 +++++++++++++++++++
include/hw/i386/x86.h | 5 +++
linux-headers/asm-x86/kvm.h | 6 ++-
linux-headers/linux/kvm.h | 11 +++++
target/i386/cpu.c | 1 +
target/i386/cpu.h | 1 +
target/i386/kvm/kvm.c | 86 ++++++++++++++++++++++++++++---------
7 files changed, 134 insertions(+), 21 deletions(-)
--
2.17.1
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH v4 1/3] linux-header: update linux header 2022-05-24 14:02 [PATCH v4 0/3] Enable notify VM exit Chenyi Qiang @ 2022-05-24 14:03 ` Chenyi Qiang 2022-05-24 14:03 ` [PATCH v4 2/3] i386: kvm: extend kvm_{get, put}_vcpu_events to support pending triple fault Chenyi Qiang 2022-05-24 14:03 ` [PATCH v4 3/3] i386: Add notify VM exit support Chenyi Qiang 2 siblings, 0 replies; 6+ messages in thread From: Chenyi Qiang @ 2022-05-24 14:03 UTC (permalink / raw) To: Paolo Bonzini, Sean Christopherson, Richard Henderson, Eduardo Habkost, Marcelo Tosatti, Xiaoyao Li Cc: Chenyi Qiang, qemu-devel, kvm This linux-header update is only a reference to include some definitions related to notify VM exit. Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com> --- linux-headers/asm-x86/kvm.h | 6 +++++- linux-headers/linux/kvm.h | 11 +++++++++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h index bf6e96011d..8791f7c228 100644 --- a/linux-headers/asm-x86/kvm.h +++ b/linux-headers/asm-x86/kvm.h @@ -325,6 +325,7 @@ struct kvm_reinject_control { #define KVM_VCPUEVENT_VALID_SHADOW 0x00000004 #define KVM_VCPUEVENT_VALID_SMM 0x00000008 #define KVM_VCPUEVENT_VALID_PAYLOAD 0x00000010 +#define KVM_VCPUEVENT_VALID_TRIPLE_FAULT 0x00000020 /* Interrupt shadow states */ #define KVM_X86_SHADOW_INT_MOV_SS 0x01 @@ -359,7 +360,10 @@ struct kvm_vcpu_events { __u8 smm_inside_nmi; __u8 latched_init; } smi; - __u8 reserved[27]; + struct { + __u8 pending; + } triple_fault; + __u8 reserved[26]; __u8 exception_has_payload; __u64 exception_payload; }; diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h index d232feaae9..81dfdcfaa1 100644 --- a/linux-headers/linux/kvm.h +++ b/linux-headers/linux/kvm.h @@ -270,6 +270,7 @@ struct kvm_xen_exit { #define KVM_EXIT_X86_BUS_LOCK 33 #define KVM_EXIT_XEN 34 #define KVM_EXIT_RISCV_SBI 35 +#define KVM_EXIT_NOTIFY 36 /* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ @@ -487,6 +488,11 @@ struct kvm_run { unsigned long args[6]; unsigned long ret[2]; } riscv_sbi; + /* KVM_EXIT_NOTIFY */ + struct { +#define KVM_NOTIFY_CONTEXT_INVALID (1 << 0) + __u32 flags; + } notify; /* Fix the size of the union. */ char padding[256]; }; @@ -1134,6 +1140,8 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_VM_GPA_BITS 207 #define KVM_CAP_XSAVE2 208 #define KVM_CAP_SYS_ATTRIBUTES 209 +#define KVM_CAP_TRIPLE_FAULT_EVENT 216 +#define KVM_CAP_X86_NOTIFY_VMEXIT 217 #ifdef KVM_CAP_IRQ_ROUTING @@ -2051,4 +2059,7 @@ struct kvm_stats_desc { /* Available with KVM_CAP_XSAVE2 */ #define KVM_GET_XSAVE2 _IOR(KVMIO, 0xcf, struct kvm_xsave) +#define KVM_X86_NOTIFY_VMEXIT_ENABLED (1ULL << 0) +#define KVM_X86_NOTIFY_VMEXIT_USER (1ULL << 1) + #endif /* __LINUX_KVM_H */ -- 2.17.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v4 2/3] i386: kvm: extend kvm_{get, put}_vcpu_events to support pending triple fault 2022-05-24 14:02 [PATCH v4 0/3] Enable notify VM exit Chenyi Qiang 2022-05-24 14:03 ` [PATCH v4 1/3] linux-header: update linux header Chenyi Qiang @ 2022-05-24 14:03 ` Chenyi Qiang 2022-05-24 14:03 ` [PATCH v4 3/3] i386: Add notify VM exit support Chenyi Qiang 2 siblings, 0 replies; 6+ messages in thread From: Chenyi Qiang @ 2022-05-24 14:03 UTC (permalink / raw) To: Paolo Bonzini, Sean Christopherson, Richard Henderson, Eduardo Habkost, Marcelo Tosatti, Xiaoyao Li Cc: Chenyi Qiang, qemu-devel, kvm For the direct triple faults, i.e. hardware detected and KVM morphed to VM-Exit, KVM will never lose them. But for triple faults sythesized by KVM, e.g. the RSM path, if KVM exits to userspace before the request is serviced, userspace could migrate the VM and lose the triple fault. A new flag KVM_VCPUEVENT_VALID_TRIPLE_FAULT is defined to signal that the event.triple_fault_pending field contains a valid state if the KVM_CAP_TRIPLE_FAULT_EVENT capability is enabled. Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com> --- target/i386/cpu.c | 1 + target/i386/cpu.h | 1 + target/i386/kvm/kvm.c | 20 ++++++++++++++++++++ 3 files changed, 22 insertions(+) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index cb6b5467d0..276058d52e 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -5998,6 +5998,7 @@ static void x86_cpu_reset(DeviceState *dev) env->exception_has_payload = false; env->exception_payload = 0; env->nmi_injected = false; + env->triple_fault_pending = false; #if !defined(CONFIG_USER_ONLY) /* We hard-wire the BSP to the first CPU. */ apic_designate_bsp(cpu->apic_state, s->cpu_index == 0); diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 982c532353..a2a9423747 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -1701,6 +1701,7 @@ typedef struct CPUArchState { uint8_t has_error_code; uint8_t exception_has_payload; uint64_t exception_payload; + bool triple_fault_pending; uint32_t ins_len; uint32_t sipi_vector; bool tsc_valid; diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c index 9cf8e03669..2f2fc18b4f 100644 --- a/target/i386/kvm/kvm.c +++ b/target/i386/kvm/kvm.c @@ -129,6 +129,7 @@ static int has_xcrs; static int has_pit_state2; static int has_sregs2; static int has_exception_payload; +static int has_triple_fault_event; static bool has_msr_mcg_ext_ctl; @@ -2385,6 +2386,16 @@ int kvm_arch_init(MachineState *ms, KVMState *s) } } + has_triple_fault_event = kvm_check_extension(s, KVM_CAP_TRIPLE_FAULT_EVENT); + if (has_triple_fault_event) { + ret = kvm_vm_enable_cap(s, KVM_CAP_TRIPLE_FAULT_EVENT, 0, true); + if (ret < 0) { + error_report("kvm: Failed to enable triple fault event cap: %s", + strerror(-ret)); + return ret; + } + } + ret = kvm_get_supported_msrs(s); if (ret < 0) { return ret; @@ -4105,6 +4116,11 @@ static int kvm_put_vcpu_events(X86CPU *cpu, int level) } } + if (has_triple_fault_event) { + events.flags |= KVM_VCPUEVENT_VALID_TRIPLE_FAULT; + events.triple_fault.pending = env->triple_fault_pending; + } + return kvm_vcpu_ioctl(CPU(cpu), KVM_SET_VCPU_EVENTS, &events); } @@ -4174,6 +4190,10 @@ static int kvm_get_vcpu_events(X86CPU *cpu) } } + if (events.flags & KVM_VCPUEVENT_VALID_TRIPLE_FAULT) { + env->triple_fault_pending = events.triple_fault.pending; + } + env->sipi_vector = events.sipi_vector; return 0; -- 2.17.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v4 3/3] i386: Add notify VM exit support 2022-05-24 14:02 [PATCH v4 0/3] Enable notify VM exit Chenyi Qiang 2022-05-24 14:03 ` [PATCH v4 1/3] linux-header: update linux header Chenyi Qiang 2022-05-24 14:03 ` [PATCH v4 2/3] i386: kvm: extend kvm_{get, put}_vcpu_events to support pending triple fault Chenyi Qiang @ 2022-05-24 14:03 ` Chenyi Qiang 2022-05-25 3:43 ` Yuan Yao 2 siblings, 1 reply; 6+ messages in thread From: Chenyi Qiang @ 2022-05-24 14:03 UTC (permalink / raw) To: Paolo Bonzini, Sean Christopherson, Richard Henderson, Eduardo Habkost, Marcelo Tosatti, Xiaoyao Li Cc: Chenyi Qiang, qemu-devel, kvm There are cases that malicious virtual machine can cause CPU stuck (due to event windows don't open up), e.g., infinite loop in microcode when nested #AC (CVE-2015-5307). No event window means no event (NMI, SMI and IRQ) can be delivered. It leads the CPU to be unavailable to host or other VMs. Notify VM exit is introduced to mitigate such kind of attacks, which will generate a VM exit if no event window occurs in VM non-root mode for a specified amount of time (notify window). A new KVM capability KVM_CAP_X86_NOTIFY_VMEXIT is exposed to user space so that the user can query the capability and set the expected notify window when creating VMs. The format of the argument when enabling this capability is as follows: Bit 63:32 - notify window specified in qemu command Bit 31:0 - some flags (e.g. KVM_X86_NOTIFY_VMEXIT_ENABLED is set to enable the feature.) Because there are some concerns, e.g. a notify VM exit may happen with VM_CONTEXT_INVALID set in exit qualification (no cases are anticipated that would set this bit), which means VM context is corrupted. To avoid the false positive and a well-behaved guest gets killed, make this feature disabled by default. Users can enable the feature by a new machine property: qemu -machine notify_vmexit=on,notify_window=0 ... A new KVM exit reason KVM_EXIT_NOTIFY is defined for notify VM exit. If it happens with VM_INVALID_CONTEXT, hypervisor exits to user space to inform the fatal case. Then user space can inject a SHUTDOWN event to the target vcpu. This is implemented by injecting a sythesized triple fault event. Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com> --- hw/i386/x86.c | 45 +++++++++++++++++++++++++++++ include/hw/i386/x86.h | 5 ++++ target/i386/kvm/kvm.c | 66 ++++++++++++++++++++++++++++++------------- 3 files changed, 96 insertions(+), 20 deletions(-) diff --git a/hw/i386/x86.c b/hw/i386/x86.c index 4cf107baea..a82f959cb9 100644 --- a/hw/i386/x86.c +++ b/hw/i386/x86.c @@ -1296,6 +1296,37 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name, qapi_free_SgxEPCList(list); } +static bool x86_machine_get_notify_vmexit(Object *obj, Error **errp) +{ + X86MachineState *x86ms = X86_MACHINE(obj); + + return x86ms->notify_vmexit; +} + +static void x86_machine_set_notify_vmexit(Object *obj, bool value, Error **errp) +{ + X86MachineState *x86ms = X86_MACHINE(obj); + + x86ms->notify_vmexit = value; +} + +static void x86_machine_get_notify_window(Object *obj, Visitor *v, + const char *name, void *opaque, Error **errp) +{ + X86MachineState *x86ms = X86_MACHINE(obj); + uint32_t notify_window = x86ms->notify_window; + + visit_type_uint32(v, name, ¬ify_window, errp); +} + +static void x86_machine_set_notify_window(Object *obj, Visitor *v, + const char *name, void *opaque, Error **errp) +{ + X86MachineState *x86ms = X86_MACHINE(obj); + + visit_type_uint32(v, name, &x86ms->notify_window, errp); +} + static void x86_machine_initfn(Object *obj) { X86MachineState *x86ms = X86_MACHINE(obj); @@ -1306,6 +1337,8 @@ static void x86_machine_initfn(Object *obj) x86ms->oem_id = g_strndup(ACPI_BUILD_APPNAME6, 6); x86ms->oem_table_id = g_strndup(ACPI_BUILD_APPNAME8, 8); x86ms->bus_lock_ratelimit = 0; + x86ms->notify_vmexit = false; + x86ms->notify_window = 0; } static void x86_machine_class_init(ObjectClass *oc, void *data) @@ -1361,6 +1394,18 @@ static void x86_machine_class_init(ObjectClass *oc, void *data) NULL, NULL); object_class_property_set_description(oc, "sgx-epc", "SGX EPC device"); + + object_class_property_add(oc, X86_MACHINE_NOTIFY_WINDOW, "uint32_t", + x86_machine_get_notify_window, + x86_machine_set_notify_window, NULL, NULL); + object_class_property_set_description(oc, X86_MACHINE_NOTIFY_WINDOW, + "Set the notify window required by notify VM exit"); + + object_class_property_add_bool(oc, X86_MACHINE_NOTIFY_VMEXIT, + x86_machine_get_notify_vmexit, + x86_machine_set_notify_vmexit); + object_class_property_set_description(oc, X86_MACHINE_NOTIFY_VMEXIT, + "Enable notify VM exit"); } static const TypeInfo x86_machine_info = { diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h index 916cc325ee..571ee8b667 100644 --- a/include/hw/i386/x86.h +++ b/include/hw/i386/x86.h @@ -80,6 +80,9 @@ struct X86MachineState { * which means no limitation on the guest's bus locks. */ uint64_t bus_lock_ratelimit; + + bool notify_vmexit; + uint32_t notify_window; }; #define X86_MACHINE_SMM "smm" @@ -87,6 +90,8 @@ struct X86MachineState { #define X86_MACHINE_OEM_ID "x-oem-id" #define X86_MACHINE_OEM_TABLE_ID "x-oem-table-id" #define X86_MACHINE_BUS_LOCK_RATELIMIT "bus-lock-ratelimit" +#define X86_MACHINE_NOTIFY_VMEXIT "notify-vmexit" +#define X86_MACHINE_NOTIFY_WINDOW "notify-window" #define TYPE_X86_MACHINE MACHINE_TYPE_NAME("x86") OBJECT_DECLARE_TYPE(X86MachineState, X86MachineClass, X86_MACHINE) diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c index 2f2fc18b4f..6aaedf3412 100644 --- a/target/i386/kvm/kvm.c +++ b/target/i386/kvm/kvm.c @@ -2345,6 +2345,10 @@ int kvm_arch_init(MachineState *ms, KVMState *s) int ret; struct utsname utsname; Error *local_err = NULL; + X86MachineState *x86ms; + + assert(object_dynamic_cast(OBJECT(ms), TYPE_X86_MACHINE)); + x86ms = X86_MACHINE(ms); /* * Initialize SEV context, if required @@ -2450,8 +2454,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s) } if (kvm_check_extension(s, KVM_CAP_X86_SMM) && - object_dynamic_cast(OBJECT(ms), TYPE_X86_MACHINE) && - x86_machine_is_smm_enabled(X86_MACHINE(ms))) { + x86_machine_is_smm_enabled(x86ms)) { smram_machine_done.notify = register_smram_listener; qemu_add_machine_init_done_notifier(&smram_machine_done); } @@ -2479,25 +2482,34 @@ int kvm_arch_init(MachineState *ms, KVMState *s) } } - if (object_dynamic_cast(OBJECT(ms), TYPE_X86_MACHINE)) { - X86MachineState *x86ms = X86_MACHINE(ms); + if (x86ms->bus_lock_ratelimit > 0) { + ret = kvm_check_extension(s, KVM_CAP_X86_BUS_LOCK_EXIT); + if (!(ret & KVM_BUS_LOCK_DETECTION_EXIT)) { + error_report("kvm: bus lock detection unsupported"); + return -ENOTSUP; + } + ret = kvm_vm_enable_cap(s, KVM_CAP_X86_BUS_LOCK_EXIT, 0, + KVM_BUS_LOCK_DETECTION_EXIT); + if (ret < 0) { + error_report("kvm: Failed to enable bus lock detection cap: %s", + strerror(-ret)); + return ret; + } + ratelimit_init(&bus_lock_ratelimit_ctrl); + ratelimit_set_speed(&bus_lock_ratelimit_ctrl, + x86ms->bus_lock_ratelimit, BUS_LOCK_SLICE_TIME); + } - if (x86ms->bus_lock_ratelimit > 0) { - ret = kvm_check_extension(s, KVM_CAP_X86_BUS_LOCK_EXIT); - if (!(ret & KVM_BUS_LOCK_DETECTION_EXIT)) { - error_report("kvm: bus lock detection unsupported"); - return -ENOTSUP; - } - ret = kvm_vm_enable_cap(s, KVM_CAP_X86_BUS_LOCK_EXIT, 0, - KVM_BUS_LOCK_DETECTION_EXIT); - if (ret < 0) { - error_report("kvm: Failed to enable bus lock detection cap: %s", - strerror(-ret)); - return ret; - } - ratelimit_init(&bus_lock_ratelimit_ctrl); - ratelimit_set_speed(&bus_lock_ratelimit_ctrl, - x86ms->bus_lock_ratelimit, BUS_LOCK_SLICE_TIME); + if (x86ms->notify_vmexit && kvm_check_extension(s, KVM_CAP_X86_NOTIFY_VMEXIT)) { + uint64_t notify_window_flags = ((uint64_t)x86ms->notify_window << 32) | + KVM_X86_NOTIFY_VMEXIT_ENABLED | + KVM_X86_NOTIFY_VMEXIT_USER; + ret = kvm_vm_enable_cap(s, KVM_CAP_X86_NOTIFY_VMEXIT, 0, + notify_window_flags); + if (ret < 0) { + error_report("kvm: Failed to enable notify vmexit cap: %s", + strerror(-ret)); + return ret; } } @@ -4940,6 +4952,7 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) X86CPU *cpu = X86_CPU(cs); uint64_t code; int ret; + struct kvm_vcpu_events events = {}; switch (run->exit_reason) { case KVM_EXIT_HLT: @@ -4995,6 +5008,19 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) /* already handled in kvm_arch_post_run */ ret = 0; break; + case KVM_EXIT_NOTIFY: + ret = 0; + if (run->notify.flags & KVM_NOTIFY_CONTEXT_INVALID) { + warn_report("KVM: invalid context due to notify vmexit"); + if (has_triple_fault_event) { + events.flags |= KVM_VCPUEVENT_VALID_TRIPLE_FAULT; + events.triple_fault.pending = true; + ret = kvm_vcpu_ioctl(cs, KVM_SET_VCPU_EVENTS, &events); + } else { + ret = -1; + } + } + break; default: fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason); ret = -1; -- 2.17.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v4 3/3] i386: Add notify VM exit support 2022-05-24 14:03 ` [PATCH v4 3/3] i386: Add notify VM exit support Chenyi Qiang @ 2022-05-25 3:43 ` Yuan Yao 2022-05-26 2:21 ` Chenyi Qiang 0 siblings, 1 reply; 6+ messages in thread From: Yuan Yao @ 2022-05-25 3:43 UTC (permalink / raw) To: Chenyi Qiang Cc: Paolo Bonzini, Sean Christopherson, Richard Henderson, Eduardo Habkost, Marcelo Tosatti, Xiaoyao Li, qemu-devel, kvm On Tue, May 24, 2022 at 10:03:02PM +0800, Chenyi Qiang wrote: > There are cases that malicious virtual machine can cause CPU stuck (due > to event windows don't open up), e.g., infinite loop in microcode when > nested #AC (CVE-2015-5307). No event window means no event (NMI, SMI and > IRQ) can be delivered. It leads the CPU to be unavailable to host or > other VMs. Notify VM exit is introduced to mitigate such kind of > attacks, which will generate a VM exit if no event window occurs in VM > non-root mode for a specified amount of time (notify window). > > A new KVM capability KVM_CAP_X86_NOTIFY_VMEXIT is exposed to user space > so that the user can query the capability and set the expected notify > window when creating VMs. The format of the argument when enabling this > capability is as follows: > Bit 63:32 - notify window specified in qemu command > Bit 31:0 - some flags (e.g. KVM_X86_NOTIFY_VMEXIT_ENABLED is set to > enable the feature.) > > Because there are some concerns, e.g. a notify VM exit may happen with > VM_CONTEXT_INVALID set in exit qualification (no cases are anticipated > that would set this bit), which means VM context is corrupted. To avoid > the false positive and a well-behaved guest gets killed, make this > feature disabled by default. Users can enable the feature by a new > machine property: > qemu -machine notify_vmexit=on,notify_window=0 ... > > A new KVM exit reason KVM_EXIT_NOTIFY is defined for notify VM exit. If > it happens with VM_INVALID_CONTEXT, hypervisor exits to user space to > inform the fatal case. Then user space can inject a SHUTDOWN event to > the target vcpu. This is implemented by injecting a sythesized triple > fault event. > > Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com> > --- > hw/i386/x86.c | 45 +++++++++++++++++++++++++++++ > include/hw/i386/x86.h | 5 ++++ > target/i386/kvm/kvm.c | 66 ++++++++++++++++++++++++++++++------------- > 3 files changed, 96 insertions(+), 20 deletions(-) > > diff --git a/hw/i386/x86.c b/hw/i386/x86.c > index 4cf107baea..a82f959cb9 100644 > --- a/hw/i386/x86.c > +++ b/hw/i386/x86.c > @@ -1296,6 +1296,37 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name, > qapi_free_SgxEPCList(list); > } > > +static bool x86_machine_get_notify_vmexit(Object *obj, Error **errp) > +{ > + X86MachineState *x86ms = X86_MACHINE(obj); > + > + return x86ms->notify_vmexit; > +} > + > +static void x86_machine_set_notify_vmexit(Object *obj, bool value, Error **errp) > +{ > + X86MachineState *x86ms = X86_MACHINE(obj); > + > + x86ms->notify_vmexit = value; > +} > + > +static void x86_machine_get_notify_window(Object *obj, Visitor *v, > + const char *name, void *opaque, Error **errp) > +{ > + X86MachineState *x86ms = X86_MACHINE(obj); > + uint32_t notify_window = x86ms->notify_window; > + > + visit_type_uint32(v, name, ¬ify_window, errp); > +} > + > +static void x86_machine_set_notify_window(Object *obj, Visitor *v, > + const char *name, void *opaque, Error **errp) > +{ > + X86MachineState *x86ms = X86_MACHINE(obj); > + > + visit_type_uint32(v, name, &x86ms->notify_window, errp); > +} > + > static void x86_machine_initfn(Object *obj) > { > X86MachineState *x86ms = X86_MACHINE(obj); > @@ -1306,6 +1337,8 @@ static void x86_machine_initfn(Object *obj) > x86ms->oem_id = g_strndup(ACPI_BUILD_APPNAME6, 6); > x86ms->oem_table_id = g_strndup(ACPI_BUILD_APPNAME8, 8); > x86ms->bus_lock_ratelimit = 0; > + x86ms->notify_vmexit = false; > + x86ms->notify_window = 0; > } > > static void x86_machine_class_init(ObjectClass *oc, void *data) > @@ -1361,6 +1394,18 @@ static void x86_machine_class_init(ObjectClass *oc, void *data) > NULL, NULL); > object_class_property_set_description(oc, "sgx-epc", > "SGX EPC device"); > + > + object_class_property_add(oc, X86_MACHINE_NOTIFY_WINDOW, "uint32_t", > + x86_machine_get_notify_window, > + x86_machine_set_notify_window, NULL, NULL); > + object_class_property_set_description(oc, X86_MACHINE_NOTIFY_WINDOW, > + "Set the notify window required by notify VM exit"); > + > + object_class_property_add_bool(oc, X86_MACHINE_NOTIFY_VMEXIT, > + x86_machine_get_notify_vmexit, > + x86_machine_set_notify_vmexit); > + object_class_property_set_description(oc, X86_MACHINE_NOTIFY_VMEXIT, > + "Enable notify VM exit"); > } > > static const TypeInfo x86_machine_info = { > diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h > index 916cc325ee..571ee8b667 100644 > --- a/include/hw/i386/x86.h > +++ b/include/hw/i386/x86.h > @@ -80,6 +80,9 @@ struct X86MachineState { > * which means no limitation on the guest's bus locks. > */ > uint64_t bus_lock_ratelimit; > + > + bool notify_vmexit; > + uint32_t notify_window; > }; > > #define X86_MACHINE_SMM "smm" > @@ -87,6 +90,8 @@ struct X86MachineState { > #define X86_MACHINE_OEM_ID "x-oem-id" > #define X86_MACHINE_OEM_TABLE_ID "x-oem-table-id" > #define X86_MACHINE_BUS_LOCK_RATELIMIT "bus-lock-ratelimit" > +#define X86_MACHINE_NOTIFY_VMEXIT "notify-vmexit" > +#define X86_MACHINE_NOTIFY_WINDOW "notify-window" > > #define TYPE_X86_MACHINE MACHINE_TYPE_NAME("x86") > OBJECT_DECLARE_TYPE(X86MachineState, X86MachineClass, X86_MACHINE) > diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c > index 2f2fc18b4f..6aaedf3412 100644 > --- a/target/i386/kvm/kvm.c > +++ b/target/i386/kvm/kvm.c > @@ -2345,6 +2345,10 @@ int kvm_arch_init(MachineState *ms, KVMState *s) > int ret; > struct utsname utsname; > Error *local_err = NULL; > + X86MachineState *x86ms; > + > + assert(object_dynamic_cast(OBJECT(ms), TYPE_X86_MACHINE)); > + x86ms = X86_MACHINE(ms); > > /* > * Initialize SEV context, if required > @@ -2450,8 +2454,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s) > } > > if (kvm_check_extension(s, KVM_CAP_X86_SMM) && > - object_dynamic_cast(OBJECT(ms), TYPE_X86_MACHINE) && > - x86_machine_is_smm_enabled(X86_MACHINE(ms))) { > + x86_machine_is_smm_enabled(x86ms)) { > smram_machine_done.notify = register_smram_listener; > qemu_add_machine_init_done_notifier(&smram_machine_done); > } > @@ -2479,25 +2482,34 @@ int kvm_arch_init(MachineState *ms, KVMState *s) > } > } > > - if (object_dynamic_cast(OBJECT(ms), TYPE_X86_MACHINE)) { The original behavior: if (object_dynamic_cast(OBJECT(ms), TYPE_X86_MACHINE)) { ... do some thing } return 0; Note that it won't throw any exceptions if ms is not instance of TYPE_X86_MACHINE, but now the assert() throws exceptions in this case. In another hand, assert() may become nothing in NDEBUG case. May move the whole BUS LOCK part and new NOTIFY_VMEXIT part to 2 separate functions and call them from kvm_arch_init() makes the readability better. > - X86MachineState *x86ms = X86_MACHINE(ms); > + if (x86ms->bus_lock_ratelimit > 0) { > + ret = kvm_check_extension(s, KVM_CAP_X86_BUS_LOCK_EXIT); > + if (!(ret & KVM_BUS_LOCK_DETECTION_EXIT)) { > + error_report("kvm: bus lock detection unsupported"); > + return -ENOTSUP; > + } > + ret = kvm_vm_enable_cap(s, KVM_CAP_X86_BUS_LOCK_EXIT, 0, > + KVM_BUS_LOCK_DETECTION_EXIT); > + if (ret < 0) { > + error_report("kvm: Failed to enable bus lock detection cap: %s", > + strerror(-ret)); > + return ret; > + } > + ratelimit_init(&bus_lock_ratelimit_ctrl); > + ratelimit_set_speed(&bus_lock_ratelimit_ctrl, > + x86ms->bus_lock_ratelimit, BUS_LOCK_SLICE_TIME); > + } > > - if (x86ms->bus_lock_ratelimit > 0) { > - ret = kvm_check_extension(s, KVM_CAP_X86_BUS_LOCK_EXIT); > - if (!(ret & KVM_BUS_LOCK_DETECTION_EXIT)) { > - error_report("kvm: bus lock detection unsupported"); > - return -ENOTSUP; > - } > - ret = kvm_vm_enable_cap(s, KVM_CAP_X86_BUS_LOCK_EXIT, 0, > - KVM_BUS_LOCK_DETECTION_EXIT); > - if (ret < 0) { > - error_report("kvm: Failed to enable bus lock detection cap: %s", > - strerror(-ret)); > - return ret; > - } > - ratelimit_init(&bus_lock_ratelimit_ctrl); > - ratelimit_set_speed(&bus_lock_ratelimit_ctrl, > - x86ms->bus_lock_ratelimit, BUS_LOCK_SLICE_TIME); > + if (x86ms->notify_vmexit && kvm_check_extension(s, KVM_CAP_X86_NOTIFY_VMEXIT)) { > + uint64_t notify_window_flags = ((uint64_t)x86ms->notify_window << 32) | > + KVM_X86_NOTIFY_VMEXIT_ENABLED | > + KVM_X86_NOTIFY_VMEXIT_USER; > + ret = kvm_vm_enable_cap(s, KVM_CAP_X86_NOTIFY_VMEXIT, 0, > + notify_window_flags); > + if (ret < 0) { > + error_report("kvm: Failed to enable notify vmexit cap: %s", > + strerror(-ret)); > + return ret; > } > } > > @@ -4940,6 +4952,7 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) > X86CPU *cpu = X86_CPU(cs); > uint64_t code; > int ret; > + struct kvm_vcpu_events events = {}; > > switch (run->exit_reason) { > case KVM_EXIT_HLT: > @@ -4995,6 +5008,19 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) > /* already handled in kvm_arch_post_run */ > ret = 0; > break; > + case KVM_EXIT_NOTIFY: > + ret = 0; > + if (run->notify.flags & KVM_NOTIFY_CONTEXT_INVALID) { > + warn_report("KVM: invalid context due to notify vmexit"); > + if (has_triple_fault_event) { > + events.flags |= KVM_VCPUEVENT_VALID_TRIPLE_FAULT; > + events.triple_fault.pending = true; > + ret = kvm_vcpu_ioctl(cs, KVM_SET_VCPU_EVENTS, &events); > + } else { > + ret = -1; > + } > + } > + break; > default: > fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason); > ret = -1; > -- > 2.17.1 > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v4 3/3] i386: Add notify VM exit support 2022-05-25 3:43 ` Yuan Yao @ 2022-05-26 2:21 ` Chenyi Qiang 0 siblings, 0 replies; 6+ messages in thread From: Chenyi Qiang @ 2022-05-26 2:21 UTC (permalink / raw) To: Yuan Yao Cc: Paolo Bonzini, Sean Christopherson, Richard Henderson, Eduardo Habkost, Marcelo Tosatti, Xiaoyao Li, qemu-devel, kvm On 5/25/2022 11:43 AM, Yuan Yao wrote: > On Tue, May 24, 2022 at 10:03:02PM +0800, Chenyi Qiang wrote: >> There are cases that malicious virtual machine can cause CPU stuck (due >> to event windows don't open up), e.g., infinite loop in microcode when >> nested #AC (CVE-2015-5307). No event window means no event (NMI, SMI and >> IRQ) can be delivered. It leads the CPU to be unavailable to host or >> other VMs. Notify VM exit is introduced to mitigate such kind of >> attacks, which will generate a VM exit if no event window occurs in VM >> non-root mode for a specified amount of time (notify window). >> >> A new KVM capability KVM_CAP_X86_NOTIFY_VMEXIT is exposed to user space >> so that the user can query the capability and set the expected notify >> window when creating VMs. The format of the argument when enabling this >> capability is as follows: >> Bit 63:32 - notify window specified in qemu command >> Bit 31:0 - some flags (e.g. KVM_X86_NOTIFY_VMEXIT_ENABLED is set to >> enable the feature.) >> >> Because there are some concerns, e.g. a notify VM exit may happen with >> VM_CONTEXT_INVALID set in exit qualification (no cases are anticipated >> that would set this bit), which means VM context is corrupted. To avoid >> the false positive and a well-behaved guest gets killed, make this >> feature disabled by default. Users can enable the feature by a new >> machine property: >> qemu -machine notify_vmexit=on,notify_window=0 ... >> >> A new KVM exit reason KVM_EXIT_NOTIFY is defined for notify VM exit. If >> it happens with VM_INVALID_CONTEXT, hypervisor exits to user space to >> inform the fatal case. Then user space can inject a SHUTDOWN event to >> the target vcpu. This is implemented by injecting a sythesized triple >> fault event. >> >> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com> >> --- >> hw/i386/x86.c | 45 +++++++++++++++++++++++++++++ >> include/hw/i386/x86.h | 5 ++++ >> target/i386/kvm/kvm.c | 66 ++++++++++++++++++++++++++++++------------- >> 3 files changed, 96 insertions(+), 20 deletions(-) >> >> diff --git a/hw/i386/x86.c b/hw/i386/x86.c >> index 4cf107baea..a82f959cb9 100644 >> --- a/hw/i386/x86.c >> +++ b/hw/i386/x86.c >> @@ -1296,6 +1296,37 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name, >> qapi_free_SgxEPCList(list); >> } >> >> +static bool x86_machine_get_notify_vmexit(Object *obj, Error **errp) >> +{ >> + X86MachineState *x86ms = X86_MACHINE(obj); >> + >> + return x86ms->notify_vmexit; >> +} >> + >> +static void x86_machine_set_notify_vmexit(Object *obj, bool value, Error **errp) >> +{ >> + X86MachineState *x86ms = X86_MACHINE(obj); >> + >> + x86ms->notify_vmexit = value; >> +} >> + >> +static void x86_machine_get_notify_window(Object *obj, Visitor *v, >> + const char *name, void *opaque, Error **errp) >> +{ >> + X86MachineState *x86ms = X86_MACHINE(obj); >> + uint32_t notify_window = x86ms->notify_window; >> + >> + visit_type_uint32(v, name, ¬ify_window, errp); >> +} >> + >> +static void x86_machine_set_notify_window(Object *obj, Visitor *v, >> + const char *name, void *opaque, Error **errp) >> +{ >> + X86MachineState *x86ms = X86_MACHINE(obj); >> + >> + visit_type_uint32(v, name, &x86ms->notify_window, errp); >> +} >> + >> static void x86_machine_initfn(Object *obj) >> { >> X86MachineState *x86ms = X86_MACHINE(obj); >> @@ -1306,6 +1337,8 @@ static void x86_machine_initfn(Object *obj) >> x86ms->oem_id = g_strndup(ACPI_BUILD_APPNAME6, 6); >> x86ms->oem_table_id = g_strndup(ACPI_BUILD_APPNAME8, 8); >> x86ms->bus_lock_ratelimit = 0; >> + x86ms->notify_vmexit = false; >> + x86ms->notify_window = 0; >> } >> >> static void x86_machine_class_init(ObjectClass *oc, void *data) >> @@ -1361,6 +1394,18 @@ static void x86_machine_class_init(ObjectClass *oc, void *data) >> NULL, NULL); >> object_class_property_set_description(oc, "sgx-epc", >> "SGX EPC device"); >> + >> + object_class_property_add(oc, X86_MACHINE_NOTIFY_WINDOW, "uint32_t", >> + x86_machine_get_notify_window, >> + x86_machine_set_notify_window, NULL, NULL); >> + object_class_property_set_description(oc, X86_MACHINE_NOTIFY_WINDOW, >> + "Set the notify window required by notify VM exit"); >> + >> + object_class_property_add_bool(oc, X86_MACHINE_NOTIFY_VMEXIT, >> + x86_machine_get_notify_vmexit, >> + x86_machine_set_notify_vmexit); >> + object_class_property_set_description(oc, X86_MACHINE_NOTIFY_VMEXIT, >> + "Enable notify VM exit"); >> } >> >> static const TypeInfo x86_machine_info = { >> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h >> index 916cc325ee..571ee8b667 100644 >> --- a/include/hw/i386/x86.h >> +++ b/include/hw/i386/x86.h >> @@ -80,6 +80,9 @@ struct X86MachineState { >> * which means no limitation on the guest's bus locks. >> */ >> uint64_t bus_lock_ratelimit; >> + >> + bool notify_vmexit; >> + uint32_t notify_window; >> }; >> >> #define X86_MACHINE_SMM "smm" >> @@ -87,6 +90,8 @@ struct X86MachineState { >> #define X86_MACHINE_OEM_ID "x-oem-id" >> #define X86_MACHINE_OEM_TABLE_ID "x-oem-table-id" >> #define X86_MACHINE_BUS_LOCK_RATELIMIT "bus-lock-ratelimit" >> +#define X86_MACHINE_NOTIFY_VMEXIT "notify-vmexit" >> +#define X86_MACHINE_NOTIFY_WINDOW "notify-window" >> >> #define TYPE_X86_MACHINE MACHINE_TYPE_NAME("x86") >> OBJECT_DECLARE_TYPE(X86MachineState, X86MachineClass, X86_MACHINE) >> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c >> index 2f2fc18b4f..6aaedf3412 100644 >> --- a/target/i386/kvm/kvm.c >> +++ b/target/i386/kvm/kvm.c >> @@ -2345,6 +2345,10 @@ int kvm_arch_init(MachineState *ms, KVMState *s) >> int ret; >> struct utsname utsname; >> Error *local_err = NULL; >> + X86MachineState *x86ms; >> + >> + assert(object_dynamic_cast(OBJECT(ms), TYPE_X86_MACHINE)); >> + x86ms = X86_MACHINE(ms); >> >> /* >> * Initialize SEV context, if required >> @@ -2450,8 +2454,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s) >> } >> >> if (kvm_check_extension(s, KVM_CAP_X86_SMM) && >> - object_dynamic_cast(OBJECT(ms), TYPE_X86_MACHINE) && >> - x86_machine_is_smm_enabled(X86_MACHINE(ms))) { >> + x86_machine_is_smm_enabled(x86ms)) { >> smram_machine_done.notify = register_smram_listener; >> qemu_add_machine_init_done_notifier(&smram_machine_done); >> } >> @@ -2479,25 +2482,34 @@ int kvm_arch_init(MachineState *ms, KVMState *s) >> } >> } >> >> - if (object_dynamic_cast(OBJECT(ms), TYPE_X86_MACHINE)) { > > The original behavior: > > if (object_dynamic_cast(OBJECT(ms), TYPE_X86_MACHINE)) { > ... do some thing } > return 0; > > Note that it won't throw any exceptions if ms is not > instance of TYPE_X86_MACHINE, but now the assert() throws > exceptions in this case. In another hand, assert() may > become nothing in NDEBUG case. > Agreed, but since it's for i386, I don't know if in any case, it would assert the exception. > May move the whole BUS LOCK part and new NOTIFY_VMEXIT part > to 2 separate functions and call them from kvm_arch_init() > makes the readability better. > I don't have preference for this. Maybe not deserve to separate the functions as current enabling code is not so complicated. >> - X86MachineState *x86ms = X86_MACHINE(ms); >> + if (x86ms->bus_lock_ratelimit > 0) { >> + ret = kvm_check_extension(s, KVM_CAP_X86_BUS_LOCK_EXIT); >> + if (!(ret & KVM_BUS_LOCK_DETECTION_EXIT)) { >> + error_report("kvm: bus lock detection unsupported"); >> + return -ENOTSUP; >> + } >> + ret = kvm_vm_enable_cap(s, KVM_CAP_X86_BUS_LOCK_EXIT, 0, >> + KVM_BUS_LOCK_DETECTION_EXIT); >> + if (ret < 0) { >> + error_report("kvm: Failed to enable bus lock detection cap: %s", >> + strerror(-ret)); >> + return ret; >> + } >> + ratelimit_init(&bus_lock_ratelimit_ctrl); >> + ratelimit_set_speed(&bus_lock_ratelimit_ctrl, >> + x86ms->bus_lock_ratelimit, BUS_LOCK_SLICE_TIME); >> + } >> >> - if (x86ms->bus_lock_ratelimit > 0) { >> - ret = kvm_check_extension(s, KVM_CAP_X86_BUS_LOCK_EXIT); >> - if (!(ret & KVM_BUS_LOCK_DETECTION_EXIT)) { >> - error_report("kvm: bus lock detection unsupported"); >> - return -ENOTSUP; >> - } >> - ret = kvm_vm_enable_cap(s, KVM_CAP_X86_BUS_LOCK_EXIT, 0, >> - KVM_BUS_LOCK_DETECTION_EXIT); >> - if (ret < 0) { >> - error_report("kvm: Failed to enable bus lock detection cap: %s", >> - strerror(-ret)); >> - return ret; >> - } >> - ratelimit_init(&bus_lock_ratelimit_ctrl); >> - ratelimit_set_speed(&bus_lock_ratelimit_ctrl, >> - x86ms->bus_lock_ratelimit, BUS_LOCK_SLICE_TIME); >> + if (x86ms->notify_vmexit && kvm_check_extension(s, KVM_CAP_X86_NOTIFY_VMEXIT)) { >> + uint64_t notify_window_flags = ((uint64_t)x86ms->notify_window << 32) | >> + KVM_X86_NOTIFY_VMEXIT_ENABLED | >> + KVM_X86_NOTIFY_VMEXIT_USER; >> + ret = kvm_vm_enable_cap(s, KVM_CAP_X86_NOTIFY_VMEXIT, 0, >> + notify_window_flags); >> + if (ret < 0) { >> + error_report("kvm: Failed to enable notify vmexit cap: %s", >> + strerror(-ret)); >> + return ret; >> } >> } >> >> @@ -4940,6 +4952,7 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) >> X86CPU *cpu = X86_CPU(cs); >> uint64_t code; >> int ret; >> + struct kvm_vcpu_events events = {}; >> >> switch (run->exit_reason) { >> case KVM_EXIT_HLT: >> @@ -4995,6 +5008,19 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run) >> /* already handled in kvm_arch_post_run */ >> ret = 0; >> break; >> + case KVM_EXIT_NOTIFY: >> + ret = 0; >> + if (run->notify.flags & KVM_NOTIFY_CONTEXT_INVALID) { >> + warn_report("KVM: invalid context due to notify vmexit"); >> + if (has_triple_fault_event) { >> + events.flags |= KVM_VCPUEVENT_VALID_TRIPLE_FAULT; >> + events.triple_fault.pending = true; >> + ret = kvm_vcpu_ioctl(cs, KVM_SET_VCPU_EVENTS, &events); >> + } else { >> + ret = -1; >> + } >> + } >> + break; >> default: >> fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason); >> ret = -1; >> -- >> 2.17.1 >> ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-05-26 2:22 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-24 14:02 [PATCH v4 0/3] Enable notify VM exit Chenyi Qiang
2022-05-24 14:03 ` [PATCH v4 1/3] linux-header: update linux header Chenyi Qiang
2022-05-24 14:03 ` [PATCH v4 2/3] i386: kvm: extend kvm_{get, put}_vcpu_events to support pending triple fault Chenyi Qiang
2022-05-24 14:03 ` [PATCH v4 3/3] i386: Add notify VM exit support Chenyi Qiang
2022-05-25 3:43 ` Yuan Yao
2022-05-26 2:21 ` Chenyi Qiang
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.