* [PATCH] KVM: nVMX: Filter out all unsupported controls when eVMCS was activated
@ 2021-09-07 16:35 Vitaly Kuznetsov
2021-09-08 21:53 ` Sean Christopherson
2021-09-22 14:07 ` Paolo Bonzini
0 siblings, 2 replies; 5+ messages in thread
From: Vitaly Kuznetsov @ 2021-09-07 16:35 UTC (permalink / raw)
To: kvm, Paolo Bonzini
Cc: Sean Christopherson, Wanpeng Li, Jim Mattson, linux-kernel
Windows Server 2022 with Hyper-V role enabled failed to boot on KVM when
enlightened VMCS is advertised. Debugging revealed there are two exposed
secondary controls it is not happy with: SECONDARY_EXEC_ENABLE_VMFUNC and
SECONDARY_EXEC_SHADOW_VMCS. These controls are known to be unsupported,
as there are no corresponding fields in eVMCSv1 (see the comment above
EVMCS1_UNSUPPORTED_2NDEXEC definition).
Previously, commit 31de3d2500e4 ("x86/kvm/hyper-v: move VMX controls
sanitization out of nested_enable_evmcs()") introduced the required
filtering mechanism for VMX MSRs but for some reason put only known
to be problematic (and not full EVMCS1_UNSUPPORTED_* lists) controls
there.
Note, Windows Server 2022 seems to have gained some sanity check for VMX
MSRs: it doesn't even try to launch a guest when there's something it
doesn't like, nested_evmcs_check_controls() mechanism can't catch the
problem.
Let's be bold this time and instead of playing whack-a-mole just filter out
all unsupported controls from VMX MSRs.
Fixes: 31de3d2500e4 ("x86/kvm/hyper-v: move VMX controls sanitization out of nested_enable_evmcs()")
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
arch/x86/kvm/vmx/evmcs.c | 12 +++++++++---
arch/x86/kvm/vmx/vmx.c | 9 +++++----
2 files changed, 14 insertions(+), 7 deletions(-)
diff --git a/arch/x86/kvm/vmx/evmcs.c b/arch/x86/kvm/vmx/evmcs.c
index 0dab1b7b529f..ba6f99f584ac 100644
--- a/arch/x86/kvm/vmx/evmcs.c
+++ b/arch/x86/kvm/vmx/evmcs.c
@@ -353,14 +353,20 @@ void nested_evmcs_filter_control_msr(u32 msr_index, u64 *pdata)
switch (msr_index) {
case MSR_IA32_VMX_EXIT_CTLS:
case MSR_IA32_VMX_TRUE_EXIT_CTLS:
- ctl_high &= ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL;
+ ctl_high &= ~EVMCS1_UNSUPPORTED_VMEXIT_CTRL;
break;
case MSR_IA32_VMX_ENTRY_CTLS:
case MSR_IA32_VMX_TRUE_ENTRY_CTLS:
- ctl_high &= ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL;
+ ctl_high &= ~EVMCS1_UNSUPPORTED_VMENTRY_CTRL;
break;
case MSR_IA32_VMX_PROCBASED_CTLS2:
- ctl_high &= ~SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES;
+ ctl_high &= ~EVMCS1_UNSUPPORTED_2NDEXEC;
+ break;
+ case MSR_IA32_VMX_PINBASED_CTLS:
+ ctl_high &= ~EVMCS1_UNSUPPORTED_PINCTRL;
+ break;
+ case MSR_IA32_VMX_VMFUNC:
+ ctl_low &= ~EVMCS1_UNSUPPORTED_VMFUNC;
break;
}
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index fada1055f325..d7c5257eb5c0 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -1837,10 +1837,11 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
&msr_info->data))
return 1;
/*
- * Enlightened VMCS v1 doesn't have certain fields, but buggy
- * Hyper-V versions are still trying to use corresponding
- * features when they are exposed. Filter out the essential
- * minimum.
+ * Enlightened VMCS v1 doesn't have certain VMCS fields but
+ * instead of just ignoring the features, different Hyper-V
+ * versions are either trying to use them and fail or do some
+ * sanity checking and refuse to boot. Filter all unsupported
+ * features out.
*/
if (!msr_info->host_initiated &&
vmx->nested.enlightened_vmcs_enabled)
--
2.31.1
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH] KVM: nVMX: Filter out all unsupported controls when eVMCS was activated
2021-09-07 16:35 [PATCH] KVM: nVMX: Filter out all unsupported controls when eVMCS was activated Vitaly Kuznetsov
@ 2021-09-08 21:53 ` Sean Christopherson
2021-09-09 7:03 ` Vitaly Kuznetsov
2021-09-22 14:07 ` Paolo Bonzini
1 sibling, 1 reply; 5+ messages in thread
From: Sean Christopherson @ 2021-09-08 21:53 UTC (permalink / raw)
To: Vitaly Kuznetsov
Cc: kvm, Paolo Bonzini, Wanpeng Li, Jim Mattson, linux-kernel
On Tue, Sep 07, 2021, Vitaly Kuznetsov wrote:
> Let's be bold this time and instead of playing whack-a-mole just filter out
> all unsupported controls from VMX MSRs.
Out of curiosity, why didn't we do this from the get-go?
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] KVM: nVMX: Filter out all unsupported controls when eVMCS was activated
2021-09-08 21:53 ` Sean Christopherson
@ 2021-09-09 7:03 ` Vitaly Kuznetsov
2021-09-22 14:07 ` Paolo Bonzini
0 siblings, 1 reply; 5+ messages in thread
From: Vitaly Kuznetsov @ 2021-09-09 7:03 UTC (permalink / raw)
To: Sean Christopherson
Cc: kvm, Paolo Bonzini, Wanpeng Li, Jim Mattson, linux-kernel
Sean Christopherson <seanjc@google.com> writes:
> On Tue, Sep 07, 2021, Vitaly Kuznetsov wrote:
>> Let's be bold this time and instead of playing whack-a-mole just filter out
>> all unsupported controls from VMX MSRs.
>
> Out of curiosity, why didn't we do this from the get-go?
We actually did, the initial implementation (57b119da3594f) was
filtering out everything but then things changed in "only clear controls
which are known to cause issues" (31de3d2500e4). I forgot everything
already but was able to google this suggestion from Paolo:
https://www.lkml.org/lkml/2020/1/22/1108
so finally we've settled on a shortened list. Now as new Windows version
is out, we have new problems to solve)
--
Vitaly
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] KVM: nVMX: Filter out all unsupported controls when eVMCS was activated
2021-09-09 7:03 ` Vitaly Kuznetsov
@ 2021-09-22 14:07 ` Paolo Bonzini
0 siblings, 0 replies; 5+ messages in thread
From: Paolo Bonzini @ 2021-09-22 14:07 UTC (permalink / raw)
To: Vitaly Kuznetsov, Sean Christopherson
Cc: kvm, Wanpeng Li, Jim Mattson, linux-kernel
On 09/09/21 09:03, Vitaly Kuznetsov wrote:
> Sean Christopherson <seanjc@google.com> writes:
>
>> On Tue, Sep 07, 2021, Vitaly Kuznetsov wrote:
>>> Let's be bold this time and instead of playing whack-a-mole just filter out
>>> all unsupported controls from VMX MSRs.
>>
>> Out of curiosity, why didn't we do this from the get-go?
>
> We actually did, the initial implementation (57b119da3594f) was
> filtering out everything but then things changed in "only clear controls
> which are known to cause issues" (31de3d2500e4). I forgot everything
> already but was able to google this suggestion from Paolo:
>
> https://www.lkml.org/lkml/2020/1/22/1108
The doubt was whether userspaces could be enabling eVMCS blindly, and
thus would lose features for Linux guests.
Paolo
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] KVM: nVMX: Filter out all unsupported controls when eVMCS was activated
2021-09-07 16:35 [PATCH] KVM: nVMX: Filter out all unsupported controls when eVMCS was activated Vitaly Kuznetsov
2021-09-08 21:53 ` Sean Christopherson
@ 2021-09-22 14:07 ` Paolo Bonzini
1 sibling, 0 replies; 5+ messages in thread
From: Paolo Bonzini @ 2021-09-22 14:07 UTC (permalink / raw)
To: Vitaly Kuznetsov, kvm
Cc: Sean Christopherson, Wanpeng Li, Jim Mattson, linux-kernel
On 07/09/21 18:35, Vitaly Kuznetsov wrote:
> Windows Server 2022 with Hyper-V role enabled failed to boot on KVM when
> enlightened VMCS is advertised. Debugging revealed there are two exposed
> secondary controls it is not happy with: SECONDARY_EXEC_ENABLE_VMFUNC and
> SECONDARY_EXEC_SHADOW_VMCS. These controls are known to be unsupported,
> as there are no corresponding fields in eVMCSv1 (see the comment above
> EVMCS1_UNSUPPORTED_2NDEXEC definition).
>
> Previously, commit 31de3d2500e4 ("x86/kvm/hyper-v: move VMX controls
> sanitization out of nested_enable_evmcs()") introduced the required
> filtering mechanism for VMX MSRs but for some reason put only known
> to be problematic (and not full EVMCS1_UNSUPPORTED_* lists) controls
> there.
>
> Note, Windows Server 2022 seems to have gained some sanity check for VMX
> MSRs: it doesn't even try to launch a guest when there's something it
> doesn't like, nested_evmcs_check_controls() mechanism can't catch the
> problem.
>
> Let's be bold this time and instead of playing whack-a-mole just filter out
> all unsupported controls from VMX MSRs.
>
> Fixes: 31de3d2500e4 ("x86/kvm/hyper-v: move VMX controls sanitization out of nested_enable_evmcs()")
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
> arch/x86/kvm/vmx/evmcs.c | 12 +++++++++---
> arch/x86/kvm/vmx/vmx.c | 9 +++++----
> 2 files changed, 14 insertions(+), 7 deletions(-)
>
> diff --git a/arch/x86/kvm/vmx/evmcs.c b/arch/x86/kvm/vmx/evmcs.c
> index 0dab1b7b529f..ba6f99f584ac 100644
> --- a/arch/x86/kvm/vmx/evmcs.c
> +++ b/arch/x86/kvm/vmx/evmcs.c
> @@ -353,14 +353,20 @@ void nested_evmcs_filter_control_msr(u32 msr_index, u64 *pdata)
> switch (msr_index) {
> case MSR_IA32_VMX_EXIT_CTLS:
> case MSR_IA32_VMX_TRUE_EXIT_CTLS:
> - ctl_high &= ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL;
> + ctl_high &= ~EVMCS1_UNSUPPORTED_VMEXIT_CTRL;
> break;
> case MSR_IA32_VMX_ENTRY_CTLS:
> case MSR_IA32_VMX_TRUE_ENTRY_CTLS:
> - ctl_high &= ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL;
> + ctl_high &= ~EVMCS1_UNSUPPORTED_VMENTRY_CTRL;
> break;
> case MSR_IA32_VMX_PROCBASED_CTLS2:
> - ctl_high &= ~SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES;
> + ctl_high &= ~EVMCS1_UNSUPPORTED_2NDEXEC;
> + break;
> + case MSR_IA32_VMX_PINBASED_CTLS:
> + ctl_high &= ~EVMCS1_UNSUPPORTED_PINCTRL;
> + break;
> + case MSR_IA32_VMX_VMFUNC:
> + ctl_low &= ~EVMCS1_UNSUPPORTED_VMFUNC;
> break;
> }
>
> diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
> index fada1055f325..d7c5257eb5c0 100644
> --- a/arch/x86/kvm/vmx/vmx.c
> +++ b/arch/x86/kvm/vmx/vmx.c
> @@ -1837,10 +1837,11 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
> &msr_info->data))
> return 1;
> /*
> - * Enlightened VMCS v1 doesn't have certain fields, but buggy
> - * Hyper-V versions are still trying to use corresponding
> - * features when they are exposed. Filter out the essential
> - * minimum.
> + * Enlightened VMCS v1 doesn't have certain VMCS fields but
> + * instead of just ignoring the features, different Hyper-V
> + * versions are either trying to use them and fail or do some
> + * sanity checking and refuse to boot. Filter all unsupported
> + * features out.
> */
> if (!msr_info->host_initiated &&
> vmx->nested.enlightened_vmcs_enabled)
>
Queued, thanks.
Paolo
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-09-22 14:07 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-07 16:35 [PATCH] KVM: nVMX: Filter out all unsupported controls when eVMCS was activated Vitaly Kuznetsov
2021-09-08 21:53 ` Sean Christopherson
2021-09-09 7:03 ` Vitaly Kuznetsov
2021-09-22 14:07 ` Paolo Bonzini
2021-09-22 14:07 ` Paolo Bonzini
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).