[PATCH 0/2] KVM: nSVM: TLB_CONTROL / FLUSHBYASID "fixes"

[PATCH 0/2] KVM: nSVM: TLB_CONTROL / FLUSHBYASID "fixes"

[Sean Christopherson]

Two "fixes" to play nice with running VMware Workstation on top of KVM,
in quotes because patch 2 isn't really a fix.

Sean Christopherson (2):
  Revert "nSVM: Check for reserved encodings of TLB_CONTROL in nested
    VMCB"
  KVM: nSVM: Advertise support for flush-by-ASID

 arch/x86/kvm/svm/nested.c | 15 ---------------
 arch/x86/kvm/svm/svm.c    |  1 +
 2 files changed, 1 insertion(+), 15 deletions(-)


base-commit: 437bba5ad2bba00c2056c896753a32edf80860cc
-- 
2.42.0.655.g421f12c284-goog

[PATCH 1/2] Revert "nSVM: Check for reserved encodings of TLB_CONTROL in nested VMCB"

[Sean Christopherson]

Revert KVM's made-up consistency check on SVM's TLB control.  The APM says
that unsupported encodings are reserved, but the APM doesn't state that
VMRUN checks for a supported encoding.  Unless something is called out
in "Canonicalization and Consistency Checks" or listed as MBZ (Must Be
Zero), AMD behavior is typically to let software shoot itself in the foot.

This reverts commit 174a921b6975ef959dd82ee9e8844067a62e3ec1.

Fixes: 174a921b6975 ("nSVM: Check for reserved encodings of TLB_CONTROL in nested VMCB")
Reported-by: Stefan Sterz <s.sterz@proxmox.com>
Closes: https://lkml.kernel.org/r/b9915c9c-4cf6-051a-2d91-44cc6380f455%40proxmox.com
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/kvm/svm/nested.c | 15 ---------------
 1 file changed, 15 deletions(-)

diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index 3fea8c47679e..60891b9ce25f 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -247,18 +247,6 @@ static bool nested_svm_check_bitmap_pa(struct kvm_vcpu *vcpu, u64 pa, u32 size)
 	    kvm_vcpu_is_legal_gpa(vcpu, addr + size - 1);
 }
 
-static bool nested_svm_check_tlb_ctl(struct kvm_vcpu *vcpu, u8 tlb_ctl)
-{
-	/* Nested FLUSHBYASID is not supported yet.  */
-	switch(tlb_ctl) {
-		case TLB_CONTROL_DO_NOTHING:
-		case TLB_CONTROL_FLUSH_ALL_ASID:
-			return true;
-		default:
-			return false;
-	}
-}
-
 static bool __nested_vmcb_check_controls(struct kvm_vcpu *vcpu,
 					 struct vmcb_ctrl_area_cached *control)
 {
@@ -278,9 +266,6 @@ static bool __nested_vmcb_check_controls(struct kvm_vcpu *vcpu,
 					   IOPM_SIZE)))
 		return false;
 
-	if (CC(!nested_svm_check_tlb_ctl(vcpu, control->tlb_ctl)))
-		return false;
-
 	if (CC((control->int_ctl & V_NMI_ENABLE_MASK) &&
 	       !vmcb12_is_intercept(control, INTERCEPT_NMI))) {
 		return false;
-- 
2.42.0.655.g421f12c284-goog

[PATCH 2/2] KVM: nSVM: Advertise support for flush-by-ASID

[Sean Christopherson]

Advertise support for FLUSHBYASID when nested SVM is enabled, as KVM can
always emulate flushing TLB entries for a vmcb12 ASID, e.g. by running L2
with a new, fresh ASID in vmcb02.  Some modern hypervisors, e.g. VMWare
Workstation 17, require FLUSHBYASID support and will refuse to run if it's
not present.

Punt on proper support, as "Honor L1's request to flush an ASID on nested
VMRUN" is one of the TODO items in the (incomplete) list of issues that
need to be addressed in order for KVM to NOT do a full TLB flush on every
nested SVM transition (see nested_svm_transition_tlb_flush()).

Reported-by: Stefan Sterz <s.sterz@proxmox.com>
Closes: https://lkml.kernel.org/r/b9915c9c-4cf6-051a-2d91-44cc6380f455%40proxmox.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/kvm/svm/svm.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 1785de7dc98b..9cf7eef161ff 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -5083,6 +5083,7 @@ static __init void svm_set_cpu_caps(void)
 	if (nested) {
 		kvm_cpu_cap_set(X86_FEATURE_SVM);
 		kvm_cpu_cap_set(X86_FEATURE_VMCBCLEAN);
+		kvm_cpu_cap_set(X86_FEATURE_FLUSHBYASID);
 
 		if (nrips)
 			kvm_cpu_cap_set(X86_FEATURE_NRIPS);
-- 
2.42.0.655.g421f12c284-goog

Re: [PATCH 0/2] KVM: nSVM: TLB_CONTROL / FLUSHBYASID "fixes"

[Stefan Sterz]

On Wed Oct 18, 2023 at 9:41 PM CEST, Sean Christopherson wrote:
> Two "fixes" to play nice with running VMware Workstation on top of KVM,
> in quotes because patch 2 isn't really a fix.
>

Hey, thanks for providing these patches. Tested them here with ESXi 7,
they work like a charm!

As a sidenote regarding my tests: had to fiddle a bit with the reverted
commit as I applied them on top of the v6.2 tag, because that is still
the kernel version we use over here.

> Sean Christopherson (2):
>   Revert "nSVM: Check for reserved encodings of TLB_CONTROL in nested
>     VMCB"
>   KVM: nSVM: Advertise support for flush-by-ASID
>
>  arch/x86/kvm/svm/nested.c | 15 ---------------
>  arch/x86/kvm/svm/svm.c    |  1 +
>  2 files changed, 1 insertion(+), 15 deletions(-)
>
>
> base-commit: 437bba5ad2bba00c2056c896753a32edf80860cc
> --
> 2.42.0.655.g421f12c284-goog

Re: [PATCH 1/2] Revert "nSVM: Check for reserved encodings of TLB_CONTROL in nested VMCB"

[Maxim Levitsky]

On Wed, 2023-10-18 at 12:41 -0700, Sean Christopherson wrote:
> Revert KVM's made-up consistency check on SVM's TLB control.  The APM says
> that unsupported encodings are reserved, but the APM doesn't state that
> VMRUN checks for a supported encoding.  Unless something is called out
> in "Canonicalization and Consistency Checks" or listed as MBZ (Must Be
> Zero), AMD behavior is typically to let software shoot itself in the foot.
> 
> This reverts commit 174a921b6975ef959dd82ee9e8844067a62e3ec1.
> 
> Fixes: 174a921b6975 ("nSVM: Check for reserved encodings of TLB_CONTROL in nested VMCB")
> Reported-by: Stefan Sterz <s.sterz@proxmox.com>
> Closes: https://lkml.kernel.org/r/b9915c9c-4cf6-051a-2d91-44cc6380f455%40proxmox.com
> Cc: stable@vger.kernel.org
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>  arch/x86/kvm/svm/nested.c | 15 ---------------
>  1 file changed, 15 deletions(-)
> 
> diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
> index 3fea8c47679e..60891b9ce25f 100644
> --- a/arch/x86/kvm/svm/nested.c
> +++ b/arch/x86/kvm/svm/nested.c
> @@ -247,18 +247,6 @@ static bool nested_svm_check_bitmap_pa(struct kvm_vcpu *vcpu, u64 pa, u32 size)
>  	    kvm_vcpu_is_legal_gpa(vcpu, addr + size - 1);
>  }
>  
> -static bool nested_svm_check_tlb_ctl(struct kvm_vcpu *vcpu, u8 tlb_ctl)
> -{
> -	/* Nested FLUSHBYASID is not supported yet.  */
> -	switch(tlb_ctl) {
> -		case TLB_CONTROL_DO_NOTHING:
> -		case TLB_CONTROL_FLUSH_ALL_ASID:
> -			return true;
> -		default:
> -			return false;
> -	}
> -}
> -
>  static bool __nested_vmcb_check_controls(struct kvm_vcpu *vcpu,
>  					 struct vmcb_ctrl_area_cached *control)
>  {
> @@ -278,9 +266,6 @@ static bool __nested_vmcb_check_controls(struct kvm_vcpu *vcpu,
>  					   IOPM_SIZE)))
>  		return false;
>  
> -	if (CC(!nested_svm_check_tlb_ctl(vcpu, control->tlb_ctl)))
> -		return false;
> -
>  	if (CC((control->int_ctl & V_NMI_ENABLE_MASK) &&
>  	       !vmcb12_is_intercept(control, INTERCEPT_NMI))) {
>  		return false;


Yes, after checking Jim's comment (*) on this I still agree that revert is OK.
KVM never passes through the tlb_ctl field (but does copy it to the cache),
thus there is no need to sanitize it.

https://www.spinics.net/lists/kvm/msg316072.html


Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>

Best regards,
	Maxim Levitsky

Re: [PATCH 2/2] KVM: nSVM: Advertise support for flush-by-ASID

[Maxim Levitsky]

On Wed, 2023-10-18 at 12:41 -0700, Sean Christopherson wrote:
> Advertise support for FLUSHBYASID when nested SVM is enabled, as KVM can
> always emulate flushing TLB entries for a vmcb12 ASID, e.g. by running L2
> with a new, fresh ASID in vmcb02.  Some modern hypervisors, e.g. VMWare
> Workstation 17, require FLUSHBYASID support and will refuse to run if it's
> not present.
> 
> Punt on proper support, as "Honor L1's request to flush an ASID on nested
> VMRUN" is one of the TODO items in the (incomplete) list of issues that
> need to be addressed in order for KVM to NOT do a full TLB flush on every
> nested SVM transition (see nested_svm_transition_tlb_flush()).
> 
> Reported-by: Stefan Sterz <s.sterz@proxmox.com>
> Closes: https://lkml.kernel.org/r/b9915c9c-4cf6-051a-2d91-44cc6380f455%40proxmox.com
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>  arch/x86/kvm/svm/svm.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
> index 1785de7dc98b..9cf7eef161ff 100644
> --- a/arch/x86/kvm/svm/svm.c
> +++ b/arch/x86/kvm/svm/svm.c
> @@ -5083,6 +5083,7 @@ static __init void svm_set_cpu_caps(void)
>  	if (nested) {
>  		kvm_cpu_cap_set(X86_FEATURE_SVM);
>  		kvm_cpu_cap_set(X86_FEATURE_VMCBCLEAN);
> +		kvm_cpu_cap_set(X86_FEATURE_FLUSHBYASID);
>  
>  		if (nrips)
>  			kvm_cpu_cap_set(X86_FEATURE_NRIPS);

Nitpick: if you think that this is worth it,
maybe we can add a comment here on why we can 'support' the flushbyasid feature?
in addition to the commit message.

Other than that:

Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>

Best regards,
	Maxim Levitsky

Re: [PATCH 0/2] KVM: nSVM: TLB_CONTROL / FLUSHBYASID "fixes"

[Sean Christopherson]

On Wed, 18 Oct 2023 12:41:02 -0700, Sean Christopherson wrote:
> Two "fixes" to play nice with running VMware Workstation on top of KVM,
> in quotes because patch 2 isn't really a fix.
> 
> Sean Christopherson (2):
>   Revert "nSVM: Check for reserved encodings of TLB_CONTROL in nested
>     VMCB"
>   KVM: nSVM: Advertise support for flush-by-ASID
> 
> [...]

Applied to kvm-x86 svm, with a comment as suggested by Maxim.

[1/2] Revert "nSVM: Check for reserved encodings of TLB_CONTROL in nested VMCB"
      https://github.com/kvm-x86/linux/commit/a484755ab252
[2/2] KVM: nSVM: Advertise support for flush-by-ASID
      https://github.com/kvm-x86/linux/commit/176bfc5b17fe

--
https://github.com/kvm-x86/linux/tree/next