[PATCH 0/2 v2] KVM: nVMX: Check GUEST_DR7 on vmentry of nested guests

[PATCH 0/2 v2] KVM: nVMX: Check GUEST_DR7 on vmentry of nested guests

[Krish Sadhukhan]

I sent out v1 a few months back. Here are the changes:

v1 -> v2:
	1. Patch# 1 has been dropped as we do not want to check GUEST_DEBUGCTL
	   software.
	2. Patch# 3 has been dropped.
	3. Patch# 4 has been modified to include only the tests for GUEST_DR7
	   and to be run directly on hardware as well.


[PATCH 1/2 v2] KVM: nVMX: Check GUEST_DR7 on vmentry of nested guests
[PATCH 2/2 v2] kvm-unit-test: nVMX: Test GUEST_DR7 on vmentry of nested guests

 arch/x86/kvm/vmx/nested.c | 6 ++++++
 arch/x86/kvm/x86.c        | 2 +-
 arch/x86/kvm/x86.h        | 6 ++++++
 3 files changed, 13 insertions(+), 1 deletion(-)

Krish Sadhukhan (1):
      nVMX: Check GUEST_DR7 on vmentry of nested guests

 x86/vmx_tests.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

Krish Sadhukhan (1):
      nVMX: Test GUEST_DR7 on vmentry of nested guests

[PATCH 1/2 v2] KVM: nVMX: Check GUEST_DR7 on vmentry of nested guests

[Krish Sadhukhan]

According to section "Checks on Guest Control Registers, Debug Registers, and
and MSRs" in Intel SDM vol 3C, the following checks are performed on vmentry
of nested guests:

    If the "load debug controls" VM-entry control is 1, bits 63:32 in the DR7
    field must be 0.

Signed-off-by: Krish Sadhukhan <krish.sadhukhan@oracle.com>
Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
---
 arch/x86/kvm/vmx/nested.c | 6 ++++++
 arch/x86/kvm/x86.c        | 2 +-
 arch/x86/kvm/x86.h        | 6 ++++++
 3 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index 4aea7d304beb..acde8a2f13e2 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -2899,6 +2899,12 @@ static int nested_vmx_check_guest_state(struct kvm_vcpu *vcpu,
 	    CC(!nested_guest_cr4_valid(vcpu, vmcs12->guest_cr4)))
 		return -EINVAL;
 
+#ifdef CONFIG_X86_64
+	if ((vmcs12->vm_entry_controls & VM_ENTRY_LOAD_DEBUG_CONTROLS) &&
+	    !kvm_dr7_valid(vmcs12->guest_dr7))
+		return -EINVAL;
+#endif
+
 	if ((vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_PAT) &&
 	    CC(!kvm_pat_valid(vmcs12->guest_ia32_pat)))
 		return -EINVAL;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index cf917139de6b..220f20a2f9c3 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -1064,7 +1064,7 @@ static int __kvm_set_dr(struct kvm_vcpu *vcpu, int dr, unsigned long val)
 	case 5:
 		/* fall through */
 	default: /* 7 */
-		if (val & 0xffffffff00000000ULL)
+		if (!kvm_dr7_valid(val))
 			return -1; /* #GP */
 		vcpu->arch.dr7 = (val & DR7_VOLATILE) | DR7_FIXED_1;
 		kvm_update_dr7(vcpu);
diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
index 29391af8871d..76cd389ecf60 100644
--- a/arch/x86/kvm/x86.h
+++ b/arch/x86/kvm/x86.h
@@ -369,6 +369,12 @@ static inline bool kvm_pat_valid(u64 data)
 	return (data | ((data & 0x0202020202020202ull) << 1)) == data;
 }
 
+static inline bool kvm_dr7_valid(u64 data)
+{
+	/* Bits [63:32] are reserved */
+	return ((data & 0xFFFFFFFF00000000ull) ? false : true);
+}
+
 void kvm_load_guest_xsave_state(struct kvm_vcpu *vcpu);
 void kvm_load_host_xsave_state(struct kvm_vcpu *vcpu);
 
-- 
2.20.1

Re: [PATCH 1/2 v2] KVM: nVMX: Check GUEST_DR7 on vmentry of nested guests

[Sean Christopherson]

On Tue, Jan 14, 2020 at 08:25:40PM -0500, Krish Sadhukhan wrote:
> According to section "Checks on Guest Control Registers, Debug Registers, and
> and MSRs" in Intel SDM vol 3C, the following checks are performed on vmentry
> of nested guests:
> 
>     If the "load debug controls" VM-entry control is 1, bits 63:32 in the DR7
>     field must be 0.

Please explain *why* the check is being added to KVM.  Quoting the SDM is
very helpful in proving the correctness of the code, but it doesn't provide
any insight into why a guest field is being checked in software.  A tweaked
version of Jim's anaylsis from v1[*] would be perfect.

https://lkml.kernel.org/r/CALMp9eR2GQ_aerH-arOEpa08k8ZdtYCA5ftxHfDCo5fS1r3VtA@mail.gmail.com

> Signed-off-by: Krish Sadhukhan <krish.sadhukhan@oracle.com>
> Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
> ---
>  arch/x86/kvm/vmx/nested.c | 6 ++++++
>  arch/x86/kvm/x86.c        | 2 +-
>  arch/x86/kvm/x86.h        | 6 ++++++
>  3 files changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
> index 4aea7d304beb..acde8a2f13e2 100644
> --- a/arch/x86/kvm/vmx/nested.c
> +++ b/arch/x86/kvm/vmx/nested.c
> @@ -2899,6 +2899,12 @@ static int nested_vmx_check_guest_state(struct kvm_vcpu *vcpu,
>  	    CC(!nested_guest_cr4_valid(vcpu, vmcs12->guest_cr4)))
>  		return -EINVAL;
>  
> +#ifdef CONFIG_X86_64

Hmm, I'd prefer not to wrap this with CONFIG_X86_64.  From an architectural
perspective, the consistency check is performed if the CPU *supports* long
mode, irrespective of whether the CPU is actually in long mode.  KVM could
technically do something like static_cpu_has(X86_FEATURE_LM), but that's a
waste of code for everyone except the 0.00000000000001% of the population
running on Yonah, and nested 32-bit on 64-bit already fudges things with
respect to 64-bit CPU behavior.

Functionally, it'll be the same end result (and possibly a waste of cycles
on 32-bit KVM if the compiler doesn't optimize out kvm_dr7_valid()) as
having the CONFIG_X86_64 since kvm_dr7_valid() will always return true on
32-bit KVM (assuming @data is changed to an unsigned long).

Architecturally, 32-bit KVM on 64-bit harware is already in a grey area,
e.g. hardware VM-Entry still performs checks like GUEST_DR7[63:32]!=0,
they just can't fail on 32-bit KVM because KVM's VMWRITE to propgate
vmcs12->guest_dr7 to vmcs02.GUEST_DR7 will drop bits 63:32.

In other words, it's not an issue of functionality, I'd just prefer to keep
keep the constency checks themselves aligned with the SDM.

> +	if ((vmcs12->vm_entry_controls & VM_ENTRY_LOAD_DEBUG_CONTROLS) &&
> +	    !kvm_dr7_valid(vmcs12->guest_dr7))

Wrap !kvm_dr7_valid() with CC() so that it's traced.

> +		return -EINVAL;
> +#endif
> +
>  	if ((vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_PAT) &&
>  	    CC(!kvm_pat_valid(vmcs12->guest_ia32_pat)))
>  		return -EINVAL;
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index cf917139de6b..220f20a2f9c3 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -1064,7 +1064,7 @@ static int __kvm_set_dr(struct kvm_vcpu *vcpu, int dr, unsigned long val)
>  	case 5:
>  		/* fall through */
>  	default: /* 7 */
> -		if (val & 0xffffffff00000000ULL)
> +		if (!kvm_dr7_valid(val))
>  			return -1; /* #GP */
>  		vcpu->arch.dr7 = (val & DR7_VOLATILE) | DR7_FIXED_1;
>  		kvm_update_dr7(vcpu);
> diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h
> index 29391af8871d..76cd389ecf60 100644
> --- a/arch/x86/kvm/x86.h
> +++ b/arch/x86/kvm/x86.h
> @@ -369,6 +369,12 @@ static inline bool kvm_pat_valid(u64 data)
>  	return (data | ((data & 0x0202020202020202ull) << 1)) == data;
>  }
>  
> +static inline bool kvm_dr7_valid(u64 data)

Per Jim's feedback on v1, @data should be "unsigned long".

> +{
> +	/* Bits [63:32] are reserved */
> +	return ((data & 0xFFFFFFFF00000000ull) ? false : true);

Per Jim's feedback in v1, the ternary operator and second set of
parantheses are unnecessary.

	return !(data & 0xFFFFFFFF00000000ull);

or

	return data == (u32)data;

or

	return !(data >> 32);

I prefer the last one because (IMO) it's easier to visually parse than the
"& 0xFF..." variant and more explicit in what it's doing than the casting
variant.

> +}
> +
>  void kvm_load_guest_xsave_state(struct kvm_vcpu *vcpu);
>  void kvm_load_host_xsave_state(struct kvm_vcpu *vcpu);
>  
> -- 
> 2.20.1
> 

[PATCH 2/2 v2] kvm-unit-test: nVMX: Test GUEST_DR7 on vmentry of nested guests

[Krish Sadhukhan]

According to section "Checks on Guest Control Registers, Debug Registers, and
and MSRs" in Intel SDM vol 3C, the following checks are performed on vmentry
of nested guests:

   If the "load debug controls" VM-entry control is 1,

      - bits 63:32 in the DR7 field must be 0.

Signed-off-by: Krish Sadhukhan <krish.sadhukhan@oracle.com>
Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
Co-developed-by: Jim Mattson <jmattson@google.com>
Signed-off-by: Jim Mattson <jmattson@google.com>
---
 x86/vmx_tests.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/x86/vmx_tests.c b/x86/vmx_tests.c
index fce773c..b773872 100644
--- a/x86/vmx_tests.c
+++ b/x86/vmx_tests.c
@@ -7442,6 +7442,49 @@ static void vmx_host_state_area_test(void)
 	test_load_host_perf_global_ctrl();
 }
 
+/*
+ * If the "load debug controls" VM-entry control is 1, bits 63:32 in
+ * the DR7 field must be 0.
+ *
+ * [Intel SDM]
+ */
+static void test_guest_dr7(void)
+{
+	u32 ent_saved = vmcs_read(ENT_CONTROLS);
+	u64 dr7_saved = vmcs_read(GUEST_DR7);
+	u64 val;
+	int i;
+
+	if (ctrl_enter_rev.set & ENT_LOAD_DBGCTLS) {
+		vmcs_clear_bits(ENT_CONTROLS, ENT_LOAD_DBGCTLS);
+		for (i = 0; i < 64; i++) {
+			val = 1ull << i;
+			vmcs_write(GUEST_DR7, val);
+			enter_guest();
+			report_guest_state_test("ENT_LOAD_DBGCTLS disabled",
+						VMX_VMCALL, val, "GUEST_DR7");
+		}
+	}
+	if (ctrl_enter_rev.clr & ENT_LOAD_DBGCTLS) {
+		vmcs_set_bits(ENT_CONTROLS, ENT_LOAD_DBGCTLS);
+		for (i = 0; i < 64; i++) {
+			val = 1ull << i;
+			vmcs_write(GUEST_DR7, val);
+			if (i < 32)
+				enter_guest();
+			else
+				enter_guest_with_invalid_guest_state();
+			report_guest_state_test("ENT_LOAD_DBGCTLS enabled",
+						i < 32 ? VMX_VMCALL :
+						VMX_ENTRY_FAILURE |
+						VMX_FAIL_STATE,
+						val, "GUEST_DR7");
+		}
+	}
+	vmcs_write(GUEST_DR7, dr7_saved);
+	vmcs_write(ENT_CONTROLS, ent_saved);
+}
+
 /*
  *  If the "load IA32_PAT" VM-entry control is 1, the value of the field
  *  for the IA32_PAT MSR must be one that could be written by WRMSR
@@ -7480,6 +7523,7 @@ static void vmx_guest_state_area_test(void)
 	test_canonical(GUEST_SYSENTER_ESP, "GUEST_SYSENTER_ESP", false);
 	test_canonical(GUEST_SYSENTER_EIP, "GUEST_SYSENTER_EIP", false);
 
+	test_guest_dr7();
 	test_load_guest_pat();
 	test_guest_efer();
 	test_load_guest_perf_global_ctrl();
-- 
2.20.1