[PATCH v3 1/3] KVM: X86: Fix operand size during instruction decoding

[PATCH v3 1/3] KVM: X86: Fix operand size during instruction decoding

[Wanpeng Li]

From: Wanpeng Li <wanpeng.li@hotmail.com>

Pedro reported:
  During tests that we conducted on KVM, we noticed that executing a "PUSH %ES"
  instruction under KVM produces different results on both memory and the SP
  register depending on whether EPT support is enabled. With EPT the SP is
  reduced by 4 bytes (and the written value is 0-padded) but without EPT support
  it is only reduced by 2 bytes. The difference can be observed when the CS.DB
  field is 1 (32-bit) but not when it's 0 (16-bit).

The internal segment descriptor cache exist even in real/vm8096 mode. The CS.D
also should be respected instead of just default operand-size/66H prefix during
instruction decoding. This patch fixes it by also adjusting operand-size according
to CS.D.

Reported-by: Pedro Fonseca <pfonseca@cs.washington.edu>
Tested-by: Pedro Fonseca <pfonseca@cs.washington.edu>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Nadav Amit <nadav.amit@gmail.com>
Cc: Pedro Fonseca <pfonseca@cs.washington.edu>
Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
---
v2 -> v3:
 * cleanup the codes 
v1 -> v2:
 * respect cs.d for real/vm8096, other modes have already 
   been considered in init_emulate_ctxt().

 arch/x86/kvm/emulate.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index 8079d14..6ebc4cb 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -5000,6 +5000,8 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len)
 	bool op_prefix = false;
 	bool has_seg_override = false;
 	struct opcode opcode;
+	u16 dummy;
+	struct desc_struct desc;
 
 	ctxt->memop.type = OP_NONE;
 	ctxt->memopp = NULL;
@@ -5020,6 +5022,11 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len)
 	case X86EMUL_MODE_VM86:
 	case X86EMUL_MODE_PROT16:
 		def_op_bytes = def_ad_bytes = 2;
+		if (mode < X86EMUL_MODE_PROT16) {
+			ctxt->ops->get_segment(ctxt, &dummy, &desc, NULL, VCPU_SREG_CS);
+			if (desc.d)
+				def_op_bytes = 4;
+		}
 		break;
 	case X86EMUL_MODE_PROT32:
 		def_op_bytes = def_ad_bytes = 4;
-- 
2.7.4

[PATCH v3 2/3] KVM: nVMX: Validate the IA32_BNDCFGS on nested VM-entry

[Wanpeng Li]

From: Wanpeng Li <wanpeng.li@hotmail.com>

According to the SDM, if the "load IA32_BNDCFGS" VM-entry controls is 1, the
following checks are performed on the field for the IA32_BNDCFGS MSR:
 - Bits reserved in the IA32_BNDCFGS MSR must be 0.
 - The linear address in bits 63:12 must be canonical.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Jim Mattson <jmattson@google.com>
Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
---
 arch/x86/kvm/vmx.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index e6c8ffa..f29f57d 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -10805,6 +10805,13 @@ static int check_vmentry_postreqs(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,
 			return 1;
 	}
 
+	if (kvm_mpx_supported() &&
+		(vmcs12->vm_entry_controls & VM_ENTRY_LOAD_BNDCFGS)) {
+		if (is_noncanonical_address(vmcs12->guest_bndcfgs & PAGE_MASK, vcpu) ||
+		    (vmcs12->guest_bndcfgs & MSR_IA32_BNDCFGS_RSVD))
+			return 1;
+	}
+
 	return 0;
 }
 
-- 
2.7.4

[PATCH v3 3/3] KVM: nVMX: Fix mmu context after VMLAUNCH/VMRESUME failure

[Wanpeng Li]

From: Wanpeng Li <wanpeng.li@hotmail.com>

Commit 4f350c6dbcb (kvm: nVMX: Handle deferred early VMLAUNCH/VMRESUME failure 
properly) can result in L1(run kvm-unit-tests/run_tests.sh vmx_controls in L1) 
null pointer deference and also L0 calltrace when EPT=0 on both L0 and L1.

In L1:

BUG: unable to handle kernel paging request at ffffffffc015bf8f
 IP: vmx_vcpu_run+0x202/0x510 [kvm_intel]
 PGD 146e13067 P4D 146e13067 PUD 146e15067 PMD 3d2686067 PTE 3d4af9161
 Oops: 0003 [#1] PREEMPT SMP
 CPU: 2 PID: 1798 Comm: qemu-system-x86 Not tainted 4.14.0-rc4+ #6
 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
 RIP: 0010:vmx_vcpu_run+0x202/0x510 [kvm_intel]
 Call Trace:
 WARNING: kernel stack frame pointer at ffffb86f4988bc18 in qemu-system-x86:1798 has bad value 0000000000000002

In L0:

-----------[ cut here ]------------
] WARNING: CPU: 6 PID: 4460 at /home/kernel/linux/arch/x86/kvm//vmx.c:9845 vmx_inject_page_fault_nested+0x130/0x140 [kvm_intel]
] CPU: 6 PID: 4460 Comm: qemu-system-x86 Tainted: G           OE   4.14.0-rc7+ #25
] RIP: 0010:vmx_inject_page_fault_nested+0x130/0x140 [kvm_intel]
 Call Trace:
  paging64_page_fault+0x500/0xde0 [kvm]
  ? paging32_gva_to_gpa_nested+0x120/0x120 [kvm]
  ? nonpaging_page_fault+0x3b0/0x3b0 [kvm]
  ? __asan_storeN+0x12/0x20
  ? paging64_gva_to_gpa+0xb0/0x120 [kvm]
  ? paging64_walk_addr_generic+0x11a0/0x11a0 [kvm]
  ? lock_acquire+0x2c0/0x2c0
  ? vmx_read_guest_seg_ar+0x97/0x100 [kvm_intel]
  ? vmx_get_segment+0x2a6/0x310 [kvm_intel]
  ? sched_clock+0x1f/0x30
  ? check_chain_key+0x137/0x1e0
  ? __lock_acquire+0x83c/0x2420
  ? kvm_multiple_exception+0xf2/0x220 [kvm]
  ? debug_check_no_locks_freed+0x240/0x240
  ? debug_smp_processor_id+0x17/0x20
  ? __lock_is_held+0x9e/0x100
  kvm_mmu_page_fault+0x90/0x180 [kvm]
  kvm_handle_page_fault+0x15c/0x310 [kvm]
  ? __lock_is_held+0x9e/0x100
  handle_exception+0x3c7/0x4d0 [kvm_intel]
  vmx_handle_exit+0x103/0x1010 [kvm_intel]
  ? kvm_arch_vcpu_ioctl_run+0x1628/0x2e20 [kvm]

The commit avoids to load host state of vmcs12 as vmcs01's guest state 
since vmcs12 is not modified (except for the VM-instruction error field)
if the checking of vmcs control area fails. However, the mmu context is 
switched to nested mmu in prepare_vmcs02() and it will not be reloaded 
since load_vmcs12_host_state() is skipped when nested VMLAUNCH/VMRESUME 
fails. This patch fixes it by reloading mmu context when nested 
VMLAUNCH/VMRESUME fails.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Jim Mattson <jmattson@google.com>
Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
---
 arch/x86/kvm/vmx.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index e6c8ffa..39bfa33 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -11406,6 +11406,7 @@ static void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 exit_reason,
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
 	struct vmcs12 *vmcs12 = get_vmcs12(vcpu);
+	u32 entry_failure_code;
 
 	/* trying to cancel vmlaunch/vmresume is a bug */
 	WARN_ON_ONCE(vmx->nested.nested_run_pending);
@@ -11534,6 +11535,19 @@ static void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 exit_reason,
 	 * accordingly.
 	 */
 	nested_vmx_failValid(vcpu, VMXERR_ENTRY_INVALID_CONTROL_FIELD);
+
+	nested_ept_uninit_mmu_context(vcpu);
+
+	/*
+	 * Only PDPTE load can fail as the value of cr3 was checked on entry and
+	 * couldn't have changed.
+	 */
+	if (nested_vmx_load_cr3(vcpu, vmcs12->host_cr3, false, &entry_failure_code))
+		nested_vmx_abort(vcpu, VMX_ABORT_LOAD_HOST_PDPTE_FAIL);
+
+	if (!enable_ept)
+		vcpu->arch.walk_mmu->inject_page_fault = kvm_inject_page_fault;
+
 	/*
 	 * The emulated instruction was already skipped in
 	 * nested_vmx_run, but the updated RIP was never
-- 
2.7.4

Re: [PATCH v3 2/3] KVM: nVMX: Validate the IA32_BNDCFGS on nested VM-entry

[Konrad Rzeszutek Wilk]

On Thu, Nov 02, 2017 at 01:31:38AM -0700, Wanpeng Li wrote:
> From: Wanpeng Li <wanpeng.li@hotmail.com>
> 
> According to the SDM, if the "load IA32_BNDCFGS" VM-entry controls is 1, the
> following checks are performed on the field for the IA32_BNDCFGS MSR:
>  - Bits reserved in the IA32_BNDCFGS MSR must be 0.
>  - The linear address in bits 63:12 must be canonical.
> 
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Krčmář <rkrcmar@redhat.com>
> Cc: Jim Mattson <jmattson@google.com>
> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>

Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>

> ---
>  arch/x86/kvm/vmx.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> index e6c8ffa..f29f57d 100644
> --- a/arch/x86/kvm/vmx.c
> +++ b/arch/x86/kvm/vmx.c
> @@ -10805,6 +10805,13 @@ static int check_vmentry_postreqs(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,
>  			return 1;
>  	}
>  
> +	if (kvm_mpx_supported() &&
> +		(vmcs12->vm_entry_controls & VM_ENTRY_LOAD_BNDCFGS)) {
> +		if (is_noncanonical_address(vmcs12->guest_bndcfgs & PAGE_MASK, vcpu) ||
> +		    (vmcs12->guest_bndcfgs & MSR_IA32_BNDCFGS_RSVD))
> +			return 1;
> +	}
> +
>  	return 0;
>  }
>  
> -- 
> 2.7.4
> 

Re: [PATCH v3 1/3] KVM: X86: Fix operand size during instruction decoding

[Paolo Bonzini]

On 02/11/2017 09:31, Wanpeng Li wrote:
> From: Wanpeng Li <wanpeng.li@hotmail.com>
> 
> Pedro reported:
>   During tests that we conducted on KVM, we noticed that executing a "PUSH %ES"
>   instruction under KVM produces different results on both memory and the SP
>   register depending on whether EPT support is enabled. With EPT the SP is
>   reduced by 4 bytes (and the written value is 0-padded) but without EPT support
>   it is only reduced by 2 bytes. The difference can be observed when the CS.DB
>   field is 1 (32-bit) but not when it's 0 (16-bit).
> 
> The internal segment descriptor cache exist even in real/vm8096 mode. The CS.D
> also should be respected instead of just default operand-size/66H prefix during
> instruction decoding. This patch fixes it by also adjusting operand-size according
> to CS.D.
> 
> Reported-by: Pedro Fonseca <pfonseca@cs.washington.edu>
> Tested-by: Pedro Fonseca <pfonseca@cs.washington.edu>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Krčmář <rkrcmar@redhat.com>
> Cc: Nadav Amit <nadav.amit@gmail.com>
> Cc: Pedro Fonseca <pfonseca@cs.washington.edu>
> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
> ---
> v2 -> v3:
>  * cleanup the codes 
> v1 -> v2:
>  * respect cs.d for real/vm8096, other modes have already 
>    been considered in init_emulate_ctxt().
> 
>  arch/x86/kvm/emulate.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
> index 8079d14..6ebc4cb 100644
> --- a/arch/x86/kvm/emulate.c
> +++ b/arch/x86/kvm/emulate.c
> @@ -5000,6 +5000,8 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len)
>  	bool op_prefix = false;
>  	bool has_seg_override = false;
>  	struct opcode opcode;
> +	u16 dummy;
> +	struct desc_struct desc;
>  
>  	ctxt->memop.type = OP_NONE;
>  	ctxt->memopp = NULL;
> @@ -5020,6 +5022,11 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len)
>  	case X86EMUL_MODE_VM86:
>  	case X86EMUL_MODE_PROT16:
>  		def_op_bytes = def_ad_bytes = 2;
> +		if (mode < X86EMUL_MODE_PROT16) {
> +			ctxt->ops->get_segment(ctxt, &dummy, &desc, NULL, VCPU_SREG_CS);
> +			if (desc.d)
> +				def_op_bytes = 4;

def_ad_bytes must be changed to 4 as well.  With that change, you should
probably separate X86EMUL_MODE_PROT16 altogether from the others.

Paolo

> +		}
>  		break;
>  	case X86EMUL_MODE_PROT32:
>  		def_op_bytes = def_ad_bytes = 4;
> 

Re: [PATCH v3 2/3] KVM: nVMX: Validate the IA32_BNDCFGS on nested VM-entry

[Paolo Bonzini]

On 02/11/2017 09:31, Wanpeng Li wrote:
> From: Wanpeng Li <wanpeng.li@hotmail.com>
> 
> According to the SDM, if the "load IA32_BNDCFGS" VM-entry controls is 1, the
> following checks are performed on the field for the IA32_BNDCFGS MSR:
>  - Bits reserved in the IA32_BNDCFGS MSR must be 0.
>  - The linear address in bits 63:12 must be canonical.
> 
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Krčmář <rkrcmar@redhat.com>
> Cc: Jim Mattson <jmattson@google.com>
> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
> ---
>  arch/x86/kvm/vmx.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> index e6c8ffa..f29f57d 100644
> --- a/arch/x86/kvm/vmx.c
> +++ b/arch/x86/kvm/vmx.c
> @@ -10805,6 +10805,13 @@ static int check_vmentry_postreqs(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,
>  			return 1;
>  	}
>  
> +	if (kvm_mpx_supported() &&
> +		(vmcs12->vm_entry_controls & VM_ENTRY_LOAD_BNDCFGS)) {

Whenever !kvm_mpx_supported() is true, the bit is not set in
vmcs12->vm_entry_controls (see nested_vmx_setup_ctls_msrs), so the
condition can be simplified.  Please also use && instead of nested "if"s.

Thanks,

Paolo

> +		if (is_noncanonical_address(vmcs12->guest_bndcfgs & PAGE_MASK, vcpu) ||
> +		    (vmcs12->guest_bndcfgs & MSR_IA32_BNDCFGS_RSVD))
> +			return 1;
> +	}
> +
>  	return 0;
>  }
>  
> 

Re: [PATCH v3 3/3] KVM: nVMX: Fix mmu context after VMLAUNCH/VMRESUME failure

[Paolo Bonzini]

On 02/11/2017 09:31, Wanpeng Li wrote:
> +
> +	nested_ept_uninit_mmu_context(vcpu);
> +
> +	/*
> +	 * Only PDPTE load can fail as the value of cr3 was checked on entry and
> +	 * couldn't have changed.
> +	 */
> +	if (nested_vmx_load_cr3(vcpu, vmcs12->host_cr3, false, &entry_failure_code))
> +		nested_vmx_abort(vcpu, VMX_ABORT_LOAD_HOST_PDPTE_FAIL);
> +
> +	if (!enable_ept)
> +		vcpu->arch.walk_mmu->inject_page_fault = kvm_inject_page_fault;
> +

This code is duplicated from load_vmcs12_host_state.  Therefore, please
move it to a new function load_vmcs12_mmu_host_state.

Thanks,

Paolo

Re: [PATCH v3 1/3] KVM: X86: Fix operand size during instruction decoding

[Wanpeng Li]

2017-11-03 1:45 GMT+08:00 Paolo Bonzini <pbonzini@redhat.com>:
> On 02/11/2017 09:31, Wanpeng Li wrote:
>> From: Wanpeng Li <wanpeng.li@hotmail.com>
>>
>> Pedro reported:
>>   During tests that we conducted on KVM, we noticed that executing a "PUSH %ES"
>>   instruction under KVM produces different results on both memory and the SP
>>   register depending on whether EPT support is enabled. With EPT the SP is
>>   reduced by 4 bytes (and the written value is 0-padded) but without EPT support
>>   it is only reduced by 2 bytes. The difference can be observed when the CS.DB
>>   field is 1 (32-bit) but not when it's 0 (16-bit).
>>
>> The internal segment descriptor cache exist even in real/vm8096 mode. The CS.D
>> also should be respected instead of just default operand-size/66H prefix during
>> instruction decoding. This patch fixes it by also adjusting operand-size according
>> to CS.D.
>>
>> Reported-by: Pedro Fonseca <pfonseca@cs.washington.edu>
>> Tested-by: Pedro Fonseca <pfonseca@cs.washington.edu>
>> Cc: Paolo Bonzini <pbonzini@redhat.com>
>> Cc: Radim Krčmář <rkrcmar@redhat.com>
>> Cc: Nadav Amit <nadav.amit@gmail.com>
>> Cc: Pedro Fonseca <pfonseca@cs.washington.edu>
>> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
>> ---
>> v2 -> v3:
>>  * cleanup the codes
>> v1 -> v2:
>>  * respect cs.d for real/vm8096, other modes have already
>>    been considered in init_emulate_ctxt().
>>
>>  arch/x86/kvm/emulate.c | 7 +++++++
>>  1 file changed, 7 insertions(+)
>>
>> diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
>> index 8079d14..6ebc4cb 100644
>> --- a/arch/x86/kvm/emulate.c
>> +++ b/arch/x86/kvm/emulate.c
>> @@ -5000,6 +5000,8 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len)
>>       bool op_prefix = false;
>>       bool has_seg_override = false;
>>       struct opcode opcode;
>> +     u16 dummy;
>> +     struct desc_struct desc;
>>
>>       ctxt->memop.type = OP_NONE;
>>       ctxt->memopp = NULL;
>> @@ -5020,6 +5022,11 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len)
>>       case X86EMUL_MODE_VM86:
>>       case X86EMUL_MODE_PROT16:
>>               def_op_bytes = def_ad_bytes = 2;
>> +             if (mode < X86EMUL_MODE_PROT16) {
>> +                     ctxt->ops->get_segment(ctxt, &dummy, &desc, NULL, VCPU_SREG_CS);
>> +                     if (desc.d)
>> +                             def_op_bytes = 4;
>
> def_ad_bytes must be changed to 4 as well.  With that change, you should
> probably separate X86EMUL_MODE_PROT16 altogether from the others.

I just handle all the comments of the three patches in v5. :)

Regards,
Wanpeng Li