RFC: Task switch emulation fails for VM86 mode

RFC: Task switch emulation fails for VM86 mode

[Wanpeng Li]

Hi all,

I found that task switch emulation fails to work for VM86 mode if
guest state is invalid. It can be reproduced by running
kvm-unit-tests/taskswitch2.flat, EPT = 0 or EPT=1,
unrestricted_guest=N, emulate_invalid_guest_state=Y.

When EPT=1, unrestricted_guest=Y, emulate_invalid_state=Y, the trace
of kvm-unit-tests/taskswitch2.flat is like below:

kvm_entry: vcpu 0
kvm_exit: reason TASK_SWITCH rip 0x4008d0 info 40000058 0
kvm_entry: vcpu 0
kvm_exit: reason EXCEPTION_NMI rip 0x0 info 0 80000306
kvm_emulate_insn: 42000:0:0f 0b (0x2)
kvm_inj_exception: #UD (0x0)
kvm_entry: vcpu 0
kvm_exit: reason TASK_SWITCH rip 0x0 info c0000050 0
kvm_entry: vcpu 0

When EPT=1, unrestricted_guest=Y, emulate_invalid_state=Y, the trace
of kvm-unit-tests/taskswitch2.flat is like below, when emulating "0f
0b(#UD)" fails, it injects another #UD and triggers a task switch in
the kvm-unit-tests/taskswitch2.flat again.

kvm_exit: reason TASK_SWITCH rip 0x0 info 40000058 0
kvm_emulate_insn: 42000:0:0f 0b (0x2)
kvm_emulate_insn: 42000:0:0f 0b (0x2) failed
kvm_inj_exception: #UD (0x0)
kvm_entry: vcpu 0
kvm_exit: reason TASK_SWITCH rip 0x0 info 40000058 0
kvm_emulate_insn: 42000:0:0f 0b (0x2)
kvm_emulate_insn: 42000:0:0f 0b (0x2) failed
kvm_inj_exception: #UD (0x0)
kvm_entry: vcpu 0
kvm_exit: reason TASK_SWITCH rip 0x0 info 40000058 0
kvm_emulate_insn: 42000:0:0f 0b (0x2)
kvm_emulate_insn: 42000:0:0f 0b (0x2) failed
kvm_inj_exception: #UD (0x0)
.....................

Then I try to add the task switch emulation in the
handle_invalid_guest_state() path, however, I will get TRIPLE FAULT.

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index f76efad..758f728 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -6312,11 +6312,15 @@ static int handle_invalid_guest_state(struct
kvm_vcpu *vcpu)
     u32 cpu_exec_ctrl;
     bool intr_window_requested;
     unsigned count = 130;
+    u32 exit_reason = vmx->exit_reason;

     cpu_exec_ctrl = vmcs_read32(CPU_BASED_VM_EXEC_CONTROL);
     intr_window_requested = cpu_exec_ctrl & CPU_BASED_VIRTUAL_INTR_PENDING;

     while (vmx->emulation_required && count-- != 0) {
+        if (exit_reason == EXIT_REASON_TASK_SWITCH)
+            return handle_task_switch(vcpu);
+
         if (intr_window_requested && vmx_interrupt_allowed(vcpu))
             return handle_interrupt_window(&vmx->vcpu);


kvm_exit: reason TASK_SWITCH rip 0x4008d0 info 40000058 0
kvm_entry: vcpu 0
kvm_exit: reason TASK_SWITCH rip 0x0 info 40000058 0
kvm_entry: vcpu 0
kvm_exit: reason TRIPLE_FAULT rip 0xffff info 0 0
kvm_userspace_exit: reason KVM_EXIT_SHUTDOWN (8)

Any proposal is a great appreciated. :)

Regards,
Wanpeng Li

Re: RFC: Task switch emulation fails for VM86 mode

[Wanpeng Li]

Cc Nadav, Jan,
2017-07-09 15:30 GMT+08:00 Wanpeng Li <kernellwp@gmail.com>:
> Hi all,
>
> I found that task switch emulation fails to work for VM86 mode if
> guest state is invalid. It can be reproduced by running
> kvm-unit-tests/taskswitch2.flat, EPT = 0 or EPT=1,
> unrestricted_guest=N, emulate_invalid_guest_state=Y.
>
> When EPT=1, unrestricted_guest=Y, emulate_invalid_state=Y, the trace
> of kvm-unit-tests/taskswitch2.flat is like below:
>
> kvm_entry: vcpu 0
> kvm_exit: reason TASK_SWITCH rip 0x4008d0 info 40000058 0
> kvm_entry: vcpu 0
> kvm_exit: reason EXCEPTION_NMI rip 0x0 info 0 80000306
> kvm_emulate_insn: 42000:0:0f 0b (0x2)
> kvm_inj_exception: #UD (0x0)
> kvm_entry: vcpu 0
> kvm_exit: reason TASK_SWITCH rip 0x0 info c0000050 0
> kvm_entry: vcpu 0
>
> When EPT=1, unrestricted_guest=Y, emulate_invalid_state=Y, the trace
> of kvm-unit-tests/taskswitch2.flat is like below, when emulating "0f
> 0b(#UD)" fails, it injects another #UD and triggers a task switch in
> the kvm-unit-tests/taskswitch2.flat again.
>
> kvm_exit: reason TASK_SWITCH rip 0x0 info 40000058 0
> kvm_emulate_insn: 42000:0:0f 0b (0x2)
> kvm_emulate_insn: 42000:0:0f 0b (0x2) failed
> kvm_inj_exception: #UD (0x0)
> kvm_entry: vcpu 0
> kvm_exit: reason TASK_SWITCH rip 0x0 info 40000058 0
> kvm_emulate_insn: 42000:0:0f 0b (0x2)
> kvm_emulate_insn: 42000:0:0f 0b (0x2) failed
> kvm_inj_exception: #UD (0x0)
> kvm_entry: vcpu 0
> kvm_exit: reason TASK_SWITCH rip 0x0 info 40000058 0
> kvm_emulate_insn: 42000:0:0f 0b (0x2)
> kvm_emulate_insn: 42000:0:0f 0b (0x2) failed
> kvm_inj_exception: #UD (0x0)
> .....................
>
> Then I try to add the task switch emulation in the
> handle_invalid_guest_state() path, however, I will get TRIPLE FAULT.
>
> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> index f76efad..758f728 100644
> --- a/arch/x86/kvm/vmx.c
> +++ b/arch/x86/kvm/vmx.c
> @@ -6312,11 +6312,15 @@ static int handle_invalid_guest_state(struct
> kvm_vcpu *vcpu)
>      u32 cpu_exec_ctrl;
>      bool intr_window_requested;
>      unsigned count = 130;
> +    u32 exit_reason = vmx->exit_reason;
>
>      cpu_exec_ctrl = vmcs_read32(CPU_BASED_VM_EXEC_CONTROL);
>      intr_window_requested = cpu_exec_ctrl & CPU_BASED_VIRTUAL_INTR_PENDING;
>
>      while (vmx->emulation_required && count-- != 0) {
> +        if (exit_reason == EXIT_REASON_TASK_SWITCH)
> +            return handle_task_switch(vcpu);
> +
>          if (intr_window_requested && vmx_interrupt_allowed(vcpu))
>              return handle_interrupt_window(&vmx->vcpu);
>
>
> kvm_exit: reason TASK_SWITCH rip 0x4008d0 info 40000058 0
> kvm_entry: vcpu 0
> kvm_exit: reason TASK_SWITCH rip 0x0 info 40000058 0
> kvm_entry: vcpu 0
> kvm_exit: reason TRIPLE_FAULT rip 0xffff info 0 0
> kvm_userspace_exit: reason KVM_EXIT_SHUTDOWN (8)
>
> Any proposal is a great appreciated. :)
>
> Regards,
> Wanpeng Li

Re: RFC: Task switch emulation fails for VM86 mode

[Nadav Amit]

Wanpeng Li <kernellwp@gmail.com> wrote:

> Cc Nadav, Jan,
> 2017-07-09 15:30 GMT+08:00 Wanpeng Li <kernellwp@gmail.com>:
>> Hi all,
>> 
>> I found that task switch emulation fails to work for VM86 mode if
>> guest state is invalid. It can be reproduced by running
>> kvm-unit-tests/taskswitch2.flat, EPT = 0 or EPT=1,
>> unrestricted_guest=N, emulate_invalid_guest_state=Y.
>> 
>> When EPT=1, unrestricted_guest=Y, emulate_invalid_state=Y, the trace
>> of kvm-unit-tests/taskswitch2.flat is like below:
>> 
>> kvm_entry: vcpu 0
>> kvm_exit: reason TASK_SWITCH rip 0x4008d0 info 40000058 0
>> kvm_entry: vcpu 0
>> kvm_exit: reason EXCEPTION_NMI rip 0x0 info 0 80000306
>> kvm_emulate_insn: 42000:0:0f 0b (0x2)
>> kvm_inj_exception: #UD (0x0)
>> kvm_entry: vcpu 0
>> kvm_exit: reason TASK_SWITCH rip 0x0 info c0000050 0
>> kvm_entry: vcpu 0
>> 
>> When EPT=1, unrestricted_guest=Y, emulate_invalid_state=Y, the trace
>> of kvm-unit-tests/taskswitch2.flat is like below, when emulating "0f
>> 0b(#UD)" fails, it injects another #UD and triggers a task switch in
>> the kvm-unit-tests/taskswitch2.flat again.
>> 
>> kvm_exit: reason TASK_SWITCH rip 0x0 info 40000058 0
>> kvm_emulate_insn: 42000:0:0f 0b (0x2)
>> kvm_emulate_insn: 42000:0:0f 0b (0x2) failed
>> kvm_inj_exception: #UD (0x0)
>> kvm_entry: vcpu 0
>> kvm_exit: reason TASK_SWITCH rip 0x0 info 40000058 0
>> kvm_emulate_insn: 42000:0:0f 0b (0x2)
>> kvm_emulate_insn: 42000:0:0f 0b (0x2) failed
>> kvm_inj_exception: #UD (0x0)
>> kvm_entry: vcpu 0
>> kvm_exit: reason TASK_SWITCH rip 0x0 info 40000058 0
>> kvm_emulate_insn: 42000:0:0f 0b (0x2)
>> kvm_emulate_insn: 42000:0:0f 0b (0x2) failed
>> kvm_inj_exception: #UD (0x0)
>> .....................
>> 
>> Then I try to add the task switch emulation in the
>> handle_invalid_guest_state() path, however, I will get TRIPLE FAULT.
>> 
>> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
>> index f76efad..758f728 100644
>> --- a/arch/x86/kvm/vmx.c
>> +++ b/arch/x86/kvm/vmx.c
>> @@ -6312,11 +6312,15 @@ static int handle_invalid_guest_state(struct
>> kvm_vcpu *vcpu)
>>     u32 cpu_exec_ctrl;
>>     bool intr_window_requested;
>>     unsigned count = 130;
>> +    u32 exit_reason = vmx->exit_reason;
>> 
>>     cpu_exec_ctrl = vmcs_read32(CPU_BASED_VM_EXEC_CONTROL);
>>     intr_window_requested = cpu_exec_ctrl & CPU_BASED_VIRTUAL_INTR_PENDING;
>> 
>>     while (vmx->emulation_required && count-- != 0) {
>> +        if (exit_reason == EXIT_REASON_TASK_SWITCH)
>> +            return handle_task_switch(vcpu);
>> +
>>         if (intr_window_requested && vmx_interrupt_allowed(vcpu))
>>             return handle_interrupt_window(&vmx->vcpu);
>> 
>> 
>> kvm_exit: reason TASK_SWITCH rip 0x4008d0 info 40000058 0
>> kvm_entry: vcpu 0
>> kvm_exit: reason TASK_SWITCH rip 0x0 info 40000058 0
>> kvm_entry: vcpu 0
>> kvm_exit: reason TRIPLE_FAULT rip 0xffff info 0 0
>> kvm_userspace_exit: reason KVM_EXIT_SHUTDOWN (8)
>> 
>> Any proposal is a great appreciated. :)

I don’t see a (very) easy solution. The code was (apparently) never built to
deal with a task switch during an instruction emulation.

AFAIU kvm_task_switch() expects information about the task-switch from the
CPU “task-switch assist” mechanisms, and this information (or even the fact
that a task-switch is needed due to an exception) are unavailable from the
instruction emulator. The instruction emulator itself does not know to
emulate task-switches, e.g., during far CALL and JMP.

A complete solution is therefore complicated and requires some work. Your
specific problem may be addressed by detecting the injection of an exception
while having invalid guest state in vm86 in vmx_queue_exception() or in
handle_invalid_guest_state(), and emulating the “task-switch assist”
mechanism.

Re: RFC: Task switch emulation fails for VM86 mode

[Paolo Bonzini]

On 10/07/2017 17:48, Nadav Amit wrote:
>>>
>>> Any proposal is a great appreciated. :)
> I don’t see a (very) easy solution. The code was (apparently) never built to
> deal with a task switch during an instruction emulation.
> 
> AFAIU kvm_task_switch() expects information about the task-switch from the
> CPU “task-switch assist” mechanisms, and this information (or even the fact
> that a task-switch is needed due to an exception) are unavailable from the
> instruction emulator. The instruction emulator itself does not know to
> emulate task-switches, e.g., during far CALL and JMP.
> 
> A complete solution is therefore complicated and requires some work. Your
> specific problem may be addressed by detecting the injection of an exception
> while having invalid guest state in vm86 in vmx_queue_exception() or in
> handle_invalid_guest_state(), and emulating the “task-switch assist”
> mechanism.

I agree, the right solution would be to read the IDT in
vmx_queue_exception if vmx->emulation_required, and inject the exception
manually.  It would be an extension of what
kvm_inject_realmode_interrupt already does.

Paolo

Re: RFC: Task switch emulation fails for VM86 mode

[Nadav Amit]

Paolo Bonzini <pbonzini@redhat.com> wrote:

> On 10/07/2017 17:48, Nadav Amit wrote:
>>>> Any proposal is a great appreciated. :)
>> I don’t see a (very) easy solution. The code was (apparently) never built to
>> deal with a task switch during an instruction emulation.
>> 
>> AFAIU kvm_task_switch() expects information about the task-switch from the
>> CPU “task-switch assist” mechanisms, and this information (or even the fact
>> that a task-switch is needed due to an exception) are unavailable from the
>> instruction emulator. The instruction emulator itself does not know to
>> emulate task-switches, e.g., during far CALL and JMP.
>> 
>> A complete solution is therefore complicated and requires some work. Your
>> specific problem may be addressed by detecting the injection of an exception
>> while having invalid guest state in vm86 in vmx_queue_exception() or in
>> handle_invalid_guest_state(), and emulating the “task-switch assist”
>> mechanism.
> 
> I agree, the right solution would be to read the IDT in
> vmx_queue_exception if vmx->emulation_required, and inject the exception
> manually.  It would be an extension of what
> kvm_inject_realmode_interrupt already does.

I take it back. While everything I said is true, there is no reason for the
guest state to be invalid in vm86, at least in the unit-test. It appears
that the task-switch emulation updates rflags (and vm86 flag) only after the
segments are loaded, causing vmx->emulation_required to be set, when in fact
emulation is not needed.

And indeed adding in the end of handle_task_switch():

	vmx->emulation_required = emulation_required(vcpu); 

solves the problem for me.