Re: KVM_SET_NESTED_STATE not yet stable

From: Jan Kiszka <jan.kiszka@web.de>
To: Paolo Bonzini <pbonzini@redhat.com>,
	Ralf Ramsauer <ralf.ramsauer@oth-regensburg.de>,
	"Raslan, KarimAllah" <karahmed@amazon.de>,
	"jmattson@google.com" <jmattson@google.com>,
	"liran.alon@oracle.com" <liran.alon@oracle.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>
Subject: Re: KVM_SET_NESTED_STATE not yet stable
Date: Sun, 21 Jul 2019 11:05:26 +0200	[thread overview]
Message-ID: <c44c9d59-3ef0-cda9-fd4a-6e6c67fd9e71@web.de> (raw)
In-Reply-To: <68880241-ff91-1cb1-1bd5-ab5d2e307bec@redhat.com>

On 19.07.19 18:38, Paolo Bonzini wrote:
> On 11/07/19 19:30, Paolo Bonzini wrote:
>> On 11/07/19 13:37, Ralf Ramsauer wrote:
>>> I can reproduce and confirm this issue. A system_reset of qemu after
>>> Jailhouse is enabled leads to the crash listed below, on all machines.
>>>
>>> On the Xeon Gold, e.g., Qemu reports:
>>>
>>> EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000f61
>>> ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
>>> EIP=0000fff0 EFL=00000246 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
>>> ES =0000 00000000 0000ffff 00009300
>>> CS =f000 ffff0000 0000ffff 00a09b00
>>> SS =0000 00000000 0000ffff 00c09300
>>> DS =0000 00000000 0000ffff 00009300
>>> FS =0000 00000000 0000ffff 00009300
>>> GS =0000 00000000 0000ffff 00009300
>>> LDT=0000 00000000 0000ffff 00008200
>>> TR =0000 00000000 0000ffff 00008b00
>>> GDT=     00000000 0000ffff
>>> IDT=     00000000 0000ffff
>>> CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000680
>>> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
>>> DR3=0000000000000000
>>> DR6=00000000ffff0ff0 DR7=0000000000000400
>>> EFER=0000000000000000
>>> Code=00 66 89 d8 66 e8 af a1 ff ff 66 83 c4 0c 66 5b 66 5e 66 c3 <ea> 5b
>>> e0 00 f0 30 36 2f 32 33 2f 39 39 00 fc 00 00 00 00 00 00 00 00 00 00 00
>>> 00 00 00 00
>>>
>>> Kernel:
>>> [ 1868.804515] kvm: vmptrld           (null)/6b8640000000 failed
>>> [ 1868.804568] kvm: vmclear fail:           (null)/6b8640000000
>>>
>>> And the host freezes unrecoverably. Hosts use standard distro kernels
>>
>> Thanks.  I'm going to look at it tomorrow.
>
> Ok, it was only tomorrow modulo 7, but the first fix I got is trivial:
>
> diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
> index 6e88f459b323..6119b30347c6 100644
> --- a/arch/x86/kvm/vmx/nested.c
> +++ b/arch/x86/kvm/vmx/nested.c
> @@ -194,6 +194,7 @@ static void vmx_disable_shadow_vmcs(struct vcpu_vmx *vmx)
>  {
>  	secondary_exec_controls_clearbit(vmx, SECONDARY_EXEC_SHADOW_VMCS);
>  	vmcs_write64(VMCS_LINK_POINTER, -1ull);
> +	vmx->nested.need_vmcs12_to_shadow_sync = false;
>  }
>
>  static inline void nested_release_evmcs(struct kvm_vcpu *vcpu)
>
> Can you try it and see what you get?
>

Confirmed that this fixes the host crashes for me as well.

Now I'm only still seeing guest corruptions on vmport/vmmouse accesses from L2.
Looking into that right now.

Jan