Re: [PATCH v3 1/3] KVM: nVMX: Handle dynamic MSR intercept toggling

From: Sean Christopherson <seanjc@google.com>
To: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
	Alexander Graf <graf@amazon.com>,
	Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [PATCH v3 1/3] KVM: nVMX: Handle dynamic MSR intercept toggling
Date: Mon, 8 Nov 2021 17:24:54 +0000	[thread overview]
Message-ID: <YYldZjBA0YOHjUdZ@google.com> (raw)
In-Reply-To: <87k0hioasv.fsf@vitty.brq.redhat.com>

On Mon, Nov 08, 2021, Vitaly Kuznetsov wrote:
> Sean Christopherson <seanjc@google.com> writes:
> > @@ -6749,7 +6686,9 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)
> >  	 * If the L02 MSR bitmap does not intercept the MSR, then we need to
> >  	 * save it.
> >  	 */
> > -	if (unlikely(!msr_write_intercepted(vcpu, MSR_IA32_SPEC_CTRL)))
> > +	if (unlikely(cpu_has_vmx_msr_bitmap() &&
> > +		     vmx_test_msr_bitmap_write(vmx->loaded_vmcs->msr_bitmap,
> > +					       MSR_IA32_SPEC_CTRL)))

Ugh, I inverted the check, '1' == intercept.  IIRC, I open coded the intercept
check because SPEC_CTRL is really the only case where should be reading _only_
the current VMCS's MSR bitmap.

I'll spin a new version of the series and test with SPEC_CTRL disabled in a VM,
and maybe revist my reasoning for this.

Thanks!

> >  		vmx->spec_ctrl = native_read_msr(MSR_IA32_SPEC_CTRL);
> 
> I smoke-tested this patch by running (unrelated) selftests when I tried
> to put in into my 'Enlightened MSR Bitmap v4' series and my dmesg got
> flooded with:
> 
> [   87.210214] unchecked MSR access error: RDMSR from 0x48 at rIP: 0xffffffffc04e0284 (native_read_msr+0x4/0x30 [kvm_intel])
> [   87.210325] Call Trace:
> [   87.210355]  vmx_vcpu_run+0xcc7/0x12b0 [kvm_intel]
> [   87.210405]  ? vmx_prepare_switch_to_guest+0x138/0x1f0 [kvm_intel]
> [   87.210466]  vcpu_enter_guest+0x98c/0x1380 [kvm]
> [   87.210631]  ? vmx_vcpu_put+0x2e/0x1f0 [kvm_intel]
> [   87.210678]  ? vmx_vcpu_load+0x21/0x60 [kvm_intel]
> [   87.210729]  kvm_arch_vcpu_ioctl_run+0xdf/0x580 [kvm]
> [   87.210844]  kvm_vcpu_ioctl+0x274/0x660 [kvm]
> [   87.210950]  __x64_sys_ioctl+0x83/0xb0
> [   87.210996]  do_syscall_64+0x3b/0x90
> [   87.211039]  entry_SYSCALL_64_after_hwframe+0x44/0xae
> [   87.211093] RIP: 0033:0x7f6ef7f9a307
> [   87.211134] Code: 44 00 00 48 8b 05 69 1b 2d 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 39 1b 2d 00 f7 d8 64 89 01 48
> [   87.211293] RSP: 002b:00007ffcacfb3b18 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
> [   87.211367] RAX: ffffffffffffffda RBX: 0000000000a2f300 RCX: 00007f6ef7f9a307
> [   87.211434] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000007
> [   87.211500] RBP: 0000000000000000 R08: 000000000040e769 R09: 0000000000000000
> [   87.211559] R10: 0000000000a2f001 R11: 0000000000000246 R12: 0000000000a2d010
> [   87.211622] R13: 0000000000a2d010 R14: 0000000000402a15 R15: 00000000ffff0ff0
> [   87.212520] Call Trace:
> [   87.212597]  vmx_vcpu_run+0xcc7/0x12b0 [kvm_intel]
> [   87.212683]  ? vmx_prepare_switch_to_guest+0x138/0x1f0 [kvm_intel]
> [   87.212789]  vcpu_enter_guest+0x98c/0x1380 [kvm]
> [   87.213059]  ? vmx_vcpu_put+0x2e/0x1f0 [kvm_intel]
> [   87.213141]  ? schedule+0x44/0xa0
> [   87.213200]  kvm_arch_vcpu_ioctl_run+0xdf/0x580 [kvm]
> [   87.213428]  kvm_vcpu_ioctl+0x274/0x660 [kvm]
> [   87.213633]  __x64_sys_ioctl+0x83/0xb0
> [   87.213705]  do_syscall_64+0x3b/0x90
> [   87.213766]  entry_SYSCALL_64_after_hwframe+0x44/0xae
> ...
> 
> this was an old 'E5-2603 v3' CPU. Any idea what's wrong?