Re: [PATCH v5 3/7] KVM: X86: Expose IA32_PKRS MSR

From: Sean Christopherson <seanjc@google.com>
To: Chenyi Qiang <chenyi.qiang@intel.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>, Joerg Roedel <joro@8bytes.org>,
	Xiaoyao Li <xiaoyao.li@intel.com>,
	kvm@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v5 3/7] KVM: X86: Expose IA32_PKRS MSR
Date: Mon, 8 Nov 2021 20:18:14 +0000	[thread overview]
Message-ID: <YYmGBhIbzgz+dyqp@google.com> (raw)
In-Reply-To: <20210811101126.8973-4-chenyi.qiang@intel.com>

On Wed, Aug 11, 2021, Chenyi Qiang wrote:
> @@ -7207,6 +7257,19 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
>  
>  	/* Refresh #PF interception to account for MAXPHYADDR changes. */
>  	vmx_update_exception_bitmap(vcpu);
> +
> +	if (kvm_cpu_cap_has(X86_FEATURE_PKS) &&
> +	    guest_cpuid_has(vcpu, X86_FEATURE_PKS)) {

Ah, this confused me for a second.  It's not wrong to clear the entry/exit controls
in the "else" path, but it's surprisingly hard to follow because it reads as if the
entry/exit controls are paired with the MSR behavior.

Oh, and more importantly, it's "hiding" a bug: the MSR bitmap needs to be _set_
if userspace disables X86_FEATURE_PKS in guest CPUID, e.g. if for some reason
userspace exposed PKS and then yanked it away.

Oof, two bugs actually.  This will fail to re-enable the entry/exit bits if
userspace hides PKS and then re-enables PKS.

Heh, make that three bugs.  If userspace never sets CPUID, KVM will run with
the entry/exit bits set.  That's arguably not a bug since functionally it's fine,
but it's a bug in the sense that KVM loads an MSR when it doesn't inted to do so.

So this should be:

	if (kvm_vcpu_cap_has(X86_FEATURE_PKS) {
		if (guest_cpuid_has(vcpu, X86_FEATURE_PKS)) {
			vmx_disable_intercept_for_msr(vcpu, MSR_IA32_PKRS, MSR_TYPE_RW);

			vm_entry_controls_setbit(vmx, VM_ENTRY_LOAD_IA32_PKRS);
			vm_exit_controls_setbit(vmx, VM_EXIT_LOAD_IA32_PKRS)

		} else {
			vmx_enable_intercept_for_msr(vcpu, MSR_IA32_PKRS, MSR_TYPE_RW);

			vm_entry_controls_clearbit(vmx, VM_ENTRY_LOAD_IA32_PKRS);
			vm_exit_controls_clearbit(vmx, VM_EXIT_LOAD_IA32_PKRS)
		}
	}

and then the bits need to be masked in vmx_vmexit_ctrl() and vmx_vmentry_ctrl(),
a la EFER and PERF_GLOBAL_CTRL.

> +		vmx_disable_intercept_for_msr(vcpu, MSR_IA32_PKRS, MSR_TYPE_RW);
> +	} else {
> +		/*
> +		 * Remove VM control in case guest VM doesn't support PKS to mitigate
> +		 * overhead during VM-{exit,entry}. They are present by default
> +		 * if supported.
> +		 */
> +		vm_entry_controls_clearbit(vmx, VM_ENTRY_LOAD_IA32_PKRS);
> +		vm_exit_controls_clearbit(vmx, VM_EXIT_LOAD_IA32_PKRS);
> +	}
>  }