Re: [PATCH v3 3/4] KVM: x86: Tie Intel and AMD behavior for MSR_TSC_AUX to guest CPU model

[Sean Christopherson <seanjc@google.com>]

On Sat, Apr 24, 2021, Reiji Watanabe wrote:
> > --- a/arch/x86/kvm/x86.c
> > +++ b/arch/x86/kvm/x86.c
> > @@ -1610,6 +1610,29 @@ static int __kvm_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data,
> >                  * invokes 64-bit SYSENTER.
> >                  */
> >                 data = get_canonical(data, vcpu_virt_addr_bits(vcpu));
> > +               break;
> > +       case MSR_TSC_AUX:
> > +               if (!kvm_cpu_cap_has(X86_FEATURE_RDTSCP))
> > +                       return 1;
> > +
> > +               if (!host_initiated &&
> > +                   !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP))
> > +                       return 1;
> > +
> > +               /*
> > +                * Per Intel's SDM, bits 63:32 are reserved, but AMD's APM has
> > +                * incomplete and conflicting architectural behavior.  Current
> > +                * AMD CPUs completely ignore bits 63:32, i.e. they aren't
> > +                * reserved and always read as zeros.  Enforce Intel's reserved
> > +                * bits check if and only if the guest CPU is Intel, and clear
> > +                * the bits in all other cases.  This ensures cross-vendor
> > +                * migration will provide consistent behavior for the guest.
> > +                */
> > +               if (guest_cpuid_is_intel(vcpu) && (data >> 32) != 0)
> > +                       return 1;
> > +
> > +               data = (u32)data;
> > +               break;
> >         }
> >
> >         msr.data = data;
> > @@ -1646,6 +1669,17 @@ int __kvm_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data,
> >         if (!host_initiated && !kvm_msr_allowed(vcpu, index, KVM_MSR_FILTER_READ))
> >                 return KVM_MSR_RET_FILTERED;
> >
> > +       switch (index) {
> > +       case MSR_TSC_AUX:
> > +               if (!kvm_cpu_cap_has(X86_FEATURE_RDTSCP))
> > +                       return 1;
> > +
> > +               if (!host_initiated &&
> > +                   !guest_cpuid_has(vcpu, X86_FEATURE_RDTSCP))
> > +                       return 1;
> 
> 
> It looks Table 2-2 of the Intel SDM Vol4 (April 2021) says
> TSC_AUX is supported:
> 
>    If CPUID.80000001H:EDX[27] = 1 or CPUID.(EAX=7,ECX=0):ECX[22] = 1
> 
> Should we also check X86_FEATURE_RDPID before returning 1
> due to no RDTSCP support ?

Yep.  VMX should also clear RDPID if the ENABLE_RDTSCP control isn't supported.
That bug isn't fatal because KVM emulates RDPID on #UD, but it would be a
notieable performance hit for the guest.

There is also a kernel bug lurking; vgetcpu_cpu_init() doesn't check
X86_FEATURE_RDPID and will fail to initialize MSR_TSC_AUX if RDPID is supported
but RDTSCP is not, and __getcpu() uses RDPID.  I'll verify that's broken and
send a patch for that one too.

> There doesn't seem to be a similar description in the APM though.

AMD also documents this in Appendix E:

  CPUID Fn0000_0007_EBX_x0 Structured Extended Feature Identifiers (ECX=0)
  Bits   Field   Name
  ...
  22     RDPID   RDPID instruction and TSC_AUX MSR support.