[PATCH v4 0/8] KVM: x86: VMX ctrl MSR + KVM quirk fixes

[Oliver Upton <oupton@google.com>]

There are a few bits in the VMX entry/exit control MSRs where KVM
intervenes. The "load IA32_PERF_GLOBAL_CTRL" and "{load,clear}
IA32_BNDCFGS" VM-{Entry,Exit} control bits are under KVM control and
conditionally exposed based on the guest CPUID. If the guest CPUID
provides a supporting vPMU or MPX, the respective VMX control bits are
enabled.

These rules have not been upheld in all cases, though. Since commit
aedbaf4f6afd ("KVM: x86: Extract kvm_update_cpuid_runtime() from
kvm_update_cpuid()") KVM will only apply its updates to the MSRs
when the guest CPUID is set. Before, KVM called kvm_update_cpuid()
frequently when running a guest, which had the effect of overriding
any userspace setting of these MSRs.

If an unsuspecting VMM writes to these VMX control MSRs after the
CPUID has been set, KVM fails to configure the appropriate bits.
There does not exist any ordering requirements between setting CPUID
and writing to an MSR.

At the same time, we probably want to get KVM out of the business of
fiddling with these control MSRs. This series adds a quirk that allows
userspace to opt-out of KVM tweaks to these MSRs.

[Patch 1-2]
Fix the immediate issue by hooking writes to the VMX control MSRs. If
userspace writes to one of the affected MSRs, reapply KVMs tweaks to
these registers. Note that these patches employ the minimal change
required to fix the issue, in case they are worthy of a backport.

[Patch 3]
With the hook added in Patch 2, updating
IA32_VMX_TRUE_{ENTRY,EXIT}_CTLS MSRs is unnecessary on PMU refresh. Drop
everything related to updating these controls on PMU refresh.

[Patch 4]
KVM_CAP_DISABLE_QUIRKS2 is broken beyond repair. Create a new capability
that makes quirks discoverable and rejects invalid bits.

[Patch 5]
Add a quirk to opt out of KVM ownership of the aforementioned MSRs. It
is really userspace's responsibility to set up sane vCPU state.

[Patches 6-8]
Add test cases to verify expected behavior with the quirk enabled (KVM
control) and quirk disabled (userspace control).

Applies cleanly to kvm/queue, at the following commit:

  625e7ef7da1a ("KVM: selftests: Add test to verify KVM handling of ICR")

Tested with the included selftest on an Intel Skylake machine.

v3: http://lore.kernel.org/r/20220225200823.2522321-1-oupton@google.com

v3 -> v4:
 - Rebased to kvm/queue. Avoids conflicts with new CAPs and commit
   0bcd556e15f9 ("KVM: nVMX: Refactor PMU refresh to avoid referencing
   kvm_x86_ops.pmu_ops") on kvm/queue.
 - Grabbed KVM_CAP_DISABLE_QUIRKS2 patch, since this series also
   introduces a quirk.
 - Fix typo in KVM_CAP_DISABLE_QUIRKS2 documentation (Sean)
 - Eliminated the need to refresh 'load IA32_PGC' bits from PMU refresh.
 - Use consistent formatting to make test cases more easily readable
   (David Dunn)
 - Use correct 'Fixes: ' tag and correct a typo in Patch 2 changelog.

Oliver Upton (8):
  KVM: nVMX: Keep KVM updates to BNDCFGS ctrl bits across MSR write
  KVM: nVMX: Keep KVM updates to PERF_GLOBAL_CTRL ctrl bits across MSR
    write
  KVM: nVMX: Drop nested_vmx_pmu_refresh()
  KVM: x86: Introduce KVM_CAP_DISABLE_QUIRKS2
  KVM: nVMX: Add a quirk for KVM tweaks to VMX control MSRs
  selftests: KVM: Separate static alloc from KVM_GET_SUPPORTED_CPUID
    call
  selftests: KVM: Add test for PERF_GLOBAL_CTRL VMX control MSR bits
  selftests: KVM: Add test for BNDCFGS VMX control MSR bits

 Documentation/virt/kvm/api.rst                |  74 +++++
 arch/x86/include/asm/kvm_host.h               |   8 +
 arch/x86/include/uapi/asm/kvm.h               |  11 +-
 arch/x86/kvm/pmu.h                            |   5 +
 arch/x86/kvm/vmx/nested.c                     |  31 +--
 arch/x86/kvm/vmx/nested.h                     |   2 -
 arch/x86/kvm/vmx/pmu_intel.c                  |   3 -
 arch/x86/kvm/vmx/vmx.c                        |  17 +-
 arch/x86/kvm/vmx/vmx.h                        |   2 +
 arch/x86/kvm/x86.c                            |   8 +
 include/uapi/linux/kvm.h                      |   1 +
 tools/testing/selftests/kvm/.gitignore        |   1 +
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/include/x86_64/processor.h  |   1 +
 .../selftests/kvm/include/x86_64/vmx.h        |   2 +
 .../selftests/kvm/lib/x86_64/processor.c      |  33 ++-
 .../kvm/x86_64/vmx_control_msrs_test.c        | 257 ++++++++++++++++++
 17 files changed, 418 insertions(+), 39 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86_64/vmx_control_msrs_test.c

-- 
2.35.1.574.g5d30c73bfb-goog