[GIT PULL] KVM: x86: LAM support for 6.8

[Sean Christopherson <seanjc@google.com>]

LAM virtualization support.  FWIW, I intended to send this in early-ish
December as you've asked in the past, but December was basically a lost cause
for me in terms of doing upstream work.  :-/

The following changes since commit e9e60c82fe391d04db55a91c733df4a017c28b2f:

  selftests/kvm: fix compilation on non-x86_64 platforms (2023-11-21 11:58:25 -0500)

are available in the Git repository at:

  https://github.com/kvm-x86/linux.git tags/kvm-x86-lam-6.8

for you to fetch changes up to 183bdd161c2b773a62f01d1c030f5a3a5b7c33b5:

  KVM: x86: Use KVM-governed feature framework to track "LAM enabled" (2023-11-28 17:54:09 -0800)

----------------------------------------------------------------
KVM x86 support for virtualizing Linear Address Masking (LAM)

Add KVM support for Linear Address Masking (LAM).  LAM tweaks the canonicality
checks for most virtual address usage in 64-bit mode, such that only the most
significant bit of the untranslated address bits must match the polarity of the
last translated address bit.  This allows software to use ignored, untranslated
address bits for metadata, e.g. to efficiently tag pointers for address
sanitization.

LAM can be enabled separately for user pointers and supervisor pointers, and
for userspace LAM can be select between 48-bit and 57-bit masking

 - 48-bit LAM: metadata bits 62:48, i.e. LAM width of 15.
 - 57-bit LAM: metadata bits 62:57, i.e. LAM width of 6.

For user pointers, LAM enabling utilizes two previously-reserved high bits from
CR3 (similar to how PCID_NOFLUSH uses bit 63): LAM_U48 and LAM_U57, bits 62 and
61 respectively.  Note, if LAM_57 is set, LAM_U48 is ignored, i.e.:

 - CR3.LAM_U48=0 && CR3.LAM_U57=0 == LAM disabled for user pointers
 - CR3.LAM_U48=1 && CR3.LAM_U57=0 == LAM-48 enabled for user pointers
 - CR3.LAM_U48=x && CR3.LAM_U57=1 == LAM-57 enabled for user pointers

For supervisor pointers, LAM is controlled by a single bit, CR4.LAM_SUP, with
the 48-bit versus 57-bit LAM behavior following the current paging mode, i.e.:

 - CR4.LAM_SUP=0 && CR4.LA57=x == LAM disabled for supervisor pointers
 - CR4.LAM_SUP=1 && CR4.LA57=0 == LAM-48 enabled for supervisor pointers
 - CR4.LAM_SUP=1 && CR4.LA57=1 == LAM-57 enabled for supervisor pointers

The modified LAM canonicality checks:
 - LAM_S48                : [ 1 ][ metadata ][ 1 ]
                              63               47
 - LAM_U48                : [ 0 ][ metadata ][ 0 ]
                              63               47
 - LAM_S57                : [ 1 ][ metadata ][ 1 ]
                              63               56
 - LAM_U57 + 5-lvl paging : [ 0 ][ metadata ][ 0 ]
                              63               56
 - LAM_U57 + 4-lvl paging : [ 0 ][ metadata ][ 0...0 ]
                              63               56..47

The bulk of KVM support for LAM is to emulate LAM's modified canonicality
checks.  The approach taken by KVM is to "fill" the metadata bits using the
highest bit of the translated address, e.g. for LAM-48, bit 47 is sign-extended
to bits 62:48.  The most significant bit, 63, is *not* modified, i.e. its value
from the raw, untagged virtual address is kept for the canonicality check. This
untagging allows

Aside from emulating LAM's canonical checks behavior, LAM has the usual KVM
touchpoints for selectable features: enumeration (CPUID.7.1:EAX.LAM[bit 26],
enabling via CR3 and CR4 bits, etc.

----------------------------------------------------------------
Binbin Wu (9):
      KVM: x86: Consolidate flags for __linearize()
      KVM: x86: Add an emulation flag for implicit system access
      KVM: x86: Add X86EMUL_F_INVLPG and pass it in em_invlpg()
      KVM: x86/mmu: Drop non-PA bits when getting GFN for guest's PGD
      KVM: x86: Add & use kvm_vcpu_is_legal_cr3() to check CR3's legality
      KVM: x86: Remove kvm_vcpu_is_illegal_gpa()
      KVM: x86: Introduce get_untagged_addr() in kvm_x86_ops and call it in emulator
      KVM: x86: Untag addresses for LAM emulation where applicable
      KVM: x86: Use KVM-governed feature framework to track "LAM enabled"

Robert Hoo (3):
      KVM: x86: Virtualize LAM for supervisor pointer
      KVM: x86: Virtualize LAM for user pointer
      KVM: x86: Advertise and enable LAM (user and supervisor)

 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  5 +++-
 arch/x86/kvm/cpuid.c               |  2 +-
 arch/x86/kvm/cpuid.h               | 13 +++++----
 arch/x86/kvm/emulate.c             | 27 ++++++++++---------
 arch/x86/kvm/governed_features.h   |  1 +
 arch/x86/kvm/kvm_emulate.h         |  9 +++++++
 arch/x86/kvm/mmu.h                 |  8 ++++++
 arch/x86/kvm/mmu/mmu.c             |  2 +-
 arch/x86/kvm/mmu/mmu_internal.h    |  1 +
 arch/x86/kvm/mmu/paging_tmpl.h     |  2 +-
 arch/x86/kvm/svm/nested.c          |  4 +--
 arch/x86/kvm/vmx/nested.c          | 11 +++++---
 arch/x86/kvm/vmx/sgx.c             |  1 +
 arch/x86/kvm/vmx/vmx.c             | 55 ++++++++++++++++++++++++++++++++++++--
 arch/x86/kvm/vmx/vmx.h             |  2 ++
 arch/x86/kvm/x86.c                 | 18 +++++++++++--
 arch/x86/kvm/x86.h                 |  2 ++
 18 files changed, 134 insertions(+), 30 deletions(-)