[PATCH 00/28] KVM: x86/mmu: Overhaul TDP MMU zapping and flushing

[Sean Christopherson <seanjc@google.com>]

Overhaul TDP MMU's handling of zapping and TLB flushing to reduce the
number of TLB flushes, and to clean up the zapping code.  The final patch
realizes the biggest change, which is to use RCU to defer any TLB flush
due to zapping a SP to the caller.  The largest cleanup is to separate the
flows for zapping roots (zap _everything_), zapping leaf SPTEs (zap guest
mappings for whatever reason), and zapping a specific SP (NX recovery).
They're currently smushed into a single zap_gfn_range(), which was a good
idea at the time, but became a mess when trying to handle the different
rules, e.g. TLB flushes aren't needed when zapping a root because KVM can
safely zap a root if and only if it's unreachable.

For booting an 8 vCPU, remote_tlb_flush (requests) goes from roughly
180 (600) to 130 (215).

Please don't apply patches 02 and 03, they've been posted elsehwere and by
other people.  I included them here because some of the patches have
pseudo-dependencies on their changes.  Patch 01 is also posted separately.
I had a brain fart and sent it out realizing that doing so would lead to
oddities.

Hou Wenlong (1):
  KVM: x86/mmu: Skip tlb flush if it has been done in zap_gfn_range()

Sean Christopherson (27):
  KVM: x86/mmu: Use yield-safe TDP MMU root iter in MMU notifier
    unmapping
  KVM: x86/mmu: Remove spurious TLB flushes in TDP MMU zap collapsible
    path
  KVM: x86/mmu: Retry page fault if root is invalidated by memslot
    update
  KVM: x86/mmu: Check for present SPTE when clearing dirty bit in TDP
    MMU
  KVM: x86/mmu: Formalize TDP MMU's (unintended?) deferred TLB flush
    logic
  KVM: x86/mmu: Document that zapping invalidated roots doesn't need to
    flush
  KVM: x86/mmu: Drop unused @kvm param from kvm_tdp_mmu_get_root()
  KVM: x86/mmu: Require mmu_lock be held for write in unyielding root
    iter
  KVM: x86/mmu: Allow yielding when zapping GFNs for defunct TDP MMU
    root
  KVM: x86/mmu: Check for !leaf=>leaf, not PFN change, in TDP MMU SP
    removal
  KVM: x86/mmu: Batch TLB flushes from TDP MMU for MMU notifier
    change_spte
  KVM: x86/mmu: Drop RCU after processing each root in MMU notifier
    hooks
  KVM: x86/mmu: Add helpers to read/write TDP MMU SPTEs and document RCU
  KVM: x86/mmu: Take TDP MMU roots off list when invalidating all roots
  KVM: x86/mmu: WARN if old _or_ new SPTE is REMOVED in non-atomic path
  KVM: x86/mmu: Terminate yield-friendly walk if invalid root observed
  KVM: x86/mmu: Refactor low-level TDP MMU set SPTE helper to take raw
    vals
  KVM: x86/mmu: Zap only the target TDP MMU shadow page in NX recovery
  KVM: x86/mmu: Use common TDP MMU zap helper for MMU notifier unmap
    hook
  KVM: x86/mmu: Add TDP MMU helper to zap a root
  KVM: x86/mmu: Skip remote TLB flush when zapping all of TDP MMU
  KVM: x86/mmu: Use "zap root" path for "slow" zap of all TDP MMU SPTEs
  KVM: x86/mmu: Add dedicated helper to zap TDP MMU root shadow page
  KVM: x86/mmu: Require mmu_lock be held for write to zap TDP MMU range
  KVM: x86/mmu: Zap only TDP MMU leafs in kvm_zap_gfn_range()
  KVM: x86/mmu: Do remote TLB flush before dropping RCU in TDP MMU
    resched
  KVM: x86/mmu: Defer TLB flush to caller when freeing TDP MMU shadow
    pages

 arch/x86/kvm/mmu/mmu.c          |  74 +++--
 arch/x86/kvm/mmu/mmu_internal.h |   7 +-
 arch/x86/kvm/mmu/paging_tmpl.h  |   3 +-
 arch/x86/kvm/mmu/tdp_iter.c     |   6 +-
 arch/x86/kvm/mmu/tdp_iter.h     |  15 +-
 arch/x86/kvm/mmu/tdp_mmu.c      | 526 +++++++++++++++++++-------------
 arch/x86/kvm/mmu/tdp_mmu.h      |  48 +--
 7 files changed, 406 insertions(+), 273 deletions(-)

-- 
2.34.0.rc2.393.gf8c9666880-goog