[PATCH v4 24/30] KVM: x86/mmu: Zap defunct roots via asynchronous worker

[Paolo Bonzini <pbonzini@redhat.com>]

Zap defunct roots, a.k.a. roots that have been invalidated after their
last reference was initially dropped, asynchronously via the system work
queue instead of forcing the work upon the unfortunate task that happened
to drop the last reference.

If a vCPU task drops the last reference, the vCPU is effectively blocked
by the host for the entire duration of the zap.  If the root being zapped
happens be fully populated with 4kb leaf SPTEs, e.g. due to dirty logging
being active, the zap can take several hundred seconds.  Unsurprisingly,
most guests are unhappy if a vCPU disappears for hundreds of seconds.

E.g. running a synthetic selftest that triggers a vCPU root zap with
~64tb of guest memory and 4kb SPTEs blocks the vCPU for 900+ seconds.
Offloading the zap to a worker drops the block time to <100ms.

Co-developed-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Ben Gardon <bgardon@google.com>
Message-Id: <20220226001546.360188-23-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 arch/x86/kvm/mmu/tdp_mmu.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
index e24a1bff9218..2456f880508d 100644
--- a/arch/x86/kvm/mmu/tdp_mmu.c
+++ b/arch/x86/kvm/mmu/tdp_mmu.c
@@ -170,13 +170,24 @@ void kvm_tdp_mmu_put_root(struct kvm *kvm, struct kvm_mmu_page *root,
 	 */
 	if (!kvm_tdp_root_mark_invalid(root)) {
 		refcount_set(&root->tdp_mmu_root_count, 1);
-		tdp_mmu_zap_root(kvm, root, shared);
 
 		/*
-		 * Give back the reference that was added back above.  We now
+		 * If the struct kvm is alive, we might as well zap the root
+		 * in a worker.  The worker takes ownership of the reference we
+		 * just added to root and is flushed before the struct kvm dies.
+		 */
+		if (likely(refcount_read(&kvm->users_count))) {
+			tdp_mmu_schedule_zap_root(kvm, root);
+			return;
+		}
+
+		/*
+		 * The struct kvm is being destroyed, zap synchronously and give
+		 * back immediately the reference that was added above.  We now
 		 * know that the root is invalid, so go ahead and free it if
 		 * no one has taken a reference in the meanwhile.
 		 */
+		tdp_mmu_zap_root(kvm, root, shared);
 		if (!refcount_dec_and_test(&root->tdp_mmu_root_count))
 			return;
 	}
-- 
2.31.1