tlb_flush stat on Intel/AMD

[Jacob Xu <jacobhxu@google.com>]

Hi all,

I wrote a kvm selftest to enter a 1-vcpu guest VM running a nop in one
thread and check the VM's debugfs entry for # of tlb flushes while
that was running. Installing the latest upstream kernel and running
this on an intel host showed a tlb_flush count of 30, while running it
on an amd host shows the tlb_flush count at 0.

Do we have an inconsistency between Intel and AMD in how VCPU_STAT is
incremented?

From browsing the code, we see that the stat only gets incremented in
the kvm_ wrappers of the x86_ops functions tlb_flush_all,
tlb_flush_current, and tlb_flush_guest. These wrappers are only called
via the KVM request api (referred to as deferred flushes in some other
threads), and there other instances of calling the x86_ops tlb_flush
methods directly (immediate flush).

It looks like most of the tlb flush calls are deferred, but there are
a few instances using the immediate flush where it's not counted
(kvm_mmu_load, svm_set_cr4, vmx_set_apic_access_page,
nested_prepare_vmcb_control). Is there a guideline on when to
deferred/immediate tlb_flush?

Could this be a cause for the lower tlb_flush stat seen on an AMD
host? Or perhaps there's another reason for the difference due to the
(too) simple selftest?

In the case of svm_tlb_flush, it seems like the tlb flush is deferred
anyway since the response to setting a tlb flush control bit in the
VMCB is not acted upon until entering the guest. So it seems we could
count tlb flushes on svm more easily by incrementing the counter by
checking the control bit before KVM_RUN. Though perhaps there's
another case we'd like to count as tlb flush when the guest switches
ASID (where would we track this?).

Would switching to this alternative for incrementing tlb_flush stat in
svm be much different than what we do right now?

Thanks,

Jacob