[PATCH 0/4] KVM: MMU: fast cleanup D bit based on fast write protect

From: Zhuangyanying <ann.zhuangyanying@huawei.com>
To: <xiaoguangrong@tencent.com>, <pbonzini@redhat.com>,
	<arei.gonglei@huawei.com>
Cc: liu.jinsong@huawei.com, wangxinxin.wang@huawei.com,
	Zhuang Yanying <ann.zhuangyanying@huawei.com>,
	qemu-devel@nongnu.org, kvm@vger.kernel.org
Subject: [PATCH 0/4] KVM: MMU: fast cleanup D bit based on fast write protect
Date: Thu, 17 Jan 2019 13:55:27 +0000	[thread overview]
Message-ID: <1547733331-16140-1-git-send-email-ann.zhuangyanying@huawei.com> (raw)

From: Zhuang Yanying <ann.zhuangyanying@huawei.com>

Recently I tested live-migration with large-memory guests, find vcpu may hang for a long time while starting migration, such as 9s for 2048G(linux-5.0.0-rc2+qemu-3.1.0).
The reason is memory_global_dirty_log_start() taking too long, and the vcpu is waiting for BQL. The page-by-page D bit clearup is the main time consumption.
I think that the idea of "KVM: MMU: fast write protect" by xiaoguangrong, especially the function kvm_mmu_write_protect_all_pages(), is very helpful.
After a little modifcation, on his patch, can solve this problem, 9s to 0.5s.

At the begining of live migration, write protection is only applied to the top-level SPTE. Then the write from vm trigger the EPT violation, with for_each_shadow_entry write protection is performed at dirct_map.
Finally the Dirty bit of the target page(at level 1 page table) is cleared, and the dirty page tracking is started. Of coure, the page where GPA is located is marked dirty when mmu_set_spte.
A similar implementation on xen, just emt instead of write protection.

What do you think about this solution?

Xiao Guangrong (3):
  KVM: MMU: correct the behavior of mmu_spte_update_no_track
  KVM: MMU: introduce possible_writable_spte_bitmap
  KVM: MMU: introduce kvm_mmu_write_protect_all_pages

Zhuang Yanying (1):
  KVM: MMU: fast cleanup D bit based on fast write protect

 arch/x86/include/asm/kvm_host.h |  25 +++-
 arch/x86/kvm/mmu.c              | 259 ++++++++++++++++++++++++++++++++++++++--
 arch/x86/kvm/mmu.h              |   1 +
 arch/x86/kvm/paging_tmpl.h      |  13 +-
 arch/x86/kvm/vmx/vmx.c          |   3 +-
 5 files changed, 286 insertions(+), 15 deletions(-)

-- 
1.8.3.1