From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marcelo Tosatti Subject: Re: [PATCH 1/2] KVM: MMU: Mark sp mmio cached when creating mmio spte Date: Wed, 13 Mar 2013 22:58:21 -0300 Message-ID: <20130314015821.GA13261@amt.cnet> References: <20130312174333.7f76148e.yoshikawa_takuya_b1@lab.ntt.co.jp> <20130312174440.5d5199ee.yoshikawa_takuya_b1@lab.ntt.co.jp> <5140094F.5080700@linux.vnet.ibm.com> <20130313162816.c62899dc.yoshikawa_takuya_b1@lab.ntt.co.jp> <51402DDA.607@linux.vnet.ibm.com> <20130313123358.GM11223@redhat.com> <51407441.4020200@linux.vnet.ibm.com> <20130313224056.8c9c87f4d95b332d2273a685@gmail.com> <514087A0.1000704@linux.vnet.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Takuya Yoshikawa , Gleb Natapov , Takuya Yoshikawa , kvm@vger.kernel.org To: Xiao Guangrong Return-path: Received: from mx1.redhat.com ([209.132.183.28]:39538 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934355Ab3CNB7k (ORCPT ); Wed, 13 Mar 2013 21:59:40 -0400 Content-Disposition: inline In-Reply-To: <514087A0.1000704@linux.vnet.ibm.com> Sender: kvm-owner@vger.kernel.org List-ID: On Wed, Mar 13, 2013 at 10:05:20PM +0800, Xiao Guangrong wrote: > On 03/13/2013 09:40 PM, Takuya Yoshikawa wrote: > > On Wed, 13 Mar 2013 20:42:41 +0800 > > Xiao Guangrong wrote: > > > >>>>>> How about save all mmio spte into a mmio-rmap? > >>>>> > >>>>> The problem is that other mmu code would need to care about the pointers > >>>>> stored in the new rmap list: when mmu_shrink zaps shadow pages for example. > >>>> > >>>> It is not hard... all the codes have been wrapped by *zap_spte*. > >>>> > >>> So are you going to send a patch? What do you think about applying this > >>> as temporary solution? > >> > >> Hi Gleb, > >> > >> Since it only needs small change based on this patch, I think we can directly > >> apply the rmap-based way. > >> > >> Takuya, could you please do this? ;) > > > > Though I'm fine with my making the patch better, I'm still thinking > > about the bad side of it, though. > > > > In zap_spte, don't we need to search the pointer to be removed from the > > global mmio-rmap list? How long can that list be? > > It is not bad. On softmmu, the rmap list has already been long more than 300. > On hardmmu, normally the mmio spte is not frequently zapped (just set not clear). > > The worst case is zap-all-mmio-spte that removes all mmio-spte. This operation > can be speed up after applying my previous patch: > KVM: MMU: fast drop all spte on the pte_list > > > > > Implementing it will/may not be difficult but I'm not sure if we would > > get pure improvement. Unless it becomes 99% sure, I think we should > > first take a basic approach. > > I definitely sure zapping all mmio-sptes is fast than zapping mmio shadow > pages. ;) With a huge number of shadow pages (think 512GB guest, 262144 pte-level shadow pages to map), it might be a problem. > > What do you think? > > I am considering if zap all shadow page is faster enough (after my patchset), do > we really need to care it? Still needed: your patch reduces kvm_mmu_zap_all() time, but as you can see with huge memory sized guests 100% improvement over the current situation will be a bottleneck (and as you noted the deletion case is still unsolved). Suppose another improvement angle is to zap only whats necessary for the given operation (say there is the memslot hint available, but unused for x86).