From: "Li, Liang Z" <liang.z.li@intel.com>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: "Hansen, Dave" <dave.hansen@intel.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"virtualization@lists.linux-foundation.org"
<virtualization@lists.linux-foundation.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"virtio-dev@lists.oasis-open.org"
<virtio-dev@lists.oasis-open.org>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
"dgilbert@redhat.com" <dgilbert@redhat.com>,
"quintela@redhat.com" <quintela@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
"Vlastimil Babka" <vbabka@suse.cz>,
Mel Gorman <mgorman@techsingularity.net>,
"Paolo Bonzini" <pbonzini@redhat.com>,
Cornelia Huck <cornelia.huck@de.ibm.com>,
Amit Shah <amit.shah@redhat.com>
Subject: RE: [virtio-dev] Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process
Date: Fri, 29 Jul 2016 00:46:14 +0000 [thread overview]
Message-ID: <F2CBF3009FA73547804AE4C663CAB28E04214C0B@shsmsx102.ccr.corp.intel.com> (raw)
In-Reply-To: <20160729003759-mutt-send-email-mst@kernel.org>
> On Thu, Jul 28, 2016 at 06:36:18AM +0000, Li, Liang Z wrote:
> > > > > This ends up doing a 1MB kmalloc() right? That seems a _bit_ big.
> > > > > How big was the pfn buffer before?
> > > >
> > > > Yes, it is if the max pfn is more than 32GB.
> > > > The size of the pfn buffer use before is 256*4 = 1024 Bytes, it's
> > > > too small, and it's the main reason for bad performance.
> > > > Use the max 1MB kmalloc is a balance between performance and
> > > > flexibility, a large page bitmap covers the range of all the
> > > > memory is no good for a system with huge amount of memory. If the
> > > > bitmap is too small, it means we have to traverse a long list for
> > > > many times, and it's bad
> > > for performance.
> > > >
> > > > Thanks!
> > > > Liang
> > >
> > > There are all your implementation decisions though.
> > >
> > > If guest memory is so fragmented that you only have order 0 4k
> > > pages, then allocating a huge 1M contigious chunk is very problematic in
> and of itself.
> > >
> >
> > The memory is allocated in the probe stage. This will not happen if
> > the driver is loaded when booting the guest.
> >
> > > Most people rarely migrate and do not care how fast that happens.
> > > Wasting a large chunk of memory (and it's zeroed for no good reason,
> > > so you actually request host memory for it) for everyone to speed it
> > > up when it does happen is not really an option.
> > >
> > If people don't plan to do inflating/deflating, they should not enable
> > the virtio-balloon at the beginning, once they decide to use it, the
> > driver should provide better performance as much as possible.
>
> The reason people inflate/deflate is so they can overcommit memory.
> Do they need to overcommit very quickly? I don't see why.
> So let's get what we can for free but I don't really believe people would want
> to pay for it.
>
> > 1MB is a very small portion for a VM with more than 32GB memory and
> > it's the *worst case*, for VM with less than 32GB memory, the amount
> > of RAM depends on VM's memory size and will be less than 1MB.
>
> It's guest memmory so might all be in swap and never touched, your memset
> at probe time will fault it in and make hypervisor actually pay for it.
>
> > If 1MB is too big, how about 512K, or 256K? 32K seems too small.
> >
> > Liang
>
> It's only small because it makes you rescan the free list.
> So maybe you should do something else.
> I looked at it a bit. Instead of scanning the free list, how about scanning actual
> page structures? If page is unused, pass it to host.
> Solves the problem of rescanning multiple times, does it not?
>
Yes, agree.
>
> Another idea: allocate a small bitmap at probe time (e.g. for deflate), allocate
> a bunch more on each request. Use something like GFP_ATOMIC and a
> scatter/gather, if that fails use the smaller bitmap.
>
So, the aim of v3 is to use a smaller bitmap without too heavy performance penalty.
Thanks a lot!
Liang
next prev parent reply other threads:[~2016-07-29 0:46 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-07-27 1:23 [PATCH v2 repost 0/7] Extend virtio-balloon for fast (de)inflating & fast live migration Liang Li
2016-07-27 1:23 ` [PATCH v2 repost 1/7] virtio-balloon: rework deflate to add page to a list Liang Li
2016-07-27 1:23 ` [PATCH v2 repost 2/7] virtio-balloon: define new feature bit and page bitmap head Liang Li
2016-07-27 1:23 ` [PATCH v2 repost 3/7] mm: add a function to get the max pfn Liang Li
2016-07-27 22:08 ` Michael S. Tsirkin
2016-07-27 22:52 ` Dave Hansen
2016-07-27 1:23 ` [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process Liang Li
2016-07-27 16:03 ` Dave Hansen
2016-07-27 21:39 ` Michael S. Tsirkin
2016-07-28 3:30 ` Li, Liang Z
2016-07-28 22:15 ` Michael S. Tsirkin
2016-07-29 1:08 ` [virtio-dev] " Li, Liang Z
2016-07-28 1:13 ` Li, Liang Z
2016-07-28 1:45 ` Michael S. Tsirkin
2016-07-28 6:36 ` [virtio-dev] " Li, Liang Z
2016-07-28 21:51 ` Michael S. Tsirkin
2016-07-29 0:46 ` Li, Liang Z [this message]
2016-07-29 19:48 ` Dave Hansen
2016-08-02 0:28 ` Li, Liang Z
2016-07-27 21:36 ` Michael S. Tsirkin
2016-07-28 3:06 ` Li, Liang Z
2016-07-28 22:17 ` Michael S. Tsirkin
2016-07-29 0:38 ` Li, Liang Z
2016-07-27 22:07 ` Michael S. Tsirkin
2016-07-28 3:48 ` Li, Liang Z
2016-07-27 1:23 ` [PATCH v2 repost 5/7] virtio-balloon: define feature bit and head for misc virt queue Liang Li
2016-07-27 1:23 ` [PATCH v2 repost 6/7] mm: add the related functions to get free page info Liang Li
2016-07-27 16:40 ` Dave Hansen
2016-07-27 22:05 ` Michael S. Tsirkin
2016-07-27 22:16 ` Dave Hansen
2016-07-27 23:05 ` Michael S. Tsirkin
2016-07-28 4:36 ` Li, Liang Z
2016-07-28 0:10 ` Li, Liang Z
2016-07-28 0:17 ` Michael S. Tsirkin
2016-07-27 22:13 ` Michael S. Tsirkin
2016-07-28 5:30 ` [virtio-dev] " Li, Liang Z
2016-07-27 1:23 ` [PATCH v2 repost 7/7] virtio-balloon: tell host vm's " Liang Li
2016-07-27 22:00 ` Michael S. Tsirkin
2016-07-28 7:50 ` Li, Liang Z
2016-07-28 21:37 ` Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=F2CBF3009FA73547804AE4C663CAB28E04214C0B@shsmsx102.ccr.corp.intel.com \
--to=liang.z.li@intel.com \
--cc=akpm@linux-foundation.org \
--cc=amit.shah@redhat.com \
--cc=cornelia.huck@de.ibm.com \
--cc=dave.hansen@intel.com \
--cc=dgilbert@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@techsingularity.net \
--cc=mst@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=vbabka@suse.cz \
--cc=virtio-dev@lists.oasis-open.org \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).