From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753707AbcEYI5c (ORCPT ); Wed, 25 May 2016 04:57:32 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37887 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751969AbcEYI52 (ORCPT ); Wed, 25 May 2016 04:57:28 -0400 Date: Wed, 25 May 2016 11:57:23 +0300 From: "Michael S. Tsirkin" To: "Li, Liang Z" Cc: "linux-kernel@vger.kernel.org" , "qemu-devel@nongnu.org" , "virtualization@lists.linux-foundation.org" , "akpm@linux-foundation.org" , "pbonzini@redhat.com" , "dgilbert@redhat.com" , "amit.shah@redhat.com" , "kvm@vger.kernel.org" Subject: Re: [PATCH RFC kernel] balloon: speed up inflating/deflating process Message-ID: <20160525115419-mutt-send-email-mst@redhat.com> References: <1463738386-30868-1-git-send-email-liang.z.li@intel.com> <20160520120038.GA28757@redhat.com> <20160524130041-mutt-send-email-mst@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 25 May 2016 08:57:28 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, May 25, 2016 at 08:48:17AM +0000, Li, Liang Z wrote: > > > > Suggestion to address all above comments: > > > > 1. allocate a bunch of pages and link them up, > > > > calculating the min and the max pfn. > > > > if max-min exceeds the allocated bitmap size, > > > > tell host. > > > > > > I am not sure if it works well in some cases, e.g. The allocated pages > > > are across a wide range and the max-min > limit is very frequently to be > > true. > > > Then, there will be many times of virtio transmission and it's bad for > > > performance improvement. Right? > > > > It's a tradeoff for sure. Measure it, see what the overhead is. > > > > Hi MST, > > I have measured the performance when using a 32K page bitmap, Just to make sure. Do you mean a 32Kbyte bitmap? Covering 1Gbyte of memory? > and inflate the balloon to 3GB > of an idle guest with 4GB RAM. Should take 3 requests then, right? > Now: > total inflating time: 338ms > the count of virtio data transmission: 373 Why was this so high? I would expect 3 transmissions. > the call count of madvise: 865 > > before: > total inflating time: 175ms > the count of virtio data transmission: 1 > the call count of madvise: 42 > > Maybe the result will be worse if the guest is not idle, or the guest has more RAM. > Do you want more data? > > Is it worth to do that? > > Liang Either my math is wrong or there's an implementation bug. > > > > > > > 2. limit allocated bitmap size to something reasonable. > > > > How about 32Kbytes? This is 256kilo bit in the map, which comes > > > > out to 1Giga bytes of memory in the balloon. > > > > > > So, even the VM has 1TB of RAM, the page bitmap will take 32MB of > > memory. > > > Maybe it's better to use a big page bitmap the save the pages > > > allocated by balloon, and split the big page bitmap to 32K bytes unit, then > > transfer one unit at a time. > > > > How is this different from what I said? > > > > > > > > Should we use a page bitmap to replace 'vb->pages' ? > > > > > > How about rolling back to use PFNs if the count of requested pages is a > > small number? > > > > > > Liang > > > > That's why we have start pfn. you can use that to pass even a single page > > without a lot of overhead. > > > > > > > -- > > > > > 1.9.1 > > > > -- > > > > To unsubscribe from this list: send the line "unsubscribe kvm" in > > > > the body of a message to majordomo@vger.kernel.org More majordomo > > > > info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [PATCH RFC kernel] balloon: speed up inflating/deflating process Date: Wed, 25 May 2016 11:57:23 +0300 Message-ID: <20160525115419-mutt-send-email-mst@redhat.com> References: <1463738386-30868-1-git-send-email-liang.z.li@intel.com> <20160520120038.GA28757@redhat.com> <20160524130041-mutt-send-email-mst@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: "kvm@vger.kernel.org" , "qemu-devel@nongnu.org" , "linux-kernel@vger.kernel.org" , "virtualization@lists.linux-foundation.org" , "amit.shah@redhat.com" , "pbonzini@redhat.com" , "akpm@linux-foundation.org" , "dgilbert@redhat.com" To: "Li, Liang Z" Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org List-Id: kvm.vger.kernel.org On Wed, May 25, 2016 at 08:48:17AM +0000, Li, Liang Z wrote: > > > > Suggestion to address all above comments: > > > > 1. allocate a bunch of pages and link them up, > > > > calculating the min and the max pfn. > > > > if max-min exceeds the allocated bitmap size, > > > > tell host. > > > > > > I am not sure if it works well in some cases, e.g. The allocated pages > > > are across a wide range and the max-min > limit is very frequently to be > > true. > > > Then, there will be many times of virtio transmission and it's bad for > > > performance improvement. Right? > > > > It's a tradeoff for sure. Measure it, see what the overhead is. > > > > Hi MST, > > I have measured the performance when using a 32K page bitmap, Just to make sure. Do you mean a 32Kbyte bitmap? Covering 1Gbyte of memory? > and inflate the balloon to 3GB > of an idle guest with 4GB RAM. Should take 3 requests then, right? > Now: > total inflating time: 338ms > the count of virtio data transmission: 373 Why was this so high? I would expect 3 transmissions. > the call count of madvise: 865 > > before: > total inflating time: 175ms > the count of virtio data transmission: 1 > the call count of madvise: 42 > > Maybe the result will be worse if the guest is not idle, or the guest has more RAM. > Do you want more data? > > Is it worth to do that? > > Liang Either my math is wrong or there's an implementation bug. > > > > > > > 2. limit allocated bitmap size to something reasonable. > > > > How about 32Kbytes? This is 256kilo bit in the map, which comes > > > > out to 1Giga bytes of memory in the balloon. > > > > > > So, even the VM has 1TB of RAM, the page bitmap will take 32MB of > > memory. > > > Maybe it's better to use a big page bitmap the save the pages > > > allocated by balloon, and split the big page bitmap to 32K bytes unit, then > > transfer one unit at a time. > > > > How is this different from what I said? > > > > > > > > Should we use a page bitmap to replace 'vb->pages' ? > > > > > > How about rolling back to use PFNs if the count of requested pages is a > > small number? > > > > > > Liang > > > > That's why we have start pfn. you can use that to pass even a single page > > without a lot of overhead. > > > > > > > -- > > > > > 1.9.1 > > > > -- > > > > To unsubscribe from this list: send the line "unsubscribe kvm" in > > > > the body of a message to majordomo@vger.kernel.org More majordomo > > > > info at http://vger.kernel.org/majordomo-info.html From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45459) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5UdB-00055L-Ph for qemu-devel@nongnu.org; Wed, 25 May 2016 04:57:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b5Ud7-0005W8-LL for qemu-devel@nongnu.org; Wed, 25 May 2016 04:57:32 -0400 Received: from mx1.redhat.com ([209.132.183.28]:35132) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5Ud7-0005Vt-DW for qemu-devel@nongnu.org; Wed, 25 May 2016 04:57:29 -0400 Date: Wed, 25 May 2016 11:57:23 +0300 From: "Michael S. Tsirkin" Message-ID: <20160525115419-mutt-send-email-mst@redhat.com> References: <1463738386-30868-1-git-send-email-liang.z.li@intel.com> <20160520120038.GA28757@redhat.com> <20160524130041-mutt-send-email-mst@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] [PATCH RFC kernel] balloon: speed up inflating/deflating process List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Li, Liang Z" Cc: "linux-kernel@vger.kernel.org" , "qemu-devel@nongnu.org" , "virtualization@lists.linux-foundation.org" , "akpm@linux-foundation.org" , "pbonzini@redhat.com" , "dgilbert@redhat.com" , "amit.shah@redhat.com" , "kvm@vger.kernel.org" On Wed, May 25, 2016 at 08:48:17AM +0000, Li, Liang Z wrote: > > > > Suggestion to address all above comments: > > > > 1. allocate a bunch of pages and link them up, > > > > calculating the min and the max pfn. > > > > if max-min exceeds the allocated bitmap size, > > > > tell host. > > > > > > I am not sure if it works well in some cases, e.g. The allocated pages > > > are across a wide range and the max-min > limit is very frequently to be > > true. > > > Then, there will be many times of virtio transmission and it's bad for > > > performance improvement. Right? > > > > It's a tradeoff for sure. Measure it, see what the overhead is. > > > > Hi MST, > > I have measured the performance when using a 32K page bitmap, Just to make sure. Do you mean a 32Kbyte bitmap? Covering 1Gbyte of memory? > and inflate the balloon to 3GB > of an idle guest with 4GB RAM. Should take 3 requests then, right? > Now: > total inflating time: 338ms > the count of virtio data transmission: 373 Why was this so high? I would expect 3 transmissions. > the call count of madvise: 865 > > before: > total inflating time: 175ms > the count of virtio data transmission: 1 > the call count of madvise: 42 > > Maybe the result will be worse if the guest is not idle, or the guest has more RAM. > Do you want more data? > > Is it worth to do that? > > Liang Either my math is wrong or there's an implementation bug. > > > > > > > 2. limit allocated bitmap size to something reasonable. > > > > How about 32Kbytes? This is 256kilo bit in the map, which comes > > > > out to 1Giga bytes of memory in the balloon. > > > > > > So, even the VM has 1TB of RAM, the page bitmap will take 32MB of > > memory. > > > Maybe it's better to use a big page bitmap the save the pages > > > allocated by balloon, and split the big page bitmap to 32K bytes unit, then > > transfer one unit at a time. > > > > How is this different from what I said? > > > > > > > > Should we use a page bitmap to replace 'vb->pages' ? > > > > > > How about rolling back to use PFNs if the count of requested pages is a > > small number? > > > > > > Liang > > > > That's why we have start pfn. you can use that to pass even a single page > > without a lot of overhead. > > > > > > > -- > > > > > 1.9.1 > > > > -- > > > > To unsubscribe from this list: send the line "unsubscribe kvm" in > > > > the body of a message to majordomo@vger.kernel.org More majordomo > > > > info at http://vger.kernel.org/majordomo-info.html