From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757182AbcG2BIQ (ORCPT ); Thu, 28 Jul 2016 21:08:16 -0400 Received: from mga04.intel.com ([192.55.52.120]:31676 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752545AbcG2BIN convert rfc822-to-8bit (ORCPT ); Thu, 28 Jul 2016 21:08:13 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.28,436,1464678000"; d="scan'208";a="1015898409" From: "Li, Liang Z" To: "Michael S. Tsirkin" CC: "Hansen, Dave" , "linux-kernel@vger.kernel.org" , "virtualization@lists.linux-foundation.org" , "linux-mm@kvack.org" , "virtio-dev@lists.oasis-open.org" , "kvm@vger.kernel.org" , "qemu-devel@nongnu.org" , "dgilbert@redhat.com" , "quintela@redhat.com" , Andrew Morton , "Vlastimil Babka" , Mel Gorman , "Paolo Bonzini" , Cornelia Huck , Amit Shah Subject: RE: [virtio-dev] Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process Thread-Topic: [virtio-dev] Re: [PATCH v2 repost 4/7] virtio-balloon: speed up inflate/deflate process Thread-Index: AQHR56aT5WMR4KLffEuOEzyB2S9p86Ar62GAgABd2ACAAON34IAAuQSAgACyC9A= Date: Fri, 29 Jul 2016 01:08:09 +0000 Message-ID: References: <1469582616-5729-1-git-send-email-liang.z.li@intel.com> <1469582616-5729-5-git-send-email-liang.z.li@intel.com> <5798DB49.7030803@intel.com> <20160728003644-mutt-send-email-mst@kernel.org> <20160728221533.GA789@redhat.com> In-Reply-To: <20160728221533.GA789@redhat.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiODg5M2YwYTItMGY4Yi00NTA1LWE0Y2YtNjY3YjAyOTI3MzY3IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE1LjkuNi42IiwiVHJ1c3RlZExhYmVsSGFzaCI6IlNicjl2b2JxbE5ZZE5YeE0yb3N1SXIxSXNQVUVudzFkdEJ5aGFwRUQrQTQ9In0= x-ctpclassification: CTP_IC x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8BIT MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > > > On Wed, Jul 27, 2016 at 09:03:21AM -0700, Dave Hansen wrote: > > > > On 07/26/2016 06:23 PM, Liang Li wrote: > > > > > + vb->pfn_limit = VIRTIO_BALLOON_PFNS_LIMIT; > > > > > + vb->pfn_limit = min(vb->pfn_limit, get_max_pfn()); > > > > > + vb->bmap_len = ALIGN(vb->pfn_limit, BITS_PER_LONG) / > > > > > + BITS_PER_BYTE + 2 * sizeof(unsigned long); > > > > > + hdr_len = sizeof(struct balloon_bmap_hdr); > > > > > + vb->bmap_hdr = kzalloc(hdr_len + vb->bmap_len, > GFP_KERNEL); > > > > > > > > This ends up doing a 1MB kmalloc() right? That seems a _bit_ big. > > > > How big was the pfn buffer before? > > > > > > > > > Yes I would limit this to 1G memory in a go, will result in a 32KByte bitmap. > > > > > > -- > > > MST > > > > Limit to 1G is bad for the performance, I sent you the test result several > weeks ago. > > > > Paste it bellow: > > ---------------------------------------------------------------------- > > -------------------------------------------------- > > About the size of page bitmap, I have test the performance of filling > > the balloon to 15GB with a 16GB RAM VM. > > > > =============================== > > 32K Byte (cover 1GB of RAM) > > > > Time spends on inflating: 2031ms > > --------------------------------------------- > > 64K Byte (cover 2GB of RAM) > > > > Time spends on inflating: 1507ms > > -------------------------------------------- > > 512K Byte (cover 16GB of RAM) > > > > Time spends on inflating: 1237ms > > ================================ > > > > If possible, a big bitmap is better for performance. > > > > Liang > > Earlier you said: > a. allocating pages (6.5%) > b. sending PFNs to host (68.3%) > c. address translation (6.1%) > d. madvise (19%) > > Here sending PFNs to host with 512K Byte map should be almost free. > > So is something else taking up the time? > I just want to show you the benefits of using a big bitmap. :) I did not measure the time spend on each stage after optimization(I will do it later), but I have tried to allocate the page with big chunk and found it can make things faster. Without allocating big chunk page, the performance improvement is about 85%, and with allocating big chunk page, the improvement is about 94%. Liang > > -- > MST