From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932621AbcLGPp1 (ORCPT ); Wed, 7 Dec 2016 10:45:27 -0500 Received: from mga04.intel.com ([192.55.52.120]:14061 "EHLO mga04.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932179AbcLGPpZ (ORCPT ); Wed, 7 Dec 2016 10:45:25 -0500 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,310,1477983600"; d="scan'208";a="1078757927" Subject: Re: [PATCH kernel v5 0/5] Extend virtio-balloon for fast (de)inflating & fast live migration To: David Hildenbrand , "Li, Liang Z" , "kvm@vger.kernel.org" References: <1480495397-23225-1-git-send-email-liang.z.li@intel.com> <0b18c636-ee67-cbb4-1ba3-81a06150db76@redhat.com> Cc: "virtio-dev@lists.oasis-open.org" , "mhocko@suse.com" , "mst@redhat.com" , "qemu-devel@nongnu.org" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , "kirill.shutemov@linux.intel.com" , "pbonzini@redhat.com" , "akpm@linux-foundation.org" , "virtualization@lists.linux-foundation.org" , "dgilbert@redhat.com" , Andrea Arcangeli From: Dave Hansen Message-ID: <0b83db29-ebad-2a70-8d61-756d33e33a48@intel.com> Date: Wed, 7 Dec 2016 07:45:21 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <0b18c636-ee67-cbb4-1ba3-81a06150db76@redhat.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 12/07/2016 07:42 AM, David Hildenbrand wrote: > Am 07.12.2016 um 14:35 schrieb Li, Liang Z: >>> Am 30.11.2016 um 09:43 schrieb Liang Li: >>>> This patch set contains two parts of changes to the virtio-balloon. >>>> >>>> One is the change for speeding up the inflating & deflating process, >>>> the main idea of this optimization is to use bitmap to send the page >>>> information to host instead of the PFNs, to reduce the overhead of >>>> virtio data transmission, address translation and madvise(). This can >>>> help to improve the performance by about 85%. >>> >>> Do you have some statistics/some rough feeling how many consecutive >>> bits are >>> usually set in the bitmaps? Is it really just purely random or is >>> there some >>> granularity that is usually consecutive? >>> >> >> I did something similar. Filled the balloon with 15GB for a 16GB idle >> guest, by >> using bitmap, the madvise count was reduced to 605. when using the >> PFNs, the madvise count >> was 3932160. It means there are quite a lot consecutive bits in the >> bitmap. >> I didn't test for a guest with heavy memory workload. > > Would it then even make sense to go one step further and report {pfn, > length} combinations? > > So simply send over an array of {pfn, length}? Li's current patches do that. Well, maybe not pfn/length, but they do take a pfn and page-order, which fits perfectly with the kernel's concept of high-order pages. > And it makes sense if you think about: > > a) hugetlb backing: The host may only be able to free huge pages (we > might want to communicate that to the guest later, that's another > story). Still we would have to send bitmaps full of 4k frames (512 bits > for 2mb frames). Of course, we could add a way to communicate that we > are using a different bitmap-granularity. Yeah, please read the patches. If they're not clear, then the descriptions need work, but this is done already.