From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756870AbcCCMpb (ORCPT ); Thu, 3 Mar 2016 07:45:31 -0500 Received: from mx1.redhat.com ([209.132.183.28]:53394 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751420AbcCCMp0 (ORCPT ); Thu, 3 Mar 2016 07:45:26 -0500 Date: Thu, 3 Mar 2016 12:45:20 +0000 From: "Daniel P. Berrange" To: Liang Li Cc: quintela@redhat.com, amit.shah@redhat.com, qemu-devel@nongnu.org, linux-kernel@vger.kernel.org, ehabkost@redhat.com, kvm@vger.kernel.org, mst@redhat.com, dgilbert@redhat.com, virtualization@lists.linux-foundation.org, linux-mm@kvack.org, pbonzini@redhat.com, akpm@linux-foundation.org, rth@twiddle.net Subject: Re: [Qemu-devel] [RFC qemu 4/4] migration: filter out guest's free pages in ram bulk stage Message-ID: <20160303124520.GE32270@redhat.com> Reply-To: "Daniel P. Berrange" References: <1457001868-15949-1-git-send-email-liang.z.li@intel.com> <1457001868-15949-5-git-send-email-liang.z.li@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <1457001868-15949-5-git-send-email-liang.z.li@intel.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 03 Mar 2016 12:45:26 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Mar 03, 2016 at 06:44:28PM +0800, Liang Li wrote: > Get the free pages information through virtio and filter out the free > pages in the ram bulk stage. This can significantly reduce the total > live migration time as well as network traffic. > > Signed-off-by: Liang Li > --- > migration/ram.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++------ > 1 file changed, 46 insertions(+), 6 deletions(-) > @@ -1945,6 +1971,20 @@ static int ram_save_setup(QEMUFile *f, void *opaque) > DIRTY_MEMORY_MIGRATION); > } > memory_global_dirty_log_start(); > + > + if (balloon_free_pages_support() && > + balloon_get_free_pages(migration_bitmap_rcu->free_pages_bmap, > + &free_pages_count) == 0) { > + qemu_mutex_unlock_iothread(); > + while (balloon_get_free_pages(migration_bitmap_rcu->free_pages_bmap, > + &free_pages_count) == 0) { > + usleep(1000); > + } > + qemu_mutex_lock_iothread(); > + > + filter_out_guest_free_pages(migration_bitmap_rcu->free_pages_bmap); > + } IIUC, this code is synchronous wrt to the guest OS balloon drive. ie it is asking the geust for free pages and waiting for a response. If the guest OS has crashed this is going to mean QEMU waits forever and thus migration won't complete. Similarly you need to consider that the guest OS may be malicious and simply never respond. So if the migration code is going to use the guest balloon driver to get info about free pages it has to be done in an asynchronous manner so that migration can never be stalled by a slow/crashed/malicious guest driver. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Daniel P. Berrange" Subject: Re: [Qemu-devel] [RFC qemu 4/4] migration: filter out guest's free pages in ram bulk stage Date: Thu, 3 Mar 2016 12:45:20 +0000 Message-ID: <20160303124520.GE32270@redhat.com> References: <1457001868-15949-1-git-send-email-liang.z.li@intel.com> <1457001868-15949-5-git-send-email-liang.z.li@intel.com> Reply-To: "Daniel P. Berrange" Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Cc: quintela@redhat.com, amit.shah@redhat.com, qemu-devel@nongnu.org, linux-kernel@vger.kernel.org, ehabkost@redhat.com, kvm@vger.kernel.org, mst@redhat.com, dgilbert@redhat.com, virtualization@lists.linux-foundation.org, linux-mm@kvack.org, pbonzini@redhat.com, akpm@linux-foundation.org, rth@twiddle.net To: Liang Li Return-path: Content-Disposition: inline In-Reply-To: <1457001868-15949-5-git-send-email-liang.z.li@intel.com> Sender: owner-linux-mm@kvack.org List-Id: kvm.vger.kernel.org On Thu, Mar 03, 2016 at 06:44:28PM +0800, Liang Li wrote: > Get the free pages information through virtio and filter out the free > pages in the ram bulk stage. This can significantly reduce the total > live migration time as well as network traffic. > > Signed-off-by: Liang Li > --- > migration/ram.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++------ > 1 file changed, 46 insertions(+), 6 deletions(-) > @@ -1945,6 +1971,20 @@ static int ram_save_setup(QEMUFile *f, void *opaque) > DIRTY_MEMORY_MIGRATION); > } > memory_global_dirty_log_start(); > + > + if (balloon_free_pages_support() && > + balloon_get_free_pages(migration_bitmap_rcu->free_pages_bmap, > + &free_pages_count) == 0) { > + qemu_mutex_unlock_iothread(); > + while (balloon_get_free_pages(migration_bitmap_rcu->free_pages_bmap, > + &free_pages_count) == 0) { > + usleep(1000); > + } > + qemu_mutex_lock_iothread(); > + > + filter_out_guest_free_pages(migration_bitmap_rcu->free_pages_bmap); > + } IIUC, this code is synchronous wrt to the guest OS balloon drive. ie it is asking the geust for free pages and waiting for a response. If the guest OS has crashed this is going to mean QEMU waits forever and thus migration won't complete. Similarly you need to consider that the guest OS may be malicious and simply never respond. So if the migration code is going to use the guest balloon driver to get info about free pages it has to be done in an asynchronous manner so that migration can never be stalled by a slow/crashed/malicious guest driver. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42006) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1abSdG-0005Cl-Cv for qemu-devel@nongnu.org; Thu, 03 Mar 2016 07:45:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1abSdC-0006oc-NC for qemu-devel@nongnu.org; Thu, 03 Mar 2016 07:45:30 -0500 Received: from mx1.redhat.com ([209.132.183.28]:42369) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1abSdC-0006oY-HS for qemu-devel@nongnu.org; Thu, 03 Mar 2016 07:45:26 -0500 Date: Thu, 3 Mar 2016 12:45:20 +0000 From: "Daniel P. Berrange" Message-ID: <20160303124520.GE32270@redhat.com> References: <1457001868-15949-1-git-send-email-liang.z.li@intel.com> <1457001868-15949-5-git-send-email-liang.z.li@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <1457001868-15949-5-git-send-email-liang.z.li@intel.com> Subject: Re: [Qemu-devel] [RFC qemu 4/4] migration: filter out guest's free pages in ram bulk stage Reply-To: "Daniel P. Berrange" List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Liang Li Cc: ehabkost@redhat.com, kvm@vger.kernel.org, quintela@redhat.com, mst@redhat.com, linux-kernel@vger.kernel.org, qemu-devel@nongnu.org, linux-mm@kvack.org, amit.shah@redhat.com, pbonzini@redhat.com, akpm@linux-foundation.org, virtualization@lists.linux-foundation.org, dgilbert@redhat.com, rth@twiddle.net On Thu, Mar 03, 2016 at 06:44:28PM +0800, Liang Li wrote: > Get the free pages information through virtio and filter out the free > pages in the ram bulk stage. This can significantly reduce the total > live migration time as well as network traffic. > > Signed-off-by: Liang Li > --- > migration/ram.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++------ > 1 file changed, 46 insertions(+), 6 deletions(-) > @@ -1945,6 +1971,20 @@ static int ram_save_setup(QEMUFile *f, void *opaque) > DIRTY_MEMORY_MIGRATION); > } > memory_global_dirty_log_start(); > + > + if (balloon_free_pages_support() && > + balloon_get_free_pages(migration_bitmap_rcu->free_pages_bmap, > + &free_pages_count) == 0) { > + qemu_mutex_unlock_iothread(); > + while (balloon_get_free_pages(migration_bitmap_rcu->free_pages_bmap, > + &free_pages_count) == 0) { > + usleep(1000); > + } > + qemu_mutex_lock_iothread(); > + > + filter_out_guest_free_pages(migration_bitmap_rcu->free_pages_bmap); > + } IIUC, this code is synchronous wrt to the guest OS balloon drive. ie it is asking the geust for free pages and waiting for a response. If the guest OS has crashed this is going to mean QEMU waits forever and thus migration won't complete. Similarly you need to consider that the guest OS may be malicious and simply never respond. So if the migration code is going to use the guest balloon driver to get info about free pages it has to be done in an asynchronous manner so that migration can never be stalled by a slow/crashed/malicious guest driver. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|