From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:46500) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UIjVh-0007RZ-R9 for qemu-devel@nongnu.org; Thu, 21 Mar 2013 13:42:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UIjVg-00027d-E7 for qemu-devel@nongnu.org; Thu, 21 Mar 2013 13:42:41 -0400 Received: from mx1.redhat.com ([209.132.183.28]:26601) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UIjVg-00027W-5e for qemu-devel@nongnu.org; Thu, 21 Mar 2013 13:42:40 -0400 From: Juan Quintela In-Reply-To: <1363881940-27505-1-git-send-email-owasserm@redhat.com> (Orit Wasserman's message of "Thu, 21 Mar 2013 18:05:31 +0200") References: <1363881940-27505-1-git-send-email-owasserm@redhat.com> Date: Thu, 21 Mar 2013 18:42:45 +0100 Message-ID: <87boacps2y.fsf@elfo.elfo> MIME-Version: 1.0 Content-Type: text/plain Subject: Re: [Qemu-devel] [PATCH v3 0/9] Migration: Remove copying of guest ram pages Reply-To: quintela@redhat.com List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Orit Wasserman Cc: pbonzini@redhat.com, chegu_vinod@hp.com, qemu-devel@nongnu.org, mst@redhat.com Orit Wasserman wrote: > In migration all data is copied to a static buffer in QEMUFile, > this hurts our network bandwidth and CPU usage especially with large guests. > We switched to iovec for storing different buffers to send (even a byte field is > considered as a buffer) and use writev to send the iovec. > writev was chosen (as apposed to sendmsg) because it supprts non socket fds. > > Guest memory pages are not copied by calling a new function > qemu_put_buffer_no_copy. > The page header data and device state data are still copied into the static > buffer. This data consists of a lot of bytes and integer fields and the static > buffer is used to store it during batching. > Another improvement is changing qemu_putbe64/32/16 to create a single > buffer instead of several byte sized buffer. > > git repository: git://github.com/oritwas/qemu.git sendv_v2 it is still sendv_v2 or sendv_v3 O:-) > Change from v2: > Always send data for the iovec even if writev_buffer is not implemented. > Coalesce adjacent iovecs to create one big buffer from small adjacent buffer. > > Changes from v1: > Use iov_send for socket. > Make writev_buffer optional and if it is not implemented use put_buffer > > Future work: Make number of iovec changeable > > Orit Wasserman (9): > Add QemuFileWritevBuffer QemuFileOps > Add socket_writev_buffer function > Update bytes_xfer in qemu_put_byte > Store the data to send also in iovec > Use writev ops if available > More optimized qemu_put_be64/32/16 > Add qemu_put_buffer_no_copy > Use qemu_put_buffer_no_copy for guest memory pages > coalesce adjacent iovecs > > arch_init.c | 2 +- > include/migration/qemu-file.h | 12 ++++ > savevm.c | 127 ++++++++++++++++++++++++++++++++++-------- > 3 files changed, 116 insertions(+), 25 deletions(-) Really nice series. Just a couple of small nits. Later, Juan.