From mboxrd@z Thu Jan 1 00:00:00 1970 From: Yoshiaki Tamura Subject: [PATCH 00/19] Kemari for KVM v0.2.7 Date: Wed, 26 Jan 2011 18:41:52 +0900 Message-ID: <1296034931-10843-1-git-send-email-tamura.yoshiaki@lab.ntt.co.jp> Cc: avi@redhat.com, anthony@codemonkey.ws, aliguori@us.ibm.com, mtosatti@redhat.com, dlaor@redhat.com, mst@redhat.com, kwolf@redhat.com, ananth@in.ibm.com, psuriset@linux.vnet.ibm.com, vatsa@linux.vnet.ibm.com, stefanha@linux.vnet.ibm.com, blauwirbel@gmail.com, ohmura.kei@lab.ntt.co.jp, Yoshiaki Tamura To: kvm@vger.kernel.org, qemu-devel@nongnu.org Return-path: Received: from sh.osrg.net ([192.16.179.4]:49680 "EHLO sh.osrg.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751906Ab1AZJnx (ORCPT ); Wed, 26 Jan 2011 04:43:53 -0500 Sender: kvm-owner@vger.kernel.org List-ID: Hi, This patch series is a revised version of Kemari for KVM, which applied comments for the previous post. The current code is based on qemu.git 0bfe006c5380c5f8a485a55ded3329fbbc224396. The changes from v0.2.6 -> v0.2.7 are: - add AIOCB, AIOPool and cancel functions (Kevin) - insert event-tap for bdrv_flush (Kevin) - add error handing when calling bdrv functions (Kevin) - fix usage of qemu_aio_flush and bdrv_flush (Kevin) - use bs in AIOCB on the primary (Kevin) - reorder event-tap functions to gather with block/net (Kevin) - fix checking bs->device_name (Kevin) The changes from v0.2.5 -> v0.2.6 are: - use qemu_{put,get}_be32() to save/load niov in event-tap The changes from v0.2.4 -> v0.2.5 are: - fixed braces and trailing spaces by using Blue's checkpatch.pl (Blue) - event-tap: don't try to send blk_req if it's a bdrv_aio_flush event The changes from v0.2.3 -> v0.2.4 are: - call vm_start() before event_tap_flush_one() to avoid failure in virtio-net assertion - add vm_change_state_handler to turn off ft_mode - use qemu_iovec functions in event-tap - remove duplicated code in migration - remove unnecessary new line for error_report in ft_trans_file The changes from v0.2.2 -> v0.2.3 are: - queue async net requests without copying (MST) -- if not async, contents of the packets are sent to the secondary - better description for option -k (MST) - fix memory transfer failure - fix ft transaction initiation failure The changes from v0.2.1 -> v0.2.2 are: - decrement last_avaid_idx with inuse before saving (MST) - remove qemu_aio_flush() and bdrv_flush_all() in migrate_ft_trans_commit() The changes from v0.2 -> v0.2.1 are: - Move event-tap to net/block layer and use stubs (Blue, Paul, MST, Kevin) - Tap bdrv_aio_flush (Marcelo) - Remove multiwrite interface in event-tap (Stefan) - Fix event-tap to use pio/mmio to replay both net/block (Stefan) - Improve error handling in event-tap (Stefan) - Fix leak in event-tap (Stefan) - Revise virtio last_avail_idx manipulation (MST) - Clean up migration.c hook (Marcelo) - Make deleting change state handler robust (Isaku, Anthony) The changes from v0.1.1 -> v0.2 are: - Introduce a queue in event-tap to make VM sync live. - Change transaction receiver to a state machine for async receiving. - Replace net/block layer functions with event-tap proxy functions. - Remove dirty bitmap optimization for now. - convert DPRINTF() in ft_trans_file to trace functions. - convert fprintf() in ft_trans_file to error_report(). - improved error handling in ft_trans_file. - add a tmp pointer to qemu_del_vm_change_state_handler. The changes from v0.1 -> v0.1.1 are: - events are tapped in net/block layer instead of device emulation layer. - Introduce a new option for -incoming to accept FT transaction. - Removed writev() support to QEMUFile and FdMigrationState for now. I would post this work in a different series. - Modified virtio-blk save/load handler to send inuse variable to correctly replay. - Removed configure --enable-ft-mode. - Removed unnecessary check for qemu_realloc(). The first 6 patches modify several functions of qemu to prepare introducing Kemari specific components. The next 6 patches are the components of Kemari. They introduce event-tap and the FT transaction protocol file based on buffered file. The design document of FT transaction protocol can be found at, http://wiki.qemu.org/images/b/b1/Kemari_sender_receiver_0.5a.pdf Then the following 2 patches modifies net/block layer functions with event-tap functions. Please note that if Kemari is off, event-tap will just passthrough, and there is most no intrusion to exisiting functions including normal live migration. Finally, the migration layer are modified to support Kemari in the last 5 patches. Again, there shouldn't be any affection if a user doesn't specify Kemari specific options. The transaction is now async on both sender and receiver side. The sender side respects the max_downtime to decide when to switch from async to sync mode. The repository contains all patches I'm sending with this message. For those who want to try, please pull the following repository. It also includes dirty bitmap optimization which aren't ready for posting yet. To remove the dirty bitmap optimization, please look at HEAD~5 of the tree. git://kemari.git.sourceforge.net/gitroot/kemari/kemari next Thanks, Yoshi Yoshiaki Tamura (19): Make QEMUFile buf expandable, and introduce qemu_realloc_buffer() and qemu_clear_buffer(). Introduce read() to FdMigrationState. Introduce skip_header parameter to qemu_loadvm_state(). qemu-char: export socket_set_nodelay(). vl.c: add deleted flag for deleting the handler. virtio: decrement last_avail_idx with inuse before saving. Introduce fault tolerant VM transaction QEMUFile and ft_mode. savevm: introduce util functions to control ft_trans_file from savevm layer. Introduce event-tap. Call init handler of event-tap at main() in vl.c. ioport: insert event_tap_ioport() to ioport_write(). Insert event_tap_mmio() to cpu_physical_memory_rw() in exec.c. net: insert event-tap to qemu_send_packet() and qemu_sendv_packet_async(). block: insert event-tap to bdrv_aio_writev() and bdrv_aio_flush(). savevm: introduce qemu_savevm_trans_{begin,commit}. migration: introduce migrate_ft_trans_{put,get}_ready(), and modify migrate_fd_put_ready() when ft_mode is on. migration-tcp: modify tcp_accept_incoming_migration() to handle ft_mode, and add a hack not to close fd when ft_mode is enabled. Introduce -k option to enable FT migration mode (Kemari). migration: add a parser to accept FT migration incoming mode. Makefile.objs | 1 + Makefile.target | 1 + block.c | 15 + event-tap.c | 925 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ event-tap.h | 44 +++ exec.c | 4 + ft_trans_file.c | 624 +++++++++++++++++++++++++++++++++++++ ft_trans_file.h | 72 +++++ hmp-commands.hx | 7 +- hw/hw.h | 7 + hw/virtio.c | 10 +- ioport.c | 2 + migration-tcp.c | 82 +++++- migration.c | 295 ++++++++++++++++++- migration.h | 3 + net.c | 9 + qemu-char.c | 2 +- qemu-tool.c | 28 ++ qemu_socket.h | 1 + qmp-commands.hx | 7 +- savevm.c | 286 ++++++++++++++++- sysemu.h | 4 +- trace-events | 25 ++ vl.c | 16 +- 24 files changed, 2442 insertions(+), 28 deletions(-) create mode 100644 event-tap.c create mode 100644 event-tap.h create mode 100644 ft_trans_file.c create mode 100644 ft_trans_file.h From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=51166 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Pi1vG-0004HU-Tb for qemu-devel@nongnu.org; Wed, 26 Jan 2011 04:44:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Pi1uc-00067d-1S for qemu-devel@nongnu.org; Wed, 26 Jan 2011 04:43:44 -0500 Received: from sh.osrg.net ([192.16.179.4]:45157) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Pi1ub-00066Q-H7 for qemu-devel@nongnu.org; Wed, 26 Jan 2011 04:43:37 -0500 From: Yoshiaki Tamura Date: Wed, 26 Jan 2011 18:41:52 +0900 Message-Id: <1296034931-10843-1-git-send-email-tamura.yoshiaki@lab.ntt.co.jp> Subject: [Qemu-devel] [PATCH 00/19] Kemari for KVM v0.2.7 List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: kvm@vger.kernel.org, qemu-devel@nongnu.org Cc: kwolf@redhat.com, aliguori@us.ibm.com, mtosatti@redhat.com, ananth@in.ibm.com, mst@redhat.com, dlaor@redhat.com, vatsa@linux.vnet.ibm.com, Yoshiaki Tamura , blauwirbel@gmail.com, ohmura.kei@lab.ntt.co.jp, avi@redhat.com, psuriset@linux.vnet.ibm.com, stefanha@linux.vnet.ibm.com Hi, This patch series is a revised version of Kemari for KVM, which applied comments for the previous post. The current code is based on qemu.git 0bfe006c5380c5f8a485a55ded3329fbbc224396. The changes from v0.2.6 -> v0.2.7 are: - add AIOCB, AIOPool and cancel functions (Kevin) - insert event-tap for bdrv_flush (Kevin) - add error handing when calling bdrv functions (Kevin) - fix usage of qemu_aio_flush and bdrv_flush (Kevin) - use bs in AIOCB on the primary (Kevin) - reorder event-tap functions to gather with block/net (Kevin) - fix checking bs->device_name (Kevin) The changes from v0.2.5 -> v0.2.6 are: - use qemu_{put,get}_be32() to save/load niov in event-tap The changes from v0.2.4 -> v0.2.5 are: - fixed braces and trailing spaces by using Blue's checkpatch.pl (Blue) - event-tap: don't try to send blk_req if it's a bdrv_aio_flush event The changes from v0.2.3 -> v0.2.4 are: - call vm_start() before event_tap_flush_one() to avoid failure in virtio-net assertion - add vm_change_state_handler to turn off ft_mode - use qemu_iovec functions in event-tap - remove duplicated code in migration - remove unnecessary new line for error_report in ft_trans_file The changes from v0.2.2 -> v0.2.3 are: - queue async net requests without copying (MST) -- if not async, contents of the packets are sent to the secondary - better description for option -k (MST) - fix memory transfer failure - fix ft transaction initiation failure The changes from v0.2.1 -> v0.2.2 are: - decrement last_avaid_idx with inuse before saving (MST) - remove qemu_aio_flush() and bdrv_flush_all() in migrate_ft_trans_commit() The changes from v0.2 -> v0.2.1 are: - Move event-tap to net/block layer and use stubs (Blue, Paul, MST, Kevin) - Tap bdrv_aio_flush (Marcelo) - Remove multiwrite interface in event-tap (Stefan) - Fix event-tap to use pio/mmio to replay both net/block (Stefan) - Improve error handling in event-tap (Stefan) - Fix leak in event-tap (Stefan) - Revise virtio last_avail_idx manipulation (MST) - Clean up migration.c hook (Marcelo) - Make deleting change state handler robust (Isaku, Anthony) The changes from v0.1.1 -> v0.2 are: - Introduce a queue in event-tap to make VM sync live. - Change transaction receiver to a state machine for async receiving. - Replace net/block layer functions with event-tap proxy functions. - Remove dirty bitmap optimization for now. - convert DPRINTF() in ft_trans_file to trace functions. - convert fprintf() in ft_trans_file to error_report(). - improved error handling in ft_trans_file. - add a tmp pointer to qemu_del_vm_change_state_handler. The changes from v0.1 -> v0.1.1 are: - events are tapped in net/block layer instead of device emulation layer. - Introduce a new option for -incoming to accept FT transaction. - Removed writev() support to QEMUFile and FdMigrationState for now. I would post this work in a different series. - Modified virtio-blk save/load handler to send inuse variable to correctly replay. - Removed configure --enable-ft-mode. - Removed unnecessary check for qemu_realloc(). The first 6 patches modify several functions of qemu to prepare introducing Kemari specific components. The next 6 patches are the components of Kemari. They introduce event-tap and the FT transaction protocol file based on buffered file. The design document of FT transaction protocol can be found at, http://wiki.qemu.org/images/b/b1/Kemari_sender_receiver_0.5a.pdf Then the following 2 patches modifies net/block layer functions with event-tap functions. Please note that if Kemari is off, event-tap will just passthrough, and there is most no intrusion to exisiting functions including normal live migration. Finally, the migration layer are modified to support Kemari in the last 5 patches. Again, there shouldn't be any affection if a user doesn't specify Kemari specific options. The transaction is now async on both sender and receiver side. The sender side respects the max_downtime to decide when to switch from async to sync mode. The repository contains all patches I'm sending with this message. For those who want to try, please pull the following repository. It also includes dirty bitmap optimization which aren't ready for posting yet. To remove the dirty bitmap optimization, please look at HEAD~5 of the tree. git://kemari.git.sourceforge.net/gitroot/kemari/kemari next Thanks, Yoshi Yoshiaki Tamura (19): Make QEMUFile buf expandable, and introduce qemu_realloc_buffer() and qemu_clear_buffer(). Introduce read() to FdMigrationState. Introduce skip_header parameter to qemu_loadvm_state(). qemu-char: export socket_set_nodelay(). vl.c: add deleted flag for deleting the handler. virtio: decrement last_avail_idx with inuse before saving. Introduce fault tolerant VM transaction QEMUFile and ft_mode. savevm: introduce util functions to control ft_trans_file from savevm layer. Introduce event-tap. Call init handler of event-tap at main() in vl.c. ioport: insert event_tap_ioport() to ioport_write(). Insert event_tap_mmio() to cpu_physical_memory_rw() in exec.c. net: insert event-tap to qemu_send_packet() and qemu_sendv_packet_async(). block: insert event-tap to bdrv_aio_writev() and bdrv_aio_flush(). savevm: introduce qemu_savevm_trans_{begin,commit}. migration: introduce migrate_ft_trans_{put,get}_ready(), and modify migrate_fd_put_ready() when ft_mode is on. migration-tcp: modify tcp_accept_incoming_migration() to handle ft_mode, and add a hack not to close fd when ft_mode is enabled. Introduce -k option to enable FT migration mode (Kemari). migration: add a parser to accept FT migration incoming mode. Makefile.objs | 1 + Makefile.target | 1 + block.c | 15 + event-tap.c | 925 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ event-tap.h | 44 +++ exec.c | 4 + ft_trans_file.c | 624 +++++++++++++++++++++++++++++++++++++ ft_trans_file.h | 72 +++++ hmp-commands.hx | 7 +- hw/hw.h | 7 + hw/virtio.c | 10 +- ioport.c | 2 + migration-tcp.c | 82 +++++- migration.c | 295 ++++++++++++++++++- migration.h | 3 + net.c | 9 + qemu-char.c | 2 +- qemu-tool.c | 28 ++ qemu_socket.h | 1 + qmp-commands.hx | 7 +- savevm.c | 286 ++++++++++++++++- sysemu.h | 4 +- trace-events | 25 ++ vl.c | 16 +- 24 files changed, 2442 insertions(+), 28 deletions(-) create mode 100644 event-tap.c create mode 100644 event-tap.h create mode 100644 ft_trans_file.c create mode 100644 ft_trans_file.h