From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: [PATCH 00/27] Libxl migration v2 Date: Mon, 15 Jun 2015 14:44:13 +0100 Message-ID: <1434375880-30914-1-git-send-email-andrew.cooper3@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Xen-devel Cc: Wei Liu , Yang Hongyang , Ian Jackson , Ian Campbell , Andrew Cooper List-Id: xen-devel@lists.xenproject.org This series adds support for the libxl migration v2 stream, and untangles the existing layering violations of the toolstack and qemu records. At the end of the series, legacy migration is no longer used. Note: Remus support is broken and (RFC) fixed in separate patches in this series. It was too tangled to fix in a bisectable fashon. Plain suspend/migrate/resume however is (should be) bisectable along the entire series. There are a couple of outstanding questions: 1) What to do about the toolstack/xenstore record. It is currently by being passed around as a blob, but it might be better to split it out. 2) What (if any) ABI/API qualifications are needed? (Particularly in reference to patch 21) The Remus code is untested by me, but is hopefully in the correct ballpark. All other combinations of suspend/migrate/resume have been tested with PV and HVM guests (qemu-trad and qemu-upstream), including 32 -> 64 bit migration (which was the underlying bug causing us to write migration v2 in the first place). There are some further improvements which could be made. In particular, it appears that sending the toolstack record on each checkpoint is redundant, and there is certainly room for some more pruning of the legacy migration code. Anyway, thoughts/comments welcome. Please test! ~Andrew Andrew Cooper (22): tools/libxl: Fix libxl__ev_child_inuse() check for not-yet-initialised children tools/libxc: Always compile the compat qemu variables into xc_sr_context tools/libxl: Stash all restore parameters in domain_create_state tools/xl: Mandatory flag indicating the format of the migration stream tools/libxl: Introduce ROUNDUP() tools/libxl: Extra APIs for the save helper tools/libxl: Pass restore_fd as a parameter to libxl__xc_domain_restore() docs: Libxl migration v2 stream specification tools/python: Libxc migration v2 infrastructure tools/python: Libxl migration v2 infrastructure tools/python: Verification utility for v2 stream spec compliance tools/python: Conversion utility for legacy migration streams tools/libxl: Support converting a legacy stream to a v2 stream tools/libxl: Convert a legacy stream if needed tools/libxc+libxl+xl: Restore v2 streams tools/libxc+libxl+xl: Save v2 streams docs/libxl: [RFC] Introduce CHECKPOINT_END to support migration v2 remus streams tools/libxl: [RFC] Write checkpoint records into the stream tools/libx{c,l}: [RFC] Introduce restore_callbacks.checkpoint() tools/libxl: [RFC] Handle checkpoint records in a libxl migration v2 stream tools/libxc: Drop all XG_LIBXL_HVM_COMPAT code from libxc tools/libxl: Drop all knowledge of toolstack callbacks Ian Jackson (2): libxl: cancellation: Preparations for save/restore cancellation libxl: cancellation: Handle SIGTERM in save/restore helper Ross Lagerwall (3): tools/libxl: Migration v2 stream format tools/libxl: Infrastructure for reading a libxl migration v2 stream tools/libxl: Infrastructure for writing a v2 stream docs/specs/libxl-migration-stream.pandoc | 218 ++++++++ tools/libxc/Makefile | 2 - tools/libxc/include/xenguest.h | 3 + tools/libxc/xc_sr_common.h | 5 - tools/libxc/xc_sr_restore.c | 33 +- tools/libxc/xc_sr_restore_x86_hvm.c | 124 ----- tools/libxc/xc_sr_save_x86_hvm.c | 36 -- tools/libxl/Makefile | 2 + tools/libxl/libxl_aoutils.c | 7 + tools/libxl/libxl_convert_callout.c | 146 ++++++ tools/libxl/libxl_create.c | 80 +-- tools/libxl/libxl_dom.c | 61 +-- tools/libxl/libxl_internal.h | 140 ++++- tools/libxl/libxl_save_callout.c | 63 +-- tools/libxl/libxl_save_helper.c | 95 ++-- tools/libxl/libxl_save_msgs_gen.pl | 9 +- tools/libxl/libxl_sr_stream_format.h | 58 +++ tools/libxl/libxl_stream_read.c | 663 ++++++++++++++++++++++++ tools/libxl/libxl_stream_write.c | 640 +++++++++++++++++++++++ tools/libxl/libxl_types.idl | 2 + tools/libxl/xl_cmdimpl.c | 9 +- tools/python/Makefile | 4 + tools/python/scripts/convert-legacy-stream.py | 683 +++++++++++++++++++++++++ tools/python/scripts/verify-stream-v2.py | 174 +++++++ tools/python/setup.py | 1 + tools/python/xen/migration/libxc.py | 446 ++++++++++++++++ tools/python/xen/migration/libxl.py | 199 +++++++ tools/python/xen/migration/tests.py | 54 ++ tools/python/xen/migration/verify.py | 37 ++ 29 files changed, 3638 insertions(+), 356 deletions(-) create mode 100644 docs/specs/libxl-migration-stream.pandoc create mode 100644 tools/libxl/libxl_convert_callout.c create mode 100644 tools/libxl/libxl_sr_stream_format.h create mode 100644 tools/libxl/libxl_stream_read.c create mode 100644 tools/libxl/libxl_stream_write.c create mode 100755 tools/python/scripts/convert-legacy-stream.py create mode 100755 tools/python/scripts/verify-stream-v2.py create mode 100644 tools/python/xen/migration/__init__.py create mode 100644 tools/python/xen/migration/libxc.py create mode 100644 tools/python/xen/migration/libxl.py create mode 100644 tools/python/xen/migration/tests.py create mode 100644 tools/python/xen/migration/verify.py -- 1.7.10.4