From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Xen-devel <xen-devel@lists.xen.org>
Cc: Wei Liu <wei.liu2@citrix.com>,
Yang Hongyang <yanghy@cn.fujitsu.com>,
Ian Jackson <Ian.Jackson@eu.citrix.com>,
Ian Campbell <Ian.Campbell@citrix.com>,
Andrew Cooper <andrew.cooper3@citrix.com>
Subject: [PATCH 00/27] Libxl migration v2
Date: Mon, 15 Jun 2015 14:44:13 +0100 [thread overview]
Message-ID: <1434375880-30914-1-git-send-email-andrew.cooper3@citrix.com> (raw)
This series adds support for the libxl migration v2 stream, and untangles the
existing layering violations of the toolstack and qemu records.
At the end of the series, legacy migration is no longer used.
Note: Remus support is broken and (RFC) fixed in separate patches in this
series. It was too tangled to fix in a bisectable fashon. Plain
suspend/migrate/resume however is (should be) bisectable along the entire
series.
There are a couple of outstanding questions:
1) What to do about the toolstack/xenstore record. It is currently by being
passed around as a blob, but it might be better to split it out.
2) What (if any) ABI/API qualifications are needed? (Particularly in reference
to patch 21)
The Remus code is untested by me, but is hopefully in the correct ballpark.
All other combinations of suspend/migrate/resume have been tested with PV and
HVM guests (qemu-trad and qemu-upstream), including 32 -> 64 bit migration
(which was the underlying bug causing us to write migration v2 in the first
place).
There are some further improvements which could be made. In particular, it
appears that sending the toolstack record on each checkpoint is redundant, and
there is certainly room for some more pruning of the legacy migration code.
Anyway, thoughts/comments welcome. Please test!
~Andrew
Andrew Cooper (22):
tools/libxl: Fix libxl__ev_child_inuse() check for not-yet-initialised children
tools/libxc: Always compile the compat qemu variables into xc_sr_context
tools/libxl: Stash all restore parameters in domain_create_state
tools/xl: Mandatory flag indicating the format of the migration stream
tools/libxl: Introduce ROUNDUP()
tools/libxl: Extra APIs for the save helper
tools/libxl: Pass restore_fd as a parameter to libxl__xc_domain_restore()
docs: Libxl migration v2 stream specification
tools/python: Libxc migration v2 infrastructure
tools/python: Libxl migration v2 infrastructure
tools/python: Verification utility for v2 stream spec compliance
tools/python: Conversion utility for legacy migration streams
tools/libxl: Support converting a legacy stream to a v2 stream
tools/libxl: Convert a legacy stream if needed
tools/libxc+libxl+xl: Restore v2 streams
tools/libxc+libxl+xl: Save v2 streams
docs/libxl: [RFC] Introduce CHECKPOINT_END to support migration v2 remus streams
tools/libxl: [RFC] Write checkpoint records into the stream
tools/libx{c,l}: [RFC] Introduce restore_callbacks.checkpoint()
tools/libxl: [RFC] Handle checkpoint records in a libxl migration v2 stream
tools/libxc: Drop all XG_LIBXL_HVM_COMPAT code from libxc
tools/libxl: Drop all knowledge of toolstack callbacks
Ian Jackson (2):
libxl: cancellation: Preparations for save/restore cancellation
libxl: cancellation: Handle SIGTERM in save/restore helper
Ross Lagerwall (3):
tools/libxl: Migration v2 stream format
tools/libxl: Infrastructure for reading a libxl migration v2 stream
tools/libxl: Infrastructure for writing a v2 stream
docs/specs/libxl-migration-stream.pandoc | 218 ++++++++
tools/libxc/Makefile | 2 -
tools/libxc/include/xenguest.h | 3 +
tools/libxc/xc_sr_common.h | 5 -
tools/libxc/xc_sr_restore.c | 33 +-
tools/libxc/xc_sr_restore_x86_hvm.c | 124 -----
tools/libxc/xc_sr_save_x86_hvm.c | 36 --
tools/libxl/Makefile | 2 +
tools/libxl/libxl_aoutils.c | 7 +
tools/libxl/libxl_convert_callout.c | 146 ++++++
tools/libxl/libxl_create.c | 80 +--
tools/libxl/libxl_dom.c | 61 +--
tools/libxl/libxl_internal.h | 140 ++++-
tools/libxl/libxl_save_callout.c | 63 +--
tools/libxl/libxl_save_helper.c | 95 ++--
tools/libxl/libxl_save_msgs_gen.pl | 9 +-
tools/libxl/libxl_sr_stream_format.h | 58 +++
tools/libxl/libxl_stream_read.c | 663 ++++++++++++++++++++++++
tools/libxl/libxl_stream_write.c | 640 +++++++++++++++++++++++
tools/libxl/libxl_types.idl | 2 +
tools/libxl/xl_cmdimpl.c | 9 +-
tools/python/Makefile | 4 +
tools/python/scripts/convert-legacy-stream.py | 683 +++++++++++++++++++++++++
tools/python/scripts/verify-stream-v2.py | 174 +++++++
tools/python/setup.py | 1 +
tools/python/xen/migration/libxc.py | 446 ++++++++++++++++
tools/python/xen/migration/libxl.py | 199 +++++++
tools/python/xen/migration/tests.py | 54 ++
tools/python/xen/migration/verify.py | 37 ++
29 files changed, 3638 insertions(+), 356 deletions(-)
create mode 100644 docs/specs/libxl-migration-stream.pandoc
create mode 100644 tools/libxl/libxl_convert_callout.c
create mode 100644 tools/libxl/libxl_sr_stream_format.h
create mode 100644 tools/libxl/libxl_stream_read.c
create mode 100644 tools/libxl/libxl_stream_write.c
create mode 100755 tools/python/scripts/convert-legacy-stream.py
create mode 100755 tools/python/scripts/verify-stream-v2.py
create mode 100644 tools/python/xen/migration/__init__.py
create mode 100644 tools/python/xen/migration/libxc.py
create mode 100644 tools/python/xen/migration/libxl.py
create mode 100644 tools/python/xen/migration/tests.py
create mode 100644 tools/python/xen/migration/verify.py
--
1.7.10.4
next reply other threads:[~2015-06-15 13:44 UTC|newest]
Thread overview: 107+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-15 13:44 Andrew Cooper [this message]
2015-06-15 13:44 ` [PATCH 01/27] tools/libxl: Fix libxl__ev_child_inuse() check for not-yet-initialised children Andrew Cooper
2015-06-16 13:21 ` Ian Campbell
2015-06-16 13:36 ` Andrew Cooper
2015-06-16 13:47 ` Ian Jackson
2015-06-16 14:05 ` Andrew Cooper
2015-06-16 15:26 ` Ian Campbell
2015-06-16 15:24 ` Ian Campbell
2015-06-16 13:39 ` Ian Jackson
2015-06-15 13:44 ` [PATCH 02/27] tools/libxc: Always compile the compat qemu variables into xc_sr_context Andrew Cooper
2015-06-16 13:22 ` Ian Campbell
2015-06-15 13:44 ` [PATCH 03/27] tools/libxl: Stash all restore parameters in domain_create_state Andrew Cooper
2015-06-16 13:37 ` Ian Campbell
2015-06-16 14:09 ` Andrew Cooper
2015-06-18 2:32 ` Yang Hongyang
2015-06-15 13:44 ` [PATCH 04/27] tools/xl: Mandatory flag indicating the format of the migration stream Andrew Cooper
2015-06-16 13:39 ` Ian Campbell
2015-06-16 14:10 ` Andrew Cooper
2015-06-15 13:44 ` [PATCH 05/27] tools/libxl: Introduce ROUNDUP() Andrew Cooper
2015-06-16 13:39 ` Ian Campbell
2015-06-15 13:44 ` [PATCH 06/27] libxl: cancellation: Preparations for save/restore cancellation Andrew Cooper
2015-06-15 13:44 ` [PATCH 07/27] libxl: cancellation: Handle SIGTERM in save/restore helper Andrew Cooper
2015-06-15 13:44 ` [PATCH 08/27] tools/libxl: Extra APIs for the save helper Andrew Cooper
2015-06-16 13:50 ` Ian Campbell
2015-06-16 15:03 ` Andrew Cooper
2015-06-15 13:44 ` [PATCH 09/27] tools/libxl: Pass restore_fd as a parameter to libxl__xc_domain_restore() Andrew Cooper
2015-06-16 13:53 ` Ian Campbell
2015-06-15 13:44 ` [PATCH 10/27] docs: Libxl migration v2 stream specification Andrew Cooper
2015-06-16 13:58 ` Ian Campbell
2015-07-08 13:49 ` Andrew Cooper
2015-07-08 13:58 ` Ian Campbell
2015-06-15 13:44 ` [PATCH 11/27] tools/python: Libxc migration v2 infrastructure Andrew Cooper
2015-06-16 14:01 ` Ian Campbell
2015-06-15 13:44 ` [PATCH 12/27] tools/python: Libxl " Andrew Cooper
2015-06-15 13:44 ` [PATCH 13/27] tools/python: Verification utility for v2 stream spec compliance Andrew Cooper
2015-06-15 13:44 ` [PATCH 14/27] tools/python: Conversion utility for legacy migration streams Andrew Cooper
2015-06-16 14:01 ` Ian Campbell
2015-06-15 13:44 ` [PATCH 15/27] tools/libxl: Migration v2 stream format Andrew Cooper
2015-06-16 14:04 ` Ian Campbell
2015-06-15 13:44 ` [PATCH 16/27] tools/libxl: Infrastructure for reading a libxl migration v2 stream Andrew Cooper
2015-06-16 14:31 ` Ian Campbell
2015-06-16 15:01 ` Andrew Cooper
2015-06-16 15:35 ` Ian Campbell
2015-06-16 15:46 ` Andrew Cooper
2015-06-17 3:09 ` Wen Congyang
2015-06-17 10:15 ` Ian Campbell
2015-06-17 10:49 ` Wen Congyang
2015-06-17 10:55 ` Ian Campbell
2015-06-17 6:03 ` Wen Congyang
2015-06-17 9:47 ` Andrew Cooper
2015-06-17 7:57 ` Wen Congyang
2015-06-17 9:50 ` Andrew Cooper
2015-06-17 10:01 ` Wen Congyang
2015-06-17 10:48 ` Andrew Cooper
2015-06-15 13:44 ` [PATCH 17/27] tools/libxl: Support converting a legacy stream to a " Andrew Cooper
2015-06-16 14:38 ` Ian Campbell
2015-06-16 15:13 ` Andrew Cooper
2015-06-16 15:38 ` Ian Campbell
2015-06-15 13:44 ` [PATCH 18/27] tools/libxl: Convert a legacy stream if needed Andrew Cooper
2015-06-15 13:44 ` [PATCH 19/27] tools/libxc+libxl+xl: Restore v2 streams Andrew Cooper
2015-06-16 14:53 ` Ian Campbell
2015-06-16 15:23 ` Andrew Cooper
2015-06-16 15:39 ` Ian Campbell
2015-06-15 13:44 ` [PATCH 20/27] tools/libxl: Infrastructure for writing a v2 stream Andrew Cooper
2015-06-16 14:57 ` Ian Campbell
2015-06-16 15:28 ` Andrew Cooper
2015-06-17 1:31 ` Yang Hongyang
2015-06-17 9:51 ` Andrew Cooper
2015-06-17 1:39 ` Wen Congyang
2015-06-17 2:24 ` Wen Congyang
2015-06-17 7:38 ` Yang Hongyang
2015-06-17 10:14 ` Wen Congyang
2015-07-10 10:55 ` Ian Campbell
2015-07-10 11:03 ` Andrew Cooper
2015-07-10 11:05 ` Ian Campbell
2015-06-15 13:44 ` [PATCH 21/27] tools/libxc+libxl+xl: Save v2 streams Andrew Cooper
2015-06-16 14:59 ` Ian Campbell
2015-06-15 13:44 ` [PATCH 22/27] docs/libxl: [RFC] Introduce CHECKPOINT_END to support migration v2 remus streams Andrew Cooper
2015-06-16 15:00 ` Ian Campbell
2015-06-16 15:30 ` Andrew Cooper
2015-06-17 3:30 ` Wen Congyang
2015-06-15 13:44 ` [PATCH 23/27] tools/libxl: [RFC] Write checkpoint records into the stream Andrew Cooper
2015-06-16 15:03 ` Ian Campbell
2015-06-16 15:53 ` Andrew Cooper
2015-06-17 7:30 ` Ian Campbell
2015-06-17 9:55 ` Andrew Cooper
2015-06-18 3:13 ` Wen Congyang
2015-06-18 9:44 ` Andrew Cooper
2015-06-15 13:44 ` [PATCH 24/27] tools/libx{c, l}: [RFC] Introduce restore_callbacks.checkpoint() Andrew Cooper
2015-06-16 2:23 ` Yang Hongyang
2015-06-17 8:20 ` Yang Hongyang
2015-06-15 13:44 ` [PATCH 25/27] tools/libxl: [RFC] Handle checkpoint records in a libxl migration v2 stream Andrew Cooper
2015-06-17 7:28 ` Wen Congyang
2015-06-15 13:44 ` [PATCH 26/27] tools/libxc: Drop all XG_LIBXL_HVM_COMPAT code from libxc Andrew Cooper
2015-06-16 15:03 ` Ian Campbell
2015-06-15 13:44 ` [PATCH 27/27] tools/libxl: Drop all knowledge of toolstack callbacks Andrew Cooper
2015-06-16 15:04 ` Ian Campbell
2015-06-16 15:06 ` Andrew Cooper
2015-06-17 10:14 ` Ian Campbell
2015-06-17 10:43 ` Andrew Cooper
2015-06-17 10:53 ` Ian Campbell
2015-06-16 2:21 ` [PATCH 00/27] Libxl migration v2 Yang Hongyang
2015-06-17 1:55 ` Wen Congyang
2015-06-17 9:45 ` Andrew Cooper
2015-07-02 7:33 ` Yang Hongyang
2015-07-02 9:26 ` Andrew Cooper
2015-07-02 9:33 ` Yang Hongyang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1434375880-30914-1-git-send-email-andrew.cooper3@citrix.com \
--to=andrew.cooper3@citrix.com \
--cc=Ian.Campbell@citrix.com \
--cc=Ian.Jackson@eu.citrix.com \
--cc=wei.liu2@citrix.com \
--cc=xen-devel@lists.xen.org \
--cc=yanghy@cn.fujitsu.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.