All of lore.kernel.org
 help / color / mirror / Atom feed
* [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams
@ 2019-12-24 15:19 Andrew Cooper
  2019-12-24 15:19 ` [Xen-devel] [PATCH 01/12] libxc/save: Shrink code volume where possible Andrew Cooper
                   ` (13 more replies)
  0 siblings, 14 replies; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Marek Marczykowski-Górecki,
	Jan Beulich, Ian Jackson

As an early christmas present, here is the pumbing work to include CPUID/MSR
data in migration streams.

For bisectability reasons, this is a large amount of plumbing with no overall
change in behaviour.  Switching to use the CPUID/MSR data in the stream is
going to require some careful coordinated changes with libxl, which I haven't
quite managed to get working yet.

However, this subset of the work is now in a position to review.


The first 6 patches are various bits of cleanup.  Patches 7-9 introduce
Migration v3 and the STATIC_DATA_END record.  Patches 10-12 introduce the
X86_{CPUID,MSR}_POLICY records.

~Andrew

Andrew Cooper (12):
  libxc/save: Shrink code volume where possible
  libxc/restore: Introduce functionality to simplify blob handling
  libxc/migration: Rationalise the 'checkpointed' field to 'stream_type'
  libxc/migration: Adjust layout of struct xc_sr_context
  tools/migration: Drop IHDR_VERSION constant from libxc and python
  docs/migration Specify migration v3 and STATIC_DATA_END
  python/migration: Update validation logic to understand a v3 stream
  libxc/restore: Support v3 streams, and cope with v2 compatibilty
  libxc/save: Write a v3 stream
  docs/migration: Specify X86_{CPUID,MSR}_POLICY records
  libxc/restore: Handle X86_{CPUID,MSR}_DATA records
  libxc/save: Write X86_{CPUID,MSR}_DATA records

 docs/specs/libxc-migration-stream.pandoc   |  81 +++++++-
 tools/libxc/include/xenguest.h             |  15 +-
 tools/libxc/xc_nomigrate.c                 |   4 +-
 tools/libxc/xc_sr_common.c                 |   3 +
 tools/libxc/xc_sr_common.h                 | 205 ++++++++++++-------
 tools/libxc/xc_sr_common_x86.c             |  97 +++++++++
 tools/libxc/xc_sr_common_x86.h             |  25 +++
 tools/libxc/xc_sr_common_x86_pv.c          |  50 ++---
 tools/libxc/xc_sr_common_x86_pv.h          |   4 +-
 tools/libxc/xc_sr_restore.c                |  82 ++++++--
 tools/libxc/xc_sr_restore_x86_hvm.c        |  35 ++--
 tools/libxc/xc_sr_restore_x86_pv.c         | 316 +++++++++++++++--------------
 tools/libxc/xc_sr_save.c                   |  65 +++---
 tools/libxc/xc_sr_save_x86_hvm.c           |  31 +--
 tools/libxc/xc_sr_save_x86_pv.c            | 225 +++++++++-----------
 tools/libxc/xc_sr_stream_format.h          |   4 +-
 tools/libxl/libxl_save_helper.c            |   4 +-
 tools/python/scripts/convert-legacy-stream |  13 +-
 tools/python/scripts/verify-stream-v2      |   2 +-
 tools/python/xen/migration/libxc.py        |  74 ++++++-
 20 files changed, 854 insertions(+), 481 deletions(-)

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 01/12] libxc/save: Shrink code volume where possible
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
@ 2019-12-24 15:19 ` Andrew Cooper
  2020-01-14 16:48   ` Ian Jackson
  2019-12-24 15:19 ` [Xen-devel] [PATCH 02/12] libxc/restore: Introduce functionality to simplify blob handling Andrew Cooper
                   ` (12 subsequent siblings)
  13 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel; +Cc: Andrew Cooper, Wei Liu, Ian Jackson

A property of how the error handling (0 on success, nonzero otherwise)
allows these calls to be chained together with the ternary operatior.

No functional change, but far less boilerplate code.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 tools/libxc/xc_sr_save.c         |  7 ++---
 tools/libxc/xc_sr_save_x86_hvm.c | 21 +++------------
 tools/libxc/xc_sr_save_x86_pv.c  | 58 ++++++++--------------------------------
 3 files changed, 16 insertions(+), 70 deletions(-)

diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
index f89e12c99f..9764aa743f 100644
--- a/tools/libxc/xc_sr_save.c
+++ b/tools/libxc/xc_sr_save.c
@@ -845,11 +845,8 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type)
 
     xc_report_progress_single(xch, "Start of stream");
 
-    rc = write_headers(ctx, guest_type);
-    if ( rc )
-        goto err;
-
-    rc = ctx->save.ops.start_of_stream(ctx);
+    rc = (write_headers(ctx, guest_type) ?:
+          ctx->save.ops.start_of_stream(ctx));
     if ( rc )
         goto err;
 
diff --git a/tools/libxc/xc_sr_save_x86_hvm.c b/tools/libxc/xc_sr_save_x86_hvm.c
index 3d86cb0600..d925a81999 100644
--- a/tools/libxc/xc_sr_save_x86_hvm.c
+++ b/tools/libxc/xc_sr_save_x86_hvm.c
@@ -187,24 +187,9 @@ static int x86_hvm_check_vm_state(struct xc_sr_context *ctx)
 
 static int x86_hvm_end_of_checkpoint(struct xc_sr_context *ctx)
 {
-    int rc;
-
-    /* Write the TSC record. */
-    rc = write_x86_tsc_info(ctx);
-    if ( rc )
-        return rc;
-
-    /* Write the HVM_CONTEXT record. */
-    rc = write_hvm_context(ctx);
-    if ( rc )
-        return rc;
-
-    /* Write HVM_PARAMS record contains applicable HVM params. */
-    rc = write_hvm_params(ctx);
-    if ( rc )
-        return rc;
-
-    return 0;
+    return (write_x86_tsc_info(ctx) ?:
+            write_hvm_context(ctx) ?:
+            write_hvm_params(ctx));
 }
 
 static int x86_hvm_cleanup(struct xc_sr_context *ctx)
diff --git a/tools/libxc/xc_sr_save_x86_pv.c b/tools/libxc/xc_sr_save_x86_pv.c
index 3ebc5a2bf8..94d0f68911 100644
--- a/tools/libxc/xc_sr_save_x86_pv.c
+++ b/tools/libxc/xc_sr_save_x86_pv.c
@@ -768,19 +768,10 @@ static int write_all_vcpu_information(struct xc_sr_context *ctx)
         if ( !vinfo.online )
             continue;
 
-        rc = write_one_vcpu_basic(ctx, i);
-        if ( rc )
-            return rc;
-
-        rc = write_one_vcpu_extended(ctx, i);
-        if ( rc )
-            return rc;
-
-        rc = write_one_vcpu_xsave(ctx, i);
-        if ( rc )
-            return rc;
-
-        rc = write_one_vcpu_msrs(ctx, i);
+        rc = (write_one_vcpu_basic(ctx, i) ?:
+              write_one_vcpu_extended(ctx, i) ?:
+              write_one_vcpu_xsave(ctx, i) ?:
+              write_one_vcpu_msrs(ctx, i));
         if ( rc )
             return rc;
     }
@@ -1031,25 +1022,10 @@ static int x86_pv_normalise_page(struct xc_sr_context *ctx, xen_pfn_t type,
  */
 static int x86_pv_setup(struct xc_sr_context *ctx)
 {
-    int rc;
-
-    rc = x86_pv_domain_info(ctx);
-    if ( rc )
-        return rc;
-
-    rc = x86_pv_map_m2p(ctx);
-    if ( rc )
-        return rc;
-
-    rc = map_shinfo(ctx);
-    if ( rc )
-        return rc;
-
-    rc = map_p2m(ctx);
-    if ( rc )
-        return rc;
-
-    return 0;
+    return (x86_pv_domain_info(ctx) ?:
+            x86_pv_map_m2p(ctx) ?:
+            map_shinfo(ctx) ?:
+            map_p2m(ctx));
 }
 
 static int x86_pv_start_of_stream(struct xc_sr_context *ctx)
@@ -1080,21 +1056,9 @@ static int x86_pv_start_of_checkpoint(struct xc_sr_context *ctx)
 
 static int x86_pv_end_of_checkpoint(struct xc_sr_context *ctx)
 {
-    int rc;
-
-    rc = write_x86_tsc_info(ctx);
-    if ( rc )
-        return rc;
-
-    rc = write_shared_info(ctx);
-    if ( rc )
-        return rc;
-
-    rc = write_all_vcpu_information(ctx);
-    if ( rc )
-        return rc;
-
-    return 0;
+    return (write_x86_tsc_info(ctx) ?:
+            write_shared_info(ctx) ?:
+            write_all_vcpu_information(ctx));
 }
 
 static int x86_pv_check_vm_state(struct xc_sr_context *ctx)
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 02/12] libxc/restore: Introduce functionality to simplify blob handling
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
  2019-12-24 15:19 ` [Xen-devel] [PATCH 01/12] libxc/save: Shrink code volume where possible Andrew Cooper
@ 2019-12-24 15:19 ` Andrew Cooper
  2020-01-14 16:50   ` Ian Jackson
  2019-12-24 15:19 ` [Xen-devel] [PATCH 03/12] libxc/migration: Rationalise the 'checkpointed' field to 'stream_type' Andrew Cooper
                   ` (11 subsequent siblings)
  13 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel; +Cc: Andrew Cooper, Wei Liu, Ian Jackson

During migration, we buffer several blobs of data which ultimately need
handing back to Xen at an appropriate time.

Currently, this is all handled in an ad-hoc manner, but more blobs are soon
going to be added.  Introduce xc_sr_blob to encapsulate a ptr/size pair, and
update_blob() to handle the memory management aspects.

Switch the HVM_CONTEXT and the four PV_VCPU_* blobs over to this new
infrastructure.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 tools/libxc/xc_sr_common.h          | 45 ++++++++++++++++++++-----
 tools/libxc/xc_sr_restore_x86_hvm.c | 21 ++++--------
 tools/libxc/xc_sr_restore_x86_pv.c  | 67 ++++++++++++++-----------------------
 3 files changed, 68 insertions(+), 65 deletions(-)

diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h
index b66d785e50..19b053911f 100644
--- a/tools/libxc/xc_sr_common.h
+++ b/tools/libxc/xc_sr_common.h
@@ -165,13 +165,40 @@ struct xc_sr_restore_ops
     int (*cleanup)(struct xc_sr_context *ctx);
 };
 
-/* x86 PV per-vcpu storage structure for blobs heading Xen-wards. */
-struct xc_sr_x86_pv_restore_vcpu
+/* Wrapper for blobs of data heading Xen-wards. */
+struct xc_sr_blob
 {
-    void *basic, *extd, *xsave, *msr;
-    size_t basicsz, extdsz, xsavesz, msrsz;
+    void *ptr;
+    size_t size;
 };
 
+/*
+ * Update a blob.  Duplicate src/size, freeing the old blob if necessary.  May
+ * fail due to memory allocation.
+ */
+static inline int update_blob(struct xc_sr_blob *blob,
+                              const void *src, size_t size)
+{
+    void *ptr;
+
+    if ( !src || !size )
+    {
+        errno = EINVAL;
+        return -1;
+    }
+
+    if ( (ptr = malloc(size)) == NULL )
+        return -1;
+
+    memcpy(ptr, src, size);
+
+    free(blob->ptr);
+    blob->ptr = memcpy(ptr, src, size);
+    blob->size = size;
+
+    return 0;
+}
+
 struct xc_sr_context
 {
     xc_interface *xch;
@@ -306,8 +333,11 @@ struct xc_sr_context
                     /* Types for each page (bounded by max_pfn). */
                     uint32_t *pfn_types;
 
-                    /* Vcpu context blobs. */
-                    struct xc_sr_x86_pv_restore_vcpu *vcpus;
+                    /* x86 PV per-vcpu storage structure for blobs. */
+                    struct xc_sr_x86_pv_restore_vcpu
+                    {
+                        struct xc_sr_blob basic, extd, xsave, msr;
+                    } *vcpus;
                     unsigned int nr_vcpus;
                 } restore;
             };
@@ -327,8 +357,7 @@ struct xc_sr_context
                 struct
                 {
                     /* HVM context blob. */
-                    void *context;
-                    size_t contextsz;
+                    struct xc_sr_blob context;
                 } restore;
             };
         } x86_hvm;
diff --git a/tools/libxc/xc_sr_restore_x86_hvm.c b/tools/libxc/xc_sr_restore_x86_hvm.c
index 4a24dc0137..fe7be9bde6 100644
--- a/tools/libxc/xc_sr_restore_x86_hvm.c
+++ b/tools/libxc/xc_sr_restore_x86_hvm.c
@@ -10,21 +10,12 @@ static int handle_hvm_context(struct xc_sr_context *ctx,
                               struct xc_sr_record *rec)
 {
     xc_interface *xch = ctx->xch;
-    void *p;
+    int rc = update_blob(&ctx->x86_hvm.restore.context, rec->data, rec->length);
 
-    p = malloc(rec->length);
-    if ( !p )
-    {
+    if ( rc )
         ERROR("Unable to allocate %u bytes for hvm context", rec->length);
-        return -1;
-    }
 
-    free(ctx->x86_hvm.restore.context);
-
-    ctx->x86_hvm.restore.context = memcpy(p, rec->data, rec->length);
-    ctx->x86_hvm.restore.contextsz = rec->length;
-
-    return 0;
+    return rc;
 }
 
 /*
@@ -210,8 +201,8 @@ static int x86_hvm_stream_complete(struct xc_sr_context *ctx)
     }
 
     rc = xc_domain_hvm_setcontext(xch, ctx->domid,
-                                  ctx->x86_hvm.restore.context,
-                                  ctx->x86_hvm.restore.contextsz);
+                                  ctx->x86_hvm.restore.context.ptr,
+                                  ctx->x86_hvm.restore.context.size);
     if ( rc < 0 )
     {
         PERROR("Unable to restore HVM context");
@@ -234,7 +225,7 @@ static int x86_hvm_stream_complete(struct xc_sr_context *ctx)
 
 static int x86_hvm_cleanup(struct xc_sr_context *ctx)
 {
-    free(ctx->x86_hvm.restore.context);
+    free(ctx->x86_hvm.restore.context.ptr);
 
     return 0;
 }
diff --git a/tools/libxc/xc_sr_restore_x86_pv.c b/tools/libxc/xc_sr_restore_x86_pv.c
index c0598af8b7..0ec506632a 100644
--- a/tools/libxc/xc_sr_restore_x86_pv.c
+++ b/tools/libxc/xc_sr_restore_x86_pv.c
@@ -236,7 +236,7 @@ static int process_vcpu_basic(struct xc_sr_context *ctx,
                               unsigned int vcpuid)
 {
     xc_interface *xch = ctx->xch;
-    vcpu_guest_context_any_t *vcpu = ctx->x86_pv.restore.vcpus[vcpuid].basic;
+    vcpu_guest_context_any_t *vcpu = ctx->x86_pv.restore.vcpus[vcpuid].basic.ptr;
     xen_pfn_t pfn, mfn;
     unsigned int i, gdt_count;
     int rc = -1;
@@ -380,7 +380,7 @@ static int process_vcpu_extended(struct xc_sr_context *ctx,
 
     domctl.cmd = XEN_DOMCTL_set_ext_vcpucontext;
     domctl.domain = ctx->domid;
-    memcpy(&domctl.u.ext_vcpucontext, vcpu->extd, vcpu->extdsz);
+    memcpy(&domctl.u.ext_vcpucontext, vcpu->extd.ptr, vcpu->extd.size);
 
     if ( xc_domctl(xch, &domctl) != 0 )
     {
@@ -404,21 +404,21 @@ static int process_vcpu_xsave(struct xc_sr_context *ctx,
     DECLARE_DOMCTL;
     DECLARE_HYPERCALL_BUFFER(void, buffer);
 
-    buffer = xc_hypercall_buffer_alloc(xch, buffer, vcpu->xsavesz);
+    buffer = xc_hypercall_buffer_alloc(xch, buffer, vcpu->xsave.size);
     if ( !buffer )
     {
         ERROR("Unable to allocate %zu bytes for xsave hypercall buffer",
-              vcpu->xsavesz);
+              vcpu->xsave.size);
         return -1;
     }
 
     domctl.cmd = XEN_DOMCTL_setvcpuextstate;
     domctl.domain = ctx->domid;
     domctl.u.vcpuextstate.vcpu = vcpuid;
-    domctl.u.vcpuextstate.size = vcpu->xsavesz;
+    domctl.u.vcpuextstate.size = vcpu->xsave.size;
     set_xen_guest_handle(domctl.u.vcpuextstate.buffer, buffer);
 
-    memcpy(buffer, vcpu->xsave, vcpu->xsavesz);
+    memcpy(buffer, vcpu->xsave.ptr, vcpu->xsave.size);
 
     rc = xc_domctl(xch, &domctl);
     if ( rc )
@@ -442,21 +442,21 @@ static int process_vcpu_msrs(struct xc_sr_context *ctx,
     DECLARE_DOMCTL;
     DECLARE_HYPERCALL_BUFFER(void, buffer);
 
-    buffer = xc_hypercall_buffer_alloc(xch, buffer, vcpu->msrsz);
+    buffer = xc_hypercall_buffer_alloc(xch, buffer, vcpu->msr.size);
     if ( !buffer )
     {
         ERROR("Unable to allocate %zu bytes for msr hypercall buffer",
-              vcpu->msrsz);
+              vcpu->msr.size);
         return -1;
     }
 
     domctl.cmd = XEN_DOMCTL_set_vcpu_msrs;
     domctl.domain = ctx->domid;
     domctl.u.vcpu_msrs.vcpu = vcpuid;
-    domctl.u.vcpu_msrs.msr_count = vcpu->msrsz / sizeof(xen_domctl_vcpu_msr_t);
+    domctl.u.vcpu_msrs.msr_count = vcpu->msr.size / sizeof(xen_domctl_vcpu_msr_t);
     set_xen_guest_handle(domctl.u.vcpu_msrs.msrs, buffer);
 
-    memcpy(buffer, vcpu->msr, vcpu->msrsz);
+    memcpy(buffer, vcpu->msr.ptr, vcpu->msr.size);
 
     rc = xc_domctl(xch, &domctl);
     if ( rc )
@@ -481,7 +481,7 @@ static int update_vcpu_context(struct xc_sr_context *ctx)
     {
         vcpu = &ctx->x86_pv.restore.vcpus[i];
 
-        if ( vcpu->basic )
+        if ( vcpu->basic.ptr )
         {
             rc = process_vcpu_basic(ctx, i);
             if ( rc )
@@ -493,21 +493,21 @@ static int update_vcpu_context(struct xc_sr_context *ctx)
             return -1;
         }
 
-        if ( vcpu->extd )
+        if ( vcpu->extd.ptr )
         {
             rc = process_vcpu_extended(ctx, i);
             if ( rc )
                 return rc;
         }
 
-        if ( vcpu->xsave )
+        if ( vcpu->xsave.ptr )
         {
             rc = process_vcpu_xsave(ctx, i);
             if ( rc )
                 return rc;
         }
 
-        if ( vcpu->msr )
+        if ( vcpu->msr.ptr )
         {
             rc = process_vcpu_msrs(ctx, i);
             if ( rc )
@@ -738,7 +738,7 @@ static int handle_x86_pv_vcpu_blob(struct xc_sr_context *ctx,
     struct xc_sr_x86_pv_restore_vcpu *vcpu;
     const char *rec_name;
     size_t blobsz;
-    void *blob;
+    struct xc_sr_blob *blob;
     int rc = -1;
 
     switch ( rec->type )
@@ -812,6 +812,7 @@ static int handle_x86_pv_vcpu_blob(struct xc_sr_context *ctx,
                   rec_name, sizeof(*vhdr) + vcpusz, rec->length);
             goto out;
         }
+        blob = &vcpu->basic;
         break;
     }
 
@@ -822,6 +823,7 @@ static int handle_x86_pv_vcpu_blob(struct xc_sr_context *ctx,
                   rec_name, sizeof(*vhdr) + 128, rec->length);
             goto out;
         }
+        blob = &vcpu->extd;
         break;
 
     case REC_TYPE_X86_PV_VCPU_XSAVE:
@@ -831,6 +833,7 @@ static int handle_x86_pv_vcpu_blob(struct xc_sr_context *ctx,
                   rec_name, sizeof(*vhdr) + 128, rec->length);
             goto out;
         }
+        blob = &vcpu->xsave;
         break;
 
     case REC_TYPE_X86_PV_VCPU_MSRS:
@@ -840,34 +843,14 @@ static int handle_x86_pv_vcpu_blob(struct xc_sr_context *ctx,
                   rec_name, blobsz, sizeof(xen_domctl_vcpu_msr_t));
             goto out;
         }
+        blob = &vcpu->msr;
         break;
     }
 
-    /* Allocate memory. */
-    blob = malloc(blobsz);
-    if ( !blob )
-    {
+    rc = update_blob(blob, vhdr->context, blobsz);
+    if ( rc )
         ERROR("Unable to allocate %zu bytes for vcpu%u %s blob",
               blobsz, vhdr->vcpu_id, rec_name);
-        goto out;
-    }
-
-    memcpy(blob, &vhdr->context, blobsz);
-
-    /* Stash sideways for later. */
-    switch ( rec->type )
-    {
-#define RECSTORE(x, y) case REC_TYPE_X86_PV_ ## x: \
-        free(y); (y) = blob; (y ## sz) = blobsz; break
-
-        RECSTORE(VCPU_BASIC,    vcpu->basic);
-        RECSTORE(VCPU_EXTENDED, vcpu->extd);
-        RECSTORE(VCPU_XSAVE,    vcpu->xsave);
-        RECSTORE(VCPU_MSRS,     vcpu->msr);
-#undef RECSTORE
-    }
-
-    rc = 0;
 
  out:
     return rc;
@@ -1159,10 +1142,10 @@ static int x86_pv_cleanup(struct xc_sr_context *ctx)
             struct xc_sr_x86_pv_restore_vcpu *vcpu =
                 &ctx->x86_pv.restore.vcpus[i];
 
-            free(vcpu->basic);
-            free(vcpu->extd);
-            free(vcpu->xsave);
-            free(vcpu->msr);
+            free(vcpu->basic.ptr);
+            free(vcpu->extd.ptr);
+            free(vcpu->xsave.ptr);
+            free(vcpu->msr.ptr);
         }
 
         free(ctx->x86_pv.restore.vcpus);
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 03/12] libxc/migration: Rationalise the 'checkpointed' field to 'stream_type'
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
  2019-12-24 15:19 ` [Xen-devel] [PATCH 01/12] libxc/save: Shrink code volume where possible Andrew Cooper
  2019-12-24 15:19 ` [Xen-devel] [PATCH 02/12] libxc/restore: Introduce functionality to simplify blob handling Andrew Cooper
@ 2019-12-24 15:19 ` Andrew Cooper
  2020-01-14 15:58   ` Ian Jackson
  2019-12-24 15:19 ` [Xen-devel] [PATCH 04/12] libxc/migration: Adjust layout of struct xc_sr_context Andrew Cooper
                   ` (10 subsequent siblings)
  13 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel; +Cc: Andrew Cooper, Wei Liu, Ian Jackson

Originally, 'checkpointed' was a boolean signalling the difference between a
plain and a Remus stream.  COLO was added later, but several bits of code
retained boolean-style logic.  While correct, it is confusing to follow.

Additionally, XC_MIG_STREAM_NONE means "no checkpoints" but reads as "no
stream".

Consolidate all the logic on the term 'stream_type', and rename STREAM_NONE
to STREAM_PLAIN.  Re-position the stream_type variable so it isn't
duplicated in both the save and restore unions.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 tools/libxc/include/xenguest.h  | 15 +++++++-------
 tools/libxc/xc_nomigrate.c      |  4 ++--
 tools/libxc/xc_sr_common.h      |  9 +++------
 tools/libxc/xc_sr_restore.c     | 33 +++++++++++++++++++------------
 tools/libxc/xc_sr_save.c        | 44 ++++++++++++++++++++++++-----------------
 tools/libxl/libxl_save_helper.c |  4 ++--
 6 files changed, 61 insertions(+), 48 deletions(-)

diff --git a/tools/libxc/include/xenguest.h b/tools/libxc/include/xenguest.h
index b4b2e19619..9ba09af743 100644
--- a/tools/libxc/include/xenguest.h
+++ b/tools/libxc/include/xenguest.h
@@ -115,11 +115,12 @@ struct save_callbacks {
     void* data;
 };
 
+/* Type of stream.  Plain, or using a continuous replication protocol? */
 typedef enum {
-    XC_MIG_STREAM_NONE, /* plain stream */
-    XC_MIG_STREAM_REMUS,
-    XC_MIG_STREAM_COLO,
-} xc_migration_stream_t;
+    XC_STREAM_PLAIN,
+    XC_STREAM_REMUS,
+    XC_STREAM_COLO,
+} xc_stream_type_t;
 
 /**
  * This function will save a running domain.
@@ -127,14 +128,14 @@ typedef enum {
  * @parm xch a handle to an open hypervisor interface
  * @parm fd the file descriptor to save a domain to
  * @parm dom the id of the domain
- * @param stream_type XC_MIG_STREAM_NONE if the far end of the stream
+ * @param stream_type XC_STREAM_PLAIN if the far end of the stream
  *        doesn't use checkpointing
  * @return 0 on success, -1 on failure
  */
 int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom,
                    uint32_t flags /* XCFLAGS_xxx */,
                    struct save_callbacks* callbacks, int hvm,
-                   xc_migration_stream_t stream_type, int recv_fd);
+                   xc_stream_type_t stream_type, int recv_fd);
 
 /* callbacks provided by xc_domain_restore */
 struct restore_callbacks {
@@ -198,7 +199,7 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom,
                       uint32_t store_domid, unsigned int console_evtchn,
                       unsigned long *console_mfn, uint32_t console_domid,
                       unsigned int hvm, unsigned int pae,
-                      xc_migration_stream_t stream_type,
+                      xc_stream_type_t stream_type,
                       struct restore_callbacks *callbacks, int send_back_fd);
 
 /**
diff --git a/tools/libxc/xc_nomigrate.c b/tools/libxc/xc_nomigrate.c
index 6d6169d5ad..3099e3278c 100644
--- a/tools/libxc/xc_nomigrate.c
+++ b/tools/libxc/xc_nomigrate.c
@@ -22,7 +22,7 @@
 
 int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t flags,
                    struct save_callbacks* callbacks, int hvm,
-                   xc_migration_stream_t stream_type, int recv_fd)
+                   xc_stream_type_t stream_type, int recv_fd)
 {
     errno = ENOSYS;
     return -1;
@@ -33,7 +33,7 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom,
                       uint32_t store_domid, unsigned int console_evtchn,
                       unsigned long *console_mfn, uint32_t console_domid,
                       unsigned int hvm, unsigned int pae,
-                      xc_migration_stream_t stream_type,
+                      xc_stream_type_t stream_type,
                       struct restore_callbacks *callbacks, int send_back_fd)
 {
     errno = ENOSYS;
diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h
index 19b053911f..4db63a63b2 100644
--- a/tools/libxc/xc_sr_common.h
+++ b/tools/libxc/xc_sr_common.h
@@ -205,6 +205,9 @@ struct xc_sr_context
     uint32_t domid;
     int fd;
 
+    /* Plain VM, or checkpoints over time. */
+    xc_stream_type_t stream_type;
+
     xc_dominfo_t dominfo;
 
     union /* Common save or restore data. */
@@ -219,9 +222,6 @@ struct xc_sr_context
             /* Live migrate vs non live suspend. */
             bool live;
 
-            /* Plain VM, or checkpoints over time. */
-            int checkpointed;
-
             /* Further debugging information in the stream. */
             bool debug;
 
@@ -252,9 +252,6 @@ struct xc_sr_context
             uint32_t guest_type;
             uint32_t guest_page_size;
 
-            /* Plain VM, or checkpoints over time. */
-            int checkpointed;
-
             /* Currently buffering records between a checkpoint */
             bool buffer_all_records;
 
diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c
index 98f3fe4098..7872b71ab5 100644
--- a/tools/libxc/xc_sr_restore.c
+++ b/tools/libxc/xc_sr_restore.c
@@ -511,7 +511,7 @@ static int handle_checkpoint(struct xc_sr_context *ctx)
     int rc = 0, ret;
     unsigned int i;
 
-    if ( !ctx->restore.checkpointed )
+    if ( ctx->stream_type == XC_STREAM_PLAIN )
     {
         ERROR("Found checkpoint in non-checkpointed stream");
         rc = -1;
@@ -553,7 +553,7 @@ static int handle_checkpoint(struct xc_sr_context *ctx)
     else
         ctx->restore.buffer_all_records = true;
 
-    if ( ctx->restore.checkpointed == XC_MIG_STREAM_COLO )
+    if ( ctx->stream_type == XC_STREAM_COLO )
     {
 #define HANDLE_CALLBACK_RETURN_VALUE(ret)                   \
     do {                                                    \
@@ -672,7 +672,7 @@ static int setup(struct xc_sr_context *ctx)
     DECLARE_HYPERCALL_BUFFER_SHADOW(unsigned long, dirty_bitmap,
                                     &ctx->restore.dirty_bitmap_hbuf);
 
-    if ( ctx->restore.checkpointed == XC_MIG_STREAM_COLO )
+    if ( ctx->stream_type == XC_STREAM_COLO )
     {
         dirty_bitmap = xc_hypercall_buffer_alloc_pages(
             xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->restore.p2m_size)));
@@ -723,7 +723,7 @@ static void cleanup(struct xc_sr_context *ctx)
     for ( i = 0; i < ctx->restore.buffered_rec_num; i++ )
         free(ctx->restore.buffered_records[i].data);
 
-    if ( ctx->restore.checkpointed == XC_MIG_STREAM_COLO )
+    if ( ctx->stream_type == XC_STREAM_COLO )
         xc_hypercall_buffer_free_pages(
             xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->restore.p2m_size)));
 
@@ -793,7 +793,7 @@ static int restore(struct xc_sr_context *ctx)
     } while ( rec.type != REC_TYPE_END );
 
  remus_failover:
-    if ( ctx->restore.checkpointed == XC_MIG_STREAM_COLO )
+    if ( ctx->stream_type == XC_STREAM_COLO )
     {
         /* With COLO, we have already called stream_complete */
         rc = 0;
@@ -834,13 +834,14 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom,
                       uint32_t store_domid, unsigned int console_evtchn,
                       unsigned long *console_gfn, uint32_t console_domid,
                       unsigned int hvm, unsigned int pae,
-                      xc_migration_stream_t stream_type,
+                      xc_stream_type_t stream_type,
                       struct restore_callbacks *callbacks, int send_back_fd)
 {
     xen_pfn_t nr_pfns;
     struct xc_sr_context ctx = {
         .xch = xch,
         .fd = io_fd,
+        .stream_type = stream_type,
     };
 
     /* GCC 4.4 (of CentOS 6.x vintage) can' t initialise anonymous unions. */
@@ -848,21 +849,27 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom,
     ctx.restore.console_domid = console_domid;
     ctx.restore.xenstore_evtchn = store_evtchn;
     ctx.restore.xenstore_domid = store_domid;
-    ctx.restore.checkpointed = stream_type;
     ctx.restore.callbacks = callbacks;
     ctx.restore.send_back_fd = send_back_fd;
 
-    /* Sanity checks for callbacks. */
-    if ( stream_type )
-        assert(callbacks->checkpoint);
-
-    if ( ctx.restore.checkpointed == XC_MIG_STREAM_COLO )
+    /* Sanity check stream_type-related parameters */
+    switch ( stream_type )
     {
-        /* this is COLO restore */
+    case XC_STREAM_COLO:
         assert(callbacks->suspend &&
                callbacks->postcopy &&
                callbacks->wait_checkpoint &&
                callbacks->restore_results);
+        /* Fallthrough */
+    case XC_STREAM_REMUS:
+        assert(callbacks->checkpoint);
+        /* Fallthrough */
+    case XC_STREAM_PLAIN:
+        break;
+
+    default:
+        assert(!"Bad stream_type");
+        break;
     }
 
     DPRINTF("fd %d, dom %u, hvm %u, pae %u, stream_type %d",
diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
index 9764aa743f..5467965b08 100644
--- a/tools/libxc/xc_sr_save.c
+++ b/tools/libxc/xc_sr_save.c
@@ -658,7 +658,7 @@ static int suspend_and_send_dirty(struct xc_sr_context *ctx)
 
     bitmap_or(dirty_bitmap, ctx->save.deferred_pages, ctx->save.p2m_size);
 
-    if ( !ctx->save.live && ctx->save.checkpointed == XC_MIG_STREAM_COLO )
+    if ( !ctx->save.live && ctx->stream_type == XC_STREAM_COLO )
     {
         rc = colo_merge_secondary_dirty_bitmap(ctx);
         if ( rc )
@@ -735,7 +735,7 @@ static int send_domain_memory_live(struct xc_sr_context *ctx)
     if ( rc )
         goto out;
 
-    if ( ctx->save.debug && ctx->save.checkpointed != XC_MIG_STREAM_NONE )
+    if ( ctx->save.debug && ctx->stream_type != XC_STREAM_PLAIN )
     {
         rc = verify_frames(ctx);
         if ( rc )
@@ -861,7 +861,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type)
 
         if ( ctx->save.live )
             rc = send_domain_memory_live(ctx);
-        else if ( ctx->save.checkpointed != XC_MIG_STREAM_NONE )
+        else if ( ctx->stream_type != XC_STREAM_PLAIN )
             rc = send_domain_memory_checkpointed(ctx);
         else
             rc = send_domain_memory_nonlive(ctx);
@@ -881,7 +881,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type)
         if ( rc )
             goto err;
 
-        if ( ctx->save.checkpointed != XC_MIG_STREAM_NONE )
+        if ( ctx->stream_type != XC_STREAM_PLAIN )
         {
             /*
              * We have now completed the initial live portion of the checkpoint
@@ -894,7 +894,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type)
             if ( rc )
                 goto err;
 
-            if ( ctx->save.checkpointed == XC_MIG_STREAM_COLO )
+            if ( ctx->stream_type == XC_STREAM_COLO )
             {
                 rc = ctx->save.callbacks->checkpoint(ctx->save.callbacks->data);
                 if ( !rc )
@@ -908,14 +908,14 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type)
             if ( rc <= 0 )
                 goto err;
 
-            if ( ctx->save.checkpointed == XC_MIG_STREAM_COLO )
+            if ( ctx->stream_type == XC_STREAM_COLO )
             {
                 rc = ctx->save.callbacks->wait_checkpoint(
                     ctx->save.callbacks->data);
                 if ( rc <= 0 )
                     goto err;
             }
-            else if ( ctx->save.checkpointed == XC_MIG_STREAM_REMUS )
+            else if ( ctx->stream_type == XC_STREAM_REMUS )
             {
                 rc = ctx->save.callbacks->checkpoint(ctx->save.callbacks->data);
                 if ( rc <= 0 )
@@ -928,7 +928,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type)
                 goto err;
             }
         }
-    } while ( ctx->save.checkpointed != XC_MIG_STREAM_NONE );
+    } while ( ctx->stream_type != XC_STREAM_PLAIN );
 
     xc_report_progress_single(xch, "End of stream");
 
@@ -958,32 +958,40 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type)
 
 int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom,
                    uint32_t flags, struct save_callbacks *callbacks,
-                   int hvm, xc_migration_stream_t stream_type, int recv_fd)
+                   int hvm, xc_stream_type_t stream_type, int recv_fd)
 {
     struct xc_sr_context ctx = {
         .xch = xch,
         .fd = io_fd,
+        .stream_type = stream_type,
     };
 
     /* GCC 4.4 (of CentOS 6.x vintage) can' t initialise anonymous unions. */
     ctx.save.callbacks = callbacks;
     ctx.save.live  = !!(flags & XCFLAGS_LIVE);
     ctx.save.debug = !!(flags & XCFLAGS_DEBUG);
-    ctx.save.checkpointed = stream_type;
     ctx.save.recv_fd = recv_fd;
 
-    /* If altering migration_stream update this assert too. */
-    assert(stream_type == XC_MIG_STREAM_NONE ||
-           stream_type == XC_MIG_STREAM_REMUS ||
-           stream_type == XC_MIG_STREAM_COLO);
+    /* Sanity check stream_type-related parameters */
+    switch ( stream_type )
+    {
+    case XC_STREAM_COLO:
+        assert(callbacks->wait_checkpoint);
+        /* Fallthrough */
+    case XC_STREAM_REMUS:
+        assert(callbacks->checkpoint && callbacks->postcopy);
+        /* Fallthrough */
+    case XC_STREAM_PLAIN:
+        break;
+
+    default:
+        assert(!"Bad stream_type");
+        break;
+    }
 
     /* Sanity checks for callbacks. */
     if ( hvm )
         assert(callbacks->switch_qemu_logdirty);
-    if ( ctx.save.checkpointed )
-        assert(callbacks->checkpoint && callbacks->postcopy);
-    if ( ctx.save.checkpointed == XC_MIG_STREAM_COLO )
-        assert(callbacks->wait_checkpoint);
 
     DPRINTF("fd %d, dom %u, flags %u, hvm %d", io_fd, dom, flags, hvm);
 
diff --git a/tools/libxl/libxl_save_helper.c b/tools/libxl/libxl_save_helper.c
index 38089a002d..398df00dd6 100644
--- a/tools/libxl/libxl_save_helper.c
+++ b/tools/libxl/libxl_save_helper.c
@@ -254,7 +254,7 @@ int main(int argc, char **argv)
         uint32_t flags =                    strtoul(NEXTARG,0,10);
         int hvm =                           atoi(NEXTARG);
         unsigned cbflags =                  strtoul(NEXTARG,0,10);
-        xc_migration_stream_t stream_type = strtoul(NEXTARG,0,10);
+        xc_stream_type_t stream_type =      strtoul(NEXTARG,0,10);
         assert(!*++argv);
 
         helper_setcallbacks_save(&helper_save_callbacks, cbflags);
@@ -278,7 +278,7 @@ int main(int argc, char **argv)
         unsigned int hvm =                  strtoul(NEXTARG,0,10);
         unsigned int pae =                  strtoul(NEXTARG,0,10);
         unsigned cbflags =                  strtoul(NEXTARG,0,10);
-        xc_migration_stream_t stream_type = strtoul(NEXTARG,0,10);
+        xc_stream_type_t stream_type =      strtoul(NEXTARG,0,10);
         assert(!*++argv);
 
         helper_setcallbacks_restore(&helper_restore_callbacks, cbflags);
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 04/12] libxc/migration: Adjust layout of struct xc_sr_context
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
                   ` (2 preceding siblings ...)
  2019-12-24 15:19 ` [Xen-devel] [PATCH 03/12] libxc/migration: Rationalise the 'checkpointed' field to 'stream_type' Andrew Cooper
@ 2019-12-24 15:19 ` Andrew Cooper
  2020-01-14 16:04   ` Ian Jackson
  2019-12-24 15:19 ` [Xen-devel] [PATCH 05/12] tools/migration: Drop IHDR_VERSION constant from libxc and python Andrew Cooper
                   ` (9 subsequent siblings)
  13 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel; +Cc: Andrew Cooper, Wei Liu, Ian Jackson

We are shortly going to want to introduce some common x86 fields, so having
x86_pv and x86_hvm as the top level objects is a problem.  Insert a
surrounding struct x86 and drop the x86 prefix from the pv/hvm objects.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>

This is much more easily reviewed with git-diff's --ignore-all-space option,
which highlights the relevant struct difference.

  diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h
  index 4963b30c4b..2d02e80ee3 100644
  --- a/tools/libxc/xc_sr_common.h
  +++ b/tools/libxc/xc_sr_common.h
  @@ -281,6 +281,8 @@ struct xc_sr_context

       union /* Guest-arch specific data. */
       {
  +        struct /* x86 */
  +        {
               struct /* x86 PV guest. */
               {
                   /* 4 or 8; 32 or 64 bit domain */
  @@ -332,7 +334,7 @@ struct xc_sr_context
                           unsigned int nr_vcpus;
                       } restore;
                   };
  -        } x86_pv;
  +            } pv;

               struct /* x86 HVM guest. */
               {
  @@ -351,7 +353,9 @@ struct xc_sr_context
                           struct xc_sr_blob context;
                       } restore;
                   };
  -        } x86_hvm;
  +            } hvm;
  +
  +        } x86;
       };
   };
---
 tools/libxc/xc_sr_common.h          | 132 ++++++++++-----------
 tools/libxc/xc_sr_common_x86_pv.c   |  50 ++++----
 tools/libxc/xc_sr_common_x86_pv.h   |   4 +-
 tools/libxc/xc_sr_restore_x86_hvm.c |  12 +-
 tools/libxc/xc_sr_restore_x86_pv.c  | 224 ++++++++++++++++++------------------
 tools/libxc/xc_sr_save_x86_hvm.c    |   4 +-
 tools/libxc/xc_sr_save_x86_pv.c     | 158 ++++++++++++-------------
 7 files changed, 294 insertions(+), 290 deletions(-)

diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h
index 4db63a63b2..0289c01e13 100644
--- a/tools/libxc/xc_sr_common.h
+++ b/tools/libxc/xc_sr_common.h
@@ -287,77 +287,81 @@ struct xc_sr_context
 
     union /* Guest-arch specific data. */
     {
-        struct /* x86 PV guest. */
+        struct /* x86 */
         {
-            /* 4 or 8; 32 or 64 bit domain */
-            unsigned int width;
-            /* 3 or 4 pagetable levels */
-            unsigned int levels;
-
-            /* Maximum Xen frame */
-            xen_pfn_t max_mfn;
-            /* Read-only machine to phys map */
-            xen_pfn_t *m2p;
-            /* first mfn of the compat m2p (Only needed for 32bit PV guests) */
-            xen_pfn_t compat_m2p_mfn0;
-            /* Number of m2p frames mapped */
-            unsigned long nr_m2p_frames;
-
-            /* Maximum guest frame */
-            xen_pfn_t max_pfn;
-
-            /* Number of frames making up the p2m */
-            unsigned int p2m_frames;
-            /* Guest's phys to machine map.  Mapped read-only (save) or
-             * allocated locally (restore).  Uses guest unsigned longs. */
-            void *p2m;
-            /* The guest pfns containing the p2m leaves */
-            xen_pfn_t *p2m_pfns;
-
-            /* Read-only mapping of guests shared info page */
-            shared_info_any_t *shinfo;
-
-            /* p2m generation count for verifying validity of local p2m. */
-            uint64_t p2m_generation;
-
-            union
+            struct /* x86 PV guest. */
             {
-                struct
+                /* 4 or 8; 32 or 64 bit domain */
+                unsigned int width;
+                /* 3 or 4 pagetable levels */
+                unsigned int levels;
+
+                /* Maximum Xen frame */
+                xen_pfn_t max_mfn;
+                /* Read-only machine to phys map */
+                xen_pfn_t *m2p;
+                /* first mfn of the compat m2p (Only needed for 32bit PV guests) */
+                xen_pfn_t compat_m2p_mfn0;
+                /* Number of m2p frames mapped */
+                unsigned long nr_m2p_frames;
+
+                /* Maximum guest frame */
+                xen_pfn_t max_pfn;
+
+                /* Number of frames making up the p2m */
+                unsigned int p2m_frames;
+                /* Guest's phys to machine map.  Mapped read-only (save) or
+                 * allocated locally (restore).  Uses guest unsigned longs. */
+                void *p2m;
+                /* The guest pfns containing the p2m leaves */
+                xen_pfn_t *p2m_pfns;
+
+                /* Read-only mapping of guests shared info page */
+                shared_info_any_t *shinfo;
+
+                /* p2m generation count for verifying validity of local p2m. */
+                uint64_t p2m_generation;
+
+                union
                 {
-                    /* State machine for the order of received records. */
-                    bool seen_pv_info;
-
-                    /* Types for each page (bounded by max_pfn). */
-                    uint32_t *pfn_types;
-
-                    /* x86 PV per-vcpu storage structure for blobs. */
-                    struct xc_sr_x86_pv_restore_vcpu
+                    struct
                     {
-                        struct xc_sr_blob basic, extd, xsave, msr;
-                    } *vcpus;
-                    unsigned int nr_vcpus;
-                } restore;
-            };
-        } x86_pv;
-
-        struct /* x86 HVM guest. */
-        {
-            union
+                        /* State machine for the order of received records. */
+                        bool seen_pv_info;
+
+                        /* Types for each page (bounded by max_pfn). */
+                        uint32_t *pfn_types;
+
+                        /* x86 PV per-vcpu storage structure for blobs. */
+                        struct xc_sr_x86_pv_restore_vcpu
+                        {
+                            struct xc_sr_blob basic, extd, xsave, msr;
+                        } *vcpus;
+                        unsigned int nr_vcpus;
+                    } restore;
+                };
+            } pv;
+
+            struct /* x86 HVM guest. */
             {
-                struct
+                union
                 {
-                    /* Whether qemu enabled logdirty mode, and we should
-                     * disable on cleanup. */
-                    bool qemu_enabled_logdirty;
-                } save;
+                    struct
+                    {
+                        /* Whether qemu enabled logdirty mode, and we should
+                         * disable on cleanup. */
+                        bool qemu_enabled_logdirty;
+                    } save;
 
-                struct
-                {
-                    /* HVM context blob. */
-                    struct xc_sr_blob context;
-                } restore;
-            };
-        } x86_hvm;
+                    struct
+                    {
+                        /* HVM context blob. */
+                        struct xc_sr_blob context;
+                    } restore;
+                };
+            } hvm;
+
+        } x86;
     };
 };
 
diff --git a/tools/libxc/xc_sr_common_x86_pv.c b/tools/libxc/xc_sr_common_x86_pv.c
index ec433fad70..d3d425cb82 100644
--- a/tools/libxc/xc_sr_common_x86_pv.c
+++ b/tools/libxc/xc_sr_common_x86_pv.c
@@ -4,16 +4,16 @@
 
 xen_pfn_t mfn_to_pfn(struct xc_sr_context *ctx, xen_pfn_t mfn)
 {
-    assert(mfn <= ctx->x86_pv.max_mfn);
-    return ctx->x86_pv.m2p[mfn];
+    assert(mfn <= ctx->x86.pv.max_mfn);
+    return ctx->x86.pv.m2p[mfn];
 }
 
 bool mfn_in_pseudophysmap(struct xc_sr_context *ctx, xen_pfn_t mfn)
 {
-    return ((mfn <= ctx->x86_pv.max_mfn) &&
-            (mfn_to_pfn(ctx, mfn) <= ctx->x86_pv.max_pfn) &&
-            (xc_pfn_to_mfn(mfn_to_pfn(ctx, mfn), ctx->x86_pv.p2m,
-                           ctx->x86_pv.width) == mfn));
+    return ((mfn <= ctx->x86.pv.max_mfn) &&
+            (mfn_to_pfn(ctx, mfn) <= ctx->x86.pv.max_pfn) &&
+            (xc_pfn_to_mfn(mfn_to_pfn(ctx, mfn), ctx->x86.pv.p2m,
+                           ctx->x86.pv.width) == mfn));
 }
 
 void dump_bad_pseudophysmap_entry(struct xc_sr_context *ctx, xen_pfn_t mfn)
@@ -21,23 +21,23 @@ void dump_bad_pseudophysmap_entry(struct xc_sr_context *ctx, xen_pfn_t mfn)
     xc_interface *xch = ctx->xch;
     xen_pfn_t pfn = ~0UL;
 
-    ERROR("mfn %#lx, max %#lx", mfn, ctx->x86_pv.max_mfn);
+    ERROR("mfn %#lx, max %#lx", mfn, ctx->x86.pv.max_mfn);
 
-    if ( (mfn != ~0UL) && (mfn <= ctx->x86_pv.max_mfn) )
+    if ( (mfn != ~0UL) && (mfn <= ctx->x86.pv.max_mfn) )
     {
-        pfn = ctx->x86_pv.m2p[mfn];
+        pfn = ctx->x86.pv.m2p[mfn];
         ERROR("  m2p[%#lx] = %#lx, max_pfn %#lx",
-              mfn, pfn, ctx->x86_pv.max_pfn);
+              mfn, pfn, ctx->x86.pv.max_pfn);
     }
 
-    if ( (pfn != ~0UL) && (pfn <= ctx->x86_pv.max_pfn) )
+    if ( (pfn != ~0UL) && (pfn <= ctx->x86.pv.max_pfn) )
         ERROR("  p2m[%#lx] = %#lx",
-              pfn, xc_pfn_to_mfn(pfn, ctx->x86_pv.p2m, ctx->x86_pv.width));
+              pfn, xc_pfn_to_mfn(pfn, ctx->x86.pv.p2m, ctx->x86.pv.width));
 }
 
 xen_pfn_t cr3_to_mfn(struct xc_sr_context *ctx, uint64_t cr3)
 {
-    if ( ctx->x86_pv.width == 8 )
+    if ( ctx->x86.pv.width == 8 )
         return cr3 >> 12;
     else
     {
@@ -53,7 +53,7 @@ uint64_t mfn_to_cr3(struct xc_sr_context *ctx, xen_pfn_t _mfn)
 {
     uint64_t mfn = _mfn;
 
-    if ( ctx->x86_pv.width == 8 )
+    if ( ctx->x86.pv.width == 8 )
         return mfn << 12;
     else
     {
@@ -86,8 +86,8 @@ int x86_pv_domain_info(struct xc_sr_context *ctx)
         ERROR("Invalid guest width %d.  Expected 32 or 64", guest_width * 8);
         return -1;
     }
-    ctx->x86_pv.width = guest_width;
-    ctx->x86_pv.levels = guest_levels;
+    ctx->x86.pv.width = guest_width;
+    ctx->x86.pv.levels = guest_levels;
 
     DPRINTF("%d bits, %d levels", guest_width * 8, guest_levels);
 
@@ -108,9 +108,9 @@ int x86_pv_map_m2p(struct xc_sr_context *ctx)
         goto err;
     }
 
-    ctx->x86_pv.max_mfn = max_page;
-    m2p_size   = M2P_SIZE(ctx->x86_pv.max_mfn);
-    m2p_chunks = M2P_CHUNKS(ctx->x86_pv.max_mfn);
+    ctx->x86.pv.max_mfn = max_page;
+    m2p_size   = M2P_SIZE(ctx->x86.pv.max_mfn);
+    m2p_chunks = M2P_CHUNKS(ctx->x86.pv.max_mfn);
 
     extents_start = malloc(m2p_chunks * sizeof(xen_pfn_t));
     if ( !extents_start )
@@ -137,27 +137,27 @@ int x86_pv_map_m2p(struct xc_sr_context *ctx)
     for ( i = 0; i < m2p_chunks; ++i )
         entries[i].mfn = extents_start[i];
 
-    ctx->x86_pv.m2p = xc_map_foreign_ranges(
+    ctx->x86.pv.m2p = xc_map_foreign_ranges(
         xch, DOMID_XEN, m2p_size, PROT_READ,
         M2P_CHUNK_SIZE, entries, m2p_chunks);
 
-    if ( !ctx->x86_pv.m2p )
+    if ( !ctx->x86.pv.m2p )
     {
         PERROR("Failed to mmap() m2p ranges");
         goto err;
     }
 
-    ctx->x86_pv.nr_m2p_frames = (M2P_CHUNK_SIZE >> PAGE_SHIFT) * m2p_chunks;
+    ctx->x86.pv.nr_m2p_frames = (M2P_CHUNK_SIZE >> PAGE_SHIFT) * m2p_chunks;
 
 #ifdef __i386__
     /* 32 bit toolstacks automatically get the compat m2p */
-    ctx->x86_pv.compat_m2p_mfn0 = entries[0].mfn;
+    ctx->x86.pv.compat_m2p_mfn0 = entries[0].mfn;
 #else
     /* 64 bit toolstacks need to ask Xen specially for it */
     {
         struct xen_machphys_mfn_list xmml = {
             .max_extents = 1,
-            .extent_start = { &ctx->x86_pv.compat_m2p_mfn0 },
+            .extent_start = { &ctx->x86.pv.compat_m2p_mfn0 },
         };
 
         rc = do_memory_op(xch, XENMEM_machphys_compat_mfn_list,
@@ -173,7 +173,7 @@ int x86_pv_map_m2p(struct xc_sr_context *ctx)
 
     /* All Done */
     rc = 0;
-    DPRINTF("max_mfn %#lx", ctx->x86_pv.max_mfn);
+    DPRINTF("max_mfn %#lx", ctx->x86.pv.max_mfn);
 
  err:
     free(entries);
diff --git a/tools/libxc/xc_sr_common_x86_pv.h b/tools/libxc/xc_sr_common_x86_pv.h
index f80c75349a..2ed03309af 100644
--- a/tools/libxc/xc_sr_common_x86_pv.h
+++ b/tools/libxc/xc_sr_common_x86_pv.h
@@ -73,7 +73,7 @@ static inline uint64_t merge_pte(uint64_t pte, xen_pfn_t mfn)
 /*
  * Get current domain information.
  *
- * Fills ctx->x86_pv
+ * Fills ctx->x86.pv
  * - .width
  * - .levels
  * - .fpp
@@ -89,7 +89,7 @@ int x86_pv_domain_info(struct xc_sr_context *ctx);
 /*
  * Maps the Xen M2P.
  *
- * Fills ctx->x86_pv.
+ * Fills ctx->x86.pv.
  * - .max_mfn
  * - .m2p
  *
diff --git a/tools/libxc/xc_sr_restore_x86_hvm.c b/tools/libxc/xc_sr_restore_x86_hvm.c
index fe7be9bde6..3f78248f32 100644
--- a/tools/libxc/xc_sr_restore_x86_hvm.c
+++ b/tools/libxc/xc_sr_restore_x86_hvm.c
@@ -10,7 +10,7 @@ static int handle_hvm_context(struct xc_sr_context *ctx,
                               struct xc_sr_record *rec)
 {
     xc_interface *xch = ctx->xch;
-    int rc = update_blob(&ctx->x86_hvm.restore.context, rec->data, rec->length);
+    int rc = update_blob(&ctx->x86.hvm.restore.context, rec->data, rec->length);
 
     if ( rc )
         ERROR("Unable to allocate %u bytes for hvm context", rec->length);
@@ -129,14 +129,14 @@ static int x86_hvm_setup(struct xc_sr_context *ctx)
 
     if ( ctx->restore.guest_type != DHDR_TYPE_X86_HVM )
     {
-        ERROR("Unable to restore %s domain into an x86_hvm domain",
+        ERROR("Unable to restore %s domain into an x86 HVM domain",
               dhdr_type_to_str(ctx->restore.guest_type));
         return -1;
     }
 
     if ( ctx->restore.guest_page_size != PAGE_SIZE )
     {
-        ERROR("Invalid page size %u for x86_hvm domains",
+        ERROR("Invalid page size %u for x86 HVM domains",
               ctx->restore.guest_page_size);
         return -1;
     }
@@ -201,8 +201,8 @@ static int x86_hvm_stream_complete(struct xc_sr_context *ctx)
     }
 
     rc = xc_domain_hvm_setcontext(xch, ctx->domid,
-                                  ctx->x86_hvm.restore.context.ptr,
-                                  ctx->x86_hvm.restore.context.size);
+                                  ctx->x86.hvm.restore.context.ptr,
+                                  ctx->x86.hvm.restore.context.size);
     if ( rc < 0 )
     {
         PERROR("Unable to restore HVM context");
@@ -225,7 +225,7 @@ static int x86_hvm_stream_complete(struct xc_sr_context *ctx)
 
 static int x86_hvm_cleanup(struct xc_sr_context *ctx)
 {
-    free(ctx->x86_hvm.restore.context.ptr);
+    free(ctx->x86.hvm.restore.context.ptr);
 
     return 0;
 }
diff --git a/tools/libxc/xc_sr_restore_x86_pv.c b/tools/libxc/xc_sr_restore_x86_pv.c
index 0ec506632a..8f61a5e8b9 100644
--- a/tools/libxc/xc_sr_restore_x86_pv.c
+++ b/tools/libxc/xc_sr_restore_x86_pv.c
@@ -4,9 +4,9 @@
 
 static xen_pfn_t pfn_to_mfn(const struct xc_sr_context *ctx, xen_pfn_t pfn)
 {
-    assert(pfn <= ctx->x86_pv.max_pfn);
+    assert(pfn <= ctx->x86.pv.max_pfn);
 
-    return xc_pfn_to_mfn(pfn, ctx->x86_pv.p2m, ctx->x86_pv.width);
+    return xc_pfn_to_mfn(pfn, ctx->x86.pv.p2m, ctx->x86.pv.width);
 }
 
 /*
@@ -18,8 +18,8 @@ static xen_pfn_t pfn_to_mfn(const struct xc_sr_context *ctx, xen_pfn_t pfn)
 static int expand_p2m(struct xc_sr_context *ctx, unsigned long max_pfn)
 {
     xc_interface *xch = ctx->xch;
-    unsigned long old_max = ctx->x86_pv.max_pfn, i;
-    unsigned int fpp = PAGE_SIZE / ctx->x86_pv.width;
+    unsigned long old_max = ctx->x86.pv.max_pfn, i;
+    unsigned int fpp = PAGE_SIZE / ctx->x86.pv.width;
     unsigned long end_frame = (max_pfn / fpp) + 1;
     unsigned long old_end_frame = (old_max / fpp) + 1;
     xen_pfn_t *p2m = NULL, *p2m_pfns = NULL;
@@ -28,35 +28,35 @@ static int expand_p2m(struct xc_sr_context *ctx, unsigned long max_pfn)
 
     assert(max_pfn > old_max);
 
-    p2msz = (max_pfn + 1) * ctx->x86_pv.width;
-    p2m = realloc(ctx->x86_pv.p2m, p2msz);
+    p2msz = (max_pfn + 1) * ctx->x86.pv.width;
+    p2m = realloc(ctx->x86.pv.p2m, p2msz);
     if ( !p2m )
     {
         ERROR("Failed to (re)alloc %zu bytes for p2m", p2msz);
         return -1;
     }
-    ctx->x86_pv.p2m = p2m;
+    ctx->x86.pv.p2m = p2m;
 
     pfn_typesz = (max_pfn + 1) * sizeof(*pfn_types);
-    pfn_types = realloc(ctx->x86_pv.restore.pfn_types, pfn_typesz);
+    pfn_types = realloc(ctx->x86.pv.restore.pfn_types, pfn_typesz);
     if ( !pfn_types )
     {
         ERROR("Failed to (re)alloc %zu bytes for pfn_types", pfn_typesz);
         return -1;
     }
-    ctx->x86_pv.restore.pfn_types = pfn_types;
+    ctx->x86.pv.restore.pfn_types = pfn_types;
 
     p2m_pfnsz = (end_frame + 1) * sizeof(*p2m_pfns);
-    p2m_pfns = realloc(ctx->x86_pv.p2m_pfns, p2m_pfnsz);
+    p2m_pfns = realloc(ctx->x86.pv.p2m_pfns, p2m_pfnsz);
     if ( !p2m_pfns )
     {
         ERROR("Failed to (re)alloc %zu bytes for p2m frame list", p2m_pfnsz);
         return -1;
     }
-    ctx->x86_pv.p2m_frames = end_frame;
-    ctx->x86_pv.p2m_pfns = p2m_pfns;
+    ctx->x86.pv.p2m_frames = end_frame;
+    ctx->x86.pv.p2m_pfns = p2m_pfns;
 
-    ctx->x86_pv.max_pfn = max_pfn;
+    ctx->x86.pv.max_pfn = max_pfn;
     for ( i = (old_max ? old_max + 1 : 0); i <= max_pfn; ++i )
     {
         ctx->restore.ops.set_gfn(ctx, i, INVALID_MFN);
@@ -64,7 +64,7 @@ static int expand_p2m(struct xc_sr_context *ctx, unsigned long max_pfn)
     }
 
     for ( i = (old_end_frame ? old_end_frame + 1 : 0); i <= end_frame; ++i )
-        ctx->x86_pv.p2m_pfns[i] = INVALID_MFN;
+        ctx->x86.pv.p2m_pfns[i] = INVALID_MFN;
 
     DPRINTF("Changed max_pfn from %#lx to %#lx", old_max, max_pfn);
     return 0;
@@ -79,13 +79,13 @@ static int pin_pagetables(struct xc_sr_context *ctx)
     unsigned long i, nr_pins;
     struct mmuext_op pin[MAX_PIN_BATCH];
 
-    for ( i = nr_pins = 0; i <= ctx->x86_pv.max_pfn; ++i )
+    for ( i = nr_pins = 0; i <= ctx->x86.pv.max_pfn; ++i )
     {
-        if ( (ctx->x86_pv.restore.pfn_types[i] &
+        if ( (ctx->x86.pv.restore.pfn_types[i] &
               XEN_DOMCTL_PFINFO_LPINTAB) == 0 )
             continue;
 
-        switch ( (ctx->x86_pv.restore.pfn_types[i] &
+        switch ( (ctx->x86.pv.restore.pfn_types[i] &
                   XEN_DOMCTL_PFINFO_LTABTYPE_MASK) )
         {
         case XEN_DOMCTL_PFINFO_L1TAB:
@@ -138,18 +138,18 @@ static int process_start_info(struct xc_sr_context *ctx,
     start_info_any_t *guest_start_info = NULL;
     int rc = -1;
 
-    pfn = GET_FIELD(vcpu, user_regs.edx, ctx->x86_pv.width);
+    pfn = GET_FIELD(vcpu, user_regs.edx, ctx->x86.pv.width);
 
-    if ( pfn > ctx->x86_pv.max_pfn )
+    if ( pfn > ctx->x86.pv.max_pfn )
     {
         ERROR("Start Info pfn %#lx out of range", pfn);
         goto err;
     }
 
-    if ( ctx->x86_pv.restore.pfn_types[pfn] != XEN_DOMCTL_PFINFO_NOTAB )
+    if ( ctx->x86.pv.restore.pfn_types[pfn] != XEN_DOMCTL_PFINFO_NOTAB )
     {
         ERROR("Start Info pfn %#lx has bad type %u", pfn,
-              (ctx->x86_pv.restore.pfn_types[pfn] >>
+              (ctx->x86.pv.restore.pfn_types[pfn] >>
                XEN_DOMCTL_PFINFO_LTAB_SHIFT));
         goto err;
     }
@@ -162,7 +162,7 @@ static int process_start_info(struct xc_sr_context *ctx,
         goto err;
     }
 
-    SET_FIELD(vcpu, user_regs.edx, mfn, ctx->x86_pv.width);
+    SET_FIELD(vcpu, user_regs.edx, mfn, ctx->x86.pv.width);
     guest_start_info = xc_map_foreign_range(
         xch, ctx->domid, PAGE_SIZE, PROT_READ | PROT_WRITE, mfn);
     if ( !guest_start_info )
@@ -172,8 +172,8 @@ static int process_start_info(struct xc_sr_context *ctx,
     }
 
     /* Deal with xenstore stuff */
-    pfn = GET_FIELD(guest_start_info, store_mfn, ctx->x86_pv.width);
-    if ( pfn > ctx->x86_pv.max_pfn )
+    pfn = GET_FIELD(guest_start_info, store_mfn, ctx->x86.pv.width);
+    if ( pfn > ctx->x86.pv.max_pfn )
     {
         ERROR("XenStore pfn %#lx out of range", pfn);
         goto err;
@@ -188,13 +188,13 @@ static int process_start_info(struct xc_sr_context *ctx,
     }
 
     ctx->restore.xenstore_gfn = mfn;
-    SET_FIELD(guest_start_info, store_mfn, mfn, ctx->x86_pv.width);
+    SET_FIELD(guest_start_info, store_mfn, mfn, ctx->x86.pv.width);
     SET_FIELD(guest_start_info, store_evtchn,
-              ctx->restore.xenstore_evtchn, ctx->x86_pv.width);
+              ctx->restore.xenstore_evtchn, ctx->x86.pv.width);
 
     /* Deal with console stuff */
-    pfn = GET_FIELD(guest_start_info, console.domU.mfn, ctx->x86_pv.width);
-    if ( pfn > ctx->x86_pv.max_pfn )
+    pfn = GET_FIELD(guest_start_info, console.domU.mfn, ctx->x86.pv.width);
+    if ( pfn > ctx->x86.pv.max_pfn )
     {
         ERROR("Console pfn %#lx out of range", pfn);
         goto err;
@@ -209,16 +209,16 @@ static int process_start_info(struct xc_sr_context *ctx,
     }
 
     ctx->restore.console_gfn = mfn;
-    SET_FIELD(guest_start_info, console.domU.mfn, mfn, ctx->x86_pv.width);
+    SET_FIELD(guest_start_info, console.domU.mfn, mfn, ctx->x86.pv.width);
     SET_FIELD(guest_start_info, console.domU.evtchn,
-              ctx->restore.console_evtchn, ctx->x86_pv.width);
+              ctx->restore.console_evtchn, ctx->x86.pv.width);
 
     /* Set other information */
     SET_FIELD(guest_start_info, nr_pages,
-              ctx->x86_pv.max_pfn + 1, ctx->x86_pv.width);
+              ctx->x86.pv.max_pfn + 1, ctx->x86.pv.width);
     SET_FIELD(guest_start_info, shared_info,
-              ctx->dominfo.shared_info_frame << PAGE_SHIFT, ctx->x86_pv.width);
-    SET_FIELD(guest_start_info, flags, 0, ctx->x86_pv.width);
+              ctx->dominfo.shared_info_frame << PAGE_SHIFT, ctx->x86.pv.width);
+    SET_FIELD(guest_start_info, flags, 0, ctx->x86.pv.width);
 
     rc = 0;
 
@@ -236,7 +236,7 @@ static int process_vcpu_basic(struct xc_sr_context *ctx,
                               unsigned int vcpuid)
 {
     xc_interface *xch = ctx->xch;
-    vcpu_guest_context_any_t *vcpu = ctx->x86_pv.restore.vcpus[vcpuid].basic.ptr;
+    vcpu_guest_context_any_t *vcpu = ctx->x86.pv.restore.vcpus[vcpuid].basic.ptr;
     xen_pfn_t pfn, mfn;
     unsigned int i, gdt_count;
     int rc = -1;
@@ -251,10 +251,10 @@ static int process_vcpu_basic(struct xc_sr_context *ctx,
     }
 
     SET_FIELD(vcpu, flags,
-              GET_FIELD(vcpu, flags, ctx->x86_pv.width) | VGCF_online,
-              ctx->x86_pv.width);
+              GET_FIELD(vcpu, flags, ctx->x86.pv.width) | VGCF_online,
+              ctx->x86.pv.width);
 
-    gdt_count = GET_FIELD(vcpu, gdt_ents, ctx->x86_pv.width);
+    gdt_count = GET_FIELD(vcpu, gdt_ents, ctx->x86.pv.width);
     if ( gdt_count > FIRST_RESERVED_GDT_ENTRY )
     {
         ERROR("GDT entry count (%u) out of range (max %u)",
@@ -267,17 +267,17 @@ static int process_vcpu_basic(struct xc_sr_context *ctx,
     /* Convert GDT frames to mfns. */
     for ( i = 0; i < gdt_count; ++i )
     {
-        pfn = GET_FIELD(vcpu, gdt_frames[i], ctx->x86_pv.width);
-        if ( pfn > ctx->x86_pv.max_pfn )
+        pfn = GET_FIELD(vcpu, gdt_frames[i], ctx->x86.pv.width);
+        if ( pfn > ctx->x86.pv.max_pfn )
         {
             ERROR("GDT frame %u (pfn %#lx) out of range", i, pfn);
             goto err;
         }
 
-        if ( (ctx->x86_pv.restore.pfn_types[pfn] != XEN_DOMCTL_PFINFO_NOTAB) )
+        if ( (ctx->x86.pv.restore.pfn_types[pfn] != XEN_DOMCTL_PFINFO_NOTAB) )
         {
             ERROR("GDT frame %u (pfn %#lx) has bad type %u", i, pfn,
-                  (ctx->x86_pv.restore.pfn_types[pfn] >>
+                  (ctx->x86.pv.restore.pfn_types[pfn] >>
                    XEN_DOMCTL_PFINFO_LTAB_SHIFT));
             goto err;
         }
@@ -290,25 +290,25 @@ static int process_vcpu_basic(struct xc_sr_context *ctx,
             goto err;
         }
 
-        SET_FIELD(vcpu, gdt_frames[i], mfn, ctx->x86_pv.width);
+        SET_FIELD(vcpu, gdt_frames[i], mfn, ctx->x86.pv.width);
     }
 
     /* Convert CR3 to an mfn. */
-    pfn = cr3_to_mfn(ctx, GET_FIELD(vcpu, ctrlreg[3], ctx->x86_pv.width));
-    if ( pfn > ctx->x86_pv.max_pfn )
+    pfn = cr3_to_mfn(ctx, GET_FIELD(vcpu, ctrlreg[3], ctx->x86.pv.width));
+    if ( pfn > ctx->x86.pv.max_pfn )
     {
         ERROR("cr3 (pfn %#lx) out of range", pfn);
         goto err;
     }
 
-    if ( (ctx->x86_pv.restore.pfn_types[pfn] &
+    if ( (ctx->x86.pv.restore.pfn_types[pfn] &
           XEN_DOMCTL_PFINFO_LTABTYPE_MASK) !=
-         (((xen_pfn_t)ctx->x86_pv.levels) << XEN_DOMCTL_PFINFO_LTAB_SHIFT) )
+         (((xen_pfn_t)ctx->x86.pv.levels) << XEN_DOMCTL_PFINFO_LTAB_SHIFT) )
     {
         ERROR("cr3 (pfn %#lx) has bad type %u, expected %u", pfn,
-              (ctx->x86_pv.restore.pfn_types[pfn] >>
+              (ctx->x86.pv.restore.pfn_types[pfn] >>
                XEN_DOMCTL_PFINFO_LTAB_SHIFT),
-              ctx->x86_pv.levels);
+              ctx->x86.pv.levels);
         goto err;
     }
 
@@ -320,27 +320,27 @@ static int process_vcpu_basic(struct xc_sr_context *ctx,
         goto err;
     }
 
-    SET_FIELD(vcpu, ctrlreg[3], mfn_to_cr3(ctx, mfn), ctx->x86_pv.width);
+    SET_FIELD(vcpu, ctrlreg[3], mfn_to_cr3(ctx, mfn), ctx->x86.pv.width);
 
     /* 64bit guests: Convert CR1 (guest pagetables) to mfn. */
-    if ( ctx->x86_pv.levels == 4 && (vcpu->x64.ctrlreg[1] & 1) )
+    if ( ctx->x86.pv.levels == 4 && (vcpu->x64.ctrlreg[1] & 1) )
     {
         pfn = vcpu->x64.ctrlreg[1] >> PAGE_SHIFT;
 
-        if ( pfn > ctx->x86_pv.max_pfn )
+        if ( pfn > ctx->x86.pv.max_pfn )
         {
             ERROR("cr1 (pfn %#lx) out of range", pfn);
             goto err;
         }
 
-        if ( (ctx->x86_pv.restore.pfn_types[pfn] &
+        if ( (ctx->x86.pv.restore.pfn_types[pfn] &
               XEN_DOMCTL_PFINFO_LTABTYPE_MASK) !=
-             (((xen_pfn_t)ctx->x86_pv.levels) << XEN_DOMCTL_PFINFO_LTAB_SHIFT) )
+             (((xen_pfn_t)ctx->x86.pv.levels) << XEN_DOMCTL_PFINFO_LTAB_SHIFT) )
         {
             ERROR("cr1 (pfn %#lx) has bad type %u, expected %u", pfn,
-                  (ctx->x86_pv.restore.pfn_types[pfn] >>
+                  (ctx->x86.pv.restore.pfn_types[pfn] >>
                    XEN_DOMCTL_PFINFO_LTAB_SHIFT),
-                  ctx->x86_pv.levels);
+                  ctx->x86.pv.levels);
             goto err;
         }
 
@@ -375,7 +375,7 @@ static int process_vcpu_extended(struct xc_sr_context *ctx,
 {
     xc_interface *xch = ctx->xch;
     struct xc_sr_x86_pv_restore_vcpu *vcpu =
-        &ctx->x86_pv.restore.vcpus[vcpuid];
+        &ctx->x86.pv.restore.vcpus[vcpuid];
     DECLARE_DOMCTL;
 
     domctl.cmd = XEN_DOMCTL_set_ext_vcpucontext;
@@ -399,7 +399,7 @@ static int process_vcpu_xsave(struct xc_sr_context *ctx,
 {
     xc_interface *xch = ctx->xch;
     struct xc_sr_x86_pv_restore_vcpu *vcpu =
-        &ctx->x86_pv.restore.vcpus[vcpuid];
+        &ctx->x86.pv.restore.vcpus[vcpuid];
     int rc;
     DECLARE_DOMCTL;
     DECLARE_HYPERCALL_BUFFER(void, buffer);
@@ -437,7 +437,7 @@ static int process_vcpu_msrs(struct xc_sr_context *ctx,
 {
     xc_interface *xch = ctx->xch;
     struct xc_sr_x86_pv_restore_vcpu *vcpu =
-        &ctx->x86_pv.restore.vcpus[vcpuid];
+        &ctx->x86.pv.restore.vcpus[vcpuid];
     int rc;
     DECLARE_DOMCTL;
     DECLARE_HYPERCALL_BUFFER(void, buffer);
@@ -477,9 +477,9 @@ static int update_vcpu_context(struct xc_sr_context *ctx)
     unsigned int i;
     int rc = 0;
 
-    for ( i = 0; i < ctx->x86_pv.restore.nr_vcpus; ++i )
+    for ( i = 0; i < ctx->x86.pv.restore.nr_vcpus; ++i )
     {
-        vcpu = &ctx->x86_pv.restore.vcpus[i];
+        vcpu = &ctx->x86.pv.restore.vcpus[i];
 
         if ( vcpu->basic.ptr )
         {
@@ -530,21 +530,21 @@ static int update_guest_p2m(struct xc_sr_context *ctx)
     unsigned int i;
     int rc = -1;
 
-    for ( i = 0; i < ctx->x86_pv.p2m_frames; ++i )
+    for ( i = 0; i < ctx->x86.pv.p2m_frames; ++i )
     {
-        pfn = ctx->x86_pv.p2m_pfns[i];
+        pfn = ctx->x86.pv.p2m_pfns[i];
 
-        if ( pfn > ctx->x86_pv.max_pfn )
+        if ( pfn > ctx->x86.pv.max_pfn )
         {
             ERROR("pfn (%#lx) for p2m_frame_list[%u] out of range",
                   pfn, i);
             goto err;
         }
 
-        if ( (ctx->x86_pv.restore.pfn_types[pfn] != XEN_DOMCTL_PFINFO_NOTAB) )
+        if ( (ctx->x86.pv.restore.pfn_types[pfn] != XEN_DOMCTL_PFINFO_NOTAB) )
         {
             ERROR("pfn (%#lx) for p2m_frame_list[%u] has bad type %u", pfn, i,
-                  (ctx->x86_pv.restore.pfn_types[pfn] >>
+                  (ctx->x86.pv.restore.pfn_types[pfn] >>
                    XEN_DOMCTL_PFINFO_LTAB_SHIFT));
             goto err;
         }
@@ -557,25 +557,25 @@ static int update_guest_p2m(struct xc_sr_context *ctx)
             goto err;
         }
 
-        ctx->x86_pv.p2m_pfns[i] = mfn;
+        ctx->x86.pv.p2m_pfns[i] = mfn;
     }
 
     guest_p2m = xc_map_foreign_pages(xch, ctx->domid, PROT_WRITE,
-                                     ctx->x86_pv.p2m_pfns,
-                                     ctx->x86_pv.p2m_frames);
+                                     ctx->x86.pv.p2m_pfns,
+                                     ctx->x86.pv.p2m_frames);
     if ( !guest_p2m )
     {
         PERROR("Failed to map p2m frames");
         goto err;
     }
 
-    memcpy(guest_p2m, ctx->x86_pv.p2m,
-           (ctx->x86_pv.max_pfn + 1) * ctx->x86_pv.width);
+    memcpy(guest_p2m, ctx->x86.pv.p2m,
+           (ctx->x86.pv.max_pfn + 1) * ctx->x86.pv.width);
     rc = 0;
 
  err:
     if ( guest_p2m )
-        munmap(guest_p2m, ctx->x86_pv.p2m_frames * PAGE_SIZE);
+        munmap(guest_p2m, ctx->x86.pv.p2m_frames * PAGE_SIZE);
 
     return rc;
 }
@@ -604,7 +604,7 @@ static int handle_x86_pv_info(struct xc_sr_context *ctx,
     xc_interface *xch = ctx->xch;
     struct xc_sr_rec_x86_pv_info *info = rec->data;
 
-    if ( ctx->x86_pv.restore.seen_pv_info )
+    if ( ctx->x86.pv.restore.seen_pv_info )
     {
         ERROR("Already received X86_PV_INFO record");
         return -1;
@@ -628,7 +628,7 @@ static int handle_x86_pv_info(struct xc_sr_context *ctx,
      * PV domains default to native width.  For an incomming compat domain, we
      * will typically be the first entity to inform Xen.
      */
-    if ( info->guest_width != ctx->x86_pv.width )
+    if ( info->guest_width != ctx->x86.pv.width )
     {
         struct xen_domctl domctl = {
             .domain = ctx->domid,
@@ -654,16 +654,16 @@ static int handle_x86_pv_info(struct xc_sr_context *ctx,
     }
 
     /* Sanity check (possibly new) domain settings. */
-    if ( (info->guest_width != ctx->x86_pv.width) ||
-         (info->pt_levels   != ctx->x86_pv.levels) )
+    if ( (info->guest_width != ctx->x86.pv.width) ||
+         (info->pt_levels   != ctx->x86.pv.levels) )
     {
         ERROR("X86_PV_INFO width/pt_levels settings %u/%u mismatch with d%d %u/%u",
               info->guest_width, info->pt_levels, ctx->domid,
-              ctx->x86_pv.width, ctx->x86_pv.levels);
+              ctx->x86.pv.width, ctx->x86.pv.levels);
         return -1;
     }
 
-    ctx->x86_pv.restore.seen_pv_info = true;
+    ctx->x86.pv.restore.seen_pv_info = true;
     return 0;
 }
 
@@ -676,10 +676,10 @@ static int handle_x86_pv_p2m_frames(struct xc_sr_context *ctx,
 {
     xc_interface *xch = ctx->xch;
     struct xc_sr_rec_x86_pv_p2m_frames *data = rec->data;
-    unsigned int start, end, x, fpp = PAGE_SIZE / ctx->x86_pv.width;
+    unsigned int start, end, x, fpp = PAGE_SIZE / ctx->x86.pv.width;
     int rc;
 
-    if ( !ctx->x86_pv.restore.seen_pv_info )
+    if ( !ctx->x86.pv.restore.seen_pv_info )
     {
         ERROR("Not yet received X86_PV_INFO record");
         return -1;
@@ -711,7 +711,7 @@ static int handle_x86_pv_p2m_frames(struct xc_sr_context *ctx,
         return -1;
     }
 
-    if ( data->end_pfn > ctx->x86_pv.max_pfn )
+    if ( data->end_pfn > ctx->x86.pv.max_pfn )
     {
         rc = expand_p2m(ctx, data->end_pfn);
         if ( rc )
@@ -719,7 +719,7 @@ static int handle_x86_pv_p2m_frames(struct xc_sr_context *ctx,
     }
 
     for ( x = 0; x < (end - start); ++x )
-        ctx->x86_pv.p2m_pfns[start + x] = data->p2m_pfns[x];
+        ctx->x86.pv.p2m_pfns[start + x] = data->p2m_pfns[x];
 
     return 0;
 }
@@ -788,21 +788,21 @@ static int handle_x86_pv_vcpu_blob(struct xc_sr_context *ctx,
     }
 
     /* Check that the vcpu id is within range. */
-    if ( vhdr->vcpu_id >= ctx->x86_pv.restore.nr_vcpus )
+    if ( vhdr->vcpu_id >= ctx->x86.pv.restore.nr_vcpus )
     {
         ERROR("%s record vcpu_id (%u) exceeds domain max (%u)",
-              rec_name, vhdr->vcpu_id, ctx->x86_pv.restore.nr_vcpus - 1);
+              rec_name, vhdr->vcpu_id, ctx->x86.pv.restore.nr_vcpus - 1);
         goto out;
     }
 
-    vcpu = &ctx->x86_pv.restore.vcpus[vhdr->vcpu_id];
+    vcpu = &ctx->x86.pv.restore.vcpus[vhdr->vcpu_id];
 
     /* Further per-record checks, where possible. */
     switch ( rec->type )
     {
     case REC_TYPE_X86_PV_VCPU_BASIC:
     {
-        size_t vcpusz = ctx->x86_pv.width == 8 ?
+        size_t vcpusz = ctx->x86.pv.width == 8 ?
             sizeof(vcpu_guest_context_x86_64_t) :
             sizeof(vcpu_guest_context_x86_32_t);
 
@@ -868,7 +868,7 @@ static int handle_shared_info(struct xc_sr_context *ctx,
     shared_info_any_t *guest_shinfo = NULL;
     const shared_info_any_t *old_shinfo = rec->data;
 
-    if ( !ctx->x86_pv.restore.seen_pv_info )
+    if ( !ctx->x86.pv.restore.seen_pv_info )
     {
         ERROR("Not yet received X86_PV_INFO record");
         return -1;
@@ -891,18 +891,18 @@ static int handle_shared_info(struct xc_sr_context *ctx,
         goto err;
     }
 
-    MEMCPY_FIELD(guest_shinfo, old_shinfo, vcpu_info, ctx->x86_pv.width);
-    MEMCPY_FIELD(guest_shinfo, old_shinfo, arch, ctx->x86_pv.width);
+    MEMCPY_FIELD(guest_shinfo, old_shinfo, vcpu_info, ctx->x86.pv.width);
+    MEMCPY_FIELD(guest_shinfo, old_shinfo, arch, ctx->x86.pv.width);
 
     SET_FIELD(guest_shinfo, arch.pfn_to_mfn_frame_list_list,
-              0, ctx->x86_pv.width);
+              0, ctx->x86.pv.width);
 
-    MEMSET_ARRAY_FIELD(guest_shinfo, evtchn_pending, 0, ctx->x86_pv.width);
+    MEMSET_ARRAY_FIELD(guest_shinfo, evtchn_pending, 0, ctx->x86.pv.width);
     for ( i = 0; i < XEN_LEGACY_MAX_VCPUS; i++ )
         SET_FIELD(guest_shinfo, vcpu_info[i].evtchn_pending_sel,
-                  0, ctx->x86_pv.width);
+                  0, ctx->x86.pv.width);
 
-    MEMSET_ARRAY_FIELD(guest_shinfo, evtchn_mask, 0xff, ctx->x86_pv.width);
+    MEMSET_ARRAY_FIELD(guest_shinfo, evtchn_mask, 0xff, ctx->x86.pv.width);
 
     rc = 0;
 
@@ -916,30 +916,30 @@ static int handle_shared_info(struct xc_sr_context *ctx,
 /* restore_ops function. */
 static bool x86_pv_pfn_is_valid(const struct xc_sr_context *ctx, xen_pfn_t pfn)
 {
-    return pfn <= ctx->x86_pv.max_pfn;
+    return pfn <= ctx->x86.pv.max_pfn;
 }
 
 /* restore_ops function. */
 static void x86_pv_set_page_type(struct xc_sr_context *ctx, xen_pfn_t pfn,
                                  unsigned long type)
 {
-    assert(pfn <= ctx->x86_pv.max_pfn);
+    assert(pfn <= ctx->x86.pv.max_pfn);
 
-    ctx->x86_pv.restore.pfn_types[pfn] = type;
+    ctx->x86.pv.restore.pfn_types[pfn] = type;
 }
 
 /* restore_ops function. */
 static void x86_pv_set_gfn(struct xc_sr_context *ctx, xen_pfn_t pfn,
                            xen_pfn_t mfn)
 {
-    assert(pfn <= ctx->x86_pv.max_pfn);
+    assert(pfn <= ctx->x86.pv.max_pfn);
 
-    if ( ctx->x86_pv.width == sizeof(uint64_t) )
+    if ( ctx->x86.pv.width == sizeof(uint64_t) )
         /* 64 bit guest.  Need to expand INVALID_MFN for 32 bit toolstacks. */
-        ((uint64_t *)ctx->x86_pv.p2m)[pfn] = mfn == INVALID_MFN ? ~0ULL : mfn;
+        ((uint64_t *)ctx->x86.pv.p2m)[pfn] = mfn == INVALID_MFN ? ~0ULL : mfn;
     else
         /* 32 bit guest.  Can truncate INVALID_MFN for 64 bit toolstacks. */
-        ((uint32_t *)ctx->x86_pv.p2m)[pfn] = mfn;
+        ((uint32_t *)ctx->x86.pv.p2m)[pfn] = mfn;
 }
 
 /*
@@ -1043,10 +1043,10 @@ static int x86_pv_setup(struct xc_sr_context *ctx)
     if ( rc )
         return rc;
 
-    ctx->x86_pv.restore.nr_vcpus = ctx->dominfo.max_vcpu_id + 1;
-    ctx->x86_pv.restore.vcpus = calloc(sizeof(struct xc_sr_x86_pv_restore_vcpu),
-                                       ctx->x86_pv.restore.nr_vcpus);
-    if ( !ctx->x86_pv.restore.vcpus )
+    ctx->x86.pv.restore.nr_vcpus = ctx->dominfo.max_vcpu_id + 1;
+    ctx->x86.pv.restore.vcpus = calloc(sizeof(struct xc_sr_x86_pv_restore_vcpu),
+                                       ctx->x86.pv.restore.nr_vcpus);
+    if ( !ctx->x86.pv.restore.vcpus )
     {
         errno = ENOMEM;
         return -1;
@@ -1130,17 +1130,17 @@ static int x86_pv_stream_complete(struct xc_sr_context *ctx)
  */
 static int x86_pv_cleanup(struct xc_sr_context *ctx)
 {
-    free(ctx->x86_pv.p2m);
-    free(ctx->x86_pv.p2m_pfns);
+    free(ctx->x86.pv.p2m);
+    free(ctx->x86.pv.p2m_pfns);
 
-    if ( ctx->x86_pv.restore.vcpus )
+    if ( ctx->x86.pv.restore.vcpus )
     {
         unsigned int i;
 
-        for ( i = 0; i < ctx->x86_pv.restore.nr_vcpus; ++i )
+        for ( i = 0; i < ctx->x86.pv.restore.nr_vcpus; ++i )
         {
             struct xc_sr_x86_pv_restore_vcpu *vcpu =
-                &ctx->x86_pv.restore.vcpus[i];
+                &ctx->x86.pv.restore.vcpus[i];
 
             free(vcpu->basic.ptr);
             free(vcpu->extd.ptr);
@@ -1148,13 +1148,13 @@ static int x86_pv_cleanup(struct xc_sr_context *ctx)
             free(vcpu->msr.ptr);
         }
 
-        free(ctx->x86_pv.restore.vcpus);
+        free(ctx->x86.pv.restore.vcpus);
     }
 
-    free(ctx->x86_pv.restore.pfn_types);
+    free(ctx->x86.pv.restore.pfn_types);
 
-    if ( ctx->x86_pv.m2p )
-        munmap(ctx->x86_pv.m2p, ctx->x86_pv.nr_m2p_frames * PAGE_SIZE);
+    if ( ctx->x86.pv.m2p )
+        munmap(ctx->x86.pv.m2p, ctx->x86.pv.nr_m2p_frames * PAGE_SIZE);
 
     return 0;
 }
diff --git a/tools/libxc/xc_sr_save_x86_hvm.c b/tools/libxc/xc_sr_save_x86_hvm.c
index d925a81999..58722118ae 100644
--- a/tools/libxc/xc_sr_save_x86_hvm.c
+++ b/tools/libxc/xc_sr_save_x86_hvm.c
@@ -165,7 +165,7 @@ static int x86_hvm_setup(struct xc_sr_context *ctx)
         return -1;
     }
 
-    ctx->x86_hvm.save.qemu_enabled_logdirty = true;
+    ctx->x86.hvm.save.qemu_enabled_logdirty = true;
 
     return 0;
 }
@@ -197,7 +197,7 @@ static int x86_hvm_cleanup(struct xc_sr_context *ctx)
     xc_interface *xch = ctx->xch;
 
     /* If qemu successfully enabled logdirty mode, attempt to disable. */
-    if ( ctx->x86_hvm.save.qemu_enabled_logdirty &&
+    if ( ctx->x86.hvm.save.qemu_enabled_logdirty &&
          ctx->save.callbacks->switch_qemu_logdirty(
              ctx->domid, 0, ctx->save.callbacks->data) )
     {
diff --git a/tools/libxc/xc_sr_save_x86_pv.c b/tools/libxc/xc_sr_save_x86_pv.c
index 94d0f68911..c1c6892666 100644
--- a/tools/libxc/xc_sr_save_x86_pv.c
+++ b/tools/libxc/xc_sr_save_x86_pv.c
@@ -16,9 +16,9 @@ static int map_shinfo(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
 
-    ctx->x86_pv.shinfo = xc_map_foreign_range(
+    ctx->x86.pv.shinfo = xc_map_foreign_range(
         xch, ctx->domid, PAGE_SIZE, PROT_READ, ctx->dominfo.shared_info_frame);
-    if ( !ctx->x86_pv.shinfo )
+    if ( !ctx->x86.pv.shinfo )
     {
         PERROR("Failed to map shared info frame at mfn %#lx",
                ctx->dominfo.shared_info_frame);
@@ -37,7 +37,7 @@ static int copy_mfns_from_guest(const struct xc_sr_context *ctx,
 {
     size_t x;
 
-    if ( ctx->x86_pv.width == sizeof(unsigned long) )
+    if ( ctx->x86.pv.width == sizeof(unsigned long) )
         memcpy(dst, src, count * sizeof(*dst));
     else
     {
@@ -82,18 +82,18 @@ static int map_p2m_leaves(struct xc_sr_context *ctx, xen_pfn_t *mfns,
     xc_interface *xch = ctx->xch;
     unsigned int x;
 
-    ctx->x86_pv.p2m = xc_map_foreign_pages(xch, ctx->domid, PROT_READ,
+    ctx->x86.pv.p2m = xc_map_foreign_pages(xch, ctx->domid, PROT_READ,
                                            mfns, n_mfns);
-    if ( !ctx->x86_pv.p2m )
+    if ( !ctx->x86.pv.p2m )
     {
         PERROR("Failed to map p2m frames");
         return -1;
     }
 
-    ctx->save.p2m_size = ctx->x86_pv.max_pfn + 1;
-    ctx->x86_pv.p2m_frames = n_mfns;
-    ctx->x86_pv.p2m_pfns = malloc(n_mfns * sizeof(*mfns));
-    if ( !ctx->x86_pv.p2m_pfns )
+    ctx->save.p2m_size = ctx->x86.pv.max_pfn + 1;
+    ctx->x86.pv.p2m_frames = n_mfns;
+    ctx->x86.pv.p2m_pfns = malloc(n_mfns * sizeof(*mfns));
+    if ( !ctx->x86.pv.p2m_pfns )
     {
         ERROR("Cannot allocate %zu bytes for p2m pfns list",
               n_mfns * sizeof(*mfns));
@@ -111,7 +111,7 @@ static int map_p2m_leaves(struct xc_sr_context *ctx, xen_pfn_t *mfns,
             return -1;
         }
 
-        ctx->x86_pv.p2m_pfns[x] = mfn_to_pfn(ctx, mfns[x]);
+        ctx->x86.pv.p2m_pfns[x] = mfn_to_pfn(ctx, mfns[x]);
     }
 
     return 0;
@@ -144,17 +144,17 @@ static int map_p2m_tree(struct xc_sr_context *ctx)
     void *guest_fl = NULL;
     size_t local_fl_size;
 
-    fpp = PAGE_SIZE / ctx->x86_pv.width;
-    fll_entries = (ctx->x86_pv.max_pfn / (fpp * fpp)) + 1;
+    fpp = PAGE_SIZE / ctx->x86.pv.width;
+    fll_entries = (ctx->x86.pv.max_pfn / (fpp * fpp)) + 1;
     if ( fll_entries > fpp )
     {
-        ERROR("max_pfn %#lx too large for p2m tree", ctx->x86_pv.max_pfn);
+        ERROR("max_pfn %#lx too large for p2m tree", ctx->x86.pv.max_pfn);
         goto err;
     }
 
-    fll_mfn = GET_FIELD(ctx->x86_pv.shinfo, arch.pfn_to_mfn_frame_list_list,
-                        ctx->x86_pv.width);
-    if ( fll_mfn == 0 || fll_mfn > ctx->x86_pv.max_mfn )
+    fll_mfn = GET_FIELD(ctx->x86.pv.shinfo, arch.pfn_to_mfn_frame_list_list,
+                        ctx->x86.pv.width);
+    if ( fll_mfn == 0 || fll_mfn > ctx->x86.pv.max_mfn )
     {
         ERROR("Bad mfn %#lx for p2m frame list list", fll_mfn);
         goto err;
@@ -189,7 +189,7 @@ static int map_p2m_tree(struct xc_sr_context *ctx)
     saved_x = 0;
     for ( x = 0; x < fll_entries; ++x )
     {
-        if ( local_fll[x] == 0 || local_fll[x] > ctx->x86_pv.max_mfn )
+        if ( local_fll[x] == 0 || local_fll[x] > ctx->x86.pv.max_mfn )
         {
             ERROR("Bad mfn %#lx at index %u (of %u) in p2m frame list list",
                   local_fll[x], x, fll_entries);
@@ -213,15 +213,15 @@ static int map_p2m_tree(struct xc_sr_context *ctx)
      * needed for p2m and logdirty map.
      */
     max_pfn = (saved_x + 1) * fpp * fpp - 1;
-    if ( max_pfn < ctx->x86_pv.max_pfn )
+    if ( max_pfn < ctx->x86.pv.max_pfn )
     {
-        ctx->x86_pv.max_pfn = max_pfn;
-        fll_entries = (ctx->x86_pv.max_pfn / (fpp * fpp)) + 1;
+        ctx->x86.pv.max_pfn = max_pfn;
+        fll_entries = (ctx->x86.pv.max_pfn / (fpp * fpp)) + 1;
     }
-    ctx->x86_pv.p2m_frames = (ctx->x86_pv.max_pfn + fpp) / fpp;
-    DPRINTF("max_pfn %#lx, p2m_frames %d", ctx->x86_pv.max_pfn,
-            ctx->x86_pv.p2m_frames);
-    fl_entries  = (ctx->x86_pv.max_pfn / fpp) + 1;
+    ctx->x86.pv.p2m_frames = (ctx->x86.pv.max_pfn + fpp) / fpp;
+    DPRINTF("max_pfn %#lx, p2m_frames %d", ctx->x86.pv.max_pfn,
+            ctx->x86.pv.p2m_frames);
+    fl_entries  = (ctx->x86.pv.max_pfn / fpp) + 1;
 
     /* Map the guest mid p2m frames. */
     guest_fl = xc_map_foreign_pages(xch, ctx->domid, PROT_READ,
@@ -249,7 +249,7 @@ static int map_p2m_tree(struct xc_sr_context *ctx)
 
     for ( x = 0; x < fl_entries; ++x )
     {
-        if ( local_fl[x] == 0 || local_fl[x] > ctx->x86_pv.max_mfn )
+        if ( local_fl[x] == 0 || local_fl[x] > ctx->x86.pv.max_mfn )
         {
             ERROR("Bad mfn %#lx at index %u (of %u) in p2m frame list",
                   local_fl[x], x, fl_entries);
@@ -281,11 +281,11 @@ static int get_p2m_generation(struct xc_sr_context *ctx)
     uint64_t p2m_generation;
     int rc;
 
-    p2m_generation = GET_FIELD(ctx->x86_pv.shinfo, arch.p2m_generation,
-                               ctx->x86_pv.width);
+    p2m_generation = GET_FIELD(ctx->x86.pv.shinfo, arch.p2m_generation,
+                               ctx->x86.pv.width);
 
-    rc = (p2m_generation == ctx->x86_pv.p2m_generation) ? 0 : -1;
-    ctx->x86_pv.p2m_generation = p2m_generation;
+    rc = (p2m_generation == ctx->x86.pv.p2m_generation) ? 0 : -1;
+    ctx->x86.pv.p2m_generation = p2m_generation;
 
     return rc;
 }
@@ -322,7 +322,7 @@ static int map_p2m_list(struct xc_sr_context *ctx, uint64_t p2m_cr3)
 
     p2m_mfn = cr3_to_mfn(ctx, p2m_cr3);
     assert(p2m_mfn != 0);
-    if ( p2m_mfn > ctx->x86_pv.max_mfn )
+    if ( p2m_mfn > ctx->x86.pv.max_mfn )
     {
         ERROR("Bad p2m_cr3 value %#" PRIx64, p2m_cr3);
         errno = ERANGE;
@@ -331,13 +331,13 @@ static int map_p2m_list(struct xc_sr_context *ctx, uint64_t p2m_cr3)
 
     get_p2m_generation(ctx);
 
-    p2m_vaddr = GET_FIELD(ctx->x86_pv.shinfo, arch.p2m_vaddr,
-                          ctx->x86_pv.width);
-    fpp = PAGE_SIZE / ctx->x86_pv.width;
-    ctx->x86_pv.p2m_frames = ctx->x86_pv.max_pfn / fpp + 1;
-    p2m_end = p2m_vaddr + ctx->x86_pv.p2m_frames * PAGE_SIZE - 1;
+    p2m_vaddr = GET_FIELD(ctx->x86.pv.shinfo, arch.p2m_vaddr,
+                          ctx->x86.pv.width);
+    fpp = PAGE_SIZE / ctx->x86.pv.width;
+    ctx->x86.pv.p2m_frames = ctx->x86.pv.max_pfn / fpp + 1;
+    p2m_end = p2m_vaddr + ctx->x86.pv.p2m_frames * PAGE_SIZE - 1;
 
-    if ( ctx->x86_pv.width == 8 )
+    if ( ctx->x86.pv.width == 8 )
     {
         mask = 0x0000ffffffffffffULL;
         if ( !is_canonical_address(p2m_vaddr) ||
@@ -368,8 +368,8 @@ static int map_p2m_list(struct xc_sr_context *ctx, uint64_t p2m_cr3)
 
     DPRINTF("p2m list from %#" PRIx64 " to %#" PRIx64 ", root at %#lx",
             p2m_vaddr, p2m_end, p2m_mfn);
-    DPRINTF("max_pfn %#lx, p2m_frames %d", ctx->x86_pv.max_pfn,
-            ctx->x86_pv.p2m_frames);
+    DPRINTF("max_pfn %#lx, p2m_frames %d", ctx->x86.pv.max_pfn,
+            ctx->x86.pv.p2m_frames);
 
     mfns = malloc(sizeof(*mfns));
     if ( !mfns )
@@ -382,7 +382,7 @@ static int map_p2m_list(struct xc_sr_context *ctx, uint64_t p2m_cr3)
     saved_mfn = 0;
     idx_start = idx_end = saved_idx = 0;
 
-    for ( level = ctx->x86_pv.levels; level > 0; level-- )
+    for ( level = ctx->x86.pv.levels; level > 0; level-- )
     {
         n_pages = idx_end - idx_start + 1;
         ptes = xc_map_foreign_pages(xch, ctx->domid, PROT_READ, mfns, n_pages);
@@ -407,7 +407,7 @@ static int map_p2m_list(struct xc_sr_context *ctx, uint64_t p2m_cr3)
         for ( idx = idx_start; idx <= idx_end; idx++ )
         {
             mfn = pte_to_frame(ptes[idx]);
-            if ( mfn == 0 || mfn > ctx->x86_pv.max_mfn )
+            if ( mfn == 0 || mfn > ctx->x86.pv.max_mfn )
             {
                 ERROR("Bad mfn %#lx during page table walk for vaddr %#" PRIx64 " at level %d of p2m list",
                       mfn, off + ((xen_vaddr_t)idx << shift), level);
@@ -432,11 +432,11 @@ static int map_p2m_list(struct xc_sr_context *ctx, uint64_t p2m_cr3)
             if ( saved_idx == idx_end )
                 saved_idx++;
             max_pfn = ((xen_pfn_t)saved_idx << 9) * fpp - 1;
-            if ( max_pfn < ctx->x86_pv.max_pfn )
+            if ( max_pfn < ctx->x86.pv.max_pfn )
             {
-                ctx->x86_pv.max_pfn = max_pfn;
-                ctx->x86_pv.p2m_frames = (ctx->x86_pv.max_pfn + fpp) / fpp;
-                p2m_end = p2m_vaddr + ctx->x86_pv.p2m_frames * PAGE_SIZE - 1;
+                ctx->x86.pv.max_pfn = max_pfn;
+                ctx->x86.pv.p2m_frames = (ctx->x86.pv.max_pfn + fpp) / fpp;
+                p2m_end = p2m_vaddr + ctx->x86.pv.p2m_frames * PAGE_SIZE - 1;
                 idx_end = idx_start + saved_idx;
             }
         }
@@ -466,10 +466,10 @@ static int map_p2m(struct xc_sr_context *ctx)
 {
     uint64_t p2m_cr3;
 
-    ctx->x86_pv.p2m_generation = ~0ULL;
-    ctx->x86_pv.max_pfn = GET_FIELD(ctx->x86_pv.shinfo, arch.max_pfn,
-                                    ctx->x86_pv.width) - 1;
-    p2m_cr3 = GET_FIELD(ctx->x86_pv.shinfo, arch.p2m_cr3, ctx->x86_pv.width);
+    ctx->x86.pv.p2m_generation = ~0ULL;
+    ctx->x86.pv.max_pfn = GET_FIELD(ctx->x86.pv.shinfo, arch.max_pfn,
+                                    ctx->x86.pv.width) - 1;
+    p2m_cr3 = GET_FIELD(ctx->x86.pv.shinfo, arch.p2m_cr3, ctx->x86.pv.width);
 
     return p2m_cr3 ? map_p2m_list(ctx, p2m_cr3) : map_p2m_tree(ctx);
 }
@@ -503,7 +503,7 @@ static int write_one_vcpu_basic(struct xc_sr_context *ctx, uint32_t id)
     /* Vcpu0 is special: Convert the suspend record to a pfn. */
     if ( id == 0 )
     {
-        mfn = GET_FIELD(&vcpu, user_regs.edx, ctx->x86_pv.width);
+        mfn = GET_FIELD(&vcpu, user_regs.edx, ctx->x86.pv.width);
         if ( !mfn_in_pseudophysmap(ctx, mfn) )
         {
             ERROR("Bad mfn for suspend record");
@@ -512,10 +512,10 @@ static int write_one_vcpu_basic(struct xc_sr_context *ctx, uint32_t id)
             goto err;
         }
         SET_FIELD(&vcpu, user_regs.edx, mfn_to_pfn(ctx, mfn),
-                  ctx->x86_pv.width);
+                  ctx->x86.pv.width);
     }
 
-    gdt_count = GET_FIELD(&vcpu, gdt_ents, ctx->x86_pv.width);
+    gdt_count = GET_FIELD(&vcpu, gdt_ents, ctx->x86.pv.width);
     if ( gdt_count > FIRST_RESERVED_GDT_ENTRY )
     {
         ERROR("GDT entry count (%u) out of range (max %u)",
@@ -528,7 +528,7 @@ static int write_one_vcpu_basic(struct xc_sr_context *ctx, uint32_t id)
     /* Convert GDT frames to pfns. */
     for ( i = 0; i < gdt_count; ++i )
     {
-        mfn = GET_FIELD(&vcpu, gdt_frames[i], ctx->x86_pv.width);
+        mfn = GET_FIELD(&vcpu, gdt_frames[i], ctx->x86.pv.width);
         if ( !mfn_in_pseudophysmap(ctx, mfn) )
         {
             ERROR("Bad mfn for frame %u of vcpu%u's GDT", i, id);
@@ -537,11 +537,11 @@ static int write_one_vcpu_basic(struct xc_sr_context *ctx, uint32_t id)
             goto err;
         }
         SET_FIELD(&vcpu, gdt_frames[i], mfn_to_pfn(ctx, mfn),
-                  ctx->x86_pv.width);
+                  ctx->x86.pv.width);
     }
 
     /* Convert CR3 to a pfn. */
-    mfn = cr3_to_mfn(ctx, GET_FIELD(&vcpu, ctrlreg[3], ctx->x86_pv.width));
+    mfn = cr3_to_mfn(ctx, GET_FIELD(&vcpu, ctrlreg[3], ctx->x86.pv.width));
     if ( !mfn_in_pseudophysmap(ctx, mfn) )
     {
         ERROR("Bad mfn for vcpu%u's cr3", id);
@@ -550,10 +550,10 @@ static int write_one_vcpu_basic(struct xc_sr_context *ctx, uint32_t id)
         goto err;
     }
     pfn = mfn_to_pfn(ctx, mfn);
-    SET_FIELD(&vcpu, ctrlreg[3], mfn_to_cr3(ctx, pfn), ctx->x86_pv.width);
+    SET_FIELD(&vcpu, ctrlreg[3], mfn_to_cr3(ctx, pfn), ctx->x86.pv.width);
 
     /* 64bit guests: Convert CR1 (guest pagetables) to pfn. */
-    if ( ctx->x86_pv.levels == 4 && vcpu.x64.ctrlreg[1] )
+    if ( ctx->x86.pv.levels == 4 && vcpu.x64.ctrlreg[1] )
     {
         mfn = vcpu.x64.ctrlreg[1] >> PAGE_SHIFT;
         if ( !mfn_in_pseudophysmap(ctx, mfn) )
@@ -567,7 +567,7 @@ static int write_one_vcpu_basic(struct xc_sr_context *ctx, uint32_t id)
         vcpu.x64.ctrlreg[1] = 1 | ((uint64_t)pfn << PAGE_SHIFT);
     }
 
-    if ( ctx->x86_pv.width == 8 )
+    if ( ctx->x86.pv.width == 8 )
         rc = write_split_record(ctx, &rec, &vcpu, sizeof(vcpu.x64));
     else
         rc = write_split_record(ctx, &rec, &vcpu, sizeof(vcpu.x32));
@@ -785,8 +785,8 @@ static int write_all_vcpu_information(struct xc_sr_context *ctx)
 static int write_x86_pv_info(struct xc_sr_context *ctx)
 {
     struct xc_sr_rec_x86_pv_info info = {
-        .guest_width = ctx->x86_pv.width,
-        .pt_levels = ctx->x86_pv.levels,
+        .guest_width = ctx->x86.pv.width,
+        .pt_levels = ctx->x86.pv.levels,
     };
     struct xc_sr_record rec = {
         .type = REC_TYPE_X86_PV_INFO,
@@ -805,10 +805,10 @@ static int write_x86_pv_p2m_frames(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
     int rc; unsigned int i;
-    size_t datasz = ctx->x86_pv.p2m_frames * sizeof(uint64_t);
+    size_t datasz = ctx->x86.pv.p2m_frames * sizeof(uint64_t);
     uint64_t *data = NULL;
     struct xc_sr_rec_x86_pv_p2m_frames hdr = {
-        .end_pfn = ctx->x86_pv.max_pfn,
+        .end_pfn = ctx->x86.pv.max_pfn,
     };
     struct xc_sr_record rec = {
         .type = REC_TYPE_X86_PV_P2M_FRAMES,
@@ -817,7 +817,7 @@ static int write_x86_pv_p2m_frames(struct xc_sr_context *ctx)
     };
 
     /* No need to translate if sizeof(uint64_t) == sizeof(xen_pfn_t). */
-    if ( sizeof(uint64_t) != sizeof(*ctx->x86_pv.p2m_pfns) )
+    if ( sizeof(uint64_t) != sizeof(*ctx->x86.pv.p2m_pfns) )
     {
         if ( !(data = malloc(datasz)) )
         {
@@ -826,15 +826,15 @@ static int write_x86_pv_p2m_frames(struct xc_sr_context *ctx)
             return -1;
         }
 
-        for ( i = 0; i < ctx->x86_pv.p2m_frames; ++i )
-            data[i] = ctx->x86_pv.p2m_pfns[i];
+        for ( i = 0; i < ctx->x86.pv.p2m_frames; ++i )
+            data[i] = ctx->x86.pv.p2m_pfns[i];
     }
     else
-        data = (uint64_t *)ctx->x86_pv.p2m_pfns;
+        data = (uint64_t *)ctx->x86.pv.p2m_pfns;
 
     rc = write_split_record(ctx, &rec, data, datasz);
 
-    if ( data != (uint64_t *)ctx->x86_pv.p2m_pfns )
+    if ( data != (uint64_t *)ctx->x86.pv.p2m_pfns )
         free(data);
 
     return rc;
@@ -848,7 +848,7 @@ static int write_shared_info(struct xc_sr_context *ctx)
     struct xc_sr_record rec = {
         .type = REC_TYPE_SHARED_INFO,
         .length = PAGE_SIZE,
-        .data = ctx->x86_pv.shinfo,
+        .data = ctx->x86.pv.shinfo,
     };
 
     return write_record(ctx, &rec);
@@ -867,7 +867,7 @@ static int normalise_pagetable(struct xc_sr_context *ctx, const uint64_t *src,
 
     type &= XEN_DOMCTL_PFINFO_LTABTYPE_MASK;
 
-    if ( ctx->x86_pv.levels == 4 )
+    if ( ctx->x86.pv.levels == 4 )
     {
         /* 64bit guests only have Xen mappings in their L4 tables. */
         if ( type == XEN_DOMCTL_PFINFO_L4TAB )
@@ -899,7 +899,7 @@ static int normalise_pagetable(struct xc_sr_context *ctx, const uint64_t *src,
              * are normal but only a few will have Xen mappings.
              */
             i = (HYPERVISOR_VIRT_START_X86_32 >> L2_PAGETABLE_SHIFT_PAE) & 511;
-            if ( pte_to_frame(src[i]) == ctx->x86_pv.compat_m2p_mfn0 )
+            if ( pte_to_frame(src[i]) == ctx->x86.pv.compat_m2p_mfn0 )
             {
                 xen_first = i;
                 xen_last = (HYPERVISOR_VIRT_END_X86_32 >>
@@ -980,9 +980,9 @@ static int normalise_pagetable(struct xc_sr_context *ctx, const uint64_t *src,
 static xen_pfn_t x86_pv_pfn_to_gfn(const struct xc_sr_context *ctx,
                                    xen_pfn_t pfn)
 {
-    assert(pfn <= ctx->x86_pv.max_pfn);
+    assert(pfn <= ctx->x86.pv.max_pfn);
 
-    return xc_pfn_to_mfn(pfn, ctx->x86_pv.p2m, ctx->x86_pv.width);
+    return xc_pfn_to_mfn(pfn, ctx->x86.pv.p2m, ctx->x86.pv.width);
 }
 
 
@@ -1063,7 +1063,7 @@ static int x86_pv_end_of_checkpoint(struct xc_sr_context *ctx)
 
 static int x86_pv_check_vm_state(struct xc_sr_context *ctx)
 {
-    if ( ctx->x86_pv.p2m_generation == ~0ULL )
+    if ( ctx->x86.pv.p2m_generation == ~0ULL )
         return 0;
 
     return x86_pv_check_vm_state_p2m_list(ctx);
@@ -1071,16 +1071,16 @@ static int x86_pv_check_vm_state(struct xc_sr_context *ctx)
 
 static int x86_pv_cleanup(struct xc_sr_context *ctx)
 {
-    free(ctx->x86_pv.p2m_pfns);
+    free(ctx->x86.pv.p2m_pfns);
 
-    if ( ctx->x86_pv.p2m )
-        munmap(ctx->x86_pv.p2m, ctx->x86_pv.p2m_frames * PAGE_SIZE);
+    if ( ctx->x86.pv.p2m )
+        munmap(ctx->x86.pv.p2m, ctx->x86.pv.p2m_frames * PAGE_SIZE);
 
-    if ( ctx->x86_pv.shinfo )
-        munmap(ctx->x86_pv.shinfo, PAGE_SIZE);
+    if ( ctx->x86.pv.shinfo )
+        munmap(ctx->x86.pv.shinfo, PAGE_SIZE);
 
-    if ( ctx->x86_pv.m2p )
-        munmap(ctx->x86_pv.m2p, ctx->x86_pv.nr_m2p_frames * PAGE_SIZE);
+    if ( ctx->x86.pv.m2p )
+        munmap(ctx->x86.pv.m2p, ctx->x86.pv.nr_m2p_frames * PAGE_SIZE);
 
     return 0;
 }
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 05/12] tools/migration: Drop IHDR_VERSION constant from libxc and python
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
                   ` (3 preceding siblings ...)
  2019-12-24 15:19 ` [Xen-devel] [PATCH 04/12] libxc/migration: Adjust layout of struct xc_sr_context Andrew Cooper
@ 2019-12-24 15:19 ` Andrew Cooper
  2020-01-14 16:05   ` Ian Jackson
  2019-12-24 15:19 ` [Xen-devel] [PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END Andrew Cooper
                   ` (8 subsequent siblings)
  13 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel
  Cc: Andrew Cooper, Marek Marczykowski-Górecki, Wei Liu, Ian Jackson

Migration v3 is in the process of being introduced, meaning that the code has
to cope with both versions.  Use an explicit 2 for now.

For the verify-stream-v2 and convert-legacy-stream scripts, update text to say
"v2 (or later)".  What matters is the distinction vs legacy streams.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/libxc/xc_sr_restore.c                | 6 +++---
 tools/libxc/xc_sr_save.c                   | 2 +-
 tools/libxc/xc_sr_stream_format.h          | 1 -
 tools/python/scripts/convert-legacy-stream | 6 +++---
 tools/python/scripts/verify-stream-v2      | 2 +-
 tools/python/xen/migration/libxc.py        | 9 ++++-----
 6 files changed, 12 insertions(+), 14 deletions(-)

diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c
index 7872b71ab5..0280e55d4b 100644
--- a/tools/libxc/xc_sr_restore.c
+++ b/tools/libxc/xc_sr_restore.c
@@ -35,10 +35,10 @@ static int read_headers(struct xc_sr_context *ctx)
         return -1;
     }
 
-    if ( ihdr.version != IHDR_VERSION )
+    if ( ihdr.version != 2 )
     {
-        ERROR("Invalid Version: Expected %d, Got %d",
-              IHDR_VERSION, ihdr.version);
+        ERROR("Invalid Version: Expected 2, Got %d",
+              ihdr.version);
         return -1;
     }
 
diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
index 5467965b08..fa1a2e6c2a 100644
--- a/tools/libxc/xc_sr_save.c
+++ b/tools/libxc/xc_sr_save.c
@@ -13,7 +13,7 @@ static int write_headers(struct xc_sr_context *ctx, uint16_t guest_type)
     struct xc_sr_ihdr ihdr = {
         .marker  = IHDR_MARKER,
         .id      = htonl(IHDR_ID),
-        .version = htonl(IHDR_VERSION),
+        .version = htonl(2),
         .options = htons(IHDR_OPT_LITTLE_ENDIAN),
     };
     struct xc_sr_dhdr dhdr = {
diff --git a/tools/libxc/xc_sr_stream_format.h b/tools/libxc/xc_sr_stream_format.h
index 37a7da6eab..ae7c0de393 100644
--- a/tools/libxc/xc_sr_stream_format.h
+++ b/tools/libxc/xc_sr_stream_format.h
@@ -23,7 +23,6 @@ struct xc_sr_ihdr
 
 #define IHDR_MARKER  0xffffffffffffffffULL
 #define IHDR_ID      0x58454E46U
-#define IHDR_VERSION 2
 
 #define _IHDR_OPT_ENDIAN 0
 #define IHDR_OPT_LITTLE_ENDIAN (0 << _IHDR_OPT_ENDIAN)
diff --git a/tools/python/scripts/convert-legacy-stream b/tools/python/scripts/convert-legacy-stream
index 2922fb3185..02a194178f 100755
--- a/tools/python/scripts/convert-legacy-stream
+++ b/tools/python/scripts/convert-legacy-stream
@@ -79,7 +79,7 @@ def write_libxc_ihdr():
     stream_write(pack(libxc.IHDR_FORMAT,
                       libxc.IHDR_MARKER,  # Marker
                       libxc.IHDR_IDENT,   # Ident
-                      libxc.IHDR_VERSION, # Version
+                      2,                  # Version
                       libxc.IHDR_OPT_LE,  # Options
                       0, 0))              # Reserved
 
@@ -632,13 +632,13 @@ def main():
                           usage = ("%prog [options] -i INPUT -o OUTPUT"
                                    " -w WIDTH -g GUEST"),
                           description =
-                          "Convert a legacy stream to a v2 stream")
+                          "Convert a legacy stream to a v2 (or later) stream")
 
     # Required options
     parser.add_option("-i", "--in", dest = "fin", metavar = "<FD or FILE>",
                       help = "Legacy input to convert")
     parser.add_option("-o", "--out", dest = "fout", metavar = "<FD or FILE>",
-                      help = "v2 format output")
+                      help = "v2 (or later) format output")
     parser.add_option("-w", "--width", dest = "twidth",
                       metavar = "<32/64>", choices = ["32", "64"],
                       help = "Legacy toolstack bitness")
diff --git a/tools/python/scripts/verify-stream-v2 b/tools/python/scripts/verify-stream-v2
index 8bac04d566..fe82b86c11 100755
--- a/tools/python/scripts/verify-stream-v2
+++ b/tools/python/scripts/verify-stream-v2
@@ -108,7 +108,7 @@ def main():
 
     parser = OptionParser(usage = "%prog [options]",
                           description =
-                          "Verify a stream according to the v2 spec")
+                          "Verify a stream according to the v2 (or later) spec")
 
     # Optional options
     parser.add_option("-i", "--in", dest = "fin", metavar = "<FD or FILE>",
diff --git a/tools/python/xen/migration/libxc.py b/tools/python/xen/migration/libxc.py
index 8a800df980..63b3558029 100644
--- a/tools/python/xen/migration/libxc.py
+++ b/tools/python/xen/migration/libxc.py
@@ -19,7 +19,6 @@
 
 IHDR_MARKER  = 0xffffffffffffffff
 IHDR_IDENT   = 0x58454E46 # "XENF" in ASCII
-IHDR_VERSION = 2
 
 IHDR_OPT_BIT_ENDIAN = 0
 IHDR_OPT_LE = (0 << IHDR_OPT_BIT_ENDIAN)
@@ -113,7 +112,7 @@
 HVM_PARAMS_FORMAT         = "II"
 
 class VerifyLibxc(VerifyBase):
-    """ Verify a Libxc v2 stream """
+    """ Verify a Libxc v2 (or later) stream """
 
     def __init__(self, info, read):
         VerifyBase.__init__(self, info, read)
@@ -144,9 +143,9 @@ def verify_ihdr(self):
             raise StreamError("Bad image id: Expected 0x%x, got 0x%x" %
                               (IHDR_IDENT, ident))
 
-        if version != IHDR_VERSION:
-            raise StreamError("Unknown image version: Expected %d, got %d" %
-                              (IHDR_VERSION, version))
+        if version != 2:
+            raise StreamError("Unknown image version: Expected 2, got %d" %
+                              (version, ))
 
         if options & IHDR_OPT_RESZ_MASK:
             raise StreamError("Reserved bits set in image options field: 0x%x" %
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
                   ` (4 preceding siblings ...)
  2019-12-24 15:19 ` [Xen-devel] [PATCH 05/12] tools/migration: Drop IHDR_VERSION constant from libxc and python Andrew Cooper
@ 2019-12-24 15:19 ` Andrew Cooper
  2020-01-03 14:44   ` Jan Beulich
  2020-01-14 16:07   ` Ian Jackson
  2019-12-24 15:19 ` [Xen-devel] [PATCH 07/12] python/migration: Update validation logic to understand a v3 stream Andrew Cooper
                   ` (7 subsequent siblings)
  13 siblings, 2 replies; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Marek Marczykowski-Górecki,
	Jan Beulich, Ian Jackson

Migration data can be split into two parts - that which is invariant of
guest execution, and that which is not.  Separate these two with the
STATIC_DATA_END record.

The short term, we want to move the x86 CPU Policy data into the stream.
In the longer term, we want to provisionally send the static data only
to the destination as a more robust compatibility check.  In both cases,
we will want a callback into the higher level toolstack.

Mandate the presence of the STATIC_DATA_END record, and declare this v3,
along with instructions for how to compatibly interpret a v2 stream.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien@xen.org>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 docs/specs/libxc-migration-stream.pandoc | 39 +++++++++++++++++++++++++++++---
 tools/libxc/xc_sr_common.c               |  1 +
 tools/libxc/xc_sr_stream_format.h        |  1 +
 tools/python/xen/migration/libxc.py      |  2 ++
 4 files changed, 40 insertions(+), 3 deletions(-)

diff --git a/docs/specs/libxc-migration-stream.pandoc b/docs/specs/libxc-migration-stream.pandoc
index a7a8a08936..22ff306e0b 100644
--- a/docs/specs/libxc-migration-stream.pandoc
+++ b/docs/specs/libxc-migration-stream.pandoc
@@ -127,7 +127,7 @@ marker      0xFFFFFFFFFFFFFFFF.
 
 id          0x58454E46 ("XENF" in ASCII).
 
-version     0x00000002.  The version of this specification.
+version     0x00000003.  The version of this specification.
 
 options     bit 0: Endianness.  0 = little-endian, 1 = big-endian.
 
@@ -620,6 +620,21 @@ The count of pfns is: record->length/sizeof(uint64_t).
 
 \clearpage
 
+STATIC_DATA_END
+---------------
+
+A static data end record marks the end of the static state.  I.e. state which
+is invariant of guest execution.
+
+
+     0     1     2     3     4     5     6     7 octet
+    +-------------------------------------------------+
+
+The end record contains no fields; its body_length is 0.
+
+\clearpage
+
+
 Layout
 ======
 
@@ -639,7 +654,9 @@ A typical save record for an x86 PV guest image would look like:
 
 * Image header
 * Domain header
-* X86_PV_INFO record
+* Static data records:
+    * X86_PV_INFO record
+    * STATIC_DATA_END
 * X86_PV_P2M_FRAMES record
 * Many PAGE_DATA records
 * X86_TSC_INFO
@@ -667,6 +684,8 @@ A typical save record for an x86 HVM guest image would look like:
 
 * Image header
 * Domain header
+* Static data records:
+    * STATIC_DATA_END
 * Many PAGE_DATA records
 * X86_TSC_INFO
 * HVM_PARAMS
@@ -675,9 +694,23 @@ A typical save record for an x86 HVM guest image would look like:
 HVM_PARAMS must precede HVM_CONTEXT, as certain parameters can affect
 the validity of architectural state in the context.
 
+Compatibility with older versions
+=================================
+
+v3 compat with v2
+-----------------
+
+A v3 stream is compatible with a v2 stream, but mandates the presense of a
+STATIC_DATA_END record ahead of any memory/register content.  This is to ease
+the introduction of new static configuration records over time.
+
+A v3-compatible reciever interpreting a v2 stream should infer the position of
+STATIC_DATA_END based on finding the first X86_PV_P2M_FRAMES record (for PV
+guests), or PAGE_DATA record (for HVM guests) and behave as if STATIC_DATA_END
+had been sent.
 
 Legacy Images (x86 only)
-========================
+------------------------
 
 Restoring legacy images from older tools shall be handled by
 translating the legacy format image into this new format.
diff --git a/tools/libxc/xc_sr_common.c b/tools/libxc/xc_sr_common.c
index dd9a11b4b5..7f22cf0365 100644
--- a/tools/libxc/xc_sr_common.c
+++ b/tools/libxc/xc_sr_common.c
@@ -36,6 +36,7 @@ static const char *const mandatory_rec_types[] =
     [REC_TYPE_VERIFY]                       = "Verify",
     [REC_TYPE_CHECKPOINT]                   = "Checkpoint",
     [REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST]    = "Checkpoint dirty pfn list",
+    [REC_TYPE_STATIC_DATA_END]              = "Static data end",
 };
 
 const char *rec_type_to_str(uint32_t type)
diff --git a/tools/libxc/xc_sr_stream_format.h b/tools/libxc/xc_sr_stream_format.h
index ae7c0de393..81c9765b0a 100644
--- a/tools/libxc/xc_sr_stream_format.h
+++ b/tools/libxc/xc_sr_stream_format.h
@@ -73,6 +73,7 @@ struct xc_sr_rhdr
 #define REC_TYPE_VERIFY                     0x0000000dU
 #define REC_TYPE_CHECKPOINT                 0x0000000eU
 #define REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST  0x0000000fU
+#define REC_TYPE_STATIC_DATA_END            0x00000010U
 
 #define REC_TYPE_OPTIONAL             0x80000000U
 
diff --git a/tools/python/xen/migration/libxc.py b/tools/python/xen/migration/libxc.py
index 63b3558029..d0c4f3527d 100644
--- a/tools/python/xen/migration/libxc.py
+++ b/tools/python/xen/migration/libxc.py
@@ -56,6 +56,7 @@
 REC_TYPE_verify                     = 0x0000000d
 REC_TYPE_checkpoint                 = 0x0000000e
 REC_TYPE_checkpoint_dirty_pfn_list  = 0x0000000f
+REC_TYPE_static_data_end            = 0x00000010
 
 rec_type_to_str = {
     REC_TYPE_end                        : "End",
@@ -74,6 +75,7 @@
     REC_TYPE_verify                     : "Verify",
     REC_TYPE_checkpoint                 : "Checkpoint",
     REC_TYPE_checkpoint_dirty_pfn_list  : "Checkpoint dirty pfn list",
+    REC_TYPE_static_data_end            : "Static data end",
 }
 
 # page_data
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 07/12] python/migration: Update validation logic to understand a v3 stream
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
                   ` (5 preceding siblings ...)
  2019-12-24 15:19 ` [Xen-devel] [PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END Andrew Cooper
@ 2019-12-24 15:19 ` Andrew Cooper
  2019-12-24 15:19 ` [Xen-devel] [PATCH 08/12] libxc/restore: Support v3 streams, and cope with v2 compatibilty Andrew Cooper
                   ` (6 subsequent siblings)
  13 siblings, 0 replies; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel; +Cc: Andrew Cooper, Marek Marczykowski-Górecki

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/python/xen/migration/libxc.py | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)

diff --git a/tools/python/xen/migration/libxc.py b/tools/python/xen/migration/libxc.py
index d0c4f3527d..5fb51b56ac 100644
--- a/tools/python/xen/migration/libxc.py
+++ b/tools/python/xen/migration/libxc.py
@@ -119,6 +119,7 @@ class VerifyLibxc(VerifyBase):
     def __init__(self, info, read):
         VerifyBase.__init__(self, info, read)
 
+        self.version = 0
         self.squashed_pagedata_records = 0
 
 
@@ -145,9 +146,12 @@ def verify_ihdr(self):
             raise StreamError("Bad image id: Expected 0x%x, got 0x%x" %
                               (IHDR_IDENT, ident))
 
-        if version != 2:
-            raise StreamError("Unknown image version: Expected 2, got %d" %
-                              (version, ))
+        if not (2 <= version <= 3):
+            raise StreamError(
+                "Unknown image version: Expected 2 <= ver <= 3, got %d" %
+                (version, ))
+
+        self.version = version
 
         if options & IHDR_OPT_RESZ_MASK:
             raise StreamError("Reserved bits set in image options field: 0x%x" %
@@ -164,7 +168,8 @@ def verify_ihdr(self):
                 "Stream is not native endianess - unable to validate")
 
         endian = ["little", "big"][options & IHDR_OPT_LE]
-        self.info("Libxc Image Header: %s endian" % (endian, ))
+        self.info("Libxc Image Header: Version %d, %s endian" %
+                  (version, endian))
 
 
     def verify_dhdr(self):
@@ -424,6 +429,16 @@ def verify_record_checkpoint_dirty_pfn_list(self, content):
         raise RecordError("Found checkpoint dirty pfn list record in stream")
 
 
+    def verify_record_static_data_end(self, content):
+        """ static data end record """
+
+        if len(content) != 0:
+            raise RecordError("End record with non-zero length")
+
+        if self.version < 3:
+            raise RecordError("Static data end record found in v2 stream")
+
+
 record_verifiers = {
     REC_TYPE_end:
         VerifyLibxc.verify_record_end,
@@ -465,4 +480,7 @@ def verify_record_checkpoint_dirty_pfn_list(self, content):
         VerifyLibxc.verify_record_checkpoint,
     REC_TYPE_checkpoint_dirty_pfn_list:
         VerifyLibxc.verify_record_checkpoint_dirty_pfn_list,
+
+    REC_TYPE_static_data_end:
+        VerifyLibxc.verify_record_static_data_end,
     }
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 08/12] libxc/restore: Support v3 streams, and cope with v2 compatibilty
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
                   ` (6 preceding siblings ...)
  2019-12-24 15:19 ` [Xen-devel] [PATCH 07/12] python/migration: Update validation logic to understand a v3 stream Andrew Cooper
@ 2019-12-24 15:19 ` Andrew Cooper
  2020-01-14 17:02   ` Ian Jackson
  2019-12-24 15:19 ` [Xen-devel] [PATCH 09/12] libxc/save: Write a v3 stream Andrew Cooper
                   ` (5 subsequent siblings)
  13 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel; +Cc: Andrew Cooper, Wei Liu, Ian Jackson

Introduce a static_data_complete() hook which is called when a
STATIC_DATA_END record is found (v3) or inferred (v2).

Modify handle_page_data() and handle_x86_pv_p2m_frames() to infer the
position in v2 streams.

The implementation of x86_static_data_complete() needs to wait until
more plumbing is in place, to make a combined libxl/libxc change to
maintain (functional) bisectability.

No practical change to behaviour - this is all plumbing work.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 tools/libxc/xc_sr_common.h          | 19 +++++++++++++++
 tools/libxc/xc_sr_common_x86.c      |  7 ++++++
 tools/libxc/xc_sr_common_x86.h      |  5 ++++
 tools/libxc/xc_sr_restore.c         | 47 +++++++++++++++++++++++++++++++++++--
 tools/libxc/xc_sr_restore_x86_hvm.c |  1 +
 tools/libxc/xc_sr_restore_x86_pv.c  | 18 ++++++++++++++
 6 files changed, 95 insertions(+), 2 deletions(-)

diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h
index 0289c01e13..2e9a4bc587 100644
--- a/tools/libxc/xc_sr_common.h
+++ b/tools/libxc/xc_sr_common.h
@@ -153,6 +153,13 @@ struct xc_sr_restore_ops
     int (*process_record)(struct xc_sr_context *ctx, struct xc_sr_record *rec);
 
     /**
+     * Perform any actions required after the static data has arrived.  Called
+     * when the STATIC_DATA_COMPLETE record has been recieved (or inferred in
+     * v2-compatibility mode).
+     */
+    int (*static_data_complete)(struct xc_sr_context *ctx);
+
+    /**
      * Perform any actions required after the stream has been finished. Called
      * after the END record has been received.
      */
@@ -255,6 +262,12 @@ struct xc_sr_context
             /* Currently buffering records between a checkpoint */
             bool buffer_all_records;
 
+            /*
+             * Whether a STATIC_DATA_END record has been seen (or implied for
+             * v2 compatibility).
+             */
+            bool seen_static_data_end;
+
 /*
  * With Remus/COLO, we buffer the records sent by the primary at checkpoint,
  * in case the primary will fail, we can recover from the last
@@ -427,6 +440,12 @@ int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec);
 int populate_pfns(struct xc_sr_context *ctx, unsigned int count,
                   const xen_pfn_t *original_pfns, const uint32_t *types);
 
+/*
+ * Handle a STATIC_DATA_END record.  For a v2 compat stream, the position of
+ * this record is inferred from other records.
+ */
+int handle_static_data_end(struct xc_sr_context *ctx);
+
 #endif
 /*
  * Local variables:
diff --git a/tools/libxc/xc_sr_common_x86.c b/tools/libxc/xc_sr_common_x86.c
index 011684df97..083454d256 100644
--- a/tools/libxc/xc_sr_common_x86.c
+++ b/tools/libxc/xc_sr_common_x86.c
@@ -42,6 +42,13 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec)
     return 0;
 }
 
+int x86_static_data_complete(struct xc_sr_context *ctx)
+{
+    /* TODO - something useful. */
+
+    return 0;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libxc/xc_sr_common_x86.h b/tools/libxc/xc_sr_common_x86.h
index ebc4355bd1..7c2d42efe8 100644
--- a/tools/libxc/xc_sr_common_x86.h
+++ b/tools/libxc/xc_sr_common_x86.h
@@ -14,6 +14,11 @@ int write_x86_tsc_info(struct xc_sr_context *ctx);
  */
 int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec);
 
+/*
+ * Perform common x86 actions required after the static data has arrived.
+ */
+int x86_static_data_complete(struct xc_sr_context *ctx);
+
 #endif
 /*
  * Local variables:
diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c
index 0280e55d4b..d4bd60a31e 100644
--- a/tools/libxc/xc_sr_restore.c
+++ b/tools/libxc/xc_sr_restore.c
@@ -35,9 +35,9 @@ static int read_headers(struct xc_sr_context *ctx)
         return -1;
     }
 
-    if ( ihdr.version != 2 )
+    if ( ihdr.version < 2 || ihdr.version > 3 )
     {
-        ERROR("Invalid Version: Expected 2, Got %d",
+        ERROR("Invalid Version: Expected 2 <= ver <= 3, Got %d",
               ihdr.version);
         return -1;
     }
@@ -342,6 +342,30 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec)
     xen_pfn_t *pfns = NULL, pfn;
     uint32_t *types = NULL, type;
 
+    /*
+     * This is a bit of a bodge, but it is less bad than duplicating
+     * handle_page_data() between different architectures.
+     */
+#if defined(__i386__) || defined(__x86_64__)
+    /* v2 compat.  Infer the position of STATIC_DATA_END. */
+    if ( ctx->restore.format_version < 3 && !ctx->restore.seen_static_data_end )
+    {
+        rc = handle_static_data_end(ctx);
+        if ( rc )
+        {
+            ERROR("Inferred STATIC_DATA_END record failed");
+            goto err;
+        }
+        rc = -1;
+    }
+
+    if ( !ctx->restore.seen_static_data_end )
+    {
+        ERROR("No STATIC_DATA_END seen");
+        goto err;
+    }
+#endif
+
     if ( rec->length < sizeof(*pages) )
     {
         ERROR("PAGE_DATA record truncated: length %u, min %zu",
@@ -631,6 +655,21 @@ static int buffer_record(struct xc_sr_context *ctx, struct xc_sr_record *rec)
     return 0;
 }
 
+int handle_static_data_end(struct xc_sr_context *ctx)
+{
+    xc_interface *xch = ctx->xch;
+
+    if ( ctx->restore.seen_static_data_end )
+    {
+        ERROR("Multiple STATIC_DATA_END records found");
+        return -1;
+    }
+
+    ctx->restore.seen_static_data_end = true;
+
+    return ctx->restore.ops.static_data_complete(ctx);
+}
+
 static int process_record(struct xc_sr_context *ctx, struct xc_sr_record *rec)
 {
     xc_interface *xch = ctx->xch;
@@ -654,6 +693,10 @@ static int process_record(struct xc_sr_context *ctx, struct xc_sr_record *rec)
         rc = handle_checkpoint(ctx);
         break;
 
+    case REC_TYPE_STATIC_DATA_END:
+        rc = handle_static_data_end(ctx);
+        break;
+
     default:
         rc = ctx->restore.ops.process_record(ctx, rec);
         break;
diff --git a/tools/libxc/xc_sr_restore_x86_hvm.c b/tools/libxc/xc_sr_restore_x86_hvm.c
index 3f78248f32..94f47f2589 100644
--- a/tools/libxc/xc_sr_restore_x86_hvm.c
+++ b/tools/libxc/xc_sr_restore_x86_hvm.c
@@ -239,6 +239,7 @@ struct xc_sr_restore_ops restore_ops_x86_hvm =
     .localise_page   = x86_hvm_localise_page,
     .setup           = x86_hvm_setup,
     .process_record  = x86_hvm_process_record,
+    .static_data_complete = x86_static_data_complete,
     .stream_complete = x86_hvm_stream_complete,
     .cleanup         = x86_hvm_cleanup,
 };
diff --git a/tools/libxc/xc_sr_restore_x86_pv.c b/tools/libxc/xc_sr_restore_x86_pv.c
index 8f61a5e8b9..90b1e5427b 100644
--- a/tools/libxc/xc_sr_restore_x86_pv.c
+++ b/tools/libxc/xc_sr_restore_x86_pv.c
@@ -679,6 +679,23 @@ static int handle_x86_pv_p2m_frames(struct xc_sr_context *ctx,
     unsigned int start, end, x, fpp = PAGE_SIZE / ctx->x86.pv.width;
     int rc;
 
+    /* v2 compat.  Infer the position of STATIC_DATA_END. */
+    if ( ctx->restore.format_version < 3 && !ctx->restore.seen_static_data_end )
+    {
+        rc = handle_static_data_end(ctx);
+        if ( rc )
+        {
+            ERROR("Inferred STATIC_DATA_END record failed");
+            return rc;
+        }
+    }
+
+    if ( !ctx->restore.seen_static_data_end )
+    {
+        ERROR("No STATIC_DATA_END seen");
+        return -1;
+    }
+
     if ( !ctx->x86.pv.restore.seen_pv_info )
     {
         ERROR("Not yet received X86_PV_INFO record");
@@ -1168,6 +1185,7 @@ struct xc_sr_restore_ops restore_ops_x86_pv =
     .localise_page   = x86_pv_localise_page,
     .setup           = x86_pv_setup,
     .process_record  = x86_pv_process_record,
+    .static_data_complete = x86_static_data_complete,
     .stream_complete = x86_pv_stream_complete,
     .cleanup         = x86_pv_cleanup,
 };
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 09/12] libxc/save: Write a v3 stream
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
                   ` (7 preceding siblings ...)
  2019-12-24 15:19 ` [Xen-devel] [PATCH 08/12] libxc/restore: Support v3 streams, and cope with v2 compatibilty Andrew Cooper
@ 2019-12-24 15:19 ` Andrew Cooper
  2020-01-14 17:05   ` Ian Jackson
  2019-12-24 15:19 ` [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records Andrew Cooper
                   ` (4 subsequent siblings)
  13 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel
  Cc: Andrew Cooper, Marek Marczykowski-Górecki, Wei Liu, Ian Jackson

Introduce a new static_data() hook which is responsible for writing out
any static data records.  The HVM side continues to be a no-op, while
the PV side moves write_x86_pv_info() into this earlier hook.  The the
common code writes out a STATIC_DATA_END record, and the stream version
is bumped to 3.

Update convert-legacy-stream to write a v3 stream, because this will
bypass the compatiblity logic in libxc.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 tools/libxc/xc_sr_common.h                 | 10 ++++++++--
 tools/libxc/xc_sr_save.c                   | 14 +++++++++++++-
 tools/libxc/xc_sr_save_x86_hvm.c           |  6 ++++++
 tools/libxc/xc_sr_save_x86_pv.c            | 10 ++++++----
 tools/python/scripts/convert-legacy-stream |  9 ++++++++-
 5 files changed, 41 insertions(+), 8 deletions(-)

diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h
index 2e9a4bc587..3f52ef7d53 100644
--- a/tools/libxc/xc_sr_common.h
+++ b/tools/libxc/xc_sr_common.h
@@ -63,8 +63,14 @@ struct xc_sr_save_ops
     int (*setup)(struct xc_sr_context *ctx);
 
     /**
-     * Send records which need to be at the start of the stream.  This is
-     * called once, after the Image and Domain headers are written.
+     * Send static records at the head of the stream.  This is called once,
+     * after the Image and Domain headers are written.
+     */
+    int (*static_data)(struct xc_sr_context *ctx);
+
+    /**
+     * Send dynamic records which need to be at the start of the stream.  This
+     * is called after the STATIC_DATA_END record is written.
      */
     int (*start_of_stream)(struct xc_sr_context *ctx);
 
diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
index fa1a2e6c2a..f24dd0d00b 100644
--- a/tools/libxc/xc_sr_save.c
+++ b/tools/libxc/xc_sr_save.c
@@ -13,7 +13,7 @@ static int write_headers(struct xc_sr_context *ctx, uint16_t guest_type)
     struct xc_sr_ihdr ihdr = {
         .marker  = IHDR_MARKER,
         .id      = htonl(IHDR_ID),
-        .version = htonl(2),
+        .version = htonl(3),
         .options = htons(IHDR_OPT_LITTLE_ENDIAN),
     };
     struct xc_sr_dhdr dhdr = {
@@ -55,6 +55,16 @@ static int write_end_record(struct xc_sr_context *ctx)
 }
 
 /*
+ * Writes a STATIC_DATA_END record into the stream.
+ */
+static int write_static_data_end_record(struct xc_sr_context *ctx)
+{
+    struct xc_sr_record end = { .type = REC_TYPE_STATIC_DATA_END };
+
+    return write_record(ctx, &end);
+}
+
+/*
  * Writes a CHECKPOINT record into the stream.
  */
 static int write_checkpoint_record(struct xc_sr_context *ctx)
@@ -846,6 +856,8 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type)
     xc_report_progress_single(xch, "Start of stream");
 
     rc = (write_headers(ctx, guest_type) ?:
+          ctx->save.ops.static_data(ctx) ?:
+          write_static_data_end_record(ctx) ?:
           ctx->save.ops.start_of_stream(ctx));
     if ( rc )
         goto err;
diff --git a/tools/libxc/xc_sr_save_x86_hvm.c b/tools/libxc/xc_sr_save_x86_hvm.c
index 58722118ae..c146e827bc 100644
--- a/tools/libxc/xc_sr_save_x86_hvm.c
+++ b/tools/libxc/xc_sr_save_x86_hvm.c
@@ -170,6 +170,11 @@ static int x86_hvm_setup(struct xc_sr_context *ctx)
     return 0;
 }
 
+static int x86_hvm_static_data(struct xc_sr_context *ctx)
+{
+    return 0;
+}
+
 static int x86_hvm_start_of_stream(struct xc_sr_context *ctx)
 {
     return 0;
@@ -213,6 +218,7 @@ struct xc_sr_save_ops save_ops_x86_hvm =
     .pfn_to_gfn          = x86_hvm_pfn_to_gfn,
     .normalise_page      = x86_hvm_normalise_page,
     .setup               = x86_hvm_setup,
+    .static_data         = x86_hvm_static_data,
     .start_of_stream     = x86_hvm_start_of_stream,
     .start_of_checkpoint = x86_hvm_start_of_checkpoint,
     .end_of_checkpoint   = x86_hvm_end_of_checkpoint,
diff --git a/tools/libxc/xc_sr_save_x86_pv.c b/tools/libxc/xc_sr_save_x86_pv.c
index c1c6892666..9d1ca6ead7 100644
--- a/tools/libxc/xc_sr_save_x86_pv.c
+++ b/tools/libxc/xc_sr_save_x86_pv.c
@@ -1028,14 +1028,15 @@ static int x86_pv_setup(struct xc_sr_context *ctx)
             map_p2m(ctx));
 }
 
+static int x86_pv_static_data(struct xc_sr_context *ctx)
+{
+    return write_x86_pv_info(ctx);
+}
+
 static int x86_pv_start_of_stream(struct xc_sr_context *ctx)
 {
     int rc;
 
-    rc = write_x86_pv_info(ctx);
-    if ( rc )
-        return rc;
-
     /*
      * Ideally should be able to change during migration.  Currently
      * corruption will occur if the contents or location of the P2M changes
@@ -1090,6 +1091,7 @@ struct xc_sr_save_ops save_ops_x86_pv =
     .pfn_to_gfn          = x86_pv_pfn_to_gfn,
     .normalise_page      = x86_pv_normalise_page,
     .setup               = x86_pv_setup,
+    .static_data         = x86_pv_static_data,
     .start_of_stream     = x86_pv_start_of_stream,
     .start_of_checkpoint = x86_pv_start_of_checkpoint,
     .end_of_checkpoint   = x86_pv_end_of_checkpoint,
diff --git a/tools/python/scripts/convert-legacy-stream b/tools/python/scripts/convert-legacy-stream
index 02a194178f..ca93a93848 100755
--- a/tools/python/scripts/convert-legacy-stream
+++ b/tools/python/scripts/convert-legacy-stream
@@ -79,7 +79,7 @@ def write_libxc_ihdr():
     stream_write(pack(libxc.IHDR_FORMAT,
                       libxc.IHDR_MARKER,  # Marker
                       libxc.IHDR_IDENT,   # Ident
-                      2,                  # Version
+                      3,                  # Version
                       libxc.IHDR_OPT_LE,  # Options
                       0, 0))              # Reserved
 
@@ -166,6 +166,9 @@ def write_libxc_hvm_params(params):
                  pack(libxc.HVM_PARAMS_FORMAT, len(params) / 2, 0),
                  pack("Q" * len(params), *params))
 
+def write_libxc_static_data_end():
+    write_record(libxc.REC_TYPE_static_data_end)
+
 def write_libxl_end():
     write_record(libxl.REC_TYPE_end)
 
@@ -590,6 +593,10 @@ def read_legacy_stream(vm):
 
         if pv:
             read_pv_extended_info(vm)
+
+        write_libxc_static_data_end()
+
+        if pv:
             read_pv_p2m_frames(vm)
 
         read_chunks(vm)
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
                   ` (8 preceding siblings ...)
  2019-12-24 15:19 ` [Xen-devel] [PATCH 09/12] libxc/save: Write a v3 stream Andrew Cooper
@ 2019-12-24 15:19 ` Andrew Cooper
  2020-01-03 14:49   ` Jan Beulich
  2020-01-14 16:08   ` Ian Jackson
  2019-12-24 15:19 ` [Xen-devel] [PATCH 11/12] libxc/restore: Handle X86_{CPUID, MSR}_DATA records Andrew Cooper
                   ` (3 subsequent siblings)
  13 siblings, 2 replies; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel
  Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Andrew Cooper, Tim Deegan,
	Marek Marczykowski-Górecki, Julien Grall, Jan Beulich,
	Ian Jackson

These two records move blobs from the XEN_DOMCTL_{get,set}_cpu_policy
hypercall.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Tim Deegan <tim@xen.org>
CC: Wei Liu <wl@xen.org>
CC: Julien Grall <julien.grall@arm.com>
CC: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
 docs/specs/libxc-migration-stream.pandoc | 42 +++++++++++++++++++++++++++++++
 tools/libxc/xc_sr_common.c               |  2 ++
 tools/libxc/xc_sr_stream_format.h        |  2 ++
 tools/python/xen/migration/libxc.py      | 43 ++++++++++++++++++++++++++++++++
 4 files changed, 89 insertions(+)

diff --git a/docs/specs/libxc-migration-stream.pandoc b/docs/specs/libxc-migration-stream.pandoc
index 22ff306e0b..3a0915b795 100644
--- a/docs/specs/libxc-migration-stream.pandoc
+++ b/docs/specs/libxc-migration-stream.pandoc
@@ -634,6 +634,46 @@ The end record contains no fields; its body_length is 0.
 
 \clearpage
 
+X86_CPUID_POLICY
+----------------
+
+CPUID policy content, as accessed by the XEN_DOMCTL_{get,set}_cpu_policy
+hypercall sub-ops.
+
+     0     1     2     3     4     5     6     7 octet
+    +-------------------------------------------------+
+    | CPUID_policy                                    |
+    ...
+    +-------------------------------------------------+
+
+--------------------------------------------------------------------
+Field            Description
+------------     ---------------------------------------------------
+CPUID_policy     Array of xen_cpuid_leaf_t[]'s
+--------------------------------------------------------------------
+
+\clearpage
+
+X86_MSR_POLICY
+--------------
+
+MSR policy content, as accessed by the XEN_DOMCTL_{get,set}_cpu_policy
+hypercall sub-ops.
+
+     0     1     2     3     4     5     6     7 octet
+    +-------------------------------------------------+
+    | MSR_policy                                      |
+    ...
+    +-------------------------------------------------+
+
+--------------------------------------------------------------------
+Field            Description
+----------       ---------------------------------------------------
+MSR_policy       Array of xen_msr_entry_t[]'s
+--------------------------------------------------------------------
+
+\clearpage
+
 
 Layout
 ======
@@ -656,6 +696,7 @@ A typical save record for an x86 PV guest image would look like:
 * Domain header
 * Static data records:
     * X86_PV_INFO record
+    * X86_{CPUID,MSR}_POLICY
     * STATIC_DATA_END
 * X86_PV_P2M_FRAMES record
 * Many PAGE_DATA records
@@ -685,6 +726,7 @@ A typical save record for an x86 HVM guest image would look like:
 * Image header
 * Domain header
 * Static data records:
+    * X86_{CPUID,MSR}_POLICY
     * STATIC_DATA_END
 * Many PAGE_DATA records
 * X86_TSC_INFO
diff --git a/tools/libxc/xc_sr_common.c b/tools/libxc/xc_sr_common.c
index 7f22cf0365..7c54b03414 100644
--- a/tools/libxc/xc_sr_common.c
+++ b/tools/libxc/xc_sr_common.c
@@ -37,6 +37,8 @@ static const char *const mandatory_rec_types[] =
     [REC_TYPE_CHECKPOINT]                   = "Checkpoint",
     [REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST]    = "Checkpoint dirty pfn list",
     [REC_TYPE_STATIC_DATA_END]              = "Static data end",
+    [REC_TYPE_X86_CPUID_POLICY]             = "x86 CPUID policy",
+    [REC_TYPE_X86_MSR_POLICY]               = "x86 MSR policy",
 };
 
 const char *rec_type_to_str(uint32_t type)
diff --git a/tools/libxc/xc_sr_stream_format.h b/tools/libxc/xc_sr_stream_format.h
index 81c9765b0a..8a0da26f75 100644
--- a/tools/libxc/xc_sr_stream_format.h
+++ b/tools/libxc/xc_sr_stream_format.h
@@ -74,6 +74,8 @@ struct xc_sr_rhdr
 #define REC_TYPE_CHECKPOINT                 0x0000000eU
 #define REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST  0x0000000fU
 #define REC_TYPE_STATIC_DATA_END            0x00000010U
+#define REC_TYPE_X86_CPUID_POLICY           0x00000011U
+#define REC_TYPE_X86_MSR_POLICY             0x00000012U
 
 #define REC_TYPE_OPTIONAL             0x80000000U
 
diff --git a/tools/python/xen/migration/libxc.py b/tools/python/xen/migration/libxc.py
index 5fb51b56ac..9881f5ced4 100644
--- a/tools/python/xen/migration/libxc.py
+++ b/tools/python/xen/migration/libxc.py
@@ -57,6 +57,8 @@
 REC_TYPE_checkpoint                 = 0x0000000e
 REC_TYPE_checkpoint_dirty_pfn_list  = 0x0000000f
 REC_TYPE_static_data_end            = 0x00000010
+REC_TYPE_x86_cpuid_policy           = 0x00000011
+REC_TYPE_x86_msr_policy             = 0x00000012
 
 rec_type_to_str = {
     REC_TYPE_end                        : "End",
@@ -76,6 +78,8 @@
     REC_TYPE_checkpoint                 : "Checkpoint",
     REC_TYPE_checkpoint_dirty_pfn_list  : "Checkpoint dirty pfn list",
     REC_TYPE_static_data_end            : "Static data end",
+    REC_TYPE_x86_cpuid_policy           : "x86 CPUID policy",
+    REC_TYPE_x86_msr_policy             : "x86 MSR policy",
 }
 
 # page_data
@@ -113,6 +117,12 @@
 HVM_PARAMS_ENTRY_FORMAT   = "QQ"
 HVM_PARAMS_FORMAT         = "II"
 
+# x86_cpuid_policy => xen_cpuid_leaf_t[]
+X86_CPUID_POLICY_FORMAT   = "IIIIII"
+
+# x86_msr_policy => xen_msr_entry_t[]
+X86_MSR_POLICY_FORMAT     = "QII"
+
 class VerifyLibxc(VerifyBase):
     """ Verify a Libxc v2 (or later) stream """
 
@@ -439,6 +449,34 @@ def verify_record_static_data_end(self, content):
             raise RecordError("Static data end record found in v2 stream")
 
 
+    def verify_record_x86_cpuid_policy(self, content):
+        """ x86 CPUID policy record """
+
+        if self.version < 3:
+            raise RecordError("x86 CPUID policy record found in v2 stream")
+
+        sz = calcsize(X86_CPUID_POLICY_FORMAT)
+        contentsz = len(content)
+
+        if contentsz < sz or (contentsz % sz) != 0:
+            raise RecordError("Record length %u, expected multiple of %u" %
+                              (contentsz, sz))
+
+
+    def verify_record_x86_msr_policy(self, content):
+        """ x86 MSR policy record """
+
+        if self.version < 3:
+            raise RecordError("x86 MSR policy record found in v2 stream")
+
+        sz = calcsize(X86_MSR_POLICY_FORMAT)
+        contentsz = len(content)
+
+        if contentsz < sz or (contentsz % sz) != 0:
+            raise RecordError("Record length %u, expected multiple of %u" %
+                              (contentsz, sz))
+
+
 record_verifiers = {
     REC_TYPE_end:
         VerifyLibxc.verify_record_end,
@@ -483,4 +521,9 @@ def verify_record_static_data_end(self, content):
 
     REC_TYPE_static_data_end:
         VerifyLibxc.verify_record_static_data_end,
+
+    REC_TYPE_x86_cpuid_policy:
+        VerifyLibxc.verify_record_x86_cpuid_policy,
+    REC_TYPE_x86_msr_policy:
+        VerifyLibxc.verify_record_x86_msr_policy,
     }
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 11/12] libxc/restore: Handle X86_{CPUID, MSR}_DATA records
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
                   ` (9 preceding siblings ...)
  2019-12-24 15:19 ` [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records Andrew Cooper
@ 2019-12-24 15:19 ` Andrew Cooper
  2020-01-14 17:16   ` Ian Jackson
  2019-12-24 15:19 ` [Xen-devel] [PATCH 12/12] libxc/save: Write " Andrew Cooper
                   ` (2 subsequent siblings)
  13 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel; +Cc: Andrew Cooper, Wei Liu, Ian Jackson

For now, the data are just stashed, and discarded at the end.  This will
be addressed when the TODO in x86_static_data_complete() is addressed.

No practical change to behaviour - this is all plumbing work.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 tools/libxc/xc_sr_common.h          | 10 ++++++++++
 tools/libxc/xc_sr_common_x86.c      | 40 +++++++++++++++++++++++++++++++++++++
 tools/libxc/xc_sr_common_x86.h      | 14 +++++++++++++
 tools/libxc/xc_sr_restore_x86_hvm.c |  9 +++++++++
 tools/libxc/xc_sr_restore_x86_pv.c  |  9 +++++++++
 5 files changed, 82 insertions(+)

diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h
index 3f52ef7d53..337c746e17 100644
--- a/tools/libxc/xc_sr_common.h
+++ b/tools/libxc/xc_sr_common.h
@@ -308,6 +308,16 @@ struct xc_sr_context
     {
         struct /* x86 */
         {
+            /* Common save/restore data. */
+            union
+            {
+                struct
+                {
+                    /* X86_{CPUID,MSR}_DATA blobs for CPU Policy. */
+                    struct xc_sr_blob cpuid, msr;
+                } restore;
+            };
+
             struct /* x86 PV guest. */
             {
                 /* 4 or 8; 32 or 64 bit domain */
diff --git a/tools/libxc/xc_sr_common_x86.c b/tools/libxc/xc_sr_common_x86.c
index 083454d256..d4d86bb12f 100644
--- a/tools/libxc/xc_sr_common_x86.c
+++ b/tools/libxc/xc_sr_common_x86.c
@@ -49,6 +49,46 @@ int x86_static_data_complete(struct xc_sr_context *ctx)
     return 0;
 }
 
+int handle_x86_cpuid_policy(struct xc_sr_context *ctx, struct xc_sr_record *rec)
+{
+    xc_interface *xch = ctx->xch;
+    int rc;
+
+    if ( rec->length == 0 ||
+         rec->length % sizeof(xen_cpuid_leaf_t) != 0 )
+    {
+        ERROR("X86_CPUID_POLICY size %u should be multiple of %zu",
+              rec->length, sizeof(xen_cpuid_leaf_t));
+        return -1;
+    }
+
+    rc = update_blob(&ctx->x86.restore.cpuid, rec->data, rec->length);
+    if ( rc )
+        ERROR("Unable to allocate %u bytes for X86_CPUID_POLICY", rec->length);
+
+    return rc;
+}
+
+int handle_x86_msr_policy(struct xc_sr_context *ctx, struct xc_sr_record *rec)
+{
+    xc_interface *xch = ctx->xch;
+    int rc;
+
+    if ( rec->length == 0 ||
+         rec->length % sizeof(xen_msr_entry_t) != 0 )
+    {
+        ERROR("X86_MSR_POLICY size %u should be multiple of %zu",
+              rec->length, sizeof(xen_cpuid_leaf_t));
+        return -1;
+    }
+
+    rc = update_blob(&ctx->x86.restore.msr, rec->data, rec->length);
+    if ( rc )
+        ERROR("Unable to allocate %u bytes for X86_MSR_POLICY", rec->length);
+
+    return rc;
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libxc/xc_sr_common_x86.h b/tools/libxc/xc_sr_common_x86.h
index 7c2d42efe8..f7fa4fe62b 100644
--- a/tools/libxc/xc_sr_common_x86.h
+++ b/tools/libxc/xc_sr_common_x86.h
@@ -19,6 +19,20 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec);
  */
 int x86_static_data_complete(struct xc_sr_context *ctx);
 
+/*
+ * Parses an X86_CPUID_POLICY record and stashes the content for application
+ * when a STATIC_DATA_END record is encountered.
+ */
+int handle_x86_cpuid_policy(struct xc_sr_context *ctx,
+                            struct xc_sr_record *rec);
+
+/*
+ * Parses an X86_MSR_POLICY record and stashes the content for application
+ * when a STATIC_DATA_END record is encountered.
+ */
+int handle_x86_msr_policy(struct xc_sr_context *ctx,
+                          struct xc_sr_record *rec);
+
 #endif
 /*
  * Local variables:
diff --git a/tools/libxc/xc_sr_restore_x86_hvm.c b/tools/libxc/xc_sr_restore_x86_hvm.c
index 94f47f2589..9190edaee7 100644
--- a/tools/libxc/xc_sr_restore_x86_hvm.c
+++ b/tools/libxc/xc_sr_restore_x86_hvm.c
@@ -171,6 +171,12 @@ static int x86_hvm_process_record(struct xc_sr_context *ctx,
     case REC_TYPE_HVM_PARAMS:
         return handle_hvm_params(ctx, rec);
 
+    case REC_TYPE_X86_CPUID_POLICY:
+        return handle_x86_cpuid_policy(ctx, rec);
+
+    case REC_TYPE_X86_MSR_POLICY:
+        return handle_x86_msr_policy(ctx, rec);
+
     default:
         return RECORD_NOT_PROCESSED;
     }
@@ -227,6 +233,9 @@ static int x86_hvm_cleanup(struct xc_sr_context *ctx)
 {
     free(ctx->x86.hvm.restore.context.ptr);
 
+    free(ctx->x86.restore.cpuid.ptr);
+    free(ctx->x86.restore.msr.ptr);
+
     return 0;
 }
 
diff --git a/tools/libxc/xc_sr_restore_x86_pv.c b/tools/libxc/xc_sr_restore_x86_pv.c
index 90b1e5427b..3d593753ab 100644
--- a/tools/libxc/xc_sr_restore_x86_pv.c
+++ b/tools/libxc/xc_sr_restore_x86_pv.c
@@ -1102,6 +1102,12 @@ static int x86_pv_process_record(struct xc_sr_context *ctx,
     case REC_TYPE_X86_TSC_INFO:
         return handle_x86_tsc_info(ctx, rec);
 
+    case REC_TYPE_X86_CPUID_POLICY:
+        return handle_x86_cpuid_policy(ctx, rec);
+
+    case REC_TYPE_X86_MSR_POLICY:
+        return handle_x86_msr_policy(ctx, rec);
+
     default:
         return RECORD_NOT_PROCESSED;
     }
@@ -1173,6 +1179,9 @@ static int x86_pv_cleanup(struct xc_sr_context *ctx)
     if ( ctx->x86.pv.m2p )
         munmap(ctx->x86.pv.m2p, ctx->x86.pv.nr_m2p_frames * PAGE_SIZE);
 
+    free(ctx->x86.restore.cpuid.ptr);
+    free(ctx->x86.restore.msr.ptr);
+
     return 0;
 }
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 12/12] libxc/save: Write X86_{CPUID, MSR}_DATA records
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
                   ` (10 preceding siblings ...)
  2019-12-24 15:19 ` [Xen-devel] [PATCH 11/12] libxc/restore: Handle X86_{CPUID, MSR}_DATA records Andrew Cooper
@ 2019-12-24 15:19 ` Andrew Cooper
  2020-01-14 17:21   ` Ian Jackson
  2020-01-03 13:06 ` [Xen-devel] [PATCH] Use CPUID/MSR data from migration streams Andrew Cooper
  2020-01-15 18:53 ` [Xen-devel] [PATCH 0.5/12] tools/migration: Formatting and style cleanup Andrew Cooper
  13 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2019-12-24 15:19 UTC (permalink / raw)
  To: Xen-devel; +Cc: Andrew Cooper, Wei Liu, Ian Jackson

With all other plumbing in place, obtain the CPU Policy from Xen and
write it into the migration stream.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 tools/libxc/xc_sr_common_x86.c   | 50 ++++++++++++++++++++++++++++++++++++++++
 tools/libxc/xc_sr_common_x86.h   |  6 +++++
 tools/libxc/xc_sr_save_x86_hvm.c |  2 +-
 tools/libxc/xc_sr_save_x86_pv.c  |  3 ++-
 4 files changed, 59 insertions(+), 2 deletions(-)

diff --git a/tools/libxc/xc_sr_common_x86.c b/tools/libxc/xc_sr_common_x86.c
index d4d86bb12f..09fb1d93d6 100644
--- a/tools/libxc/xc_sr_common_x86.c
+++ b/tools/libxc/xc_sr_common_x86.c
@@ -49,6 +49,56 @@ int x86_static_data_complete(struct xc_sr_context *ctx)
     return 0;
 }
 
+int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
+{
+    xc_interface *xch = ctx->xch;
+    struct xc_sr_record cpuid = { .type = REC_TYPE_X86_CPUID_POLICY, };
+    struct xc_sr_record msrs  = { .type = REC_TYPE_X86_MSR_POLICY, };
+    uint32_t nr_leaves = 0, nr_msrs = 0;
+    int rc;
+
+    if ( xc_get_cpu_policy_size(xch, &nr_leaves, &nr_msrs) < 0 )
+    {
+        PERROR("Unable to get CPU Policy size");
+        return -1;
+    }
+
+    cpuid.data = malloc(nr_leaves * sizeof(xen_cpuid_leaf_t));
+    msrs.data  = malloc(nr_msrs   * sizeof(xen_msr_entry_t));
+    if ( !cpuid.data || !msrs.data )
+    {
+        ERROR("Cannot allocate memory for CPU Policy");
+        rc = -1;
+        goto out;
+    }
+
+    if ( xc_get_domain_cpu_policy(xch, ctx->domid, &nr_leaves, cpuid.data,
+                                  &nr_msrs, msrs.data) )
+    {
+        PERROR("Unable to get d%d CPU Policy", ctx->domid);
+        rc = -1;
+        goto out;
+    }
+
+    cpuid.length = nr_leaves * sizeof(xen_cpuid_leaf_t);
+    if ( cpuid.length )
+    {
+        rc = write_record(ctx, &cpuid);
+        if ( rc )
+            goto out;
+    }
+
+    msrs.length = nr_msrs * sizeof(xen_msr_entry_t);
+    if ( msrs.length )
+        rc = write_record(ctx, &msrs);
+
+ out:
+    free(cpuid.data);
+    free(msrs.data);
+
+    return rc;
+}
+
 int handle_x86_cpuid_policy(struct xc_sr_context *ctx, struct xc_sr_record *rec)
 {
     xc_interface *xch = ctx->xch;
diff --git a/tools/libxc/xc_sr_common_x86.h b/tools/libxc/xc_sr_common_x86.h
index f7fa4fe62b..7b1823eca9 100644
--- a/tools/libxc/xc_sr_common_x86.h
+++ b/tools/libxc/xc_sr_common_x86.h
@@ -20,6 +20,12 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec);
 int x86_static_data_complete(struct xc_sr_context *ctx);
 
 /*
+ * Obtains a domains CPU Policy from Xen, and writes X86_{CPUID,MSR}_POLICY
+ * records into the stream.
+ */
+int write_x86_cpu_policy_records(struct xc_sr_context *ctx);
+
+/*
  * Parses an X86_CPUID_POLICY record and stashes the content for application
  * when a STATIC_DATA_END record is encountered.
  */
diff --git a/tools/libxc/xc_sr_save_x86_hvm.c b/tools/libxc/xc_sr_save_x86_hvm.c
index c146e827bc..5b95d20b19 100644
--- a/tools/libxc/xc_sr_save_x86_hvm.c
+++ b/tools/libxc/xc_sr_save_x86_hvm.c
@@ -172,7 +172,7 @@ static int x86_hvm_setup(struct xc_sr_context *ctx)
 
 static int x86_hvm_static_data(struct xc_sr_context *ctx)
 {
-    return 0;
+    return write_x86_cpu_policy_records(ctx);
 }
 
 static int x86_hvm_start_of_stream(struct xc_sr_context *ctx)
diff --git a/tools/libxc/xc_sr_save_x86_pv.c b/tools/libxc/xc_sr_save_x86_pv.c
index 9d1ca6ead7..2e9aa9ea16 100644
--- a/tools/libxc/xc_sr_save_x86_pv.c
+++ b/tools/libxc/xc_sr_save_x86_pv.c
@@ -1030,7 +1030,8 @@ static int x86_pv_setup(struct xc_sr_context *ctx)
 
 static int x86_pv_static_data(struct xc_sr_context *ctx)
 {
-    return write_x86_pv_info(ctx);
+    return (write_x86_pv_info(ctx) ?:
+            write_x86_cpu_policy_records(ctx));
 }
 
 static int x86_pv_start_of_stream(struct xc_sr_context *ctx)
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH] Use CPUID/MSR data from migration streams
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
                   ` (11 preceding siblings ...)
  2019-12-24 15:19 ` [Xen-devel] [PATCH 12/12] libxc/save: Write " Andrew Cooper
@ 2020-01-03 13:06 ` Andrew Cooper
  2020-01-03 13:06   ` [Xen-devel] [PATCH 15/20] fixup tools/migration: Formatting and style cleanup Andrew Cooper
                     ` (5 more replies)
  2020-01-15 18:53 ` [Xen-devel] [PATCH 0.5/12] tools/migration: Formatting and style cleanup Andrew Cooper
  13 siblings, 6 replies; 60+ messages in thread
From: Andrew Cooper @ 2020-01-03 13:06 UTC (permalink / raw)
  To: Xen-devel; +Cc: Anthony PERARD, Andrew Cooper, Wei Liu, Ian Jackson

This is the incremental work to "Support CPUID/MSR data in migration streams"
which juggles libxl sufficiently to allow us to use the data sent in migration
streams.

The two patches:
  tools/libxl: Code-gen improvements for libxl_save_msgs_gen.pl
  tools/libxl: Reposition build_pre() logic between architectures

posted independently are logically patches 13/14 of the series, and these six
make it up to 20.

Future versions of the series, if necessary, will post them all togther.

Andrew Cooper (6):
  fixup tools/migration: Formatting and style cleanup
  tools/libxl: Simplify callback handling in libxl-save-helper
  tools/libx[cl]: Plumb static_data_done() up into libxl
  tools/libxl: Plumb domain_create_state down into libxl__build_pre()
  tools/libxl: Re-position CPUID handling during domain construction
  tools/libxc: Restore CPUID/MSR data found in the migration stream

 tools/libxc/include/xenguest.h     | 44 +++++++++++++++++++++++-------------
 tools/libxc/xc_sr_common_x86.c     | 35 +++++++++++++++++++++++++++--
 tools/libxc/xc_sr_restore.c        |  1 +
 tools/libxc/xc_sr_save.c           |  8 +++----
 tools/libxl/libxl_create.c         | 46 +++++++++++++++++++++++++++++++-------
 tools/libxl/libxl_dm.c             |  5 +++--
 tools/libxl/libxl_dom.c            | 16 ++++++++-----
 tools/libxl/libxl_internal.h       | 11 ++++-----
 tools/libxl/libxl_save_helper.c    | 15 +++++--------
 tools/libxl/libxl_save_msgs_gen.pl |  3 ++-
 10 files changed, 130 insertions(+), 54 deletions(-)

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 15/20] fixup tools/migration: Formatting and style cleanup
  2020-01-03 13:06 ` [Xen-devel] [PATCH] Use CPUID/MSR data from migration streams Andrew Cooper
@ 2020-01-03 13:06   ` Andrew Cooper
  2020-01-03 13:06   ` [Xen-devel] [PATCH 16/20] tools/libxl: Simplify callback handling in libxl-save-helper Andrew Cooper
                     ` (4 subsequent siblings)
  5 siblings, 0 replies; 60+ messages in thread
From: Andrew Cooper @ 2020-01-03 13:06 UTC (permalink / raw)
  To: Xen-devel; +Cc: Andrew Cooper

Further cleanup.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 tools/libxc/include/xenguest.h | 37 ++++++++++++++++++++-----------------
 tools/libxc/xc_sr_save.c       |  8 ++++----
 2 files changed, 24 insertions(+), 21 deletions(-)

diff --git a/tools/libxc/include/xenguest.h b/tools/libxc/include/xenguest.h
index 9ba09af743..d81baa6cc2 100644
--- a/tools/libxc/include/xenguest.h
+++ b/tools/libxc/include/xenguest.h
@@ -55,10 +55,11 @@ typedef int (*precopy_policy_t)(struct precopy_stats, void *);
 
 /* callbacks provided by xc_domain_save */
 struct save_callbacks {
-    /* Called after expiration of checkpoint interval,
+    /*
+     * Called after expiration of checkpoint interval,
      * to suspend the guest.
      */
-    int (*suspend)(void* data);
+    int (*suspend)(void *data);
 
     /*
      * Called before and after every batch of page data sent during
@@ -82,7 +83,7 @@ struct save_callbacks {
      * xc_domain_save then flushes the output buffer, while the
      *  guest continues to run.
      */
-    int (*postcopy)(void* data);
+    int (*postcopy)(void *data);
 
     /*
      * Called after the memory checkpoint has been flushed
@@ -97,7 +98,7 @@ struct save_callbacks {
      * 0: terminate checkpointing gracefully
      * 1: take another checkpoint
      */
-    int (*checkpoint)(void* data);
+    int (*checkpoint)(void *data);
 
     /*
      * Called after the checkpoint callback.
@@ -106,13 +107,13 @@ struct save_callbacks {
      * 0: terminate checkpointing gracefully
      * 1: take another checkpoint
      */
-    int (*wait_checkpoint)(void* data);
+    int (*wait_checkpoint)(void *data);
 
     /* Enable qemu-dm logging dirty pages to xen */
     int (*switch_qemu_logdirty)(uint32_t domid, unsigned enable, void *data); /* HVM only */
 
     /* to be provided as the last argument to each callback function */
-    void* data;
+    void *data;
 };
 
 /* Type of stream.  Plain, or using a continuous replication protocol? */
@@ -134,27 +135,29 @@ typedef enum {
  */
 int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom,
                    uint32_t flags /* XCFLAGS_xxx */,
-                   struct save_callbacks* callbacks, int hvm,
+                   struct save_callbacks *callbacks, int hvm,
                    xc_stream_type_t stream_type, int recv_fd);
 
 /* callbacks provided by xc_domain_restore */
 struct restore_callbacks {
-    /* Called after a new checkpoint to suspend the guest.
-     */
-    int (*suspend)(void* data);
+    /* Called after a new checkpoint to suspend the guest. */
+    int (*suspend)(void *data);
 
-    /* Called after the secondary vm is ready to resume.
+    /*
+     * Called after the secondary vm is ready to resume.
      * Callback function resumes the guest & the device model,
      * returns to xc_domain_restore.
      */
-    int (*postcopy)(void* data);
+    int (*postcopy)(void *data);
 
-    /* A checkpoint record has been found in the stream.
-     * returns: */
+    /*
+     * A checkpoint record has been found in the stream.
+     * returns:
+     */
 #define XGR_CHECKPOINT_ERROR    0 /* Terminate processing */
 #define XGR_CHECKPOINT_SUCCESS  1 /* Continue reading more data from the stream */
 #define XGR_CHECKPOINT_FAILOVER 2 /* Failover and resume VM */
-    int (*checkpoint)(void* data);
+    int (*checkpoint)(void *data);
 
     /*
      * Called after the checkpoint callback.
@@ -163,7 +166,7 @@ struct restore_callbacks {
      * 0: terminate checkpointing gracefully
      * 1: take another checkpoint
      */
-    int (*wait_checkpoint)(void* data);
+    int (*wait_checkpoint)(void *data);
 
     /*
      * callback to send store gfn and console gfn to xl
@@ -174,7 +177,7 @@ struct restore_callbacks {
                             void *data);
 
     /* to be provided as the last argument to each callback function */
-    void* data;
+    void *data;
 };
 
 /**
diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
index f24dd0d00b..0e66f31aae 100644
--- a/tools/libxc/xc_sr_save.c
+++ b/tools/libxc/xc_sr_save.c
@@ -579,7 +579,7 @@ static int send_memory_live(struct xc_sr_context *ctx)
 static int colo_merge_secondary_dirty_bitmap(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
-    struct xc_sr_record rec = { 0, 0, NULL };
+    struct xc_sr_record rec;
     uint64_t *pfns = NULL;
     uint64_t pfn;
     unsigned int count, i;
@@ -593,14 +593,14 @@ static int colo_merge_secondary_dirty_bitmap(struct xc_sr_context *ctx)
 
     if ( rec.type != REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST )
     {
-        PERROR("Expect dirty bitmap record, but received %u", rec.type );
+        PERROR("Expect dirty bitmap record, but received %u", rec.type);
         rc = -1;
         goto err;
     }
 
     if ( rec.length % sizeof(*pfns) )
     {
-        PERROR("Invalid dirty pfn list record length %u", rec.length );
+        PERROR("Invalid dirty pfn list record length %u", rec.length);
         rc = -1;
         goto err;
     }
@@ -611,7 +611,7 @@ static int colo_merge_secondary_dirty_bitmap(struct xc_sr_context *ctx)
     for ( i = 0; i < count; i++ )
     {
         pfn = pfns[i];
-        if (pfn > ctx->save.p2m_size)
+        if ( pfn > ctx->save.p2m_size )
         {
             PERROR("Invalid pfn 0x%" PRIx64, pfn);
             rc = -1;
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 16/20] tools/libxl: Simplify callback handling in libxl-save-helper
  2020-01-03 13:06 ` [Xen-devel] [PATCH] Use CPUID/MSR data from migration streams Andrew Cooper
  2020-01-03 13:06   ` [Xen-devel] [PATCH 15/20] fixup tools/migration: Formatting and style cleanup Andrew Cooper
@ 2020-01-03 13:06   ` Andrew Cooper
  2020-01-14 17:27     ` Ian Jackson
  2020-01-03 13:06   ` [Xen-devel] [PATCH 17/20] tools/libx[cl]: Plumb static_data_done() up into libxl Andrew Cooper
                     ` (3 subsequent siblings)
  5 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2020-01-03 13:06 UTC (permalink / raw)
  To: Xen-devel; +Cc: Anthony PERARD, Andrew Cooper, Wei Liu, Ian Jackson

The {save,restore}_callback helpers can have their scope reduced vastly, and
helper_setcallbacks_{save,restore}() doesn't need to use a ternary operator to
write 0 (meaning NULL) into an already zeroed structure.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/libxl/libxl_save_helper.c    | 15 ++++++---------
 tools/libxl/libxl_save_msgs_gen.pl |  2 +-
 2 files changed, 7 insertions(+), 10 deletions(-)

diff --git a/tools/libxl/libxl_save_helper.c b/tools/libxl/libxl_save_helper.c
index 398df00dd6..a91f36db73 100644
--- a/tools/libxl/libxl_save_helper.c
+++ b/tools/libxl/libxl_save_helper.c
@@ -217,8 +217,6 @@ int helper_getreply(void *user)
 
 /*----- other callbacks -----*/
 
-static struct save_callbacks helper_save_callbacks;
-
 static void startup(const char *op) {
     xtl_log(&logger,XTL_DEBUG,0,program,"starting %s",op);
 
@@ -234,8 +232,6 @@ static void complete(int retval) {
     exit(0);
 }
 
-static struct restore_callbacks helper_restore_callbacks;
-
 int main(int argc, char **argv)
 {
     int r;
@@ -247,6 +243,7 @@ int main(int argc, char **argv)
     assert(mode);
 
     if (!strcmp(mode,"--save-domain")) {
+        static struct save_callbacks cb;
 
         io_fd =                             atoi(NEXTARG);
         recv_fd =                           atoi(NEXTARG);
@@ -257,16 +254,17 @@ int main(int argc, char **argv)
         xc_stream_type_t stream_type =      strtoul(NEXTARG,0,10);
         assert(!*++argv);
 
-        helper_setcallbacks_save(&helper_save_callbacks, cbflags);
+        helper_setcallbacks_save(&cb, cbflags);
 
         startup("save");
         setup_signals(save_signal_handler);
 
-        r = xc_domain_save(xch, io_fd, dom, flags, &helper_save_callbacks,
+        r = xc_domain_save(xch, io_fd, dom, flags, &cb,
                            hvm, stream_type, recv_fd);
         complete(r);
 
     } else if (!strcmp(mode,"--restore-domain")) {
+        static struct restore_callbacks cb;
 
         io_fd =                             atoi(NEXTARG);
         send_back_fd =                      atoi(NEXTARG);
@@ -281,7 +279,7 @@ int main(int argc, char **argv)
         xc_stream_type_t stream_type =      strtoul(NEXTARG,0,10);
         assert(!*++argv);
 
-        helper_setcallbacks_restore(&helper_restore_callbacks, cbflags);
+        helper_setcallbacks_restore(&cb, cbflags);
 
         unsigned long store_mfn = 0;
         unsigned long console_mfn = 0;
@@ -292,8 +290,7 @@ int main(int argc, char **argv)
         r = xc_domain_restore(xch, io_fd, dom, store_evtchn, &store_mfn,
                               store_domid, console_evtchn, &console_mfn,
                               console_domid, hvm, pae,
-                              stream_type,
-                              &helper_restore_callbacks, send_back_fd);
+                              stream_type, &cb, send_back_fd);
         helper_stub_restore_results(store_mfn,console_mfn,0);
         complete(r);
 
diff --git a/tools/libxl/libxl_save_msgs_gen.pl b/tools/libxl/libxl_save_msgs_gen.pl
index 6f1d79f821..831a15e0bb 100755
--- a/tools/libxl/libxl_save_msgs_gen.pl
+++ b/tools/libxl/libxl_save_msgs_gen.pl
@@ -333,7 +333,7 @@ END_ALWAYS
         my $c_v = "(1u<<$msgnum)";
         my $c_cb = "cbs->$name";
         $f_more_sr->("    if ($c_cb) cbflags |= $c_v;\n", $enumcallbacks);
-        $f_more_sr->("    $c_cb = (cbflags & $c_v) ? ${encode}_${name} : 0;\n",
+        $f_more_sr->("    if (cbflags & $c_v) $c_cb = ${encode}_${name};\n",
                      $setcallbacks);
     }
     $f_more_sr->("        return 1;\n    }\n\n");
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 17/20] tools/libx[cl]: Plumb static_data_done() up into libxl
  2020-01-03 13:06 ` [Xen-devel] [PATCH] Use CPUID/MSR data from migration streams Andrew Cooper
  2020-01-03 13:06   ` [Xen-devel] [PATCH 15/20] fixup tools/migration: Formatting and style cleanup Andrew Cooper
  2020-01-03 13:06   ` [Xen-devel] [PATCH 16/20] tools/libxl: Simplify callback handling in libxl-save-helper Andrew Cooper
@ 2020-01-03 13:06   ` Andrew Cooper
  2020-01-14 17:30     ` Ian Jackson
  2020-01-03 13:06   ` [Xen-devel] [PATCH 18/20] tools/libxl: Plumb domain_create_state down into libxl__build_pre() Andrew Cooper
                     ` (2 subsequent siblings)
  5 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2020-01-03 13:06 UTC (permalink / raw)
  To: Xen-devel; +Cc: Anthony PERARD, Andrew Cooper, Wei Liu, Ian Jackson

libxl is going to have to provide compatibility for pre 4.14 streams which
don't contain CPUID information.  Introduce the static_data_done() callback
and plumb it up into libxl.

No overall change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/libxc/include/xenguest.h     |  9 +++++++++
 tools/libxc/xc_sr_common_x86.c     | 13 ++++++++++++-
 tools/libxc/xc_sr_restore.c        |  1 +
 tools/libxl/libxl_create.c         | 13 +++++++++++++
 tools/libxl/libxl_save_msgs_gen.pl |  1 +
 5 files changed, 36 insertions(+), 1 deletion(-)

diff --git a/tools/libxc/include/xenguest.h b/tools/libxc/include/xenguest.h
index d81baa6cc2..be80544bd0 100644
--- a/tools/libxc/include/xenguest.h
+++ b/tools/libxc/include/xenguest.h
@@ -140,6 +140,15 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom,
 
 /* callbacks provided by xc_domain_restore */
 struct restore_callbacks {
+    /*
+     * Called once the STATIC_DATA_END record has been received/inferred.
+     * Passes in the blocks of static data which have not been received, which
+     * the higher level toolstack must provide backwards compatibility for.
+     */
+#define XGR_SDD_MISSING_CPUID (1 << 0)
+#define XGR_SDD_MISSING_MSR   (1 << 1)
+    int (*static_data_done)(unsigned int missing, void *data);
+
     /* Called after a new checkpoint to suspend the guest. */
     int (*suspend)(void *data);
 
diff --git a/tools/libxc/xc_sr_common_x86.c b/tools/libxc/xc_sr_common_x86.c
index 09fb1d93d6..c3d1d30d91 100644
--- a/tools/libxc/xc_sr_common_x86.c
+++ b/tools/libxc/xc_sr_common_x86.c
@@ -44,9 +44,20 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec)
 
 int x86_static_data_complete(struct xc_sr_context *ctx)
 {
+    xc_interface *xch = ctx->xch;
+    unsigned int missing = 0;
+    int rc;
+
     /* TODO - something useful. */
+    missing = XGR_SDD_MISSING_MSR | XGR_SDD_MISSING_CPUID;
 
-    return 0;
+    rc = ctx->restore.callbacks->static_data_done(
+        missing, ctx->restore.callbacks->data);
+
+    if ( rc )
+        ERROR("static_data_done() callback failed: %d\n", rc);
+
+    return rc;
 }
 
 int write_x86_cpu_policy_records(struct xc_sr_context *ctx)
diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c
index d4bd60a31e..09d8a08316 100644
--- a/tools/libxc/xc_sr_restore.c
+++ b/tools/libxc/xc_sr_restore.c
@@ -908,6 +908,7 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom,
         assert(callbacks->checkpoint);
         /* Fallthrough */
     case XC_STREAM_PLAIN:
+        assert(callbacks->static_data_done);
         break;
 
     default:
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 32d45dcef0..fdc76917dc 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -1227,6 +1227,7 @@ static void domcreate_bootloader_done(libxl__egc *egc,
     dcs->srs.dcs = dcs;
 
     /* Restore */
+    callbacks->static_data_done = libxl__srm_callout_callback_static_data_done;
     callbacks->restore_results = libxl__srm_callout_callback_restore_results;
 
     /* COLO only supports HVM now because it does not work very
@@ -1296,6 +1297,18 @@ static void libxl__colo_restore_setup_done(libxl__egc *egc,
     libxl__stream_read_start(egc, &dcs->srs);
 }
 
+int libxl__srm_callout_callback_static_data_done(unsigned int missing,
+                                                 void *user)
+{
+    libxl__save_helper_state *shs = user;
+    libxl__domain_create_state *dcs = shs->caller_state;
+    STATE_AO_GC(dcs->ao);
+
+    /* TODO - something useful. */
+
+    return 0;
+}
+
 void libxl__srm_callout_callback_restore_results(xen_pfn_t store_mfn,
           xen_pfn_t console_mfn, void *user)
 {
diff --git a/tools/libxl/libxl_save_msgs_gen.pl b/tools/libxl/libxl_save_msgs_gen.pl
index 831a15e0bb..5bfbd4fd10 100755
--- a/tools/libxl/libxl_save_msgs_gen.pl
+++ b/tools/libxl/libxl_save_msgs_gen.pl
@@ -29,6 +29,7 @@ our @msgs = (
     [ 'srcxA',  "wait_checkpoint", [] ],
     [ 'scxA',   "switch_qemu_logdirty",  [qw(uint32_t domid
                                           unsigned enable)] ],
+    [ 'rcxW',   "static_data_done",      [qw(unsigned missing)] ],
     [ 'rcx',    "restore_results",       ['xen_pfn_t', 'store_gfn',
                                           'xen_pfn_t', 'console_gfn'] ],
     [ 'srW',    "complete",              [qw(int retval
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 18/20] tools/libxl: Plumb domain_create_state down into libxl__build_pre()
  2020-01-03 13:06 ` [Xen-devel] [PATCH] Use CPUID/MSR data from migration streams Andrew Cooper
                     ` (2 preceding siblings ...)
  2020-01-03 13:06   ` [Xen-devel] [PATCH 17/20] tools/libx[cl]: Plumb static_data_done() up into libxl Andrew Cooper
@ 2020-01-03 13:06   ` Andrew Cooper
  2020-01-14 17:32     ` Ian Jackson
  2020-01-03 13:06   ` [Xen-devel] [PATCH 19/20] tools/libxl: Re-position CPUID handling during domain construction Andrew Cooper
  2020-01-03 13:06   ` [Xen-devel] [PATCH 20/20] tools/libxc: Restore CPUID/MSR data found in the migration stream Andrew Cooper
  5 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2020-01-03 13:06 UTC (permalink / raw)
  To: Xen-devel; +Cc: Anthony PERARD, Andrew Cooper, Wei Liu, Ian Jackson

To fix CPUID handling, libxl__build_pre() is going to have to distinguish
between a brand new VM vs one which is being migrated-in/resumed.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>

The data handling is completely chaotic, but I can't find a less bad way of
doing this.  It really doesn't help that DCS is the superset contining
domain_{build,create}_config and domain_build_state, rather than being
symetric with its similarly named counterparts.

Suggestions welcome.
---
 tools/libxl/libxl_create.c   | 16 ++++++++--------
 tools/libxl/libxl_dm.c       |  5 +++--
 tools/libxl/libxl_dom.c      |  4 +++-
 tools/libxl/libxl_internal.h | 11 ++++-------
 4 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index fdc76917dc..a8fa4ca11b 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -431,17 +431,17 @@ static void init_console_info(libxl__gc *gc,
        Only 'channels' when mapped to consoles have a string name. */
 }
 
-int libxl__domain_build(libxl__gc *gc,
-                        libxl_domain_config *d_config,
-                        uint32_t domid,
-                        libxl__domain_build_state *state)
+int libxl__domain_build(libxl__gc *gc, uint32_t domid,
+                        libxl__domain_create_state *dcs)
 {
+    libxl_domain_config *d_config = dcs->guest_config;
     libxl_domain_build_info *const info = &d_config->b_info;
+    libxl__domain_build_state *state = &dcs->build_state;
     char **vments = NULL, **localents = NULL;
     struct timeval start_time;
     int i, ret;
 
-    ret = libxl__build_pre(gc, domid, d_config, state);
+    ret = libxl__build_pre(gc, domid, dcs);
     if (ret)
         goto out;
 
@@ -1218,7 +1218,7 @@ static void domcreate_bootloader_done(libxl__egc *egc,
     dcs->sdss.callback = domcreate_devmodel_started;
 
     if (restore_fd < 0 && dcs->domid_soft_reset == INVALID_DOMID) {
-        rc = libxl__domain_build(gc, d_config, domid, state);
+        rc = libxl__domain_build(gc, domid, dcs);
         domcreate_rebuild_done(egc, dcs, rc);
         return;
     }
@@ -1245,7 +1245,7 @@ static void domcreate_bootloader_done(libxl__egc *egc,
         goto out;
     }
 
-    rc = libxl__build_pre(gc, domid, d_config, state);
+    rc = libxl__build_pre(gc, domid, dcs);
     if (rc)
         goto out;
 
@@ -1567,7 +1567,7 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *multidev,
 
         dcs->sdss.dm.guest_domid = domid;
         if (libxl_defbool_val(d_config->b_info.device_model_stubdomain))
-            libxl__spawn_stub_dm(egc, &dcs->sdss);
+            libxl__spawn_stub_dm(egc, dcs);
         else
             libxl__spawn_local_dm(egc, &dcs->sdss.dm);
 
diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index dac1b8ddb8..b595dc03b3 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -2094,8 +2094,9 @@ char *libxl__stub_dm_name(libxl__gc *gc, const char *guest_name)
     return GCSPRINTF("%s-dm", guest_name);
 }
 
-void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss)
+void libxl__spawn_stub_dm(libxl__egc *egc, libxl__domain_create_state *dcs)
 {
+    libxl__stub_dm_spawn_state *sdss = &dcs->sdss;
     STATE_AO_GC(sdss->dm.spawn.ao);
     libxl_ctx *ctx = libxl__gc_owner(gc);
     int ret;
@@ -2194,7 +2195,7 @@ void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss)
     if (ret)
         goto out;
     uint32_t dm_domid = sdss->pvqemu.guest_domid;
-    ret = libxl__domain_build(gc, dm_config, dm_domid, stubdom_state);
+    ret = libxl__domain_build(gc, dm_domid, dcs);
     if (ret)
         goto out;
 
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index 573c63692b..01aefa6ce4 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -244,9 +244,11 @@ static int numa_place_domain(libxl__gc *gc, uint32_t domid,
 }
 
 int libxl__build_pre(libxl__gc *gc, uint32_t domid,
-              libxl_domain_config *d_config, libxl__domain_build_state *state)
+                     libxl__domain_create_state *dcs)
 {
+    libxl_domain_config *d_config = dcs->guest_config;
     libxl_domain_build_info *const info = &d_config->b_info;
+    libxl__domain_build_state *state = &dcs->build_state;
     libxl_ctx *ctx = libxl__gc_owner(gc);
     char *xs_domid, *con_domid;
     int rc;
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index b5adbfe4b7..16dc02e8a3 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1389,8 +1389,7 @@ _hidden void libxl__domain_build_state_init(libxl__domain_build_state *s);
 _hidden void libxl__domain_build_state_dispose(libxl__domain_build_state *s);
 
 _hidden int libxl__build_pre(libxl__gc *gc, uint32_t domid,
-              libxl_domain_config * const d_config,
-              libxl__domain_build_state *state);
+                             libxl__domain_create_state *dcs);
 _hidden int libxl__build_post(libxl__gc *gc, uint32_t domid,
                libxl_domain_build_info *info, libxl__domain_build_state *state,
                char **vms_ents, char **local_ents);
@@ -1960,10 +1959,8 @@ _hidden int libxl__domain_make(libxl__gc *gc,
                                libxl__domain_build_state *state,
                                uint32_t *domid);
 
-_hidden int libxl__domain_build(libxl__gc *gc,
-                                libxl_domain_config *d_config,
-                                uint32_t domid,
-                                libxl__domain_build_state *state);
+_hidden int libxl__domain_build(libxl__gc *gc, uint32_t domid,
+                                libxl__domain_create_state *dcs);
 
 /* for device model creation */
 _hidden const char *libxl__domain_device_model(libxl__gc *gc,
@@ -4114,7 +4111,7 @@ typedef struct {
     libxl__xswait_state xswait;
 } libxl__stub_dm_spawn_state;
 
-_hidden void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state*);
+_hidden void libxl__spawn_stub_dm(libxl__egc *egc, libxl__domain_create_state *dcs);
 
 _hidden char *libxl__stub_dm_name(libxl__gc *gc, const char * guest_name);
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 19/20] tools/libxl: Re-position CPUID handling during domain construction
  2020-01-03 13:06 ` [Xen-devel] [PATCH] Use CPUID/MSR data from migration streams Andrew Cooper
                     ` (3 preceding siblings ...)
  2020-01-03 13:06   ` [Xen-devel] [PATCH 18/20] tools/libxl: Plumb domain_create_state down into libxl__build_pre() Andrew Cooper
@ 2020-01-03 13:06   ` Andrew Cooper
  2020-01-14 17:33     ` Ian Jackson
  2020-01-03 13:06   ` [Xen-devel] [PATCH 20/20] tools/libxc: Restore CPUID/MSR data found in the migration stream Andrew Cooper
  5 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2020-01-03 13:06 UTC (permalink / raw)
  To: Xen-devel; +Cc: Anthony PERARD, Andrew Cooper, Wei Liu, Ian Jackson

CPUID handling needs to be earlier in construction.  Move it from its current
position in libxl__build_post() to libxl__build_pre() for fresh builds, and
libxl__srm_callout_callback_static_data_done() for the migration/resume case.

In the migration case, take account of XGR_SDD_MISSING_CPUID.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
CC: Anthony PERARD <anthony.perard@citrix.com>
---
 tools/libxl/libxl_create.c | 19 ++++++++++++++++++-
 tools/libxl/libxl_dom.c    | 12 ++++++++----
 2 files changed, 26 insertions(+), 5 deletions(-)

diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index a8fa4ca11b..0d9a91aeeb 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -1303,8 +1303,25 @@ int libxl__srm_callout_callback_static_data_done(unsigned int missing,
     libxl__save_helper_state *shs = user;
     libxl__domain_create_state *dcs = shs->caller_state;
     STATE_AO_GC(dcs->ao);
+    libxl_ctx *ctx = libxl__gc_owner(gc);
+
+    const libxl_domain_config *d_config = dcs->guest_config;
+    const libxl_domain_build_info *info = &d_config->b_info;
 
-    /* TODO - something useful. */
+    /*
+     * For pre-4.14 streams, no CPUID/MSR information will be sent.  Retain
+     * the old behaviour by regenerating CPUID from first principles.
+     */
+    if (missing & XGR_SDD_MISSING_CPUID)
+        libxl_cpuid_apply_policy(ctx, dcs->guest_domid);
+
+    /*
+     * In all cases, cpuid=[] needs re-evaluating.  The common case is that it
+     * will match libxl_cpuid_apply_policy() and/or whatever is in the stream,
+     * but the legacy XEND 'k' modifier passes through host values.
+     */
+    if (info->cpuid != NULL)
+        libxl_cpuid_set(ctx, dcs->guest_domid, info->cpuid);
 
     return 0;
 }
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index 01aefa6ce4..099a913019 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -254,6 +254,8 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,
     int rc;
     uint64_t size;
 
+    const int restore_fd = dcs->restore_fd;
+
     if (xc_domain_max_vcpus(ctx->xch, domid, info->max_vcpus) != 0) {
         LOG(ERROR, "Couldn't set max vcpu count");
         return ERROR_FAIL;
@@ -376,6 +378,12 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,
         return ERROR_FAIL;
     }
 
+    if (restore_fd < 0) {
+        libxl_cpuid_apply_policy(ctx, domid);
+        if (info->cpuid != NULL)
+            libxl_cpuid_set(ctx, domid, info->cpuid);
+    }
+
     xs_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenstored/domid", NULL);
     state->store_domid = xs_domid ? atoi(xs_domid) : 0;
     free(xs_domid);
@@ -456,10 +464,6 @@ int libxl__build_post(libxl__gc *gc, uint32_t domid,
     if (rc)
         return rc;
 
-    libxl_cpuid_apply_policy(ctx, domid);
-    if (info->cpuid != NULL)
-        libxl_cpuid_set(ctx, domid, info->cpuid);
-
     if (info->type == LIBXL_DOMAIN_TYPE_HVM
         && !libxl_ms_vm_genid_is_zero(&info->u.hvm.ms_vm_genid)) {
         rc = libxl__ms_vm_genid_set(gc, domid,
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 20/20] tools/libxc: Restore CPUID/MSR data found in the migration stream
  2020-01-03 13:06 ` [Xen-devel] [PATCH] Use CPUID/MSR data from migration streams Andrew Cooper
                     ` (4 preceding siblings ...)
  2020-01-03 13:06   ` [Xen-devel] [PATCH 19/20] tools/libxl: Re-position CPUID handling during domain construction Andrew Cooper
@ 2020-01-03 13:06   ` Andrew Cooper
  2020-01-14 17:34     ` Ian Jackson
  5 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2020-01-03 13:06 UTC (permalink / raw)
  To: Xen-devel; +Cc: Andrew Cooper, Wei Liu, Ian Jackson

With libxl suitably adjusted, it is now safe to restore the CPUID/MSR data
directly from the migration stream.  Adjust the XGR_SDD_MISSING_* flags for
the static_data_done() callback appropriately.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 tools/libxc/xc_sr_common_x86.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/tools/libxc/xc_sr_common_x86.c b/tools/libxc/xc_sr_common_x86.c
index c3d1d30d91..5c5b3e7635 100644
--- a/tools/libxc/xc_sr_common_x86.c
+++ b/tools/libxc/xc_sr_common_x86.c
@@ -45,11 +45,31 @@ int handle_x86_tsc_info(struct xc_sr_context *ctx, struct xc_sr_record *rec)
 int x86_static_data_complete(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
+    uint32_t nr_leaves = 0, nr_msrs = 0;
+    uint32_t err_l = ~0, err_s = ~0, err_m = ~0;
     unsigned int missing = 0;
     int rc;
 
-    /* TODO - something useful. */
-    missing = XGR_SDD_MISSING_MSR | XGR_SDD_MISSING_CPUID;
+    if ( ctx->x86.restore.cpuid.ptr )
+        nr_leaves = ctx->x86.restore.cpuid.size / sizeof(xen_cpuid_leaf_t);
+    else
+        missing |= XGR_SDD_MISSING_CPUID;
+
+    if ( ctx->x86.restore.msr.ptr )
+        nr_msrs = ctx->x86.restore.msr.size / sizeof(xen_msr_entry_t);
+    else
+        missing |= XGR_SDD_MISSING_MSR;
+
+    if ( (nr_leaves || nr_msrs) &&
+         xc_set_domain_cpu_policy(xch, ctx->domid,
+                                  nr_leaves, ctx->x86.restore.cpuid.ptr,
+                                  nr_msrs,   ctx->x86.restore.msr.ptr,
+                                  &err_l, &err_s, &err_m) )
+    {
+        PERROR("Failed to set CPUID policy: leaf %08x, subleaf %08x, msr %08x",
+               err_l, err_s, err_m);
+        return -1;
+    }
 
     rc = ctx->restore.callbacks->static_data_done(
         missing, ctx->restore.callbacks->data);
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END
  2019-12-24 15:19 ` [Xen-devel] [PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END Andrew Cooper
@ 2020-01-03 14:44   ` Jan Beulich
  2020-01-09 14:54     ` Andrew Cooper
  2020-01-14 16:07   ` Ian Jackson
  1 sibling, 1 reply; 60+ messages in thread
From: Jan Beulich @ 2020-01-03 14:44 UTC (permalink / raw)
  To: Andrew Cooper
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Marek Marczykowski-Górecki, Ian Jackson,
	Xen-devel

On 24.12.2019 16:19, Andrew Cooper wrote:
> Migration data can be split into two parts - that which is invariant of
> guest execution, and that which is not.  Separate these two with the
> STATIC_DATA_END record.
> 
> The short term, we want to move the x86 CPU Policy data into the stream.
> In the longer term, we want to provisionally send the static data only
> to the destination as a more robust compatibility check.  In both cases,
> we will want a callback into the higher level toolstack.
> 
> Mandate the presence of the STATIC_DATA_END record, and declare this v3,
> along with instructions for how to compatibly interpret a v2 stream.

What doesn't become clear (to me) from all of the above is why this
record is needed (wanted), and hence why it is to be mandatory.
After all ...

> @@ -675,9 +694,23 @@ A typical save record for an x86 HVM guest image would look like:
>  HVM_PARAMS must precede HVM_CONTEXT, as certain parameters can affect
>  the validity of architectural state in the context.
>  
> +Compatibility with older versions
> +=================================
> +
> +v3 compat with v2
> +-----------------
> +
> +A v3 stream is compatible with a v2 stream, but mandates the presense of a
> +STATIC_DATA_END record ahead of any memory/register content.  This is to ease
> +the introduction of new static configuration records over time.
> +
> +A v3-compatible reciever interpreting a v2 stream should infer the position of
> +STATIC_DATA_END based on finding the first X86_PV_P2M_FRAMES record (for PV
> +guests), or PAGE_DATA record (for HVM guests) and behave as if STATIC_DATA_END
> +had been sent.

... you already imply a v3 receiver can deal with its absence.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records
  2019-12-24 15:19 ` [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records Andrew Cooper
@ 2020-01-03 14:49   ` Jan Beulich
  2020-01-03 14:55     ` Andrew Cooper
  2020-01-14 16:08   ` Ian Jackson
  1 sibling, 1 reply; 60+ messages in thread
From: Jan Beulich @ 2020-01-03 14:49 UTC (permalink / raw)
  To: Andrew Cooper
  Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Tim Deegan, Marek Marczykowski-Górecki,
	Julien Grall, Ian Jackson, Xen-devel

On 24.12.2019 16:19, Andrew Cooper wrote:
> @@ -439,6 +449,34 @@ def verify_record_static_data_end(self, content):
>              raise RecordError("Static data end record found in v2 stream")
>  
>  
> +    def verify_record_x86_cpuid_policy(self, content):
> +        """ x86 CPUID policy record """
> +
> +        if self.version < 3:
> +            raise RecordError("x86 CPUID policy record found in v2 stream")
> +
> +        sz = calcsize(X86_CPUID_POLICY_FORMAT)
> +        contentsz = len(content)
> +
> +        if contentsz < sz or (contentsz % sz) != 0:
> +            raise RecordError("Record length %u, expected multiple of %u" %
> +                              (contentsz, sz))
> +
> +
> +    def verify_record_x86_msr_policy(self, content):
> +        """ x86 MSR policy record """
> +
> +        if self.version < 3:
> +            raise RecordError("x86 MSR policy record found in v2 stream")
> +
> +        sz = calcsize(X86_MSR_POLICY_FORMAT)
> +        contentsz = len(content)
> +
> +        if contentsz < sz or (contentsz % sz) != 0:
> +            raise RecordError("Record length %u, expected multiple of %u" %
> +                              (contentsz, sz))

While I can't even see a theoretical case of the CPUID array
having zero elements, is it really entirely implausible to have
an empty MSRs array? I.e. wouldn't the left side of the "or"
better go away?

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records
  2020-01-03 14:49   ` Jan Beulich
@ 2020-01-03 14:55     ` Andrew Cooper
  2020-01-03 15:30       ` Jan Beulich
  0 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2020-01-03 14:55 UTC (permalink / raw)
  To: Jan Beulich
  Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Tim Deegan, Marek Marczykowski-Górecki,
	Julien Grall, Ian Jackson, Xen-devel

On 03/01/2020 14:49, Jan Beulich wrote:
> On 24.12.2019 16:19, Andrew Cooper wrote:
>> @@ -439,6 +449,34 @@ def verify_record_static_data_end(self, content):
>>              raise RecordError("Static data end record found in v2 stream")
>>  
>>  
>> +    def verify_record_x86_cpuid_policy(self, content):
>> +        """ x86 CPUID policy record """
>> +
>> +        if self.version < 3:
>> +            raise RecordError("x86 CPUID policy record found in v2 stream")
>> +
>> +        sz = calcsize(X86_CPUID_POLICY_FORMAT)
>> +        contentsz = len(content)
>> +
>> +        if contentsz < sz or (contentsz % sz) != 0:
>> +            raise RecordError("Record length %u, expected multiple of %u" %
>> +                              (contentsz, sz))
>> +
>> +
>> +    def verify_record_x86_msr_policy(self, content):
>> +        """ x86 MSR policy record """
>> +
>> +        if self.version < 3:
>> +            raise RecordError("x86 MSR policy record found in v2 stream")
>> +
>> +        sz = calcsize(X86_MSR_POLICY_FORMAT)
>> +        contentsz = len(content)
>> +
>> +        if contentsz < sz or (contentsz % sz) != 0:
>> +            raise RecordError("Record length %u, expected multiple of %u" %
>> +                              (contentsz, sz))
> While I can't even see a theoretical case of the CPUID array
> having zero elements, is it really entirely implausible to have
> an empty MSRs array? I.e. wouldn't the left side of the "or"
> better go away?

MSRs will never have 0 entries, because unlike CPUID, we can't omit
records with 0s as their content.  This becomes ambiguous when the
policy default is nonzero.

When we do gain more MSRs, I will see about organising elision based on
CPUID features, so we don't have to send a 0 for every single MSR in the
policy, but MSRs without CPUID enumeration must always be sent.

This means that the one MSR we have currently (MSR_INTEL_PLATFORM_INFO
for CPUID Faulting, which we also virtualise on AMD hardware) shall
unconditionally be present forever more.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records
  2020-01-03 14:55     ` Andrew Cooper
@ 2020-01-03 15:30       ` Jan Beulich
  2020-01-09 15:30         ` Andrew Cooper
  0 siblings, 1 reply; 60+ messages in thread
From: Jan Beulich @ 2020-01-03 15:30 UTC (permalink / raw)
  To: Andrew Cooper
  Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Tim Deegan, Marek Marczykowski-Górecki,
	Julien Grall, Ian Jackson, Xen-devel

On 03.01.2020 15:55, Andrew Cooper wrote:
> On 03/01/2020 14:49, Jan Beulich wrote:
>> On 24.12.2019 16:19, Andrew Cooper wrote:
>>> @@ -439,6 +449,34 @@ def verify_record_static_data_end(self, content):
>>>              raise RecordError("Static data end record found in v2 stream")
>>>  
>>>  
>>> +    def verify_record_x86_cpuid_policy(self, content):
>>> +        """ x86 CPUID policy record """
>>> +
>>> +        if self.version < 3:
>>> +            raise RecordError("x86 CPUID policy record found in v2 stream")
>>> +
>>> +        sz = calcsize(X86_CPUID_POLICY_FORMAT)
>>> +        contentsz = len(content)
>>> +
>>> +        if contentsz < sz or (contentsz % sz) != 0:
>>> +            raise RecordError("Record length %u, expected multiple of %u" %
>>> +                              (contentsz, sz))
>>> +
>>> +
>>> +    def verify_record_x86_msr_policy(self, content):
>>> +        """ x86 MSR policy record """
>>> +
>>> +        if self.version < 3:
>>> +            raise RecordError("x86 MSR policy record found in v2 stream")
>>> +
>>> +        sz = calcsize(X86_MSR_POLICY_FORMAT)
>>> +        contentsz = len(content)
>>> +
>>> +        if contentsz < sz or (contentsz % sz) != 0:
>>> +            raise RecordError("Record length %u, expected multiple of %u" %
>>> +                              (contentsz, sz))
>> While I can't even see a theoretical case of the CPUID array
>> having zero elements, is it really entirely implausible to have
>> an empty MSRs array? I.e. wouldn't the left side of the "or"
>> better go away?
> 
> MSRs will never have 0 entries, because unlike CPUID, we can't omit
> records with 0s as their content.  This becomes ambiguous when the
> policy default is nonzero.

Isn't the same true for CPUID, in particular some of the non-boolean
fields?

> When we do gain more MSRs, I will see about organising elision based on
> CPUID features, so we don't have to send a 0 for every single MSR in the
> policy, but MSRs without CPUID enumeration must always be sent.
> 
> This means that the one MSR we have currently (MSR_INTEL_PLATFORM_INFO
> for CPUID Faulting, which we also virtualise on AMD hardware) shall
> unconditionally be present forever more.

Hmm, yes. Still the special casing of there needing to be at least
one entry looks a little odd here (and also for CPUID). I would
find it more logical if there was just the remainder-must-be-zero
check. But this is libxc code, so I'm not the one to really judge
anyway.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END
  2020-01-03 14:44   ` Jan Beulich
@ 2020-01-09 14:54     ` Andrew Cooper
  0 siblings, 0 replies; 60+ messages in thread
From: Andrew Cooper @ 2020-01-09 14:54 UTC (permalink / raw)
  To: Jan Beulich
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Marek Marczykowski-Górecki, Ian Jackson,
	Xen-devel

On 03/01/2020 14:44, Jan Beulich wrote:
> On 24.12.2019 16:19, Andrew Cooper wrote:
>> Migration data can be split into two parts - that which is invariant of
>> guest execution, and that which is not.  Separate these two with the
>> STATIC_DATA_END record.
>>
>> The short term, we want to move the x86 CPU Policy data into the stream.
>> In the longer term, we want to provisionally send the static data only
>> to the destination as a more robust compatibility check.  In both cases,
>> we will want a callback into the higher level toolstack.
>>
>> Mandate the presence of the STATIC_DATA_END record, and declare this v3,
>> along with instructions for how to compatibly interpret a v2 stream.
> What doesn't become clear (to me) from all of the above is why this
> record is needed (wanted), and hence why it is to be mandatory.
> After all ...
>
>> @@ -675,9 +694,23 @@ A typical save record for an x86 HVM guest image would look like:
>>  HVM_PARAMS must precede HVM_CONTEXT, as certain parameters can affect
>>  the validity of architectural state in the context.
>>  
>> +Compatibility with older versions
>> +=================================
>> +
>> +v3 compat with v2
>> +-----------------
>> +
>> +A v3 stream is compatible with a v2 stream, but mandates the presense of a
>> +STATIC_DATA_END record ahead of any memory/register content.  This is to ease
>> +the introduction of new static configuration records over time.
>> +
>> +A v3-compatible reciever interpreting a v2 stream should infer the position of
>> +STATIC_DATA_END based on finding the first X86_PV_P2M_FRAMES record (for PV
>> +guests), or PAGE_DATA record (for HVM guests) and behave as if STATIC_DATA_END
>> +had been sent.
> ... you already imply a v3 receiver can deal with its absence.

It provides an easier way to reason about the stream, as new features
get added, and this is helpful when debugging.

Most importantly however, it offers a clean way to deprecate/remove
support for obsolete senders.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records
  2020-01-03 15:30       ` Jan Beulich
@ 2020-01-09 15:30         ` Andrew Cooper
  2020-01-14 16:12           ` Ian Jackson
  0 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2020-01-09 15:30 UTC (permalink / raw)
  To: Jan Beulich
  Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Tim Deegan, Marek Marczykowski-Górecki,
	Julien Grall, Ian Jackson, Xen-devel

On 03/01/2020 15:30, Jan Beulich wrote:
> On 03.01.2020 15:55, Andrew Cooper wrote:
>> On 03/01/2020 14:49, Jan Beulich wrote:
>>> On 24.12.2019 16:19, Andrew Cooper wrote:
>>>> @@ -439,6 +449,34 @@ def verify_record_static_data_end(self, content):
>>>>              raise RecordError("Static data end record found in v2 stream")
>>>>  
>>>>  
>>>> +    def verify_record_x86_cpuid_policy(self, content):
>>>> +        """ x86 CPUID policy record """
>>>> +
>>>> +        if self.version < 3:
>>>> +            raise RecordError("x86 CPUID policy record found in v2 stream")
>>>> +
>>>> +        sz = calcsize(X86_CPUID_POLICY_FORMAT)
>>>> +        contentsz = len(content)
>>>> +
>>>> +        if contentsz < sz or (contentsz % sz) != 0:
>>>> +            raise RecordError("Record length %u, expected multiple of %u" %
>>>> +                              (contentsz, sz))
>>>> +
>>>> +
>>>> +    def verify_record_x86_msr_policy(self, content):
>>>> +        """ x86 MSR policy record """
>>>> +
>>>> +        if self.version < 3:
>>>> +            raise RecordError("x86 MSR policy record found in v2 stream")
>>>> +
>>>> +        sz = calcsize(X86_MSR_POLICY_FORMAT)
>>>> +        contentsz = len(content)
>>>> +
>>>> +        if contentsz < sz or (contentsz % sz) != 0:
>>>> +            raise RecordError("Record length %u, expected multiple of %u" %
>>>> +                              (contentsz, sz))
>>> While I can't even see a theoretical case of the CPUID array
>>> having zero elements, is it really entirely implausible to have
>>> an empty MSRs array? I.e. wouldn't the left side of the "or"
>>> better go away?
>> MSRs will never have 0 entries, because unlike CPUID, we can't omit
>> records with 0s as their content.  This becomes ambiguous when the
>> policy default is nonzero.
> Isn't the same true for CPUID, in particular some of the non-boolean
> fields?

I perhaps misspoke.  We can omit CPUID leave(s) based on information
already sent (max_leaf and/or the absence of an enumerating feature). 
In these cases, the destination side can still fill the remainder in
with 0's.

Top level leaves without an encompassing feature do need sending in full.

The best practical example is for not sending all 63 subleaves of the
xsave state leaf, when most of them are outside of the policy XCR0|XSS bits.

>
>> When we do gain more MSRs, I will see about organising elision based on
>> CPUID features, so we don't have to send a 0 for every single MSR in the
>> policy, but MSRs without CPUID enumeration must always be sent.
>>
>> This means that the one MSR we have currently (MSR_INTEL_PLATFORM_INFO
>> for CPUID Faulting, which we also virtualise on AMD hardware) shall
>> unconditionally be present forever more.
> Hmm, yes. Still the special casing of there needing to be at least
> one entry looks a little odd here (and also for CPUID). I would
> find it more logical if there was just the remainder-must-be-zero
> check. But this is libxc code, so I'm not the one to really judge
> anyway.

The migration stream is split into records with no playload (markers
with external control flow meaning), and data records, which have a payload.

It is an error for a data record to have no payload, because it means
there is a source side generation bug.  In the case of Xen returning 0
MSRs, the record would be omitted entirely, rather than be sent with 0
MSRs worth of data.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 03/12] libxc/migration: Rationalise the 'checkpointed' field to 'stream_type'
  2019-12-24 15:19 ` [Xen-devel] [PATCH 03/12] libxc/migration: Rationalise the 'checkpointed' field to 'stream_type' Andrew Cooper
@ 2020-01-14 15:58   ` Ian Jackson
  0 siblings, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 15:58 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Xen-devel, Wei Liu

Andrew Cooper writes ("[PATCH 03/12] libxc/migration: Rationalise the 'checkpointed' field to 'stream_type'"):
> Originally, 'checkpointed' was a boolean signalling the difference between a
> plain and a Remus stream.  COLO was added later, but several bits of code
> retained boolean-style logic.  While correct, it is confusing to follow.
> 
> Additionally, XC_MIG_STREAM_NONE means "no checkpoints" but reads as "no
> stream".
> 
> Consolidate all the logic on the term 'stream_type', and rename STREAM_NONE
> to STREAM_PLAIN.  Re-position the stream_type variable so it isn't
> duplicated in both the save and restore unions.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 04/12] libxc/migration: Adjust layout of struct xc_sr_context
  2019-12-24 15:19 ` [Xen-devel] [PATCH 04/12] libxc/migration: Adjust layout of struct xc_sr_context Andrew Cooper
@ 2020-01-14 16:04   ` Ian Jackson
  0 siblings, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 16:04 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Xen-devel, Wei Liu

Andrew Cooper writes ("[PATCH 04/12] libxc/migration: Adjust layout of struct xc_sr_context"):
> We are shortly going to want to introduce some common x86 fields, so having
> x86_pv and x86_hvm as the top level objects is a problem.  Insert a
> surrounding struct x86 and drop the x86 prefix from the pv/hvm objects.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 05/12] tools/migration: Drop IHDR_VERSION constant from libxc and python
  2019-12-24 15:19 ` [Xen-devel] [PATCH 05/12] tools/migration: Drop IHDR_VERSION constant from libxc and python Andrew Cooper
@ 2020-01-14 16:05   ` Ian Jackson
  2020-01-15 15:29     ` Andrew Cooper
  0 siblings, 1 reply; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 16:05 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Xen-devel, Marek Marczykowski-Górecki, Wei Liu

Andrew Cooper writes ("[PATCH 05/12] tools/migration: Drop IHDR_VERSION constant from libxc and python"):
> Migration v3 is in the process of being introduced, meaning that the code has
> to cope with both versions.  Use an explicit 2 for now.
> 
> For the verify-stream-v2 and convert-legacy-stream scripts, update text to say
> "v2 (or later)".  What matters is the distinction vs legacy streams.

How about introducing
  enum { IHDR_VERSION_2 = 2 }
or some such ?

In C it can be hard otherwise to find all the relevant tests.  Being
able to grep for IHDR_VERSION would help.  So I would prefer manifest
constants of some kind to unvarnished integers.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END
  2019-12-24 15:19 ` [Xen-devel] [PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END Andrew Cooper
  2020-01-03 14:44   ` Jan Beulich
@ 2020-01-14 16:07   ` Ian Jackson
  1 sibling, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 16:07 UTC (permalink / raw)
  To: Andrew Cooper
  Cc: Stefano Stabellini, Julien Grall, Wei Liu, Konrad Rzeszutek Wilk,
	George Dunlap, Marek Marczykowski-Górecki, Jan Beulich,
	Xen-devel

Andrew Cooper writes ("[PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END"):
> Migration data can be split into two parts - that which is invariant of
> guest execution, and that which is not.  Separate these two with the
> STATIC_DATA_END record.
> 
> The short term, we want to move the x86 CPU Policy data into the stream.
> In the longer term, we want to provisionally send the static data only
> to the destination as a more robust compatibility check.  In both cases,
> we will want a callback into the higher level toolstack.
> 
> Mandate the presence of the STATIC_DATA_END record, and declare this v3,
> along with instructions for how to compatibly interpret a v2 stream.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records
  2019-12-24 15:19 ` [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records Andrew Cooper
  2020-01-03 14:49   ` Jan Beulich
@ 2020-01-14 16:08   ` Ian Jackson
  2020-01-15 15:36     ` Andrew Cooper
  1 sibling, 1 reply; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 16:08 UTC (permalink / raw)
  To: Andrew Cooper
  Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk, Tim (Xen.org),
	George Dunlap, Marek Marczykowski-Górecki, Julien Grall,
	Jan Beulich, Xen-devel

Andrew Cooper writes ("[PATCH 10/12] docs/migration: Specify X86_{CPUID,MSR}_POLICY records"):
> These two records move blobs from the XEN_DOMCTL_{get,set}_cpu_policy
> hypercall.

We had an extensive IRL discussion recently about the compatibility
implications of this.  Is that written down somewhere ?  I was
expecting to see it in this patch.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records
  2020-01-09 15:30         ` Andrew Cooper
@ 2020-01-14 16:12           ` Ian Jackson
  2020-01-15 15:48             ` Andrew Cooper
  0 siblings, 1 reply; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 16:12 UTC (permalink / raw)
  To: Andrew Cooper
  Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk, Tim (Xen.org),
	George Dunlap, Marek Marczykowski-Górecki, Julien Grall,
	Jan Beulich, Xen-devel

Andrew Cooper writes ("Re: [PATCH 10/12] docs/migration: Specify X86_{CPUID,MSR}_POLICY records"):
> The migration stream is split into records with no playload (markers
> with external control flow meaning), and data records, which have a payload.

I remember thinking at the time you specified this (some time ago, in
migration v2) that this was anomalous.

At the time it made no difference in practice, because all existing
payload types required nonempty payloads anyway, so I didn't argue.

> It is an error for a data record to have no payload, because it means
> there is a source side generation bug.  In the case of Xen returning 0
> MSRs, the record would be omitted entirely, rather than be sent with 0
> MSRs worth of data.

I think it would be better to say instead that data records may have a
0-length payload.  This allows a record with a 0-length payload to
have a different semantic meaning ("here is this information and the
information is the empty set") from an omitted record ("this
information is not available/provided").

Whether a record is a marker ought to be inferred from its type.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 01/12] libxc/save: Shrink code volume where possible
  2019-12-24 15:19 ` [Xen-devel] [PATCH 01/12] libxc/save: Shrink code volume where possible Andrew Cooper
@ 2020-01-14 16:48   ` Ian Jackson
  2020-01-14 16:55     ` Ian Jackson
  2020-01-15 19:22     ` Andrew Cooper
  0 siblings, 2 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 16:48 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Xen-devel, Wei Liu

Andrew Cooper writes ("[PATCH 01/12] libxc/save: Shrink code volume where possible"):
> A property of how the error handling (0 on success, nonzero otherwise)
> allows these calls to be chained together with the ternary operatior.

I'm quite surprised to find a suggestion like this coming from you in
particular.  I think if we are going to adopt this thing in general,
it ought to be in a CODING_STYLE somewhere.

I'm distinctly unsure about the merits of the pattern.  It does make
the code much shorter and less repetitive.  OTOH ?: is a
not-very-frequently used GNU extension and my representative sample of
programmers had to think about what this idiom meant and it wasn't
universally liked.  On the third hand, if this idiom becomes dominant
you only have to think about it once.

Maybe it would be better to have
    #define MUST(call) ({ rc = (call); if (rc) goto error; })
and write
    MUST( write_one_vcpu_basic(ctx, i) );

Or just to permit
   rc = write_one_vcpu_basic(ctx, i);    if (rc) goto error;
(ie on a single line).

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 02/12] libxc/restore: Introduce functionality to simplify blob handling
  2019-12-24 15:19 ` [Xen-devel] [PATCH 02/12] libxc/restore: Introduce functionality to simplify blob handling Andrew Cooper
@ 2020-01-14 16:50   ` Ian Jackson
  0 siblings, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 16:50 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Xen-devel, Wei Liu

Andrew Cooper writes ("[PATCH 02/12] libxc/restore: Introduce functionality to simplify blob handling"):
> During migration, we buffer several blobs of data which ultimately need
> handing back to Xen at an appropriate time.
> 
> Currently, this is all handled in an ad-hoc manner, but more blobs are soon
> going to be added.  Introduce xc_sr_blob to encapsulate a ptr/size pair, and
> update_blob() to handle the memory management aspects.
> 
> Switch the HVM_CONTEXT and the four PV_VCPU_* blobs over to this new
> infrastructure.

I like the new approach.

I was expecting to see "no functional change in the commit message".

If you think that is true, and with that added,
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 01/12] libxc/save: Shrink code volume where possible
  2020-01-14 16:48   ` Ian Jackson
@ 2020-01-14 16:55     ` Ian Jackson
  2020-01-15 19:22     ` Andrew Cooper
  1 sibling, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 16:55 UTC (permalink / raw)
  To: Andrew Cooper, Xen-devel, Wei Liu

Ian Jackson writes ("Re: [PATCH 01/12] libxc/save: Shrink code volume where possible"):
> Maybe it would be better to have
>     #define MUST(call) ({ rc = (call); if (rc) goto error; })
> and write
>     MUST( write_one_vcpu_basic(ctx, i) );

This is not uncommon.  BIND9 does something like it:
    https://git.uis.cam.ac.uk/x/uis/ipreg/bind9.git/blob/HEAD:/lib/dns/zone.c#l515

A friend points out that
  #define MUST(x) ({ int rc_ = (x); if (rc_) { rc = rc_; goto error; } })
is better because it keeps rc uninitialised until the last moment.
That means the compiler can spot exit paths where you fail to set rc.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 08/12] libxc/restore: Support v3 streams, and cope with v2 compatibilty
  2019-12-24 15:19 ` [Xen-devel] [PATCH 08/12] libxc/restore: Support v3 streams, and cope with v2 compatibilty Andrew Cooper
@ 2020-01-14 17:02   ` Ian Jackson
  0 siblings, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 17:02 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Xen-devel, Wei Liu, Ian Jackson

Andrew Cooper writes ("[PATCH 08/12] libxc/restore: Support v3 streams, and cope with v2 compatibilty"):
> Introduce a static_data_complete() hook which is called when a
> STATIC_DATA_END record is found (v3) or inferred (v2).
> 
> Modify handle_page_data() and handle_x86_pv_p2m_frames() to infer the
> position in v2 streams.
> 
> The implementation of x86_static_data_complete() needs to wait until
> more plumbing is in place, to make a combined libxl/libxc change to
> maintain (functional) bisectability.
> 
> No practical change to behaviour - this is all plumbing work.

These parts are confusing to me:

> +    /*
> +     * This is a bit of a bodge, but it is less bad than duplicating
> +     * handle_page_data() between different architectures.
> +     */
> +#if defined(__i386__) || defined(__x86_64__)
> +    /* v2 compat.  Infer the position of STATIC_DATA_END. */
> +    if ( ctx->restore.format_version < 3 && !ctx->restore.seen_static_data_end )
> +    {
> +        rc = handle_static_data_end(ctx);
> +        if ( rc )
> +        {
> +            ERROR("Inferred STATIC_DATA_END record failed");
> +            goto err;
> +        }
> +        rc = -1;
> +    }
> +
> +    if ( !ctx->restore.seen_static_data_end )
> +    {
> +        ERROR("No STATIC_DATA_END seen");
> +        goto err;
> +    }
> +#endif

...

> +    /* v2 compat.  Infer the position of STATIC_DATA_END. */
> +    if ( ctx->restore.format_version < 3 && !ctx->restore.seen_static_data_end )
> +    {
> +        rc = handle_static_data_end(ctx);
> +        if ( rc )
> +        {
> +            ERROR("Inferred STATIC_DATA_END record failed");
> +            return rc;
> +        }
> +    }
> +
> +    if ( !ctx->restore.seen_static_data_end )
> +    {
> +        ERROR("No STATIC_DATA_END seen");
> +        return -1;
> +    }

Firstly, this code is remarkably similar.  Surely it should be
factored out into something like
  possible_implicit_static_data_end()
?

And secondly, I don't understand why the first part in
handle_page_data is arch-qualified.  Maybe this would make more sense
if I looked at the code in context rather than just the diff ?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 09/12] libxc/save: Write a v3 stream
  2019-12-24 15:19 ` [Xen-devel] [PATCH 09/12] libxc/save: Write a v3 stream Andrew Cooper
@ 2020-01-14 17:05   ` Ian Jackson
  0 siblings, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 17:05 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Xen-devel, Marek Marczykowski-Górecki, Wei Liu

Andrew Cooper writes ("[PATCH 09/12] libxc/save: Write a v3 stream"):
> Introduce a new static_data() hook which is responsible for writing out
> any static data records.  The HVM side continues to be a no-op, while
> the PV side moves write_x86_pv_info() into this earlier hook.  The the
> common code writes out a STATIC_DATA_END record, and the stream version
> is bumped to 3.
> 
> Update convert-legacy-stream to write a v3 stream, because this will
> bypass the compatiblity logic in libxc.

This seems like it contains the parts I would expect.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 11/12] libxc/restore: Handle X86_{CPUID, MSR}_DATA records
  2019-12-24 15:19 ` [Xen-devel] [PATCH 11/12] libxc/restore: Handle X86_{CPUID, MSR}_DATA records Andrew Cooper
@ 2020-01-14 17:16   ` Ian Jackson
  0 siblings, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 17:16 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Xen-devel, Wei Liu

Andrew Cooper writes ("[PATCH 11/12] libxc/restore: Handle X86_{CPUID,MSR}_DATA records"):
> For now, the data are just stashed, and discarded at the end.  This will
> be addressed when the TODO in x86_static_data_complete() is addressed.
> 
> No practical change to behaviour - this is all plumbing work.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 12/12] libxc/save: Write X86_{CPUID, MSR}_DATA records
  2019-12-24 15:19 ` [Xen-devel] [PATCH 12/12] libxc/save: Write " Andrew Cooper
@ 2020-01-14 17:21   ` Ian Jackson
  2020-01-15 15:52     ` Andrew Cooper
  0 siblings, 1 reply; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 17:21 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Xen-devel, Wei Liu

Andrew Cooper writes ("[PATCH 12/12] libxc/save: Write X86_{CPUID,MSR}_DATA records"):
> With all other plumbing in place, obtain the CPU Policy from Xen and
> write it into the migration stream.

This looks good to me but:

This patch may need revision to handle the results of our discussion
about the ?: error handling idiom.

And I am still missing the text discussing compatibility.  Maybe I
have just overlooked it ?

Anyway, if it weren't for those two issues I would have acked the
patch.

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 16/20] tools/libxl: Simplify callback handling in libxl-save-helper
  2020-01-03 13:06   ` [Xen-devel] [PATCH 16/20] tools/libxl: Simplify callback handling in libxl-save-helper Andrew Cooper
@ 2020-01-14 17:27     ` Ian Jackson
  2020-01-15 16:16       ` Andrew Cooper
  0 siblings, 1 reply; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 17:27 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Anthony Perard, Xen-devel, Wei Liu

Andrew Cooper writes ("[PATCH 16/20] tools/libxl: Simplify callback handling in libxl-save-helper"):
> The {save,restore}_callback helpers can have their scope reduced vastly,

This part is OK with me although it would have been nicer to review if
the the move and the rename were separate patches.  I don't know why
it is valuable.

> and helper_setcallbacks_{save,restore}() doesn't need to use a
> ternary operator to write 0 (meaning NULL) into an already zeroed
> structure.

Is this unrelated ?  I think so.

>          my $c_cb = "cbs->$name";
>          $f_more_sr->("    if ($c_cb) cbflags |= $c_v;\n", $enumcallbacks);
> -        $f_more_sr->("    $c_cb = (cbflags & $c_v) ? ${encode}_${name} : 0;\n",
> +        $f_more_sr->("    if (cbflags & $c_v) $c_cb = ${encode}_${name};\n",
>                       $setcallbacks);

It is a long time since I edited this code but I think your reasoning
is "cbs is already zero on entry because it is static; therefore
cbs->$name must be null, so there is no need to write 0 into it in the
else case".

However, the line you are touching is preceded by "if ($c_cb)" which
only makes sense if the variable might be non-null.

So something is not right here.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 17/20] tools/libx[cl]: Plumb static_data_done() up into libxl
  2020-01-03 13:06   ` [Xen-devel] [PATCH 17/20] tools/libx[cl]: Plumb static_data_done() up into libxl Andrew Cooper
@ 2020-01-14 17:30     ` Ian Jackson
  2020-01-15 16:34       ` Andrew Cooper
  0 siblings, 1 reply; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 17:30 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Anthony Perard, Xen-devel, Wei Liu

Andrew Cooper writes ("[PATCH 17/20] tools/libx[cl]: Plumb static_data_done() up into libxl"):
>  /* callbacks provided by xc_domain_restore */
>  struct restore_callbacks {
> +    /*
> +     * Called once the STATIC_DATA_END record has been received/inferred.
> +     * Passes in the blocks of static data which have not been received, which
> +     * the higher level toolstack must provide backwards compatibility for.
> +     */
> +#define XGR_SDD_MISSING_CPUID (1 << 0)
> +#define XGR_SDD_MISSING_MSR   (1 << 1)
> +    int (*static_data_done)(unsigned int missing, void *data);

This is a bit weird, isn't it ?  I mean: if these blocks of data *are*
received then libxc handles them; but if they are not, libxc's caller
must do so.

I appreciate that the interface at the top of libxc is already rather
complex and uneven but this doesn't seem to be helping...

The actual code looks OK to me.

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 18/20] tools/libxl: Plumb domain_create_state down into libxl__build_pre()
  2020-01-03 13:06   ` [Xen-devel] [PATCH 18/20] tools/libxl: Plumb domain_create_state down into libxl__build_pre() Andrew Cooper
@ 2020-01-14 17:32     ` Ian Jackson
  0 siblings, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 17:32 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Anthony Perard, Xen-devel, Wei Liu

Andrew Cooper writes ("[PATCH 18/20] tools/libxl: Plumb domain_create_state down into libxl__build_pre()"):
> To fix CPUID handling, libxl__build_pre() is going to have to distinguish
> between a brand new VM vs one which is being migrated-in/resumed.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 19/20] tools/libxl: Re-position CPUID handling during domain construction
  2020-01-03 13:06   ` [Xen-devel] [PATCH 19/20] tools/libxl: Re-position CPUID handling during domain construction Andrew Cooper
@ 2020-01-14 17:33     ` Ian Jackson
  2020-01-14 17:51       ` Andrew Cooper
  0 siblings, 1 reply; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 17:33 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Anthony Perard, Xen-devel, Wei Liu

Andrew Cooper writes ("[PATCH 19/20] tools/libxl: Re-position CPUID handling during domain construction"):
> CPUID handling needs to be earlier in construction.  Move it from its current
> position in libxl__build_post() to libxl__build_pre() for fresh builds, and
> libxl__srm_callout_callback_static_data_done() for the migration/resume case.
> 
> In the migration case, take account of XGR_SDD_MISSING_CPUID.

Is it possible to split out the change to the sequencing, from the new
functionality ?  If so, please could you do so.

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 20/20] tools/libxc: Restore CPUID/MSR data found in the migration stream
  2020-01-03 13:06   ` [Xen-devel] [PATCH 20/20] tools/libxc: Restore CPUID/MSR data found in the migration stream Andrew Cooper
@ 2020-01-14 17:34     ` Ian Jackson
  0 siblings, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 17:34 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Xen-devel, Wei Liu

Andrew Cooper writes ("[PATCH 20/20] tools/libxc: Restore CPUID/MSR data found in the migration stream"):
> With libxl suitably adjusted, it is now safe to restore the CPUID/MSR data
> directly from the migration stream.  Adjust the XGR_SDD_MISSING_* flags for
> the static_data_done() callback appropriately.

Subject to my comments earlier about (i) the unevenness of the
resulting libxc API and (ii) text about the compatibility plan,

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 19/20] tools/libxl: Re-position CPUID handling during domain construction
  2020-01-14 17:33     ` Ian Jackson
@ 2020-01-14 17:51       ` Andrew Cooper
  2020-01-14 18:12         ` Ian Jackson
  0 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2020-01-14 17:51 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Anthony Perard, Xen-devel, Wei Liu

On 14/01/2020 17:33, Ian Jackson wrote:
> Andrew Cooper writes ("[PATCH 19/20] tools/libxl: Re-position CPUID handling during domain construction"):
>> CPUID handling needs to be earlier in construction.  Move it from its current
>> position in libxl__build_post() to libxl__build_pre() for fresh builds, and
>> libxl__srm_callout_callback_static_data_done() for the migration/resume case.
>>
>> In the migration case, take account of XGR_SDD_MISSING_CPUID.
> Is it possible to split out the change to the sequencing, from the new
> functionality ?

Not easily, no.

While libxc is discarding the CPUID record, libxl needs to make the
calls unconditionally for VMs to function.

If you recall, an early patch in the series leaves a todo in libxc
(which is there for this very bisection reason), which passes
XGR_SDD_MISSING_CPUID back unconditionally (making this patch alone
effectively no net change).

A subsequence patch makes the passing of XGR_SDD_MISSING_CPUID
conditional on the content in the stream, now that we are no longer
discarding the CPUID records found.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 19/20] tools/libxl: Re-position CPUID handling during domain construction
  2020-01-14 17:51       ` Andrew Cooper
@ 2020-01-14 18:12         ` Ian Jackson
  0 siblings, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-14 18:12 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Anthony Perard, Xen-devel, Wei Liu

Andrew Cooper writes ("Re: [PATCH 19/20] tools/libxl: Re-position CPUID handling during domain construction"):
> On 14/01/2020 17:33, Ian Jackson wrote:
> > Is it possible to split out the change to the sequencing, from the new
> > functionality ?
> 
> Not easily, no.

Thanks for the explanation.  I read it again with that in mind.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 05/12] tools/migration: Drop IHDR_VERSION constant from libxc and python
  2020-01-14 16:05   ` Ian Jackson
@ 2020-01-15 15:29     ` Andrew Cooper
  0 siblings, 0 replies; 60+ messages in thread
From: Andrew Cooper @ 2020-01-15 15:29 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Xen-devel, Marek Marczykowski-Górecki, Wei Liu

On 14/01/2020 16:05, Ian Jackson wrote:
> Andrew Cooper writes ("[PATCH 05/12] tools/migration: Drop IHDR_VERSION constant from libxc and python"):
>> Migration v3 is in the process of being introduced, meaning that the code has
>> to cope with both versions.  Use an explicit 2 for now.
>>
>> For the verify-stream-v2 and convert-legacy-stream scripts, update text to say
>> "v2 (or later)".  What matters is the distinction vs legacy streams.
> How about introducing
>   enum { IHDR_VERSION_2 = 2 }
> or some such ?
>
> In C it can be hard otherwise to find all the relevant tests.  Being
> able to grep for IHDR_VERSION would help.  So I would prefer manifest
> constants of some kind to unvarnished integers.

There is exactly (and only ever) one place where this constant is
checked.  This is a consequence of it being read out of a pipe.

It is highly unlikely that the code will gain a second check.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records
  2020-01-14 16:08   ` Ian Jackson
@ 2020-01-15 15:36     ` Andrew Cooper
  0 siblings, 0 replies; 60+ messages in thread
From: Andrew Cooper @ 2020-01-15 15:36 UTC (permalink / raw)
  To: Ian Jackson
  Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk, Tim (Xen.org),
	George Dunlap, Marek Marczykowski-Górecki, Julien Grall,
	Jan Beulich, Xen-devel

On 14/01/2020 16:08, Ian Jackson wrote:
> Andrew Cooper writes ("[PATCH 10/12] docs/migration: Specify X86_{CPUID,MSR}_POLICY records"):
>> These two records move blobs from the XEN_DOMCTL_{get,set}_cpu_policy
>> hypercall.
> We had an extensive IRL discussion recently about the compatibility
> implications of this.  Is that written down somewhere ?  I was
> expecting to see it in this patch.

Sadly clairvoyance isn't a skill I'm terribly good at.  (This email
predates our conversation by 2 weeks or so.)

v2 of the series will have appropriate adjustments, although none of it
was relevant to this patch specifically.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records
  2020-01-14 16:12           ` Ian Jackson
@ 2020-01-15 15:48             ` Andrew Cooper
  0 siblings, 0 replies; 60+ messages in thread
From: Andrew Cooper @ 2020-01-15 15:48 UTC (permalink / raw)
  To: Ian Jackson
  Cc: Stefano Stabellini, Wei Liu, Konrad Rzeszutek Wilk, Tim (Xen.org),
	George Dunlap, Marek Marczykowski-Górecki, Julien Grall,
	Jan Beulich, Xen-devel

On 14/01/2020 16:12, Ian Jackson wrote:
> Andrew Cooper writes ("Re: [PATCH 10/12] docs/migration: Specify X86_{CPUID,MSR}_POLICY records"):
>> The migration stream is split into records with no playload (markers
>> with external control flow meaning), and data records, which have a payload.
> I remember thinking at the time you specified this (some time ago, in
> migration v2) that this was anomalous.

It was, and remains, very deliberate.

> Whether a record is a marker ought to be inferred from its type.

All records have explicit semantics as specified by their types.  This
includes the semantics as to whether it shall have zero or non-zero payload.

A data record with no payload is nonsensical.  It is prohibited
specifically because it helps the protocol verification logic spot bugs,
and we really did spot several hypercall (preexiting) and save-side bugs
because of this rule.

If a plausible use for payload-less data appears, then we can take a
judgement call as to whether it outweighs the utility of improved error
detection.  Making this change would require a change to the spec, and
an adjustment to the pre-exiting receive side logic.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 12/12] libxc/save: Write X86_{CPUID, MSR}_DATA records
  2020-01-14 17:21   ` Ian Jackson
@ 2020-01-15 15:52     ` Andrew Cooper
  0 siblings, 0 replies; 60+ messages in thread
From: Andrew Cooper @ 2020-01-15 15:52 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Xen-devel, Wei Liu

On 14/01/2020 17:21, Ian Jackson wrote:
> Andrew Cooper writes ("[PATCH 12/12] libxc/save: Write X86_{CPUID,MSR}_DATA records"):
>> With all other plumbing in place, obtain the CPU Policy from Xen and
>> write it into the migration stream.
> This looks good to me but:
>
> This patch may need revision to handle the results of our discussion
> about the ?: error handling idiom.
>
> And I am still missing the text discussing compatibility.  Maybe I
> have just overlooked it ?

In all cases with migration development, the receive side logic
(previous patch) has to come before the save side logic (this patch), or
the result will break bisection with the receive side choking on an
unknown record type.

From the "whole series" point of view, compatibility is also the
destination side discarding the data because libxl still needs its order
of CPUID handling shuffling to cope.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 16/20] tools/libxl: Simplify callback handling in libxl-save-helper
  2020-01-14 17:27     ` Ian Jackson
@ 2020-01-15 16:16       ` Andrew Cooper
  0 siblings, 0 replies; 60+ messages in thread
From: Andrew Cooper @ 2020-01-15 16:16 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Anthony Perard, Xen-devel, Wei Liu

On 14/01/2020 17:27, Ian Jackson wrote:
> Andrew Cooper writes ("[PATCH 16/20] tools/libxl: Simplify callback handling in libxl-save-helper"):
>> The {save,restore}_callback helpers can have their scope reduced vastly,
> This part is OK with me although it would have been nicer to review if
> the the move and the rename were separate patches.  I don't know why
> it is valuable.
>
>> and helper_setcallbacks_{save,restore}() doesn't need to use a
>> ternary operator to write 0 (meaning NULL) into an already zeroed
>> structure.
> Is this unrelated ?  I think so.

This change is specifically to make the generated C easier to follow,
because I had to debug it yet again.

>>          my $c_cb = "cbs->$name";
>>          $f_more_sr->("    if ($c_cb) cbflags |= $c_v;\n", $enumcallbacks);
>> -        $f_more_sr->("    $c_cb = (cbflags & $c_v) ? ${encode}_${name} : 0;\n",
>> +        $f_more_sr->("    if (cbflags & $c_v) $c_cb = ${encode}_${name};\n",
>>                       $setcallbacks);
> It is a long time since I edited this code but I think your reasoning
> is "cbs is already zero on entry because it is static; therefore
> cbs->$name must be null, so there is no need to write 0 into it in the
> else case".

Correct.

>
> However, the line you are touching is preceded by "if ($c_cb)" which
> only makes sense if the variable might be non-null.
>
> So something is not right here.

This is all perl to me, but the two adjacent-looking lines of C don't
end up adjacent in the generated code.

The first line ends up in
libxl__srm_callout_enumcallbacks_{save,restore}() (in libxl.so), while
the second line ends up in helper_setcallbacks_{save,restore}() (in
libxl-save-helper).

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 17/20] tools/libx[cl]: Plumb static_data_done() up into libxl
  2020-01-14 17:30     ` Ian Jackson
@ 2020-01-15 16:34       ` Andrew Cooper
  2020-05-29 15:58         ` Ian Jackson
  0 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2020-01-15 16:34 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Anthony Perard, Xen-devel, Wei Liu

On 14/01/2020 17:30, Ian Jackson wrote:
> Andrew Cooper writes ("[PATCH 17/20] tools/libx[cl]: Plumb static_data_done() up into libxl"):
>>  /* callbacks provided by xc_domain_restore */
>>  struct restore_callbacks {
>> +    /*
>> +     * Called once the STATIC_DATA_END record has been received/inferred.
>> +     * Passes in the blocks of static data which have not been received, which
>> +     * the higher level toolstack must provide backwards compatibility for.
>> +     */
>> +#define XGR_SDD_MISSING_CPUID (1 << 0)
>> +#define XGR_SDD_MISSING_MSR   (1 << 1)
>> +    int (*static_data_done)(unsigned int missing, void *data);
> This is a bit weird, isn't it ?  I mean: if these blocks of data *are*
> received then libxc handles them; but if they are not, libxc's caller
> must do so.
>
> I appreciate that the interface at the top of libxc is already rather
> complex and uneven but this doesn't seem to be helping...

There are several things going on here.

One is the control flow marker of "We reached the end of the static
data".  A higher level toolstack needs to know this unconditionally,
which is why the callback is mandatory.

For v2 compatibility, its callers cope with "this is where an end of
static data would be in a v3 stream", but that abstracted away so the
higher level toolstack doesn't know or need to care.

The missing parameter is "p.s. here are the things we were expecting but
didn't get - you need to pick up the pieces".  For now, it is synonymous
with "here is a v2 stream without CPUID data", but that won't be
accurate in the future if/when new static data records get retrofitted.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* [Xen-devel] [PATCH 0.5/12] tools/migration: Formatting and style cleanup
  2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
                   ` (12 preceding siblings ...)
  2020-01-03 13:06 ` [Xen-devel] [PATCH] Use CPUID/MSR data from migration streams Andrew Cooper
@ 2020-01-15 18:53 ` Andrew Cooper
  2020-01-15 21:26   ` Ian Jackson
  13 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2020-01-15 18:53 UTC (permalink / raw)
  To: Xen-devel; +Cc: Andrew Cooper, Wei Liu, Ian Jackson

The code has devating from the prevailing style in many ways.  Adjust spacing,
indentation, position of operators, layout of multiline comments, removal of
superfluous comments, constness, trailing commas, and use of unqualified
'unsigned'.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Ian Jackson <Ian.Jackson@citrix.com>
CC: Wei Liu <wl@xen.org>
---
 tools/libxc/include/xenguest.h             |  35 ++++----
 tools/libxc/xc_sr_common.c                 |   9 +--
 tools/libxc/xc_sr_common.h                 |  10 +--
 tools/libxc/xc_sr_common_x86.c             |   5 +-
 tools/libxc/xc_sr_common_x86_pv.c          |  12 +--
 tools/libxc/xc_sr_restore.c                |  39 ++++-----
 tools/libxc/xc_sr_restore_x86_pv.c         |  21 ++---
 tools/libxc/xc_sr_save.c                   |  74 ++++++++---------
 tools/libxc/xc_sr_save_x86_hvm.c           |   7 +-
 tools/libxc/xc_sr_save_x86_pv.c            | 101 +++++++++--------------
 tools/python/scripts/convert-legacy-stream |  20 ++---
 tools/python/xen/migration/libxc.py        | 124 +++++++++++++++--------------
 tools/python/xen/migration/libxl.py        |  39 ++++-----
 13 files changed, 235 insertions(+), 261 deletions(-)

diff --git a/tools/libxc/include/xenguest.h b/tools/libxc/include/xenguest.h
index 237603373c..19d828a7f2 100644
--- a/tools/libxc/include/xenguest.h
+++ b/tools/libxc/include/xenguest.h
@@ -52,10 +52,11 @@ typedef int (*precopy_policy_t)(struct precopy_stats, void *);
 
 /* callbacks provided by xc_domain_save */
 struct save_callbacks {
-    /* Called after expiration of checkpoint interval,
+    /*
+     * Called after expiration of checkpoint interval,
      * to suspend the guest.
      */
-    int (*suspend)(void* data);
+    int (*suspend)(void *data);
 
     /*
      * Called before and after every batch of page data sent during
@@ -79,7 +80,7 @@ struct save_callbacks {
      * xc_domain_save then flushes the output buffer, while the
      *  guest continues to run.
      */
-    int (*postcopy)(void* data);
+    int (*postcopy)(void *data);
 
     /*
      * Called after the memory checkpoint has been flushed
@@ -94,7 +95,7 @@ struct save_callbacks {
      * 0: terminate checkpointing gracefully
      * 1: take another checkpoint
      */
-    int (*checkpoint)(void* data);
+    int (*checkpoint)(void *data);
 
     /*
      * Called after the checkpoint callback.
@@ -103,13 +104,13 @@ struct save_callbacks {
      * 0: terminate checkpointing gracefully
      * 1: take another checkpoint
      */
-    int (*wait_checkpoint)(void* data);
+    int (*wait_checkpoint)(void *data);
 
     /* Enable qemu-dm logging dirty pages to xen */
     int (*switch_qemu_logdirty)(uint32_t domid, unsigned enable, void *data); /* HVM only */
 
     /* to be provided as the last argument to each callback function */
-    void* data;
+    void *data;
 };
 
 /* Type of stream.  Plain, or using a continuous replication protocol? */
@@ -138,22 +139,24 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom,
 
 /* callbacks provided by xc_domain_restore */
 struct restore_callbacks {
-    /* Called after a new checkpoint to suspend the guest.
-     */
-    int (*suspend)(void* data);
+    /* Called after a new checkpoint to suspend the guest. */
+    int (*suspend)(void *data);
 
-    /* Called after the secondary vm is ready to resume.
+    /*
+     * Called after the secondary vm is ready to resume.
      * Callback function resumes the guest & the device model,
      * returns to xc_domain_restore.
      */
-    int (*postcopy)(void* data);
+    int (*postcopy)(void *data);
 
-    /* A checkpoint record has been found in the stream.
-     * returns: */
+    /*
+     * A checkpoint record has been found in the stream.
+     * returns:
+     */
 #define XGR_CHECKPOINT_ERROR    0 /* Terminate processing */
 #define XGR_CHECKPOINT_SUCCESS  1 /* Continue reading more data from the stream */
 #define XGR_CHECKPOINT_FAILOVER 2 /* Failover and resume VM */
-    int (*checkpoint)(void* data);
+    int (*checkpoint)(void *data);
 
     /*
      * Called after the checkpoint callback.
@@ -162,7 +165,7 @@ struct restore_callbacks {
      * 0: terminate checkpointing gracefully
      * 1: take another checkpoint
      */
-    int (*wait_checkpoint)(void* data);
+    int (*wait_checkpoint)(void *data);
 
     /*
      * callback to send store gfn and console gfn to xl
@@ -173,7 +176,7 @@ struct restore_callbacks {
                             void *data);
 
     /* to be provided as the last argument to each callback function */
-    void* data;
+    void *data;
 };
 
 /**
diff --git a/tools/libxc/xc_sr_common.c b/tools/libxc/xc_sr_common.c
index 46fd928de2..dd9a11b4b5 100644
--- a/tools/libxc/xc_sr_common.c
+++ b/tools/libxc/xc_sr_common.c
@@ -4,7 +4,7 @@
 
 #include <xen-tools/libs.h>
 
-static const char *dhdr_types[] =
+static const char *const dhdr_types[] =
 {
     [DHDR_TYPE_X86_PV]  = "x86 PV",
     [DHDR_TYPE_X86_HVM] = "x86 HVM",
@@ -18,7 +18,7 @@ const char *dhdr_type_to_str(uint32_t type)
     return "Reserved";
 }
 
-static const char *mandatory_rec_types[] =
+static const char *const mandatory_rec_types[] =
 {
     [REC_TYPE_END]                          = "End",
     [REC_TYPE_PAGE_DATA]                    = "Page data",
@@ -58,13 +58,12 @@ int write_split_record(struct xc_sr_context *ctx, struct xc_sr_record *rec,
     xc_interface *xch = ctx->xch;
     typeof(rec->length) combined_length = rec->length + sz;
     size_t record_length = ROUNDUP(combined_length, REC_ALIGN_ORDER);
-    struct iovec parts[] =
-    {
+    struct iovec parts[] = {
         { &rec->type,       sizeof(rec->type) },
         { &combined_length, sizeof(combined_length) },
         { rec->data,        rec->length },
         { buf,              sz },
-        { (void*)zeroes,    record_length - combined_length },
+        { (void *)zeroes,   record_length - combined_length },
     };
 
     if ( record_length > REC_LENGTH_MAX )
diff --git a/tools/libxc/xc_sr_common.h b/tools/libxc/xc_sr_common.h
index b923ad5c10..e43b858a0a 100644
--- a/tools/libxc/xc_sr_common.h
+++ b/tools/libxc/xc_sr_common.h
@@ -228,7 +228,7 @@ struct xc_sr_context
             struct precopy_stats stats;
 
             xen_pfn_t *batch_pfns;
-            unsigned nr_batch_pfns;
+            unsigned int nr_batch_pfns;
             unsigned long *deferred_pages;
             unsigned long nr_deferred_pages;
             xc_hypercall_buffer_t dirty_bitmap_hbuf;
@@ -262,8 +262,8 @@ struct xc_sr_context
  */
 #define DEFAULT_BUF_RECORDS 1024
             struct xc_sr_record *buffered_records;
-            unsigned allocated_rec_num;
-            unsigned buffered_rec_num;
+            unsigned int allocated_rec_num;
+            unsigned int buffered_rec_num;
 
             /*
              * Xenstore and Console parameters.
@@ -333,7 +333,7 @@ struct xc_sr_context
                     {
                         struct xc_sr_blob basic, extd, xsave, msr;
                     } *vcpus;
-                    unsigned nr_vcpus;
+                    unsigned int nr_vcpus;
                 } restore;
             };
         } x86_pv;
@@ -418,7 +418,7 @@ int read_record(struct xc_sr_context *ctx, int fd, struct xc_sr_record *rec);
  * x86_pv_localise_page() if we receive pagetables frames ahead of the
  * contents of the frames they point at.
  */
-int populate_pfns(struct xc_sr_context *ctx, unsigned count,
+int populate_pfns(struct xc_sr_context *ctx, unsigned int count,
                   const xen_pfn_t *original_pfns, const uint32_t *types);
 
 #endif
diff --git a/tools/libxc/xc_sr_common_x86.c b/tools/libxc/xc_sr_common_x86.c
index 1dee840a03..011684df97 100644
--- a/tools/libxc/xc_sr_common_x86.c
+++ b/tools/libxc/xc_sr_common_x86.c
@@ -4,11 +4,10 @@ int write_x86_tsc_info(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
     struct xc_sr_rec_x86_tsc_info tsc = {};
-    struct xc_sr_record rec =
-    {
+    struct xc_sr_record rec = {
         .type = REC_TYPE_X86_TSC_INFO,
         .length = sizeof(tsc),
-        .data = &tsc
+        .data = &tsc,
     };
 
     if ( xc_domain_get_tsc_info(xch, ctx->domid, &tsc.mode,
diff --git a/tools/libxc/xc_sr_common_x86_pv.c b/tools/libxc/xc_sr_common_x86_pv.c
index f233c8799e..ec433fad70 100644
--- a/tools/libxc/xc_sr_common_x86_pv.c
+++ b/tools/libxc/xc_sr_common_x86_pv.c
@@ -10,10 +10,10 @@ xen_pfn_t mfn_to_pfn(struct xc_sr_context *ctx, xen_pfn_t mfn)
 
 bool mfn_in_pseudophysmap(struct xc_sr_context *ctx, xen_pfn_t mfn)
 {
-    return ( (mfn <= ctx->x86_pv.max_mfn) &&
-             (mfn_to_pfn(ctx, mfn) <= ctx->x86_pv.max_pfn) &&
-             (xc_pfn_to_mfn(mfn_to_pfn(ctx, mfn), ctx->x86_pv.p2m,
-                            ctx->x86_pv.width) == mfn) );
+    return ((mfn <= ctx->x86_pv.max_mfn) &&
+            (mfn_to_pfn(ctx, mfn) <= ctx->x86_pv.max_pfn) &&
+            (xc_pfn_to_mfn(mfn_to_pfn(ctx, mfn), ctx->x86_pv.p2m,
+                           ctx->x86_pv.width) == mfn));
 }
 
 void dump_bad_pseudophysmap_entry(struct xc_sr_context *ctx, xen_pfn_t mfn)
@@ -157,7 +157,7 @@ int x86_pv_map_m2p(struct xc_sr_context *ctx)
     {
         struct xen_machphys_mfn_list xmml = {
             .max_extents = 1,
-            .extent_start = { &ctx->x86_pv.compat_m2p_mfn0 }
+            .extent_start = { &ctx->x86_pv.compat_m2p_mfn0 },
         };
 
         rc = do_memory_op(xch, XENMEM_machphys_compat_mfn_list,
@@ -175,7 +175,7 @@ int x86_pv_map_m2p(struct xc_sr_context *ctx)
     rc = 0;
     DPRINTF("max_mfn %#lx", ctx->x86_pv.max_mfn);
 
-err:
+ err:
     free(entries);
     free(extents_start);
 
diff --git a/tools/libxc/xc_sr_restore.c b/tools/libxc/xc_sr_restore.c
index d9bf6fb5a1..5e31908ca8 100644
--- a/tools/libxc/xc_sr_restore.c
+++ b/tools/libxc/xc_sr_restore.c
@@ -134,13 +134,13 @@ static int pfn_set_populated(struct xc_sr_context *ctx, xen_pfn_t pfn)
  * unpopulated subset.  If types is NULL, no page type checking is performed
  * and all unpopulated pfns are populated.
  */
-int populate_pfns(struct xc_sr_context *ctx, unsigned count,
+int populate_pfns(struct xc_sr_context *ctx, unsigned int count,
                   const xen_pfn_t *original_pfns, const uint32_t *types)
 {
     xc_interface *xch = ctx->xch;
     xen_pfn_t *mfns = malloc(count * sizeof(*mfns)),
         *pfns = malloc(count * sizeof(*pfns));
-    unsigned i, nr_pfns = 0;
+    unsigned int i, nr_pfns = 0;
     int rc = -1;
 
     if ( !mfns || !pfns )
@@ -202,7 +202,7 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned count,
  * stream, populate and record their types, map the relevant subset and copy
  * the data into the guest.
  */
-static int process_page_data(struct xc_sr_context *ctx, unsigned count,
+static int process_page_data(struct xc_sr_context *ctx, unsigned int count,
                              xen_pfn_t *pfns, uint32_t *types, void *page_data)
 {
     xc_interface *xch = ctx->xch;
@@ -210,8 +210,8 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned count,
     int *map_errs = malloc(count * sizeof(*map_errs));
     int rc;
     void *mapping = NULL, *guest_page = NULL;
-    unsigned i,    /* i indexes the pfns from the record. */
-        j,         /* j indexes the subset of pfns we decide to map. */
+    unsigned int i, /* i indexes the pfns from the record. */
+        j,          /* j indexes the subset of pfns we decide to map. */
         nr_pages = 0;
 
     if ( !mfns || !map_errs )
@@ -258,8 +258,8 @@ static int process_page_data(struct xc_sr_context *ctx, unsigned count,
     if ( nr_pages == 0 )
         goto done;
 
-    mapping = guest_page = xenforeignmemory_map(xch->fmem,
-        ctx->domid, PROT_READ | PROT_WRITE,
+    mapping = guest_page = xenforeignmemory_map(
+        xch->fmem, ctx->domid, PROT_READ | PROT_WRITE,
         nr_pages, mfns, map_errs);
     if ( !mapping )
     {
@@ -336,7 +336,7 @@ static int handle_page_data(struct xc_sr_context *ctx, struct xc_sr_record *rec)
 {
     xc_interface *xch = ctx->xch;
     struct xc_sr_rec_page_data_header *pages = rec->data;
-    unsigned i, pages_of_data = 0;
+    unsigned int i, pages_of_data = 0;
     int rc = -1;
 
     xen_pfn_t *pfns = NULL, pfn;
@@ -424,12 +424,11 @@ static int send_checkpoint_dirty_pfn_list(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
     int rc = -1;
-    unsigned count, written;
+    unsigned int count, written;
     uint64_t i, *pfns = NULL;
     struct iovec *iov = NULL;
     xc_shadow_op_stats_t stats = { 0, ctx->restore.p2m_size };
-    struct xc_sr_record rec =
-    {
+    struct xc_sr_record rec = {
         .type = REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST,
     };
     DECLARE_HYPERCALL_BUFFER_SHADOW(unsigned long, dirty_bitmap,
@@ -510,7 +509,7 @@ static int handle_checkpoint(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
     int rc = 0, ret;
-    unsigned i;
+    unsigned int i;
 
     if ( ctx->stream_type == XC_STREAM_PLAIN )
     {
@@ -587,7 +586,7 @@ static int handle_checkpoint(struct xc_sr_context *ctx)
 
         /* Wait for a new checkpoint */
         ret = ctx->restore.callbacks->wait_checkpoint(
-                                                ctx->restore.callbacks->data);
+            ctx->restore.callbacks->data);
         HANDLE_CALLBACK_RETURN_VALUE(ret);
 
         /* suspend secondary vm */
@@ -608,7 +607,7 @@ static int handle_checkpoint(struct xc_sr_context *ctx)
 static int buffer_record(struct xc_sr_context *ctx, struct xc_sr_record *rec)
 {
     xc_interface *xch = ctx->xch;
-    unsigned new_alloc_num;
+    unsigned int new_alloc_num;
     struct xc_sr_record *p;
 
     if ( ctx->restore.buffered_rec_num >= ctx->restore.allocated_rec_num )
@@ -675,8 +674,8 @@ static int setup(struct xc_sr_context *ctx)
 
     if ( ctx->stream_type == XC_STREAM_COLO )
     {
-        dirty_bitmap = xc_hypercall_buffer_alloc_pages(xch, dirty_bitmap,
-                                NRPAGES(bitmap_size(ctx->restore.p2m_size)));
+        dirty_bitmap = xc_hypercall_buffer_alloc_pages(
+            xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->restore.p2m_size)));
 
         if ( !dirty_bitmap )
         {
@@ -717,7 +716,7 @@ static int setup(struct xc_sr_context *ctx)
 static void cleanup(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
-    unsigned i;
+    unsigned int i;
     DECLARE_HYPERCALL_BUFFER_SHADOW(unsigned long, dirty_bitmap,
                                     &ctx->restore.dirty_bitmap_hbuf);
 
@@ -725,10 +724,12 @@ static void cleanup(struct xc_sr_context *ctx)
         free(ctx->restore.buffered_records[i].data);
 
     if ( ctx->stream_type == XC_STREAM_COLO )
-        xc_hypercall_buffer_free_pages(xch, dirty_bitmap,
-                                   NRPAGES(bitmap_size(ctx->restore.p2m_size)));
+        xc_hypercall_buffer_free_pages(
+            xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->restore.p2m_size)));
+
     free(ctx->restore.buffered_records);
     free(ctx->restore.populated_pfns);
+
     if ( ctx->restore.ops.cleanup(ctx) )
         PERROR("Failed to clean up");
 }
diff --git a/tools/libxc/xc_sr_restore_x86_pv.c b/tools/libxc/xc_sr_restore_x86_pv.c
index dcc2a7f8c5..9299861236 100644
--- a/tools/libxc/xc_sr_restore_x86_pv.c
+++ b/tools/libxc/xc_sr_restore_x86_pv.c
@@ -222,7 +222,7 @@ static int process_start_info(struct xc_sr_context *ctx,
 
     rc = 0;
 
-err:
+ err:
     if ( guest_start_info )
         munmap(guest_start_info, PAGE_SIZE);
 
@@ -238,7 +238,7 @@ static int process_vcpu_basic(struct xc_sr_context *ctx,
     xc_interface *xch = ctx->xch;
     vcpu_guest_context_any_t *vcpu = ctx->x86_pv.restore.vcpus[vcpuid].basic.ptr;
     xen_pfn_t pfn, mfn;
-    unsigned i, gdt_count;
+    unsigned int i, gdt_count;
     int rc = -1;
 
     /* Vcpu 0 is special: Convert the suspend record to an mfn. */
@@ -474,7 +474,7 @@ static int update_vcpu_context(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
     struct xc_sr_x86_pv_restore_vcpu *vcpu;
-    unsigned i;
+    unsigned int i;
     int rc = 0;
 
     for ( i = 0; i < ctx->x86_pv.restore.nr_vcpus; ++i )
@@ -527,7 +527,7 @@ static int update_guest_p2m(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
     xen_pfn_t mfn, pfn, *guest_p2m = NULL;
-    unsigned i;
+    unsigned int i;
     int rc = -1;
 
     for ( i = 0; i < ctx->x86_pv.p2m_frames; ++i )
@@ -562,7 +562,7 @@ static int update_guest_p2m(struct xc_sr_context *ctx)
 
     guest_p2m = xc_map_foreign_pages(xch, ctx->domid, PROT_WRITE,
                                      ctx->x86_pv.p2m_pfns,
-                                     ctx->x86_pv.p2m_frames );
+                                     ctx->x86_pv.p2m_frames);
     if ( !guest_p2m )
     {
         PERROR("Failed to map p2m frames");
@@ -572,6 +572,7 @@ static int update_guest_p2m(struct xc_sr_context *ctx)
     memcpy(guest_p2m, ctx->x86_pv.p2m,
            (ctx->x86_pv.max_pfn + 1) * ctx->x86_pv.width);
     rc = 0;
+
  err:
     if ( guest_p2m )
         munmap(guest_p2m, ctx->x86_pv.p2m_frames * PAGE_SIZE);
@@ -675,7 +676,7 @@ static int handle_x86_pv_p2m_frames(struct xc_sr_context *ctx,
 {
     xc_interface *xch = ctx->xch;
     struct xc_sr_rec_x86_pv_p2m_frames *data = rec->data;
-    unsigned start, end, x, fpp = PAGE_SIZE / ctx->x86_pv.width;
+    unsigned int start, end, x, fpp = PAGE_SIZE / ctx->x86_pv.width;
     int rc;
 
     if ( !ctx->x86_pv.restore.seen_pv_info )
@@ -862,7 +863,7 @@ static int handle_shared_info(struct xc_sr_context *ctx,
                               struct xc_sr_record *rec)
 {
     xc_interface *xch = ctx->xch;
-    unsigned i;
+    unsigned int i;
     int rc = -1;
     shared_info_any_t *guest_shinfo = NULL;
     const shared_info_any_t *old_shinfo = rec->data;
@@ -904,8 +905,8 @@ static int handle_shared_info(struct xc_sr_context *ctx,
     MEMSET_ARRAY_FIELD(guest_shinfo, evtchn_mask, 0xff, ctx->x86_pv.width);
 
     rc = 0;
- err:
 
+ err:
     if ( guest_shinfo )
         munmap(guest_shinfo, PAGE_SIZE);
 
@@ -952,7 +953,7 @@ static int x86_pv_localise_page(struct xc_sr_context *ctx,
     xc_interface *xch = ctx->xch;
     uint64_t *table = page;
     uint64_t pte;
-    unsigned i, to_populate;
+    unsigned int i, to_populate;
     xen_pfn_t pfns[(PAGE_SIZE / sizeof(uint64_t))];
 
     type &= XEN_DOMCTL_PFINFO_LTABTYPE_MASK;
@@ -1134,7 +1135,7 @@ static int x86_pv_cleanup(struct xc_sr_context *ctx)
 
     if ( ctx->x86_pv.restore.vcpus )
     {
-        unsigned i;
+        unsigned int i;
 
         for ( i = 0; i < ctx->x86_pv.restore.nr_vcpus; ++i )
         {
diff --git a/tools/libxc/xc_sr_save.c b/tools/libxc/xc_sr_save.c
index 0651fa92bc..fa736a311f 100644
--- a/tools/libxc/xc_sr_save.c
+++ b/tools/libxc/xc_sr_save.c
@@ -10,20 +10,18 @@ static int write_headers(struct xc_sr_context *ctx, uint16_t guest_type)
 {
     xc_interface *xch = ctx->xch;
     int32_t xen_version = xc_version(xch, XENVER_version, NULL);
-    struct xc_sr_ihdr ihdr =
-        {
-            .marker  = IHDR_MARKER,
-            .id      = htonl(IHDR_ID),
-            .version = htonl(IHDR_VERSION),
-            .options = htons(IHDR_OPT_LITTLE_ENDIAN),
-        };
-    struct xc_sr_dhdr dhdr =
-        {
-            .type       = guest_type,
-            .page_shift = XC_PAGE_SHIFT,
-            .xen_major  = (xen_version >> 16) & 0xffff,
-            .xen_minor  = (xen_version)       & 0xffff,
-        };
+    struct xc_sr_ihdr ihdr = {
+        .marker  = IHDR_MARKER,
+        .id      = htonl(IHDR_ID),
+        .version = htonl(IHDR_VERSION),
+        .options = htons(IHDR_OPT_LITTLE_ENDIAN),
+    };
+    struct xc_sr_dhdr dhdr = {
+        .type       = guest_type,
+        .page_shift = XC_PAGE_SHIFT,
+        .xen_major  = (xen_version >> 16) & 0xffff,
+        .xen_minor  = (xen_version)       & 0xffff,
+    };
 
     if ( xen_version < 0 )
     {
@@ -51,7 +49,7 @@ static int write_headers(struct xc_sr_context *ctx, uint16_t guest_type)
  */
 static int write_end_record(struct xc_sr_context *ctx)
 {
-    struct xc_sr_record end = { REC_TYPE_END, 0, NULL };
+    struct xc_sr_record end = { .type = REC_TYPE_END };
 
     return write_record(ctx, &end);
 }
@@ -61,7 +59,7 @@ static int write_end_record(struct xc_sr_context *ctx)
  */
 static int write_checkpoint_record(struct xc_sr_context *ctx)
 {
-    struct xc_sr_record checkpoint = { REC_TYPE_CHECKPOINT, 0, NULL };
+    struct xc_sr_record checkpoint = { .type = REC_TYPE_CHECKPOINT };
 
     return write_record(ctx, &checkpoint);
 }
@@ -84,14 +82,13 @@ static int write_batch(struct xc_sr_context *ctx)
     void **guest_data = NULL;
     void **local_pages = NULL;
     int *errors = NULL, rc = -1;
-    unsigned i, p, nr_pages = 0, nr_pages_mapped = 0;
-    unsigned nr_pfns = ctx->save.nr_batch_pfns;
+    unsigned int i, p, nr_pages = 0, nr_pages_mapped = 0;
+    unsigned int nr_pfns = ctx->save.nr_batch_pfns;
     void *page, *orig_page;
     uint64_t *rec_pfns = NULL;
     struct iovec *iov = NULL; int iovcnt = 0;
     struct xc_sr_rec_page_data_header hdr = { 0 };
-    struct xc_sr_record rec =
-    {
+    struct xc_sr_record rec = {
         .type = REC_TYPE_PAGE_DATA,
     };
 
@@ -153,8 +150,8 @@ static int write_batch(struct xc_sr_context *ctx)
 
     if ( nr_pages > 0 )
     {
-        guest_mapping = xenforeignmemory_map(xch->fmem,
-            ctx->domid, PROT_READ, nr_pages, mfns, errors);
+        guest_mapping = xenforeignmemory_map(
+            xch->fmem, ctx->domid, PROT_READ, nr_pages, mfns, errors);
         if ( !guest_mapping )
         {
             PERROR("Failed to map guest pages");
@@ -481,7 +478,7 @@ static int update_progress_string(struct xc_sr_context *ctx, char **str)
 static int simple_precopy_policy(struct precopy_stats stats, void *user)
 {
     return ((stats.dirty_count >= 0 &&
-            stats.dirty_count < SPP_TARGET_DIRTY_COUNT) ||
+             stats.dirty_count < SPP_TARGET_DIRTY_COUNT) ||
             stats.iteration >= SPP_MAX_ITERATIONS)
         ? XGS_POLICY_STOP_AND_COPY
         : XGS_POLICY_CONTINUE_PRECOPY;
@@ -511,12 +508,13 @@ static int send_memory_live(struct xc_sr_context *ctx)
     if ( rc )
         goto out;
 
-    ctx->save.stats = (struct precopy_stats)
-        { .dirty_count   = ctx->save.p2m_size };
+    ctx->save.stats = (struct precopy_stats){
+        .dirty_count = ctx->save.p2m_size,
+    };
     policy_stats = &ctx->save.stats;
 
     if ( precopy_policy == NULL )
-         precopy_policy = simple_precopy_policy;
+        precopy_policy = simple_precopy_policy;
 
     bitmap_set(dirty_bitmap, ctx->save.p2m_size);
 
@@ -546,7 +544,7 @@ static int send_memory_live(struct xc_sr_context *ctx)
         policy_decision = precopy_policy(*policy_stats, data);
 
         if ( policy_decision != XGS_POLICY_CONTINUE_PRECOPY )
-           break;
+            break;
 
         if ( xc_shadow_control(
                  xch, ctx->domid, XEN_DOMCTL_SHADOW_OP_CLEAN,
@@ -571,10 +569,10 @@ static int send_memory_live(struct xc_sr_context *ctx)
 static int colo_merge_secondary_dirty_bitmap(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
-    struct xc_sr_record rec = { 0, 0, NULL };
+    struct xc_sr_record rec;
     uint64_t *pfns = NULL;
     uint64_t pfn;
-    unsigned count, i;
+    unsigned int count, i;
     int rc;
     DECLARE_HYPERCALL_BUFFER_SHADOW(unsigned long, dirty_bitmap,
                                     &ctx->save.dirty_bitmap_hbuf);
@@ -585,14 +583,14 @@ static int colo_merge_secondary_dirty_bitmap(struct xc_sr_context *ctx)
 
     if ( rec.type != REC_TYPE_CHECKPOINT_DIRTY_PFN_LIST )
     {
-        PERROR("Expect dirty bitmap record, but received %u", rec.type );
+        PERROR("Expect dirty bitmap record, but received %u", rec.type);
         rc = -1;
         goto err;
     }
 
     if ( rec.length % sizeof(*pfns) )
     {
-        PERROR("Invalid dirty pfn list record length %u", rec.length );
+        PERROR("Invalid dirty pfn list record length %u", rec.length);
         rc = -1;
         goto err;
     }
@@ -603,7 +601,7 @@ static int colo_merge_secondary_dirty_bitmap(struct xc_sr_context *ctx)
     for ( i = 0; i < count; i++ )
     {
         pfn = pfns[i];
-        if (pfn > ctx->save.p2m_size)
+        if ( pfn > ctx->save.p2m_size )
         {
             PERROR("Invalid pfn 0x%" PRIx64, pfn);
             rc = -1;
@@ -688,11 +686,7 @@ static int verify_frames(struct xc_sr_context *ctx)
     xc_interface *xch = ctx->xch;
     xc_shadow_op_stats_t stats = { 0, ctx->save.p2m_size };
     int rc;
-    struct xc_sr_record rec =
-    {
-        .type = REC_TYPE_VERIFY,
-        .length = 0,
-    };
+    struct xc_sr_record rec = { .type = REC_TYPE_VERIFY };
 
     DPRINTF("Enabling verify mode");
 
@@ -748,7 +742,7 @@ static int send_domain_memory_live(struct xc_sr_context *ctx)
             goto out;
     }
 
-  out:
+ out:
     return rc;
 }
 
@@ -795,7 +789,7 @@ static int setup(struct xc_sr_context *ctx)
         goto err;
 
     dirty_bitmap = xc_hypercall_buffer_alloc_pages(
-                   xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->save.p2m_size)));
+        xch, dirty_bitmap, NRPAGES(bitmap_size(ctx->save.p2m_size)));
     ctx->save.batch_pfns = malloc(MAX_BATCH_SIZE *
                                   sizeof(*ctx->save.batch_pfns));
     ctx->save.deferred_pages = calloc(1, bitmap_size(ctx->save.p2m_size));
@@ -966,7 +960,7 @@ static int save(struct xc_sr_context *ctx, uint16_t guest_type)
 };
 
 int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom,
-                   uint32_t flags, struct save_callbacks* callbacks,
+                   uint32_t flags, struct save_callbacks *callbacks,
                    xc_stream_type_t stream_type, int recv_fd)
 {
     struct xc_sr_context ctx = {
diff --git a/tools/libxc/xc_sr_save_x86_hvm.c b/tools/libxc/xc_sr_save_x86_hvm.c
index 90ff35a439..3d86cb0600 100644
--- a/tools/libxc/xc_sr_save_x86_hvm.c
+++ b/tools/libxc/xc_sr_save_x86_hvm.c
@@ -11,8 +11,7 @@ static int write_hvm_context(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
     int rc, hvm_buf_size;
-    struct xc_sr_record hvm_rec =
-    {
+    struct xc_sr_record hvm_rec = {
         .type = REC_TYPE_HVM_CONTEXT,
     };
 
@@ -134,7 +133,6 @@ static xen_pfn_t x86_hvm_pfn_to_gfn(const struct xc_sr_context *ctx,
 static int x86_hvm_normalise_page(struct xc_sr_context *ctx,
                                   xen_pfn_t type, void **page)
 {
-    /* no-op */
     return 0;
 }
 
@@ -174,19 +172,16 @@ static int x86_hvm_setup(struct xc_sr_context *ctx)
 
 static int x86_hvm_start_of_stream(struct xc_sr_context *ctx)
 {
-    /* no-op */
     return 0;
 }
 
 static int x86_hvm_start_of_checkpoint(struct xc_sr_context *ctx)
 {
-    /* no-op */
     return 0;
 }
 
 static int x86_hvm_check_vm_state(struct xc_sr_context *ctx)
 {
-    /* no-op */
     return 0;
 }
 
diff --git a/tools/libxc/xc_sr_save_x86_pv.c b/tools/libxc/xc_sr_save_x86_pv.c
index 09b6efeee3..3ebc5a2bf8 100644
--- a/tools/libxc/xc_sr_save_x86_pv.c
+++ b/tools/libxc/xc_sr_save_x86_pv.c
@@ -80,7 +80,7 @@ static int map_p2m_leaves(struct xc_sr_context *ctx, xen_pfn_t *mfns,
                           size_t n_mfns)
 {
     xc_interface *xch = ctx->xch;
-    unsigned x;
+    unsigned int x;
 
     ctx->x86_pv.p2m = xc_map_foreign_pages(xch, ctx->domid, PROT_READ,
                                            mfns, n_mfns);
@@ -133,7 +133,7 @@ static int map_p2m_tree(struct xc_sr_context *ctx)
      */
     xc_interface *xch = ctx->xch;
     int rc = -1;
-    unsigned x, saved_x, fpp, fll_entries, fl_entries;
+    unsigned int x, saved_x, fpp, fll_entries, fl_entries;
     xen_pfn_t fll_mfn, saved_mfn, max_pfn;
 
     xen_pfn_t *local_fll = NULL;
@@ -260,8 +260,7 @@ static int map_p2m_tree(struct xc_sr_context *ctx)
     /* Map the p2m leaves themselves. */
     rc = map_p2m_leaves(ctx, local_fl, fl_entries);
 
-err:
-
+ err:
     free(local_fl);
     if ( guest_fl )
         munmap(guest_fl, fll_entries * PAGE_SIZE);
@@ -318,7 +317,7 @@ static int map_p2m_list(struct xc_sr_context *ctx, uint64_t p2m_cr3)
     xen_pfn_t p2m_mfn, mfn, saved_mfn, max_pfn;
     uint64_t *ptes = NULL;
     xen_pfn_t *mfns = NULL;
-    unsigned fpp, n_pages, level, shift, idx_start, idx_end, idx, saved_idx;
+    unsigned int fpp, n_pages, level, shift, idx_start, idx_end, idx, saved_idx;
     int rc = -1;
 
     p2m_mfn = cr3_to_mfn(ctx, p2m_cr3);
@@ -450,7 +449,7 @@ static int map_p2m_list(struct xc_sr_context *ctx, uint64_t p2m_cr3)
     /* Map the p2m leaves themselves. */
     rc = map_p2m_leaves(ctx, mfns, idx_end - idx_start + 1);
 
-err:
+ err:
     free(mfns);
     if ( ptes )
         munmap(ptes, n_pages * PAGE_SIZE);
@@ -483,15 +482,13 @@ static int write_one_vcpu_basic(struct xc_sr_context *ctx, uint32_t id)
 {
     xc_interface *xch = ctx->xch;
     xen_pfn_t mfn, pfn;
-    unsigned i, gdt_count;
+    unsigned int i, gdt_count;
     int rc = -1;
     vcpu_guest_context_any_t vcpu;
-    struct xc_sr_rec_x86_pv_vcpu_hdr vhdr =
-    {
+    struct xc_sr_rec_x86_pv_vcpu_hdr vhdr = {
         .vcpu_id = id,
     };
-    struct xc_sr_record rec =
-    {
+    struct xc_sr_record rec = {
         .type = REC_TYPE_X86_PV_VCPU_BASIC,
         .length = sizeof(vhdr),
         .data = &vhdr,
@@ -586,18 +583,15 @@ static int write_one_vcpu_basic(struct xc_sr_context *ctx, uint32_t id)
 static int write_one_vcpu_extended(struct xc_sr_context *ctx, uint32_t id)
 {
     xc_interface *xch = ctx->xch;
-    struct xc_sr_rec_x86_pv_vcpu_hdr vhdr =
-    {
+    struct xc_sr_rec_x86_pv_vcpu_hdr vhdr = {
         .vcpu_id = id,
     };
-    struct xc_sr_record rec =
-    {
+    struct xc_sr_record rec = {
         .type = REC_TYPE_X86_PV_VCPU_EXTENDED,
         .length = sizeof(vhdr),
         .data = &vhdr,
     };
-    struct xen_domctl domctl =
-    {
+    struct xen_domctl domctl = {
         .cmd = XEN_DOMCTL_get_ext_vcpucontext,
         .domain = ctx->domid,
         .u.ext_vcpucontext.vcpu = id,
@@ -626,18 +620,15 @@ static int write_one_vcpu_xsave(struct xc_sr_context *ctx, uint32_t id)
     xc_interface *xch = ctx->xch;
     int rc = -1;
     DECLARE_HYPERCALL_BUFFER(void, buffer);
-    struct xc_sr_rec_x86_pv_vcpu_hdr vhdr =
-    {
+    struct xc_sr_rec_x86_pv_vcpu_hdr vhdr = {
         .vcpu_id = id,
     };
-    struct xc_sr_record rec =
-    {
+    struct xc_sr_record rec = {
         .type = REC_TYPE_X86_PV_VCPU_XSAVE,
         .length = sizeof(vhdr),
         .data = &vhdr,
     };
-    struct xen_domctl domctl =
-    {
+    struct xen_domctl domctl = {
         .cmd = XEN_DOMCTL_getvcpuextstate,
         .domain = ctx->domid,
         .u.vcpuextstate.vcpu = id,
@@ -695,18 +686,15 @@ static int write_one_vcpu_msrs(struct xc_sr_context *ctx, uint32_t id)
     int rc = -1;
     size_t buffersz;
     DECLARE_HYPERCALL_BUFFER(void, buffer);
-    struct xc_sr_rec_x86_pv_vcpu_hdr vhdr =
-    {
+    struct xc_sr_rec_x86_pv_vcpu_hdr vhdr = {
         .vcpu_id = id,
     };
-    struct xc_sr_record rec =
-    {
+    struct xc_sr_record rec = {
         .type = REC_TYPE_X86_PV_VCPU_MSRS,
         .length = sizeof(vhdr),
         .data = &vhdr,
     };
-    struct xen_domctl domctl =
-    {
+    struct xen_domctl domctl = {
         .cmd = XEN_DOMCTL_get_vcpu_msrs,
         .domain = ctx->domid,
         .u.vcpu_msrs.vcpu = id,
@@ -805,17 +793,15 @@ static int write_all_vcpu_information(struct xc_sr_context *ctx)
  */
 static int write_x86_pv_info(struct xc_sr_context *ctx)
 {
-    struct xc_sr_rec_x86_pv_info info =
-        {
-            .guest_width = ctx->x86_pv.width,
-            .pt_levels = ctx->x86_pv.levels,
-        };
-    struct xc_sr_record rec =
-        {
-            .type = REC_TYPE_X86_PV_INFO,
-            .length = sizeof(info),
-            .data = &info
-        };
+    struct xc_sr_rec_x86_pv_info info = {
+        .guest_width = ctx->x86_pv.width,
+        .pt_levels = ctx->x86_pv.levels,
+    };
+    struct xc_sr_record rec = {
+        .type = REC_TYPE_X86_PV_INFO,
+        .length = sizeof(info),
+        .data = &info,
+    };
 
     return write_record(ctx, &rec);
 }
@@ -827,20 +813,17 @@ static int write_x86_pv_info(struct xc_sr_context *ctx)
 static int write_x86_pv_p2m_frames(struct xc_sr_context *ctx)
 {
     xc_interface *xch = ctx->xch;
-    int rc; unsigned i;
+    int rc; unsigned int i;
     size_t datasz = ctx->x86_pv.p2m_frames * sizeof(uint64_t);
     uint64_t *data = NULL;
-    struct xc_sr_rec_x86_pv_p2m_frames hdr =
-        {
-            .start_pfn = 0,
-            .end_pfn = ctx->x86_pv.max_pfn,
-        };
-    struct xc_sr_record rec =
-        {
-            .type = REC_TYPE_X86_PV_P2M_FRAMES,
-            .length = sizeof(hdr),
-            .data = &hdr,
-        };
+    struct xc_sr_rec_x86_pv_p2m_frames hdr = {
+        .end_pfn = ctx->x86_pv.max_pfn,
+    };
+    struct xc_sr_record rec = {
+        .type = REC_TYPE_X86_PV_P2M_FRAMES,
+        .length = sizeof(hdr),
+        .data = &hdr,
+    };
 
     /* No need to translate if sizeof(uint64_t) == sizeof(xen_pfn_t). */
     if ( sizeof(uint64_t) != sizeof(*ctx->x86_pv.p2m_pfns) )
@@ -871,8 +854,7 @@ static int write_x86_pv_p2m_frames(struct xc_sr_context *ctx)
  */
 static int write_shared_info(struct xc_sr_context *ctx)
 {
-    struct xc_sr_record rec =
-    {
+    struct xc_sr_record rec = {
         .type = REC_TYPE_SHARED_INFO,
         .length = PAGE_SIZE,
         .data = ctx->x86_pv.shinfo,
@@ -890,7 +872,7 @@ static int normalise_pagetable(struct xc_sr_context *ctx, const uint64_t *src,
 {
     xc_interface *xch = ctx->xch;
     uint64_t pte;
-    unsigned i, xen_first = -1, xen_last = -1; /* Indices of Xen mappings. */
+    unsigned int i, xen_first = -1, xen_last = -1; /* Indices of Xen mappings. */
 
     type &= XEN_DOMCTL_PFINFO_LTABTYPE_MASK;
 
@@ -1004,7 +986,6 @@ static int normalise_pagetable(struct xc_sr_context *ctx, const uint64_t *src,
     return 0;
 }
 
-/* save_ops function. */
 static xen_pfn_t x86_pv_pfn_to_gfn(const struct xc_sr_context *ctx,
                                    xen_pfn_t pfn)
 {
@@ -1040,7 +1021,7 @@ static int x86_pv_normalise_page(struct xc_sr_context *ctx, xen_pfn_t type,
     rc = normalise_pagetable(ctx, *page, local_page, type);
     *page = local_page;
 
-  out:
+ out:
     return rc;
 }
 
@@ -1071,9 +1052,6 @@ static int x86_pv_setup(struct xc_sr_context *ctx)
     return 0;
 }
 
-/*
- * save_ops function.  Writes PV header records into the stream.
- */
 static int x86_pv_start_of_stream(struct xc_sr_context *ctx)
 {
     int rc;
@@ -1127,9 +1105,6 @@ static int x86_pv_check_vm_state(struct xc_sr_context *ctx)
     return x86_pv_check_vm_state_p2m_list(ctx);
 }
 
-/*
- * save_ops function.  Cleanup.
- */
 static int x86_pv_cleanup(struct xc_sr_context *ctx)
 {
     free(ctx->x86_pv.p2m_pfns);
diff --git a/tools/python/scripts/convert-legacy-stream b/tools/python/scripts/convert-legacy-stream
index a5da30b1ce..2922fb3185 100755
--- a/tools/python/scripts/convert-legacy-stream
+++ b/tools/python/scripts/convert-legacy-stream
@@ -257,8 +257,8 @@ def read_pv_extended_info(vm):
 
 
     if so_far != total_length:
-        raise StreamError("Overshot Extended Info size by %d bytes"
-                          % (so_far - total_length,))
+        raise StreamError("Overshot Extended Info size by %d bytes" %
+                          (so_far - total_length, ))
 
 def read_pv_p2m_frames(vm):
     fpp = 4096 / vm.width
@@ -375,8 +375,8 @@ def read_chunks(vm):
         elif marker > 0:
 
             if marker > legacy.MAX_BATCH:
-                raise StreamError("Page batch (%d) exceeded MAX_BATCH (%d)"
-                                  % (marker, legacy.MAX_BATCH))
+                raise StreamError("Page batch (%d) exceeded MAX_BATCH (%d)" %
+                                  (marker, legacy.MAX_BATCH))
             pfns = unpack_ulongs(marker)
 
             # xc_domain_save() leaves many XEN_DOMCTL_PFINFO_XTAB records for
@@ -398,8 +398,8 @@ def read_chunks(vm):
             max_id, = unpack_exact("i")
 
             if max_id > legacy.MAX_VCPU_ID:
-                raise StreamError("Vcpu max_id out of range: %d > %d"
-                                  % (max_id, legacy.MAX_VCPU_ID))
+                raise StreamError("Vcpu max_id out of range: %d > %d" %
+                                  (max_id, legacy.MAX_VCPU_ID))
 
             vm.max_vcpu_id = max_id
             bitmap = unpack_exact("Q" * ((max_id/64) + 1))
@@ -414,8 +414,8 @@ def read_chunks(vm):
                     bit_idx += 1
                     word >>= 1
 
-            info("  Vcpu info: max_id %d, online map %s"
-                 % (vm.max_vcpu_id, vm.online_vcpu_map))
+            info("  Vcpu info: max_id %d, online map %s" %
+                 (vm.max_vcpu_id, vm.online_vcpu_map))
 
         elif marker == legacy.CHUNK_hvm_ident_pt:
             _, ident_pt = unpack_exact("=IQ")
@@ -512,7 +512,7 @@ def read_chunks(vm):
                 [public.HVM_PARAM_NR_IOREQ_SERVER_PAGES, nr_pages])
 
         else:
-            raise StreamError("Unrecognised chunk %d" % (marker,))
+            raise StreamError("Unrecognised chunk %d" % (marker, ))
 
 def read_hvm_tail(vm):
 
@@ -579,7 +579,7 @@ def read_legacy_stream(vm):
 
     try:
         vm.p2m_size, = unpack_ulongs(1)
-        info("P2M Size: 0x%x" % (vm.p2m_size,))
+        info("P2M Size: 0x%x" % (vm.p2m_size, ))
 
         if vm.libxl:
             write_libxl_hdr()
diff --git a/tools/python/xen/migration/libxc.py b/tools/python/xen/migration/libxc.py
index 432750c9fa..8a800df980 100644
--- a/tools/python/xen/migration/libxc.py
+++ b/tools/python/xen/migration/libxc.py
@@ -74,7 +74,7 @@
     REC_TYPE_x86_pv_vcpu_msrs           : "x86 PV vcpu msrs",
     REC_TYPE_verify                     : "Verify",
     REC_TYPE_checkpoint                 : "Checkpoint",
-    REC_TYPE_checkpoint_dirty_pfn_list  : "Checkpoint dirty pfn list"
+    REC_TYPE_checkpoint_dirty_pfn_list  : "Checkpoint dirty pfn list",
 }
 
 # page_data
@@ -137,24 +137,25 @@ def verify_ihdr(self):
             self.unpack_exact(IHDR_FORMAT)
 
         if marker != IHDR_MARKER:
-            raise StreamError("Bad image marker: Expected 0x%x, got 0x%x"
-                              % (IHDR_MARKER, marker))
+            raise StreamError("Bad image marker: Expected 0x%x, got 0x%x" %
+                              (IHDR_MARKER, marker))
 
         if ident != IHDR_IDENT:
-            raise StreamError("Bad image id: Expected 0x%x, got 0x%x"
-                              % (IHDR_IDENT, ident))
+            raise StreamError("Bad image id: Expected 0x%x, got 0x%x" %
+                              (IHDR_IDENT, ident))
 
         if version != IHDR_VERSION:
-            raise StreamError("Unknown image version: Expected %d, got %d"
-                              % (IHDR_VERSION, version))
+            raise StreamError("Unknown image version: Expected %d, got %d" %
+                              (IHDR_VERSION, version))
 
         if options & IHDR_OPT_RESZ_MASK:
-            raise StreamError("Reserved bits set in image options field: 0x%x"
-                              % (options & IHDR_OPT_RESZ_MASK))
+            raise StreamError("Reserved bits set in image options field: 0x%x" %
+                              (options & IHDR_OPT_RESZ_MASK))
 
         if res1 != 0 or res2 != 0:
-            raise StreamError("Reserved bits set in image header: 0x%04x:0x%08x"
-                              % (res1, res2))
+            raise StreamError(
+                "Reserved bits set in image header: 0x%04x:0x%08x" %
+                (res1, res2))
 
         if ( (sys.byteorder == "little") and
              ((options & IHDR_OPT_BIT_ENDIAN) != IHDR_OPT_LE) ):
@@ -175,19 +176,19 @@ def verify_dhdr(self):
             raise StreamError("Unrecognised domain type 0x%x" % (gtype, ))
 
         if res1 != 0:
-            raise StreamError("Reserved bits set in domain header 0x%04x"
-                              % (res1, ))
+            raise StreamError("Reserved bits set in domain header 0x%04x" %
+                              (res1, ))
 
         if page_shift != 12:
-            raise StreamError("Page shift expected to be 12.  Got %d"
-                              % (page_shift, ))
+            raise StreamError("Page shift expected to be 12.  Got %d" %
+                              (page_shift, ))
 
         if major == 0:
-            self.info("Domain Header: legacy converted %s"
-                      % (dhdr_type_to_str[gtype], ))
+            self.info("Domain Header: legacy converted %s" %
+                      (dhdr_type_to_str[gtype], ))
         else:
-            self.info("Domain Header: %s from Xen %d.%d"
-                      % (dhdr_type_to_str[gtype], major, minor))
+            self.info("Domain Header: %s from Xen %d.%d" %
+                      (dhdr_type_to_str[gtype], major, minor))
 
 
     def verify_record(self):
@@ -204,12 +205,12 @@ def verify_record(self):
         if rtype != REC_TYPE_page_data:
 
             if self.squashed_pagedata_records > 0:
-                self.info("Squashed %d Page Data records together"
-                          % (self.squashed_pagedata_records, ))
+                self.info("Squashed %d Page Data records together" %
+                          (self.squashed_pagedata_records, ))
                 self.squashed_pagedata_records = 0
 
-            self.info("Libxc Record: %s, length %d"
-                      % (rec_type_to_str[rtype], length))
+            self.info("Libxc Record: %s, length %d" %
+                      (rec_type_to_str[rtype], length))
 
         else:
             self.squashed_pagedata_records += 1
@@ -219,8 +220,9 @@ def verify_record(self):
             raise StreamError("Padding containing non0 bytes found")
 
         if rtype not in record_verifiers:
-            raise RuntimeError("No verification function for libxc record '%s'"
-                               % rec_type_to_str[rtype])
+            raise RuntimeError(
+                "No verification function for libxc record '%s'" %
+                rec_type_to_str[rtype])
         else:
             record_verifiers[rtype](self, content[:length])
 
@@ -239,32 +241,32 @@ def verify_record_page_data(self, content):
         minsz = calcsize(PAGE_DATA_FORMAT)
 
         if len(content) <= minsz:
-            raise RecordError("PAGE_DATA record must be at least %d bytes long"
-                              % (minsz, ))
+            raise RecordError(
+                "PAGE_DATA record must be at least %d bytes long" % (minsz, ))
 
         count, res1 = unpack(PAGE_DATA_FORMAT, content[:minsz])
 
         if res1 != 0:
-            raise StreamError("Reserved bits set in PAGE_DATA record 0x%04x"
-                              % (res1, ))
+            raise StreamError(
+                "Reserved bits set in PAGE_DATA record 0x%04x" % (res1, ))
 
         pfnsz = count * 8
         if (len(content) - minsz) < pfnsz:
-            raise RecordError("PAGE_DATA record must contain a pfn record for "
-                              "each count")
+            raise RecordError(
+                "PAGE_DATA record must contain a pfn record for each count")
 
-        pfns = list(unpack("=%dQ" % (count,), content[minsz:minsz + pfnsz]))
+        pfns = list(unpack("=%dQ" % (count, ), content[minsz:minsz + pfnsz]))
 
         nr_pages = 0
         for idx, pfn in enumerate(pfns):
 
             if pfn & PAGE_DATA_PFN_RESZ_MASK:
-                raise RecordError("Reserved bits set in pfn[%d]: 0x%016x",
-                                  idx, pfn & PAGE_DATA_PFN_RESZ_MASK)
+                raise RecordError("Reserved bits set in pfn[%d]: 0x%016x" %
+                                  (idx, pfn & PAGE_DATA_PFN_RESZ_MASK))
 
             if pfn >> PAGE_DATA_TYPE_SHIFT in (5, 6, 7, 8):
-                raise RecordError("Invalid type value in pfn[%d]: 0x%016x",
-                                  idx, pfn & PAGE_DATA_TYPE_LTAB_MASK)
+                raise RecordError("Invalid type value in pfn[%d]: 0x%016x" %
+                                  (idx, pfn & PAGE_DATA_TYPE_LTAB_MASK))
 
             # We expect page data for each normal page or pagetable
             if PAGE_DATA_TYPE_NOTAB <= (pfn & PAGE_DATA_TYPE_LTABTYPE_MASK) \
@@ -273,8 +275,8 @@ def verify_record_page_data(self, content):
 
         pagesz = nr_pages * 4096
         if len(content) != minsz + pfnsz + pagesz:
-            raise RecordError("Expected %u + %u + %u, got %u"
-                              % (minsz, pfnsz, pagesz, len(content)))
+            raise RecordError("Expected %u + %u + %u, got %u" %
+                              (minsz, pfnsz, pagesz, len(content)))
 
 
     def verify_record_x86_pv_info(self, content):
@@ -282,8 +284,8 @@ def verify_record_x86_pv_info(self, content):
 
         expectedsz = calcsize(X86_PV_INFO_FORMAT)
         if len(content) != expectedsz:
-            raise RecordError("x86_pv_info: expected length of %d, got %d"
-                              % (expectedsz, len(content)))
+            raise RecordError("x86_pv_info: expected length of %d, got %d" %
+                              (expectedsz, len(content)))
 
         width, levels, res1, res2 = unpack(X86_PV_INFO_FORMAT, content)
 
@@ -294,8 +296,9 @@ def verify_record_x86_pv_info(self, content):
             raise RecordError("Expected levels of 3 or 4, got %d" % (levels, ))
 
         if res1 != 0 or res2 != 0:
-            raise StreamError("Reserved bits set in X86_PV_INFO: 0x%04x 0x%08x"
-                              % (res1, res2))
+            raise StreamError(
+                "Reserved bits set in X86_PV_INFO: 0x%04x 0x%08x" %
+                (res1, res2))
 
         bitness = {4:32, 8:64}[width]
         self.info("  %sbit guest, %d levels of pagetables" % (bitness, levels))
@@ -309,8 +312,8 @@ def verify_record_x86_pv_p2m_frames(self, content):
                               " least 8 bytes long")
 
         if len(content) % 8 != 0:
-            raise RecordError("Length expected to be a multiple of 8, not %d"
-                              % (len(content), ))
+            raise RecordError("Length expected to be a multiple of 8, not %d" %
+                              (len(content), ))
 
         start, end = unpack("=II", content[:8])
         self.info("  Start pfn 0x%x, End 0x%x" % (start, end))
@@ -321,30 +324,32 @@ def verify_record_x86_pv_vcpu_generic(self, content, name):
         minsz = calcsize(X86_PV_VCPU_HDR_FORMAT)
 
         if len(content) < minsz:
-            raise RecordError("X86_PV_VCPU_%s record length must be at least %d"
-                              " bytes long" % (name, minsz))
+            raise RecordError(
+                "X86_PV_VCPU_%s record length must be at least %d bytes long" %
+                (name, minsz))
 
         if len(content) == minsz:
-            self.info("Warning: X86_PV_VCPU_%s record with zero content"
-                      % (name, ))
+            self.info("Warning: X86_PV_VCPU_%s record with zero content" %
+                      (name, ))
 
         vcpuid, res1 = unpack(X86_PV_VCPU_HDR_FORMAT, content[:minsz])
 
         if res1 != 0:
             raise StreamError(
-                "Reserved bits set in x86_pv_vcpu_%s record 0x%04x"
-                              % (name, res1))
+                "Reserved bits set in x86_pv_vcpu_%s record 0x%04x" %
+                (name, res1))
 
-        self.info("  vcpu%d %s context, %d bytes"
-                  % (vcpuid, name, len(content) - minsz))
+        self.info("  vcpu%d %s context, %d bytes" %
+                  (vcpuid, name, len(content) - minsz))
 
 
     def verify_record_shared_info(self, content):
         """ shared info record """
 
-        if len(content) != 4096:
-            raise RecordError("Length expected to be 4906 bytes, not %d"
-                              % (len(content), ))
+        contentsz = len(content)
+        if contentsz != 4096:
+            raise RecordError("Length expected to be 4906 bytes, not %d" %
+                              (contentsz, ))
 
 
     def verify_record_tsc_info(self, content):
@@ -358,11 +363,11 @@ def verify_record_tsc_info(self, content):
         mode, khz, nsec, incarn, res1 = unpack(X86_TSC_INFO_FORMAT, content)
 
         if res1 != 0:
-            raise StreamError("Reserved bits set in X86_TSC_INFO: 0x%08x"
-                              % (res1, ))
+            raise StreamError("Reserved bits set in X86_TSC_INFO: 0x%08x" %
+                              (res1, ))
 
-        self.info("  Mode %u, %u kHz, %u ns, incarnation %d"
-                  % (mode, khz, nsec, incarn))
+        self.info("  Mode %u, %u kHz, %u ns, incarnation %d" %
+                  (mode, khz, nsec, incarn))
 
 
     def verify_record_hvm_context(self, content):
@@ -412,6 +417,7 @@ def verify_record_checkpoint(self, content):
         if len(content) != 0:
             raise RecordError("Checkpoint record with non-zero length")
 
+
     def verify_record_checkpoint_dirty_pfn_list(self, content):
         """ checkpoint dirty pfn list """
         raise RecordError("Found checkpoint dirty pfn list record in stream")
diff --git a/tools/python/xen/migration/libxl.py b/tools/python/xen/migration/libxl.py
index 79f4024e72..5c4d4fe063 100644
--- a/tools/python/xen/migration/libxl.py
+++ b/tools/python/xen/migration/libxl.py
@@ -45,7 +45,7 @@
     REC_TYPE_emulator_xenstore_data : "Emulator xenstore data",
     REC_TYPE_emulator_context       : "Emulator context",
     REC_TYPE_checkpoint_end         : "Checkpoint end",
-    REC_TYPE_checkpoint_state       : "Checkpoint state"
+    REC_TYPE_checkpoint_state       : "Checkpoint state",
 }
 
 # emulator_* header
@@ -90,16 +90,16 @@ def verify_hdr(self):
         ident, version, options = self.unpack_exact(HDR_FORMAT)
 
         if ident != HDR_IDENT:
-            raise StreamError("Bad image id: Expected 0x%x, got 0x%x"
-                              % (HDR_IDENT, ident))
+            raise StreamError("Bad image id: Expected 0x%x, got 0x%x" %
+                              (HDR_IDENT, ident))
 
         if version != HDR_VERSION:
-            raise StreamError("Unknown image version: Expected %d, got %d"
-                              % (HDR_VERSION, version))
+            raise StreamError("Unknown image version: Expected %d, got %d" %
+                              (HDR_VERSION, version))
 
         if options & HDR_OPT_RESZ_MASK:
-            raise StreamError("Reserved bits set in image options field: 0x%x"
-                              % (options & HDR_OPT_RESZ_MASK))
+            raise StreamError("Reserved bits set in image options field: 0x%x" %
+                              (options & HDR_OPT_RESZ_MASK))
 
         if ( (sys.byteorder == "little") and
              ((options & HDR_OPT_BIT_ENDIAN) != HDR_OPT_LE) ):
@@ -121,8 +121,8 @@ def verify_record(self):
         if rtype not in rec_type_to_str:
             raise StreamError("Unrecognised record type %x" % (rtype, ))
 
-        self.info("Libxl Record: %s, length %d"
-                  % (rec_type_to_str[rtype], length))
+        self.info("Libxl Record: %s, length %d" %
+                  (rec_type_to_str[rtype], length))
 
         contentsz = (length + 7) & ~7
         content = self.rdexact(contentsz)
@@ -132,8 +132,9 @@ def verify_record(self):
             raise StreamError("Padding containing non0 bytes found")
 
         if rtype not in record_verifiers:
-            raise RuntimeError("No verification function for libxl record '%s'"
-                               % rec_type_to_str[rtype])
+            raise RuntimeError(
+                "No verification function for libxl record '%s'" %
+                rec_type_to_str[rtype])
         else:
             record_verifiers[rtype](self, content[:length])
 
@@ -162,16 +163,16 @@ def verify_record_emulator_xenstore_data(self, content):
         minsz = calcsize(EMULATOR_HEADER_FORMAT)
 
         if len(content) < minsz:
-            raise RecordError("Length must be at least %d bytes, got %d"
-                              % (minsz, len(content)))
+            raise RecordError("Length must be at least %d bytes, got %d" %
+                              (minsz, len(content)))
 
         emu_id, emu_idx = unpack(EMULATOR_HEADER_FORMAT, content[:minsz])
 
         if emu_id not in emulator_id_to_str:
             raise RecordError("Unrecognised emulator id 0x%x" % (emu_id, ))
 
-        self.info("Emulator Xenstore Data (%s, idx %d)"
-                  % (emulator_id_to_str[emu_id], emu_idx))
+        self.info("Emulator Xenstore Data (%s, idx %d)" %
+                  (emulator_id_to_str[emu_id], emu_idx))
 
         # Chop off the emulator header
         content = content[minsz:]
@@ -185,8 +186,8 @@ def verify_record_emulator_xenstore_data(self, content):
             parts = content[:-1].split("\x00")
 
             if (len(parts) % 2) != 0:
-                raise RecordError("Expected an even number of strings, got %d"
-                                  % (len(parts), ))
+                raise RecordError("Expected an even number of strings, got %d" %
+                                  (len(parts), ))
 
             for key, val in zip(parts[0::2], parts[1::2]):
                 self.info("  '%s' = '%s'" % (key, val))
@@ -197,8 +198,8 @@ def verify_record_emulator_context(self, content):
         minsz = calcsize(EMULATOR_HEADER_FORMAT)
 
         if len(content) < minsz:
-            raise RecordError("Length must be at least %d bytes, got %d"
-                              % (minsz, len(content)))
+            raise RecordError("Length must be at least %d bytes, got %d" %
+                              (minsz, len(content)))
 
         emu_id, emu_idx = unpack(EMULATOR_HEADER_FORMAT, content[:minsz])
 
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply related	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 01/12] libxc/save: Shrink code volume where possible
  2020-01-14 16:48   ` Ian Jackson
  2020-01-14 16:55     ` Ian Jackson
@ 2020-01-15 19:22     ` Andrew Cooper
  2020-04-27 17:19       ` Ian Jackson
  1 sibling, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2020-01-15 19:22 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Xen-devel, Wei Liu

On 14/01/2020 16:48, Ian Jackson wrote:
> Andrew Cooper writes ("[PATCH 01/12] libxc/save: Shrink code volume where possible"):
>> A property of how the error handling (0 on success, nonzero otherwise)
>> allows these calls to be chained together with the ternary operatior.
> I'm quite surprised to find a suggestion like this coming from you in
> particular.

What probably is relevant is that ?: is a common construct in the
hypervisor, which I suppose does colour my expectation of people knowing
exactly what it means and how it behaves.

> Maybe it would be better to have
>     #define MUST(call) ({ rc = (call); if (rc) goto error; })
> and write
>     MUST( write_one_vcpu_basic(ctx, i) );
>
> Or just to permit
>    rc = write_one_vcpu_basic(ctx, i);    if (rc) goto error;
> (ie on a single line).

OTOH, it should come as no surprise that I'd rather drop this patch
entirely than go with these alternatives, both of which detract from
code clarity.  The former for hiding control flow, and the latter for
being atypical layout which unnecessary cognitive load to follow.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [Xen-devel] [PATCH 0.5/12] tools/migration: Formatting and style cleanup
  2020-01-15 18:53 ` [Xen-devel] [PATCH 0.5/12] tools/migration: Formatting and style cleanup Andrew Cooper
@ 2020-01-15 21:26   ` Ian Jackson
  0 siblings, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-01-15 21:26 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Xen-devel, Wei Liu

Andrew Cooper writes ("[PATCH 0.5/12] tools/migration: Formatting and style cleanup"):
> The code has devating from the prevailing style in many ways.  Adjust spacing,
> indentation, position of operators, layout of multiline comments, removal of
> superfluous comments, constness, trailing commas, and use of unqualified
> 'unsigned'.
> 
> No functional change.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [PATCH 01/12] libxc/save: Shrink code volume where possible
  2020-01-15 19:22     ` Andrew Cooper
@ 2020-04-27 17:19       ` Ian Jackson
  2020-04-27 19:55         ` Wei Liu
  0 siblings, 1 reply; 60+ messages in thread
From: Ian Jackson @ 2020-04-27 17:19 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Xen-devel, Wei Liu

Andrew Cooper writes ("Re: [PATCH 01/12] libxc/save: Shrink code volume where possible"):
> On 14/01/2020 16:48, Ian Jackson wrote:
> > Andrew Cooper writes ("[PATCH 01/12] libxc/save: Shrink code volume where possible"):
> >> A property of how the error handling (0 on success, nonzero otherwise)
> >> allows these calls to be chained together with the ternary operatior.
> > I'm quite surprised to find a suggestion like this coming from you in
> > particular.
> 
> What probably is relevant is that ?: is a common construct in the
> hypervisor, which I suppose does colour my expectation of people knowing
> exactly what it means and how it behaves.

I expect other C programmers to know what ?: does, too.  But I think
using it to implement the error monad is quite unusual.  I asked
around a bit and my feeling is still that this isn't an improvement.

> > Or just to permit
> >    rc = write_one_vcpu_basic(ctx, i);    if (rc) goto error;
> > (ie on a single line).
> 
> OTOH, it should come as no surprise that I'd rather drop this patch
> entirely than go with these alternatives, both of which detract from
> code clarity. The former for hiding control flow, and the latter for
> being atypical layout which unnecessary cognitive load to follow.

I think, then, that it would be best to drop this patch, unless Wei
(or someone else) disagrees with me.

Sorry,
Ian.


^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [PATCH 01/12] libxc/save: Shrink code volume where possible
  2020-04-27 17:19       ` Ian Jackson
@ 2020-04-27 19:55         ` Wei Liu
  2020-04-27 20:00           ` Andrew Cooper
  0 siblings, 1 reply; 60+ messages in thread
From: Wei Liu @ 2020-04-27 19:55 UTC (permalink / raw)
  To: Ian Jackson; +Cc: Andrew Cooper, Wei Liu, Xen-devel

On Mon, Apr 27, 2020 at 06:19:37PM +0100, Ian Jackson wrote:
> Andrew Cooper writes ("Re: [PATCH 01/12] libxc/save: Shrink code volume where possible"):
> > On 14/01/2020 16:48, Ian Jackson wrote:
> > > Andrew Cooper writes ("[PATCH 01/12] libxc/save: Shrink code volume where possible"):
> > >> A property of how the error handling (0 on success, nonzero otherwise)
> > >> allows these calls to be chained together with the ternary operatior.
> > > I'm quite surprised to find a suggestion like this coming from you in
> > > particular.
> > 
> > What probably is relevant is that ?: is a common construct in the
> > hypervisor, which I suppose does colour my expectation of people knowing
> > exactly what it means and how it behaves.
> 
> I expect other C programmers to know what ?: does, too.  But I think
> using it to implement the error monad is quite unusual.  I asked
> around a bit and my feeling is still that this isn't an improvement.
> 
> > > Or just to permit
> > >    rc = write_one_vcpu_basic(ctx, i);    if (rc) goto error;
> > > (ie on a single line).
> > 
> > OTOH, it should come as no surprise that I'd rather drop this patch
> > entirely than go with these alternatives, both of which detract from
> > code clarity. The former for hiding control flow, and the latter for
> > being atypical layout which unnecessary cognitive load to follow.
> 
> I think, then, that it would be best to drop this patch, unless Wei
> (or someone else) disagrees with me.

I don't feel strongly either way.

Wei.


^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [PATCH 01/12] libxc/save: Shrink code volume where possible
  2020-04-27 19:55         ` Wei Liu
@ 2020-04-27 20:00           ` Andrew Cooper
  2020-04-28  9:46             ` Wei Liu
  0 siblings, 1 reply; 60+ messages in thread
From: Andrew Cooper @ 2020-04-27 20:00 UTC (permalink / raw)
  To: Wei Liu, Ian Jackson; +Cc: Xen-devel

On 27/04/2020 20:55, Wei Liu wrote:
> On Mon, Apr 27, 2020 at 06:19:37PM +0100, Ian Jackson wrote:
>> Andrew Cooper writes ("Re: [PATCH 01/12] libxc/save: Shrink code volume where possible"):
>>> On 14/01/2020 16:48, Ian Jackson wrote:
>>>> Andrew Cooper writes ("[PATCH 01/12] libxc/save: Shrink code volume where possible"):
>>>>> A property of how the error handling (0 on success, nonzero otherwise)
>>>>> allows these calls to be chained together with the ternary operatior.
>>>> I'm quite surprised to find a suggestion like this coming from you in
>>>> particular.
>>> What probably is relevant is that ?: is a common construct in the
>>> hypervisor, which I suppose does colour my expectation of people knowing
>>> exactly what it means and how it behaves.
>> I expect other C programmers to know what ?: does, too.  But I think
>> using it to implement the error monad is quite unusual.  I asked
>> around a bit and my feeling is still that this isn't an improvement.
>>
>>>> Or just to permit
>>>>    rc = write_one_vcpu_basic(ctx, i);    if (rc) goto error;
>>>> (ie on a single line).
>>> OTOH, it should come as no surprise that I'd rather drop this patch
>>> entirely than go with these alternatives, both of which detract from
>>> code clarity. The former for hiding control flow, and the latter for
>>> being atypical layout which unnecessary cognitive load to follow.
>> I think, then, that it would be best to drop this patch, unless Wei
>> (or someone else) disagrees with me.
> I don't feel strongly either way.

I'm confused... I dropped this 3 and a half months ago, because it was
blindingly obvious it was going nowhere.

This is the v1 series which was totally superseded by the v2 series also
posted in January.

~Andrew


^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [PATCH 01/12] libxc/save: Shrink code volume where possible
  2020-04-27 20:00           ` Andrew Cooper
@ 2020-04-28  9:46             ` Wei Liu
  0 siblings, 0 replies; 60+ messages in thread
From: Wei Liu @ 2020-04-28  9:46 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Ian Jackson, Wei Liu, Xen-devel

On Mon, Apr 27, 2020 at 09:00:30PM +0100, Andrew Cooper wrote:
> On 27/04/2020 20:55, Wei Liu wrote:
> > On Mon, Apr 27, 2020 at 06:19:37PM +0100, Ian Jackson wrote:
> >> Andrew Cooper writes ("Re: [PATCH 01/12] libxc/save: Shrink code volume where possible"):
> >>> On 14/01/2020 16:48, Ian Jackson wrote:
> >>>> Andrew Cooper writes ("[PATCH 01/12] libxc/save: Shrink code volume where possible"):
> >>>>> A property of how the error handling (0 on success, nonzero otherwise)
> >>>>> allows these calls to be chained together with the ternary operatior.
> >>>> I'm quite surprised to find a suggestion like this coming from you in
> >>>> particular.
> >>> What probably is relevant is that ?: is a common construct in the
> >>> hypervisor, which I suppose does colour my expectation of people knowing
> >>> exactly what it means and how it behaves.
> >> I expect other C programmers to know what ?: does, too.  But I think
> >> using it to implement the error monad is quite unusual.  I asked
> >> around a bit and my feeling is still that this isn't an improvement.
> >>
> >>>> Or just to permit
> >>>>    rc = write_one_vcpu_basic(ctx, i);    if (rc) goto error;
> >>>> (ie on a single line).
> >>> OTOH, it should come as no surprise that I'd rather drop this patch
> >>> entirely than go with these alternatives, both of which detract from
> >>> code clarity. The former for hiding control flow, and the latter for
> >>> being atypical layout which unnecessary cognitive load to follow.
> >> I think, then, that it would be best to drop this patch, unless Wei
> >> (or someone else) disagrees with me.
> > I don't feel strongly either way.
> 
> I'm confused... I dropped this 3 and a half months ago, because it was
> blindingly obvious it was going nowhere.
> 
> This is the v1 series which was totally superseded by the v2 series also
> posted in January.

OK. I saw Ian's reply on 27th and thought it was still in progress.

Wei.


^ permalink raw reply	[flat|nested] 60+ messages in thread

* Re: [PATCH 17/20] tools/libx[cl]: Plumb static_data_done() up into libxl
  2020-01-15 16:34       ` Andrew Cooper
@ 2020-05-29 15:58         ` Ian Jackson
  0 siblings, 0 replies; 60+ messages in thread
From: Ian Jackson @ 2020-05-29 15:58 UTC (permalink / raw)
  To: Andrew Cooper; +Cc: Anthony Perard, Xen-devel, Wei Liu

Andrew Cooper writes ("Re: [PATCH 17/20] tools/libx[cl]: Plumb static_data_done() up into libxl"):
> There are several things going on here.
> 
> One is the control flow marker of "We reached the end of the static
> data".  A higher level toolstack needs to know this unconditionally,
> which is why the callback is mandatory.
> 
> For v2 compatibility, its callers cope with "this is where an end of
> static data would be in a v3 stream", but that abstracted away so the
> higher level toolstack doesn't know or need to care.
> 
> The missing parameter is "p.s. here are the things we were expecting but
> didn't get - you need to pick up the pieces".  For now, it is synonymous
> with "here is a v2 stream without CPUID data", but that won't be
> accurate in the future if/when new static data records get retrofitted.

Thanks for the explanation.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>


^ permalink raw reply	[flat|nested] 60+ messages in thread

end of thread, other threads:[~2020-05-29 15:59 UTC | newest]

Thread overview: 60+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-24 15:19 [Xen-devel] [PATCH 00/12] Support CPUID/MSR data in migration streams Andrew Cooper
2019-12-24 15:19 ` [Xen-devel] [PATCH 01/12] libxc/save: Shrink code volume where possible Andrew Cooper
2020-01-14 16:48   ` Ian Jackson
2020-01-14 16:55     ` Ian Jackson
2020-01-15 19:22     ` Andrew Cooper
2020-04-27 17:19       ` Ian Jackson
2020-04-27 19:55         ` Wei Liu
2020-04-27 20:00           ` Andrew Cooper
2020-04-28  9:46             ` Wei Liu
2019-12-24 15:19 ` [Xen-devel] [PATCH 02/12] libxc/restore: Introduce functionality to simplify blob handling Andrew Cooper
2020-01-14 16:50   ` Ian Jackson
2019-12-24 15:19 ` [Xen-devel] [PATCH 03/12] libxc/migration: Rationalise the 'checkpointed' field to 'stream_type' Andrew Cooper
2020-01-14 15:58   ` Ian Jackson
2019-12-24 15:19 ` [Xen-devel] [PATCH 04/12] libxc/migration: Adjust layout of struct xc_sr_context Andrew Cooper
2020-01-14 16:04   ` Ian Jackson
2019-12-24 15:19 ` [Xen-devel] [PATCH 05/12] tools/migration: Drop IHDR_VERSION constant from libxc and python Andrew Cooper
2020-01-14 16:05   ` Ian Jackson
2020-01-15 15:29     ` Andrew Cooper
2019-12-24 15:19 ` [Xen-devel] [PATCH 06/12] docs/migration Specify migration v3 and STATIC_DATA_END Andrew Cooper
2020-01-03 14:44   ` Jan Beulich
2020-01-09 14:54     ` Andrew Cooper
2020-01-14 16:07   ` Ian Jackson
2019-12-24 15:19 ` [Xen-devel] [PATCH 07/12] python/migration: Update validation logic to understand a v3 stream Andrew Cooper
2019-12-24 15:19 ` [Xen-devel] [PATCH 08/12] libxc/restore: Support v3 streams, and cope with v2 compatibilty Andrew Cooper
2020-01-14 17:02   ` Ian Jackson
2019-12-24 15:19 ` [Xen-devel] [PATCH 09/12] libxc/save: Write a v3 stream Andrew Cooper
2020-01-14 17:05   ` Ian Jackson
2019-12-24 15:19 ` [Xen-devel] [PATCH 10/12] docs/migration: Specify X86_{CPUID, MSR}_POLICY records Andrew Cooper
2020-01-03 14:49   ` Jan Beulich
2020-01-03 14:55     ` Andrew Cooper
2020-01-03 15:30       ` Jan Beulich
2020-01-09 15:30         ` Andrew Cooper
2020-01-14 16:12           ` Ian Jackson
2020-01-15 15:48             ` Andrew Cooper
2020-01-14 16:08   ` Ian Jackson
2020-01-15 15:36     ` Andrew Cooper
2019-12-24 15:19 ` [Xen-devel] [PATCH 11/12] libxc/restore: Handle X86_{CPUID, MSR}_DATA records Andrew Cooper
2020-01-14 17:16   ` Ian Jackson
2019-12-24 15:19 ` [Xen-devel] [PATCH 12/12] libxc/save: Write " Andrew Cooper
2020-01-14 17:21   ` Ian Jackson
2020-01-15 15:52     ` Andrew Cooper
2020-01-03 13:06 ` [Xen-devel] [PATCH] Use CPUID/MSR data from migration streams Andrew Cooper
2020-01-03 13:06   ` [Xen-devel] [PATCH 15/20] fixup tools/migration: Formatting and style cleanup Andrew Cooper
2020-01-03 13:06   ` [Xen-devel] [PATCH 16/20] tools/libxl: Simplify callback handling in libxl-save-helper Andrew Cooper
2020-01-14 17:27     ` Ian Jackson
2020-01-15 16:16       ` Andrew Cooper
2020-01-03 13:06   ` [Xen-devel] [PATCH 17/20] tools/libx[cl]: Plumb static_data_done() up into libxl Andrew Cooper
2020-01-14 17:30     ` Ian Jackson
2020-01-15 16:34       ` Andrew Cooper
2020-05-29 15:58         ` Ian Jackson
2020-01-03 13:06   ` [Xen-devel] [PATCH 18/20] tools/libxl: Plumb domain_create_state down into libxl__build_pre() Andrew Cooper
2020-01-14 17:32     ` Ian Jackson
2020-01-03 13:06   ` [Xen-devel] [PATCH 19/20] tools/libxl: Re-position CPUID handling during domain construction Andrew Cooper
2020-01-14 17:33     ` Ian Jackson
2020-01-14 17:51       ` Andrew Cooper
2020-01-14 18:12         ` Ian Jackson
2020-01-03 13:06   ` [Xen-devel] [PATCH 20/20] tools/libxc: Restore CPUID/MSR data found in the migration stream Andrew Cooper
2020-01-14 17:34     ` Ian Jackson
2020-01-15 18:53 ` [Xen-devel] [PATCH 0.5/12] tools/migration: Formatting and style cleanup Andrew Cooper
2020-01-15 21:26   ` Ian Jackson

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.