* [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot.
@ 2018-03-13 8:33 junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 01/10] RFC: Add save and support snapshot dependency function to block driver junyan.he
` (12 more replies)
0 siblings, 13 replies; 17+ messages in thread
From: junyan.he @ 2018-03-13 8:33 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, mreitz, pbonzini, crosthwaite.peter, quintela, rth,
dgilbert, famz, Junyan He
From: Junyan He <junyan.he@intel.com>
The nvdimm size is huge, sometimes it is more than 256G or even more.
This is a huge burden for snapshot saving. One snapshot point with
nvdimm may occupy more than 50G disk space even with compression
enabled.
We need to introduce dependent snapshot manner to solve this problem.
The first snapshot point should always be saved completely, and enable
dirty log trace after saving for nvdimm memory region. The later snapshot
point should add the reference to previous snapshot's nvdimm data and
just saving dirty pages. This can save a lot of disk and time if the
snapshot operations are triggered frequently.
We add save_snapshot_dependency functions to QCOW2 file system firstly, the
later snapshot will add reference to previous dependent snapshot's data
cluster. There is an alignment problem here, the dependent data should
always be cluster aligned. We need to add some padding data when saving
the snapshot to make it always cluster aligned.
The logic between nvdimm and ram for snapshot saving is a little confused
now, we need to exclude nvdimm kind memory region from ram list and the
dirty log tracing setting is also not very clear. Maybe we can separate the
snapshot saving from the migration logic later to make code clean.
In theory, this kind of manner can apply to any kind of memory. But because
it need to turn dirty log trace on, the performance may decline. So we just
enable it for nvdimm kind memory firstly.
Signed-off-by: Junyan He <junyan.he@intel.com>
---
Makefile.target | 1 +
block/qcow2-snapshot.c | 154 ++++++++++++++++++++++
block/qcow2.c | 2 +
block/qcow2.h | 7 +
block/snapshot.c | 45 +++++++
exec.c | 7 +
hw/ppc/spapr.c | 2 +-
hw/s390x/s390-stattrib.c | 2 +-
include/block/block_int.h | 9 ++
include/block/snapshot.h | 7 +
include/exec/memory.h | 9 ++
include/exec/ram_addr.h | 2 +
include/migration/misc.h | 4 +
include/migration/register.h | 2 +-
include/migration/snapshot.h | 3 +
memory.c | 18 ++-
migration/block.c | 2 +-
migration/nvdimm.c | 1033 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
migration/qemu-file.c | 61 +++++++++
migration/qemu-file.h | 14 ++
migration/ram.c | 19 ++-
migration/savevm.c | 62 ++++++++-
vl.c | 1 +
23 files changed, 1452 insertions(+), 14 deletions(-)
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH 01/10] RFC: Add save and support snapshot dependency function to block driver.
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
@ 2018-03-13 8:33 ` junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 02/10] RFC: Implement qcow2's snapshot dependent saving function junyan.he
` (11 subsequent siblings)
12 siblings, 0 replies; 17+ messages in thread
From: junyan.he @ 2018-03-13 8:33 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, mreitz, pbonzini, crosthwaite.peter, quintela, rth,
dgilbert, famz, Junyan He
From: Junyan He <junyan.he@intel.com>
We want to support incremental snapshot saving, this needs the file
system support dependency saving. Later snapshots may ref the dependent
snapshot's content, and most time should be cluster aligned.
Add a query function to check whether the file system support this, and
use the save_dependency function to do the real work.
Signed-off-by: Junyan He <junyan.he@intel.com>
---
include/block/block_int.h | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 64a5700..be1eca3 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -274,6 +274,15 @@ struct BlockDriver {
const char *snapshot_id,
const char *name,
Error **errp);
+ int (*bdrv_snapshot_save_dependency)(BlockDriverState *bs,
+ const char *depend_snapshot_id,
+ int64_t depend_offset,
+ int64_t depend_size,
+ int64_t offset,
+ Error **errp);
+ int (*bdrv_snapshot_support_dependency)(BlockDriverState *bs,
+ int32_t *alignment);
+
int (*bdrv_get_info)(BlockDriverState *bs, BlockDriverInfo *bdi);
ImageInfoSpecific *(*bdrv_get_specific_info)(BlockDriverState *bs);
--
2.7.4
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH 02/10] RFC: Implement qcow2's snapshot dependent saving function.
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 01/10] RFC: Add save and support snapshot dependency function to block driver junyan.he
@ 2018-03-13 8:33 ` junyan.he
2018-05-08 14:50 ` Eric Blake
2018-03-13 8:33 ` [Qemu-devel] [PATCH 03/10] RFC: Implement save and support snapshot dependency in block driver layer junyan.he
` (10 subsequent siblings)
12 siblings, 1 reply; 17+ messages in thread
From: junyan.he @ 2018-03-13 8:33 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, mreitz, pbonzini, crosthwaite.peter, quintela, rth,
dgilbert, famz, Junyan He
From: Junyan He <junyan.he@intel.com>
For qcow2 format, we can increase the cluster's reference count of
dependent snapshot content and link the offset to the L2 table of
the new snapshot point. This way can avoid obvious snapshot's dependent
relationship, so when we delete some snapshot point, just decrease the
cluster count and no need to check further.
Signed-off-by: Junyan He <junyan.he@intel.com>
---
block/qcow2-snapshot.c | 154 +++++++++++++++++++++++++++++++++++++++++++++++++
block/qcow2.c | 2 +
block/qcow2.h | 7 +++
3 files changed, 163 insertions(+)
diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c
index cee25f5..8e83084 100644
--- a/block/qcow2-snapshot.c
+++ b/block/qcow2-snapshot.c
@@ -736,3 +736,157 @@ int qcow2_snapshot_load_tmp(BlockDriverState *bs,
return 0;
}
+
+int qcow2_snapshot_save_dependency(BlockDriverState *bs,
+ const char *depend_snapshot_id,
+ int64_t depend_offset,
+ int64_t depend_size,
+ int64_t offset,
+ Error **errp)
+{
+ int snapshot_index;
+ BDRVQcow2State *s = bs->opaque;
+ QCowSnapshot *sn;
+ int ret;
+ int64_t i;
+ int64_t total_bytes = depend_size;
+ int64_t depend_offset1, offset1;
+ uint64_t *depend_l1_table = NULL;
+ uint64_t depend_l1_bytes;
+ uint64_t *depend_l2_table = NULL;
+ uint64_t depend_l2_offset;
+ uint64_t depend_entry;
+ QCowL2Meta l2meta;
+
+ assert(bs->read_only == false);
+
+ if (depend_snapshot_id == NULL) {
+ return 0;
+ }
+
+ if (!QEMU_IS_ALIGNED(depend_offset, s->cluster_size)) {
+ error_setg(errp, "Specified snapshot offset is not multiple of %u",
+ s->cluster_size);
+ return -EINVAL;
+ }
+
+ if (!QEMU_IS_ALIGNED(offset, s->cluster_size)) {
+ error_setg(errp, "Offset is not multiple of %u", s->cluster_size);
+ return -EINVAL;
+ }
+
+ if (!QEMU_IS_ALIGNED(depend_size, s->cluster_size)) {
+ error_setg(errp, "depend_size is not multiple of %u", s->cluster_size);
+ return -EINVAL;
+ }
+
+ snapshot_index = find_snapshot_by_id_and_name(bs, NULL, depend_snapshot_id);
+ /* Search the snapshot */
+ if (snapshot_index < 0) {
+ error_setg(errp, "Can't find snapshot");
+ return -ENOENT;
+ }
+
+ sn = &s->snapshots[snapshot_index];
+ if (sn->disk_size != bs->total_sectors * BDRV_SECTOR_SIZE) {
+ error_report("qcow2: depend on the snapshots with different disk "
+ "size is not implemented");
+ return -ENOTSUP;
+ }
+
+ /* Only can save dependency of snapshot's vmstate data */
+ depend_offset1 = depend_offset + qcow2_vm_state_offset(s);
+ offset1 = offset + qcow2_vm_state_offset(s);
+
+ depend_l1_bytes = s->l1_size * sizeof(uint64_t);
+ depend_l1_table = g_try_malloc0(depend_l1_bytes);
+ if (depend_l1_table == NULL) {
+ return -ENOMEM;
+ }
+
+ ret = bdrv_pread(bs->file, sn->l1_table_offset, depend_l1_table,
+ depend_l1_bytes);
+ if (ret < 0) {
+ g_free(depend_l1_table);
+ goto out;
+ }
+ for (i = 0; i < depend_l1_bytes / sizeof(uint64_t); i++) {
+ be64_to_cpus(&depend_l1_table[i]);
+ }
+
+ while (total_bytes) {
+ assert(total_bytes > 0);
+ /* Find the cluster of depend */
+ depend_l2_offset =
+ depend_l1_table[depend_offset1 >> (s->l2_bits + s->cluster_bits)];
+ depend_l2_offset &= L1E_OFFSET_MASK;
+ if (depend_l2_offset == 0) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ if (offset_into_cluster(s, depend_l2_offset)) {
+ qcow2_signal_corruption(bs, true, -1, -1, "L2 table offset %#"
+ PRIx64 " unaligned (L1 index: %#"
+ PRIx64 ")",
+ depend_l2_offset,
+ depend_offset1 >>
+ (s->l2_bits + s->cluster_bits));
+ return -EIO;
+ }
+
+ ret = qcow2_cache_get(bs, s->l2_table_cache, depend_l2_offset,
+ (void **)(&depend_l2_table));
+ if (ret < 0) {
+ goto out;
+ }
+
+ depend_entry =
+ be64_to_cpu(
+ depend_l2_table[offset_to_l2_index(s, depend_offset1)]);
+ if (depend_entry == 0) {
+ ret = -EINVAL;
+ qcow2_cache_put(s->l2_table_cache, (void **)(&depend_l2_table));
+ goto out;
+ }
+
+ memset(&l2meta, 0, sizeof(l2meta));
+ l2meta.offset = offset1;
+ l2meta.alloc_offset = (depend_entry & L2E_OFFSET_MASK);
+ l2meta.nb_clusters = 1;
+ /* Add a ref to this cluster */
+ ret = qcow2_update_cluster_refcount(
+ bs, l2meta.alloc_offset >> s->cluster_bits,
+ 1, false, QCOW2_DISCARD_SNAPSHOT);
+ if (ret < 0) {
+ qcow2_cache_put(s->l2_table_cache, (void **)(&depend_l2_table));
+ goto out;
+ }
+
+ ret = qcow2_alloc_cluster_link_l2(bs, &l2meta);
+ if (ret < 0) {
+ qcow2_cache_put(s->l2_table_cache, (void **)(&depend_l2_table));
+ goto out;
+ }
+
+ total_bytes -= s->cluster_size;
+ offset1 += s->cluster_size;
+ depend_offset1 += s->cluster_size;
+
+ qcow2_cache_put(s->l2_table_cache, (void **)(&depend_l2_table));
+ }
+
+out:
+ g_free(depend_l1_table);
+ return ret;
+}
+
+int qcow2_snapshot_support_dependency(BlockDriverState *bs, int32_t *alignment)
+{
+ BDRVQcow2State *s = bs->opaque;
+ if (alignment) {
+ *alignment = s->cluster_size;
+ }
+
+ return 1;
+}
diff --git a/block/qcow2.c b/block/qcow2.c
index 071dc4d..9786ba4 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -4371,6 +4371,8 @@ BlockDriver bdrv_qcow2 = {
.bdrv_snapshot_delete = qcow2_snapshot_delete,
.bdrv_snapshot_list = qcow2_snapshot_list,
.bdrv_snapshot_load_tmp = qcow2_snapshot_load_tmp,
+ .bdrv_snapshot_support_dependency = qcow2_snapshot_support_dependency,
+ .bdrv_snapshot_save_dependency = qcow2_snapshot_save_dependency,
.bdrv_measure = qcow2_measure,
.bdrv_get_info = qcow2_get_info,
.bdrv_get_specific_info = qcow2_get_specific_info,
diff --git a/block/qcow2.h b/block/qcow2.h
index 1a84cc7..dc7ef45 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -640,6 +640,13 @@ int qcow2_snapshot_load_tmp(BlockDriverState *bs,
void qcow2_free_snapshots(BlockDriverState *bs);
int qcow2_read_snapshots(BlockDriverState *bs);
+int qcow2_snapshot_save_dependency(BlockDriverState *bs,
+ const char *depend_snapshot_id,
+ int64_t depend_offset,
+ int64_t depend_size,
+ int64_t offset,
+ Error **errp);
+int qcow2_snapshot_support_dependency(BlockDriverState *bs, int32_t *alignment);
/* qcow2-cache.c functions */
Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables,
--
2.7.4
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH 03/10] RFC: Implement save and support snapshot dependency in block driver layer.
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 01/10] RFC: Add save and support snapshot dependency function to block driver junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 02/10] RFC: Implement qcow2's snapshot dependent saving function junyan.he
@ 2018-03-13 8:33 ` junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 04/10] RFC: Set memory_region_set_log available for more client junyan.he
` (9 subsequent siblings)
12 siblings, 0 replies; 17+ messages in thread
From: junyan.he @ 2018-03-13 8:33 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, mreitz, pbonzini, crosthwaite.peter, quintela, rth,
dgilbert, famz, Junyan He
From: Junyan He <junyan.he@intel.com>
Signed-off-by: Junyan He <junyan.he@intel.com>
---
block/snapshot.c | 45 +++++++++++++++++++++++++++++++++++++++++++++
include/block/snapshot.h | 7 +++++++
2 files changed, 52 insertions(+)
diff --git a/block/snapshot.c b/block/snapshot.c
index eacc1f1..8cc40ac 100644
--- a/block/snapshot.c
+++ b/block/snapshot.c
@@ -401,6 +401,51 @@ int bdrv_snapshot_load_tmp_by_id_or_name(BlockDriverState *bs,
return ret;
}
+int bdrv_snapshot_save_dependency(BlockDriverState *bs,
+ const char *depend_snapshot_id,
+ int64_t depend_offset,
+ int64_t depend_size,
+ int64_t offset,
+ Error **errp)
+{
+ BlockDriver *drv = bs->drv;
+
+ if (!drv) {
+ return -ENOMEDIUM;
+ }
+
+ if (drv->bdrv_snapshot_save_dependency) {
+ return drv->bdrv_snapshot_save_dependency(bs, depend_snapshot_id,
+ depend_offset, depend_size,
+ offset, errp);
+ }
+
+ if (bs->file) {
+ return bdrv_snapshot_save_dependency(bs->file->bs, depend_snapshot_id,
+ depend_offset, depend_size,
+ offset, errp);
+ }
+
+ return -ENOTSUP;
+}
+
+int bdrv_snapshot_support_dependency(BlockDriverState *bs, int32_t *alignment)
+{
+ BlockDriver *drv = bs->drv;
+ if (!drv || !bdrv_is_inserted(bs) || bdrv_is_read_only(bs)) {
+ return 0;
+ }
+
+ if (drv->bdrv_snapshot_support_dependency) {
+ return drv->bdrv_snapshot_support_dependency(bs, alignment);
+ }
+
+ if (bs->file != NULL) {
+ return bdrv_snapshot_support_dependency(bs->file->bs, alignment);
+ }
+
+ return -ENOTSUP;
+}
/* Group operations. All block drivers are involved.
* These functions will properly handle dataplane (take aio_context_acquire
diff --git a/include/block/snapshot.h b/include/block/snapshot.h
index f73d109..e5bf06f 100644
--- a/include/block/snapshot.h
+++ b/include/block/snapshot.h
@@ -73,6 +73,13 @@ int bdrv_snapshot_load_tmp(BlockDriverState *bs,
int bdrv_snapshot_load_tmp_by_id_or_name(BlockDriverState *bs,
const char *id_or_name,
Error **errp);
+int bdrv_snapshot_save_dependency(BlockDriverState *bs,
+ const char *depend_snapshot_id,
+ int64_t depend_offset,
+ int64_t depend_size,
+ int64_t offset,
+ Error **errp);
+int bdrv_snapshot_support_dependency(BlockDriverState *bs, int32_t *alignment);
/* Group operations. All block drivers are involved.
--
2.7.4
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH 04/10] RFC: Set memory_region_set_log available for more client.
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
` (2 preceding siblings ...)
2018-03-13 8:33 ` [Qemu-devel] [PATCH 03/10] RFC: Implement save and support snapshot dependency in block driver layer junyan.he
@ 2018-03-13 8:33 ` junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 05/10] RFC: Add memory region snapshot bitmap get function junyan.he
` (8 subsequent siblings)
12 siblings, 0 replies; 17+ messages in thread
From: junyan.he @ 2018-03-13 8:33 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, mreitz, pbonzini, crosthwaite.peter, quintela, rth,
dgilbert, famz, Junyan He
From: Junyan He <junyan.he@intel.com>
We need to collect dirty log for nvdimm kind memory, need to enable
memory_region_set_log for more clients rather than just VGA.
Signed-off-by: Junyan He <junyan.he@intel.com>
---
memory.c | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/memory.c b/memory.c
index e70b64b..4a8a2fe 100644
--- a/memory.c
+++ b/memory.c
@@ -1921,11 +1921,12 @@ void memory_region_set_log(MemoryRegion *mr, bool log, unsigned client)
uint8_t mask = 1 << client;
uint8_t old_logging;
- assert(client == DIRTY_MEMORY_VGA);
- old_logging = mr->vga_logging_count;
- mr->vga_logging_count += log ? 1 : -1;
- if (!!old_logging == !!mr->vga_logging_count) {
- return;
+ if (client == DIRTY_MEMORY_VGA) {
+ old_logging = mr->vga_logging_count;
+ mr->vga_logging_count += log ? 1 : -1;
+ if (!!old_logging == !!mr->vga_logging_count) {
+ return;
+ }
}
memory_region_transaction_begin();
--
2.7.4
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH 05/10] RFC: Add memory region snapshot bitmap get function.
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
` (3 preceding siblings ...)
2018-03-13 8:33 ` [Qemu-devel] [PATCH 04/10] RFC: Set memory_region_set_log available for more client junyan.he
@ 2018-03-13 8:33 ` junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 06/10] RFC: Add save dependency functions to qemu_file junyan.he
` (7 subsequent siblings)
12 siblings, 0 replies; 17+ messages in thread
From: junyan.he @ 2018-03-13 8:33 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, mreitz, pbonzini, crosthwaite.peter, quintela, rth,
dgilbert, famz, Junyan He
From: Junyan He <junyan.he@intel.com>
We need to get the bitmap content of the snapshot when enable dirty
log trace for nvdimm.
Signed-off-by: Junyan He <junyan.he@intel.com>
---
exec.c | 7 +++++++
include/exec/memory.h | 9 +++++++++
include/exec/ram_addr.h | 2 ++
memory.c | 7 +++++++
4 files changed, 25 insertions(+)
diff --git a/exec.c b/exec.c
index a9181e6..3d2bf0d 100644
--- a/exec.c
+++ b/exec.c
@@ -1235,6 +1235,13 @@ bool cpu_physical_memory_snapshot_get_dirty(DirtyBitmapSnapshot *snap,
return false;
}
+unsigned long *cpu_physical_memory_snapshot_get_dirty_bitmap
+ (DirtyBitmapSnapshot *snap)
+{
+ assert(snap);
+ return snap->dirty;
+}
+
/* Called from RCU critical section */
hwaddr memory_region_section_get_iotlb(CPUState *cpu,
MemoryRegionSection *section,
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 31eae0a..f742995 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -1179,6 +1179,15 @@ bool memory_region_snapshot_get_dirty(MemoryRegion *mr,
hwaddr addr, hwaddr size);
/**
+ * memory_region_snapshot_get_dirty_bitmap: Get the dirty bitmap data of
+ * snapshot.
+ *
+ * @snap: the dirty bitmap snapshot
+ */
+unsigned long *memory_region_snapshot_get_dirty_bitmap
+ (DirtyBitmapSnapshot *snap);
+
+/**
* memory_region_reset_dirty: Mark a range of pages as clean, for a specified
* client.
*
diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
index cf2446a..ce366c1 100644
--- a/include/exec/ram_addr.h
+++ b/include/exec/ram_addr.h
@@ -371,6 +371,8 @@ DirtyBitmapSnapshot *cpu_physical_memory_snapshot_and_clear_dirty
bool cpu_physical_memory_snapshot_get_dirty(DirtyBitmapSnapshot *snap,
ram_addr_t start,
ram_addr_t length);
+unsigned long *cpu_physical_memory_snapshot_get_dirty_bitmap
+ (DirtyBitmapSnapshot *snap);
static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
ram_addr_t length)
diff --git a/memory.c b/memory.c
index 4a8a2fe..68f17f0 100644
--- a/memory.c
+++ b/memory.c
@@ -1991,6 +1991,13 @@ DirtyBitmapSnapshot *memory_region_snapshot_and_clear_dirty(MemoryRegion *mr,
memory_region_get_ram_addr(mr) + addr, size, client);
}
+unsigned long *memory_region_snapshot_get_dirty_bitmap
+ (DirtyBitmapSnapshot *snap)
+{
+ assert(snap);
+ return cpu_physical_memory_snapshot_get_dirty_bitmap(snap);
+}
+
bool memory_region_snapshot_get_dirty(MemoryRegion *mr, DirtyBitmapSnapshot *snap,
hwaddr addr, hwaddr size)
{
--
2.7.4
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH 06/10] RFC: Add save dependency functions to qemu_file
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
` (4 preceding siblings ...)
2018-03-13 8:33 ` [Qemu-devel] [PATCH 05/10] RFC: Add memory region snapshot bitmap get function junyan.he
@ 2018-03-13 8:33 ` junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 07/10] RFC: Add get_current_snapshot_info to get the snapshot state junyan.he
` (6 subsequent siblings)
12 siblings, 0 replies; 17+ messages in thread
From: junyan.he @ 2018-03-13 8:33 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, mreitz, pbonzini, crosthwaite.peter, quintela, rth,
dgilbert, famz, Junyan He
From: Junyan He <junyan.he@intel.com>
When we save snapshot, we need qemu_file to support save dependency
operations. It should call brv_driver's save dependency functions
to implement these operations.
Signed-off-by: Junyan He <junyan.he@intel.com>
---
migration/qemu-file.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++
migration/qemu-file.h | 14 ++++++++++++
migration/savevm.c | 33 +++++++++++++++++++++++++---
3 files changed, 105 insertions(+), 3 deletions(-)
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 2ab2bf3..9d2a39a 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -46,10 +46,13 @@ struct QEMUFile {
int buf_index;
int buf_size; /* 0 when writing */
uint8_t buf[IO_BUF_SIZE];
+ char ref_name_str[128]; /* maybe snapshot id */
DECLARE_BITMAP(may_free, MAX_IOV_SIZE);
struct iovec iov[MAX_IOV_SIZE];
unsigned int iovcnt;
+ bool support_dependency;
+ int32_t dependency_aligment;
int last_error;
};
@@ -745,3 +748,61 @@ void qemu_file_set_blocking(QEMUFile *f, bool block)
f->ops->set_blocking(f->opaque, block);
}
}
+
+void qemu_file_set_support_dependency(QEMUFile *f, int32_t alignment)
+{
+ f->dependency_aligment = alignment;
+ f->support_dependency = true;
+}
+
+bool qemu_file_is_support_dependency(QEMUFile *f, int32_t *alignment)
+{
+ if (f->support_dependency && alignment) {
+ *alignment = f->dependency_aligment;
+ }
+
+ return f->support_dependency;
+}
+
+/* This function set the reference name for snapshot usage. Sometimes it needs
+ * to depend on other snapshot's data to avoid redundance.
+ */
+bool qemu_file_set_ref_name(QEMUFile *f, const char *name)
+{
+ if (strlen(name) + 1 > sizeof(f->ref_name_str)) {
+ return false;
+ }
+
+ memcpy(f->ref_name_str, name, strlen(name) + 1);
+ return true;
+}
+
+ssize_t qemu_file_save_dependency(QEMUFile *f, int64_t depend_offset,
+ int64_t size)
+{
+ ssize_t ret;
+
+ if (f->support_dependency == false) {
+ return -1;
+ }
+
+ assert(f->ops->save_dependency);
+
+ if (!QEMU_IS_ALIGNED(depend_offset, f->dependency_aligment)) {
+ return -1;
+ }
+
+ qemu_fflush(f);
+
+ if (!QEMU_IS_ALIGNED(f->pos, f->dependency_aligment)) {
+ return -1;
+ }
+
+ ret = f->ops->save_dependency(f->opaque, f->ref_name_str,
+ depend_offset, size, f->pos);
+ if (ret > 0) {
+ f->pos += size;
+ }
+
+ return ret;
+}
diff --git a/migration/qemu-file.h b/migration/qemu-file.h
index aae4e5e..137b917 100644
--- a/migration/qemu-file.h
+++ b/migration/qemu-file.h
@@ -57,6 +57,14 @@ typedef ssize_t (QEMUFileWritevBufferFunc)(void *opaque, struct iovec *iov,
int iovcnt, int64_t pos);
/*
+ * This function add reference to the dependency data in snapshot specified by
+ * ref_name_str to this file's offset
+ */
+typedef ssize_t (QEMUFileSaveDependencyFunc)(void *opaque, const char *name,
+ int64_t depend_offset,
+ int64_t offset, int64_t size);
+
+/*
* This function provides hooks around different
* stages of RAM migration.
* 'opaque' is the backend specific data in QEMUFile
@@ -104,6 +112,7 @@ typedef struct QEMUFileOps {
QEMUFileWritevBufferFunc *writev_buffer;
QEMURetPathFunc *get_return_path;
QEMUFileShutdownFunc *shut_down;
+ QEMUFileSaveDependencyFunc *save_dependency;
} QEMUFileOps;
typedef struct QEMUFileHooks {
@@ -153,6 +162,11 @@ int qemu_file_shutdown(QEMUFile *f);
QEMUFile *qemu_file_get_return_path(QEMUFile *f);
void qemu_fflush(QEMUFile *f);
void qemu_file_set_blocking(QEMUFile *f, bool block);
+bool qemu_file_set_ref_name(QEMUFile *f, const char *name);
+void qemu_file_set_support_dependency(QEMUFile *f, int32_t alignment);
+bool qemu_file_is_support_dependency(QEMUFile *f, int32_t *alignment);
+ssize_t qemu_file_save_dependency(QEMUFile *f, int64_t depend_offset,
+ int64_t size);
size_t qemu_get_counted_string(QEMUFile *f, char buf[256]);
diff --git a/migration/savevm.c b/migration/savevm.c
index 358c5b5..1bbd6aa 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -196,6 +196,20 @@ static ssize_t block_writev_buffer(void *opaque, struct iovec *iov, int iovcnt,
return qiov.size;
}
+static ssize_t block_save_dependency(void *opaque, const char *id_name,
+ int64_t depend_offset,
+ int64_t offset, int64_t size)
+{
+ int ret = bdrv_snapshot_save_dependency(opaque, id_name,
+ depend_offset, offset,
+ size, NULL);
+ if (ret < 0) {
+ return ret;
+ }
+
+ return size;
+}
+
static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos,
size_t size)
{
@@ -213,15 +227,28 @@ static const QEMUFileOps bdrv_read_ops = {
};
static const QEMUFileOps bdrv_write_ops = {
- .writev_buffer = block_writev_buffer,
- .close = bdrv_fclose
+ .writev_buffer = block_writev_buffer,
+ .close = bdrv_fclose,
+ .save_dependency = block_save_dependency
};
static QEMUFile *qemu_fopen_bdrv(BlockDriverState *bs, int is_writable)
{
+ int ret = 0;
+ int32_t alignment = 0;
+ QEMUFile *f = NULL;
+
if (is_writable) {
- return qemu_fopen_ops(bs, &bdrv_write_ops);
+ f = qemu_fopen_ops(bs, &bdrv_write_ops);
+
+ ret = bdrv_snapshot_support_dependency(bs, &alignment);
+ if (ret > 0) {
+ qemu_file_set_support_dependency(f, alignment);
+ }
+
+ return f;
}
+
return qemu_fopen_ops(bs, &bdrv_read_ops);
}
--
2.7.4
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH 07/10] RFC: Add get_current_snapshot_info to get the snapshot state.
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
` (5 preceding siblings ...)
2018-03-13 8:33 ` [Qemu-devel] [PATCH 06/10] RFC: Add save dependency functions to qemu_file junyan.he
@ 2018-03-13 8:33 ` junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 08/10] RFC: Add a section_id parameter to save_live_iterate call junyan.he
` (5 subsequent siblings)
12 siblings, 0 replies; 17+ messages in thread
From: junyan.he @ 2018-03-13 8:33 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, mreitz, pbonzini, crosthwaite.peter, quintela, rth,
dgilbert, famz, Junyan He
From: Junyan He <junyan.he@intel.com>
We need to know the snapshot saving information when we do dependent
snapshot saving, e.g the name of previous snapshot. Add this global
function to query the snapshot status is usable.
Signed-off-by: Junyan He <junyan.he@intel.com>
---
include/migration/snapshot.h | 3 +++
migration/savevm.c | 27 +++++++++++++++++++++++++++
2 files changed, 30 insertions(+)
diff --git a/include/migration/snapshot.h b/include/migration/snapshot.h
index c85b6ec..0b950ce 100644
--- a/include/migration/snapshot.h
+++ b/include/migration/snapshot.h
@@ -15,7 +15,10 @@
#ifndef QEMU_MIGRATION_SNAPSHOT_H
#define QEMU_MIGRATION_SNAPSHOT_H
+#include "block/snapshot.h"
+
int save_snapshot(const char *name, Error **errp);
int load_snapshot(const char *name, Error **errp);
+int get_current_snapshot_info(QEMUSnapshotInfo *sn);
#endif
diff --git a/migration/savevm.c b/migration/savevm.c
index 1bbd6aa..3a9b904 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -2212,6 +2212,29 @@ int qemu_loadvm_state(QEMUFile *f)
return ret;
}
+static int in_snap_saving;
+static QEMUSnapshotInfo in_snap_saving_sn;
+
+int get_current_snapshot_info(QEMUSnapshotInfo *sn)
+{
+ if (in_snap_saving && sn) {
+ memcpy(sn, &in_snap_saving_sn, sizeof(QEMUSnapshotInfo));
+ }
+
+ return in_snap_saving;
+}
+
+static void set_current_snapshot_info(QEMUSnapshotInfo *sn)
+{
+ if (sn) {
+ memcpy(&in_snap_saving_sn, sn, sizeof(QEMUSnapshotInfo));
+ in_snap_saving = 1;
+ } else {
+ memset(&in_snap_saving_sn, 0, sizeof(QEMUSnapshotInfo));
+ in_snap_saving = 0;
+ }
+}
+
int save_snapshot(const char *name, Error **errp)
{
BlockDriverState *bs, *bs1;
@@ -2282,6 +2305,8 @@ int save_snapshot(const char *name, Error **errp)
strftime(sn->name, sizeof(sn->name), "vm-%Y%m%d%H%M%S", &tm);
}
+ set_current_snapshot_info(sn);
+
/* save the VM state */
f = qemu_fopen_bdrv(bs, 1);
if (!f) {
@@ -2313,6 +2338,8 @@ int save_snapshot(const char *name, Error **errp)
ret = 0;
the_end:
+ set_current_snapshot_info(NULL);
+
if (aio_context) {
aio_context_release(aio_context);
}
--
2.7.4
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH 08/10] RFC: Add a section_id parameter to save_live_iterate call.
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
` (6 preceding siblings ...)
2018-03-13 8:33 ` [Qemu-devel] [PATCH 07/10] RFC: Add get_current_snapshot_info to get the snapshot state junyan.he
@ 2018-03-13 8:33 ` junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 09/10] RFC: Add nvdimm snapshot saving to migration junyan.he
` (4 subsequent siblings)
12 siblings, 0 replies; 17+ messages in thread
From: junyan.he @ 2018-03-13 8:33 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, mreitz, pbonzini, crosthwaite.peter, quintela, rth,
dgilbert, famz, Junyan He
From: Junyan He <junyan.he@intel.com>
We need to know the section_id when we do snapshot saving.
Add a parameter to save_live_iterate function call.
Signed-off-by: Junyan He <junyan.he@intel.com>
---
hw/ppc/spapr.c | 2 +-
hw/s390x/s390-stattrib.c | 2 +-
include/migration/register.h | 2 +-
migration/block.c | 2 +-
migration/ram.c | 2 +-
migration/savevm.c | 2 +-
6 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 7e1c858..4cde4f4 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1974,7 +1974,7 @@ static int htab_save_later_pass(QEMUFile *f, sPAPRMachineState *spapr,
#define MAX_ITERATION_NS 5000000 /* 5 ms */
#define MAX_KVM_BUF_SIZE 2048
-static int htab_save_iterate(QEMUFile *f, void *opaque)
+static int htab_save_iterate(QEMUFile *f, void *opaque, int section_id)
{
sPAPRMachineState *spapr = opaque;
int fd;
diff --git a/hw/s390x/s390-stattrib.c b/hw/s390x/s390-stattrib.c
index adf07ef..18ece84 100644
--- a/hw/s390x/s390-stattrib.c
+++ b/hw/s390x/s390-stattrib.c
@@ -246,7 +246,7 @@ static int cmma_save(QEMUFile *f, void *opaque, int final)
return ret;
}
-static int cmma_save_iterate(QEMUFile *f, void *opaque)
+static int cmma_save_iterate(QEMUFile *f, void *opaque, int section_id)
{
return cmma_save(f, opaque, 0);
}
diff --git a/include/migration/register.h b/include/migration/register.h
index f4f7bdc..7f7df2c 100644
--- a/include/migration/register.h
+++ b/include/migration/register.h
@@ -31,7 +31,7 @@ typedef struct SaveVMHandlers {
* use data that is local to the migration thread or protected
* by other locks.
*/
- int (*save_live_iterate)(QEMUFile *f, void *opaque);
+ int (*save_live_iterate)(QEMUFile *f, void *opaque, int section_id);
/* This runs outside the iothread lock! */
int (*save_setup)(QEMUFile *f, void *opaque);
diff --git a/migration/block.c b/migration/block.c
index 1f03946..6d4c8a3 100644
--- a/migration/block.c
+++ b/migration/block.c
@@ -755,7 +755,7 @@ static int block_save_setup(QEMUFile *f, void *opaque)
return ret;
}
-static int block_save_iterate(QEMUFile *f, void *opaque)
+static int block_save_iterate(QEMUFile *f, void *opaque, int section_id)
{
int ret;
int64_t last_ftell = qemu_ftell(f);
diff --git a/migration/ram.c b/migration/ram.c
index 3b6c077..d1db422 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2249,7 +2249,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
* @f: QEMUFile where to send the data
* @opaque: RAMState pointer
*/
-static int ram_save_iterate(QEMUFile *f, void *opaque)
+static int ram_save_iterate(QEMUFile *f, void *opaque, int section_id)
{
RAMState **temp = opaque;
RAMState *rs = *temp;
diff --git a/migration/savevm.c b/migration/savevm.c
index 3a9b904..ce4133a 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -1072,7 +1072,7 @@ int qemu_savevm_state_iterate(QEMUFile *f, bool postcopy)
save_section_header(f, se, QEMU_VM_SECTION_PART);
- ret = se->ops->save_live_iterate(f, se->opaque);
+ ret = se->ops->save_live_iterate(f, se->opaque, se->section_id);
trace_savevm_section_end(se->idstr, se->section_id, ret);
save_section_footer(f, se);
--
2.7.4
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH 09/10] RFC: Add nvdimm snapshot saving to migration.
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
` (7 preceding siblings ...)
2018-03-13 8:33 ` [Qemu-devel] [PATCH 08/10] RFC: Add a section_id parameter to save_live_iterate call junyan.he
@ 2018-03-13 8:33 ` junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 10/10] RFC: Enable nvdimm snapshot functions junyan.he
` (3 subsequent siblings)
12 siblings, 0 replies; 17+ messages in thread
From: junyan.he @ 2018-03-13 8:33 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, mreitz, pbonzini, crosthwaite.peter, quintela, rth,
dgilbert, famz, Junyan He
From: Junyan He <junyan.he@intel.com>
The nvdimm size is huge, sometimes is more than 256G or even more.
This is a huge burden for snapshot saving. One snapshot point with
nvdimm may occupy more than 50G disk space even with compression
enabled.
We need to introduce dependent snapshot manner to solve this problem.
The first snapshot point should always be saved completely, and enable
dirty log trace after saving for nvdimm memory region. The later snapshot
point should add the reference to previous snapshot's nvdimm data and
just saving dirty pages. This can save a lot of disk and time if the
snapshot operations are triggered frequently.
Signed-off-by: Junyan He <junyan.he@intel.com>
---
Makefile.target | 1 +
include/migration/misc.h | 4 +
migration/nvdimm.c | 1033 ++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 1038 insertions(+)
create mode 100644 migration/nvdimm.c
diff --git a/Makefile.target b/Makefile.target
index 6549481..0259e70 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -139,6 +139,7 @@ obj-y += memory.o
obj-y += memory_mapping.o
obj-y += dump.o
obj-y += migration/ram.o
+obj-y += migration/nvdimm.o
LIBS := $(libs_softmmu) $(LIBS)
# Hardware support
diff --git a/include/migration/misc.h b/include/migration/misc.h
index 77fd4f5..0c23da8 100644
--- a/include/migration/misc.h
+++ b/include/migration/misc.h
@@ -20,6 +20,10 @@
void ram_mig_init(void);
+/* migration/nvdimm.c */
+void nvdimm_snapshot_init(void);
+bool ram_block_is_nvdimm_active(RAMBlock *block);
+
/* migration/block.c */
#ifdef CONFIG_LIVE_BLOCK_MIGRATION
diff --git a/migration/nvdimm.c b/migration/nvdimm.c
new file mode 100644
index 0000000..8516bb0
--- /dev/null
+++ b/migration/nvdimm.c
@@ -0,0 +1,1033 @@
+/*
+ * QEMU System Emulator
+ *
+ * Authors:
+ * He Junyan<Junyan.he@intel.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/mem/nvdimm.h"
+#include "cpu.h"
+#include "qemu/cutils.h"
+#include "exec/ram_addr.h"
+#include "exec/target_page.h"
+#include "qemu/rcu_queue.h"
+#include "qemu/error-report.h"
+#include "migration.h"
+#include "qapi/error.h"
+#include "migration/register.h"
+#include "migration/ram.h"
+#include "migration/qemu-file.h"
+#include "migration.h"
+#include "migration/misc.h"
+#include "migration/savevm.h"
+#include "block/snapshot.h"
+#include "migration/snapshot.h"
+
+#define NVDIMM_MIG_VERSION 0x01
+
+/* PADDING data, useless */
+#define NVDIMM_PADDING_BYTE 0xce
+/* PAGE id, is all zero */
+#define NVDIMM_ZERO_PAGE_ID 0xaabc250f
+#define NVDIMM_NONZERO_PAGE_ID 0xacbc250e
+/* No usage date, for alignment only */
+#define NVDIMM_SECTION_PADDING_ID 0xaaceccea
+/* Section for dirty log kind */
+#define NVDIMM_SECTION_DIRTY_LOG_ID 0xbbcd0c1e
+/* Section for raw data, no bitmap, dump the whole mem */
+#define NVDIMM_SECTION_DATA_ID 0x76bbcae3
+/* Section for setup */
+#define NVDIMM_SECTION_SETUP 0x7ace0cfa
+/* Section for setup */
+#define NVDIMM_SECTION_COMPLETE 0x8ace0cfa
+/* Section end symbol */
+#define NVDIMM_SECTION_END_ID 0xccbe8752
+/************************ Sections** ***********************
+Padding section
+----------------------------------------------------
+| PADDING_ID | size | PADDING_BYTE ...... | END_ID |
+----------------------------------------------------
+Dirty log section
+------------------------------------------------------------------------------------
+| DIRTY_BITMAP_ID | total size | ram name size | ram name | ram size | bitmap size |
+------------------------------------------------------------------------------------
+ -----------------------------------------------------------------
+ bitmap data... | dirty page size | dirty page data... | END_ID |
+ -----------------------------------------------------------------
+Raw data section
+---------------------------------------------------------------------------------------
+| DATA_ID | size | ram name size | ram name | ram size | data size | data... | END_ID |
+---------------------------------------------------------------------------------------
+*************************************************************/
+
+/* State of NVDimm for migration */
+struct NVDimmState {
+ /* Whether the block driver support dependency
+ between snapshots */
+ char *depend_snapshot_id;
+ int64_t depend_offset;
+ int64_t depend_size;
+ char *cur_snapshot_id;
+ int64_t cur_offset;
+ int64_t cur_size;
+ RAMBlock **blocks;
+ int block_num;
+ bool dirty_logging;
+};
+typedef struct NVDimmState NVDimmState;
+
+static NVDimmState *nvdimm_state_p;
+
+static int nvdimm_device_list_append(Object *obj, void *opaque)
+{
+ GSList **list = opaque;
+
+ if (object_dynamic_cast(obj, TYPE_NVDIMM)) {
+ *list = g_slist_append(*list, DEVICE(obj));
+ }
+
+ object_child_foreach(obj, nvdimm_device_list_append, opaque);
+ return 0;
+}
+
+static bool ram_block_is_nvdimm(RAMBlock *block)
+{
+ GSList *list = NULL;
+ GSList *device_list = NULL;
+ bool ret = false;
+
+ object_child_foreach(qdev_get_machine(),
+ nvdimm_device_list_append, &device_list);
+
+ if (device_list == NULL) {
+ return false;
+ }
+
+ for (list = device_list; list; list = list->next) {
+ NVDIMMDevice *nvd = list->data;
+ MemoryRegion *mr = &nvd->nvdimm_mr;
+ int fd = memory_region_get_fd(mr);
+
+ if (fd >= 0 && fd == block->fd) {
+ ret = true;
+ break;
+ }
+ }
+
+ g_slist_free(device_list);
+ return ret;
+}
+
+bool ram_block_is_nvdimm_active(RAMBlock *block)
+{
+ if (block == NULL) {
+ return false;
+ }
+
+ if (get_current_snapshot_info(NULL) == false) {
+ return false;
+ }
+
+ return ram_block_is_nvdimm(block);
+}
+
+/* Just support snapshot, live migration use ram's handlers */
+static bool nvdimm_is_active(void *opaque)
+{
+ RAMBlock *block;
+ int ret = get_current_snapshot_info(NULL);
+ if (ret) {
+ return true;
+ }
+
+ if (!ram_bytes_total()) {
+ return false;
+ }
+
+ rcu_read_lock();
+ RAMBLOCK_FOREACH(block)
+ {
+ if (ram_block_is_nvdimm_active(block)) {
+ rcu_read_unlock();
+ return true;
+ }
+ }
+ rcu_read_unlock();
+
+ return false;
+}
+
+static int nvdimm_padding_to_alignment(QEMUFile *f, int section_id,
+ int32_t alignment, bool add_footer)
+{
+ int64_t cur_pos;
+ int32_t padding_sz;
+ int ret = 0;
+
+ cur_pos = qemu_ftell(f);
+ /* We need to insert some padding section here. */
+ padding_sz = (int32_t)(QEMU_ALIGN_UP(cur_pos, alignment) - cur_pos);
+ ret = padding_sz;
+
+ padding_sz -= sizeof(int32_t); // NVDIMM_SECTION_PADDING_ID
+ padding_sz -= sizeof(int32_t); // NVDIMM_PADDING_BYTE size
+ padding_sz -= sizeof(int32_t); // NVDIMM_SECTION_END_ID
+ if (migrate_get_current()->send_section_footer) {
+ padding_sz -= sizeof(int8_t);
+ padding_sz -= sizeof(int32_t);
+ }
+
+ if (padding_sz <= 0) {
+ padding_sz += alignment;
+ ret += alignment;
+ }
+
+ qemu_put_be32(f, NVDIMM_SECTION_PADDING_ID);
+ qemu_put_be32(f, padding_sz);
+ while (padding_sz) {
+ qemu_put_byte(f, NVDIMM_PADDING_BYTE);
+ padding_sz--;
+ }
+ qemu_put_be32(f, NVDIMM_SECTION_END_ID);
+ if (add_footer && migrate_get_current()->send_section_footer) {
+ qemu_put_byte(f, QEMU_VM_SECTION_FOOTER);
+ qemu_put_be32(f, section_id);
+ }
+
+ cur_pos = qemu_ftell(f);
+ assert(QEMU_IS_ALIGNED(cur_pos, alignment) || add_footer == false);
+ return ret;
+}
+
+static int nvdimm_state_save_dependency(QEMUFile *f, NVDimmState *nvdimm_state)
+{
+ int64_t cur_pos;
+ int ret;
+ int32_t alignment = 0;
+
+ if (qemu_file_is_support_dependency(f, &alignment) == false) {
+ error_report("Enable nvdimm dependent snapshot without"
+ "file dependency support");
+ return -ENOTSUP;
+ }
+
+ cur_pos = qemu_ftell(f);
+ assert(QEMU_IS_ALIGNED(cur_pos, alignment));
+ assert(QEMU_IS_ALIGNED(nvdimm_state->depend_offset, alignment));
+ assert(QEMU_IS_ALIGNED(nvdimm_state->depend_size, alignment));
+
+ ret = qemu_file_set_ref_name(f, nvdimm_state->depend_snapshot_id);
+ assert(ret);
+
+ ret = qemu_file_save_dependency(f, nvdimm_state->depend_offset,
+ nvdimm_state->depend_size);
+ if (ret < 0) {
+ error_report("save file dependency failed, depend_offset = %lx "
+ "depend_size is %ld, ret is %d",
+ nvdimm_state->depend_offset,
+ nvdimm_state->depend_size, ret);
+ return ret;
+ }
+
+ cur_pos = qemu_ftell(f);
+ assert(QEMU_IS_ALIGNED(cur_pos, alignment));
+
+ return ret;
+}
+
+static inline void *nvdimm_host_from_ram_block_offset(RAMBlock *block,
+ ram_addr_t offset)
+{
+ if (!offset_in_ramblock(block, offset)) {
+ return NULL;
+ }
+
+ return block->host + offset;
+}
+
+static int nvdimm_state_save_all_pages(QEMUFile *f,
+ NVDimmState *nvdimm_state, int i)
+{
+ hwaddr addr;
+ uint64_t total_sz;
+ int name_sz;
+ uint64_t data_sz;
+ void *host_ptr;
+
+ if (memory_region_size(nvdimm_state->blocks[i]->mr) == 0) {
+ return 0;
+ }
+
+ data_sz = 0;
+ for (addr = 0; addr < memory_region_size(nvdimm_state->blocks[i]->mr);
+ addr += 1 << TARGET_PAGE_BITS) {
+ assert(QEMU_IS_ALIGNED(addr, 1 << TARGET_PAGE_BITS));
+ host_ptr =
+ nvdimm_host_from_ram_block_offset(nvdimm_state->blocks[i], addr);
+ if (!host_ptr) {
+ error_report("Illegal RAM offset " RAM_ADDR_FMT, addr);
+ return -EINVAL;
+ }
+
+ if (buffer_is_zero(host_ptr, 1 << TARGET_PAGE_BITS)) {
+ data_sz += sizeof(int); // Zero page, just a ID
+ } else {
+ data_sz += ((1 << TARGET_PAGE_BITS) + sizeof(int)); // ID + page
+ }
+ }
+
+ total_sz = sizeof(unsigned int); // NVDIMM_SECTION_DIRTY_BITMAP_ID
+ total_sz += sizeof(uint64_t); // the total size itself
+ total_sz += sizeof(int); // ram name size
+ name_sz = strlen(nvdimm_state->blocks[i]->idstr) + 1;
+ total_sz += name_sz;
+ total_sz += sizeof(uint64_t); // ram size
+ total_sz += sizeof(uint64_t); // data size
+ total_sz += data_sz;
+ total_sz += sizeof(unsigned int); // NVDIMM_SECTION_END_ID
+
+ qemu_put_be32(f, NVDIMM_SECTION_DATA_ID);
+ qemu_put_be64(f, total_sz);
+ qemu_put_be32(f, name_sz);
+ qemu_put_buffer(f, (uint8_t *)nvdimm_state->blocks[i]->idstr, name_sz);
+ qemu_put_be64(f, memory_region_size(nvdimm_state->blocks[i]->mr));
+ qemu_put_be64(f, data_sz);
+
+ for (addr = 0; addr < memory_region_size(nvdimm_state->blocks[i]->mr);
+ addr += 1 << TARGET_PAGE_BITS) {
+ host_ptr =
+ nvdimm_host_from_ram_block_offset(nvdimm_state->blocks[i], addr);
+
+ if (buffer_is_zero(host_ptr, 1 << TARGET_PAGE_BITS)) {
+ qemu_put_be32(f, NVDIMM_ZERO_PAGE_ID);
+ data_sz -= sizeof(int);
+ } else {
+ qemu_put_be32(f, NVDIMM_NONZERO_PAGE_ID);
+ data_sz -= sizeof(int);
+ qemu_put_buffer(f, host_ptr, 1 << TARGET_PAGE_BITS);
+ data_sz -= 1 << TARGET_PAGE_BITS;
+ }
+ }
+ assert(data_sz == 0);
+ qemu_put_be32(f, NVDIMM_SECTION_END_ID);
+
+ return 1;
+}
+
+static int nvdimm_state_save_dirty_pages(QEMUFile *f,
+ NVDimmState *nvdimm_state, int i)
+{
+ DirtyBitmapSnapshot *snap;
+ uint64_t bit_sz;
+ uint64_t total_sz;
+ int name_sz;
+ uint64_t data_sz;
+ hwaddr addr;
+
+ if (memory_region_size(nvdimm_state->blocks[i]->mr) == 0) {
+ return 0;
+ }
+
+ snap = memory_region_snapshot_and_clear_dirty(
+ nvdimm_state->blocks[i]->mr,
+ 0,
+ memory_region_size(nvdimm_state->blocks[i]->mr),
+ DIRTY_MEMORY_MIGRATION);
+ if (snap == NULL) {
+ error_report("Can not create snapshot bitmap for block %s",
+ nvdimm_state->blocks[i]->idstr);
+ return -1;
+ }
+
+ bit_sz =
+ memory_region_size(nvdimm_state->blocks[i]->mr) >> (TARGET_PAGE_BITS + 3);
+
+ data_sz = 0;
+ for (addr = 0; addr < memory_region_size(nvdimm_state->blocks[i]->mr);
+ addr += 1 << TARGET_PAGE_BITS) {
+ assert(QEMU_IS_ALIGNED(addr, 1 << TARGET_PAGE_BITS));
+ if (memory_region_snapshot_get_dirty(nvdimm_state->blocks[i]->mr,
+ snap, addr, 1 << TARGET_PAGE_BITS)) {
+ data_sz += 1 << TARGET_PAGE_BITS;
+ }
+ }
+
+ total_sz = sizeof(unsigned int); // NVDIMM_SECTION_DIRTY_BITMAP_ID
+ total_sz += sizeof(uint64_t); // the total size itself
+ total_sz += sizeof(int); // ram name size
+ name_sz = strlen(nvdimm_state->blocks[i]->idstr) + 1;
+ total_sz += name_sz;
+ total_sz += sizeof(uint64_t); // ram size
+ total_sz += sizeof(uint64_t); // bitmap size
+ total_sz += bit_sz;
+ total_sz += sizeof(uint64_t); // data size
+ total_sz += data_sz;
+ total_sz += sizeof(unsigned int); // NVDIMM_SECTION_END_ID
+
+ qemu_put_be32(f, NVDIMM_SECTION_DIRTY_LOG_ID);
+ qemu_put_be64(f, total_sz);
+ qemu_put_be32(f, name_sz);
+ qemu_put_buffer(f, (uint8_t *)nvdimm_state->blocks[i]->idstr, name_sz);
+ qemu_put_be64(f, memory_region_size(nvdimm_state->blocks[i]->mr));
+ qemu_put_be64(f, bit_sz);
+ qemu_put_buffer(f, (uint8_t *)memory_region_snapshot_get_dirty_bitmap(snap),
+ bit_sz);
+
+ qemu_put_be64(f, data_sz);
+ if (data_sz != 0) {
+ for (addr = 0; addr < memory_region_size(nvdimm_state->blocks[i]->mr);
+ addr += 1 << TARGET_PAGE_BITS) {
+ assert(QEMU_IS_ALIGNED(addr, 1 << TARGET_PAGE_BITS));
+ if (memory_region_snapshot_get_dirty(nvdimm_state->blocks[i]->mr,
+ snap, addr, 1 << TARGET_PAGE_BITS)) {
+ qemu_put_buffer(f, nvdimm_state->blocks[i]->host + addr,
+ 1 << TARGET_PAGE_BITS);
+ data_sz -= 1 << TARGET_PAGE_BITS;
+ }
+ }
+ assert(data_sz == 0);
+ }
+
+ qemu_put_be32(f, NVDIMM_SECTION_END_ID);
+ g_free(snap);
+
+ return 1;
+}
+
+/**
+ * nvdimm_save_iterate: iterative stage for migration
+ *
+ * Returns zero to indicate success and negative for error
+ *
+ * @f: QEMUFile where to send the data
+ * @opaque: NVDimmState pointer
+ */
+static int nvdimm_save_iterate(QEMUFile *f, void *opaque, int section_id)
+{
+ NVDimmState *nvdimm_state = *(void **)opaque;
+ int ret = 0;
+ int i;
+ int32_t alignment;
+ int64_t begin_pos, cur_pos;
+ bool padded = false;
+
+ /* Must support dependency */
+ ret = qemu_file_is_support_dependency(f, &alignment);
+ assert(ret == true);
+
+ cur_pos = qemu_ftell(f);
+ if (!QEMU_IS_ALIGNED(cur_pos, alignment)) {
+ ret = nvdimm_padding_to_alignment(f, section_id, alignment, true);
+ if (ret < 0) {
+ error_report("NVDIMM saving, failed to padding to aligment");
+ return ret;
+ }
+ padded = true;
+ }
+
+ begin_pos = qemu_ftell(f);
+ assert(QEMU_IS_ALIGNED(begin_pos, alignment));
+ nvdimm_state->cur_offset = begin_pos;
+
+ if (nvdimm_state->dirty_logging) {
+ ret = nvdimm_state_save_dependency(f, nvdimm_state);
+ if (ret < 0) {
+ error_report("NVDIMM saving, failed to save dependency");
+ return ret;
+ }
+
+ for (i = 0; i < nvdimm_state->block_num; i++) {
+ cpu_physical_memory_test_and_clear_dirty(
+ memory_region_get_ram_addr(nvdimm_state->blocks[i]->mr),
+ memory_region_size(nvdimm_state->blocks[i]->mr),
+ DIRTY_MEMORY_MIGRATION);
+ }
+
+ if (padded) {
+ qemu_put_byte(f, QEMU_VM_SECTION_PART);
+ qemu_put_be32(f, section_id);
+ }
+
+ for (i = 0; i < nvdimm_state->block_num; i++) {
+ ret = nvdimm_state_save_dirty_pages(f, nvdimm_state, i);
+ if (ret < 0) {
+ error_report("NVDIMM saving, failed to save dirty pages");
+ return ret;
+ }
+ }
+ } else {
+ if (padded) {
+ qemu_put_byte(f, QEMU_VM_SECTION_PART);
+ qemu_put_be32(f, section_id);
+ }
+
+ /* Save the whole content of nvdimm, no dependency needed */
+ for (i = 0; i < nvdimm_state->block_num; i++) {
+ ret = nvdimm_state_save_all_pages(f, nvdimm_state, i);
+ if (ret < 0) {
+ error_report("NVDIMM saving, failed to save all pages");
+ return ret;
+ }
+ }
+ }
+
+ /* Need to add padding to make the whole data aligned, include
+ QEMU_VM_SECTION_FOOTER and section_id */
+ cur_pos = qemu_ftell(f);
+ if (migrate_get_current()->send_section_footer) {
+ cur_pos += (1 + sizeof(int));
+ }
+
+ if (QEMU_IS_ALIGNED(cur_pos, alignment)) { // Already aligned
+ nvdimm_state->cur_size = cur_pos - begin_pos;
+ assert(QEMU_IS_ALIGNED(nvdimm_state->cur_size, alignment));
+ return ret;
+ }
+
+ /* Appending the footer if needed */
+ if (migrate_get_current()->send_section_footer) {
+ qemu_put_byte(f, QEMU_VM_SECTION_FOOTER);
+ qemu_put_be32(f, section_id);
+ qemu_put_byte(f, QEMU_VM_SECTION_PART);
+ qemu_put_be32(f, section_id);
+ }
+ ret = nvdimm_padding_to_alignment(f, section_id, alignment, false);
+ if (ret < 0) {
+ error_report("NVDIMM saving, failed to save all pages");
+ return ret;
+ }
+
+ cur_pos = qemu_ftell(f);
+ nvdimm_state->cur_size = cur_pos - begin_pos;
+ if (migrate_get_current()->send_section_footer) {
+ nvdimm_state->cur_size += (1 + sizeof(int));
+ }
+ assert(QEMU_IS_ALIGNED(nvdimm_state->cur_size, alignment));
+
+ return ret;
+}
+
+static void nvdimm_destroy_nvdimm_state(NVDimmState *nvdimm_state)
+{
+ if (nvdimm_state) {
+ /* disable all dirty log trace */
+ if (nvdimm_state->depend_snapshot_id) {
+ int i;
+ for (i = 0; i < nvdimm_state->block_num; i++) {
+ memory_region_set_log(nvdimm_state->blocks[i]->mr, false,
+ DIRTY_MEMORY_MIGRATION);
+ }
+ }
+
+ if (nvdimm_state->cur_snapshot_id) {
+ g_free(nvdimm_state->cur_snapshot_id);
+ }
+ if (nvdimm_state->blocks) {
+ g_free(nvdimm_state->blocks);
+ }
+ g_free(nvdimm_state);
+ }
+}
+
+static NVDimmState *nvdimm_alloc_nvdimm_state(void)
+{
+ NVDimmState *nvdimm_state = g_try_new0(NVDimmState, 1);
+ RAMBlock *block;
+
+ if (nvdimm_state == NULL) {
+ return NULL;
+ }
+
+ rcu_read_lock();
+ RAMBLOCK_FOREACH(block)
+ {
+ if (ram_block_is_nvdimm(block)) {
+ nvdimm_state->block_num++;
+ nvdimm_state->blocks =
+ g_try_renew(RAMBlock *,
+ nvdimm_state->blocks,
+ nvdimm_state->block_num);
+ if (nvdimm_state->blocks == NULL) {
+ rcu_read_unlock();
+ nvdimm_destroy_nvdimm_state(nvdimm_state);
+ return NULL;
+ }
+
+ nvdimm_state->blocks[nvdimm_state->block_num - 1] = block;
+ }
+ }
+ rcu_read_unlock();
+
+ return nvdimm_state;
+}
+
+/**
+ * nvdimm_save_setup: Setup nvdimm for migration
+ *
+ * Returns zero to indicate success and negative for error
+ *
+ * @f: QEMUFile where to send the data
+ * @opaque: NVDimmState pointer
+ */
+static int nvdimm_save_setup(QEMUFile *f, void *opaque)
+{
+ NVDimmState *nvdimm_state = *(void **)opaque;
+ int ret = 0;
+ QEMUSnapshotInfo sn;
+ RAMBlock *block;
+ int i;
+
+ ret = get_current_snapshot_info(&sn);
+ if (ret == 0) { /* Just enable in snapshot mode */
+ info_report("Not in snapshot saving, no nvdimm snapshot optimization");
+ return -1;
+ }
+
+ /* No dependency support, just let the ram common logic do its job */
+ if (qemu_file_is_support_dependency(f, NULL) == false) {
+ assert(nvdimm_state == NULL);
+ info_report("The drive file does not support dependent snapshot");
+ return -1;
+ }
+
+ if (nvdimm_state == NULL) { /* First time */
+ nvdimm_state = nvdimm_alloc_nvdimm_state();
+ if (nvdimm_state == NULL) {
+ error_report("Alloc the nvdimm state for snapshot saving failed");
+ goto failed;
+ }
+
+ nvdimm_state->cur_snapshot_id = g_strdup(sn.name);
+ *(void **)opaque = nvdimm_state;
+ } else {
+ assert(nvdimm_state->cur_snapshot_id);
+ if (nvdimm_state->depend_snapshot_id) {
+ g_free(nvdimm_state->depend_snapshot_id);
+ }
+ nvdimm_state->depend_snapshot_id = nvdimm_state->cur_snapshot_id;
+ nvdimm_state->depend_offset = nvdimm_state->cur_offset;
+ nvdimm_state->depend_size = nvdimm_state->cur_size;
+ nvdimm_state->cur_snapshot_id = g_strdup(sn.name);
+ nvdimm_state->cur_offset = 0;
+ nvdimm_state->cur_size = 0;
+
+ rcu_read_lock();
+ RAMBLOCK_FOREACH(block)
+ {
+ if (ram_block_is_nvdimm_active(block)) {
+ for (i = 0; i < nvdimm_state->block_num; i++) {
+ if (block == nvdimm_state->blocks[i]) {
+ break;
+ }
+ }
+
+ // Can not find the same block?
+ if (i == nvdimm_state->block_num) {
+ rcu_read_unlock();
+ error_report("Can not find the block %s", block->idstr);
+ goto failed;
+ }
+ }
+ }
+ rcu_read_unlock();
+ }
+
+ qemu_put_be32(f, NVDIMM_SECTION_SETUP);
+ qemu_put_be32(f, NVDIMM_SECTION_END_ID);
+
+ return ret;
+
+failed:
+ nvdimm_destroy_nvdimm_state(nvdimm_state);
+ *(void **)opaque = NULL;
+ return -1;
+}
+
+/**
+ * nvdimm_save_complete: function called to send the remaining amount of ram
+ *
+ * Returns zero to indicate success
+ *
+ * Called with iothread lock
+ *
+ * @f: QEMUFile where to send the data
+ * @opaque: NVDimmState pointer
+ */
+static int nvdimm_save_complete(QEMUFile *f, void *opaque)
+{
+ NVDimmState *nvdimm_state = *(void **)opaque;
+ int i;
+
+ for (i = 0; i < nvdimm_state->block_num; i++) {
+ memory_region_set_log(nvdimm_state->blocks[i]->mr, true,
+ DIRTY_MEMORY_MIGRATION);
+ }
+
+ /* Enable the dirty logging for next time usage */
+ nvdimm_state->dirty_logging = true;
+
+ qemu_put_be32(f, NVDIMM_SECTION_COMPLETE);
+ qemu_put_be32(f, NVDIMM_SECTION_END_ID);
+
+ return 0;
+}
+
+static bool nvdimm_has_postcopy(void *opaque)
+{
+ return false;
+}
+
+static void nvdimm_save_pending(QEMUFile *f, void *opaque, uint64_t max_size,
+ uint64_t *non_postcopiable_pending,
+ uint64_t *postcopiable_pending)
+{
+}
+
+static void nvdimm_save_cleanup(void *opaque)
+{
+ return;
+}
+
+/**
+ * nvdimm_load_setup: Setup NVDimm for migration incoming side
+ *
+ * Returns zero to indicate success and negative for error
+ *
+ * @f: QEMUFile where to receive the data
+ * @opaque: NVDimmState pointer
+ */
+static int nvdimm_load_setup(QEMUFile *f, void *opaque)
+{
+ NVDimmState *nvdimm_state = *(void **)opaque;
+
+ if (nvdimm_state) {
+ nvdimm_destroy_nvdimm_state(nvdimm_state);
+ *(void **)opaque = NULL;
+ }
+
+ nvdimm_state = nvdimm_alloc_nvdimm_state();
+ if (nvdimm_state == NULL) {
+ return -1;
+ }
+
+ *(void **)opaque = nvdimm_state;
+ return 1;
+}
+
+static int nvdimm_load_cleanup(void *opaque)
+{
+ NVDimmState *nvdimm_state = *(void **)opaque;
+ nvdimm_destroy_nvdimm_state(nvdimm_state);
+ *(void **)opaque = NULL;
+
+ return 0;
+}
+
+static int nvdimm_load_dirty_pages(QEMUFile *f, NVDimmState *nvdimm_state)
+{
+ int64_t total_sz = qemu_get_be64(f);
+ int ret = 0;
+ int name_sz;
+ int64_t sz;
+ uint8_t *name_buf = NULL;
+ uint8_t *bitmap_buf = NULL;
+ RAMBlock *block;
+ int64_t ram_sz = 0;
+ int64_t bitmap_sz = 0;
+ int64_t data_sz = 0;
+ hwaddr addr;
+ void *host_ptr;
+
+ if (total_sz <= 0) {
+ ret = -EINVAL;
+ return ret;
+ }
+
+ name_sz = qemu_get_be32(f);
+ if (name_sz <= 0) {
+ ret = -EINVAL;
+ return ret;
+ }
+
+ name_buf = g_malloc(name_sz);
+ if (name_buf == NULL) {
+ ret = -ENOMEM;
+ return ret;
+ }
+
+ sz = qemu_get_buffer(f, name_buf, name_sz);
+ if (sz != name_sz) {
+ ret = -EINVAL;
+ return ret;
+ }
+
+ block = qemu_ram_block_by_name((char *)name_buf);
+ if (block == NULL || block->host == NULL) {
+ ret = -EINVAL;
+ return ret;
+ }
+ g_free(name_buf);
+ name_buf = NULL;
+
+ ram_sz = qemu_get_be64(f);
+ if (ram_sz != memory_region_size(block->mr)) {
+ ret = -EINVAL;
+ return ret;
+ }
+
+ bitmap_sz = qemu_get_be64(f);
+ if (bitmap_sz <= 0) {
+ ret = -EINVAL;
+ return ret;
+ }
+
+ bitmap_buf = g_malloc(bitmap_sz);
+ if (bitmap_buf == NULL) {
+ ret = -ENOMEM;
+ return ret;
+ }
+
+ sz = qemu_get_buffer(f, bitmap_buf, bitmap_sz);
+ if (sz != bitmap_sz) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ data_sz = qemu_get_be64(f);
+ if (data_sz < 0) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ assert(QEMU_IS_ALIGNED(data_sz, TARGET_PAGE_SIZE));
+ addr = 0;
+ while (data_sz) {
+ addr = find_next_bit((unsigned long *)bitmap_buf,
+ ram_sz >> TARGET_PAGE_BITS, addr);
+ host_ptr = nvdimm_host_from_ram_block_offset(block, addr);
+ if (!host_ptr) {
+ error_report("Illegal RAM offset " RAM_ADDR_FMT, addr);
+ ret = -EINVAL;
+ goto out;
+ }
+
+ qemu_get_buffer(f, host_ptr, TARGET_PAGE_SIZE);
+ data_sz -= TARGET_PAGE_SIZE;
+ }
+
+ if (qemu_get_be32(f) != NVDIMM_SECTION_END_ID) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+out:
+ if (bitmap_buf) {
+ g_free(bitmap_buf);
+ }
+ return ret;
+}
+
+static int nvdimm_load_all_pages(QEMUFile *f, NVDimmState *nvdimm_state)
+{
+ int64_t total_sz = qemu_get_be64(f);
+ int64_t sz;
+ int64_t data_sz;
+ int name_sz;
+ int ret = 0;
+ uint8_t *buf = NULL;
+ RAMBlock *block;
+ hwaddr addr;
+ void *host_ptr;
+ int64_t ram_sz = 0;
+ int tag;
+
+ if (total_sz <= 0) {
+ ret = -EINVAL;
+ return ret;
+ }
+
+ name_sz = qemu_get_be32(f);
+ if (name_sz <= 0) {
+ ret = -EINVAL;
+ return ret;
+ }
+
+ buf = g_malloc(name_sz);
+ if (buf == NULL) {
+ ret = -ENOMEM;
+ return ret;
+ }
+
+ sz = qemu_get_buffer(f, buf, name_sz);
+ if (sz != name_sz) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ block = qemu_ram_block_by_name((char *)buf);
+ if (block == NULL || block->host == NULL) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ram_sz = qemu_get_be64(f);
+ if (ram_sz != memory_region_size(block->mr)) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ data_sz = qemu_get_be64(f);
+ if (data_sz <= 0) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ for (addr = 0; addr < ram_sz; addr += 1 << TARGET_PAGE_BITS) {
+ assert(QEMU_IS_ALIGNED(addr, 1 << TARGET_PAGE_BITS));
+ host_ptr = nvdimm_host_from_ram_block_offset(block, addr);
+ if (!host_ptr) {
+ error_report("Illegal RAM offset " RAM_ADDR_FMT, addr);
+ ret = -EINVAL;
+ goto out;
+ }
+
+ tag = qemu_get_be32(f);
+ if (tag == NVDIMM_ZERO_PAGE_ID) {
+ memset(host_ptr, 0, 1 << TARGET_PAGE_BITS);
+ } else {
+ assert(tag == NVDIMM_NONZERO_PAGE_ID);
+ qemu_get_buffer(f, host_ptr, TARGET_PAGE_SIZE);
+ }
+ host_ptr += TARGET_PAGE_SIZE;
+ }
+
+ tag = qemu_get_be32(f);
+ if (tag != NVDIMM_SECTION_END_ID) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+out:
+ if (buf) {
+ g_free(buf);
+ }
+
+ return ret;
+}
+
+static int nvdimm_load(QEMUFile *f, void *opaque, int version_id)
+{
+ int ret = 0;
+ unsigned int sec_id;
+ uint8_t *buf = NULL;
+ size_t sz;
+ NVDimmState *nvdimm_state = *(void **)opaque;
+
+ if (version_id != NVDIMM_MIG_VERSION) {
+ ret = -EINVAL;
+ goto failed;
+ }
+
+ sec_id = qemu_get_be32(f);
+ if (sec_id == NVDIMM_SECTION_PADDING_ID) {
+ /* Just skip all this padding section. */
+ int padding_sz = qemu_get_be32(f);
+ unsigned int end_id;
+ buf = g_malloc(padding_sz);
+ if (buf == NULL) {
+ ret = -ENOMEM;
+ goto failed;
+ }
+
+ sz = qemu_get_buffer(f, buf, padding_sz);
+ if (sz != padding_sz) {
+ ret = -EINVAL;
+ goto failed;
+ }
+ padding_sz--;
+ while (padding_sz >= 0) {
+ if (buf[padding_sz] != NVDIMM_PADDING_BYTE) {
+ ret = -EINVAL;
+ goto failed;
+ }
+ padding_sz--;
+ }
+
+ end_id = qemu_get_be32(f);
+ if (end_id != NVDIMM_SECTION_END_ID) {
+ ret = -EINVAL;
+ goto failed;
+ }
+
+ g_free(buf);
+ buf = NULL;
+ } else if (sec_id == NVDIMM_SECTION_DIRTY_LOG_ID) {
+ ret = nvdimm_load_dirty_pages(f, nvdimm_state);
+ if (ret < 0) {
+ goto failed;
+ }
+ } else if (sec_id == NVDIMM_SECTION_DATA_ID) {
+ ret = nvdimm_load_all_pages(f, nvdimm_state);
+ if (ret < 0) {
+ goto failed;
+ }
+ } else if (sec_id == NVDIMM_SECTION_SETUP ||
+ sec_id == NVDIMM_SECTION_COMPLETE) {
+ unsigned int d = qemu_get_be32(f);
+ if (d != NVDIMM_SECTION_END_ID) {
+ ret = -EINVAL;
+ goto failed;
+ }
+ } else {
+ error_report("NVDIMM load, can not recognize SEC id %d", sec_id);
+ ret = -EINVAL;
+ goto failed;
+ }
+
+ return ret;
+
+failed:
+ if (buf) {
+ g_free(buf);
+ }
+
+ return ret;
+}
+
+static SaveVMHandlers savevm_nvdimm_handlers = {
+ .is_active = nvdimm_is_active,
+ .save_setup = nvdimm_save_setup,
+ .save_live_iterate = nvdimm_save_iterate,
+ .save_live_complete_precopy = nvdimm_save_complete,
+ .has_postcopy = nvdimm_has_postcopy,
+ .save_live_pending = nvdimm_save_pending,
+ .load_state = nvdimm_load,
+ .save_cleanup = nvdimm_save_cleanup,
+ .load_setup = nvdimm_load_setup,
+ .load_cleanup = nvdimm_load_cleanup,
+};
+
+void nvdimm_snapshot_init(void)
+{
+ register_savevm_live(NULL, "nvdimm", 0, NVDIMM_MIG_VERSION,
+ &savevm_nvdimm_handlers, &nvdimm_state_p);
+}
--
2.7.4
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH 10/10] RFC: Enable nvdimm snapshot functions.
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
` (8 preceding siblings ...)
2018-03-13 8:33 ` [Qemu-devel] [PATCH 09/10] RFC: Add nvdimm snapshot saving to migration junyan.he
@ 2018-03-13 8:33 ` junyan.he
2018-03-15 13:55 ` [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot no-reply
` (2 subsequent siblings)
12 siblings, 0 replies; 17+ messages in thread
From: junyan.he @ 2018-03-13 8:33 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, mreitz, pbonzini, crosthwaite.peter, quintela, rth,
dgilbert, famz, Junyan He
From: Junyan He <junyan.he@intel.com>
In snapshot saving, all nvdimm kind memory will be saved in different way
and we exclude all nvdimm kind memory region in ram.c
Signed-off-by: Junyan He <junyan.he@intel.com>
---
migration/ram.c | 17 +++++++++++++++++
vl.c | 1 +
2 files changed, 18 insertions(+)
diff --git a/migration/ram.c b/migration/ram.c
index d1db422..ad32469 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1219,9 +1219,15 @@ static bool find_dirty_block(RAMState *rs, PageSearchStatus *pss, bool *again)
/* Didn't find anything in this RAM Block */
pss->page = 0;
pss->block = QLIST_NEXT_RCU(pss->block, next);
+ while (ram_block_is_nvdimm_active(pss->block)) {
+ pss->block = QLIST_NEXT_RCU(pss->block, next);
+ }
if (!pss->block) {
/* Hit the end of the list */
pss->block = QLIST_FIRST_RCU(&ram_list.blocks);
+ while (ram_block_is_nvdimm_active(pss->block)) {
+ pss->block = QLIST_NEXT_RCU(pss->block, next);
+ }
/* Flag that we've looped */
pss->complete_round = true;
rs->ram_bulk_stage = false;
@@ -1541,6 +1547,9 @@ static int ram_find_and_save_block(RAMState *rs, bool last_stage)
if (!pss.block) {
pss.block = QLIST_FIRST_RCU(&ram_list.blocks);
+ while (ram_block_is_nvdimm_active(pss.block)) {
+ pss.block = QLIST_NEXT_RCU(pss.block, next);
+ }
}
do {
@@ -1583,6 +1592,10 @@ uint64_t ram_bytes_total(void)
rcu_read_lock();
RAMBLOCK_FOREACH(block) {
+ if (ram_block_is_nvdimm_active(block)) {
+ // If snapshot and the block is nvdimm, let nvdimm do the job
+ continue;
+ }
total += block->used_length;
}
rcu_read_unlock();
@@ -2222,6 +2235,10 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
qemu_put_be64(f, ram_bytes_total() | RAM_SAVE_FLAG_MEM_SIZE);
RAMBLOCK_FOREACH(block) {
+ if (ram_block_is_nvdimm_active(block)) {
+ // If snapshot and the block is nvdimm, let nvdimm do the job
+ continue;
+ }
qemu_put_byte(f, strlen(block->idstr));
qemu_put_buffer(f, (uint8_t *)block->idstr, strlen(block->idstr));
qemu_put_be64(f, block->used_length);
diff --git a/vl.c b/vl.c
index 3ef04ce..1bd5711 100644
--- a/vl.c
+++ b/vl.c
@@ -4502,6 +4502,7 @@ int main(int argc, char **argv, char **envp)
blk_mig_init();
ram_mig_init();
+ nvdimm_snapshot_init();
/* If the currently selected machine wishes to override the units-per-bus
* property of its default HBA interface type, do so now. */
--
2.7.4
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot.
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
` (9 preceding siblings ...)
2018-03-13 8:33 ` [Qemu-devel] [PATCH 10/10] RFC: Enable nvdimm snapshot functions junyan.he
@ 2018-03-15 13:55 ` no-reply
2018-03-15 14:15 ` no-reply
2018-05-08 9:23 ` Stefan Hajnoczi
12 siblings, 0 replies; 17+ messages in thread
From: no-reply @ 2018-03-15 13:55 UTC (permalink / raw)
To: junyan.he; +Cc: famz, qemu-devel, kwolf
Hi,
This series seems to have some coding style problems. See output below for
more information:
Type: series
Message-id: 1520930033-18885-1-git-send-email-junyan.he@intel.com
Subject: [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
failed=1
echo
fi
n=$((n+1))
done
exit $failed
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
* [new tag] patchew/1520930033-18885-1-git-send-email-junyan.he@intel.com -> patchew/1520930033-18885-1-git-send-email-junyan.he@intel.com
Switched to a new branch 'test'
856a5e9687 RFC: Enable nvdimm snapshot functions.
4c1adceb28 RFC: Add nvdimm snapshot saving to migration.
424778c4de RFC: Add a section_id parameter to save_live_iterate call.
f9abbe71cd RFC: Add get_current_snapshot_info to get the snapshot state.
f49a5b0838 RFC: Add save dependency functions to qemu_file
adf712a616 RFC: Add memory region snapshot bitmap get function.
3fa9484cb8 RFC: Set memory_region_set_log available for more client.
9d40dbb19e RFC: Implement save and support snapshot dependency in block driver layer.
c43e8a1da2 RFC: Implement qcow2's snapshot dependent saving function.
e022968c94 RFC: Add save and support snapshot dependency function to block driver.
=== OUTPUT BEGIN ===
Checking PATCH 1/10: RFC: Add save and support snapshot dependency function to block driver....
Checking PATCH 2/10: RFC: Implement qcow2's snapshot dependent saving function....
Checking PATCH 3/10: RFC: Implement save and support snapshot dependency in block driver layer....
Checking PATCH 4/10: RFC: Set memory_region_set_log available for more client....
Checking PATCH 5/10: RFC: Add memory region snapshot bitmap get function....
Checking PATCH 6/10: RFC: Add save dependency functions to qemu_file...
Checking PATCH 7/10: RFC: Add get_current_snapshot_info to get the snapshot state....
Checking PATCH 8/10: RFC: Add a section_id parameter to save_live_iterate call....
Checking PATCH 9/10: RFC: Add nvdimm snapshot saving to migration....
WARNING: line over 80 characters
#122: FILE: migration/nvdimm.c:70:
+------------------------------------------------------------------------------------
WARNING: line over 80 characters
#123: FILE: migration/nvdimm.c:71:
+| DIRTY_BITMAP_ID | total size | ram name size | ram name | ram size | bitmap size |
WARNING: line over 80 characters
#124: FILE: migration/nvdimm.c:72:
+------------------------------------------------------------------------------------
WARNING: line over 80 characters
#129: FILE: migration/nvdimm.c:77:
+---------------------------------------------------------------------------------------
WARNING: line over 80 characters
#130: FILE: migration/nvdimm.c:78:
+| DATA_ID | size | ram name size | ram name | ram size | data size | data... | END_ID |
WARNING: line over 80 characters
#131: FILE: migration/nvdimm.c:79:
+---------------------------------------------------------------------------------------
ERROR: do not use C99 // comments
#243: FILE: migration/nvdimm.c:191:
+ padding_sz -= sizeof(int32_t); // NVDIMM_SECTION_PADDING_ID
ERROR: do not use C99 // comments
#244: FILE: migration/nvdimm.c:192:
+ padding_sz -= sizeof(int32_t); // NVDIMM_PADDING_BYTE size
ERROR: do not use C99 // comments
#245: FILE: migration/nvdimm.c:193:
+ padding_sz -= sizeof(int32_t); // NVDIMM_SECTION_END_ID
ERROR: do not use C99 // comments
#344: FILE: migration/nvdimm.c:292:
+ data_sz += sizeof(int); // Zero page, just a ID
ERROR: do not use C99 // comments
#346: FILE: migration/nvdimm.c:294:
+ data_sz += ((1 << TARGET_PAGE_BITS) + sizeof(int)); // ID + page
ERROR: do not use C99 // comments
#350: FILE: migration/nvdimm.c:298:
+ total_sz = sizeof(unsigned int); // NVDIMM_SECTION_DIRTY_BITMAP_ID
ERROR: do not use C99 // comments
#351: FILE: migration/nvdimm.c:299:
+ total_sz += sizeof(uint64_t); // the total size itself
ERROR: do not use C99 // comments
#352: FILE: migration/nvdimm.c:300:
+ total_sz += sizeof(int); // ram name size
ERROR: do not use C99 // comments
#355: FILE: migration/nvdimm.c:303:
+ total_sz += sizeof(uint64_t); // ram size
ERROR: do not use C99 // comments
#356: FILE: migration/nvdimm.c:304:
+ total_sz += sizeof(uint64_t); // data size
ERROR: do not use C99 // comments
#358: FILE: migration/nvdimm.c:306:
+ total_sz += sizeof(unsigned int); // NVDIMM_SECTION_END_ID
WARNING: line over 80 characters
#414: FILE: migration/nvdimm.c:362:
+ memory_region_size(nvdimm_state->blocks[i]->mr) >> (TARGET_PAGE_BITS + 3);
WARNING: line over 80 characters
#421: FILE: migration/nvdimm.c:369:
+ snap, addr, 1 << TARGET_PAGE_BITS)) {
ERROR: do not use C99 // comments
#426: FILE: migration/nvdimm.c:374:
+ total_sz = sizeof(unsigned int); // NVDIMM_SECTION_DIRTY_BITMAP_ID
ERROR: do not use C99 // comments
#427: FILE: migration/nvdimm.c:375:
+ total_sz += sizeof(uint64_t); // the total size itself
ERROR: do not use C99 // comments
#428: FILE: migration/nvdimm.c:376:
+ total_sz += sizeof(int); // ram name size
ERROR: do not use C99 // comments
#431: FILE: migration/nvdimm.c:379:
+ total_sz += sizeof(uint64_t); // ram size
ERROR: do not use C99 // comments
#432: FILE: migration/nvdimm.c:380:
+ total_sz += sizeof(uint64_t); // bitmap size
ERROR: do not use C99 // comments
#434: FILE: migration/nvdimm.c:382:
+ total_sz += sizeof(uint64_t); // data size
ERROR: do not use C99 // comments
#436: FILE: migration/nvdimm.c:384:
+ total_sz += sizeof(unsigned int); // NVDIMM_SECTION_END_ID
WARNING: line over 80 characters
#453: FILE: migration/nvdimm.c:401:
+ snap, addr, 1 << TARGET_PAGE_BITS)) {
ERROR: do not use C99 // comments
#552: FILE: migration/nvdimm.c:500:
+ if (QEMU_IS_ALIGNED(cur_pos, alignment)) { // Already aligned
ERROR: trailing statements should be on next line
#552: FILE: migration/nvdimm.c:500:
+ if (QEMU_IS_ALIGNED(cur_pos, alignment)) { // Already aligned
ERROR: g_free(NULL) is safe this check is probably not required
#594: FILE: migration/nvdimm.c:542:
+ if (nvdimm_state->cur_snapshot_id) {
+ g_free(nvdimm_state->cur_snapshot_id);
ERROR: g_free(NULL) is safe this check is probably not required
#597: FILE: migration/nvdimm.c:545:
+ if (nvdimm_state->blocks) {
+ g_free(nvdimm_state->blocks);
ERROR: g_free(NULL) is safe this check is probably not required
#676: FILE: migration/nvdimm.c:624:
+ if (nvdimm_state->depend_snapshot_id) {
+ g_free(nvdimm_state->depend_snapshot_id);
ERROR: do not use C99 // comments
#695: FILE: migration/nvdimm.c:643:
+ // Can not find the same block?
ERROR: g_free(NULL) is safe this check is probably not required
#896: FILE: migration/nvdimm.c:844:
+ if (bitmap_buf) {
+ g_free(bitmap_buf);
ERROR: g_free(NULL) is safe this check is probably not required
#983: FILE: migration/nvdimm.c:931:
+ if (buf) {
+ g_free(buf);
ERROR: g_free(NULL) is safe this check is probably not required
#1062: FILE: migration/nvdimm.c:1010:
+ if (buf) {
+ g_free(buf);
total: 27 errors, 9 warnings, 1050 lines checked
Your patch has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Checking PATCH 10/10: RFC: Enable nvdimm snapshot functions....
ERROR: do not use C99 // comments
#47: FILE: migration/ram.c:1596:
+ // If snapshot and the block is nvdimm, let nvdimm do the job
ERROR: do not use C99 // comments
#58: FILE: migration/ram.c:2239:
+ // If snapshot and the block is nvdimm, let nvdimm do the job
total: 2 errors, 0 warnings, 51 lines checked
Your patch has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===
Test command exited with code: 1
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@freelists.org
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot.
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
` (10 preceding siblings ...)
2018-03-15 13:55 ` [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot no-reply
@ 2018-03-15 14:15 ` no-reply
2018-05-08 9:23 ` Stefan Hajnoczi
12 siblings, 0 replies; 17+ messages in thread
From: no-reply @ 2018-03-15 14:15 UTC (permalink / raw)
To: junyan.he; +Cc: famz, qemu-devel, kwolf
Hi,
This series failed docker-mingw@fedora build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
Type: series
Message-id: 1520930033-18885-1-git-send-email-junyan.he@intel.com
Subject: [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=8
time make docker-test-mingw@fedora
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
856a5e9687 RFC: Enable nvdimm snapshot functions.
4c1adceb28 RFC: Add nvdimm snapshot saving to migration.
424778c4de RFC: Add a section_id parameter to save_live_iterate call.
f9abbe71cd RFC: Add get_current_snapshot_info to get the snapshot state.
f49a5b0838 RFC: Add save dependency functions to qemu_file
adf712a616 RFC: Add memory region snapshot bitmap get function.
3fa9484cb8 RFC: Set memory_region_set_log available for more client.
9d40dbb19e RFC: Implement save and support snapshot dependency in block driver layer.
c43e8a1da2 RFC: Implement qcow2's snapshot dependent saving function.
e022968c94 RFC: Add save and support snapshot dependency function to block driver.
=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-e7u4by_8/src/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
BUILD fedora
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-e7u4by_8/src'
GEN /var/tmp/patchew-tester-tmp-e7u4by_8/src/docker-src.2018-03-15-10.13.12.11256/qemu.tar
Cloning into '/var/tmp/patchew-tester-tmp-e7u4by_8/src/docker-src.2018-03-15-10.13.12.11256/qemu.tar.vroot'...
done.
Checking out files: 46% (2767/6005)
Checking out files: 47% (2823/6005)
Checking out files: 48% (2883/6005)
Checking out files: 49% (2943/6005)
Checking out files: 50% (3003/6005)
Checking out files: 51% (3063/6005)
Checking out files: 52% (3123/6005)
Checking out files: 53% (3183/6005)
Checking out files: 54% (3243/6005)
Checking out files: 55% (3303/6005)
Checking out files: 56% (3363/6005)
Checking out files: 57% (3423/6005)
Checking out files: 57% (3429/6005)
Checking out files: 58% (3483/6005)
Checking out files: 59% (3543/6005)
Checking out files: 60% (3603/6005)
Checking out files: 61% (3664/6005)
Checking out files: 62% (3724/6005)
Checking out files: 63% (3784/6005)
Checking out files: 64% (3844/6005)
Checking out files: 65% (3904/6005)
Checking out files: 66% (3964/6005)
Checking out files: 67% (4024/6005)
Checking out files: 68% (4084/6005)
Checking out files: 69% (4144/6005)
Checking out files: 70% (4204/6005)
Checking out files: 71% (4264/6005)
Checking out files: 72% (4324/6005)
Checking out files: 73% (4384/6005)
Checking out files: 74% (4444/6005)
Checking out files: 75% (4504/6005)
Checking out files: 76% (4564/6005)
Checking out files: 77% (4624/6005)
Checking out files: 78% (4684/6005)
Checking out files: 79% (4744/6005)
Checking out files: 80% (4804/6005)
Checking out files: 81% (4865/6005)
Checking out files: 82% (4925/6005)
Checking out files: 83% (4985/6005)
Checking out files: 84% (5045/6005)
Checking out files: 85% (5105/6005)
Checking out files: 86% (5165/6005)
Checking out files: 87% (5225/6005)
Checking out files: 88% (5285/6005)
Checking out files: 89% (5345/6005)
Checking out files: 90% (5405/6005)
Checking out files: 91% (5465/6005)
Checking out files: 92% (5525/6005)
Checking out files: 93% (5585/6005)
Checking out files: 94% (5645/6005)
Checking out files: 95% (5705/6005)
Checking out files: 96% (5765/6005)
Checking out files: 97% (5825/6005)
Checking out files: 98% (5885/6005)
Checking out files: 99% (5945/6005)
Checking out files: 100% (6005/6005)
Checking out files: 100% (6005/6005), done.
Your branch is up-to-date with 'origin/test'.
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-e7u4by_8/src/docker-src.2018-03-15-10.13.12.11256/qemu.tar.vroot/dtc'...
Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42'
Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Cloning into '/var/tmp/patchew-tester-tmp-e7u4by_8/src/docker-src.2018-03-15-10.13.12.11256/qemu.tar.vroot/ui/keycodemapdb'...
Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce'
COPY RUNNER
RUN test-mingw in qemu:fedora
Packages installed:
PyYAML-3.12-5.fc27.x86_64
SDL-devel-1.2.15-29.fc27.x86_64
bc-1.07.1-3.fc27.x86_64
bison-3.0.4-8.fc27.x86_64
bzip2-1.0.6-24.fc27.x86_64
ccache-3.3.6-1.fc27.x86_64
clang-5.0.1-3.fc27.x86_64
findutils-4.6.0-16.fc27.x86_64
flex-2.6.1-5.fc27.x86_64
gcc-7.3.1-5.fc27.x86_64
gcc-c++-7.3.1-5.fc27.x86_64
gettext-0.19.8.1-12.fc27.x86_64
git-2.14.3-3.fc27.x86_64
glib2-devel-2.54.3-2.fc27.x86_64
hostname-3.18-4.fc27.x86_64
libaio-devel-0.3.110-9.fc27.x86_64
libasan-7.3.1-5.fc27.x86_64
libfdt-devel-1.4.6-1.fc27.x86_64
libubsan-7.3.1-5.fc27.x86_64
llvm-5.0.1-3.fc27.x86_64
make-4.2.1-4.fc27.x86_64
mingw32-SDL-1.2.15-9.fc27.noarch
mingw32-bzip2-1.0.6-9.fc27.noarch
mingw32-curl-7.54.1-2.fc27.noarch
mingw32-glib2-2.54.1-1.fc27.noarch
mingw32-gmp-6.1.2-2.fc27.noarch
mingw32-gnutls-3.5.13-2.fc27.noarch
mingw32-gtk2-2.24.31-4.fc27.noarch
mingw32-gtk3-3.22.16-1.fc27.noarch
mingw32-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw32-libpng-1.6.29-2.fc27.noarch
mingw32-libssh2-1.8.0-3.fc27.noarch
mingw32-libtasn1-4.13-1.fc27.noarch
mingw32-nettle-3.3-3.fc27.noarch
mingw32-pixman-0.34.0-3.fc27.noarch
mingw32-pkg-config-0.28-9.fc27.x86_64
mingw64-SDL-1.2.15-9.fc27.noarch
mingw64-bzip2-1.0.6-9.fc27.noarch
mingw64-curl-7.54.1-2.fc27.noarch
mingw64-glib2-2.54.1-1.fc27.noarch
mingw64-gmp-6.1.2-2.fc27.noarch
mingw64-gnutls-3.5.13-2.fc27.noarch
mingw64-gtk2-2.24.31-4.fc27.noarch
mingw64-gtk3-3.22.16-1.fc27.noarch
mingw64-libjpeg-turbo-1.5.1-3.fc27.noarch
mingw64-libpng-1.6.29-2.fc27.noarch
mingw64-libssh2-1.8.0-3.fc27.noarch
mingw64-libtasn1-4.13-1.fc27.noarch
mingw64-nettle-3.3-3.fc27.noarch
mingw64-pixman-0.34.0-3.fc27.noarch
mingw64-pkg-config-0.28-9.fc27.x86_64
nettle-devel-3.4-1.fc27.x86_64
perl-5.26.1-403.fc27.x86_64
pixman-devel-0.34.0-4.fc27.x86_64
python3-3.6.2-13.fc27.x86_64
sparse-0.5.1-2.fc27.x86_64
tar-1.29-7.fc27.x86_64
which-2.21-4.fc27.x86_64
zlib-devel-1.2.11-4.fc27.x86_64
Environment variables:
TARGET_LIST=
PACKAGES=ccache gettext git tar PyYAML sparse flex bison python3 bzip2 hostname glib2-devel pixman-devel zlib-devel SDL-devel libfdt-devel gcc gcc-c++ llvm clang make perl which bc findutils libaio-devel nettle-devel libasan libubsan mingw32-pixman mingw32-glib2 mingw32-gmp mingw32-SDL mingw32-pkg-config mingw32-gtk2 mingw32-gtk3 mingw32-gnutls mingw32-nettle mingw32-libtasn1 mingw32-libjpeg-turbo mingw32-libpng mingw32-curl mingw32-libssh2 mingw32-bzip2 mingw64-pixman mingw64-glib2 mingw64-gmp mingw64-SDL mingw64-pkg-config mingw64-gtk2 mingw64-gtk3 mingw64-gnutls mingw64-nettle mingw64-libtasn1 mingw64-libjpeg-turbo mingw64-libpng mingw64-curl mingw64-libssh2 mingw64-bzip2
J=8
V=
HOSTNAME=ea3f1bf244ee
DEBUG=
SHOW_ENV=1
PWD=/
HOME=/root
CCACHE_DIR=/var/tmp/ccache
DISTTAG=f27container
QEMU_CONFIGURE_OPTS=--python=/usr/bin/python3
FGC=f27
TEST_DIR=/tmp/qemu-test
SHLVL=1
FEATURES=mingw clang pyyaml asan dtc
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MAKEFLAGS= -j8
EXTRA_CONFIGURE_OPTS=
_=/usr/bin/env
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/tmp/qemu-test/install --python=/usr/bin/python3 --cross-prefix=x86_64-w64-mingw32- --enable-trace-backends=simple --enable-gnutls --enable-nettle --enable-curl --enable-vnc --enable-bzip2 --enable-guest-agent --with-sdlabi=1.2 --with-gtkabi=2.0
Install prefix /tmp/qemu-test/install
BIOS directory /tmp/qemu-test/install
firmware path /tmp/qemu-test/install/share/qemu-firmware
binary directory /tmp/qemu-test/install
library directory /tmp/qemu-test/install/lib
module directory /tmp/qemu-test/install/lib
libexec directory /tmp/qemu-test/install/libexec
include directory /tmp/qemu-test/install/include
config directory /tmp/qemu-test/install
local state directory queried at runtime
Windows SDK no
Source path /tmp/qemu-test/src
GIT binary git
GIT submodules
C compiler x86_64-w64-mingw32-gcc
Host C compiler cc
C++ compiler x86_64-w64-mingw32-g++
Objective-C compiler clang
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/pixman-1 -I$(SRC_PATH)/dtc/libfdt -Werror -DHAS_LIBSSH2_SFTP_FSYNC -mms-bitfields -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/glib-2.0 -I/usr/x86_64-w64-mingw32/sys-root/mingw/lib/glib-2.0/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -m64 -mcx16 -mthreads -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wexpansion-to-defined -Wendif-labels -Wno-shift-negative-value -Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/p11-kit-1 -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include -I/usr/x86_64-w64-mingw32/sys-root/mingw/include/libpng16
LDFLAGS -Wl,--nxcompat -Wl,--no-seh -Wl,--dynamicbase -Wl,--warn-common -m64 -g
make make
install install
python /usr/bin/python3 -B
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
SDL support yes (1.2.15)
GTK support yes (2.24.31)
GTK GL support no
VTE support no
TLS priority NORMAL
GNUTLS support yes
GNUTLS rnd yes
libgcrypt no
libgcrypt kdf no
nettle yes (3.3)
nettle kdf yes
libtasn1 yes
curses support no
virgl support no
curl support yes
mingw32 support yes
Audio drivers dsound
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
Multipath support no
VNC support yes
VNC SASL support no
VNC JPEG support yes
VNC PNG support yes
xen support no
brlapi support no
bluez support no
Documentation no
PIE no
vde support no
netmap support no
Linux AIO support no
ATTR/XATTR support no
Install blobs yes
KVM support no
HAX support yes
HVF support no
WHPX support no
TCG support yes
TCG debug enabled no
TCG interpreter no
malloc trim support no
RDMA support no
fdt support yes
preadv support no
fdatasync no
madvise no
posix_madvise no
posix_memalign no
libcap-ng support no
vhost-net support no
vhost-crypto support no
vhost-scsi support no
vhost-vsock support no
vhost-user support no
Trace backends simple
Trace output file trace-<pid>
spice support no
rbd support no
xfsctl support no
smartcard support no
libusb no
usb net redir no
OpenGL support no
OpenGL dmabufs no
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info yes
QGA MSI support no
seccomp support no
coroutine backend win32
coroutine pool yes
debug stack usage no
crypto afalg no
GlusterFS support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support yes
TPM passthrough no
TPM emulator no
QOM debugging yes
Live block migration yes
lzo support no
snappy support no
bzip2 support yes
NUMA host support no
libxml2 no
tcmalloc support no
jemalloc support no
avx2 optimization yes
replication support yes
VxHS block device no
capstone no
WARNING: Use of GTK 2.0 is deprecated and will be removed in
WARNING: future releases. Please switch to using GTK 3.0
WARNING: Use of SDL 1.2 is deprecated and will be removed in
WARNING: future releases. Please switch to using SDL 2.0
mkdir -p dtc/libfdt
mkdir -p dtc/tests
GEN aarch64-softmmu/config-devices.mak.tmp
GEN x86_64-softmmu/config-devices.mak.tmp
GEN qapi-gen
GEN config-host.h
GEN qemu-options.def
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.h
GEN trace/generated-helpers.c
GEN module_block.h
GEN x86_64-softmmu/config-devices.mak
GEN aarch64-softmmu/config-devices.mak
GEN ui/input-keymap-atset1-to-qcode.c
GEN ui/input-keymap-linux-to-qcode.c
GEN ui/input-keymap-qcode-to-atset1.c
GEN ui/input-keymap-qcode-to-atset2.c
GEN ui/input-keymap-qcode-to-atset3.c
GEN ui/input-keymap-qcode-to-linux.c
GEN ui/input-keymap-qcode-to-qnum.c
GEN ui/input-keymap-qcode-to-sun.c
GEN ui/input-keymap-win32-to-qcode.c
GEN ui/input-keymap-usb-to-qcode.c
GEN ui/input-keymap-qnum-to-qcode.c
GEN ui/input-keymap-x11-to-qcode.c
GEN ui/input-keymap-xorgevdev-to-qcode.c
GEN ui/input-keymap-xorgkbd-to-qcode.c
GEN ui/input-keymap-xorgxquartz-to-qcode.c
GEN ui/input-keymap-xorgxwin-to-qcode.c
GEN tests/test-qapi-gen
GEN trace-root.h
GEN util/trace.h
GEN crypto/trace.h
GEN io/trace.h
GEN migration/trace.h
GEN block/trace.h
GEN chardev/trace.h
GEN hw/block/trace.h
GEN hw/block/dataplane/trace.h
GEN hw/char/trace.h
GEN hw/intc/trace.h
GEN hw/net/trace.h
GEN hw/rdma/trace.h
GEN hw/rdma/vmw/trace.h
GEN hw/virtio/trace.h
GEN hw/audio/trace.h
GEN hw/misc/trace.h
GEN hw/misc/macio/trace.h
GEN hw/usb/trace.h
GEN hw/scsi/trace.h
GEN hw/nvram/trace.h
GEN hw/display/trace.h
GEN hw/input/trace.h
GEN hw/timer/trace.h
GEN hw/dma/trace.h
GEN hw/sparc/trace.h
GEN hw/sparc64/trace.h
GEN hw/sd/trace.h
GEN hw/isa/trace.h
GEN hw/mem/trace.h
GEN hw/i386/trace.h
GEN hw/i386/xen/trace.h
GEN hw/9pfs/trace.h
GEN hw/ppc/trace.h
GEN hw/pci/trace.h
GEN hw/pci-host/trace.h
GEN hw/s390x/trace.h
GEN hw/vfio/trace.h
GEN hw/acpi/trace.h
GEN hw/arm/trace.h
GEN hw/alpha/trace.h
GEN hw/hppa/trace.h
GEN hw/xen/trace.h
GEN hw/ide/trace.h
GEN hw/tpm/trace.h
GEN ui/trace.h
GEN audio/trace.h
GEN net/trace.h
GEN target/arm/trace.h
GEN target/i386/trace.h
GEN target/mips/trace.h
GEN target/sparc/trace.h
GEN target/s390x/trace.h
GEN target/ppc/trace.h
GEN qom/trace.h
GEN linux-user/trace.h
GEN qapi/trace.h
GEN accel/tcg/trace.h
GEN accel/kvm/trace.h
GEN nbd/trace.h
GEN scsi/trace.h
GEN trace-root.c
GEN util/trace.c
GEN crypto/trace.c
GEN io/trace.c
GEN migration/trace.c
GEN block/trace.c
GEN chardev/trace.c
GEN hw/block/trace.c
GEN hw/block/dataplane/trace.c
GEN hw/char/trace.c
GEN hw/intc/trace.c
GEN hw/net/trace.c
GEN hw/rdma/trace.c
GEN hw/rdma/vmw/trace.c
GEN hw/virtio/trace.c
GEN hw/audio/trace.c
GEN hw/misc/trace.c
GEN hw/misc/macio/trace.c
GEN hw/usb/trace.c
GEN hw/scsi/trace.c
GEN hw/nvram/trace.c
GEN hw/display/trace.c
GEN hw/input/trace.c
GEN hw/timer/trace.c
GEN hw/dma/trace.c
GEN hw/sparc/trace.c
GEN hw/sparc64/trace.c
GEN hw/sd/trace.c
GEN hw/isa/trace.c
GEN hw/mem/trace.c
GEN hw/i386/trace.c
GEN hw/i386/xen/trace.c
GEN hw/9pfs/trace.c
GEN hw/ppc/trace.c
GEN hw/pci/trace.c
GEN hw/pci-host/trace.c
GEN hw/s390x/trace.c
GEN hw/vfio/trace.c
GEN hw/acpi/trace.c
GEN hw/arm/trace.c
GEN hw/alpha/trace.c
GEN hw/hppa/trace.c
GEN hw/xen/trace.c
GEN hw/ide/trace.c
GEN hw/tpm/trace.c
GEN ui/trace.c
GEN audio/trace.c
GEN net/trace.c
GEN target/arm/trace.c
GEN target/i386/trace.c
GEN target/mips/trace.c
GEN target/sparc/trace.c
GEN target/s390x/trace.c
GEN target/ppc/trace.c
GEN qom/trace.c
GEN linux-user/trace.c
GEN qapi/trace.c
GEN accel/tcg/trace.c
GEN accel/kvm/trace.c
GEN nbd/trace.c
GEN scsi/trace.c
GEN config-all-devices.mak
DEP /tmp/qemu-test/src/dtc/tests/dumptrees.c
DEP /tmp/qemu-test/src/dtc/tests/trees.S
DEP /tmp/qemu-test/src/dtc/tests/testutils.c
DEP /tmp/qemu-test/src/dtc/tests/value-labels.c
DEP /tmp/qemu-test/src/dtc/tests/asm_tree_dump.c
DEP /tmp/qemu-test/src/dtc/tests/truncated_property.c
DEP /tmp/qemu-test/src/dtc/tests/check_path.c
DEP /tmp/qemu-test/src/dtc/tests/overlay_bad_fixup.c
DEP /tmp/qemu-test/src/dtc/tests/subnode_iterate.c
DEP /tmp/qemu-test/src/dtc/tests/property_iterate.c
DEP /tmp/qemu-test/src/dtc/tests/overlay.c
DEP /tmp/qemu-test/src/dtc/tests/integer-expressions.c
DEP /tmp/qemu-test/src/dtc/tests/utilfdt_test.c
DEP /tmp/qemu-test/src/dtc/tests/path_offset_aliases.c
DEP /tmp/qemu-test/src/dtc/tests/add_subnode_with_nops.c
DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_unordered.c
DEP /tmp/qemu-test/src/dtc/tests/dtbs_equal_ordered.c
DEP /tmp/qemu-test/src/dtc/tests/dtb_reverse.c
DEP /tmp/qemu-test/src/dtc/tests/extra-terminating-null.c
DEP /tmp/qemu-test/src/dtc/tests/incbin.c
DEP /tmp/qemu-test/src/dtc/tests/boot-cpuid.c
DEP /tmp/qemu-test/src/dtc/tests/phandle_format.c
DEP /tmp/qemu-test/src/dtc/tests/path-references.c
DEP /tmp/qemu-test/src/dtc/tests/references.c
DEP /tmp/qemu-test/src/dtc/tests/string_escapes.c
DEP /tmp/qemu-test/src/dtc/tests/propname_escapes.c
DEP /tmp/qemu-test/src/dtc/tests/appendprop2.c
DEP /tmp/qemu-test/src/dtc/tests/appendprop1.c
DEP /tmp/qemu-test/src/dtc/tests/del_node.c
DEP /tmp/qemu-test/src/dtc/tests/del_property.c
DEP /tmp/qemu-test/src/dtc/tests/setprop.c
DEP /tmp/qemu-test/src/dtc/tests/set_name.c
DEP /tmp/qemu-test/src/dtc/tests/rw_tree1.c
DEP /tmp/qemu-test/src/dtc/tests/open_pack.c
DEP /tmp/qemu-test/src/dtc/tests/nopulate.c
DEP /tmp/qemu-test/src/dtc/tests/mangle-layout.c
DEP /tmp/qemu-test/src/dtc/tests/move_and_save.c
DEP /tmp/qemu-test/src/dtc/tests/sw_tree1.c
DEP /tmp/qemu-test/src/dtc/tests/nop_node.c
DEP /tmp/qemu-test/src/dtc/tests/nop_property.c
DEP /tmp/qemu-test/src/dtc/tests/setprop_inplace.c
DEP /tmp/qemu-test/src/dtc/tests/stringlist.c
DEP /tmp/qemu-test/src/dtc/tests/addr_size_cells.c
DEP /tmp/qemu-test/src/dtc/tests/notfound.c
DEP /tmp/qemu-test/src/dtc/tests/sized_cells.c
DEP /tmp/qemu-test/src/dtc/tests/char_literal.c
DEP /tmp/qemu-test/src/dtc/tests/get_alias.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_compatible.c
DEP /tmp/qemu-test/src/dtc/tests/node_check_compatible.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_phandle.c
DEP /tmp/qemu-test/src/dtc/tests/node_offset_by_prop_value.c
DEP /tmp/qemu-test/src/dtc/tests/parent_offset.c
DEP /tmp/qemu-test/src/dtc/tests/supernode_atdepth_offset.c
DEP /tmp/qemu-test/src/dtc/tests/get_path.c
DEP /tmp/qemu-test/src/dtc/tests/get_phandle.c
DEP /tmp/qemu-test/src/dtc/tests/getprop.c
DEP /tmp/qemu-test/src/dtc/tests/get_name.c
DEP /tmp/qemu-test/src/dtc/tests/path_offset.c
DEP /tmp/qemu-test/src/dtc/tests/subnode_offset.c
DEP /tmp/qemu-test/src/dtc/tests/find_property.c
DEP /tmp/qemu-test/src/dtc/tests/root_node.c
DEP /tmp/qemu-test/src/dtc/tests/get_mem_rsv.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_addresses.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_overlay.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_empty_tree.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_rw.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_strerror.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_sw.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_wip.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt_ro.c
DEP /tmp/qemu-test/src/dtc/libfdt/fdt.c
DEP /tmp/qemu-test/src/dtc/util.c
DEP /tmp/qemu-test/src/dtc/fdtoverlay.c
DEP /tmp/qemu-test/src/dtc/fdtput.c
DEP /tmp/qemu-test/src/dtc/fdtget.c
DEP /tmp/qemu-test/src/dtc/fdtdump.c
LEX convert-dtsv0-lexer.lex.c
DEP /tmp/qemu-test/src/dtc/srcpos.c
BISON dtc-parser.tab.c
LEX dtc-lexer.lex.c
DEP /tmp/qemu-test/src/dtc/treesource.c
DEP /tmp/qemu-test/src/dtc/livetree.c
DEP /tmp/qemu-test/src/dtc/fstree.c
DEP /tmp/qemu-test/src/dtc/flattree.c
DEP /tmp/qemu-test/src/dtc/dtc.c
DEP /tmp/qemu-test/src/dtc/data.c
DEP /tmp/qemu-test/src/dtc/checks.c
DEP convert-dtsv0-lexer.lex.c
DEP dtc-parser.tab.c
DEP dtc-lexer.lex.c
CHK version_gen.h
UPD version_gen.h
DEP /tmp/qemu-test/src/dtc/util.c
CC libfdt/fdt.o
CC libfdt/fdt_ro.o
CC libfdt/fdt_wip.o
CC libfdt/fdt_sw.o
CC libfdt/fdt_strerror.o
CC libfdt/fdt_rw.o
CC libfdt/fdt_empty_tree.o
CC libfdt/fdt_addresses.o
CC libfdt/fdt_overlay.o
AR libfdt/libfdt.a
x86_64-w64-mingw32-ar: creating libfdt/libfdt.a
a - libfdt/fdt.o
a - libfdt/fdt_ro.o
a - libfdt/fdt_wip.o
a - libfdt/fdt_sw.o
a - libfdt/fdt_rw.o
a - libfdt/fdt_strerror.o
a - libfdt/fdt_empty_tree.o
a - libfdt/fdt_addresses.o
a - libfdt/fdt_overlay.o
RC version.o
mkdir -p dtc/libfdt
mkdir -p dtc/tests
GEN qga/qapi-generated/qapi-gen
CC qapi/qapi-types.o
CC qapi/qapi-types-block-core.o
CC qapi/qapi-builtin-types.o
CC qapi/qapi-types-char.o
CC qapi/qapi-types-block.o
CC qapi/qapi-types-common.o
CC qapi/qapi-types-crypto.o
CC qapi/qapi-types-introspect.o
CC qapi/qapi-types-migration.o
CC qapi/qapi-types-misc.o
CC qapi/qapi-types-net.o
CC qapi/qapi-types-rocker.o
CC qapi/qapi-types-run-state.o
CC qapi/qapi-types-sockets.o
CC qapi/qapi-types-tpm.o
CC qapi/qapi-types-trace.o
CC qapi/qapi-types-transaction.o
CC qapi/qapi-types-ui.o
CC qapi/qapi-builtin-visit.o
CC qapi/qapi-visit.o
CC qapi/qapi-visit-block-core.o
CC qapi/qapi-visit-block.o
CC qapi/qapi-visit-char.o
CC qapi/qapi-visit-common.o
CC qapi/qapi-visit-crypto.o
CC qapi/qapi-visit-introspect.o
CC qapi/qapi-visit-migration.o
CC qapi/qapi-visit-misc.o
CC qapi/qapi-visit-net.o
CC qapi/qapi-visit-rocker.o
CC qapi/qapi-visit-run-state.o
CC qapi/qapi-visit-sockets.o
CC qapi/qapi-visit-tpm.o
CC qapi/qapi-visit-trace.o
CC qapi/qapi-visit-transaction.o
CC qapi/qapi-events.o
CC qapi/qapi-visit-ui.o
CC qapi/qapi-events-block-core.o
CC qapi/qapi-events-block.o
CC qapi/qapi-events-char.o
CC qapi/qapi-events-common.o
CC qapi/qapi-events-crypto.o
CC qapi/qapi-events-introspect.o
CC qapi/qapi-events-migration.o
CC qapi/qapi-events-misc.o
CC qapi/qapi-events-net.o
CC qapi/qapi-events-rocker.o
CC qapi/qapi-events-run-state.o
CC qapi/qapi-events-sockets.o
CC qapi/qapi-events-tpm.o
CC qapi/qapi-events-trace.o
CC qapi/qapi-events-transaction.o
CC qapi/qapi-events-ui.o
CC qapi/qapi-introspect.o
CC qapi/qapi-visit-core.o
CC qapi/qapi-dealloc-visitor.o
CC qapi/qobject-input-visitor.o
CC qapi/qobject-output-visitor.o
CC qapi/qmp-registry.o
CC qapi/qmp-dispatch.o
CC qapi/string-input-visitor.o
CC qapi/string-output-visitor.o
CC qapi/opts-visitor.o
CC qapi/qapi-clone-visitor.o
CC qapi/qmp-event.o
CC qapi/qapi-util.o
CC qobject/qnull.o
CC qobject/qnum.o
CC qobject/qstring.o
CC qobject/qdict.o
CC qobject/qlist.o
CC qobject/qbool.o
CC qobject/qlit.o
CC qobject/qjson.o
CC qobject/qobject.o
CC qobject/json-lexer.o
CC qobject/json-streamer.o
CC qobject/json-parser.o
CC trace/simple.o
CC trace/control.o
CC trace/qmp.o
CC util/osdep.o
CC util/cutils.o
CC util/unicode.o
CC util/qemu-timer-common.o
CC util/bufferiszero.o
CC util/lockcnt.o
CC util/aiocb.o
CC util/async.o
CC util/aio-wait.o
CC util/thread-pool.o
CC util/qemu-timer.o
CC util/main-loop.o
CC util/iohandler.o
CC util/aio-win32.o
CC util/event_notifier-win32.o
CC util/oslib-win32.o
CC util/qemu-thread-win32.o
CC util/envlist.o
CC util/path.o
CC util/module.o
CC util/host-utils.o
CC util/bitmap.o
CC util/bitops.o
CC util/hbitmap.o
CC util/fifo8.o
CC util/acl.o
CC util/cacheinfo.o
CC util/error.o
CC util/qemu-error.o
CC util/id.o
CC util/iov.o
CC util/qemu-config.o
CC util/qemu-sockets.o
CC util/uri.o
CC util/notify.o
CC util/qemu-option.o
CC util/qemu-progress.o
CC util/keyval.o
CC util/hexdump.o
CC util/crc32c.o
CC util/uuid.o
CC util/throttle.o
CC util/getauxval.o
CC util/readline.o
CC util/rcu.o
CC util/qemu-coroutine.o
CC util/qemu-coroutine-lock.o
CC util/qemu-coroutine-io.o
CC util/qemu-coroutine-sleep.o
CC util/coroutine-win32.o
CC util/buffer.o
CC util/timed-average.o
CC util/base64.o
CC util/log.o
CC util/pagesize.o
CC util/qdist.o
CC util/qht.o
CC util/range.o
CC util/systemd.o
CC util/stats64.o
CC trace-root.o
CC util/trace.o
CC crypto/trace.o
CC io/trace.o
CC migration/trace.o
CC block/trace.o
CC chardev/trace.o
CC hw/block/trace.o
CC hw/block/dataplane/trace.o
CC hw/char/trace.o
CC hw/intc/trace.o
CC hw/net/trace.o
CC hw/rdma/trace.o
CC hw/rdma/vmw/trace.o
CC hw/virtio/trace.o
CC hw/audio/trace.o
CC hw/misc/trace.o
CC hw/misc/macio/trace.o
CC hw/usb/trace.o
CC hw/scsi/trace.o
CC hw/nvram/trace.o
CC hw/display/trace.o
CC hw/input/trace.o
CC hw/timer/trace.o
CC hw/dma/trace.o
CC hw/sparc/trace.o
CC hw/sparc64/trace.o
CC hw/sd/trace.o
CC hw/isa/trace.o
CC hw/mem/trace.o
CC hw/i386/trace.o
CC hw/i386/xen/trace.o
CC hw/9pfs/trace.o
CC hw/ppc/trace.o
CC hw/pci/trace.o
CC hw/pci-host/trace.o
CC hw/s390x/trace.o
CC hw/vfio/trace.o
CC hw/acpi/trace.o
CC hw/arm/trace.o
CC hw/alpha/trace.o
CC hw/hppa/trace.o
CC hw/xen/trace.o
CC hw/ide/trace.o
CC hw/tpm/trace.o
CC ui/trace.o
CC audio/trace.o
CC net/trace.o
CC target/arm/trace.o
CC target/i386/trace.o
CC target/mips/trace.o
CC target/sparc/trace.o
CC target/s390x/trace.o
CC target/ppc/trace.o
CC qom/trace.o
CC linux-user/trace.o
CC qapi/trace.o
CC accel/tcg/trace.o
CC accel/kvm/trace.o
CC nbd/trace.o
CC scsi/trace.o
CC crypto/pbkdf-stub.o
CC stubs/arch-query-cpu-def.o
CC stubs/arch-query-cpu-model-expansion.o
CC stubs/arch-query-cpu-model-comparison.o
CC stubs/arch-query-cpu-model-baseline.o
CC stubs/bdrv-next-monitor-owned.o
CC stubs/blk-commit-all.o
CC stubs/blockdev-close-all-bdrv-states.o
CC stubs/clock-warp.o
CC stubs/cpu-get-clock.o
CC stubs/cpu-get-icount.o
CC stubs/dump.o
CC stubs/error-printf.o
CC stubs/fdset.o
CC stubs/gdbstub.o
CC stubs/get-vm-name.o
CC stubs/iothread.o
CC stubs/iothread-lock.o
CC stubs/is-daemonized.o
CC stubs/machine-init-done.o
CC stubs/migr-blocker.o
CC stubs/change-state-handler.o
CC stubs/monitor.o
CC stubs/notify-event.o
CC stubs/qtest.o
CC stubs/replay.o
CC stubs/runstate-check.o
CC stubs/set-fd-handler.o
CC stubs/slirp.o
CC stubs/sysbus.o
CC stubs/tpm.o
CC stubs/trace-control.o
CC stubs/uuid.o
CC stubs/vm-stop.o
CC stubs/vmstate.o
CC stubs/fd-register.o
CC stubs/qmp_pc_dimm.o
CC stubs/target-monitor-defs.o
CC stubs/target-get-monitor-def.o
CC stubs/pc_madt_cpu_entry.o
CC stubs/vmgenid.o
CC stubs/xen-common.o
CC stubs/xen-hvm.o
CC stubs/pci-host-piix.o
CC stubs/ram-block.o
GEN qemu-img-cmds.h
CC block.o
CC blockjob.o
CC qemu-io-cmds.o
CC replication.o
CC block/raw-format.o
CC block/qcow.o
CC block/vdi.o
CC block/vmdk.o
CC block/cloop.o
CC block/bochs.o
CC block/vpc.o
CC block/vvfat.o
CC block/dmg.o
CC block/qcow2.o
CC block/qcow2-refcount.o
CC block/qcow2-cluster.o
CC block/qcow2-snapshot.o
CC block/qcow2-cache.o
CC block/qcow2-bitmap.o
CC block/qed.o
CC block/qed-l2-cache.o
CC block/qed-table.o
CC block/qed-cluster.o
CC block/qed-check.o
CC block/vhdx.o
CC block/vhdx-endian.o
CC block/vhdx-log.o
CC block/quorum.o
CC block/parallels.o
CC block/blkverify.o
CC block/blkdebug.o
CC block/blkreplay.o
CC block/block-backend.o
CC block/snapshot.o
CC block/qapi.o
CC block/file-win32.o
CC block/win32-aio.o
CC block/null.o
CC block/mirror.o
CC block/commit.o
CC block/io.o
CC block/create.o
CC block/throttle-groups.o
CC block/nbd.o
CC block/nbd-client.o
CC block/sheepdog.o
CC block/accounting.o
CC block/dirty-bitmap.o
CC block/write-threshold.o
CC block/backup.o
CC block/replication.o
CC block/throttle.o
CC block/crypto.o
CC nbd/server.o
CC nbd/client.o
CC nbd/common.o
CC scsi/utils.o
CC block/curl.o
CC block/ssh.o
CC block/dmg-bz2.o
CC crypto/init.o
CC crypto/hash.o
CC crypto/hash-nettle.o
CC crypto/hmac.o
CC crypto/hmac-nettle.o
CC crypto/aes.o
CC crypto/desrfb.o
CC crypto/cipher.o
CC crypto/tlscreds.o
CC crypto/tlscredsanon.o
CC crypto/tlscredsx509.o
CC crypto/tlssession.o
CC crypto/secret.o
CC crypto/random-gnutls.o
CC crypto/pbkdf.o
CC crypto/pbkdf-nettle.o
CC crypto/ivgen.o
CC crypto/ivgen-essiv.o
CC crypto/ivgen-plain.o
CC crypto/ivgen-plain64.o
CC crypto/afsplit.o
CC crypto/xts.o
CC crypto/block.o
CC crypto/block-qcow.o
CC crypto/block-luks.o
CC io/channel.o
CC io/channel-file.o
CC io/channel-buffer.o
CC io/channel-command.o
CC io/channel-socket.o
CC io/channel-tls.o
CC io/channel-watch.o
CC io/channel-websock.o
CC io/channel-util.o
CC io/dns-resolver.o
CC io/net-listener.o
CC io/task.o
CC qom/object.o
CC qom/qom-qobject.o
CC qom/container.o
CC qom/object_interfaces.o
CC qemu-io.o
CC blockdev.o
CC blockdev-nbd.o
CC bootdevice.o
CC iothread.o
CC qdev-monitor.o
CC device-hotplug.o
CC os-win32.o
CC bt-host.o
CC bt-vhci.o
CC dma-helpers.o
CC vl.o
CC tpm.o
CC device_tree.o
CC qapi/qapi-commands.o
CC qapi/qapi-commands-block-core.o
CC qapi/qapi-commands-block.o
CC qapi/qapi-commands-char.o
CC qapi/qapi-commands-common.o
CC qapi/qapi-commands-crypto.o
CC qapi/qapi-commands-introspect.o
CC qapi/qapi-commands-migration.o
CC qapi/qapi-commands-misc.o
CC qapi/qapi-commands-net.o
CC qapi/qapi-commands-rocker.o
CC qapi/qapi-commands-sockets.o
CC qapi/qapi-commands-run-state.o
CC qapi/qapi-commands-tpm.o
CC qapi/qapi-commands-trace.o
CC qapi/qapi-commands-transaction.o
CC qapi/qapi-commands-ui.o
CC qmp.o
CC hmp.o
CC cpus-common.o
CC audio/audio.o
CC audio/noaudio.o
CC audio/wavaudio.o
CC audio/mixeng.o
CC audio/dsoundaudio.o
CC audio/audio_win_int.o
CC audio/wavcapture.o
CC backends/rng.o
CC backends/rng-egd.o
CC backends/tpm.o
CC backends/hostmem.o
CC backends/hostmem-ram.o
CC backends/cryptodev.o
CC backends/cryptodev-builtin.o
CC block/stream.o
CC backends/cryptodev-vhost.o
CC chardev/msmouse.o
CC chardev/wctablet.o
CC chardev/testdev.o
CC disas/arm.o
CXX disas/arm-a64.o
CC disas/i386.o
CXX disas/libvixl/vixl/utils.o
CXX disas/libvixl/vixl/compiler-intrinsics.o
CXX disas/libvixl/vixl/a64/instructions-a64.o
CXX disas/libvixl/vixl/a64/decoder-a64.o
CXX disas/libvixl/vixl/a64/disasm-a64.o
CC hw/acpi/core.o
CC hw/acpi/piix4.o
CC hw/acpi/pcihp.o
CC hw/acpi/ich9.o
CC hw/acpi/tco.o
CC hw/acpi/cpu_hotplug.o
CC hw/acpi/memory_hotplug.o
CC hw/acpi/cpu.o
CC hw/acpi/nvdimm.o
CC hw/acpi/vmgenid.o
CC hw/acpi/acpi_interface.o
CC hw/acpi/bios-linker-loader.o
CC hw/acpi/aml-build.o
CC hw/acpi/ipmi.o
CC hw/acpi/acpi-stub.o
CC hw/acpi/ipmi-stub.o
CC hw/audio/sb16.o
CC hw/audio/es1370.o
CC hw/audio/ac97.o
CC hw/audio/fmopl.o
CC hw/audio/adlib.o
CC hw/audio/gus.o
CC hw/audio/gusemu_hal.o
CC hw/audio/gusemu_mixer.o
CC hw/audio/cs4231a.o
CC hw/audio/intel-hda.o
CC hw/audio/hda-codec.o
CC hw/audio/pcspk.o
CC hw/audio/wm8750.o
CC hw/audio/pl041.o
CC hw/audio/lm4549.o
CC hw/audio/marvell_88w8618.o
CC hw/audio/soundhw.o
CC hw/block/block.o
CC hw/block/cdrom.o
CC hw/block/hd-geometry.o
CC hw/block/fdc.o
CC hw/block/m25p80.o
CC hw/block/nand.o
CC hw/block/pflash_cfi01.o
CC hw/block/pflash_cfi02.o
CC hw/block/ecc.o
CC hw/block/onenand.o
CC hw/block/nvme.o
CC hw/bt/core.o
CC hw/bt/l2cap.o
CC hw/bt/sdp.o
CC hw/bt/hci.o
CC hw/bt/hid.o
CC hw/bt/hci-csr.o
CC hw/char/ipoctal232.o
CC hw/char/parallel.o
CC hw/char/pl011.o
CC hw/char/serial.o
CC hw/char/serial-isa.o
CC hw/char/serial-pci.o
CC hw/char/virtio-console.o
CC hw/char/cadence_uart.o
CC hw/char/cmsdk-apb-uart.o
CC hw/char/debugcon.o
CC hw/char/imx_serial.o
CC hw/core/qdev.o
CC hw/core/qdev-properties.o
CC hw/core/bus.o
CC hw/core/reset.o
CC hw/core/qdev-fw.o
CC hw/core/fw-path-provider.o
CC hw/core/irq.o
CC hw/core/hotplug.o
CC hw/core/stream.o
CC hw/core/nmi.o
CC hw/core/ptimer.o
CC hw/core/sysbus.o
CC hw/core/machine.o
CC hw/core/loader.o
CC hw/core/qdev-properties-system.o
CC hw/core/register.o
CC hw/core/or-irq.o
CC hw/core/split-irq.o
CC hw/core/platform-bus.o
CC hw/cpu/core.o
CC hw/display/ads7846.o
CC hw/display/cirrus_vga.o
CC hw/display/pl110.o
CC hw/display/sii9022.o
CC hw/display/ssd0303.o
CC hw/display/ssd0323.o
CC hw/display/vga-pci.o
CC hw/display/vga-isa.o
CC hw/display/vmware_vga.o
CC hw/display/blizzard.o
CC hw/display/exynos4210_fimd.o
CC hw/display/framebuffer.o
CC hw/display/tc6393xb.o
CC hw/dma/pl080.o
CC hw/dma/pl330.o
CC hw/dma/i8257.o
CC hw/dma/xilinx_axidma.o
CC hw/dma/xlnx-zynq-devcfg.o
CC hw/gpio/max7310.o
CC hw/gpio/pl061.o
CC hw/gpio/zaurus.o
CC hw/gpio/gpio_key.o
CC hw/i2c/core.o
CC hw/i2c/smbus.o
CC hw/i2c/smbus_eeprom.o
CC hw/i2c/i2c-ddc.o
CC hw/i2c/versatile_i2c.o
CC hw/i2c/smbus_ich9.o
CC hw/i2c/pm_smbus.o
CC hw/i2c/bitbang_i2c.o
CC hw/i2c/exynos4210_i2c.o
CC hw/i2c/imx_i2c.o
CC hw/i2c/aspeed_i2c.o
CC hw/ide/core.o
CC hw/ide/atapi.o
CC hw/ide/qdev.o
CC hw/ide/pci.o
CC hw/ide/isa.o
CC hw/ide/piix.o
CC hw/ide/microdrive.o
CC hw/ide/ahci.o
CC hw/ide/ich.o
CC hw/ide/ahci-allwinner.o
CC hw/input/hid.o
CC hw/input/lm832x.o
CC hw/input/pckbd.o
CC hw/input/pl050.o
CC hw/input/ps2.o
CC hw/input/stellaris_input.o
CC hw/input/tsc2005.o
CC hw/input/virtio-input.o
CC hw/input/virtio-input-hid.o
CC hw/intc/i8259_common.o
CC hw/intc/i8259.o
CC hw/intc/pl190.o
CC hw/intc/xlnx-pmu-iomod-intc.o
CC hw/intc/xlnx-zynqmp-ipi.o
CC hw/intc/imx_avic.o
CC hw/intc/imx_gpcv2.o
CC hw/intc/realview_gic.o
CC hw/intc/ioapic_common.o
CC hw/intc/arm_gic_common.o
CC hw/intc/arm_gic.o
CC hw/intc/arm_gicv2m.o
CC hw/intc/arm_gicv3_common.o
CC hw/intc/arm_gicv3.o
CC hw/intc/arm_gicv3_dist.o
CC hw/intc/arm_gicv3_redist.o
CC hw/intc/arm_gicv3_its_common.o
CC hw/intc/intc.o
CC hw/ipack/ipack.o
CC hw/ipack/tpci200.o
CC hw/ipmi/ipmi.o
CC hw/ipmi/ipmi_bmc_sim.o
CC hw/ipmi/ipmi_bmc_extern.o
CC hw/ipmi/isa_ipmi_kcs.o
CC hw/ipmi/isa_ipmi_bt.o
CC hw/isa/isa-bus.o
CC hw/isa/apm.o
CC hw/mem/pc-dimm.o
CC hw/mem/nvdimm.o
CC hw/misc/applesmc.o
CC hw/misc/max111x.o
CC hw/misc/tmp105.o
CC hw/misc/tmp421.o
CC hw/misc/debugexit.o
CC hw/misc/sga.o
CC hw/misc/pc-testdev.o
CC hw/misc/pci-testdev.o
CC hw/misc/edu.o
CC hw/misc/unimp.o
CC hw/misc/vmcoreinfo.o
CC hw/misc/arm_l2x0.o
CC hw/misc/arm_integrator_debug.o
CC hw/misc/a9scu.o
CC hw/misc/arm11scu.o
CC hw/net/eepro100.o
CC hw/net/ne2000.o
CC hw/net/pcnet-pci.o
CC hw/net/pcnet.o
CC hw/net/e1000.o
CC hw/net/e1000x_common.o
CC hw/net/net_tx_pkt.o
CC hw/net/net_rx_pkt.o
CC hw/net/e1000e.o
CC hw/net/e1000e_core.o
CC hw/net/rtl8139.o
CC hw/net/vmxnet3.o
CC hw/net/smc91c111.o
CC hw/net/lan9118.o
CC hw/net/ne2000-isa.o
CC hw/net/xgmac.o
CC hw/net/xilinx_axienet.o
CC hw/net/allwinner_emac.o
CC hw/net/imx_fec.o
CC hw/net/cadence_gem.o
CC hw/net/stellaris_enet.o
CC hw/net/ftgmac100.o
CC hw/net/rocker/rocker.o
CC hw/net/rocker/rocker_fp.o
CC hw/net/rocker/rocker_desc.o
CC hw/net/rocker/rocker_world.o
CC hw/net/rocker/rocker_of_dpa.o
CC hw/net/can/can_sja1000.o
CC hw/net/can/can_kvaser_pci.o
CC hw/net/can/can_pcm3680_pci.o
CC hw/net/can/can_mioe3680_pci.o
CC hw/nvram/eeprom93xx.o
CC hw/nvram/eeprom_at24c.o
CC hw/nvram/fw_cfg.o
CC hw/nvram/chrp_nvram.o
CC hw/pci-bridge/pcie_root_port.o
CC hw/pci-bridge/pci_bridge_dev.o
CC hw/pci-bridge/gen_pcie_root_port.o
CC hw/pci-bridge/pcie_pci_bridge.o
CC hw/pci-bridge/pci_expander_bridge.o
CC hw/pci-bridge/xio3130_upstream.o
CC hw/pci-bridge/xio3130_downstream.o
CC hw/pci-bridge/ioh3420.o
CC hw/pci-bridge/i82801b11.o
CC hw/pci-host/pam.o
CC hw/pci-host/versatile.o
CC hw/pci-host/piix.o
CC hw/pci-host/q35.o
CC hw/pci-host/gpex.o
CC hw/pci-host/designware.o
CC hw/pci/pci.o
CC hw/pci/pci_bridge.o
CC hw/pci/msix.o
CC hw/pci/msi.o
CC hw/pci/shpc.o
CC hw/pci/slotid_cap.o
CC hw/pci/pci_host.o
CC hw/pci/pcie_host.o
CC hw/pci/pcie.o
CC hw/pci/pcie_port.o
CC hw/pci/pcie_aer.o
CC hw/pci/pci-stub.o
CC hw/pcmcia/pcmcia.o
CC hw/scsi/scsi-disk.o
CC hw/scsi/scsi-generic.o
CC hw/scsi/scsi-bus.o
CC hw/scsi/lsi53c895a.o
CC hw/scsi/mptsas.o
CC hw/scsi/mptconfig.o
CC hw/scsi/mptendian.o
CC hw/scsi/megasas.o
CC hw/scsi/vmw_pvscsi.o
CC hw/scsi/esp.o
CC hw/scsi/esp-pci.o
CC hw/sd/pl181.o
CC hw/sd/ssi-sd.o
CC hw/sd/sd.o
CC hw/sd/core.o
CC hw/sd/sdmmc-internal.o
CC hw/sd/sdhci.o
CC hw/smbios/smbios.o
CC hw/smbios/smbios_type_38.o
CC hw/smbios/smbios-stub.o
CC hw/smbios/smbios_type_38-stub.o
CC hw/ssi/pl022.o
CC hw/ssi/ssi.o
CC hw/ssi/xilinx_spips.o
CC hw/ssi/aspeed_smc.o
CC hw/ssi/stm32f2xx_spi.o
CC hw/ssi/mss-spi.o
CC hw/timer/arm_timer.o
CC hw/timer/arm_mptimer.o
CC hw/timer/armv7m_systick.o
CC hw/timer/a9gtimer.o
CC hw/timer/cadence_ttc.o
CC hw/timer/ds1338.o
CC hw/timer/hpet.o
CC hw/timer/i8254_common.o
CC hw/timer/i8254.o
CC hw/timer/twl92230.o
CC hw/timer/pl031.o
CC hw/timer/imx_epit.o
CC hw/timer/imx_gpt.o
CC hw/timer/xlnx-zynqmp-rtc.o
CC hw/timer/stm32f2xx_timer.o
CC hw/timer/aspeed_timer.o
CC hw/timer/cmsdk-apb-timer.o
CC hw/timer/mss-timer.o
CC hw/tpm/tpm_util.o
CC hw/tpm/tpm_tis.o
CC hw/tpm/tpm_crb.o
CC hw/usb/core.o
CC hw/usb/combined-packet.o
CC hw/usb/bus.o
CC hw/usb/libhw.o
CC hw/usb/desc.o
CC hw/usb/desc-msos.o
CC hw/usb/hcd-uhci.o
CC hw/usb/hcd-ohci.o
CC hw/usb/hcd-ehci.o
CC hw/usb/hcd-ehci-pci.o
CC hw/usb/hcd-ehci-sysbus.o
CC hw/usb/hcd-xhci.o
CC hw/usb/hcd-xhci-nec.o
CC hw/usb/hcd-musb.o
CC hw/usb/dev-hub.o
CC hw/usb/dev-hid.o
CC hw/usb/dev-wacom.o
CC hw/usb/dev-storage.o
CC hw/usb/dev-uas.o
CC hw/usb/dev-audio.o
CC hw/usb/dev-serial.o
CC hw/usb/dev-network.o
CC hw/usb/dev-bluetooth.o
CC hw/usb/dev-smartcard-reader.o
CC hw/usb/host-stub.o
CC hw/virtio/virtio-rng.o
CC hw/virtio/virtio-pci.o
CC hw/virtio/virtio-mmio.o
CC hw/virtio/virtio-bus.o
CC hw/virtio/vhost-stub.o
CC hw/watchdog/watchdog.o
CC hw/watchdog/wdt_i6300esb.o
CC hw/watchdog/wdt_ib700.o
CC hw/watchdog/wdt_aspeed.o
CC migration/migration.o
CC migration/socket.o
CC migration/fd.o
CC migration/exec.o
CC migration/tls.o
CC migration/channel.o
CC migration/savevm.o
CC migration/colo-comm.o
CC migration/colo.o
CC migration/colo-failover.o
CC migration/vmstate.o
CC migration/vmstate-types.o
CC migration/page_cache.o
CC migration/qemu-file.o
CC migration/global_state.o
CC migration/qemu-file-channel.o
CC migration/xbzrle.o
CC migration/postcopy-ram.o
CC migration/qjson.o
CC migration/block.o
CC net/net.o
CC net/queue.o
CC net/checksum.o
CC net/util.o
CC net/hub.o
CC net/socket.o
CC net/dump.o
CC net/eth.o
CC net/slirp.o
CC net/filter.o
CC net/filter-buffer.o
CC net/filter-mirror.o
CC net/colo-compare.o
CC net/colo.o
CC net/filter-rewriter.o
CC net/filter-replay.o
CC net/tap-win32.o
CC net/can/can_core.o
CC net/can/can_host.o
CC qom/cpu.o
CC replay/replay.o
CC replay/replay-internal.o
CC replay/replay-events.o
CC replay/replay-time.o
CC replay/replay-input.o
CC replay/replay-char.o
CC replay/replay-snapshot.o
CC replay/replay-net.o
CC replay/replay-audio.o
CC slirp/cksum.o
CC slirp/if.o
CC slirp/ip_icmp.o
CC slirp/ip6_input.o
CC slirp/ip6_icmp.o
CC slirp/ip6_output.o
CC slirp/ip_input.o
CC slirp/ip_output.o
CC slirp/dnssearch.o
CC slirp/dhcpv6.o
CC slirp/slirp.o
CC slirp/mbuf.o
CC slirp/misc.o
CC slirp/sbuf.o
CC slirp/socket.o
CC slirp/tcp_input.o
CC slirp/tcp_subr.o
CC slirp/tcp_output.o
CC slirp/tcp_timer.o
CC slirp/udp.o
CC slirp/udp6.o
CC slirp/bootp.o
CC slirp/tftp.o
CC slirp/arp_table.o
CC slirp/ndp_table.o
CC slirp/ncsi.o
CC ui/keymaps.o
CC ui/console.o
CC ui/qemu-pixman.o
CC ui/cursor.o
CC ui/input.o
CC ui/input-keymap.o
CC ui/input-legacy.o
CC ui/vnc.o
CC ui/vnc-enc-zlib.o
CC ui/vnc-enc-hextile.o
CC ui/vnc-enc-tight.o
CC ui/vnc-palette.o
CC ui/vnc-auth-vencrypt.o
CC ui/vnc-enc-zrle.o
CC ui/vnc-ws.o
CC ui/vnc-jobs.o
CC ui/sdl.o
CC ui/sdl_zoom.o
CC ui/gtk.o
CC chardev/char.o
CC chardev/char-console.o
CC chardev/char-fe.o
CC chardev/char-file.o
CC chardev/char-io.o
CC chardev/char-mux.o
CC chardev/char-null.o
CC chardev/char-pipe.o
CC chardev/char-ringbuf.o
CC chardev/char-serial.o
CC chardev/char-socket.o
CC chardev/char-stdio.o
CC chardev/char-udp.o
CC chardev/char-win.o
CC chardev/char-win-stdio.o
CC qga/commands.o
CC qga/guest-agent-command-state.o
CC qga/main.o
CC qga/commands-win32.o
CC qga/channel-win32.o
CC qga/service-win32.o
CC qga/vss-win32.o
AS optionrom/multiboot.o
CC qga/qapi-generated/qga-qapi-types.o
AS optionrom/linuxboot.o
CC optionrom/linuxboot_dma.o
CC qga/qapi-generated/qga-qapi-visit.o
CC qga/qapi-generated/qga-qapi-commands.o
AS optionrom/kvmvapic.o
BUILD optionrom/multiboot.img
BUILD optionrom/linuxboot.img
BUILD optionrom/linuxboot_dma.img
BUILD optionrom/kvmvapic.img
BUILD optionrom/multiboot.raw
AR libqemuutil.a
CC qemu-img.o
BUILD optionrom/linuxboot.raw
BUILD optionrom/linuxboot_dma.raw
BUILD optionrom/kvmvapic.raw
SIGN optionrom/linuxboot.bin
SIGN optionrom/linuxboot_dma.bin
SIGN optionrom/multiboot.bin
SIGN optionrom/kvmvapic.bin
LINK qemu-io.exe
LINK qemu-img.exe
LINK qemu-ga.exe
GEN x86_64-softmmu/hmp-commands.h
GEN x86_64-softmmu/hmp-commands-info.h
GEN x86_64-softmmu/config-target.h
GEN aarch64-softmmu/hmp-commands-info.h
GEN aarch64-softmmu/hmp-commands.h
GEN aarch64-softmmu/config-target.h
CC x86_64-softmmu/tcg/tcg-op-vec.o
CC x86_64-softmmu/exec.o
CC x86_64-softmmu/tcg/tcg-op.o
CC x86_64-softmmu/tcg/tcg-common.o
CC x86_64-softmmu/tcg/tcg.o
CC x86_64-softmmu/tcg/tcg-op-gvec.o
CC x86_64-softmmu/tcg/optimize.o
CC aarch64-softmmu/exec.o
CC x86_64-softmmu/fpu/softfloat.o
CC aarch64-softmmu/tcg/tcg.o
CC aarch64-softmmu/tcg/tcg-op.o
CC x86_64-softmmu/disas.o
CC aarch64-softmmu/tcg/tcg-op-vec.o
GEN x86_64-softmmu/gdbstub-xml.c
CC x86_64-softmmu/arch_init.o
CC aarch64-softmmu/tcg/tcg-op-gvec.o
CC aarch64-softmmu/tcg/tcg-common.o
CC aarch64-softmmu/tcg/optimize.o
CC x86_64-softmmu/cpus.o
CC x86_64-softmmu/monitor.o
CC x86_64-softmmu/gdbstub.o
CC aarch64-softmmu/fpu/softfloat.o
CC aarch64-softmmu/disas.o
CC x86_64-softmmu/balloon.o
CC x86_64-softmmu/ioport.o
CC x86_64-softmmu/numa.o
CC x86_64-softmmu/qtest.o
CC x86_64-softmmu/memory.o
CC x86_64-softmmu/memory_mapping.o
CC x86_64-softmmu/dump.o
CC x86_64-softmmu/migration/ram.o
CC x86_64-softmmu/migration/nvdimm.o
GEN aarch64-softmmu/gdbstub-xml.c
CC aarch64-softmmu/arch_init.o
CC x86_64-softmmu/accel/accel.o
CC x86_64-softmmu/accel/stubs/hvf-stub.o
CC x86_64-softmmu/accel/stubs/whpx-stub.o
CC aarch64-softmmu/cpus.o
CC x86_64-softmmu/accel/stubs/kvm-stub.o
CC x86_64-softmmu/accel/tcg/tcg-all.o
CC aarch64-softmmu/monitor.o
CC x86_64-softmmu/accel/tcg/cputlb.o
CC aarch64-softmmu/gdbstub.o
CC aarch64-softmmu/balloon.o
CC aarch64-softmmu/ioport.o
CC aarch64-softmmu/numa.o
CC aarch64-softmmu/qtest.o
CC aarch64-softmmu/memory.o
CC aarch64-softmmu/memory_mapping.o
CC aarch64-softmmu/dump.o
CC x86_64-softmmu/accel/tcg/tcg-runtime.o
CC x86_64-softmmu/accel/tcg/tcg-runtime-gvec.o
CC x86_64-softmmu/accel/tcg/cpu-exec.o
CC aarch64-softmmu/migration/ram.o
CC aarch64-softmmu/migration/nvdimm.o
CC aarch64-softmmu/accel/accel.o
/tmp/qemu-test/src/migration/nvdimm.c: In function 'nvdimm_state_save_dependency':
/tmp/qemu-test/src/migration/nvdimm.c:244:70: error: format '%lx' expects argument of type 'long unsigned int', but argument 2 has type 'int64_t {aka long long int}' [-Werror=format=]
error_report("save file dependency failed, depend_offset = %lx "
~~^
%llx
/tmp/qemu-test/src/migration/nvdimm.c:246:22:
nvdimm_state->depend_offset,
~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/nvdimm.c:244:22: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64_t {aka long long int}' [-Werror=format=]
error_report("save file dependency failed, depend_offset = %lx "
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/nvdimm.c:247:22:
nvdimm_state->depend_size, ret);
~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/nvdimm.c:245:40: note: format string is defined here
"depend_size is %ld, ret is %d",
~~^
%lld
cc1: all warnings being treated as errors
make[1]: *** [/tmp/qemu-test/src/rules.mak:66: migration/nvdimm.o] Error 1
make[1]: *** Waiting for unfinished jobs....
CC x86_64-softmmu/accel/tcg/cpu-exec-common.o
CC aarch64-softmmu/accel/stubs/hax-stub.o
CC aarch64-softmmu/accel/stubs/hvf-stub.o
CC aarch64-softmmu/accel/stubs/whpx-stub.o
CC aarch64-softmmu/accel/stubs/kvm-stub.o
CC aarch64-softmmu/accel/tcg/tcg-all.o
CC aarch64-softmmu/accel/tcg/cputlb.o
CC aarch64-softmmu/accel/tcg/tcg-runtime.o
CC aarch64-softmmu/accel/tcg/tcg-runtime-gvec.o
/tmp/qemu-test/src/migration/nvdimm.c: In function 'nvdimm_state_save_dependency':
/tmp/qemu-test/src/migration/nvdimm.c:244:70: error: format '%lx' expects argument of type 'long unsigned int', but argument 2 has type 'int64_t {aka long long int}' [-Werror=format=]
error_report("save file dependency failed, depend_offset = %lx "
~~^
%llx
/tmp/qemu-test/src/migration/nvdimm.c:246:22:
nvdimm_state->depend_offset,
~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/nvdimm.c:244:22: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'int64_t {aka long long int}' [-Werror=format=]
error_report("save file dependency failed, depend_offset = %lx "
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/nvdimm.c:247:22:
nvdimm_state->depend_size, ret);
~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/qemu-test/src/migration/nvdimm.c:245:40: note: format string is defined here
"depend_size is %ld, ret is %d",
~~^
%lld
cc1: all warnings being treated as errors
make[1]: *** [/tmp/qemu-test/src/rules.mak:66: migration/nvdimm.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:476: subdir-x86_64-softmmu] Error 2
make: *** Waiting for unfinished jobs....
make: *** [Makefile:476: subdir-aarch64-softmmu] Error 2
Traceback (most recent call last):
File "./tests/docker/docker.py", line 407, in <module>
sys.exit(main())
File "./tests/docker/docker.py", line 404, in main
return args.cmdobj.run(args, argv)
File "./tests/docker/docker.py", line 261, in run
return Docker().run(argv, args.keep, quiet=args.quiet)
File "./tests/docker/docker.py", line 229, in run
quiet=quiet)
File "./tests/docker/docker.py", line 147, in _do_check
return subprocess.check_call(self._command + cmd, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 186, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker', 'run', '--label', 'com.qemu.instance.uuid=05481f30285b11e890e752540069c830', '-u', '0', '--security-opt', 'seccomp=unconfined', '--rm', '--net=none', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=8', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/root/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-e7u4by_8/src/docker-src.2018-03-15-10.13.12.11256:/var/tmp/qemu:z,ro', 'qemu:fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2
make[1]: *** [tests/docker/Makefile.include:129: docker-run] Error 1
make[1]: Leaving directory '/var/tmp/patchew-tester-tmp-e7u4by_8/src'
make: *** [tests/docker/Makefile.include:163: docker-run-test-mingw@fedora] Error 2
real 2m23.843s
user 0m4.818s
sys 0m4.185s
=== OUTPUT END ===
Test command exited with code: 2
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@freelists.org
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot.
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
` (11 preceding siblings ...)
2018-03-15 14:15 ` no-reply
@ 2018-05-08 9:23 ` Stefan Hajnoczi
12 siblings, 0 replies; 17+ messages in thread
From: Stefan Hajnoczi @ 2018-05-08 9:23 UTC (permalink / raw)
To: junyan.he
Cc: qemu-devel, kwolf, famz, crosthwaite.peter, quintela, dgilbert,
mreitz, pbonzini, rth
[-- Attachment #1: Type: text/plain, Size: 3983 bytes --]
On Tue, Mar 13, 2018 at 04:33:43PM +0800, junyan.he@intel.com wrote:
> From: Junyan He <junyan.he@intel.com>
>
> The nvdimm size is huge, sometimes it is more than 256G or even more.
> This is a huge burden for snapshot saving. One snapshot point with
> nvdimm may occupy more than 50G disk space even with compression
> enabled.
> We need to introduce dependent snapshot manner to solve this problem.
What is the status of this patch series?
Kevin and Juan: Review is needed from the qcow2 and migration maintainers.
This patch series adds a function to clone qcow2 snapshots (sharing
clusters via the refcount) and then uses it to snapshot NVDIMM contents
incrementally during savevm. This way only dirty NVDIMM clusters need
to be written into the qcow2 file, saving a lot of space and time. The
drawback is that dirty memory logging needs to be enabled while the
guest is running.
The approach makes sense to me and could be used for other migration
state beyond NVDIMM in the future. I think it's worth iterating this
patch series and merging it.
It was difficult for me to understand some of the English. Do you have
a colleague who can review the English and suggest how to rephrase the
text?
I didn't found the "snapshot dependency" name unclear. I suggest
calling the new API .bdrv_snapshot_clone(). This makes it clear that a
new snapshot is created based on an existing one.
New APIs are missing doc comments. Please document the functions,
arguments, and the return values - especially for extern functions
(those declared in header files).
> The first snapshot point should always be saved completely, and enable
> dirty log trace after saving for nvdimm memory region. The later snapshot
> point should add the reference to previous snapshot's nvdimm data and
> just saving dirty pages. This can save a lot of disk and time if the
> snapshot operations are triggered frequently.
> We add save_snapshot_dependency functions to QCOW2 file system firstly, the
> later snapshot will add reference to previous dependent snapshot's data
> cluster. There is an alignment problem here, the dependent data should
> always be cluster aligned. We need to add some padding data when saving
> the snapshot to make it always cluster aligned.
> The logic between nvdimm and ram for snapshot saving is a little confused
> now, we need to exclude nvdimm kind memory region from ram list and the
> dirty log tracing setting is also not very clear. Maybe we can separate the
> snapshot saving from the migration logic later to make code clean.
> In theory, this kind of manner can apply to any kind of memory. But because
> it need to turn dirty log trace on, the performance may decline. So we just
> enable it for nvdimm kind memory firstly.
>
> Signed-off-by: Junyan He <junyan.he@intel.com>
> ---
> Makefile.target | 1 +
> block/qcow2-snapshot.c | 154 ++++++++++++++++++++++
> block/qcow2.c | 2 +
> block/qcow2.h | 7 +
> block/snapshot.c | 45 +++++++
> exec.c | 7 +
> hw/ppc/spapr.c | 2 +-
> hw/s390x/s390-stattrib.c | 2 +-
> include/block/block_int.h | 9 ++
> include/block/snapshot.h | 7 +
> include/exec/memory.h | 9 ++
> include/exec/ram_addr.h | 2 +
> include/migration/misc.h | 4 +
> include/migration/register.h | 2 +-
> include/migration/snapshot.h | 3 +
> memory.c | 18 ++-
> migration/block.c | 2 +-
> migration/nvdimm.c | 1033 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> migration/qemu-file.c | 61 +++++++++
> migration/qemu-file.h | 14 ++
> migration/ram.c | 19 ++-
> migration/savevm.c | 62 ++++++++-
> vl.c | 1 +
> 23 files changed, 1452 insertions(+), 14 deletions(-)
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH 02/10] RFC: Implement qcow2's snapshot dependent saving function.
2018-03-13 8:33 ` [Qemu-devel] [PATCH 02/10] RFC: Implement qcow2's snapshot dependent saving function junyan.he
@ 2018-05-08 14:50 ` Eric Blake
2018-05-14 12:59 ` Stefan Hajnoczi
0 siblings, 1 reply; 17+ messages in thread
From: Eric Blake @ 2018-05-08 14:50 UTC (permalink / raw)
To: junyan.he, qemu-devel
Cc: kwolf, famz, crosthwaite.peter, quintela, dgilbert, mreitz,
pbonzini, rth, Pavel Dovgaluk
On 03/13/2018 03:33 AM, junyan.he@intel.com wrote:
> From: Junyan He <junyan.he@intel.com>
>
> For qcow2 format, we can increase the cluster's reference count of
> dependent snapshot content and link the offset to the L2 table of
> the new snapshot point. This way can avoid obvious snapshot's dependent
> relationship, so when we delete some snapshot point, just decrease the
> cluster count and no need to check further.
>
> Signed-off-by: Junyan He <junyan.he@intel.com>
> ---
> block/qcow2-snapshot.c | 154 +++++++++++++++++++++++++++++++++++++++++++++++++
> block/qcow2.c | 2 +
> block/qcow2.h | 7 +++
> 3 files changed, 163 insertions(+)
It sounds like you are trying to modify the qcow2 spec to store more
information into the internal snapshot table (and if you aren't, why
not? If an internal snapshot depends on another one and we don't record
that information in the qcow2 metadata, then we are pushing the burden
of tracking inter-relationships onto management apps, and risk
corruption if you load a snapshot without also visiting its
dependencies). It is absolutely essential that such modifications be
reflected in docs/interop/qcow2.txt first, to make sure we agree on the
implementation. What's more, there is already another parallel proposal
that is also wanting to tweak qcow2 files:
https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg05231.html
We need to make sure both additions are coordinated.
> +
> + if (!QEMU_IS_ALIGNED(depend_offset, s->cluster_size)) {
Why two spaces here and elsewhere?
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH 02/10] RFC: Implement qcow2's snapshot dependent saving function.
2018-05-08 14:50 ` Eric Blake
@ 2018-05-14 12:59 ` Stefan Hajnoczi
0 siblings, 0 replies; 17+ messages in thread
From: Stefan Hajnoczi @ 2018-05-14 12:59 UTC (permalink / raw)
To: Eric Blake
Cc: junyan.he, qemu-devel, kwolf, famz, quintela, crosthwaite.peter,
dgilbert, mreitz, Pavel Dovgaluk, pbonzini, rth
[-- Attachment #1: Type: text/plain, Size: 1899 bytes --]
On Tue, May 08, 2018 at 09:50:00AM -0500, Eric Blake wrote:
> On 03/13/2018 03:33 AM, junyan.he@intel.com wrote:
> > From: Junyan He <junyan.he@intel.com>
> >
> > For qcow2 format, we can increase the cluster's reference count of
> > dependent snapshot content and link the offset to the L2 table of
> > the new snapshot point. This way can avoid obvious snapshot's dependent
> > relationship, so when we delete some snapshot point, just decrease the
> > cluster count and no need to check further.
> >
> > Signed-off-by: Junyan He <junyan.he@intel.com>
> > ---
> > block/qcow2-snapshot.c | 154 +++++++++++++++++++++++++++++++++++++++++++++++++
> > block/qcow2.c | 2 +
> > block/qcow2.h | 7 +++
> > 3 files changed, 163 insertions(+)
>
> It sounds like you are trying to modify the qcow2 spec to store more
> information into the internal snapshot table (and if you aren't, why not? If
> an internal snapshot depends on another one and we don't record that
> information in the qcow2 metadata, then we are pushing the burden of
> tracking inter-relationships onto management apps, and risk corruption if
> you load a snapshot without also visiting its dependencies). It is
> absolutely essential that such modifications be reflected in
> docs/interop/qcow2.txt first, to make sure we agree on the implementation.
> What's more, there is already another parallel proposal that is also wanting
> to tweak qcow2 files:
>
> https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg05231.html
>
> We need to make sure both additions are coordinated.
I think the relationship doesn't need to be stored. The point of this
snapshot clone operation is to share some of the clusters of the parent
snapshot (using qcow2's existing cluster reference counts). Cloning
avoids having to write a duplicate copy of those clusters.
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH 02/10] RFC: Implement qcow2's snapshot dependent saving function.
2018-03-14 1:20 junyan.he
@ 2018-03-14 1:20 ` junyan.he
0 siblings, 0 replies; 17+ messages in thread
From: junyan.he @ 2018-03-14 1:20 UTC (permalink / raw)
To: qemu-devel
Cc: kwolf, mreitz, pbonzini, crosthwaite.peter, quintela, rth,
dgilbert, famz, Junyan He
From: Junyan He <junyan.he@intel.com>
For qcow2 format, we can increase the cluster's reference count of
dependent snapshot content and link the offset to the L2 table of
the new snapshot point. This way can avoid obvious snapshot's dependent
relationship, so when we delete some snapshot point, just decrease the
cluster count and no need to check further.
Signed-off-by: Junyan He <junyan.he@intel.com>
---
block/qcow2-snapshot.c | 154 +++++++++++++++++++++++++++++++++++++++++++++++++
block/qcow2.c | 2 +
block/qcow2.h | 7 +++
3 files changed, 163 insertions(+)
diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c
index cee25f5..8e83084 100644
--- a/block/qcow2-snapshot.c
+++ b/block/qcow2-snapshot.c
@@ -736,3 +736,157 @@ int qcow2_snapshot_load_tmp(BlockDriverState *bs,
return 0;
}
+
+int qcow2_snapshot_save_dependency(BlockDriverState *bs,
+ const char *depend_snapshot_id,
+ int64_t depend_offset,
+ int64_t depend_size,
+ int64_t offset,
+ Error **errp)
+{
+ int snapshot_index;
+ BDRVQcow2State *s = bs->opaque;
+ QCowSnapshot *sn;
+ int ret;
+ int64_t i;
+ int64_t total_bytes = depend_size;
+ int64_t depend_offset1, offset1;
+ uint64_t *depend_l1_table = NULL;
+ uint64_t depend_l1_bytes;
+ uint64_t *depend_l2_table = NULL;
+ uint64_t depend_l2_offset;
+ uint64_t depend_entry;
+ QCowL2Meta l2meta;
+
+ assert(bs->read_only == false);
+
+ if (depend_snapshot_id == NULL) {
+ return 0;
+ }
+
+ if (!QEMU_IS_ALIGNED(depend_offset, s->cluster_size)) {
+ error_setg(errp, "Specified snapshot offset is not multiple of %u",
+ s->cluster_size);
+ return -EINVAL;
+ }
+
+ if (!QEMU_IS_ALIGNED(offset, s->cluster_size)) {
+ error_setg(errp, "Offset is not multiple of %u", s->cluster_size);
+ return -EINVAL;
+ }
+
+ if (!QEMU_IS_ALIGNED(depend_size, s->cluster_size)) {
+ error_setg(errp, "depend_size is not multiple of %u", s->cluster_size);
+ return -EINVAL;
+ }
+
+ snapshot_index = find_snapshot_by_id_and_name(bs, NULL, depend_snapshot_id);
+ /* Search the snapshot */
+ if (snapshot_index < 0) {
+ error_setg(errp, "Can't find snapshot");
+ return -ENOENT;
+ }
+
+ sn = &s->snapshots[snapshot_index];
+ if (sn->disk_size != bs->total_sectors * BDRV_SECTOR_SIZE) {
+ error_report("qcow2: depend on the snapshots with different disk "
+ "size is not implemented");
+ return -ENOTSUP;
+ }
+
+ /* Only can save dependency of snapshot's vmstate data */
+ depend_offset1 = depend_offset + qcow2_vm_state_offset(s);
+ offset1 = offset + qcow2_vm_state_offset(s);
+
+ depend_l1_bytes = s->l1_size * sizeof(uint64_t);
+ depend_l1_table = g_try_malloc0(depend_l1_bytes);
+ if (depend_l1_table == NULL) {
+ return -ENOMEM;
+ }
+
+ ret = bdrv_pread(bs->file, sn->l1_table_offset, depend_l1_table,
+ depend_l1_bytes);
+ if (ret < 0) {
+ g_free(depend_l1_table);
+ goto out;
+ }
+ for (i = 0; i < depend_l1_bytes / sizeof(uint64_t); i++) {
+ be64_to_cpus(&depend_l1_table[i]);
+ }
+
+ while (total_bytes) {
+ assert(total_bytes > 0);
+ /* Find the cluster of depend */
+ depend_l2_offset =
+ depend_l1_table[depend_offset1 >> (s->l2_bits + s->cluster_bits)];
+ depend_l2_offset &= L1E_OFFSET_MASK;
+ if (depend_l2_offset == 0) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ if (offset_into_cluster(s, depend_l2_offset)) {
+ qcow2_signal_corruption(bs, true, -1, -1, "L2 table offset %#"
+ PRIx64 " unaligned (L1 index: %#"
+ PRIx64 ")",
+ depend_l2_offset,
+ depend_offset1 >>
+ (s->l2_bits + s->cluster_bits));
+ return -EIO;
+ }
+
+ ret = qcow2_cache_get(bs, s->l2_table_cache, depend_l2_offset,
+ (void **)(&depend_l2_table));
+ if (ret < 0) {
+ goto out;
+ }
+
+ depend_entry =
+ be64_to_cpu(
+ depend_l2_table[offset_to_l2_index(s, depend_offset1)]);
+ if (depend_entry == 0) {
+ ret = -EINVAL;
+ qcow2_cache_put(s->l2_table_cache, (void **)(&depend_l2_table));
+ goto out;
+ }
+
+ memset(&l2meta, 0, sizeof(l2meta));
+ l2meta.offset = offset1;
+ l2meta.alloc_offset = (depend_entry & L2E_OFFSET_MASK);
+ l2meta.nb_clusters = 1;
+ /* Add a ref to this cluster */
+ ret = qcow2_update_cluster_refcount(
+ bs, l2meta.alloc_offset >> s->cluster_bits,
+ 1, false, QCOW2_DISCARD_SNAPSHOT);
+ if (ret < 0) {
+ qcow2_cache_put(s->l2_table_cache, (void **)(&depend_l2_table));
+ goto out;
+ }
+
+ ret = qcow2_alloc_cluster_link_l2(bs, &l2meta);
+ if (ret < 0) {
+ qcow2_cache_put(s->l2_table_cache, (void **)(&depend_l2_table));
+ goto out;
+ }
+
+ total_bytes -= s->cluster_size;
+ offset1 += s->cluster_size;
+ depend_offset1 += s->cluster_size;
+
+ qcow2_cache_put(s->l2_table_cache, (void **)(&depend_l2_table));
+ }
+
+out:
+ g_free(depend_l1_table);
+ return ret;
+}
+
+int qcow2_snapshot_support_dependency(BlockDriverState *bs, int32_t *alignment)
+{
+ BDRVQcow2State *s = bs->opaque;
+ if (alignment) {
+ *alignment = s->cluster_size;
+ }
+
+ return 1;
+}
diff --git a/block/qcow2.c b/block/qcow2.c
index 071dc4d..9786ba4 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -4371,6 +4371,8 @@ BlockDriver bdrv_qcow2 = {
.bdrv_snapshot_delete = qcow2_snapshot_delete,
.bdrv_snapshot_list = qcow2_snapshot_list,
.bdrv_snapshot_load_tmp = qcow2_snapshot_load_tmp,
+ .bdrv_snapshot_support_dependency = qcow2_snapshot_support_dependency,
+ .bdrv_snapshot_save_dependency = qcow2_snapshot_save_dependency,
.bdrv_measure = qcow2_measure,
.bdrv_get_info = qcow2_get_info,
.bdrv_get_specific_info = qcow2_get_specific_info,
diff --git a/block/qcow2.h b/block/qcow2.h
index 1a84cc7..dc7ef45 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -640,6 +640,13 @@ int qcow2_snapshot_load_tmp(BlockDriverState *bs,
void qcow2_free_snapshots(BlockDriverState *bs);
int qcow2_read_snapshots(BlockDriverState *bs);
+int qcow2_snapshot_save_dependency(BlockDriverState *bs,
+ const char *depend_snapshot_id,
+ int64_t depend_offset,
+ int64_t depend_size,
+ int64_t offset,
+ Error **errp);
+int qcow2_snapshot_support_dependency(BlockDriverState *bs, int32_t *alignment);
/* qcow2-cache.c functions */
Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables,
--
2.7.4
^ permalink raw reply related [flat|nested] 17+ messages in thread
end of thread, other threads:[~2018-05-14 12:59 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-03-13 8:33 [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 01/10] RFC: Add save and support snapshot dependency function to block driver junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 02/10] RFC: Implement qcow2's snapshot dependent saving function junyan.he
2018-05-08 14:50 ` Eric Blake
2018-05-14 12:59 ` Stefan Hajnoczi
2018-03-13 8:33 ` [Qemu-devel] [PATCH 03/10] RFC: Implement save and support snapshot dependency in block driver layer junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 04/10] RFC: Set memory_region_set_log available for more client junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 05/10] RFC: Add memory region snapshot bitmap get function junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 06/10] RFC: Add save dependency functions to qemu_file junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 07/10] RFC: Add get_current_snapshot_info to get the snapshot state junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 08/10] RFC: Add a section_id parameter to save_live_iterate call junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 09/10] RFC: Add nvdimm snapshot saving to migration junyan.he
2018-03-13 8:33 ` [Qemu-devel] [PATCH 10/10] RFC: Enable nvdimm snapshot functions junyan.he
2018-03-15 13:55 ` [Qemu-devel] [PATCH 00/10] RFC: Optimize nvdimm kind memory for snapshot no-reply
2018-03-15 14:15 ` no-reply
2018-05-08 9:23 ` Stefan Hajnoczi
2018-03-14 1:20 junyan.he
2018-03-14 1:20 ` [Qemu-devel] [PATCH 02/10] RFC: Implement qcow2's snapshot dependent saving function junyan.he
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.